2024-09-19T03:59:01.820 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T03:59:01.840 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T03:59:01.913 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626 branch: squid description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/multimds_misc} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911626' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - Scrub error on inode sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi076.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKk33/Lk6l7X+CyxNGn5zPTs4rxjY8EJw00qPjnFzaWtq9UlZd099lLHp3ZdN5E69ZNNXDcLPocTWXXNgL/4q+g= smithi116.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFskfojGWT2x250/5ry1dm5px2lpFdoHSAHjwVysFHWhKrClCFtsI2KwUGarGL61U3+1MLZBOaqIH4zTrJ0bYY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_multimds_misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T03:59:01.914 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T03:59:01.914 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T03:59:01.914 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T03:59:01.916 INFO:teuthology.task.internal:Checking packages... 2024-09-19T03:59:01.937 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T03:59:01.937 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T03:59:01.937 INFO:teuthology.packaging:ref: None 2024-09-19T03:59:01.937 INFO:teuthology.packaging:tag: None 2024-09-19T03:59:01.937 INFO:teuthology.packaging:branch: squid 2024-09-19T03:59:01.937 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T03:59:01.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-09-19T03:59:02.142 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363-g6fad5dd7-1jammy 2024-09-19T03:59:02.144 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T03:59:02.150 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T03:59:02.151 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T03:59:02.166 INFO:teuthology.task.internal:Saving configuration 2024-09-19T03:59:02.182 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T03:59:02.189 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T03:59:02.211 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi076.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:52:47.600936', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIMSY9Ez21FZeDw9Ox3c3/z0EPPKZUXlfCggXuJm64kDO'} 2024-09-19T03:59:02.229 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi116.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 03:52:47.602613', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC73l/UNnu2/qmSW1x2hYYUVPPHZCZ6WtTC8IHy5ZVmP'} 2024-09-19T03:59:02.229 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T03:59:02.236 INFO:teuthology.task.internal:roles: ubuntu@smithi076.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-19T03:59:02.236 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-19T03:59:02.236 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T03:59:02.283 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f158453f0a0>, signals=[15]) 2024-09-19T03:59:02.283 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T03:59:02.290 INFO:teuthology.task.internal:Opening connections... 2024-09-19T03:59:02.290 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T03:59:02.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:59:02.370 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T03:59:02.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T03:59:02.448 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T03:59:02.455 DEBUG:teuthology.orchestra.run.smithi076:> uname -m 2024-09-19T03:59:02.461 INFO:teuthology.orchestra.run.smithi076.stdout:x86_64 2024-09-19T03:59:02.461 DEBUG:teuthology.orchestra.run.smithi076:> cat /etc/os-release 2024-09-19T03:59:02.507 INFO:teuthology.orchestra.run.smithi076.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-19T03:59:02.507 INFO:teuthology.orchestra.run.smithi076.stdout:NAME="Ubuntu" 2024-09-19T03:59:02.507 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_ID="22.04" 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:VERSION_CODENAME=jammy 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:ID=ubuntu 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:ID_LIKE=debian 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-19T03:59:02.508 INFO:teuthology.orchestra.run.smithi076.stdout:UBUNTU_CODENAME=jammy 2024-09-19T03:59:02.509 INFO:teuthology.lock.ops:Updating smithi076.front.sepia.ceph.com on lock server 2024-09-19T03:59:02.530 DEBUG:teuthology.orchestra.run.smithi116:> uname -m 2024-09-19T03:59:02.536 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2024-09-19T03:59:02.536 DEBUG:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Ubuntu" 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="22.04" 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_CODENAME=jammy 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:ID=ubuntu 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE=debian 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-19T03:59:02.581 INFO:teuthology.orchestra.run.smithi116.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-19T03:59:02.582 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-19T03:59:02.582 INFO:teuthology.orchestra.run.smithi116.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-19T03:59:02.582 INFO:teuthology.orchestra.run.smithi116.stdout:UBUNTU_CODENAME=jammy 2024-09-19T03:59:02.582 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2024-09-19T03:59:02.597 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T03:59:02.606 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T03:59:02.610 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T03:59:02.610 DEBUG:teuthology.orchestra.run.smithi076:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:59:02.613 DEBUG:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2024-09-19T03:59:02.629 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T03:59:02.635 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T03:59:02.635 DEBUG:teuthology.orchestra.run.smithi076:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:59:02.661 DEBUG:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2024-09-19T03:59:02.689 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T03:59:02.731 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T03:59:02.746 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-19T03:59:02.746 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-19T03:59:02.746 DEBUG:teuthology.orchestra.run.smithi076:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:59:02.747 DEBUG:teuthology.orchestra.run.smithi116:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T03:59:02.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:59:02.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T03:59:02.752 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-09-19T03:59:02.752 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-09-19T03:59:02.799 INFO:teuthology.orchestra.run.smithi076.stdout:5.15.0-56-generic 2024-09-19T03:59:02.800 INFO:teuthology.task.kernel:Running kernel on smithi076: 5.15.0-56-generic 2024-09-19T03:59:02.800 DEBUG:teuthology.orchestra.run.smithi076:> sudo apt-get clean 2024-09-19T03:59:02.801 INFO:teuthology.orchestra.run.smithi116.stdout:5.15.0-56-generic 2024-09-19T03:59:02.801 INFO:teuthology.task.kernel:Running kernel on smithi116: 5.15.0-56-generic 2024-09-19T03:59:02.801 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-get clean 2024-09-19T03:59:02.936 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-get update 2024-09-19T03:59:02.940 DEBUG:teuthology.orchestra.run.smithi076:> sudo apt-get update 2024-09-19T03:59:03.167 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T03:59:03.167 INFO:teuthology.orchestra.run.smithi076.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-19T03:59:03.167 INFO:teuthology.orchestra.run.smithi076.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-19T03:59:03.183 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T03:59:03.193 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-19T03:59:03.243 INFO:teuthology.orchestra.run.smithi076.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-19T03:59:03.257 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-19T03:59:03.278 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-19T03:59:03.406 INFO:teuthology.orchestra.run.smithi076.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-19T03:59:03.425 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-19T03:59:03.489 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-19T03:59:03.491 INFO:teuthology.orchestra.run.smithi076.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [2,058 kB] 2024-09-19T03:59:03.509 INFO:teuthology.orchestra.run.smithi076.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-19T03:59:03.513 INFO:teuthology.orchestra.run.smithi076.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-19T03:59:03.516 INFO:teuthology.orchestra.run.smithi076.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-19T03:59:03.517 INFO:teuthology.orchestra.run.smithi076.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-19T03:59:03.523 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-19T03:59:03.526 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-19T03:59:03.527 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-19T03:59:03.540 INFO:teuthology.orchestra.run.smithi076.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-19T03:59:03.540 INFO:teuthology.orchestra.run.smithi076.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-19T03:59:03.543 INFO:teuthology.orchestra.run.smithi076.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-19T03:59:03.543 INFO:teuthology.orchestra.run.smithi076.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-19T03:59:03.547 INFO:teuthology.orchestra.run.smithi116.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-19T03:59:03.547 INFO:teuthology.orchestra.run.smithi116.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-19T03:59:03.548 INFO:teuthology.orchestra.run.smithi076.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-19T03:59:03.551 INFO:teuthology.orchestra.run.smithi116.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-19T03:59:03.551 INFO:teuthology.orchestra.run.smithi116.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-19T03:59:03.556 INFO:teuthology.orchestra.run.smithi076.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-19T03:59:03.557 INFO:teuthology.orchestra.run.smithi076.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-19T03:59:03.557 INFO:teuthology.orchestra.run.smithi076.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-19T03:59:03.557 INFO:teuthology.orchestra.run.smithi076.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-19T03:59:03.561 INFO:teuthology.orchestra.run.smithi116.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-19T03:59:03.570 INFO:teuthology.orchestra.run.smithi076.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-19T03:59:03.578 INFO:teuthology.orchestra.run.smithi116.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-19T03:59:03.580 INFO:teuthology.orchestra.run.smithi116.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-19T03:59:03.580 INFO:teuthology.orchestra.run.smithi116.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-19T03:59:03.581 INFO:teuthology.orchestra.run.smithi116.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-19T03:59:03.581 INFO:teuthology.orchestra.run.smithi116.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-19T03:59:03.581 INFO:teuthology.orchestra.run.smithi116.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-19T03:59:03.583 INFO:teuthology.orchestra.run.smithi116.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-19T03:59:03.584 INFO:teuthology.orchestra.run.smithi116.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-19T03:59:03.585 INFO:teuthology.orchestra.run.smithi076.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [701 kB] 2024-09-19T03:59:03.585 INFO:teuthology.orchestra.run.smithi076.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-19T03:59:03.590 INFO:teuthology.orchestra.run.smithi076.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [355 kB] 2024-09-19T03:59:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-19T03:59:03.593 INFO:teuthology.orchestra.run.smithi076.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,495 kB] 2024-09-19T03:59:03.595 INFO:teuthology.orchestra.run.smithi116.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-19T03:59:03.611 INFO:teuthology.orchestra.run.smithi076.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-19T03:59:03.612 INFO:teuthology.orchestra.run.smithi076.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [429 kB] 2024-09-19T03:59:03.612 INFO:teuthology.orchestra.run.smithi116.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-19T03:59:03.612 INFO:teuthology.orchestra.run.smithi116.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-19T03:59:03.612 INFO:teuthology.orchestra.run.smithi116.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-19T03:59:03.613 INFO:teuthology.orchestra.run.smithi116.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-19T03:59:03.613 INFO:teuthology.orchestra.run.smithi116.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-19T03:59:03.615 INFO:teuthology.orchestra.run.smithi076.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-19T03:59:03.615 INFO:teuthology.orchestra.run.smithi076.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,124 kB] 2024-09-19T03:59:03.623 INFO:teuthology.orchestra.run.smithi076.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [732 kB] 2024-09-19T03:59:03.628 INFO:teuthology.orchestra.run.smithi076.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-19T03:59:03.630 INFO:teuthology.orchestra.run.smithi076.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.1 kB] 2024-09-19T03:59:03.630 INFO:teuthology.orchestra.run.smithi076.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-19T03:59:03.631 INFO:teuthology.orchestra.run.smithi076.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-19T03:59:03.648 INFO:teuthology.orchestra.run.smithi076.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-19T03:59:03.648 INFO:teuthology.orchestra.run.smithi076.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-19T03:59:03.648 INFO:teuthology.orchestra.run.smithi076.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-19T03:59:03.649 INFO:teuthology.orchestra.run.smithi076.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-19T03:59:03.649 INFO:teuthology.orchestra.run.smithi076.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-19T03:59:03.661 INFO:teuthology.orchestra.run.smithi076.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-19T03:59:03.661 INFO:teuthology.orchestra.run.smithi076.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-19T03:59:03.662 INFO:teuthology.orchestra.run.smithi076.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-19T03:59:03.678 INFO:teuthology.orchestra.run.smithi076.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-19T03:59:03.678 INFO:teuthology.orchestra.run.smithi076.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-19T03:59:03.706 INFO:teuthology.orchestra.run.smithi116.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,839 kB] 2024-09-19T03:59:04.130 INFO:teuthology.orchestra.run.smithi116.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [544 kB] 2024-09-19T03:59:04.146 INFO:teuthology.orchestra.run.smithi116.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [298 kB] 2024-09-19T03:59:04.154 INFO:teuthology.orchestra.run.smithi116.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-19T03:59:04.155 INFO:teuthology.orchestra.run.smithi116.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-19T03:59:04.156 INFO:teuthology.orchestra.run.smithi116.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,431 kB] 2024-09-19T03:59:04.252 INFO:teuthology.orchestra.run.smithi116.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [418 kB] 2024-09-19T03:59:04.263 INFO:teuthology.orchestra.run.smithi116.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-19T03:59:04.263 INFO:teuthology.orchestra.run.smithi116.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [903 kB] 2024-09-19T03:59:04.286 INFO:teuthology.orchestra.run.smithi116.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [628 kB] 2024-09-19T03:59:04.308 INFO:teuthology.orchestra.run.smithi116.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [177 kB] 2024-09-19T03:59:04.314 INFO:teuthology.orchestra.run.smithi116.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.3 kB] 2024-09-19T03:59:04.314 INFO:teuthology.orchestra.run.smithi116.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-19T03:59:04.315 INFO:teuthology.orchestra.run.smithi116.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-19T03:59:04.315 INFO:teuthology.orchestra.run.smithi116.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-19T03:59:04.394 INFO:teuthology.orchestra.run.smithi116.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-19T03:59:09.788 INFO:teuthology.orchestra.run.smithi076.stdout:Fetched 16.2 MB in 2s (7,293 kB/s) 2024-09-19T03:59:09.852 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 16.2 MB in 2s (7,157 kB/s) 2024-09-19T03:59:11.225 INFO:teuthology.orchestra.run.smithi076.stdout:Reading package lists... 2024-09-19T03:59:11.242 DEBUG:teuthology.orchestra.run.smithi076:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T03:59:11.298 INFO:teuthology.orchestra.run.smithi076.stdout:Reading package lists... 2024-09-19T03:59:11.415 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-09-19T03:59:11.432 DEBUG:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T03:59:11.484 INFO:teuthology.orchestra.run.smithi076.stdout:Building dependency tree... 2024-09-19T03:59:11.485 INFO:teuthology.orchestra.run.smithi076.stdout:Reading state information... 2024-09-19T03:59:11.499 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-09-19T03:59:11.649 INFO:teuthology.orchestra.run.smithi076.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T03:59:11.650 INFO:teuthology.orchestra.run.smithi076.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T03:59:11.650 INFO:teuthology.orchestra.run.smithi076.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T03:59:11.650 INFO:teuthology.orchestra.run.smithi076.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T03:59:11.650 INFO:teuthology.orchestra.run.smithi076.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-19T03:59:11.650 INFO:teuthology.orchestra.run.smithi076.stdout: python2.7-minimal 2024-09-19T03:59:11.651 INFO:teuthology.orchestra.run.smithi076.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T03:59:11.651 INFO:teuthology.orchestra.run.smithi076.stdout:The following additional packages will be installed: 2024-09-19T03:59:11.652 INFO:teuthology.orchestra.run.smithi076.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T03:59:11.652 INFO:teuthology.orchestra.run.smithi076.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-19T03:59:11.652 INFO:teuthology.orchestra.run.smithi076.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-19T03:59:11.653 INFO:teuthology.orchestra.run.smithi076.stdout:Suggested packages: 2024-09-19T03:59:11.653 INFO:teuthology.orchestra.run.smithi076.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-19T03:59:11.653 INFO:teuthology.orchestra.run.smithi076.stdout:Recommended packages: 2024-09-19T03:59:11.654 INFO:teuthology.orchestra.run.smithi076.stdout: thermald 2024-09-19T03:59:11.694 INFO:teuthology.orchestra.run.smithi076.stdout:The following NEW packages will be installed: 2024-09-19T03:59:11.694 INFO:teuthology.orchestra.run.smithi076.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T03:59:11.695 INFO:teuthology.orchestra.run.smithi076.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-19T03:59:11.695 INFO:teuthology.orchestra.run.smithi076.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-19T03:59:11.696 INFO:teuthology.orchestra.run.smithi076.stdout:The following packages will be upgraded: 2024-09-19T03:59:11.697 INFO:teuthology.orchestra.run.smithi076.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-19T03:59:11.709 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2024-09-19T03:59:11.710 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2024-09-19T03:59:11.878 INFO:teuthology.orchestra.run.smithi076.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T03:59:11.878 INFO:teuthology.orchestra.run.smithi076.stdout:Need to get 113 MB of archives. 2024-09-19T03:59:11.878 INFO:teuthology.orchestra.run.smithi076.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-19T03:59:11.878 INFO:teuthology.orchestra.run.smithi076.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-19T03:59:11.899 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T03:59:11.899 INFO:teuthology.orchestra.run.smithi116.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T03:59:11.899 INFO:teuthology.orchestra.run.smithi116.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T03:59:11.899 INFO:teuthology.orchestra.run.smithi116.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T03:59:11.900 INFO:teuthology.orchestra.run.smithi116.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-19T03:59:11.900 INFO:teuthology.orchestra.run.smithi116.stdout: python2.7-minimal 2024-09-19T03:59:11.900 INFO:teuthology.orchestra.run.smithi116.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T03:59:11.901 INFO:teuthology.orchestra.run.smithi116.stdout:The following additional packages will be installed: 2024-09-19T03:59:11.902 INFO:teuthology.orchestra.run.smithi116.stdout: linux-generic linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T03:59:11.902 INFO:teuthology.orchestra.run.smithi116.stdout: linux-headers-generic linux-image-5.15.0-122-generic 2024-09-19T03:59:11.902 INFO:teuthology.orchestra.run.smithi116.stdout: linux-modules-5.15.0-122-generic linux-modules-extra-5.15.0-122-generic 2024-09-19T03:59:11.903 INFO:teuthology.orchestra.run.smithi116.stdout:Suggested packages: 2024-09-19T03:59:11.903 INFO:teuthology.orchestra.run.smithi116.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-19T03:59:11.904 INFO:teuthology.orchestra.run.smithi116.stdout:Recommended packages: 2024-09-19T03:59:11.904 INFO:teuthology.orchestra.run.smithi116.stdout: thermald 2024-09-19T03:59:11.971 INFO:teuthology.orchestra.run.smithi116.stdout:The following NEW packages will be installed: 2024-09-19T03:59:11.971 INFO:teuthology.orchestra.run.smithi116.stdout: linux-headers-5.15.0-122 linux-headers-5.15.0-122-generic 2024-09-19T03:59:11.971 INFO:teuthology.orchestra.run.smithi116.stdout: linux-image-5.15.0-122-generic linux-modules-5.15.0-122-generic 2024-09-19T03:59:11.971 INFO:teuthology.orchestra.run.smithi116.stdout: linux-modules-extra-5.15.0-122-generic 2024-09-19T03:59:11.973 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages will be upgraded: 2024-09-19T03:59:11.973 INFO:teuthology.orchestra.run.smithi116.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-19T03:59:12.153 INFO:teuthology.orchestra.run.smithi116.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T03:59:12.153 INFO:teuthology.orchestra.run.smithi116.stdout:Need to get 113 MB of archives. 2024-09-19T03:59:12.153 INFO:teuthology.orchestra.run.smithi116.stdout:After this operation, 584 MB of additional disk space will be used. 2024-09-19T03:59:12.153 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-122-generic amd64 5.15.0-122.132 [22.7 MB] 2024-09-19T03:59:13.286 INFO:teuthology.orchestra.run.smithi076.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-19T03:59:13.552 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-122-generic amd64 5.15.0-122.132 [11.5 MB] 2024-09-19T03:59:13.635 INFO:teuthology.orchestra.run.smithi076.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-19T03:59:13.952 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-122-generic amd64 5.15.0-122.132 [63.9 MB] 2024-09-19T03:59:15.759 INFO:teuthology.orchestra.run.smithi076.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-19T03:59:15.760 INFO:teuthology.orchestra.run.smithi076.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-19T03:59:15.838 INFO:teuthology.orchestra.run.smithi076.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-19T03:59:16.097 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.122.122 [1,692 B] 2024-09-19T03:59:16.097 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.122.122 [2,436 B] 2024-09-19T03:59:16.098 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122 all 5.15.0-122.132 [12.3 MB] 2024-09-19T03:59:16.197 INFO:teuthology.orchestra.run.smithi076.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-19T03:59:16.281 INFO:teuthology.orchestra.run.smithi076.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-19T03:59:16.572 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-122-generic amd64 5.15.0-122.132 [2,827 kB] 2024-09-19T03:59:16.665 INFO:teuthology.orchestra.run.smithi076.stdout:Fetched 113 MB in 5s (24.7 MB/s) 2024-09-19T03:59:16.667 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.122.122 [2,294 B] 2024-09-19T03:59:16.847 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-19T03:59:17.071 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 113 MB in 5s (24.2 MB/s) 2024-09-19T03:59:17.271 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package linux-modules-5.15.0-122-generic. 2024-09-19T03:59:18.847 INFO:teuthology.orchestra.run.smithi076.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-19T03:59:18.851 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:18.986 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:19.264 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-19T03:59:19.268 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../0-linux-modules-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:19.425 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:21.104 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-19T03:59:21.126 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:21.222 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:21.699 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-19T03:59:21.719 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:21.757 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:21.793 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package linux-image-5.15.0-122-generic. 2024-09-19T03:59:21.814 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../1-linux-image-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:21.915 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:22.413 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-122-generic. 2024-09-19T03:59:22.433 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:22.471 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:28.316 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T03:59:28.460 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T03:59:28.767 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T03:59:28.863 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T03:59:29.125 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-19T03:59:29.147 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-19T03:59:29.187 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../3-linux-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T03:59:29.191 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T03:59:29.320 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T03:59:29.678 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T03:59:29.782 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-image-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T03:59:30.076 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package linux-headers-5.15.0-122. 2024-09-19T03:59:30.098 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../5-linux-headers-5.15.0-122_5.15.0-122.132_all.deb ... 2024-09-19T03:59:30.168 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T03:59:34.474 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-19T03:59:34.510 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:34.548 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:36.071 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package linux-headers-5.15.0-122-generic. 2024-09-19T03:59:36.107 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../6-linux-headers-5.15.0-122-generic_5.15.0-122.132_amd64.deb ... 2024-09-19T03:59:36.146 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:37.515 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T03:59:37.632 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T03:59:37.972 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T03:59:38.078 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:38.179 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-19T03:59:38.299 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:39.611 INFO:teuthology.orchestra.run.smithi076.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-19T03:59:39.611 INFO:teuthology.orchestra.run.smithi076.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-19T03:59:39.611 INFO:teuthology.orchestra.run.smithi076.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-19T03:59:39.611 INFO:teuthology.orchestra.run.smithi076.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-19T03:59:39.758 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:39.955 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.122.122_amd64.deb ... 2024-09-19T03:59:40.106 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-headers-generic (5.15.0.122.122) over (5.15.0.56.54) ... 2024-09-19T03:59:40.464 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-headers-5.15.0-122 (5.15.0-122.132) ... 2024-09-19T03:59:40.619 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-headers-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:40.745 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-headers-generic (5.15.0.122.122) ... 2024-09-19T03:59:40.882 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:41.201 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-19T03:59:41.319 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-19T03:59:41.420 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:42.200 INFO:teuthology.orchestra.run.smithi116.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-19T03:59:42.200 INFO:teuthology.orchestra.run.smithi116.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-19T03:59:42.200 INFO:teuthology.orchestra.run.smithi116.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-122-generic 2024-09-19T03:59:42.200 INFO:teuthology.orchestra.run.smithi116.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-122-generic 2024-09-19T03:59:42.341 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-modules-extra-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:42.771 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:42.828 INFO:teuthology.orchestra.run.smithi076.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-19T03:59:42.829 INFO:teuthology.orchestra.run.smithi076.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-19T03:59:43.876 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-image-generic (5.15.0.122.122) ... 2024-09-19T03:59:44.002 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-generic (5.15.0.122.122) ... 2024-09-19T03:59:44.136 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-modules-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:46.005 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for linux-image-5.15.0-122-generic (5.15.0-122.132) ... 2024-09-19T03:59:46.075 INFO:teuthology.orchestra.run.smithi116.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-19T03:59:46.075 INFO:teuthology.orchestra.run.smithi116.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-122-generic 2024-09-19T03:59:56.576 INFO:teuthology.orchestra.run.smithi076.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-19T03:59:56.576 INFO:teuthology.orchestra.run.smithi076.stdout:Sourcing file `/etc/default/grub' 2024-09-19T03:59:56.586 INFO:teuthology.orchestra.run.smithi076.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T03:59:56.604 INFO:teuthology.orchestra.run.smithi076.stdout:Generating grub configuration file ... 2024-09-19T03:59:56.870 INFO:teuthology.orchestra.run.smithi076.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T03:59:56.890 INFO:teuthology.orchestra.run.smithi076.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T03:59:57.191 INFO:teuthology.orchestra.run.smithi076.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T03:59:57.197 INFO:teuthology.orchestra.run.smithi076.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T03:59:57.266 INFO:teuthology.orchestra.run.smithi076.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T03:59:57.272 INFO:teuthology.orchestra.run.smithi076.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T03:59:57.441 INFO:teuthology.orchestra.run.smithi076.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T03:59:57.441 INFO:teuthology.orchestra.run.smithi076.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T03:59:57.441 INFO:teuthology.orchestra.run.smithi076.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T03:59:57.490 INFO:teuthology.orchestra.run.smithi076.stdout:done 2024-09-19T03:59:57.890 DEBUG:teuthology.orchestra.run.smithi076:> dpkg -s linux-image-generic 2024-09-19T03:59:57.911 INFO:teuthology.orchestra.run.smithi076.stdout:Package: linux-image-generic 2024-09-19T03:59:57.911 INFO:teuthology.orchestra.run.smithi076.stdout:Status: install ok installed 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Priority: optional 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Section: kernel 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Installed-Size: 21 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: amd64 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Source: linux-meta 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Version: 5.15.0.122.122 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Recommends: thermald 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout:Description: Generic Linux kernel image 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout: This package will always depend on the latest generic kernel image 2024-09-19T03:59:57.912 INFO:teuthology.orchestra.run.smithi076.stdout: available. 2024-09-19T03:59:57.913 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-19T03:59:57.913 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T03:59:57.913 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T03:59:57.913 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-19T03:59:57.913 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi076.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T03:59:57.913 DEBUG:teuthology.orchestra.run.smithi076:> sudo apt-get clean 2024-09-19T03:59:58.026 DEBUG:teuthology.orchestra.run.smithi076:> sudo apt-get update 2024-09-19T03:59:58.135 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T03:59:58.144 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T03:59:58.161 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T03:59:58.257 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T03:59:59.564 INFO:teuthology.orchestra.run.smithi076.stdout:Reading package lists... 2024-09-19T03:59:59.582 DEBUG:teuthology.orchestra.run.smithi076:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T03:59:59.644 INFO:teuthology.orchestra.run.smithi076.stdout:Reading package lists... 2024-09-19T03:59:59.850 INFO:teuthology.orchestra.run.smithi076.stdout:Building dependency tree... 2024-09-19T03:59:59.851 INFO:teuthology.orchestra.run.smithi076.stdout:Reading state information... 2024-09-19T04:00:00.042 INFO:teuthology.orchestra.run.smithi076.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-19T04:00:00.043 INFO:teuthology.orchestra.run.smithi076.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T04:00:00.043 INFO:teuthology.orchestra.run.smithi076.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T04:00:00.043 INFO:teuthology.orchestra.run.smithi076.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T04:00:00.043 INFO:teuthology.orchestra.run.smithi076.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T04:00:00.043 INFO:teuthology.orchestra.run.smithi076.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T04:00:00.044 INFO:teuthology.orchestra.run.smithi076.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T04:00:00.044 INFO:teuthology.orchestra.run.smithi076.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T04:00:00.083 INFO:teuthology.orchestra.run.smithi116.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-19T04:00:00.083 INFO:teuthology.orchestra.run.smithi116.stdout:Sourcing file `/etc/default/grub' 2024-09-19T04:00:00.085 INFO:teuthology.orchestra.run.smithi076.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T04:00:00.087 DEBUG:teuthology.orchestra.run.smithi076:> dpkg -s linux-image-generic 2024-09-19T04:00:00.102 INFO:teuthology.orchestra.run.smithi076.stdout:Package: linux-image-generic 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Status: install ok installed 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Priority: optional 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Section: kernel 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Installed-Size: 21 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Architecture: amd64 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Source: linux-meta 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Version: 5.15.0.122.122 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Recommends: thermald 2024-09-19T04:00:00.103 INFO:teuthology.orchestra.run.smithi076.stdout:Description: Generic Linux kernel image 2024-09-19T04:00:00.104 INFO:teuthology.orchestra.run.smithi076.stdout: This package will always depend on the latest generic kernel image 2024-09-19T04:00:00.104 INFO:teuthology.orchestra.run.smithi076.stdout: available. 2024-09-19T04:00:00.104 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-09-19T04:00:00.116 INFO:teuthology.orchestra.run.smithi116.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T04:00:00.138 INFO:teuthology.orchestra.run.smithi116.stdout:Generating grub configuration file ... 2024-09-19T04:00:00.154 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.mUD06OdWRn 2024-09-19T04:00:00.154 DEBUG:teuthology.orchestra.run.smithi076:> sudo cp /boot/grub/grub.cfg /tmp/tmp.mUD06OdWRn 2024-09-19T04:00:00.210 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.mUD06OdWRn 2024-09-19T04:00:00.319 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.mUD06OdWRn is 10KB 2024-09-19T04:00:00.332 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.mUD06OdWRn 2024-09-19T04:00:00.338 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-19T04:00:00.389 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:00:00.389 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-19T04:00:00.389 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-19T04:00:00.411 INFO:teuthology.orchestra.run.smithi116.stdout:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T04:00:00.445 INFO:teuthology.orchestra.run.smithi116.stdout:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T04:00:00.460 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-19T04:00:00.460 DEBUG:teuthology.orchestra.run.smithi076:> sudo update-grub 2024-09-19T04:00:00.804 INFO:teuthology.orchestra.run.smithi116.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T04:00:00.809 INFO:teuthology.orchestra.run.smithi116.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T04:00:00.875 INFO:teuthology.orchestra.run.smithi116.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T04:00:00.880 INFO:teuthology.orchestra.run.smithi116.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T04:00:01.059 INFO:teuthology.orchestra.run.smithi116.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T04:00:01.060 INFO:teuthology.orchestra.run.smithi116.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T04:00:01.060 INFO:teuthology.orchestra.run.smithi116.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T04:00:01.111 INFO:teuthology.orchestra.run.smithi116.stdout:done 2024-09-19T04:00:01.550 INFO:teuthology.orchestra.run.smithi076.stderr:Sourcing file `/etc/default/grub' 2024-09-19T04:00:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T04:00:01.554 INFO:teuthology.orchestra.run.smithi076.stderr:Generating grub configuration file ... 2024-09-19T04:00:01.623 DEBUG:teuthology.orchestra.run.smithi116:> dpkg -s linux-image-generic 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Package: linux-image-generic 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Status: install ok installed 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Priority: optional 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Section: kernel 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Installed-Size: 21 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Architecture: amd64 2024-09-19T04:00:01.640 INFO:teuthology.orchestra.run.smithi116.stdout:Source: linux-meta 2024-09-19T04:00:01.641 INFO:teuthology.orchestra.run.smithi116.stdout:Version: 5.15.0.122.122 2024-09-19T04:00:01.641 INFO:teuthology.orchestra.run.smithi116.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T04:00:01.641 INFO:teuthology.orchestra.run.smithi116.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T04:00:01.641 INFO:teuthology.orchestra.run.smithi116.stdout:Recommends: thermald 2024-09-19T04:00:01.641 INFO:teuthology.orchestra.run.smithi116.stdout:Description: Generic Linux kernel image 2024-09-19T04:00:01.641 INFO:teuthology.orchestra.run.smithi116.stdout: This package will always depend on the latest generic kernel image 2024-09-19T04:00:01.641 INFO:teuthology.orchestra.run.smithi116.stdout: available. 2024-09-19T04:00:01.641 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-122-generic 2024-09-19T04:00:01.641 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T04:00:01.641 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T04:00:01.642 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-19T04:00:01.642 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi116.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T04:00:01.642 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-get clean 2024-09-19T04:00:01.723 INFO:teuthology.orchestra.run.smithi076.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T04:00:01.729 INFO:teuthology.orchestra.run.smithi076.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T04:00:01.832 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-get update 2024-09-19T04:00:01.957 INFO:teuthology.orchestra.run.smithi076.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T04:00:01.963 INFO:teuthology.orchestra.run.smithi076.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T04:00:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T04:00:02.011 INFO:teuthology.orchestra.run.smithi076.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T04:00:02.147 INFO:teuthology.orchestra.run.smithi076.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T04:00:02.148 INFO:teuthology.orchestra.run.smithi076.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T04:00:02.148 INFO:teuthology.orchestra.run.smithi076.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T04:00:02.158 INFO:teuthology.orchestra.run.smithi076.stderr:done 2024-09-19T04:00:02.171 DEBUG:teuthology.orchestra.run.smithi076:> sudo shutdown -r now 2024-09-19T04:00:02.319 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T04:00:02.551 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T04:00:02.626 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T04:00:02.708 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T04:00:03.911 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-09-19T04:00:03.930 DEBUG:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-19T04:00:03.991 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-09-19T04:00:04.203 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2024-09-19T04:00:04.204 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2024-09-19T04:00:04.379 INFO:teuthology.orchestra.run.smithi116.stdout:linux-image-generic is already the newest version (5.15.0.122.122). 2024-09-19T04:00:04.379 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T04:00:04.379 INFO:teuthology.orchestra.run.smithi116.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T04:00:04.380 INFO:teuthology.orchestra.run.smithi116.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T04:00:04.380 INFO:teuthology.orchestra.run.smithi116.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T04:00:04.380 INFO:teuthology.orchestra.run.smithi116.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T04:00:04.380 INFO:teuthology.orchestra.run.smithi116.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T04:00:04.380 INFO:teuthology.orchestra.run.smithi116.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T04:00:04.419 INFO:teuthology.orchestra.run.smithi116.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-19T04:00:04.421 DEBUG:teuthology.orchestra.run.smithi116:> dpkg -s linux-image-generic 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Package: linux-image-generic 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Status: install ok installed 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Priority: optional 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Section: kernel 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Installed-Size: 21 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Maintainer: Ubuntu Kernel Team 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Architecture: amd64 2024-09-19T04:00:04.433 INFO:teuthology.orchestra.run.smithi116.stdout:Source: linux-meta 2024-09-19T04:00:04.434 INFO:teuthology.orchestra.run.smithi116.stdout:Version: 5.15.0.122.122 2024-09-19T04:00:04.434 INFO:teuthology.orchestra.run.smithi116.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-122), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-19T04:00:04.434 INFO:teuthology.orchestra.run.smithi116.stdout:Depends: linux-image-5.15.0-122-generic, linux-modules-extra-5.15.0-122-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-19T04:00:04.434 INFO:teuthology.orchestra.run.smithi116.stdout:Recommends: thermald 2024-09-19T04:00:04.434 INFO:teuthology.orchestra.run.smithi116.stdout:Description: Generic Linux kernel image 2024-09-19T04:00:04.434 INFO:teuthology.orchestra.run.smithi116.stdout: This package will always depend on the latest generic kernel image 2024-09-19T04:00:04.434 INFO:teuthology.orchestra.run.smithi116.stdout: available. 2024-09-19T04:00:04.434 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-09-19T04:00:04.480 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.Liklgo8tcw 2024-09-19T04:00:04.480 DEBUG:teuthology.orchestra.run.smithi116:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Liklgo8tcw 2024-09-19T04:00:04.533 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.Liklgo8tcw 2024-09-19T04:00:04.652 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.Liklgo8tcw is 10KB 2024-09-19T04:00:04.665 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.Liklgo8tcw 2024-09-19T04:00:04.670 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-19T04:00:04.722 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:00:04.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-19T04:00:04.723 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-19T04:00:04.786 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-122-generic 2024-09-19T04:00:04.786 DEBUG:teuthology.orchestra.run.smithi116:> sudo update-grub 2024-09-19T04:00:06.064 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub' 2024-09-19T04:00:06.065 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-19T04:00:06.067 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2024-09-19T04:00:06.251 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.15.0-122-generic 2024-09-19T04:00:06.257 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.15.0-122-generic 2024-09-19T04:00:06.531 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-19T04:00:06.536 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-19T04:00:06.576 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-19T04:00:06.581 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-19T04:00:06.701 INFO:teuthology.orchestra.run.smithi116.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-19T04:00:06.701 INFO:teuthology.orchestra.run.smithi116.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-19T04:00:06.701 INFO:teuthology.orchestra.run.smithi116.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-19T04:00:06.712 INFO:teuthology.orchestra.run.smithi116.stderr:done 2024-09-19T04:00:06.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo shutdown -r now 2024-09-19T04:00:32.199 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-19T04:00:32.200 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:00:32.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:00:36.730 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-19T04:00:36.730 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-19T04:00:36.731 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:00:50.753 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-09-19T04:00:55.101 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-09-19T04:00:59.759 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:00:59.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:01:02.813 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-09-19T04:01:04.103 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-19T04:01:04.104 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:01:07.169 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-09-19T04:01:14.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:01:14.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:01:17.885 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-09-19T04:01:19.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-19T04:01:19.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:01:32.898 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:01:32.899 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:01:37.597 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2024-09-19T04:01:51.421 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.76 2024-09-19T04:01:52.599 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-19T04:01:52.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:01:52.957 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-09-19T04:01:53.490 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-19T04:01:53.490 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-122-generic"... 2024-09-19T04:01:53.490 DEBUG:teuthology.orchestra.run.smithi116:> uname -r 2024-09-19T04:01:53.537 INFO:teuthology.orchestra.run.smithi116.stdout:5.15.0-122-generic 2024-09-19T04:01:53.537 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-19T04:01:53.537 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:01:53.537 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-19T04:01:54.538 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-19T04:01:54.539 DEBUG:teuthology.orchestra.run.smithi116:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T04:01:54.608 INFO:teuthology.orchestra.run.smithi116.stdout:ttyS1 2024-09-19T04:01:54.627 DEBUG:teuthology.parallel:result is None 2024-09-19T04:02:09.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:02:09.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:02:09.720 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-09-19T04:02:10.270 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:02:10.270 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-122-generic"... 2024-09-19T04:02:10.270 DEBUG:teuthology.orchestra.run.smithi076:> uname -r 2024-09-19T04:02:10.318 INFO:teuthology.orchestra.run.smithi076.stdout:5.15.0-122-generic 2024-09-19T04:02:10.319 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-122-generic vs 5.15.0-122-generic 2024-09-19T04:02:10.319 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T04:02:10.319 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-19T04:02:11.320 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-19T04:02:11.320 DEBUG:teuthology.orchestra.run.smithi076:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-19T04:02:11.687 INFO:teuthology.orchestra.run.smithi076.stdout:ttyS1 2024-09-19T04:02:11.717 DEBUG:teuthology.parallel:result is None 2024-09-19T04:02:11.717 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T04:02:11.724 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T04:02:11.725 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:02:11.727 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T04:02:11.733 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T04:02:11.739 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T04:02:11.745 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T04:02:11.745 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:02:11.776 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T04:02:11.800 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T04:02:11.807 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T04:02:11.808 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:02:11.836 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T04:02:11.848 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:11.855 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:11.857 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:11.864 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T04:02:11.866 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T04:02:11.875 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T04:02:11.876 DEBUG:teuthology.orchestra.run.smithi076:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:02:11.904 DEBUG:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T04:02:11.931 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T04:02:11.940 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T04:02:11.940 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:02:11.960 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T04:02:11.977 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:12.011 DEBUG:teuthology.orchestra.run.smithi076:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:12.059 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:02:12.059 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:02:12.133 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T04:02:12.139 DEBUG:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T04:02:12.185 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:02:12.186 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T04:02:12.252 DEBUG:teuthology.orchestra.run.smithi076:> sudo service rsyslog restart 2024-09-19T04:02:12.254 DEBUG:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2024-09-19T04:02:12.371 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T04:02:12.378 INFO:teuthology.task.internal:Starting timer... 2024-09-19T04:02:12.378 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T04:02:12.395 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T04:02:12.406 DEBUG:teuthology.task.selinux:Excluding smithi076: OS 'ubuntu' does not support SELinux 2024-09-19T04:02:12.406 DEBUG:teuthology.task.selinux:Excluding smithi116: OS 'ubuntu' does not support SELinux 2024-09-19T04:02:12.406 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T04:02:12.406 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-19T04:02:12.406 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T04:02:12.406 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T04:02:12.417 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-19T04:02:12.420 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-19T04:02:12.569 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T04:02:12.584 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T04:02:12.585 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi076.front.sepia.ceph.com,smithi116.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T04:09:38.093 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi076.front.sepia.ceph.com'), Remote(name='ubuntu@smithi116.front.sepia.ceph.com')] 2024-09-19T04:09:38.094 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:09:38.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi076.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:09:38.179 DEBUG:teuthology.orchestra.run.smithi076:> true 2024-09-19T04:09:38.257 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi076.front.sepia.ceph.com' 2024-09-19T04:09:38.257 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-19T04:09:38.258 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T04:09:38.338 DEBUG:teuthology.orchestra.run.smithi116:> true 2024-09-19T04:09:38.417 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi116.front.sepia.ceph.com' 2024-09-19T04:09:38.417 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T04:09:38.430 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T04:09:38.430 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:09:38.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:09:38.433 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T04:09:38.433 DEBUG:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:09:38.461 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Command line: ntpd -gq 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: ---------------------------------------------------- 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: ntp-4 is maintained by Network Time Foundation, 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: corporation. Support and training for ntp-4 are 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: available at https://www.nwtime.org/support 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: ---------------------------------------------------- 2024-09-19T04:09:38.462 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: proto: precision = 0.060 usec (-24) 2024-09-19T04:09:38.463 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: basedate set to 2022-02-04 2024-09-19T04:09:38.463 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: gps base set to 2022-02-06 (week 2196) 2024-09-19T04:09:38.463 INFO:teuthology.orchestra.run.smithi076.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T04:09:38.463 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T04:09:38.463 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: restrict ::: KOD does nothing without LIMITED. 2024-09-19T04:09:38.464 INFO:teuthology.orchestra.run.smithi076.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-19T04:09:38.465 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Listen and drop on 0 v6wildcard [::]:123 2024-09-19T04:09:38.465 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-19T04:09:38.465 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Listen normally on 2 lo 127.0.0.1:123 2024-09-19T04:09:38.465 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Listen normally on 3 enp3s0f1 172.21.15.76:123 2024-09-19T04:09:38.465 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Listen normally on 4 lo [::1]:123 2024-09-19T04:09:38.465 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:47e1%5]:123 2024-09-19T04:09:38.465 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:38 ntpd[18505]: Listening on routing socket on fd #22 for interface updates 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Command line: ntpd -gq 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: ---------------------------------------------------- 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: ntp-4 is maintained by Network Time Foundation, 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: corporation. Support and training for ntp-4 are 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: available at https://www.nwtime.org/support 2024-09-19T04:09:38.487 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: ---------------------------------------------------- 2024-09-19T04:09:38.488 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: proto: precision = 0.044 usec (-24) 2024-09-19T04:09:38.488 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: basedate set to 2022-02-04 2024-09-19T04:09:38.488 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: gps base set to 2022-02-06 (week 2196) 2024-09-19T04:09:38.488 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T04:09:38.488 INFO:teuthology.orchestra.run.smithi116.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-19T04:09:38.489 INFO:teuthology.orchestra.run.smithi116.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: restrict ::: KOD does nothing without LIMITED. 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Listen and drop on 0 v6wildcard [::]:123 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Listen normally on 2 lo 127.0.0.1:123 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Listen normally on 3 enp3s0f1 172.21.15.116:123 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Listen normally on 4 lo [::1]:123 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:3f67%5]:123 2024-09-19T04:09:38.490 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:38 ntpd[18483]: Listening on routing socket on fd #22 for interface updates 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:39 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:39.461 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:39 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:39 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:39 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:39 ntpd[18505]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:39.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:39 ntpd[18505]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:39 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:39.487 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:39 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:39 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:39 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:39 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:39.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:39 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:40 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:40.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:40 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:41.461 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:41 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:41 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:41 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:41 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:41 ntpd[18505]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:41.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:41 ntpd[18505]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:41.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:41.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:41.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:41.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:41.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:41 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:42.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:42 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:42.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:42 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:42.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:42 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:42.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:42 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:42.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:42 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:42.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:42 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:43.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:43 ntpd[18505]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:43.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:43.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:43.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:43.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:43.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:43 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:44.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:44 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:44.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:44 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:44.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:44 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:44.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:44 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:44.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:44 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:44.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:44 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:45.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.462 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:45.463 INFO:teuthology.orchestra.run.smithi076.stdout:19 Sep 04:09:45 ntpd[18505]: ntpd: time slew -0.002068 s 2024-09-19T04:09:45.464 INFO:teuthology.orchestra.run.smithi076.stdout:ntpd: time slew -0.002068s 2024-09-19T04:09:45.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-19T04:09:45.464 INFO:teuthology.orchestra.run.smithi076.stderr:19 Sep 04:09:45 ntpd[18505]: can't open /var/log/ntpstats/loopstats.20240919: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:45.488 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:45.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-19T04:09:45.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: can't open /var/log/ntpstats/rawstats.20240919: Permission denied 2024-09-19T04:09:45.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-19T04:09:45.489 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: can't open /var/log/ntpstats/peerstats.20240919: Permission denied 2024-09-19T04:09:45.489 INFO:teuthology.orchestra.run.smithi116.stdout:19 Sep 04:09:45 ntpd[18483]: ntpd: time slew +0.000331 s 2024-09-19T04:09:45.489 INFO:teuthology.orchestra.run.smithi116.stdout:ntpd: time slew +0.000331s 2024-09-19T04:09:45.490 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-19T04:09:45.490 INFO:teuthology.orchestra.run.smithi116.stderr:19 Sep 04:09:45 ntpd[18483]: can't open /var/log/ntpstats/loopstats.20240919: Permission denied 2024-09-19T04:09:45.517 INFO:teuthology.orchestra.run.smithi076.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T04:09:45.517 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================== 2024-09-19T04:09:45.518 INFO:teuthology.orchestra.run.smithi076.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.518 INFO:teuthology.orchestra.run.smithi076.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.518 INFO:teuthology.orchestra.run.smithi076.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.518 INFO:teuthology.orchestra.run.smithi076.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.538 INFO:teuthology.orchestra.run.smithi116.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T04:09:45.538 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================== 2024-09-19T04:09:45.539 INFO:teuthology.orchestra.run.smithi116.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.539 INFO:teuthology.orchestra.run.smithi116.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.539 INFO:teuthology.orchestra.run.smithi116.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.539 INFO:teuthology.orchestra.run.smithi116.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-19T04:09:45.540 INFO:teuthology.run_tasks:Running task install... 2024-09-19T04:09:45.548 DEBUG:teuthology.task.install:project ceph 2024-09-19T04:09:45.549 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T04:09:45.549 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T04:09:45.549 INFO:teuthology.task.install:Using flavor: default 2024-09-19T04:09:45.555 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T04:09:45.555 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T04:09:45.555 DEBUG:teuthology.orchestra.run.smithi076:> sudo apt-key list | grep Ceph 2024-09-19T04:09:45.556 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-key list | grep Ceph 2024-09-19T04:09:45.618 INFO:teuthology.orchestra.run.smithi076.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-19T04:09:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-19T04:09:45.646 INFO:teuthology.orchestra.run.smithi076.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-19T04:09:45.646 INFO:teuthology.orchestra.run.smithi076.stdout:uid [ unknown] Ceph.com (release key) 2024-09-19T04:09:45.647 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-19T04:09:45.647 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-19T04:09:45.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:09:45.667 INFO:teuthology.orchestra.run.smithi116.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-19T04:09:45.667 INFO:teuthology.orchestra.run.smithi116.stdout:uid [ unknown] Ceph.com (release key) 2024-09-19T04:09:45.667 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-19T04:09:45.667 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-19T04:09:45.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:09:45.800 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default/ 2024-09-19T04:09:45.800 INFO:teuthology.task.install.deb:Package version is 19.1.1-361-g2f599a46-1jammy 2024-09-19T04:09:45.808 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default/ 2024-09-19T04:09:45.808 INFO:teuthology.task.install.deb:Package version is 19.1.1-361-g2f599a46-1jammy 2024-09-19T04:09:45.909 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:09:45.909 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-19T04:09:45.915 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:09:45.915 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-19T04:09:45.923 DEBUG:teuthology.orchestra.run.smithi076:> sudo apt-get update 2024-09-19T04:09:45.929 DEBUG:teuthology.orchestra.run.smithi116:> sudo apt-get update 2024-09-19T04:09:46.086 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T04:09:46.096 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T04:09:46.096 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-19T04:09:46.105 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-19T04:09:46.112 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T04:09:46.122 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-19T04:09:46.148 INFO:teuthology.orchestra.run.smithi076.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy InRelease 2024-09-19T04:09:46.151 INFO:teuthology.orchestra.run.smithi116.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy InRelease 2024-09-19T04:09:46.171 INFO:teuthology.orchestra.run.smithi076.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-19T04:09:46.172 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-09-19T04:09:46.193 INFO:teuthology.orchestra.run.smithi076.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-19T04:09:46.199 INFO:teuthology.orchestra.run.smithi116.stdout:Ign:6 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-19T04:09:46.208 INFO:teuthology.orchestra.run.smithi076.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T04:09:46.217 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-19T04:09:46.226 INFO:teuthology.orchestra.run.smithi076.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-09-19T04:09:46.242 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-19T04:09:46.252 INFO:teuthology.orchestra.run.smithi076.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-09-19T04:09:46.273 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-09-19T04:09:46.431 INFO:teuthology.orchestra.run.smithi076.stdout:Fetched 29.0 kB in 0s (86.9 kB/s) 2024-09-19T04:09:46.454 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 29.0 kB in 0s (84.5 kB/s) 2024-09-19T04:09:47.429 INFO:teuthology.orchestra.run.smithi076.stdout:Reading package lists... 2024-09-19T04:09:47.439 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-09-19T04:09:47.446 DEBUG:teuthology.orchestra.run.smithi076:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-361-g2f599a46-1jammy cephadm=19.1.1-361-g2f599a46-1jammy ceph-mds=19.1.1-361-g2f599a46-1jammy ceph-mgr=19.1.1-361-g2f599a46-1jammy ceph-common=19.1.1-361-g2f599a46-1jammy ceph-fuse=19.1.1-361-g2f599a46-1jammy ceph-test=19.1.1-361-g2f599a46-1jammy ceph-volume=19.1.1-361-g2f599a46-1jammy radosgw=19.1.1-361-g2f599a46-1jammy python3-rados=19.1.1-361-g2f599a46-1jammy python3-rgw=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy python3-rbd=19.1.1-361-g2f599a46-1jammy libcephfs2=19.1.1-361-g2f599a46-1jammy libcephfs-dev=19.1.1-361-g2f599a46-1jammy librados2=19.1.1-361-g2f599a46-1jammy librbd1=19.1.1-361-g2f599a46-1jammy rbd-fuse=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy cephfs-shell=19.1.1-361-g2f599a46-1jammy cephfs-top=19.1.1-361-g2f599a46-1jammy cephfs-mirror=19.1.1-361-g2f599a46-1jammy 2024-09-19T04:09:47.455 DEBUG:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.1.1-361-g2f599a46-1jammy cephadm=19.1.1-361-g2f599a46-1jammy ceph-mds=19.1.1-361-g2f599a46-1jammy ceph-mgr=19.1.1-361-g2f599a46-1jammy ceph-common=19.1.1-361-g2f599a46-1jammy ceph-fuse=19.1.1-361-g2f599a46-1jammy ceph-test=19.1.1-361-g2f599a46-1jammy ceph-volume=19.1.1-361-g2f599a46-1jammy radosgw=19.1.1-361-g2f599a46-1jammy python3-rados=19.1.1-361-g2f599a46-1jammy python3-rgw=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy python3-rbd=19.1.1-361-g2f599a46-1jammy libcephfs2=19.1.1-361-g2f599a46-1jammy libcephfs-dev=19.1.1-361-g2f599a46-1jammy librados2=19.1.1-361-g2f599a46-1jammy librbd1=19.1.1-361-g2f599a46-1jammy rbd-fuse=19.1.1-361-g2f599a46-1jammy python3-cephfs=19.1.1-361-g2f599a46-1jammy cephfs-shell=19.1.1-361-g2f599a46-1jammy cephfs-top=19.1.1-361-g2f599a46-1jammy cephfs-mirror=19.1.1-361-g2f599a46-1jammy 2024-09-19T04:09:47.509 INFO:teuthology.orchestra.run.smithi076.stdout:Reading package lists... 2024-09-19T04:09:47.518 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-09-19T04:09:47.716 INFO:teuthology.orchestra.run.smithi076.stdout:Building dependency tree... 2024-09-19T04:09:47.717 INFO:teuthology.orchestra.run.smithi076.stdout:Reading state information... 2024-09-19T04:09:47.727 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2024-09-19T04:09:47.728 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2024-09-19T04:09:47.922 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T04:09:47.922 INFO:teuthology.orchestra.run.smithi116.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T04:09:47.922 INFO:teuthology.orchestra.run.smithi116.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T04:09:47.922 INFO:teuthology.orchestra.run.smithi116.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T04:09:47.923 INFO:teuthology.orchestra.run.smithi116.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T04:09:47.923 INFO:teuthology.orchestra.run.smithi116.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T04:09:47.923 INFO:teuthology.orchestra.run.smithi116.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T04:09:47.924 INFO:teuthology.orchestra.run.smithi116.stdout:The following additional packages will be installed: 2024-09-19T04:09:47.924 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-19T04:09:47.924 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-19T04:09:47.924 INFO:teuthology.orchestra.run.smithi116.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T04:09:47.924 INFO:teuthology.orchestra.run.smithi116.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-19T04:09:47.926 INFO:teuthology.orchestra.run.smithi116.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-19T04:09:47.927 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout:Suggested packages: 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-19T04:09:47.928 INFO:teuthology.orchestra.run.smithi116.stdout:Recommended packages: 2024-09-19T04:09:47.929 INFO:teuthology.orchestra.run.smithi116.stdout: btrfs-tools 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout:The following additional packages will be installed: 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-19T04:09:47.930 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-19T04:09:47.931 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout:Suggested packages: 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-19T04:09:47.932 INFO:teuthology.orchestra.run.smithi076.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-19T04:09:47.933 INFO:teuthology.orchestra.run.smithi076.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-19T04:09:47.933 INFO:teuthology.orchestra.run.smithi076.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-19T04:09:47.933 INFO:teuthology.orchestra.run.smithi076.stdout:Recommended packages: 2024-09-19T04:09:47.933 INFO:teuthology.orchestra.run.smithi076.stdout: btrfs-tools 2024-09-19T04:09:48.034 INFO:teuthology.orchestra.run.smithi116.stdout:The following NEW packages will be installed: 2024-09-19T04:09:48.034 INFO:teuthology.orchestra.run.smithi116.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-19T04:09:48.034 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-19T04:09:48.034 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-19T04:09:48.036 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-19T04:09:48.036 INFO:teuthology.orchestra.run.smithi116.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-19T04:09:48.037 INFO:teuthology.orchestra.run.smithi116.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-19T04:09:48.038 INFO:teuthology.orchestra.run.smithi116.stdout: socat unzip xmlstarlet zip 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout:The following NEW packages will be installed: 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-19T04:09:48.040 INFO:teuthology.orchestra.run.smithi076.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-19T04:09:48.041 INFO:teuthology.orchestra.run.smithi076.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-19T04:09:48.042 INFO:teuthology.orchestra.run.smithi076.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-19T04:09:48.042 INFO:teuthology.orchestra.run.smithi076.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-19T04:09:48.042 INFO:teuthology.orchestra.run.smithi076.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-19T04:09:48.042 INFO:teuthology.orchestra.run.smithi076.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-19T04:09:48.042 INFO:teuthology.orchestra.run.smithi076.stdout: socat unzip xmlstarlet zip 2024-09-19T04:09:48.191 INFO:teuthology.orchestra.run.smithi076.stdout:0 upgraded, 124 newly installed, 0 to remove and 331 not upgraded. 2024-09-19T04:09:48.192 INFO:teuthology.orchestra.run.smithi076.stdout:Need to get 181 MB of archives. 2024-09-19T04:09:48.192 INFO:teuthology.orchestra.run.smithi076.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-19T04:09:48.192 INFO:teuthology.orchestra.run.smithi076.stdout:Get:1 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-361-g2f599a46-1jammy [3,590 kB] 2024-09-19T04:09:48.225 INFO:teuthology.orchestra.run.smithi116.stdout:0 upgraded, 124 newly installed, 0 to remove and 331 not upgraded. 2024-09-19T04:09:48.225 INFO:teuthology.orchestra.run.smithi116.stdout:Need to get 181 MB of archives. 2024-09-19T04:09:48.226 INFO:teuthology.orchestra.run.smithi116.stdout:After this operation, 831 MB of additional disk space will be used. 2024-09-19T04:09:48.226 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-19T04:09:48.226 INFO:teuthology.orchestra.run.smithi076.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-19T04:09:48.237 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.1.1-361-g2f599a46-1jammy [3,590 kB] 2024-09-19T04:09:48.308 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-19T04:09:48.308 INFO:teuthology.orchestra.run.smithi076.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-19T04:09:48.353 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-19T04:09:48.354 INFO:teuthology.orchestra.run.smithi076.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-19T04:09:48.381 INFO:teuthology.orchestra.run.smithi076.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-361-g2f599a46-1jammy [3,241 kB] 2024-09-19T04:09:48.529 INFO:teuthology.orchestra.run.smithi076.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-361-g2f599a46-1jammy [719 kB] 2024-09-19T04:09:48.548 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-19T04:09:48.549 INFO:teuthology.orchestra.run.smithi076.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-19T04:09:48.557 INFO:teuthology.orchestra.run.smithi076.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-361-g2f599a46-1jammy [357 kB] 2024-09-19T04:09:48.637 INFO:teuthology.orchestra.run.smithi076.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-361-g2f599a46-1jammy [32.8 kB] 2024-09-19T04:09:48.640 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-19T04:09:48.641 INFO:teuthology.orchestra.run.smithi076.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-19T04:09:48.650 INFO:teuthology.orchestra.run.smithi076.stdout:Get:11 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-361-g2f599a46-1jammy [180 kB] 2024-09-19T04:09:48.653 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-19T04:09:48.653 INFO:teuthology.orchestra.run.smithi076.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-19T04:09:48.676 INFO:teuthology.orchestra.run.smithi076.stdout:Get:13 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-361-g2f599a46-1jammy [67.8 kB] 2024-09-19T04:09:48.702 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-19T04:09:48.703 INFO:teuthology.orchestra.run.smithi076.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-19T04:09:48.717 INFO:teuthology.orchestra.run.smithi076.stdout:Get:15 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-361-g2f599a46-1jammy [333 kB] 2024-09-19T04:09:48.780 INFO:teuthology.orchestra.run.smithi076.stdout:Get:16 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-361-g2f599a46-1jammy [6,875 kB] 2024-09-19T04:09:48.876 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-19T04:09:48.876 INFO:teuthology.orchestra.run.smithi076.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-19T04:09:48.884 INFO:teuthology.orchestra.run.smithi116.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-19T04:09:48.885 INFO:teuthology.orchestra.run.smithi076.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-19T04:09:48.911 INFO:teuthology.orchestra.run.smithi116.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-19T04:09:48.912 INFO:teuthology.orchestra.run.smithi076.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-19T04:09:48.920 INFO:teuthology.orchestra.run.smithi116.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-19T04:09:48.920 INFO:teuthology.orchestra.run.smithi076.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-19T04:09:48.921 INFO:teuthology.orchestra.run.smithi116.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-19T04:09:48.921 INFO:teuthology.orchestra.run.smithi076.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-19T04:09:48.941 INFO:teuthology.orchestra.run.smithi116.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-19T04:09:48.942 INFO:teuthology.orchestra.run.smithi076.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-19T04:09:48.945 INFO:teuthology.orchestra.run.smithi116.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-19T04:09:48.947 INFO:teuthology.orchestra.run.smithi076.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-19T04:09:48.949 INFO:teuthology.orchestra.run.smithi116.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-19T04:09:48.951 INFO:teuthology.orchestra.run.smithi076.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-19T04:09:48.952 INFO:teuthology.orchestra.run.smithi116.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-19T04:09:48.953 INFO:teuthology.orchestra.run.smithi076.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-19T04:09:48.959 INFO:teuthology.orchestra.run.smithi116.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-19T04:09:48.959 INFO:teuthology.orchestra.run.smithi076.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-19T04:09:49.034 INFO:teuthology.orchestra.run.smithi116.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-19T04:09:49.035 INFO:teuthology.orchestra.run.smithi076.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-19T04:09:49.038 INFO:teuthology.orchestra.run.smithi116.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-19T04:09:49.040 INFO:teuthology.orchestra.run.smithi076.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-19T04:09:49.042 INFO:teuthology.orchestra.run.smithi116.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-19T04:09:49.043 INFO:teuthology.orchestra.run.smithi116.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-19T04:09:49.044 INFO:teuthology.orchestra.run.smithi076.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-19T04:09:49.044 INFO:teuthology.orchestra.run.smithi116.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-19T04:09:49.044 INFO:teuthology.orchestra.run.smithi076.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-19T04:09:49.045 INFO:teuthology.orchestra.run.smithi076.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-19T04:09:49.046 INFO:teuthology.orchestra.run.smithi116.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-19T04:09:49.047 INFO:teuthology.orchestra.run.smithi116.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-19T04:09:49.047 INFO:teuthology.orchestra.run.smithi076.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-19T04:09:49.047 INFO:teuthology.orchestra.run.smithi076.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-19T04:09:49.048 INFO:teuthology.orchestra.run.smithi076.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-19T04:09:49.048 INFO:teuthology.orchestra.run.smithi116.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-19T04:09:49.049 INFO:teuthology.orchestra.run.smithi116.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-19T04:09:49.049 INFO:teuthology.orchestra.run.smithi076.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-19T04:09:49.121 INFO:teuthology.orchestra.run.smithi116.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-19T04:09:49.122 INFO:teuthology.orchestra.run.smithi076.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-19T04:09:49.131 INFO:teuthology.orchestra.run.smithi116.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-19T04:09:49.131 INFO:teuthology.orchestra.run.smithi076.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-19T04:09:49.132 INFO:teuthology.orchestra.run.smithi116.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-19T04:09:49.132 INFO:teuthology.orchestra.run.smithi076.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-19T04:09:49.137 INFO:teuthology.orchestra.run.smithi116.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-19T04:09:49.137 INFO:teuthology.orchestra.run.smithi076.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-19T04:09:49.138 INFO:teuthology.orchestra.run.smithi116.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-19T04:09:49.138 INFO:teuthology.orchestra.run.smithi076.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-19T04:09:49.139 INFO:teuthology.orchestra.run.smithi116.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-19T04:09:49.139 INFO:teuthology.orchestra.run.smithi076.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-19T04:09:49.140 INFO:teuthology.orchestra.run.smithi116.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-19T04:09:49.140 INFO:teuthology.orchestra.run.smithi076.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-19T04:09:49.141 INFO:teuthology.orchestra.run.smithi116.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-19T04:09:49.142 INFO:teuthology.orchestra.run.smithi116.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-19T04:09:49.203 INFO:teuthology.orchestra.run.smithi116.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-19T04:09:49.204 INFO:teuthology.orchestra.run.smithi076.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-19T04:09:49.205 INFO:teuthology.orchestra.run.smithi076.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-19T04:09:49.206 INFO:teuthology.orchestra.run.smithi076.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-19T04:09:49.225 INFO:teuthology.orchestra.run.smithi116.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-19T04:09:49.226 INFO:teuthology.orchestra.run.smithi116.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-19T04:09:49.227 INFO:teuthology.orchestra.run.smithi116.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-19T04:09:49.237 INFO:teuthology.orchestra.run.smithi116.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-19T04:09:49.238 INFO:teuthology.orchestra.run.smithi116.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-19T04:09:49.239 INFO:teuthology.orchestra.run.smithi116.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-19T04:09:49.240 INFO:teuthology.orchestra.run.smithi116.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-19T04:09:49.285 INFO:teuthology.orchestra.run.smithi076.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-19T04:09:49.285 INFO:teuthology.orchestra.run.smithi116.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-19T04:09:49.287 INFO:teuthology.orchestra.run.smithi076.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-19T04:09:49.287 INFO:teuthology.orchestra.run.smithi076.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-19T04:09:49.289 INFO:teuthology.orchestra.run.smithi116.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-19T04:09:49.300 INFO:teuthology.orchestra.run.smithi076.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-19T04:09:49.301 INFO:teuthology.orchestra.run.smithi076.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-19T04:09:49.301 INFO:teuthology.orchestra.run.smithi076.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-19T04:09:49.366 INFO:teuthology.orchestra.run.smithi116.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-19T04:09:49.367 INFO:teuthology.orchestra.run.smithi076.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-19T04:09:49.368 INFO:teuthology.orchestra.run.smithi116.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-19T04:09:49.369 INFO:teuthology.orchestra.run.smithi076.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-19T04:09:49.370 INFO:teuthology.orchestra.run.smithi116.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-19T04:09:49.373 INFO:teuthology.orchestra.run.smithi076.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-19T04:09:49.374 INFO:teuthology.orchestra.run.smithi116.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-19T04:09:49.375 INFO:teuthology.orchestra.run.smithi076.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-19T04:09:49.375 INFO:teuthology.orchestra.run.smithi116.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-19T04:09:49.377 INFO:teuthology.orchestra.run.smithi116.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-19T04:09:49.381 INFO:teuthology.orchestra.run.smithi116.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-19T04:09:49.448 INFO:teuthology.orchestra.run.smithi076.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-19T04:09:49.450 INFO:teuthology.orchestra.run.smithi076.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-19T04:09:49.455 INFO:teuthology.orchestra.run.smithi076.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-19T04:09:49.457 INFO:teuthology.orchestra.run.smithi076.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-19T04:09:49.459 INFO:teuthology.orchestra.run.smithi076.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-19T04:09:49.459 INFO:teuthology.orchestra.run.smithi076.stdout:Get:61 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-361-g2f599a46-1jammy [112 kB] 2024-09-19T04:09:49.461 INFO:teuthology.orchestra.run.smithi076.stdout:Get:62 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-361-g2f599a46-1jammy [472 kB] 2024-09-19T04:09:49.464 INFO:teuthology.orchestra.run.smithi076.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-19T04:09:49.474 INFO:teuthology.orchestra.run.smithi076.stdout:Get:64 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-361-g2f599a46-1jammy [25.9 MB] 2024-09-19T04:09:49.541 INFO:teuthology.orchestra.run.smithi116.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-19T04:09:49.543 INFO:teuthology.orchestra.run.smithi116.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-19T04:09:49.545 INFO:teuthology.orchestra.run.smithi116.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-19T04:09:49.563 INFO:teuthology.orchestra.run.smithi116.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-19T04:09:49.565 INFO:teuthology.orchestra.run.smithi116.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-19T04:09:49.615 INFO:teuthology.orchestra.run.smithi116.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-19T04:09:49.618 INFO:teuthology.orchestra.run.smithi116.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-19T04:09:49.619 INFO:teuthology.orchestra.run.smithi116.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-19T04:09:49.628 INFO:teuthology.orchestra.run.smithi076.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-19T04:09:49.630 INFO:teuthology.orchestra.run.smithi076.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-19T04:09:49.633 INFO:teuthology.orchestra.run.smithi076.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-19T04:09:49.633 INFO:teuthology.orchestra.run.smithi076.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-19T04:09:49.635 INFO:teuthology.orchestra.run.smithi076.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-19T04:09:49.698 INFO:teuthology.orchestra.run.smithi076.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-19T04:09:49.700 INFO:teuthology.orchestra.run.smithi116.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-19T04:09:49.700 INFO:teuthology.orchestra.run.smithi116.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-19T04:09:49.701 INFO:teuthology.orchestra.run.smithi076.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-19T04:09:49.701 INFO:teuthology.orchestra.run.smithi076.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-19T04:09:49.707 INFO:teuthology.orchestra.run.smithi116.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-19T04:09:49.708 INFO:teuthology.orchestra.run.smithi116.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-19T04:09:49.709 INFO:teuthology.orchestra.run.smithi116.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-19T04:09:49.710 INFO:teuthology.orchestra.run.smithi116.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-19T04:09:49.711 INFO:teuthology.orchestra.run.smithi116.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-19T04:09:49.712 INFO:teuthology.orchestra.run.smithi116.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-19T04:09:49.718 INFO:teuthology.orchestra.run.smithi116.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-19T04:09:49.778 INFO:teuthology.orchestra.run.smithi076.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-19T04:09:49.778 INFO:teuthology.orchestra.run.smithi076.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-19T04:09:49.788 INFO:teuthology.orchestra.run.smithi116.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-19T04:09:49.792 INFO:teuthology.orchestra.run.smithi116.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-19T04:09:49.793 INFO:teuthology.orchestra.run.smithi116.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-19T04:09:49.795 INFO:teuthology.orchestra.run.smithi076.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-19T04:09:49.796 INFO:teuthology.orchestra.run.smithi076.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-19T04:09:49.796 INFO:teuthology.orchestra.run.smithi076.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-19T04:09:49.797 INFO:teuthology.orchestra.run.smithi076.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-19T04:09:49.798 INFO:teuthology.orchestra.run.smithi076.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-19T04:09:49.799 INFO:teuthology.orchestra.run.smithi076.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-19T04:09:49.805 INFO:teuthology.orchestra.run.smithi076.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-19T04:09:49.870 INFO:teuthology.orchestra.run.smithi116.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-19T04:09:49.876 INFO:teuthology.orchestra.run.smithi076.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-19T04:09:49.876 INFO:teuthology.orchestra.run.smithi116.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-19T04:09:49.877 INFO:teuthology.orchestra.run.smithi116.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-19T04:09:49.880 INFO:teuthology.orchestra.run.smithi076.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-19T04:09:49.880 INFO:teuthology.orchestra.run.smithi116.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-19T04:09:49.881 INFO:teuthology.orchestra.run.smithi116.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-19T04:09:49.881 INFO:teuthology.orchestra.run.smithi076.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-19T04:09:49.892 INFO:teuthology.orchestra.run.smithi116.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-19T04:09:49.893 INFO:teuthology.orchestra.run.smithi116.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-19T04:09:49.951 INFO:teuthology.orchestra.run.smithi116.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-19T04:09:49.952 INFO:teuthology.orchestra.run.smithi116.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-19T04:09:49.953 INFO:teuthology.orchestra.run.smithi116.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-19T04:09:49.957 INFO:teuthology.orchestra.run.smithi076.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-19T04:09:49.964 INFO:teuthology.orchestra.run.smithi076.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-19T04:09:49.965 INFO:teuthology.orchestra.run.smithi076.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-19T04:09:49.967 INFO:teuthology.orchestra.run.smithi076.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-19T04:09:49.969 INFO:teuthology.orchestra.run.smithi076.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2024-09-19T04:09:49.980 INFO:teuthology.orchestra.run.smithi076.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-19T04:09:49.981 INFO:teuthology.orchestra.run.smithi076.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-19T04:09:50.032 INFO:teuthology.orchestra.run.smithi116.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-19T04:09:50.036 INFO:teuthology.orchestra.run.smithi116.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-19T04:09:50.038 INFO:teuthology.orchestra.run.smithi116.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-19T04:09:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-19T04:09:50.039 INFO:teuthology.orchestra.run.smithi076.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-19T04:09:50.041 INFO:teuthology.orchestra.run.smithi076.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-19T04:09:50.055 INFO:teuthology.orchestra.run.smithi116.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-19T04:09:50.056 INFO:teuthology.orchestra.run.smithi116.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-19T04:09:50.057 INFO:teuthology.orchestra.run.smithi116.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-19T04:09:50.061 INFO:teuthology.orchestra.run.smithi116.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-19T04:09:50.114 INFO:teuthology.orchestra.run.smithi116.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-19T04:09:50.120 INFO:teuthology.orchestra.run.smithi076.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-19T04:09:50.124 INFO:teuthology.orchestra.run.smithi076.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-19T04:09:50.126 INFO:teuthology.orchestra.run.smithi076.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-19T04:09:50.143 INFO:teuthology.orchestra.run.smithi076.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-19T04:09:50.144 INFO:teuthology.orchestra.run.smithi076.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-19T04:09:50.145 INFO:teuthology.orchestra.run.smithi076.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-19T04:09:50.150 INFO:teuthology.orchestra.run.smithi076.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-19T04:09:50.202 INFO:teuthology.orchestra.run.smithi076.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-19T04:09:50.325 INFO:teuthology.orchestra.run.smithi076.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-361-g2f599a46-1jammy [5,145 kB] 2024-09-19T04:09:50.411 INFO:teuthology.orchestra.run.smithi076.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-361-g2f599a46-1jammy [245 kB] 2024-09-19T04:09:50.465 INFO:teuthology.orchestra.run.smithi076.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-361-g2f599a46-1jammy [124 kB] 2024-09-19T04:09:50.474 INFO:teuthology.orchestra.run.smithi076.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-361-g2f599a46-1jammy [1,555 kB] 2024-09-19T04:09:50.513 INFO:teuthology.orchestra.run.smithi076.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-361-g2f599a46-1jammy [6,142 kB] 2024-09-19T04:09:50.686 INFO:teuthology.orchestra.run.smithi076.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-361-g2f599a46-1jammy [22.8 MB] 2024-09-19T04:09:51.557 INFO:teuthology.orchestra.run.smithi076.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-361-g2f599a46-1jammy [14.1 kB] 2024-09-19T04:09:51.558 INFO:teuthology.orchestra.run.smithi076.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-361-g2f599a46-1jammy [877 kB] 2024-09-19T04:09:51.575 INFO:teuthology.orchestra.run.smithi076.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-361-g2f599a46-1jammy [2,462 kB] 2024-09-19T04:09:51.615 INFO:teuthology.orchestra.run.smithi076.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-361-g2f599a46-1jammy [794 kB] 2024-09-19T04:09:51.625 INFO:teuthology.orchestra.run.smithi076.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-361-g2f599a46-1jammy [152 kB] 2024-09-19T04:09:51.627 INFO:teuthology.orchestra.run.smithi076.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-361-g2f599a46-1jammy [2,347 kB] 2024-09-19T04:09:51.697 INFO:teuthology.orchestra.run.smithi076.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-361-g2f599a46-1jammy [8,625 kB] 2024-09-19T04:09:51.964 INFO:teuthology.orchestra.run.smithi076.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-361-g2f599a46-1jammy [14.2 kB] 2024-09-19T04:09:51.969 INFO:teuthology.orchestra.run.smithi076.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-361-g2f599a46-1jammy [51.4 MB] 2024-09-19T04:09:53.587 INFO:teuthology.orchestra.run.smithi116.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.1.1-361-g2f599a46-1jammy [3,241 kB] 2024-09-19T04:09:53.830 INFO:teuthology.orchestra.run.smithi076.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-361-g2f599a46-1jammy [130 kB] 2024-09-19T04:09:53.832 INFO:teuthology.orchestra.run.smithi076.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-361-g2f599a46-1jammy [236 kB] 2024-09-19T04:09:53.837 INFO:teuthology.orchestra.run.smithi076.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-361-g2f599a46-1jammy [34.0 kB] 2024-09-19T04:09:53.838 INFO:teuthology.orchestra.run.smithi076.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-361-g2f599a46-1jammy [23.5 kB] 2024-09-19T04:09:53.839 INFO:teuthology.orchestra.run.smithi076.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-361-g2f599a46-1jammy [39.0 kB] 2024-09-19T04:09:53.840 INFO:teuthology.orchestra.run.smithi076.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-361-g2f599a46-1jammy [13.6 MB] 2024-09-19T04:09:53.946 INFO:teuthology.orchestra.run.smithi116.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.1.1-361-g2f599a46-1jammy [719 kB] 2024-09-19T04:09:53.998 INFO:teuthology.orchestra.run.smithi116.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.1.1-361-g2f599a46-1jammy [357 kB] 2024-09-19T04:09:54.108 INFO:teuthology.orchestra.run.smithi076.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-361-g2f599a46-1jammy [91.5 kB] 2024-09-19T04:09:54.114 INFO:teuthology.orchestra.run.smithi116.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.1.1-361-g2f599a46-1jammy [32.8 kB] 2024-09-19T04:09:54.115 INFO:teuthology.orchestra.run.smithi116.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.1.1-361-g2f599a46-1jammy [180 kB] 2024-09-19T04:09:54.119 INFO:teuthology.orchestra.run.smithi116.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.1.1-361-g2f599a46-1jammy [67.8 kB] 2024-09-19T04:09:54.120 INFO:teuthology.orchestra.run.smithi116.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.1.1-361-g2f599a46-1jammy [333 kB] 2024-09-19T04:09:54.134 INFO:teuthology.orchestra.run.smithi116.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.1.1-361-g2f599a46-1jammy [6,875 kB] 2024-09-19T04:09:54.306 INFO:teuthology.orchestra.run.smithi116.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.1.1-361-g2f599a46-1jammy [112 kB] 2024-09-19T04:09:54.308 INFO:teuthology.orchestra.run.smithi116.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.1.1-361-g2f599a46-1jammy [472 kB] 2024-09-19T04:09:54.318 INFO:teuthology.orchestra.run.smithi116.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.1.1-361-g2f599a46-1jammy [25.9 MB] 2024-09-19T04:09:54.736 INFO:teuthology.orchestra.run.smithi076.stdout:Fetched 181 MB in 6s (29.9 MB/s) 2024-09-19T04:09:54.809 INFO:teuthology.orchestra.run.smithi116.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.1.1-361-g2f599a46-1jammy [5,145 kB] 2024-09-19T04:09:54.864 INFO:teuthology.orchestra.run.smithi116.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.1.1-361-g2f599a46-1jammy [245 kB] 2024-09-19T04:09:54.867 INFO:teuthology.orchestra.run.smithi116.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.1.1-361-g2f599a46-1jammy [124 kB] 2024-09-19T04:09:54.868 INFO:teuthology.orchestra.run.smithi116.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.1.1-361-g2f599a46-1jammy [1,555 kB] 2024-09-19T04:09:54.875 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-19T04:09:54.881 INFO:teuthology.orchestra.run.smithi116.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.1.1-361-g2f599a46-1jammy [6,142 kB] 2024-09-19T04:09:54.974 INFO:teuthology.orchestra.run.smithi076.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-19T04:09:54.979 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-19T04:09:54.980 INFO:teuthology.orchestra.run.smithi116.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.1.1-361-g2f599a46-1jammy [22.8 MB] 2024-09-19T04:09:55.023 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-19T04:09:55.326 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-19T04:09:55.353 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-19T04:09:55.393 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T04:09:55.455 INFO:teuthology.orchestra.run.smithi116.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.1.1-361-g2f599a46-1jammy [14.1 kB] 2024-09-19T04:09:55.456 INFO:teuthology.orchestra.run.smithi116.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.1.1-361-g2f599a46-1jammy [877 kB] 2024-09-19T04:09:55.466 INFO:teuthology.orchestra.run.smithi116.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.1.1-361-g2f599a46-1jammy [2,462 kB] 2024-09-19T04:09:55.497 INFO:teuthology.orchestra.run.smithi116.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.1.1-361-g2f599a46-1jammy [794 kB] 2024-09-19T04:09:55.507 INFO:teuthology.orchestra.run.smithi116.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.1.1-361-g2f599a46-1jammy [152 kB] 2024-09-19T04:09:55.509 INFO:teuthology.orchestra.run.smithi116.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.1.1-361-g2f599a46-1jammy [2,347 kB] 2024-09-19T04:09:55.531 INFO:teuthology.orchestra.run.smithi116.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.1.1-361-g2f599a46-1jammy [8,625 kB] 2024-09-19T04:09:55.660 INFO:teuthology.orchestra.run.smithi116.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.1.1-361-g2f599a46-1jammy [14.2 kB] 2024-09-19T04:09:55.660 INFO:teuthology.orchestra.run.smithi116.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.1.1-361-g2f599a46-1jammy [51.4 MB] 2024-09-19T04:09:55.680 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-cmd2. 2024-09-19T04:09:55.706 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-19T04:09:55.738 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-19T04:09:56.045 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-19T04:09:56.068 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-19T04:09:56.133 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T04:09:56.487 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-19T04:09:56.514 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-19T04:09:56.554 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T04:09:56.752 INFO:teuthology.orchestra.run.smithi116.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.1.1-361-g2f599a46-1jammy [130 kB] 2024-09-19T04:09:56.755 INFO:teuthology.orchestra.run.smithi116.stdout:Get:119 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.1.1-361-g2f599a46-1jammy [236 kB] 2024-09-19T04:09:56.760 INFO:teuthology.orchestra.run.smithi116.stdout:Get:120 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.1.1-361-g2f599a46-1jammy [34.0 kB] 2024-09-19T04:09:56.761 INFO:teuthology.orchestra.run.smithi116.stdout:Get:121 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.1.1-361-g2f599a46-1jammy [23.5 kB] 2024-09-19T04:09:56.761 INFO:teuthology.orchestra.run.smithi116.stdout:Get:122 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.1.1-361-g2f599a46-1jammy [39.0 kB] 2024-09-19T04:09:56.762 INFO:teuthology.orchestra.run.smithi116.stdout:Get:123 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.1.1-361-g2f599a46-1jammy [13.6 MB] 2024-09-19T04:09:56.891 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-19T04:09:56.917 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-19T04:09:56.952 INFO:teuthology.orchestra.run.smithi116.stdout:Get:124 https://4.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.1.1-361-g2f599a46-1jammy [91.5 kB] 2024-09-19T04:09:56.957 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T04:09:57.336 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-19T04:09:57.363 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T04:09:57.411 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:09:57.589 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 181 MB in 9s (20.4 MB/s) 2024-09-19T04:09:57.706 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-19T04:09:57.792 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165990 files and directories currently installed.) 2024-09-19T04:09:57.797 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-19T04:09:57.815 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-19T04:09:57.842 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T04:09:57.881 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:09:57.887 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-19T04:09:58.165 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-19T04:09:58.185 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-19T04:09:58.219 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-19T04:09:58.223 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T04:09:58.245 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T04:09:58.277 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:09:58.535 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cmd2. 2024-09-19T04:09:58.561 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-19T04:09:58.601 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-19T04:09:58.614 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-19T04:09:58.640 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-19T04:09:58.680 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T04:09:58.929 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-19T04:09:58.956 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-19T04:09:59.017 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package librados2. 2024-09-19T04:09:59.038 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T04:09:59.043 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../010-librados2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:09:59.083 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:09:59.308 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-19T04:09:59.323 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-19T04:09:59.366 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T04:09:59.495 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package librbd1. 2024-09-19T04:09:59.522 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../011-librbd1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:09:59.553 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:09:59.677 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-19T04:09:59.704 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-19T04:09:59.744 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T04:09:59.990 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libcephfs2. 2024-09-19T04:10:00.017 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:00.056 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:00.181 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-19T04:10:00.207 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T04:10:00.255 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:10:00.334 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-rados. 2024-09-19T04:10:00.361 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../013-python3-rados_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:00.401 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:00.634 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-19T04:10:00.660 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T04:10:00.700 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:10:00.913 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-19T04:10:00.939 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:00.971 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:01.053 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-19T04:10:01.080 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-19T04:10:01.138 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:10:01.215 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-cephfs. 2024-09-19T04:10:01.231 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:01.273 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:01.458 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-19T04:10:01.484 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-19T04:10:01.516 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T04:10:01.568 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-19T04:10:01.595 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:01.635 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:01.989 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-prettytable. 2024-09-19T04:10:02.015 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-19T04:10:02.052 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librados2. 2024-09-19T04:10:02.055 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-19T04:10:02.079 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../010-librados2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:02.119 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:02.291 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-rbd. 2024-09-19T04:10:02.317 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:02.349 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:02.597 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librbd1. 2024-09-19T04:10:02.623 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../011-librbd1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:02.663 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:02.669 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-19T04:10:02.695 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-19T04:10:02.727 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T04:10:03.006 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-19T04:10:03.032 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-19T04:10:03.041 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libcephfs2. 2024-09-19T04:10:03.067 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../012-libcephfs2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:03.072 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T04:10:03.099 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:03.376 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-19T04:10:03.385 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rados. 2024-09-19T04:10:03.391 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-19T04:10:03.400 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../013-python3-rados_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:03.434 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T04:10:03.443 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:03.730 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package lua5.1. 2024-09-19T04:10:03.756 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-19T04:10:03.762 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-19T04:10:03.788 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-19T04:10:03.789 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:03.829 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:04.073 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cephfs. 2024-09-19T04:10:04.089 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../015-python3-cephfs_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:04.100 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package lua-any. 2024-09-19T04:10:04.114 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-19T04:10:04.123 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:04.158 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-19T04:10:04.376 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-19T04:10:04.391 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../016-python3-ceph-common_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:04.394 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package zip. 2024-09-19T04:10:04.409 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-19T04:10:04.426 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:04.452 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking zip (3.0-12build2) ... 2024-09-19T04:10:04.712 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-prettytable. 2024-09-19T04:10:04.738 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-19T04:10:04.765 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package unzip. 2024-09-19T04:10:04.778 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-19T04:10:04.791 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-19T04:10:04.831 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-19T04:10:05.073 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rbd. 2024-09-19T04:10:05.088 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../018-python3-rbd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:05.131 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:05.202 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package luarocks. 2024-09-19T04:10:05.228 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-19T04:10:05.268 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-19T04:10:05.484 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-19T04:10:05.510 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-19T04:10:05.551 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T04:10:05.728 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package librgw2. 2024-09-19T04:10:05.754 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../027-librgw2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:05.795 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-19T04:10:05.810 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-19T04:10:05.845 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T04:10:05.928 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:06.115 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-19T04:10:06.141 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-19T04:10:06.181 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T04:10:06.464 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-rgw. 2024-09-19T04:10:06.490 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:06.492 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua5.1. 2024-09-19T04:10:06.519 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-19T04:10:06.531 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:06.559 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-19T04:10:06.867 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-19T04:10:06.887 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua-any. 2024-09-19T04:10:06.893 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-19T04:10:06.902 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-19T04:10:06.934 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T04:10:06.937 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-19T04:10:07.246 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libradosstriper1. 2024-09-19T04:10:07.272 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:07.312 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:07.340 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package zip. 2024-09-19T04:10:07.366 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-19T04:10:07.406 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking zip (3.0-12build2) ... 2024-09-19T04:10:07.718 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package unzip. 2024-09-19T04:10:07.741 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-common. 2024-09-19T04:10:07.744 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-19T04:10:07.767 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../031-ceph-common_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:07.784 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-19T04:10:07.807 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:08.180 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package luarocks. 2024-09-19T04:10:08.206 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-19T04:10:08.246 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-19T04:10:08.613 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librgw2. 2024-09-19T04:10:08.639 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../027-librgw2_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:08.679 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:09.205 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-base. 2024-09-19T04:10:09.232 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../032-ceph-base_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:09.239 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rgw. 2024-09-19T04:10:09.266 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../028-python3-rgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:09.281 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:09.315 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:09.634 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-19T04:10:09.660 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-19T04:10:09.701 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T04:10:09.818 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-19T04:10:09.838 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-19T04:10:09.877 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-19T04:10:09.995 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libradosstriper1. 2024-09-19T04:10:10.021 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../030-libradosstriper1_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:10.053 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:10.180 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-cheroot. 2024-09-19T04:10:10.207 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-19T04:10:10.247 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T04:10:10.490 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-common. 2024-09-19T04:10:10.516 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../031-ceph-common_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:10.557 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:10.584 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-19T04:10:10.610 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-19T04:10:10.650 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-19T04:10:10.920 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-19T04:10:10.946 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-19T04:10:10.978 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-19T04:10:11.282 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-19T04:10:11.308 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-19T04:10:11.348 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-19T04:10:11.652 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-tempora. 2024-09-19T04:10:11.678 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-19T04:10:11.718 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-19T04:10:11.937 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-base. 2024-09-19T04:10:11.963 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../032-ceph-base_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:11.997 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-portend. 2024-09-19T04:10:12.012 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:12.023 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-19T04:10:12.063 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-19T04:10:12.366 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-19T04:10:12.393 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-19T04:10:12.433 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-19T04:10:12.736 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-19T04:10:12.741 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-19T04:10:12.762 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-19T04:10:12.767 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-19T04:10:12.803 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-19T04:10:12.807 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-19T04:10:13.094 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cheroot. 2024-09-19T04:10:13.120 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-19T04:10:13.160 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T04:10:13.165 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-natsort. 2024-09-19T04:10:13.178 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-19T04:10:13.215 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-19T04:10:13.446 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-19T04:10:13.472 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-19T04:10:13.513 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-19T04:10:13.552 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-logutils. 2024-09-19T04:10:13.576 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-19T04:10:13.618 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-19T04:10:13.816 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-19T04:10:13.842 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-19T04:10:13.882 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-19T04:10:13.907 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-19T04:10:13.933 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-19T04:10:13.974 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-19T04:10:14.168 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-19T04:10:14.195 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-19T04:10:14.235 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-19T04:10:14.285 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-mako. 2024-09-19T04:10:14.311 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-19T04:10:14.351 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T04:10:14.479 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-tempora. 2024-09-19T04:10:14.494 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-19T04:10:14.529 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-19T04:10:14.688 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-19T04:10:14.714 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-19T04:10:14.754 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-19T04:10:14.832 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-portend. 2024-09-19T04:10:14.860 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-19T04:10:14.899 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-19T04:10:15.040 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-19T04:10:15.067 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-19T04:10:15.106 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-19T04:10:15.210 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-19T04:10:15.236 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-19T04:10:15.277 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-19T04:10:15.376 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-webob. 2024-09-19T04:10:15.392 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-19T04:10:15.434 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T04:10:15.538 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-19T04:10:15.564 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-19T04:10:15.604 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-19T04:10:15.762 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-waitress. 2024-09-19T04:10:15.788 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-19T04:10:15.831 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T04:10:15.958 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-natsort. 2024-09-19T04:10:15.984 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-19T04:10:16.024 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-19T04:10:16.344 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-logutils. 2024-09-19T04:10:16.370 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-19T04:10:16.390 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-tempita. 2024-09-19T04:10:16.410 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-19T04:10:16.416 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-19T04:10:16.456 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T04:10:16.697 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-19T04:10:16.723 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-19T04:10:16.742 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-paste. 2024-09-19T04:10:16.768 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-19T04:10:16.771 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-19T04:10:16.808 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T04:10:17.074 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-mako. 2024-09-19T04:10:17.101 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-19T04:10:17.141 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T04:10:17.153 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-19T04:10:17.180 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-19T04:10:17.219 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T04:10:17.461 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-19T04:10:17.488 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-19T04:10:17.497 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-19T04:10:17.512 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-19T04:10:17.527 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-19T04:10:17.547 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-19T04:10:17.780 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-19T04:10:17.807 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-19T04:10:17.842 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-webtest. 2024-09-19T04:10:17.846 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-19T04:10:17.868 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-19T04:10:17.908 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-19T04:10:18.133 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-webob. 2024-09-19T04:10:18.159 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-19T04:10:18.177 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pecan. 2024-09-19T04:10:18.193 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-19T04:10:18.199 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T04:10:18.227 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T04:10:18.540 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-certifi. 2024-09-19T04:10:18.544 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-waitress. 2024-09-19T04:10:18.555 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-19T04:10:18.570 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-19T04:10:18.590 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-19T04:10:18.614 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T04:10:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-idna. 2024-09-19T04:10:18.936 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-19T04:10:18.948 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-tempita. 2024-09-19T04:10:18.973 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-19T04:10:18.976 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T04:10:19.013 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T04:10:19.287 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-urllib3. 2024-09-19T04:10:19.313 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-19T04:10:19.325 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-paste. 2024-09-19T04:10:19.351 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-19T04:10:19.353 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T04:10:19.383 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T04:10:19.673 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-requests. 2024-09-19T04:10:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-19T04:10:19.710 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-19T04:10:19.726 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-19T04:10:19.740 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T04:10:19.769 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T04:10:20.038 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-19T04:10:20.051 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-19T04:10:20.065 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-19T04:10:20.077 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-19T04:10:20.105 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-19T04:10:20.117 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T04:10:20.404 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-19T04:10:20.408 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-webtest. 2024-09-19T04:10:20.418 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:20.434 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-19T04:10:20.461 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:20.474 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-19T04:10:20.783 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-19T04:10:20.809 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:20.810 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pecan. 2024-09-19T04:10:20.837 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-19T04:10:20.849 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:20.877 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T04:10:21.185 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-19T04:10:21.212 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-19T04:10:21.249 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-certifi. 2024-09-19T04:10:21.251 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-19T04:10:21.275 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-19T04:10:21.315 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-19T04:10:22.347 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-openssl. 2024-09-19T04:10:22.374 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-19T04:10:22.427 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-idna. 2024-09-19T04:10:22.453 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-19T04:10:22.493 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T04:10:22.555 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-19T04:10:22.891 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mgr. 2024-09-19T04:10:22.918 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:22.954 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-urllib3. 2024-09-19T04:10:22.958 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:22.981 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-19T04:10:23.012 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T04:10:23.257 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-requests. 2024-09-19T04:10:23.273 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-19T04:10:23.311 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mon. 2024-09-19T04:10:23.316 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T04:10:23.337 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:23.369 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:23.618 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-19T04:10:23.645 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-19T04:10:23.685 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T04:10:23.947 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-osd. 2024-09-19T04:10:23.954 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-19T04:10:23.973 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:23.982 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:24.013 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:24.021 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:24.410 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-19T04:10:24.437 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:24.484 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:24.795 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-19T04:10:24.822 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-19T04:10:24.862 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-19T04:10:25.117 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph. 2024-09-19T04:10:25.131 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-openssl. 2024-09-19T04:10:25.143 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../068-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:25.157 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-19T04:10:25.183 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:25.189 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-19T04:10:25.494 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-fuse. 2024-09-19T04:10:25.520 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:25.526 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr. 2024-09-19T04:10:25.552 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../065-ceph-mgr_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:25.560 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:25.592 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:25.872 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mds. 2024-09-19T04:10:25.898 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:25.938 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:25.979 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mon. 2024-09-19T04:10:26.005 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../066-ceph-mon_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:26.045 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:26.332 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package cephadm. 2024-09-19T04:10:26.359 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../071-cephadm_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:26.398 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:26.591 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-osd. 2024-09-19T04:10:26.617 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../067-ceph-osd_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:26.648 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:26.710 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-19T04:10:26.736 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-19T04:10:26.776 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-19T04:10:27.079 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-19T04:10:27.094 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T04:10:27.137 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T04:10:27.676 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph. 2024-09-19T04:10:27.692 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../068-ceph_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:27.751 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:28.142 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-babel. 2024-09-19T04:10:28.169 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T04:10:28.208 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T04:10:28.229 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-fuse. 2024-09-19T04:10:28.256 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../069-ceph-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:28.304 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:28.536 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-jinja2. 2024-09-19T04:10:28.562 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-19T04:10:28.602 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T04:10:28.682 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mds. 2024-09-19T04:10:28.709 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../070-ceph-mds_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:28.748 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:28.972 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-19T04:10:28.998 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:29.038 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:29.118 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephadm. 2024-09-19T04:10:29.145 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../071-cephadm_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:29.184 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:29.392 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-19T04:10:29.418 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-19T04:10:29.458 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-19T04:10:29.495 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-19T04:10:29.522 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-19T04:10:29.562 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-19T04:10:29.761 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-routes. 2024-09-19T04:10:29.787 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-19T04:10:29.827 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T04:10:29.881 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-19T04:10:29.896 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T04:10:29.939 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T04:10:30.155 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-19T04:10:30.182 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:30.221 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:30.961 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-babel. 2024-09-19T04:10:30.988 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-19T04:10:31.027 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T04:10:31.355 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jinja2. 2024-09-19T04:10:31.382 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-19T04:10:31.422 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T04:10:31.493 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-19T04:10:31.509 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-19T04:10:31.551 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T04:10:31.750 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-19T04:10:31.777 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:31.816 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:32.029 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-joblib. 2024-09-19T04:10:32.045 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-19T04:10:32.079 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T04:10:32.195 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-19T04:10:32.221 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-19T04:10:32.261 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-19T04:10:32.449 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-19T04:10:32.464 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-19T04:10:32.539 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-routes. 2024-09-19T04:10:32.554 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-19T04:10:32.589 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T04:10:32.609 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-19T04:10:32.883 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-19T04:10:32.895 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-sklearn. 2024-09-19T04:10:32.899 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:32.922 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-19T04:10:32.933 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:32.962 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T04:10:33.662 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-19T04:10:33.678 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:33.720 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:34.230 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-19T04:10:34.250 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-19T04:10:34.288 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T04:10:34.648 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-cachetools. 2024-09-19T04:10:34.675 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-19T04:10:34.714 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-19T04:10:34.767 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-joblib. 2024-09-19T04:10:34.782 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-19T04:10:34.816 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T04:10:35.034 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-19T04:10:35.061 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-19T04:10:35.100 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-19T04:10:35.178 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-19T04:10:35.206 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-19T04:10:35.330 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-19T04:10:35.429 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-19T04:10:35.456 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-19T04:10:35.495 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T04:10:35.608 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-sklearn. 2024-09-19T04:10:35.636 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-19T04:10:35.666 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T04:10:35.789 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-rsa. 2024-09-19T04:10:35.816 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-19T04:10:35.856 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-19T04:10:36.110 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-google-auth. 2024-09-19T04:10:36.136 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-19T04:10:36.176 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-19T04:10:36.392 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-19T04:10:36.420 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:36.458 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:36.512 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-19T04:10:36.539 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-19T04:10:36.578 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T04:10:36.890 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-websocket. 2024-09-19T04:10:36.917 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-19T04:10:36.956 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-19T04:10:37.387 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cachetools. 2024-09-19T04:10:37.414 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-19T04:10:37.453 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-19T04:10:37.502 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-19T04:10:37.528 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-19T04:10:37.568 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T04:10:37.764 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-19T04:10:37.792 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-19T04:10:37.830 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-19T04:10:38.134 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-19T04:10:38.150 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-19T04:10:38.184 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T04:10:38.265 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-19T04:10:38.281 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:38.323 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:38.453 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rsa. 2024-09-19T04:10:38.480 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-19T04:10:38.511 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-19T04:10:38.651 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-19T04:10:38.678 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-19T04:10:38.717 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T04:10:38.798 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-google-auth. 2024-09-19T04:10:38.826 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-19T04:10:38.865 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-19T04:10:39.037 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-19T04:10:39.064 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T04:10:39.103 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T04:10:39.185 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-19T04:10:39.212 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-19T04:10:39.251 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T04:10:39.330 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package jq. 2024-09-19T04:10:39.346 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T04:10:39.388 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-19T04:10:39.563 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-websocket. 2024-09-19T04:10:39.590 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-19T04:10:39.629 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-19T04:10:39.641 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package socat. 2024-09-19T04:10:39.656 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-19T04:10:39.699 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-19T04:10:39.966 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-19T04:10:39.993 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-19T04:10:39.994 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package xmlstarlet. 2024-09-19T04:10:40.011 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-19T04:10:40.034 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T04:10:40.052 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-19T04:10:40.423 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-test. 2024-09-19T04:10:40.450 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../099-ceph-test_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:40.497 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:40.631 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-19T04:10:40.647 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:40.689 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:40.984 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-19T04:10:41.011 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-19T04:10:41.050 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T04:10:41.379 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-19T04:10:41.406 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T04:10:41.445 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T04:10:41.706 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package jq. 2024-09-19T04:10:41.722 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-19T04:10:41.764 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-19T04:10:42.000 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package socat. 2024-09-19T04:10:42.016 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-19T04:10:42.050 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-19T04:10:42.337 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package xmlstarlet. 2024-09-19T04:10:42.364 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-19T04:10:42.403 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-19T04:10:42.685 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ceph-volume. 2024-09-19T04:10:42.701 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:42.735 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:42.740 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-test. 2024-09-19T04:10:42.757 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../099-ceph-test_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:42.790 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:43.223 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-19T04:10:43.250 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:43.288 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:43.608 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-colorama. 2024-09-19T04:10:43.636 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-19T04:10:43.674 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-19T04:10:43.935 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package cephfs-shell. 2024-09-19T04:10:43.951 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:44.019 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:44.271 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package cephfs-top. 2024-09-19T04:10:44.287 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:44.329 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:44.565 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-19T04:10:44.581 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:44.614 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:44.926 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-19T04:10:44.954 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-19T04:10:44.992 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T04:10:45.170 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-volume. 2024-09-19T04:10:45.197 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../100-ceph-volume_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:45.236 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:45.324 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-19T04:10:45.350 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-19T04:10:45.388 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-19T04:10:45.591 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-19T04:10:45.618 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../101-cephfs-mirror_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:45.657 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:45.692 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package nvme-cli. 2024-09-19T04:10:45.720 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-19T04:10:45.750 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T04:10:45.977 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-colorama. 2024-09-19T04:10:46.004 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-19T04:10:46.043 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-19T04:10:46.155 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package pkg-config. 2024-09-19T04:10:46.183 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-19T04:10:46.221 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T04:10:46.304 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephfs-shell. 2024-09-19T04:10:46.331 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../103-cephfs-shell_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:46.370 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:46.491 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-19T04:10:46.518 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-19T04:10:46.557 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-19T04:10:46.623 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephfs-top. 2024-09-19T04:10:46.647 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../104-cephfs-top_19.1.1-361-g2f599a46-1jammy_all.deb ... 2024-09-19T04:10:46.689 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:46.900 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-19T04:10:46.916 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../105-libcephfs-dev_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:46.958 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:46.977 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-19T04:10:47.005 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-19T04:10:47.043 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-19T04:10:47.278 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-19T04:10:47.305 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-19T04:10:47.344 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T04:10:47.346 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pastescript. 2024-09-19T04:10:47.374 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-19T04:10:47.412 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-19T04:10:47.707 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-19T04:10:47.715 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pluggy. 2024-09-19T04:10:47.735 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-19T04:10:47.743 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-19T04:10:47.773 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-19T04:10:47.781 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-19T04:10:48.127 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-psutil. 2024-09-19T04:10:48.128 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package nvme-cli. 2024-09-19T04:10:48.154 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-19T04:10:48.155 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2024-09-19T04:10:48.186 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T04:10:48.193 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-19T04:10:48.521 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-py. 2024-09-19T04:10:48.549 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-19T04:10:48.558 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package pkg-config. 2024-09-19T04:10:48.585 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-19T04:10:48.595 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-19T04:10:48.615 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T04:10:48.869 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-19T04:10:48.896 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-19T04:10:48.907 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pygments. 2024-09-19T04:10:48.934 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-19T04:10:48.935 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-19T04:10:48.973 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T04:10:49.347 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-19T04:10:49.374 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-19T04:10:49.413 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-19T04:10:49.444 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-19T04:10:49.472 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-19T04:10:49.510 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-19T04:10:49.716 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pastescript. 2024-09-19T04:10:49.743 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-19T04:10:49.780 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-toml. 2024-09-19T04:10:49.782 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-19T04:10:49.807 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-19T04:10:49.846 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-19T04:10:50.103 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pluggy. 2024-09-19T04:10:50.130 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-19T04:10:50.132 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-pytest. 2024-09-19T04:10:50.160 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-19T04:10:50.169 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-19T04:10:50.199 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T04:10:50.463 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-psutil. 2024-09-19T04:10:50.491 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-19T04:10:50.519 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package python3-simplejson. 2024-09-19T04:10:50.530 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-19T04:10:50.546 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-19T04:10:50.585 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-19T04:10:50.879 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-19T04:10:50.883 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-py. 2024-09-19T04:10:50.907 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-19T04:10:50.910 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-19T04:10:50.946 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-19T04:10:50.957 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-19T04:10:51.311 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pygments. 2024-09-19T04:10:51.339 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-19T04:10:51.377 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T04:10:51.658 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package radosgw. 2024-09-19T04:10:51.685 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../122-radosgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:51.732 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:51.840 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-19T04:10:51.867 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-19T04:10:51.909 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-19T04:10:52.201 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-toml. 2024-09-19T04:10:52.228 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-19T04:10:52.267 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-19T04:10:52.528 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pytest. 2024-09-19T04:10:52.544 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-19T04:10:52.578 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T04:10:52.585 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package rbd-fuse. 2024-09-19T04:10:52.613 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:52.652 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:52.940 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-simplejson. 2024-09-19T04:10:52.967 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-19T04:10:53.006 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-19T04:10:53.223 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-19T04:10:53.276 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-19T04:10:53.292 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-19T04:10:53.334 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-19T04:10:53.473 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T04:10:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T04:10:53.855 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-19T04:10:54.055 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package radosgw. 2024-09-19T04:10:54.082 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../122-radosgw_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:54.121 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:54.162 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-19T04:10:54.575 INFO:teuthology.orchestra.run.smithi076.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-19T04:10:54.918 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:54.949 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package rbd-fuse. 2024-09-19T04:10:54.965 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../123-rbd-fuse_19.1.1-361-g2f599a46-1jammy_amd64.deb ... 2024-09-19T04:10:54.999 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:55.442 INFO:teuthology.orchestra.run.smithi076.stdout:Adding system user cephadm....done 2024-09-19T04:10:55.545 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-19T04:10:55.590 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T04:10:55.796 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-19T04:10:55.858 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-19T04:10:55.921 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2024-09-19T04:10:56.112 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-19T04:10:56.181 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-19T04:10:56.238 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-19T04:10:56.489 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-19T04:10:56.505 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-19T04:10:56.740 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T04:10:56.866 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-19T04:10:56.898 INFO:teuthology.orchestra.run.smithi116.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-19T04:10:56.965 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:57.168 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T04:10:57.473 INFO:teuthology.orchestra.run.smithi116.stdout:Adding system user cephadm....done 2024-09-19T04:10:57.553 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-19T04:10:57.621 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-19T04:10:57.813 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-19T04:10:57.881 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-19T04:10:58.064 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-19T04:10:58.132 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-19T04:10:58.258 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-19T04:10:58.332 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-19T04:10:58.509 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-19T04:10:58.667 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-19T04:10:58.769 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-19T04:10:58.895 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-19T04:10:58.918 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-19T04:10:59.221 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-19T04:10:59.236 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-19T04:10:59.495 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:10:59.573 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-19T04:10:59.754 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T04:10:59.816 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-19T04:10:59.881 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T04:11:00.007 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-19T04:11:00.067 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-19T04:11:00.274 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T04:11:00.335 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-19T04:11:00.400 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-19T04:11:00.612 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-19T04:11:00.651 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-19T04:11:00.863 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-19T04:11:01.106 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-19T04:11:01.241 INFO:teuthology.orchestra.run.smithi076.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-19T04:11:01.266 INFO:teuthology.orchestra.run.smithi076.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-19T04:11:01.330 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T04:11:01.398 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-ceph-argparse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:01.456 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-19T04:11:01.641 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-19T04:11:01.767 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-19T04:11:01.823 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T04:11:01.893 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-19T04:11:01.949 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-19T04:11:02.153 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-19T04:11:02.209 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T04:11:02.278 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-19T04:11:02.452 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-19T04:11:02.529 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-19T04:11:02.770 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up zip (3.0-12build2) ... 2024-09-19T04:11:02.873 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-19T04:11:02.904 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T04:11:02.928 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-19T04:11:02.983 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-19T04:11:03.109 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-19T04:11:03.469 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-19T04:11:03.556 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T04:11:03.594 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-19T04:11:03.807 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T04:11:03.854 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-19T04:11:03.933 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-19T04:11:04.059 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T04:11:04.113 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-19T04:11:04.360 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-19T04:11:04.423 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up zip (3.0-12build2) ... 2024-09-19T04:11:04.549 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-19T04:11:04.770 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-19T04:11:05.021 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T04:11:05.192 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-19T04:11:05.397 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T04:11:05.460 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-19T04:11:05.586 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-19T04:11:05.682 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-19T04:11:05.695 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-19T04:11:05.983 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T04:11:05.988 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-19T04:11:06.310 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-19T04:11:06.372 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-19T04:11:06.561 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-19T04:11:06.654 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-19T04:11:06.695 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:06.996 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T04:11:07.008 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-19T04:11:07.318 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-19T04:11:07.347 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T04:11:07.619 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-19T04:11:07.954 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-19T04:11:08.205 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-19T04:11:08.339 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:08.567 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T04:11:08.649 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-19T04:11:08.822 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:11:08.950 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-19T04:11:08.956 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-19T04:11:09.224 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T04:11:09.350 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-19T04:11:09.476 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T04:11:09.785 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-19T04:11:10.048 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-19T04:11:10.189 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-19T04:11:10.296 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:11:10.422 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-19T04:11:10.440 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:11:10.600 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-19T04:11:10.715 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-19T04:11:10.967 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-19T04:11:11.013 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-19T04:11:11.135 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-19T04:11:11.209 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-19T04:11:11.262 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-19T04:11:11.460 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-19T04:11:11.571 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-19T04:11:11.728 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-19T04:11:11.830 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-19T04:11:11.988 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T04:11:12.073 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:11:12.113 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-19T04:11:12.174 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-19T04:11:12.406 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T04:11:12.467 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-19T04:11:12.532 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T04:11:12.735 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-19T04:11:12.842 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T04:11:13.003 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-19T04:11:13.126 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-19T04:11:13.254 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-19T04:11:13.385 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T04:11:13.496 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-19T04:11:13.529 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-19T04:11:13.655 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T04:11:13.743 INFO:teuthology.orchestra.run.smithi076.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-19T04:11:13.764 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-19T04:11:13.865 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-19T04:11:13.981 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T04:11:14.107 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T04:11:14.133 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-19T04:11:14.259 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-19T04:11:14.492 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-19T04:11:14.585 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-19T04:11:14.760 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-19T04:11:14.894 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-19T04:11:14.895 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-19T04:11:15.146 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-19T04:11:15.154 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:11:15.271 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-19T04:11:15.280 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T04:11:15.398 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-19T04:11:15.471 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-19T04:11:15.606 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T04:11:15.724 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-19T04:11:15.833 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-19T04:11:15.890 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T04:11:16.158 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-19T04:11:16.227 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-19T04:11:16.345 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-19T04:11:16.486 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-19T04:11:16.612 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-19T04:11:16.613 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-19T04:11:16.863 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-19T04:11:16.897 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T04:11:16.981 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-19T04:11:17.165 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-19T04:11:17.299 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-19T04:11:17.533 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-19T04:11:17.599 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-19T04:11:17.826 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T04:11:17.884 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-19T04:11:18.104 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-19T04:11:18.188 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T04:11:18.314 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:18.363 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-19T04:11:18.423 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:18.557 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-19T04:11:18.657 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-19T04:11:18.910 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:18.916 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-19T04:11:19.036 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:19.163 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-19T04:11:19.284 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-19T04:11:19.448 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:19.574 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:19.586 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-19T04:11:19.700 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:19.907 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T04:11:19.907 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T04:11:19.921 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-19T04:11:20.048 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librados2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.174 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libsqlite3-mod-ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.300 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-19T04:11:20.367 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.493 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.619 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.644 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libcephfs2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.754 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libradosstriper1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.754 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:20.863 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-19T04:11:20.880 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T04:11:21.131 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librbd1 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:21.232 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-modules-core (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:21.350 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:21.563 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T04:11:21.564 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-19T04:11:22.034 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libcephfs-dev (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.109 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.161 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rados (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.234 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.311 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librgw2 (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.360 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.437 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephfs-top (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.503 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.563 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-19T04:11:22.629 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:22.755 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:23.273 INFO:teuthology.orchestra.run.smithi076.stdout:Adding group ceph....done 2024-09-19T04:11:23.641 INFO:teuthology.orchestra.run.smithi076.stdout:Adding system user ceph....done 2024-09-19T04:11:23.800 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rbd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:23.926 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up rbd-fuse (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:24.052 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:24.143 INFO:teuthology.orchestra.run.smithi076.stdout:Setting system user ceph properties....done 2024-09-19T04:11:24.163 INFO:teuthology.orchestra.run.smithi076.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-19T04:11:24.170 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cephfs (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:24.288 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephfs-shell (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:24.299 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-19T04:11:24.388 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-common (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:24.671 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-19T04:11:24.931 INFO:teuthology.orchestra.run.smithi116.stdout:Adding group ceph....done 2024-09-19T04:11:25.140 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:25.266 INFO:teuthology.orchestra.run.smithi116.stdout:Adding system user ceph....done 2024-09-19T04:11:25.326 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:11:25.326 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:11:25.384 INFO:teuthology.orchestra.run.smithi116.stdout:Setting system user ceph properties....done 2024-09-19T04:11:25.399 INFO:teuthology.orchestra.run.smithi116.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-19T04:11:25.535 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-19T04:11:25.732 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:25.834 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:25.872 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-19T04:11:26.298 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:11:26.298 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:11:26.373 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephfs-mirror (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:26.583 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:11:26.583 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-19T04:11:26.752 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:27.049 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-test (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:27.102 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-19T04:11:27.192 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up radosgw (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:27.562 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:27.686 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:11:27.686 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-19T04:11:27.793 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T04:11:27.793 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T04:11:28.203 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-base (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:28.248 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:28.444 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T04:11:28.445 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T04:11:28.515 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-19T04:11:28.907 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:28.979 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mds (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:29.164 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T04:11:29.164 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T04:11:29.218 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T04:11:29.218 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-19T04:11:29.647 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:29.673 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:29.773 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:29.890 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T04:11:29.890 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-19T04:11:29.899 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:30.091 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T04:11:30.091 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T04:11:30.357 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-osd (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:30.558 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:30.634 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T04:11:30.634 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-19T04:11:30.701 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:30.827 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:30.987 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:31.101 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-k8sevents (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:31.324 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-diskprediction-local (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:31.348 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T04:11:31.457 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T04:11:31.533 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mon (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:31.601 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T04:11:31.744 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T04:11:31.744 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-19T04:11:32.218 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-cephadm (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:32.369 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:32.495 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-dashboard (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:32.646 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-volume (19.1.1-361-g2f599a46-1jammy) ... 2024-09-19T04:11:32.719 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T04:11:32.999 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T04:11:33.091 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T04:11:33.244 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T04:11:33.440 INFO:teuthology.orchestra.run.smithi076.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T04:11:33.444 DEBUG:teuthology.orchestra.run.smithi076:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-19T04:11:33.499 INFO:teuthology.orchestra.run.smithi076.stdout:Reading package lists... 2024-09-19T04:11:33.711 INFO:teuthology.orchestra.run.smithi076.stdout:Building dependency tree... 2024-09-19T04:11:33.712 INFO:teuthology.orchestra.run.smithi076.stdout:Reading state information... 2024-09-19T04:11:33.903 INFO:teuthology.orchestra.run.smithi076.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-19T04:11:33.904 INFO:teuthology.orchestra.run.smithi076.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-19T04:11:33.904 INFO:teuthology.orchestra.run.smithi076.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-19T04:11:33.904 INFO:teuthology.orchestra.run.smithi076.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T04:11:33.904 INFO:teuthology.orchestra.run.smithi076.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T04:11:33.904 INFO:teuthology.orchestra.run.smithi076.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T04:11:33.904 INFO:teuthology.orchestra.run.smithi076.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T04:11:33.905 INFO:teuthology.orchestra.run.smithi076.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T04:11:33.905 INFO:teuthology.orchestra.run.smithi076.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T04:11:33.905 INFO:teuthology.orchestra.run.smithi076.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T04:11:33.906 INFO:teuthology.orchestra.run.smithi076.stdout:The following additional packages will be installed: 2024-09-19T04:11:33.906 INFO:teuthology.orchestra.run.smithi076.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-19T04:11:33.906 INFO:teuthology.orchestra.run.smithi076.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-19T04:11:33.906 INFO:teuthology.orchestra.run.smithi076.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-19T04:11:33.906 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-19T04:11:33.906 INFO:teuthology.orchestra.run.smithi076.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T04:11:33.907 INFO:teuthology.orchestra.run.smithi076.stdout:Suggested packages: 2024-09-19T04:11:33.907 INFO:teuthology.orchestra.run.smithi076.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-19T04:11:33.907 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-19T04:11:33.907 INFO:teuthology.orchestra.run.smithi076.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-19T04:11:33.949 INFO:teuthology.orchestra.run.smithi076.stdout:The following NEW packages will be installed: 2024-09-19T04:11:33.950 INFO:teuthology.orchestra.run.smithi076.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-19T04:11:33.950 INFO:teuthology.orchestra.run.smithi076.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-19T04:11:33.950 INFO:teuthology.orchestra.run.smithi076.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-19T04:11:33.950 INFO:teuthology.orchestra.run.smithi076.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-19T04:11:33.950 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-19T04:11:33.950 INFO:teuthology.orchestra.run.smithi076.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-19T04:11:33.950 INFO:teuthology.orchestra.run.smithi076.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T04:11:33.951 INFO:teuthology.orchestra.run.smithi076.stdout:The following packages will be upgraded: 2024-09-19T04:11:33.952 INFO:teuthology.orchestra.run.smithi076.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-19T04:11:34.022 INFO:teuthology.orchestra.run.smithi076.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-19T04:11:34.022 INFO:teuthology.orchestra.run.smithi076.stdout:Need to get 55.8 MB of archives. 2024-09-19T04:11:34.022 INFO:teuthology.orchestra.run.smithi076.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-19T04:11:34.022 INFO:teuthology.orchestra.run.smithi076.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-19T04:11:34.156 INFO:teuthology.orchestra.run.smithi076.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-19T04:11:34.164 INFO:teuthology.orchestra.run.smithi076.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-19T04:11:34.205 INFO:teuthology.orchestra.run.smithi076.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-19T04:11:34.205 INFO:teuthology.orchestra.run.smithi076.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-19T04:11:34.206 INFO:teuthology.orchestra.run.smithi076.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-19T04:11:34.217 INFO:teuthology.orchestra.run.smithi076.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-19T04:11:34.217 INFO:teuthology.orchestra.run.smithi076.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-19T04:11:34.219 INFO:teuthology.orchestra.run.smithi076.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-19T04:11:34.221 INFO:teuthology.orchestra.run.smithi076.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-19T04:11:34.221 INFO:teuthology.orchestra.run.smithi076.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-19T04:11:34.223 INFO:teuthology.orchestra.run.smithi076.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-19T04:11:34.224 INFO:teuthology.orchestra.run.smithi076.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-19T04:11:34.240 INFO:teuthology.orchestra.run.smithi076.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-19T04:11:34.241 INFO:teuthology.orchestra.run.smithi076.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-19T04:11:34.242 INFO:teuthology.orchestra.run.smithi076.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-19T04:11:34.369 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T04:11:34.538 INFO:teuthology.orchestra.run.smithi076.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-19T04:11:34.540 INFO:teuthology.orchestra.run.smithi076.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-19T04:11:34.540 INFO:teuthology.orchestra.run.smithi076.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-19T04:11:34.543 INFO:teuthology.orchestra.run.smithi076.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-19T04:11:34.543 INFO:teuthology.orchestra.run.smithi076.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-19T04:11:34.547 INFO:teuthology.orchestra.run.smithi076.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-19T04:11:34.548 INFO:teuthology.orchestra.run.smithi076.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-19T04:11:34.555 INFO:teuthology.orchestra.run.smithi076.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-19T04:11:34.555 INFO:teuthology.orchestra.run.smithi076.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-19T04:11:34.576 INFO:teuthology.orchestra.run.smithi076.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-19T04:11:34.576 INFO:teuthology.orchestra.run.smithi076.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-19T04:11:34.586 INFO:teuthology.orchestra.run.smithi076.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-19T04:11:34.607 INFO:teuthology.orchestra.run.smithi076.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-19T04:11:34.610 INFO:teuthology.orchestra.run.smithi076.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-19T04:11:34.610 INFO:teuthology.orchestra.run.smithi076.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-19T04:11:34.611 INFO:teuthology.orchestra.run.smithi076.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-19T04:11:34.611 INFO:teuthology.orchestra.run.smithi076.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-19T04:11:34.622 INFO:teuthology.orchestra.run.smithi076.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-19T04:11:34.624 INFO:teuthology.orchestra.run.smithi076.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-19T04:11:34.863 INFO:teuthology.orchestra.run.smithi076.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-19T04:11:34.863 INFO:teuthology.orchestra.run.smithi076.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-19T04:11:34.863 INFO:teuthology.orchestra.run.smithi076.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-19T04:11:34.863 INFO:teuthology.orchestra.run.smithi076.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-19T04:11:34.863 INFO:teuthology.orchestra.run.smithi076.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-19T04:11:34.863 INFO:teuthology.orchestra.run.smithi076.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-19T04:11:34.874 INFO:teuthology.orchestra.run.smithi076.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-19T04:11:34.874 INFO:teuthology.orchestra.run.smithi076.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-19T04:11:35.040 INFO:teuthology.orchestra.run.smithi116.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T04:11:35.042 DEBUG:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-19T04:11:35.100 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2024-09-19T04:11:35.286 INFO:teuthology.orchestra.run.smithi076.stdout:Preconfiguring packages ... 2024-09-19T04:11:35.313 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2024-09-19T04:11:35.313 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2024-09-19T04:11:35.431 INFO:teuthology.orchestra.run.smithi076.stdout:Fetched 55.8 MB in 1s (60.9 MB/s) 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages were automatically installed and are no longer required: 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-19T04:11:35.510 INFO:teuthology.orchestra.run.smithi116.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-19T04:11:35.511 INFO:teuthology.orchestra.run.smithi116.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-19T04:11:35.511 INFO:teuthology.orchestra.run.smithi116.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-19T04:11:35.511 INFO:teuthology.orchestra.run.smithi116.stdout:The following additional packages will be installed: 2024-09-19T04:11:35.511 INFO:teuthology.orchestra.run.smithi116.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-19T04:11:35.511 INFO:teuthology.orchestra.run.smithi116.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-19T04:11:35.512 INFO:teuthology.orchestra.run.smithi116.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-19T04:11:35.512 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-19T04:11:35.512 INFO:teuthology.orchestra.run.smithi116.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T04:11:35.513 INFO:teuthology.orchestra.run.smithi116.stdout:Suggested packages: 2024-09-19T04:11:35.513 INFO:teuthology.orchestra.run.smithi116.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-19T04:11:35.513 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-19T04:11:35.513 INFO:teuthology.orchestra.run.smithi116.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-19T04:11:35.554 INFO:teuthology.orchestra.run.smithi116.stdout:The following NEW packages will be installed: 2024-09-19T04:11:35.555 INFO:teuthology.orchestra.run.smithi116.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-19T04:11:35.555 INFO:teuthology.orchestra.run.smithi116.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-19T04:11:35.556 INFO:teuthology.orchestra.run.smithi116.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-19T04:11:35.556 INFO:teuthology.orchestra.run.smithi116.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-19T04:11:35.556 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-19T04:11:35.556 INFO:teuthology.orchestra.run.smithi116.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-19T04:11:35.556 INFO:teuthology.orchestra.run.smithi116.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-19T04:11:35.557 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages will be upgraded: 2024-09-19T04:11:35.557 INFO:teuthology.orchestra.run.smithi116.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-19T04:11:35.738 INFO:teuthology.orchestra.run.smithi116.stdout:4 upgraded, 39 newly installed, 0 to remove and 327 not upgraded. 2024-09-19T04:11:35.739 INFO:teuthology.orchestra.run.smithi116.stdout:Need to get 55.8 MB of archives. 2024-09-19T04:11:35.739 INFO:teuthology.orchestra.run.smithi116.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-19T04:11:35.739 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-19T04:11:35.756 INFO:teuthology.orchestra.run.smithi076.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T04:11:35.761 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T04:11:35.883 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T04:11:36.362 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-19T04:11:36.378 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-19T04:11:36.502 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-19T04:11:36.504 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-19T04:11:36.505 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-19T04:11:36.513 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-19T04:11:36.533 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-19T04:11:36.534 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-19T04:11:36.538 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-19T04:11:36.541 INFO:teuthology.orchestra.run.smithi116.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-19T04:11:36.541 INFO:teuthology.orchestra.run.smithi116.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-19T04:11:36.544 INFO:teuthology.orchestra.run.smithi116.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-19T04:11:36.606 INFO:teuthology.orchestra.run.smithi116.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-19T04:11:36.607 INFO:teuthology.orchestra.run.smithi116.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-19T04:11:36.614 INFO:teuthology.orchestra.run.smithi116.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-19T04:11:36.687 INFO:teuthology.orchestra.run.smithi116.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-19T04:11:37.390 INFO:teuthology.orchestra.run.smithi076.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-19T04:11:37.443 INFO:teuthology.orchestra.run.smithi116.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-19T04:11:37.447 INFO:teuthology.orchestra.run.smithi116.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-19T04:11:37.448 INFO:teuthology.orchestra.run.smithi116.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-19T04:11:37.454 INFO:teuthology.orchestra.run.smithi116.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-19T04:11:37.455 INFO:teuthology.orchestra.run.smithi116.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-19T04:11:37.466 INFO:teuthology.orchestra.run.smithi116.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-19T04:11:37.470 INFO:teuthology.orchestra.run.smithi116.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-19T04:11:37.485 INFO:teuthology.orchestra.run.smithi116.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-19T04:11:37.486 INFO:teuthology.orchestra.run.smithi116.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-19T04:11:37.567 INFO:teuthology.orchestra.run.smithi116.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-19T04:11:37.571 INFO:teuthology.orchestra.run.smithi116.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-19T04:11:37.588 INFO:teuthology.orchestra.run.smithi116.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-19T04:11:37.646 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package flex. 2024-09-19T04:11:37.661 INFO:teuthology.orchestra.run.smithi116.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-19T04:11:37.667 INFO:teuthology.orchestra.run.smithi116.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-19T04:11:37.668 INFO:teuthology.orchestra.run.smithi116.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-19T04:11:37.669 INFO:teuthology.orchestra.run.smithi116.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-19T04:11:37.670 INFO:teuthology.orchestra.run.smithi116.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-19T04:11:37.700 INFO:teuthology.orchestra.run.smithi116.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-19T04:11:37.703 INFO:teuthology.orchestra.run.smithi116.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-19T04:11:37.744 INFO:teuthology.orchestra.run.smithi076.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T04:11:37.750 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-19T04:11:37.812 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-19T04:11:38.262 INFO:teuthology.orchestra.run.smithi116.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-19T04:11:38.262 INFO:teuthology.orchestra.run.smithi116.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-19T04:11:38.262 INFO:teuthology.orchestra.run.smithi116.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-19T04:11:38.262 INFO:teuthology.orchestra.run.smithi116.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-19T04:11:38.263 INFO:teuthology.orchestra.run.smithi116.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-19T04:11:38.263 INFO:teuthology.orchestra.run.smithi116.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-19T04:11:38.286 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T04:11:38.290 INFO:teuthology.orchestra.run.smithi116.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-19T04:11:38.290 INFO:teuthology.orchestra.run.smithi116.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-19T04:11:38.459 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-19T04:11:38.707 INFO:teuthology.orchestra.run.smithi116.stdout:Preconfiguring packages ... 2024-09-19T04:11:38.867 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 55.8 MB in 3s (20.5 MB/s) 2024-09-19T04:11:38.878 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T04:11:39.174 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T04:11:39.180 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T04:11:39.327 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T04:11:39.354 INFO:teuthology.orchestra.run.smithi076.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T04:11:39.359 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T04:11:39.467 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T04:11:39.893 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T04:11:39.906 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-19T04:11:40.024 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T04:11:40.351 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T04:11:40.692 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package bison. 2024-09-19T04:11:40.793 INFO:teuthology.orchestra.run.smithi076.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T04:11:40.799 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-19T04:11:40.805 INFO:teuthology.orchestra.run.smithi116.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-19T04:11:40.844 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T04:11:41.057 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package flex. 2024-09-19T04:11:41.158 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173987 files and directories currently installed.) 2024-09-19T04:11:41.164 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-19T04:11:41.194 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-19T04:11:41.204 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-19T04:11:41.222 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-19T04:11:41.252 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T04:11:41.614 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package dump. 2024-09-19T04:11:41.641 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-19T04:11:41.681 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking dump (0.4b47-3) ... 2024-09-19T04:11:41.712 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T04:11:41.885 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-19T04:11:41.968 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package indent. 2024-09-19T04:11:41.984 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:42.026 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T04:11:42.270 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-19T04:11:42.286 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-19T04:11:42.296 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T04:11:42.320 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T04:11:42.650 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-19T04:11:42.677 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-19T04:11:42.716 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T04:11:42.822 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T04:11:42.827 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T04:11:42.959 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T04:11:42.977 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-19T04:11:42.993 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-19T04:11:43.035 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T04:11:43.331 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-19T04:11:43.359 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-19T04:11:43.376 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-19T04:11:43.389 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T04:11:43.482 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-19T04:11:43.642 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-19T04:11:43.669 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-19T04:11:43.708 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T04:11:43.809 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T04:11:43.961 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libjson-perl. 2024-09-19T04:11:43.989 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-19T04:11:44.045 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-19T04:11:44.191 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package bison. 2024-09-19T04:11:44.295 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174060 files and directories currently installed.) 2024-09-19T04:11:44.301 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-19T04:11:44.333 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-19T04:11:44.349 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-19T04:11:44.351 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T04:11:44.382 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T04:11:44.760 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-19T04:11:44.788 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-19T04:11:44.827 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T04:11:45.147 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package dump. 2024-09-19T04:11:45.175 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-19T04:11:45.213 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking dump (0.4b47-3) ... 2024-09-19T04:11:45.525 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package indent. 2024-09-19T04:11:45.541 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:45.650 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T04:11:45.911 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-19T04:11:45.927 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-19T04:11:45.969 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T04:11:46.144 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-19T04:11:46.171 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T04:11:46.210 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:11:46.257 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-19T04:11:46.272 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-19T04:11:46.315 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T04:11:46.471 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-19T04:11:46.487 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T04:11:46.530 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:11:46.567 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-19T04:11:46.583 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-19T04:11:46.625 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T04:11:46.858 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-19T04:11:46.885 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T04:11:46.916 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:11:46.962 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-19T04:11:46.990 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-19T04:11:47.029 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T04:11:47.244 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-19T04:11:47.271 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-19T04:11:47.281 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-19T04:11:47.297 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-19T04:11:47.310 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T04:11:47.331 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T04:11:47.551 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libjson-perl. 2024-09-19T04:11:47.578 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-19T04:11:47.617 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-19T04:11:47.630 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-19T04:11:47.658 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T04:11:47.697 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T04:11:47.945 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-19T04:11:47.973 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-19T04:11:48.012 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T04:11:48.050 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-19T04:11:48.078 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:48.116 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:48.428 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-19T04:11:48.455 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:48.494 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:11:48.797 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-19T04:11:48.813 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:48.855 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:11:49.108 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-19T04:11:49.135 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T04:11:49.174 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T04:11:49.695 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-19T04:11:49.723 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-19T04:11:49.748 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-19T04:11:49.761 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T04:11:49.776 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T04:11:49.815 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:11:50.092 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-19T04:11:50.106 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package modemmanager. 2024-09-19T04:11:50.108 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-19T04:11:50.134 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T04:11:50.150 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:11:50.173 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:11:50.428 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-19T04:11:50.456 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T04:11:50.495 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:11:50.805 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package network-manager. 2024-09-19T04:11:50.831 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-19T04:11:50.832 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T04:11:50.858 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-19T04:11:50.880 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-19T04:11:50.897 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T04:11:51.209 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-19T04:11:51.237 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T04:11:51.275 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T04:11:51.471 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package ppp. 2024-09-19T04:11:51.498 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-19T04:11:51.578 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-19T04:11:51.606 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:51.636 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:51.769 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T04:11:51.973 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-19T04:11:52.000 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:52.039 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:11:52.076 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package pptp-linux. 2024-09-19T04:11:52.104 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-19T04:11:52.142 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-19T04:11:52.350 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-19T04:11:52.361 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-19T04:11:52.366 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:52.377 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-19T04:11:52.408 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:11:52.420 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-19T04:11:52.644 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-19T04:11:52.660 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-19T04:11:52.694 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T04:11:52.723 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-19T04:11:52.750 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-19T04:11:52.789 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking postgresql-client-common (238) ... 2024-09-19T04:11:53.060 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-19T04:11:53.076 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:53.110 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:53.189 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-19T04:11:53.217 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-19T04:11:53.256 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T04:11:53.617 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package modemmanager. 2024-09-19T04:11:53.645 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-19T04:11:53.665 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package postgresql-common. 2024-09-19T04:11:53.684 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:11:53.693 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-19T04:11:53.741 INFO:teuthology.orchestra.run.smithi076.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-19T04:11:53.771 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking postgresql-common (238) ... 2024-09-19T04:11:54.230 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package postgresql-14. 2024-09-19T04:11:54.258 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:54.292 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:54.307 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package network-manager. 2024-09-19T04:11:54.335 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-19T04:11:54.382 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-19T04:11:55.098 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ppp. 2024-09-19T04:11:55.125 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-19T04:11:55.400 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T04:11:55.425 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package postgresql. 2024-09-19T04:11:55.442 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-19T04:11:55.484 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking postgresql (14+238) ... 2024-09-19T04:11:55.702 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package pptp-linux. 2024-09-19T04:11:55.730 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-19T04:11:55.769 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-19T04:11:55.803 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package postgresql-client. 2024-09-19T04:11:55.831 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-19T04:11:55.877 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking postgresql-client (14+238) ... 2024-09-19T04:11:56.022 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-19T04:11:56.038 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-19T04:11:56.080 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-19T04:11:56.121 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-19T04:11:56.137 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-19T04:11:56.179 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-19T04:11:56.358 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-19T04:11:56.385 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-19T04:11:56.424 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-client-common (238) ... 2024-09-19T04:11:56.474 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package tcl8.6. 2024-09-19T04:11:56.502 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-19T04:11:56.540 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T04:11:56.711 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-19T04:11:56.739 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:56.777 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:56.810 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package tcl. 2024-09-19T04:11:56.826 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-19T04:11:56.872 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-19T04:11:57.179 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package wpasupplicant. 2024-09-19T04:11:57.207 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-19T04:11:57.254 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T04:11:57.424 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-common. 2024-09-19T04:11:57.452 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-19T04:11:57.510 INFO:teuthology.orchestra.run.smithi116.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-19T04:11:57.530 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-common (238) ... 2024-09-19T04:11:57.634 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-19T04:11:57.650 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-19T04:11:57.692 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-19T04:11:57.998 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-14. 2024-09-19T04:11:58.026 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-19T04:11:58.067 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:58.070 INFO:teuthology.orchestra.run.smithi076.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-19T04:11:58.098 INFO:teuthology.orchestra.run.smithi076.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-19T04:11:58.145 INFO:teuthology.orchestra.run.smithi076.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T04:11:58.407 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up postgresql-client-common (238) ... 2024-09-19T04:11:58.618 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-19T04:11:58.744 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T04:11:58.861 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:58.963 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T04:11:59.089 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T04:11:59.215 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:11:59.250 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql. 2024-09-19T04:11:59.277 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-19T04:11:59.309 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql (14+238) ... 2024-09-19T04:11:59.536 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-client. 2024-09-19T04:11:59.564 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-19T04:11:59.605 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-client (14+238) ... 2024-09-19T04:11:59.707 INFO:teuthology.orchestra.run.smithi076.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-19T04:11:59.787 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T04:11:59.846 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-19T04:11:59.862 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-19T04:11:59.913 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-19T04:12:00.158 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T04:12:00.158 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T04:12:00.241 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package tcl8.6. 2024-09-19T04:12:00.269 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-19T04:12:00.307 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T04:12:00.559 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package tcl. 2024-09-19T04:12:00.575 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-19T04:12:00.613 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-19T04:12:00.640 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T04:12:00.766 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T04:12:00.892 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T04:12:00.945 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package wpasupplicant. 2024-09-19T04:12:00.973 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-19T04:12:01.012 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T04:12:01.018 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-19T04:12:01.169 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T04:12:01.287 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T04:12:01.383 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-19T04:12:01.388 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T04:12:01.399 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-19T04:12:01.441 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-19T04:12:01.514 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T04:12:01.744 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-19T04:12:01.772 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-19T04:12:01.811 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T04:12:02.107 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-client-common (238) ... 2024-09-19T04:12:02.169 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T04:12:02.287 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up dump (0.4b47-3) ... 2024-09-19T04:12:02.317 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-19T04:12:02.364 INFO:teuthology.orchestra.run.smithi076.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-19T04:12:02.442 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-19T04:12:02.455 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:12:02.568 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:12:02.581 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T04:12:02.694 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-19T04:12:02.707 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-19T04:12:02.820 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-19T04:12:02.833 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-19T04:12:02.938 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:12:02.960 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T04:12:03.050 INFO:teuthology.orchestra.run.smithi076.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-19T04:12:03.136 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T04:12:03.262 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T04:12:03.427 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-19T04:12:03.438 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T04:12:03.519 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-19T04:12:03.556 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:12:03.665 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-19T04:12:03.825 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up postgresql-client (14+238) ... 2024-09-19T04:12:03.889 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T04:12:03.889 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-19T04:12:03.951 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-19T04:12:04.379 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-19T04:12:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:04.424 INFO:teuthology.orchestra.run.smithi076.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-19T04:12:04.424 INFO:teuthology.orchestra.run.smithi076.stdout:which means they are currently configured by ifupdown: 2024-09-19T04:12:04.424 INFO:teuthology.orchestra.run.smithi076.stdout:- enp3s0f1 2024-09-19T04:12:04.424 INFO:teuthology.orchestra.run.smithi076.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-19T04:12:04.424 INFO:teuthology.orchestra.run.smithi076.stdout:remove their configuration from /etc/network/interfaces. 2024-09-19T04:12:04.424 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:04.506 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-19T04:12:04.554 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-19T04:12:04.632 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-19T04:12:04.758 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-19T04:12:04.871 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-19T04:12:04.883 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-19T04:12:05.176 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-19T04:12:05.191 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-19T04:12:05.569 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-19T04:12:05.695 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-19T04:12:05.896 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:12:06.039 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T04:12:06.208 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-19T04:12:06.325 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-19T04:12:06.332 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up postgresql-common (238) ... 2024-09-19T04:12:06.451 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up dump (0.4b47-3) ... 2024-09-19T04:12:06.545 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-19T04:12:06.611 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:12:06.737 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-19T04:12:06.855 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-19T04:12:06.991 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-19T04:12:07.117 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-19T04:12:07.207 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-19T04:12:07.256 INFO:teuthology.orchestra.run.smithi076.stdout:Adding user postgres to group ssl-cert 2024-09-19T04:12:07.268 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-19T04:12:07.394 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-19T04:12:07.475 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:07.475 INFO:teuthology.orchestra.run.smithi076.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-19T04:12:07.571 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-19T04:12:07.688 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:12:07.689 INFO:teuthology.orchestra.run.smithi076.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-19T04:12:07.689 INFO:teuthology.orchestra.run.smithi076.stdout:Removing obsolete dictionary files: 2024-09-19T04:12:07.797 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-19T04:12:07.966 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-client (14+238) ... 2024-09-19T04:12:08.091 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-19T04:12:08.225 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-19T04:12:08.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:08.574 INFO:teuthology.orchestra.run.smithi116.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-19T04:12:08.574 INFO:teuthology.orchestra.run.smithi116.stdout:which means they are currently configured by ifupdown: 2024-09-19T04:12:08.574 INFO:teuthology.orchestra.run.smithi116.stdout:- enp3s0f1 2024-09-19T04:12:08.574 INFO:teuthology.orchestra.run.smithi116.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-19T04:12:08.574 INFO:teuthology.orchestra.run.smithi116.stdout:remove their configuration from /etc/network/interfaces. 2024-09-19T04:12:08.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:08.609 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:12:08.688 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-19T04:12:08.931 INFO:teuthology.orchestra.run.smithi076.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-19T04:12:08.950 INFO:teuthology.orchestra.run.smithi076.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout:This user must also own the server process. 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout:The database cluster will be initialized with locale "en_US". 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout:The default text search configuration will be set to "english". 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:08.990 INFO:teuthology.orchestra.run.smithi076.stdout:Data page checksums are disabled. 2024-09-19T04:12:08.991 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:08.991 INFO:teuthology.orchestra.run.smithi076.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-19T04:12:08.991 INFO:teuthology.orchestra.run.smithi076.stdout:creating subdirectories ... ok 2024-09-19T04:12:08.991 INFO:teuthology.orchestra.run.smithi076.stdout:selecting dynamic shared memory implementation ... posix 2024-09-19T04:12:09.003 INFO:teuthology.orchestra.run.smithi076.stdout:selecting default max_connections ... 100 2024-09-19T04:12:09.015 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-19T04:12:09.025 INFO:teuthology.orchestra.run.smithi076.stdout:selecting default shared_buffers ... 128MB 2024-09-19T04:12:09.035 INFO:teuthology.orchestra.run.smithi076.stdout:selecting default time zone ... Etc/UTC 2024-09-19T04:12:09.036 INFO:teuthology.orchestra.run.smithi076.stdout:creating configuration files ... ok 2024-09-19T04:12:09.150 INFO:teuthology.orchestra.run.smithi076.stdout:running bootstrap script ... ok 2024-09-19T04:12:09.324 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-19T04:12:09.785 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-19T04:12:09.837 INFO:teuthology.orchestra.run.smithi076.stdout:performing post-bootstrap initialization ... ok 2024-09-19T04:12:09.912 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-19T04:12:10.021 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-19T04:12:10.130 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-common (238) ... 2024-09-19T04:12:10.479 INFO:teuthology.orchestra.run.smithi076.stdout:syncing data to disk ... ok 2024-09-19T04:12:10.876 INFO:teuthology.orchestra.run.smithi076.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-19T04:12:11.358 INFO:teuthology.orchestra.run.smithi116.stdout:Adding user postgres to group ssl-cert 2024-09-19T04:12:11.575 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:11.575 INFO:teuthology.orchestra.run.smithi116.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-19T04:12:11.785 INFO:teuthology.orchestra.run.smithi116.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-19T04:12:11.785 INFO:teuthology.orchestra.run.smithi116.stdout:Removing obsolete dictionary files: 2024-09-19T04:12:12.353 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-19T04:12:12.740 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-19T04:12:13.080 INFO:teuthology.orchestra.run.smithi116.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-19T04:12:13.097 INFO:teuthology.orchestra.run.smithi116.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-19T04:12:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-19T04:12:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:This user must also own the server process. 2024-09-19T04:12:13.149 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:The database cluster will be initialized with locale "en_US". 2024-09-19T04:12:13.149 INFO:teuthology.orchestra.run.smithi116.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-19T04:12:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:The default text search configuration will be set to "english". 2024-09-19T04:12:13.150 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:Data page checksums are disabled. 2024-09-19T04:12:13.150 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-19T04:12:13.150 INFO:teuthology.orchestra.run.smithi116.stdout:creating subdirectories ... ok 2024-09-19T04:12:13.151 INFO:teuthology.orchestra.run.smithi116.stdout:selecting dynamic shared memory implementation ... posix 2024-09-19T04:12:13.166 INFO:teuthology.orchestra.run.smithi116.stdout:selecting default max_connections ... 100 2024-09-19T04:12:13.188 INFO:teuthology.orchestra.run.smithi116.stdout:selecting default shared_buffers ... 128MB 2024-09-19T04:12:13.197 INFO:teuthology.orchestra.run.smithi116.stdout:selecting default time zone ... Etc/UTC 2024-09-19T04:12:13.198 INFO:teuthology.orchestra.run.smithi116.stdout:creating configuration files ... ok 2024-09-19T04:12:13.313 INFO:teuthology.orchestra.run.smithi116.stdout:running bootstrap script ... ok 2024-09-19T04:12:13.640 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:12:13.775 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-19T04:12:13.901 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up postgresql (14+238) ... 2024-09-19T04:12:13.959 INFO:teuthology.orchestra.run.smithi116.stdout:performing post-bootstrap initialization ... ok 2024-09-19T04:12:14.018 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:12:14.128 INFO:teuthology.orchestra.run.smithi076.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:12:14.374 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T04:12:14.374 INFO:teuthology.orchestra.run.smithi076.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T04:12:14.643 INFO:teuthology.orchestra.run.smithi116.stdout:syncing data to disk ... ok 2024-09-19T04:12:14.905 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T04:12:14.997 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-19T04:12:15.040 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-19T04:12:15.325 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T04:12:15.477 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T04:12:16.813 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-19T04:12:16.921 INFO:teuthology.orchestra.run.smithi076.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T04:12:17.420 INFO:teuthology.orchestra.run.smithi076.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T04:12:17.425 DEBUG:teuthology.parallel:result is None 2024-09-19T04:12:17.796 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:12:17.897 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-19T04:12:18.014 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up postgresql (14+238) ... 2024-09-19T04:12:18.141 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-19T04:12:18.266 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-19T04:12:18.528 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T04:12:18.528 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-19T04:12:19.085 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-19T04:12:19.178 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-19T04:12:19.480 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-19T04:12:19.624 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-19T04:12:20.993 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-19T04:12:21.093 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-19T04:12:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-19T04:12:21.593 DEBUG:teuthology.parallel:result is None 2024-09-19T04:12:21.594 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:12:21.743 DEBUG:teuthology.orchestra.run.smithi076:> dpkg-query -W -f '${Version}' ceph 2024-09-19T04:12:21.763 INFO:teuthology.orchestra.run.smithi076.stdout:19.1.1-361-g2f599a46-1jammy 2024-09-19T04:12:21.763 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361-g2f599a46-1jammy 2024-09-19T04:12:21.763 INFO:teuthology.task.install:The correct ceph version 19.1.1-361-g2f599a46-1jammy is installed. 2024-09-19T04:12:21.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T04:12:21.906 DEBUG:teuthology.orchestra.run.smithi116:> dpkg-query -W -f '${Version}' ceph 2024-09-19T04:12:21.925 INFO:teuthology.orchestra.run.smithi116.stdout:19.1.1-361-g2f599a46-1jammy 2024-09-19T04:12:21.925 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361-g2f599a46-1jammy 2024-09-19T04:12:21.925 INFO:teuthology.task.install:The correct ceph version 19.1.1-361-g2f599a46-1jammy is installed. 2024-09-19T04:12:21.927 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T04:12:21.928 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:21.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:12:21.942 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:21.942 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T04:12:21.981 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T04:12:21.982 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:21.982 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:12:21.995 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:12:22.050 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:22.050 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T04:12:22.062 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T04:12:22.117 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T04:12:22.117 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:22.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:12:22.130 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:12:22.187 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:22.187 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T04:12:22.200 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T04:12:22.257 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T04:12:22.258 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:22.258 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:12:22.271 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:12:22.327 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:22.327 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T04:12:22.341 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T04:12:22.393 INFO:teuthology.run_tasks:Running task ceph... 2024-09-19T04:12:22.486 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-19T04:12:22.486 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 777 /var/log/ceph 2024-09-19T04:12:22.488 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 777 /var/log/ceph 2024-09-19T04:12:22.501 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-19T04:12:22.501 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T04:12:22.545 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-19T04:12:22.556 INFO:tasks.ceph:Creating extra log directories... 2024-09-19T04:12:22.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T04:12:22.597 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-19T04:12:22.617 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-19T04:12:22.617 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-19T04:12:22.617 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-19T04:12:22.617 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:22.617 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T04:12:22.655 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T04:12:22.733 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:22.733 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-19T04:12:22.739 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-19T04:12:22.819 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-19T04:12:22.819 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-19T04:12:22.819 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626', 'branch': 'squid', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/multimds_misc}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7911626', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1747, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'sleep_before_teardown': 0, 'subset': '176/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '6fad5dd73c93545e52f57f681e140a9c3529c11e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi076.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKk33/Lk6l7X+CyxNGn5zPTs4rxjY8EJw00qPjnFzaWtq9UlZd099lLHp3ZdN5E69ZNNXDcLPocTWXXNgL/4q+g=', 'smithi116.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBFskfojGWT2x250/5ry1dm5px2lpFdoHSAHjwVysFHWhKrClCFtsI2KwUGarGL61U3+1MLZBOaqIH4zTrJ0bYY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'Scrub error on inode'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_multimds_misc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_21:24:17', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-19T04:12:22.820 DEBUG:teuthology.orchestra.run.smithi076:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T04:12:22.822 DEBUG:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-19T04:12:22.862 DEBUG:teuthology.orchestra.run.smithi076:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T04:12:22.868 DEBUG:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/run/ceph 2024-09-19T04:12:22.918 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:22.918 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:12:22.927 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:12:22.927 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_1 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 889 Links: 1 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-19 04:06:08.587411736 +0000 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-19 04:06:08.323414090 +0000 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-19 04:06:08.323414090 +0000 2024-09-19T04:12:22.976 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2024-09-19T04:12:22.977 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:12:23.030 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-19T04:12:23.030 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-19T04:12:23.030 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000422142 s, 1.2 MB/s 2024-09-19T04:12:23.031 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:12:23.080 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_2 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-19 04:06:09.071407421 +0000 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-19 04:06:08.819409668 +0000 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-19 04:06:08.819409668 +0000 2024-09-19T04:12:23.128 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2024-09-19T04:12:23.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:12:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-19T04:12:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-19T04:12:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000330731 s, 1.5 MB/s 2024-09-19T04:12:23.181 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:12:23.229 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_3 2024-09-19T04:12:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:12:23.275 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-09-19T04:12:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-19 04:06:09.551403142 +0000 2024-09-19T04:12:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-19 04:06:09.299405388 +0000 2024-09-19T04:12:23.276 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-19 04:06:09.299405388 +0000 2024-09-19T04:12:23.276 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2024-09-19T04:12:23.276 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:12:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-19T04:12:23.330 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-19T04:12:23.330 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000399212 s, 1.3 MB/s 2024-09-19T04:12:23.331 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:12:23.380 DEBUG:teuthology.orchestra.run.smithi076:> stat /dev/vg_nvme/lv_4 2024-09-19T04:12:23.427 INFO:teuthology.orchestra.run.smithi076.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:12:23.427 INFO:teuthology.orchestra.run.smithi076.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-09-19T04:12:23.427 INFO:teuthology.orchestra.run.smithi076.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:Access: 2024-09-19 04:06:10.063398577 +0000 2024-09-19T04:12:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:Modify: 2024-09-19 04:06:09.803400895 +0000 2024-09-19T04:12:23.428 INFO:teuthology.orchestra.run.smithi076.stdout:Change: 2024-09-19 04:06:09.803400895 +0000 2024-09-19T04:12:23.428 INFO:teuthology.orchestra.run.smithi076.stdout: Birth: - 2024-09-19T04:12:23.428 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:12:23.481 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records in 2024-09-19T04:12:23.481 INFO:teuthology.orchestra.run.smithi076.stderr:1+0 records out 2024-09-19T04:12:23.482 INFO:teuthology.orchestra.run.smithi076.stderr:512 bytes copied, 0.000390947 s, 1.3 MB/s 2024-09-19T04:12:23.483 DEBUG:teuthology.orchestra.run.smithi076:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:12:23.532 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:23.533 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:23.533 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T04:12:23.539 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T04:12:23.539 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 2024-09-19T04:12:23.586 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T04:12:23.587 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:23.587 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 888 Links: 1 2024-09-19T04:12:23.587 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:23.587 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-19 04:06:08.818919169 +0000 2024-09-19T04:12:23.587 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-19 04:06:08.554928919 +0000 2024-09-19T04:12:23.587 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-19 04:06:08.554928919 +0000 2024-09-19T04:12:23.588 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2024-09-19T04:12:23.588 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T04:12:23.641 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-19T04:12:23.641 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-19T04:12:23.641 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000445963 s, 1.1 MB/s 2024-09-19T04:12:23.642 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T04:12:23.693 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 2024-09-19T04:12:23.746 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T04:12:23.746 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:23.746 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 901 Links: 1 2024-09-19T04:12:23.747 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:23.747 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-19 04:06:09.306901446 +0000 2024-09-19T04:12:23.747 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-19 04:06:09.038911076 +0000 2024-09-19T04:12:23.747 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-19 04:06:09.038911076 +0000 2024-09-19T04:12:23.747 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2024-09-19T04:12:23.747 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T04:12:23.800 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-19T04:12:23.800 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-19T04:12:23.800 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000371676 s, 1.4 MB/s 2024-09-19T04:12:23.802 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T04:12:23.851 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 2024-09-19T04:12:23.906 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T04:12:23.906 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:23.907 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-09-19T04:12:23.907 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:23.907 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-19 04:06:09.802883623 +0000 2024-09-19T04:12:23.907 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-19 04:06:09.566892103 +0000 2024-09-19T04:12:23.907 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-19 04:06:09.566892103 +0000 2024-09-19T04:12:23.907 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2024-09-19T04:12:23.907 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T04:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-19T04:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-19T04:12:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000400738 s, 1.3 MB/s 2024-09-19T04:12:23.961 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T04:12:24.011 DEBUG:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 938 Links: 1 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2024-09-19 04:06:10.286837904 +0000 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2024-09-19 04:06:10.038871863 +0000 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2024-09-19 04:06:10.038871863 +0000 2024-09-19T04:12:24.058 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2024-09-19T04:12:24.058 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T04:12:24.120 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2024-09-19T04:12:24.120 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2024-09-19T04:12:24.120 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000315846 s, 1.6 MB/s 2024-09-19T04:12:24.121 DEBUG:teuthology.orchestra.run.smithi116:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T04:12:24.171 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:24.171 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T04:12:24.171 INFO:tasks.ceph:Generating config... 2024-09-19T04:12:24.173 INFO:tasks.ceph:[client] client mount timeout = 600 2024-09-19T04:12:24.173 INFO:tasks.ceph:[client] debug client = 20 2024-09-19T04:12:24.173 INFO:tasks.ceph:[client] debug ms = 1 2024-09-19T04:12:24.173 INFO:tasks.ceph:[client] pid file = 2024-09-19T04:12:24.173 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-09-19T04:12:24.173 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] debug mds = 20 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] debug ms = 1 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] mds debug frag = True 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-19T04:12:24.174 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-19T04:12:24.175 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-19T04:12:24.175 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-19T04:12:24.175 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-19T04:12:24.175 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-19T04:12:24.175 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-19T04:12:24.175 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-09-19T04:12:24.175 INFO:tasks.ceph:[mon] mon warn on pool no app = False 2024-09-19T04:12:24.175 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-19T04:12:24.175 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-19T04:12:24.175 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-19T04:12:24.175 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-19T04:12:24.175 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-19T04:12:24.175 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-19T04:12:24.176 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-19T04:12:24.176 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-19T04:12:24.176 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-19T04:12:24.176 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-19T04:12:24.176 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-19T04:12:24.176 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-19T04:12:24.176 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-09-19T04:12:24.176 INFO:tasks.ceph:Setting up mon.a... 2024-09-19T04:12:24.176 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-19T04:12:24.211 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.keyring 2024-09-19T04:12:24.227 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-19T04:12:24.308 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T04:12:24.363 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.76'), ('mon.b', '172.21.15.116'), ('mon.c', '[v2:172.21.15.116:3301,v1:172.21.15.116:6790]')] 2024-09-19T04:12:24.363 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.76,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-09-19T04:12:24.364 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:24.364 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T04:12:24.412 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.76 --add b 172.21.15.116 --addv c '[v2:172.21.15.116:3301,v1:172.21.15.116:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:24.498 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: generated fsid 2bcf595d-5a10-4e84-8b11-2533969843a0 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:setting min_mon_release = quincy 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:epoch 0 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:fsid 2bcf595d-5a10-4e84-8b11-2533969843a0 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:last_changed 2024-09-19T04:12:24.501805+0000 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:created 2024-09-19T04:12:24.501805+0000 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:min_mon_release 17 (quincy) 2024-09-19T04:12:24.499 INFO:teuthology.orchestra.run.smithi076.stdout:election_strategy: 1 2024-09-19T04:12:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:0: [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] mon.a 2024-09-19T04:12:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:1: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.b 2024-09-19T04:12:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:2: [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] mon.c 2024-09-19T04:12:24.500 INFO:teuthology.orchestra.run.smithi076.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-19T04:12:24.501 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-19T04:12:24.548 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2bcf595d-5a10-4e84-8b11-2533969843a0... 2024-09-19T04:12:24.549 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T04:12:24.592 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-19T04:12:24.621 INFO:teuthology.orchestra.run.smithi076.stdout:[global] 2024-09-19T04:12:24.621 INFO:teuthology.orchestra.run.smithi076.stdout: chdir = "" 2024-09-19T04:12:24.621 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T04:12:24.621 INFO:teuthology.orchestra.run.smithi076.stdout: auth supported = cephx 2024-09-19T04:12:24.621 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: filestore xattr use omap = true 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: mon clock drift allowed = 1.000 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: osd crush chooseleaf type = 0 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: auth debug = true 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on old message = true 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: ms die on bug = true 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: mon pg warn max object skew = 0 2024-09-19T04:12:24.622 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default size = 2 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow primary affinity = true 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd allow pg remap = true 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on legacy crush tunables = false 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on crush straw calc version zero = false 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on no sortbitwise = false 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on osd down out interval zero = false 2024-09-19T04:12:24.623 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on too few osds = false 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: mon_allow_pool_size_one = true 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: osd default data pool replay window = 5 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: mon allow pool delete = true 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: mon cluster log file level = debug 2024-09-19T04:12:24.624 INFO:teuthology.orchestra.run.smithi076.stdout: debug asserts on shutdown = true 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: mon health detail to clog = false 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: mon host = "172.21.15.76,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]" 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: fsid = 2bcf595d-5a10-4e84-8b11-2533969843a0 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout:[osd] 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: osd journal size = 100 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub load threshold = 5.0 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: osd scrub max interval = 600 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: osd mclock profile = high_recovery_ops 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.625 INFO:teuthology.orchestra.run.smithi076.stdout: osd recover clone overlap = true 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd recovery max chunk = 1048576 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug shutdown = true 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug op order = true 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug verify stray on activate = true 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug trim objects = true 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd open classes on start = true 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug pg log writeout = true 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd deep scrub update digest min age = 30 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.626 INFO:teuthology.orchestra.run.smithi076.stdout: osd map max advance = 10 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: journal zero on create = true 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: filestore ondisk finisher threads = 3 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: filestore apply finisher threads = 3 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: bdev debug aio = true 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: osd debug misdirected ops = true 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore block size = 96636764160 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: bluestore fsck on mount = True 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluefs = 20 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: debug bluestore = 20 2024-09-19T04:12:24.627 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: debug osd = 20 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: debug rocksdb = 10 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd full ratio = 0.9 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd nearfull ratio = 0.8 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: osd failsafe full ratio = 0.95 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: osd objectstore = bluestore 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr] 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: debug mgr = 20 2024-09-19T04:12:24.628 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: mgr/telemetry/nag = false 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout:[mon] 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: debug mon = 20 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: debug paxos = 20 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: debug auth = 20 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: mon data avail warn = 5 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: mon mgr mkfs grace = 240 2024-09-19T04:12:24.629 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min pgs per osd = 4 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd reporter subtree level = osd 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: mon osd prime pg temp = true 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: mon reweight min bytes per osd = 10 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.630 INFO:teuthology.orchestra.run.smithi076.stdout: # 1m isn't quite enough 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: mon_down_mkfs_grace = 2m 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: mon_warn_on_filestore_osds = false 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: mon down mkfs grace = 300 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: mon op complaint time = 120 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: mon warn on pool no app = False 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout:[client] 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: rgw cache enabled = true 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable ops log = true 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: rgw enable usage log = true 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T04:12:24.631 INFO:teuthology.orchestra.run.smithi076.stdout: client mount timeout = 600 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout: debug client = 20 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout: pid file = "" 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.a] 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.b] 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:[mon.c] 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-09-19T04:12:24.632 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout:[mds] 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds = 20 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: debug mds balancer = 20 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: debug ms = 1 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug frag = True 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: mds debug scatterstat = True 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: mds op complaint time = 180 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: mds verify scatter = True 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: osd op complaint time = 180 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: rados mon op timeout = 900 2024-09-19T04:12:24.633 INFO:teuthology.orchestra.run.smithi076.stdout: rados osd op timeout = 900 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2024-09-19T04:12:24.634 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.635 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2024-09-19T04:12:24.636 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.76,172.21.15.116,[v2:172.21.15.116:3301,v1:172.21.15.116:6790]" 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 2bcf595d-5a10-4e84-8b11-2533969843a0 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.637 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd mclock profile = high_recovery_ops 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2024-09-19T04:12:24.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug trim objects = true 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2024-09-19T04:12:24.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 20 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2024-09-19T04:12:24.640 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2024-09-19T04:12:24.641 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2024-09-19T04:12:24.641 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-09-19T04:12:24.641 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.641 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2024-09-19T04:12:24.641 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-09-19T04:12:24.641 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2024-09-19T04:12:24.641 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2024-09-19T04:12:24.644 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: auth mon ticket ttl = 660 # 11m 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: auth service ticket ttl = 240 # 4m 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: # don't complain about insecure global_id in the test suite 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-19T04:12:24.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: # 1m isn't quite enough 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: mon_down_mkfs_grace = 2m 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_filestore_osds = false 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: mon down mkfs grace = 300 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on pool no app = False 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2024-09-19T04:12:24.646 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-09-19T04:12:24.647 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds balancer = 20 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 900 2024-09-19T04:12:24.648 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 900 2024-09-19T04:12:24.650 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-19T04:12:24.650 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-19T04:12:24.739 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-19T04:12:24.739 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:24.739 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-19T04:12:24.744 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:24.745 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-19T04:12:24.791 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:12:24.792 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:24.792 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T04:12:24.792 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T04:12:24.855 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:24.855 DEBUG:teuthology.orchestra.run.smithi076:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:24.903 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:12:24.903 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:24.903 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-19T04:12:24.903 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-19T04:12:24.926 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:24.926 DEBUG:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:24.974 INFO:tasks.ceph:Setting up mon nodes... 2024-09-19T04:12:24.974 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-19T04:12:24.974 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T04:12:25.019 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-19T04:12:25.022 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T04:12:25.064 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-19T04:12:25.078 INFO:tasks.ceph:Setting up mds nodes... 2024-09-19T04:12:25.078 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T04:12:25.118 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-19T04:12:25.121 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-19T04:12:25.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T04:12:25.259 INFO:teuthology.orchestra.run.smithi076.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-19T04:12:25.262 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-19T04:12:25.315 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T04:12:25.359 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-19T04:12:25.362 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-19T04:12:25.416 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T04:12:25.495 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-19T04:12:25.497 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-19T04:12:25.553 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-19T04:12:25.554 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T04:12:25.589 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-19T04:12:25.601 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-19T04:12:25.674 INFO:teuthology.orchestra.run.smithi076.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-19T04:12:25.684 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-19T04:12:25.720 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-19T04:12:25.733 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-19T04:12:25.803 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-19T04:12:25.813 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-19T04:12:25.813 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi076.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-19T04:12:25.813 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-19T04:12:25.827 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:25.827 INFO:tasks.ceph:role: osd.0 2024-09-19T04:12:25.828 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:12:25.828 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T04:12:25.891 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:25.891 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:25.891 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:25.892 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:25.892 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:25.892 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:25.892 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:26.369 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-19T04:12:26.370 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-19T04:12:26.371 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-19T04:12:26.513 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-19T04:12:26.524 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:26.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:26.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-19T04:12:26.579 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:26.579 INFO:tasks.ceph:role: osd.1 2024-09-19T04:12:26.579 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:12:26.579 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:26.643 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:27.092 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-19T04:12:27.094 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-19T04:12:27.094 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-19T04:12:27.180 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-19T04:12:27.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:27.190 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:27.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-19T04:12:27.242 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:27.242 INFO:tasks.ceph:role: osd.2 2024-09-19T04:12:27.242 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:12:27.243 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T04:12:27.302 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:27.303 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:27.303 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:27.304 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:27.304 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:27.785 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-19T04:12:27.787 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-19T04:12:27.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-19T04:12:27.880 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-19T04:12:27.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:27.892 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-19T04:12:27.947 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:27.947 INFO:tasks.ceph:role: osd.3 2024-09-19T04:12:27.947 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:12:27.947 DEBUG:teuthology.orchestra.run.smithi076:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:28.011 INFO:teuthology.orchestra.run.smithi076.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:28.012 INFO:teuthology.orchestra.run.smithi076.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:28.012 INFO:teuthology.orchestra.run.smithi076.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:28.521 INFO:teuthology.orchestra.run.smithi076.stdout:Discarding blocks...Done. 2024-09-19T04:12:28.522 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi076.front.sepia.ceph.com -o noatime 2024-09-19T04:12:28.523 DEBUG:teuthology.orchestra.run.smithi076:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-19T04:12:28.663 DEBUG:teuthology.orchestra.run.smithi076:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-19T04:12:28.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:28.676 INFO:teuthology.orchestra.run.smithi076.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:28.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:28.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:28.786+0000 7fc603e5e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-19T04:12:28.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:28.790+0000 7fc603e5e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-19T04:12:28.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:28.790+0000 7fc603e5e7c0 -1 bdev(0x555b45f13800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:28.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:28.790+0000 7fc603e5e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-19T04:12:32.175 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-19T04:12:32.188 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:32.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:32.266+0000 7f956a6a07c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-19T04:12:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:32.266+0000 7f956a6a07c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-19T04:12:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:32.266+0000 7f956a6a07c0 -1 bdev(0x55ced0f2f800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:32.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:32.266+0000 7f956a6a07c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-19T04:12:35.407 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-19T04:12:35.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:35.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:35.542+0000 7f02112dc7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-19T04:12:35.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:35.542+0000 7f02112dc7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-19T04:12:35.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:35.542+0000 7f02112dc7c0 -1 bdev(0x564075703800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:35.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:35.542+0000 7f02112dc7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-19T04:12:38.674 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-19T04:12:38.732 DEBUG:teuthology.orchestra.run.smithi076:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:38.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:38.810+0000 7f82c54657c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-19T04:12:38.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:38.818+0000 7f82c54657c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-19T04:12:38.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:38.818+0000 7f82c54657c0 -1 bdev(0x562f48bcb800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:38.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:12:38.818+0000 7f82c54657c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-19T04:12:41.954 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-19T04:12:42.008 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-19T04:12:42.022 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:42.022 INFO:tasks.ceph:role: osd.4 2024-09-19T04:12:42.022 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:12:42.023 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:42.086 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:42.087 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:42.537 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-09-19T04:12:42.540 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-09-19T04:12:42.541 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-19T04:12:42.638 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-19T04:12:42.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:42.648 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:42.648 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-19T04:12:42.699 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:42.699 INFO:tasks.ceph:role: osd.5 2024-09-19T04:12:42.699 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:12:42.699 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-19T04:12:42.766 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:42.766 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:42.766 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:42.767 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:42.767 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:42.767 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:42.767 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:42.767 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:42.767 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:42.767 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:43.232 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-09-19T04:12:43.234 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-09-19T04:12:43.235 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-19T04:12:43.386 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-19T04:12:43.397 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:43.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:43.398 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-19T04:12:43.454 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:43.454 INFO:tasks.ceph:role: osd.6 2024-09-19T04:12:43.454 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:12:43.454 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-19T04:12:43.517 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:43.517 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:43.518 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:43.995 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-09-19T04:12:43.997 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-09-19T04:12:43.997 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-19T04:12:44.138 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-19T04:12:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:44.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:44.149 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-19T04:12:44.205 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-19T04:12:44.205 INFO:tasks.ceph:role: osd.7 2024-09-19T04:12:44.205 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:12:44.205 DEBUG:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-19T04:12:44.269 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-19T04:12:44.690 INFO:teuthology.orchestra.run.smithi116.stdout:Discarding blocks...Done. 2024-09-19T04:12:44.692 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2024-09-19T04:12:44.692 DEBUG:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-19T04:12:44.777 DEBUG:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-19T04:12:44.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:12:44.835 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2024-09-19T04:12:44.835 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:44.948 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:44.944+0000 7efc9a8c37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-19T04:12:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:44.948+0000 7efc9a8c37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-19T04:12:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:44.948+0000 7efc9a8c37c0 -1 bdev(0x558bc9265800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:44.948+0000 7efc9a8c37c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-19T04:12:48.058 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-19T04:12:48.072 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:48.150 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:48.148+0000 7f7d4780e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-19T04:12:48.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:48.148+0000 7f7d4780e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-19T04:12:48.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:48.148+0000 7f7d4780e7c0 -1 bdev(0x561f9fbe3800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:48.151 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:48.148+0000 7f7d4780e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-19T04:12:51.285 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-19T04:12:51.342 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:51.424 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:51.420+0000 7fa3704e47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-19T04:12:51.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:51.420+0000 7fa3704e47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-19T04:12:51.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:51.420+0000 7fa3704e47c0 -1 bdev(0x5647fc255800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:51.425 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:51.420+0000 7fa3704e47c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-19T04:12:52.821 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:12:52.823 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:12:54.548 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-19T04:12:54.561 DEBUG:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:12:54.641 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:54.636+0000 7f31d84867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-19T04:12:54.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:54.640+0000 7f31d84867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-19T04:12:54.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:54.640+0000 7f31d84867c0 -1 bdev(0x55ef68d27800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-19T04:12:54.642 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:12:54.640+0000 7f31d84867c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-19T04:12:57.776 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-19T04:12:57.830 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-19T04:12:57.830 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:57.831 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-19T04:12:57.844 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:57.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-19T04:12:57.898 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:57.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-19T04:12:57.955 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:57.955 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-19T04:12:58.010 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:58.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-19T04:12:58.065 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:58.066 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-19T04:12:58.117 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:58.117 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-19T04:12:58.170 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.171 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-19T04:12:58.185 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.185 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-19T04:12:58.242 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.242 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-19T04:12:58.297 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.297 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-19T04:12:58.353 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.353 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-19T04:12:58.409 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.409 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-19T04:12:58.465 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.465 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-19T04:12:58.522 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:58.522 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-19T04:12:58.527 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:12:58.527 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-19T04:12:58.575 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.575 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-19T04:12:58.580 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:12:58.581 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-19T04:12:58.630 INFO:tasks.ceph:Adding keys to all mons... 2024-09-19T04:12:58.630 DEBUG:teuthology.orchestra.run.smithi076:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T04:12:58.633 DEBUG:teuthology.orchestra.run.smithi116:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-19T04:12:58.675 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.y] 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg== 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.a] 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w== 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.c] 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw== 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.0] 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ== 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.1] 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w== 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.2] 2024-09-19T04:12:58.676 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ== 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.3] 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw== 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout:[mgr.x] 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A== 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.b] 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA== 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout:[mds.d] 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA== 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.4] 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA== 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.5] 2024-09-19T04:12:58.677 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw== 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.6] 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg== 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout:[osd.7] 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA== 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout:[client.2] 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ== 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout:[client.1] 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg== 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout:[client.3] 2024-09-19T04:12:58.678 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA== 2024-09-19T04:12:58.684 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.y] 2024-09-19T04:12:58.684 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg== 2024-09-19T04:12:58.684 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2024-09-19T04:12:58.684 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w== 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw== 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.0] 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ== 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.1] 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w== 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.2] 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ== 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.3] 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw== 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.x] 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A== 2024-09-19T04:12:58.685 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA== 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA== 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.4] 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA== 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.5] 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw== 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.6] 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg== 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.7] 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA== 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2024-09-19T04:12:58.686 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:12:58.687 INFO:teuthology.orchestra.run.smithi116.stdout:[client.2] 2024-09-19T04:12:58.687 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ== 2024-09-19T04:12:58.687 INFO:teuthology.orchestra.run.smithi116.stdout:[client.1] 2024-09-19T04:12:58.687 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg== 2024-09-19T04:12:58.687 INFO:teuthology.orchestra.run.smithi116.stdout:[client.3] 2024-09-19T04:12:58.687 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA== 2024-09-19T04:12:58.688 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T04:12:58.724 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T04:12:58.776 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:58.811 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:58.861 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:58.863 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:58.949 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:58.951 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.040 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.079 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.124 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.212 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.257 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T04:12:59.296 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-19T04:12:59.341 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:59.380 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:59.433 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:59.464 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-19T04:12:59.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.548 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.588 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.628 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.670 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.708 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.753 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.791 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-19T04:12:59.835 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:12:59.872 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:12:59.920 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:12:59.958 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:13:00.007 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:13:00.048 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:13:00.099 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:13:00.101 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-19T04:13:00.141 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-19T04:13:00.141 DEBUG:teuthology.orchestra.run.smithi076:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-19T04:13:00.197 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T04:13:00.285 DEBUG:teuthology.orchestra.run.smithi076:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-19T04:13:00.295 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-19T04:13:00.309 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T04:13:00.404 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-19T04:13:00.415 DEBUG:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-19T04:13:00.467 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-19T04:13:00.560 DEBUG:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-19T04:13:00.571 DEBUG:teuthology.orchestra.run.smithi076:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:13:00.573 DEBUG:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-19T04:13:00.618 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-19T04:13:00.618 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-19T04:13:00.618 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-19T04:13:00.621 INFO:tasks.ceph.mon.a:Started 2024-09-19T04:13:00.621 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-19T04:13:00.621 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-19T04:13:00.663 INFO:tasks.ceph.mon.b:Started 2024-09-19T04:13:00.663 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-19T04:13:00.663 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-19T04:13:00.665 INFO:tasks.ceph.mon.c:Started 2024-09-19T04:13:00.666 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-19T04:13:00.666 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-19T04:13:00.666 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-19T04:13:00.668 INFO:tasks.ceph.mgr.y:Started 2024-09-19T04:13:00.668 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-19T04:13:00.668 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-19T04:13:00.671 INFO:tasks.ceph.mgr.x:Started 2024-09-19T04:13:00.671 INFO:tasks.ceph:Setting crush tunables to default 2024-09-19T04:13:00.671 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph osd crush tunables default 2024-09-19T04:13:00.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.794+0000 7f6640e4b640 1 Processor -- start 2024-09-19T04:13:00.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.794+0000 7f6640e4b640 1 -- start start 2024-09-19T04:13:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6640e4b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:00.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6640e4b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6640e4b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f663c0d0d70 con 0x7f663c15c5f0 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6640e4b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f663c0d1640 con 0x7f663c160900 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6640e4b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f663c15d130 con 0x7f663c160520 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6639d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6639d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33006/0 (socket says 172.21.15.76:33006) 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6639d9b640 1 -- 172.21.15.76:0/3463186981 learned_addr learned my addr 172.21.15.76:0/3463186981 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6639d9b640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 msgr2=0x7f663c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6639d9b640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 msgr2=0x7f663c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6639d9b640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:13:00.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f6639d9b640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:13:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f663a59c640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 msgr2=0x7f663c071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-19T04:13:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f663a59c640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 msgr2=0x7f663c071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T04:13:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:13:00.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:00.798+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T04:13:00.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f6639d9b640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:00.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f6639d9b640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 msgr2=0x7f663c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-09-19T04:13:00.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f6639d9b640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 msgr2=0x7f663c071c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T04:13:00.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:00.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f6639d9b640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:13:01.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f6639d9b640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:13:01.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f663a59c640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 msgr2=0x7f663c071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-09-19T04:13:01.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f663a59c640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 msgr2=0x7f663c071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-09-19T04:13:01.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T04:13:01.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.002+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T04:13:01.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.042+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3186965657 0 0) 0x7f6634001120 con 0x7f663c15c5f0 2024-09-19T04:13:01.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.042+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6628003530 con 0x7f663c15c5f0 2024-09-19T04:13:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 <== mon.1 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3664237509 0 0) 0x7f6628003530 con 0x7f663c15c5f0 2024-09-19T04:13:01.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 msgr2=0x7f663c071750 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.116:3300/0 conn(0x7f663c160520 0x7f663c071750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 msgr2=0x7f663c071c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 --2- 172.21.15.76:0/3463186981 >> v2:172.21.15.76:3300/0 conn(0x7f663c160900 0x7f663c071c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c15d390 con 0x7f663c15c5f0 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 <== mon.1 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6634001850 con 0x7f663c15c5f0 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 >> v1:172.21.15.116:6789/0 conn(0x7f663c15c5f0 legacy=0x7f663c15ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f6628003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6628004810 0x7f6628004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6628005220 0x7f662800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f663c0d1640 con 0x7f6628003a10 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f663c15d130 con 0x7f6628005220 2024-09-19T04:13:01.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f662800c1d0 con 0x7f6628004810 2024-09-19T04:13:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f6628003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f6639d9b640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6628004810 0x7f6628004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663ad9d640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6628005220 0x7f662800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663a59c640 1 -- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6628004810 msgr2=0x7f6628004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6628004810 0x7f6628004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663a59c640 1 -- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6628005220 msgr2=0x7f662800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:01.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6628005220 0x7f662800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663ad9d640 1 -- 172.21.15.76:0/3463186981 reap_dead start 2024-09-19T04:13:01.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.050+0000 7f663a59c640 1 -- 172.21.15.76:0/3463186981 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6628003530 con 0x7f6628003a10 2024-09-19T04:13:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f663a59c640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f6628003e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f662c009a40 tx=0x7f662c01cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e633999af29ec58 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0b2070 con 0x7f6628003a10 2024-09-19T04:13:01.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662c0a5dd0 con 0x7f6628003a10 2024-09-19T04:13:01.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f663959a640 1 -- 172.21.15.76:0/3463186981 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c009640 con 0x7f6628003a10 2024-09-19T04:13:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 -- 172.21.15.76:0/3463186981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 msgr2=0x7f6628003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f6628003e30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f662c009a40 tx=0x7f662c01cfd0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 -- 172.21.15.76:0/3463186981 shutdown_connections 2024-09-19T04:13:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 --2- 172.21.15.76:0/3463186981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f6628003e30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 -- 172.21.15.76:0/3463186981 >> 172.21.15.76:0/3463186981 conn(0x7f663c06c0c0 msgr2=0x7f663c06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 -- 172.21.15.76:0/3463186981 shutdown_connections 2024-09-19T04:13:01.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 -- 172.21.15.76:0/3463186981 wait complete. 2024-09-19T04:13:01.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 Processor -- start 2024-09-19T04:13:01.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.058+0000 7f6640e4b640 1 -- start start 2024-09-19T04:13:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f663c1e7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f663c157a60 0x7f663c157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f663c155f50 0x7f663c1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f663c15cde0 con 0x7f6628003a10 2024-09-19T04:13:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f663c06ede0 con 0x7f663c155f50 2024-09-19T04:13:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f663c15cf60 con 0x7f663c157a60 2024-09-19T04:13:01.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f663c1e7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f663c1e7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33056/0 (socket says 172.21.15.76:33056) 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 -- 172.21.15.76:0/3291846660 learned_addr learned my addr 172.21.15.76:0/3291846660 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6639d9b640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f663c157a60 0x7f663c157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663ad9d640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f663c155f50 0x7f663c1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 -- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f663c157a60 msgr2=0x7f663c157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f663c157a60 0x7f663c157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 -- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f663c155f50 msgr2=0x7f663c1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f663c155f50 0x7f663c1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 -- 172.21.15.76:0/3291846660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f663c13cf60 con 0x7f6628003a10 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663ad9d640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f663c155f50 0x7f663c1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f663a59c640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f663c1e7670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f662c01ce00 tx=0x7f6628005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6632ffd640 1 -- 172.21.15.76:0/3291846660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0be020 con 0x7f6628003a10 2024-09-19T04:13:01.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6632ffd640 1 -- 172.21.15.76:0/3291846660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f662c0adaa0 con 0x7f6628003a10 2024-09-19T04:13:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f663c13d240 con 0x7f6628003a10 2024-09-19T04:13:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6632ffd640 1 -- 172.21.15.76:0/3291846660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f662c0b2040 con 0x7f6628003a10 2024-09-19T04:13:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f663c13d800 con 0x7f6628003a10 2024-09-19T04:13:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6632ffd640 1 -- 172.21.15.76:0/3291846660 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f662c0c1070 con 0x7f6628003a10 2024-09-19T04:13:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6632ffd640 1 -- 172.21.15.76:0/3291846660 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f662c0b7460 con 0x7f6628003a10 2024-09-19T04:13:01.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.062+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f663c0d1c60 con 0x7f6628003a10 2024-09-19T04:13:01.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.066+0000 7f6632ffd640 1 -- 172.21.15.76:0/3291846660 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f662c016d20 con 0x7f6628003a10 2024-09-19T04:13:01.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.130+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f663c0d29b0 con 0x7f6628003a10 2024-09-19T04:13:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6632ffd640 1 -- 172.21.15.76:0/3291846660 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f662c0c3440 con 0x7f6628003a10 2024-09-19T04:13:01.132 INFO:teuthology.orchestra.run.smithi076.stderr:adjusted tunables profile to default 2024-09-19T04:13:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 msgr2=0x7f663c1e7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f663c1e7670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f662c01ce00 tx=0x7f6628005840 comp rx=0 tx=0).stop 2024-09-19T04:13:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 shutdown_connections 2024-09-19T04:13:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f663c155f50 0x7f663c1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f663c157a60 0x7f663c157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 --2- 172.21.15.76:0/3291846660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6628003a10 0x7f663c1e7670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 >> 172.21.15.76:0/3291846660 conn(0x7f663c06c0c0 msgr2=0x7f663c0b1de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 shutdown_connections 2024-09-19T04:13:01.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:01.134+0000 7f6640e4b640 1 -- 172.21.15.76:0/3291846660 wait complete. 2024-09-19T04:13:01.150 INFO:tasks.ceph:check_enable_crimson: False 2024-09-19T04:13:01.151 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-19T04:13:01.151 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:13:01.151 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-19T04:13:01.161 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:13:01.161 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-19T04:13:01.216 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:13:01.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-19T04:13:01.231 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:01.234+0000 7fa5b8131180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:13:01.240 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:13:01.240 DEBUG:teuthology.orchestra.run.smithi076:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-19T04:13:01.295 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:13:01.295 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-19T04:13:01.310 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:13:01.310 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-19T04:13:01.365 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:13:01.365 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-19T04:13:01.420 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:13:01.420 DEBUG:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-19T04:13:01.476 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 04ec1863-8fa9-4395-beba-35ccf9684a78 0 2024-09-19T04:13:01.635 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:01.631+0000 7f89675d8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T04:13:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f953564a640 1 Processor -- start 2024-09-19T04:13:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f953564a640 1 -- start start 2024-09-19T04:13:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f953564a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f953006ccd0 0x7f953006d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f953564a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95300af4c0 0x7f95300af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f953564a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95300b3e20 con 0x7f9530153030 2024-09-19T04:13:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f953564a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f953006d680 con 0x7f95300af4c0 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f953564a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f95300b4100 con 0x7f953006ccd0 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f953006ccd0 0x7f953006d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f953006ccd0 0x7f953006d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:49540/0 (socket says 172.21.15.116:49540) 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952e7fc640 1 -- 172.21.15.116:0/1482495026 learned_addr learned my addr 172.21.15.116:0/1482495026 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952f7fe640 1 --2- 172.21.15.116:0/1482495026 >> v2:172.21.15.76:3300/0 conn(0x7f95300af4c0 0x7f95300af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952e7fc640 1 -- 172.21.15.116:0/1482495026 >> v2:172.21.15.76:3300/0 conn(0x7f95300af4c0 msgr2=0x7f95300af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952e7fc640 1 --2- 172.21.15.116:0/1482495026 >> v2:172.21.15.76:3300/0 conn(0x7f95300af4c0 0x7f95300af930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952e7fc640 1 -- 172.21.15.116:0/1482495026 >> v1:172.21.15.76:6789/0 conn(0x7f9530153030 legacy=0x7f953015d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.639+0000 7f952e7fc640 1 -- 172.21.15.116:0/1482495026 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95300b43e0 con 0x7f953006ccd0 2024-09-19T04:13:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.643+0000 7f952e7fc640 1 --2- 172.21.15.116:0/1482495026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f953006ccd0 0x7f953006d140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9524009870 tx=0x7f952401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65462d892f97f4ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.643+0000 7f952dffb640 1 -- 172.21.15.116:0/1482495026 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240ab650 con 0x7f953006ccd0 2024-09-19T04:13:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.643+0000 7f952dffb640 1 -- 172.21.15.116:0/1482495026 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95240b0070 con 0x7f953006ccd0 2024-09-19T04:13:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.643+0000 7f952dffb640 1 -- 172.21.15.116:0/1482495026 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240aba10 con 0x7f953006ccd0 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- 172.21.15.116:0/1482495026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f953006ccd0 msgr2=0x7f953006d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 --2- 172.21.15.116:0/1482495026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f953006ccd0 0x7f953006d140 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9524009870 tx=0x7f952401c560 comp rx=0 tx=0).stop 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- 172.21.15.116:0/1482495026 shutdown_connections 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 --2- 172.21.15.116:0/1482495026 >> v2:172.21.15.76:3300/0 conn(0x7f95300af4c0 0x7f95300af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 --2- 172.21.15.116:0/1482495026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f953006ccd0 0x7f953006d140 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- 172.21.15.116:0/1482495026 >> 172.21.15.116:0/1482495026 conn(0x7f9530072690 msgr2=0x7f9530072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- 172.21.15.116:0/1482495026 shutdown_connections 2024-09-19T04:13:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- 172.21.15.116:0/1482495026 wait complete. 2024-09-19T04:13:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 Processor -- start 2024-09-19T04:13:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- start start 2024-09-19T04:13:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953006ccd0 0x7f953014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95300af4c0 0x7f95301563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9530153030 0x7f95300b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95300b5bb0 con 0x7f953006ccd0 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f95300b5a30 con 0x7f9530153030 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f953564a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9530153410 con 0x7f95300af4c0 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9530153030 0x7f95300b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95300af4c0 0x7f95301563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95300af4c0 0x7f95301563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:49550/0 (socket says 172.21.15.116:49550) 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 -- 172.21.15.116:0/991290520 learned_addr learned my addr 172.21.15.116:0/991290520 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952effd640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953006ccd0 0x7f953014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 -- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9530153030 msgr2=0x7f95300b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9530153030 0x7f95300b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 -- 172.21.15.116:0/991290520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953006ccd0 msgr2=0x7f953014b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953006ccd0 0x7f953014b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 -- 172.21.15.116:0/991290520 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953014ba40 con 0x7f95300af4c0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952f7fe640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9530153030 0x7f95300b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952effd640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953006ccd0 0x7f953014b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f952e7fc640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95300af4c0 0x7f95301563d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95240ad040 tx=0x7f952401ada0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.647+0000 7f95177fe640 1 -- 172.21.15.116:0/991290520 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240be5f0 con 0x7f95300af4c0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.651+0000 7f95177fe640 1 -- 172.21.15.116:0/991290520 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95240be790 con 0x7f95300af4c0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.651+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9530155330 con 0x7f95300af4c0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.651+0000 7f95177fe640 1 -- 172.21.15.116:0/991290520 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95240beb80 con 0x7f95300af4c0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.651+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f95301558f0 con 0x7f95300af4c0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.651+0000 7f95177fe640 1 -- 172.21.15.116:0/991290520 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f95240b4780 con 0x7f95300af4c0 2024-09-19T04:13:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.651+0000 7f95177fe640 1 -- 172.21.15.116:0/991290520 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f95240c8a20 con 0x7f95300af4c0 2024-09-19T04:13:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.651+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f953006d140 con 0x7f95300af4c0 2024-09-19T04:13:01.663 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.659+0000 7f95177fe640 1 -- 172.21.15.116:0/991290520 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f95240c4060 con 0x7f95300af4c0 2024-09-19T04:13:01.741 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.735+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "04ec1863-8fa9-4395-beba-35ccf9684a78", "id": 0} v 0) -- 0x7f9530075760 con 0x7f95300af4c0 2024-09-19T04:13:01.745 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:01.746+0000 7fa5b8131180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:13:01.749 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.743+0000 7f95177fe640 1 -- 172.21.15.116:0/991290520 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "04ec1863-8fa9-4395-beba-35ccf9684a78", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f9524016150 con 0x7f95300af4c0 2024-09-19T04:13:01.749 INFO:teuthology.orchestra.run.smithi116.stdout:0 2024-09-19T04:13:01.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95300af4c0 msgr2=0x7f95301563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.750 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95300af4c0 0x7f95301563d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95240ad040 tx=0x7f952401ada0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 shutdown_connections 2024-09-19T04:13:01.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9530153030 0x7f95300b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95300af4c0 0x7f95301563d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 --2- 172.21.15.116:0/991290520 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f953006ccd0 0x7f953014b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 >> 172.21.15.116:0/991290520 conn(0x7f9530072690 msgr2=0x7f953014fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 shutdown_connections 2024-09-19T04:13:01.751 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.747+0000 7f953564a640 1 -- 172.21.15.116:0/991290520 wait complete. 2024-09-19T04:13:01.766 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new da4116da-20bf-495a-a087-724ecd3b59dd 1 2024-09-19T04:13:01.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.871+0000 7fdb9849d640 1 Processor -- start 2024-09-19T04:13:01.876 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.871+0000 7fdb9849d640 1 -- start start 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.871+0000 7fdb9849d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb9006d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.871+0000 7fdb9849d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdb9006d890 0x7fdb9009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.871+0000 7fdb9849d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb9006dd30 con 0x7fdb9015f660 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.871+0000 7fdb9849d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fdb9006e5d0 con 0x7fdb9006d890 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.871+0000 7fdb9849d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdb900b5ca0 con 0x7fdb900b3790 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdb9006d890 0x7fdb9009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdb9006d890 0x7fdb9009aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34492/0 (socket says 172.21.15.116:34492) 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 -- 172.21.15.116:0/3642236689 learned_addr learned my addr 172.21.15.116:0/3642236689 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb96239640 1 --2- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb9006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 -- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 msgr2=0x7fdb9006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:01.878 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 --2- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb9006d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 -- 172.21.15.116:0/3642236689 >> v1:172.21.15.76:6789/0 conn(0x7fdb9015f660 legacy=0x7fdb90161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:01.879 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 -- 172.21.15.116:0/3642236689 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9006e010 con 0x7fdb9006d890 2024-09-19T04:13:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95a38640 1 --2- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb9009aa00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009870 tx=0x7fdb8001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d8b9682d86cb37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95237640 1 -- 172.21.15.116:0/3642236689 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb800b1070 con 0x7fdb9006d890 2024-09-19T04:13:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95237640 1 -- 172.21.15.116:0/3642236689 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb800ac620 con 0x7fdb9006d890 2024-09-19T04:13:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb95237640 1 -- 172.21.15.116:0/3642236689 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb800ac9f0 con 0x7fdb9006d890 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 -- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 msgr2=0x7fdb9009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 --2- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb9009aa00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009870 tx=0x7fdb8001c4f0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 -- 172.21.15.116:0/3642236689 shutdown_connections 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 --2- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb9009aa00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 --2- 172.21.15.116:0/3642236689 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb9006d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 -- 172.21.15.116:0/3642236689 >> 172.21.15.116:0/3642236689 conn(0x7fdb900b78d0 msgr2=0x7fdb900b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 -- 172.21.15.116:0/3642236689 shutdown_connections 2024-09-19T04:13:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.875+0000 7fdb9849d640 1 -- 172.21.15.116:0/3642236689 wait complete. 2024-09-19T04:13:01.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 Processor -- start 2024-09-19T04:13:01.882 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 -- start start 2024-09-19T04:13:01.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb90150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb90071330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015f660 0x7fdb901544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:01.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900b75b0 con 0x7fdb9015f660 2024-09-19T04:13:01.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb900b7430 con 0x7fdb9006d890 2024-09-19T04:13:01.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9016b0f0 con 0x7fdb900b3790 2024-09-19T04:13:01.884 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb90150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb90150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34506/0 (socket says 172.21.15.116:34506) 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 -- 172.21.15.116:0/1745326285 learned_addr learned my addr 172.21.15.116:0/1745326285 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb95a38640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb90071330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 -- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 msgr2=0x7fdb90071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb90071330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 -- 172.21.15.116:0/1745326285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015f660 msgr2=0x7fdb901544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96a3a640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015f660 0x7fdb901544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015f660 0x7fdb901544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 -- 172.21.15.116:0/1745326285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb90153720 con 0x7fdb9006d890 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb95a38640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb90071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96a3a640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015f660 0x7fdb901544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb96239640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb90150d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800e930 tx=0x7fdb8800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:01.885 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb86ffd640 1 -- 172.21.15.116:0/1745326285 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88017070 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb86ffd640 1 -- 172.21.15.116:0/1745326285 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb8800ca00 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb90153a00 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb86ffd640 1 -- 172.21.15.116:0/1745326285 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800cdb0 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901512b0 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb86ffd640 1 -- 172.21.15.116:0/1745326285 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb8801a070 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb86ffd640 1 -- 172.21.15.116:0/1745326285 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7fdb8801caa0 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.879+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9009aa00 con 0x7fdb9006d890 2024-09-19T04:13:01.886 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.883+0000 7fdb86ffd640 1 -- 172.21.15.116:0/1745326285 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fdb8801cd50 con 0x7fdb9006d890 2024-09-19T04:13:01.952 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.947+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "da4116da-20bf-495a-a087-724ecd3b59dd", "id": 1} v 0) -- 0x7fdb9006efe0 con 0x7fdb9006d890 2024-09-19T04:13:01.952 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:01.954+0000 7fa5b8131180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:13:01.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb86ffd640 1 -- 172.21.15.116:0/1745326285 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "da4116da-20bf-495a-a087-724ecd3b59dd", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7fdb880134c0 con 0x7fdb9006d890 2024-09-19T04:13:01.958 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-09-19T04:13:01.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 msgr2=0x7fdb90150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb90150d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800e930 tx=0x7fdb8800edf0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 shutdown_connections 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb9015f660 0x7fdb901544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb900b3790 0x7fdb90071330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 --2- 172.21.15.116:0/1745326285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb9006d890 0x7fdb90150d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 >> 172.21.15.116:0/1745326285 conn(0x7fdb900b78d0 msgr2=0x7fdb90161d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 shutdown_connections 2024-09-19T04:13:01.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:01.955+0000 7fdb9849d640 1 -- 172.21.15.116:0/1745326285 wait complete. 2024-09-19T04:13:01.975 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 047d1ca8-76db-48fc-9895-0abdd6738312 2 2024-09-19T04:13:02.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a57c4d640 1 Processor -- start 2024-09-19T04:13:02.084 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a57c4d640 1 -- start start 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a57c4d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a5009b110 0x7f0a501690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a57c4d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5006da60 con 0x7f0a5015f660 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a57c4d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5006dd30 con 0x7f0a500b3790 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a57c4d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0a5006e5d0 con 0x7f0a5009b110 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a559e9640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0a500b3790 0x7f0a5006d350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57832/0 (socket says 172.21.15.116:57832) 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.079+0000 7f0a559e9640 1 -- 172.21.15.116:0/199540463 learned_addr learned my addr 172.21.15.116:0/199540463 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a561ea640 1 --2- 172.21.15.116:0/199540463 >> v2:172.21.15.76:3300/0 conn(0x7f0a5009b110 0x7f0a501690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4138208117 0 0) 0x7f0a5006dd30 con 0x7f0a500b3790 2024-09-19T04:13:02.087 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a3c003530 con 0x7f0a500b3790 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 <== mon.1 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 36973345 0 0) 0x7f0a3c003530 con 0x7f0a500b3790 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 >> v2:172.21.15.76:3300/0 conn(0x7f0a5009b110 msgr2=0x7f0a501690e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 --2- 172.21.15.116:0/199540463 >> v2:172.21.15.76:3300/0 conn(0x7f0a5009b110 0x7f0a501690e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 >> v1:172.21.15.76:6789/0 conn(0x7f0a5015f660 legacy=0x7f0a5009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5006e010 con 0x7f0a500b3790 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 <== mon.1 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f0a4c001680 con 0x7f0a500b3790 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 >> v1:172.21.15.116:6789/0 conn(0x7f0a500b3790 legacy=0x7f0a5006d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a3c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3c004810 0x7f0a3c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a3c0051d0 0x7f0a3c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5006e5d0 con 0x7f0a3c004810 2024-09-19T04:13:02.088 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c00c070 con 0x7f0a3c0051d0 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0a3c00c270 con 0x7f0a3c0039c0 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a559e9640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a3c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a561ea640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a3c0051d0 0x7f0a3c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a551e8640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3c004810 0x7f0a3c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a559e9640 1 -- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a3c0051d0 msgr2=0x7f0a3c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a559e9640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a3c0051d0 0x7f0a3c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a559e9640 1 -- 172.21.15.116:0/199540463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3c004810 msgr2=0x7f0a3c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a559e9640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a3c004810 0x7f0a3c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.089 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a559e9640 1 -- 172.21.15.116:0/199540463 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a3c003530 con 0x7f0a3c0039c0 2024-09-19T04:13:02.090 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a561ea640 1 -- 172.21.15.116:0/199540463 reap_dead start 2024-09-19T04:13:02.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a559e9640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a3c003de0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00c210 tx=0x7f0a4c01f370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2344dcd2eee833d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c007d40 con 0x7f0a3c0039c0 2024-09-19T04:13:02.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4c005b00 con 0x7f0a3c0039c0 2024-09-19T04:13:02.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a549e7640 1 -- 172.21.15.116:0/199540463 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c007070 con 0x7f0a3c0039c0 2024-09-19T04:13:02.093 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a57c4d640 1 -- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 msgr2=0x7f0a3c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.083+0000 7f0a57c4d640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a3c003de0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00c210 tx=0x7f0a4c01f370 comp rx=0 tx=0).stop 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- 172.21.15.116:0/199540463 shutdown_connections 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 --2- 172.21.15.116:0/199540463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a3c003de0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- 172.21.15.116:0/199540463 >> 172.21.15.116:0/199540463 conn(0x7f0a500b78d0 msgr2=0x7f0a500b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- 172.21.15.116:0/199540463 shutdown_connections 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- 172.21.15.116:0/199540463 wait complete. 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 Processor -- start 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- start start 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a500af510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a50154b30 0x7f0a50153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a500b2e80 0x7f0a50157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a500b5f00 con 0x7f0a50154b30 2024-09-19T04:13:02.094 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0a500b5ca0 con 0x7f0a500b2e80 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0a500b3b70 con 0x7f0a3c0039c0 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a559e9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a500af510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a559e9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a500af510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:49568/0 (socket says 172.21.15.116:49568) 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a559e9640 1 -- 172.21.15.116:0/3351403747 learned_addr learned my addr 172.21.15.116:0/3351403747 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a561ea640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a500b2e80 0x7f0a50157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a551e8640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a50154b30 0x7f0a50153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a561ea640 1 -- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 msgr2=0x7f0a500af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a561ea640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a500af510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a561ea640 1 -- 172.21.15.116:0/3351403747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a50154b30 msgr2=0x7f0a50153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a561ea640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a50154b30 0x7f0a50153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.095 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a561ea640 1 -- 172.21.15.116:0/3351403747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a500aff60 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a559e9640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a500af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a551e8640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a50154b30 0x7f0a50153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a561ea640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a500b2e80 0x7f0a50157da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0a500699e0 tx=0x7f0a4800f1b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a427fc640 1 -- 172.21.15.116:0/3351403747 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800fce0 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a427fc640 1 -- 172.21.15.116:0/3351403747 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a4801d070 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a500b23a0 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a427fc640 1 -- 172.21.15.116:0/3351403747 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48015670 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a500b2960 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a427fc640 1 -- 172.21.15.116:0/3351403747 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a48015810 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a427fc640 1 -- 172.21.15.116:0/3351403747 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f0a480213c0 con 0x7f0a500b2e80 2024-09-19T04:13:02.096 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.087+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5006d350 con 0x7f0a500b2e80 2024-09-19T04:13:02.097 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.091+0000 7f0a427fc640 1 -- 172.21.15.116:0/3351403747 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0a48010040 con 0x7f0a500b2e80 2024-09-19T04:13:02.114 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:02.111+0000 7f89675d8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T04:13:02.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.159+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "047d1ca8-76db-48fc-9895-0abdd6738312", "id": 2} v 0) -- 0x7f0a5015ea00 con 0x7f0a500b2e80 2024-09-19T04:13:02.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a427fc640 1 -- 172.21.15.116:0/3351403747 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "047d1ca8-76db-48fc-9895-0abdd6738312", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f0a48015d60 con 0x7f0a500b2e80 2024-09-19T04:13:02.172 INFO:teuthology.orchestra.run.smithi116.stdout:2 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a500b2e80 msgr2=0x7f0a50157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a500b2e80 0x7f0a50157da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0a500699e0 tx=0x7f0a4800f1b0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 shutdown_connections 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a500b2e80 0x7f0a50157da0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a50154b30 0x7f0a50153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 --2- 172.21.15.116:0/3351403747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a3c0039c0 0x7f0a500af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 >> 172.21.15.116:0/3351403747 conn(0x7f0a500b78d0 msgr2=0x7f0a50160b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 shutdown_connections 2024-09-19T04:13:02.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.167+0000 7f0a57c4d640 1 -- 172.21.15.116:0/3351403747 wait complete. 2024-09-19T04:13:02.189 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 83621a44-76e6-4acd-8e4c-fcba39569ac2 3 2024-09-19T04:13:02.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f797bc4d640 1 Processor -- start 2024-09-19T04:13:02.300 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f797bc4d640 1 -- start start 2024-09-19T04:13:02.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f797bc4d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f797409ac60 0x7f797409ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f797bc4d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f797409b040 0x7f79740af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.301 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f797bc4d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79740afd80 con 0x7f79740b0e20 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f797bc4d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f79740b0060 con 0x7f797409ac60 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f797bc4d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f79740b0340 con 0x7f797409b040 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f79791e8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f797409b040 0x7f79740af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f79791e8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f797409b040 0x7f79740af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34538/0 (socket says 172.21.15.116:34538) 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f79791e8640 1 -- 172.21.15.116:0/1764709553 learned_addr learned my addr 172.21.15.116:0/1764709553 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.295+0000 7f79799e9640 1 --2- 172.21.15.116:0/1764709553 >> v2:172.21.15.76:3300/0 conn(0x7f797409ac60 0x7f797409ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79791e8640 1 -- 172.21.15.116:0/1764709553 >> v2:172.21.15.76:3300/0 conn(0x7f797409ac60 msgr2=0x7f797409ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79791e8640 1 --2- 172.21.15.116:0/1764709553 >> v2:172.21.15.76:3300/0 conn(0x7f797409ac60 0x7f797409ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79791e8640 1 -- 172.21.15.116:0/1764709553 >> v1:172.21.15.76:6789/0 conn(0x7f79740b0e20 legacy=0x7f79740b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.302 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79791e8640 1 -- 172.21.15.116:0/1764709553 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79740b0620 con 0x7f797409b040 2024-09-19T04:13:02.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79791e8640 1 --2- 172.21.15.116:0/1764709553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f79740af730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f79740b52b0 tx=0x7f796801d390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52465452c57c33fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.304 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79789e7640 1 -- 172.21.15.116:0/1764709553 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79680a5810 con 0x7f797409b040 2024-09-19T04:13:02.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79789e7640 1 -- 172.21.15.116:0/1764709553 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79680a59b0 con 0x7f797409b040 2024-09-19T04:13:02.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.299+0000 7f79789e7640 1 -- 172.21.15.116:0/1764709553 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79680a5d80 con 0x7f797409b040 2024-09-19T04:13:02.305 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- 172.21.15.116:0/1764709553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 msgr2=0x7f79740af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 --2- 172.21.15.116:0/1764709553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f79740af730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f79740b52b0 tx=0x7f796801d390 comp rx=0 tx=0).stop 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- 172.21.15.116:0/1764709553 shutdown_connections 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 --2- 172.21.15.116:0/1764709553 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f79740af730 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 --2- 172.21.15.116:0/1764709553 >> v2:172.21.15.76:3300/0 conn(0x7f797409ac60 0x7f797409ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- 172.21.15.116:0/1764709553 >> 172.21.15.116:0/1764709553 conn(0x7f79740b78d0 msgr2=0x7f79740b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- 172.21.15.116:0/1764709553 shutdown_connections 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- 172.21.15.116:0/1764709553 wait complete. 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 Processor -- start 2024-09-19T04:13:02.306 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- start start 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f797409ac60 0x7f7974150270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f797414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79740b0e20 0x7f7974150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f797415d3e0 con 0x7f79740b0e20 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f797415d260 con 0x7f797409b040 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f797415d560 con 0x7f797409ac60 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79791e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f797414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f797409ac60 0x7f7974150270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79791e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f797414cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34548/0 (socket says 172.21.15.116:34548) 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f797409ac60 0x7f7974150270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:49580/0 (socket says 172.21.15.116:49580) 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79791e8640 1 -- 172.21.15.116:0/3478091883 learned_addr learned my addr 172.21.15.116:0/3478091883 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.309 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797a1ea640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79740b0e20 0x7f7974150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 -- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 msgr2=0x7f797414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f797414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 -- 172.21.15.116:0/3478091883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79740b0e20 msgr2=0x7f7974150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79740b0e20 0x7f7974150b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 -- 172.21.15.116:0/3478091883 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79741661b0 con 0x7f797409ac60 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79791e8640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f797414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79799e9640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f797409ac60 0x7f7974150270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f797000d5f0 tx=0x7f797000dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79667fc640 1 -- 172.21.15.116:0/3478091883 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700125f0 con 0x7f797409ac60 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79667fc640 1 -- 172.21.15.116:0/3478091883 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7970019070 con 0x7f797409ac60 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f797414ea80 con 0x7f797409ac60 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f79667fc640 1 -- 172.21.15.116:0/3478091883 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79700129b0 con 0x7f797409ac60 2024-09-19T04:13:02.310 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.303+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f797414ed60 con 0x7f797409ac60 2024-09-19T04:13:02.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.307+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79740a04c0 con 0x7f797409ac60 2024-09-19T04:13:02.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.307+0000 7f79667fc640 1 -- 172.21.15.116:0/3478091883 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7970012b50 con 0x7f797409ac60 2024-09-19T04:13:02.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.307+0000 7f79667fc640 1 -- 172.21.15.116:0/3478091883 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f7970014910 con 0x7f797409ac60 2024-09-19T04:13:02.311 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.307+0000 7f79667fc640 1 -- 172.21.15.116:0/3478091883 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f7970014c40 con 0x7f797409ac60 2024-09-19T04:13:02.334 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:02.331+0000 7f89675d8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T04:13:02.381 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.379+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "83621a44-76e6-4acd-8e4c-fcba39569ac2", "id": 3} v 0) -- 0x7f7974071bd0 con 0x7f797409ac60 2024-09-19T04:13:02.387 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f79667fc640 1 -- 172.21.15.116:0/3478091883 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "83621a44-76e6-4acd-8e4c-fcba39569ac2", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f7970014480 con 0x7f797409ac60 2024-09-19T04:13:02.388 INFO:teuthology.orchestra.run.smithi116.stdout:3 2024-09-19T04:13:02.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f797409ac60 msgr2=0x7f7974150270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f797409ac60 0x7f7974150270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f797000d5f0 tx=0x7f797000dab0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 shutdown_connections 2024-09-19T04:13:02.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79740b0e20 0x7f7974150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f797409b040 0x7f797414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 --2- 172.21.15.116:0/3478091883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f797409ac60 0x7f7974150270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 >> 172.21.15.116:0/3478091883 conn(0x7f79740b78d0 msgr2=0x7f7974071020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 shutdown_connections 2024-09-19T04:13:02.390 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.383+0000 7f797bc4d640 1 -- 172.21.15.116:0/3478091883 wait complete. 2024-09-19T04:13:02.405 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 388909a1-37ed-4039-85ef-fe9f10af682c 4 2024-09-19T04:13:02.514 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:02.518+0000 7fa5b8131180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:13:02.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.511+0000 7f762a595640 1 Processor -- start 2024-09-19T04:13:02.516 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.511+0000 7f762a595640 1 -- start start 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624071ef0 0x7f7624141610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7624141b50 0x7f76240d1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7624142300 con 0x7f7624141230 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f76241425e0 con 0x7f7624141b50 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f76240d1160 con 0x7f7624071ef0 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f76237fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624071ef0 0x7f7624141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7623fff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7624141230 0x7f76240717e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57842/0 (socket says 172.21.15.116:57842) 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7623fff640 1 -- 172.21.15.116:0/3615724841 learned_addr learned my addr 172.21.15.116:0/3615724841 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-09-19T04:13:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7628b32640 1 --2- 172.21.15.116:0/3615724841 >> v2:172.21.15.116:3300/0 conn(0x7f7624141b50 0x7f76240d1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f76237fe640 1 -- 172.21.15.116:0/3615724841 >> v2:172.21.15.116:3300/0 conn(0x7f7624141b50 msgr2=0x7f76240d1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:02.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f76237fe640 1 --2- 172.21.15.116:0/3615724841 >> v2:172.21.15.116:3300/0 conn(0x7f7624141b50 0x7f76240d1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f76237fe640 1 -- 172.21.15.116:0/3615724841 >> v1:172.21.15.116:6789/0 conn(0x7f7624141230 legacy=0x7f76240717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7622ffd640 1 -- 172.21.15.116:0/3615724841 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 389194805 0 0) 0x7f7624142300 con 0x7f7624141230 2024-09-19T04:13:02.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f76237fe640 1 -- 172.21.15.116:0/3615724841 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76240d1440 con 0x7f7624071ef0 2024-09-19T04:13:02.519 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7628b32640 1 --2- 172.21.15.116:0/3615724841 >> v2:172.21.15.116:3300/0 conn(0x7f7624141b50 0x7f76240d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:02.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f76237fe640 1 --2- 172.21.15.116:0/3615724841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624071ef0 0x7f7624141610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f760c0097c0 tx=0x7f760c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94d5c1c467f3f483 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7622ffd640 1 -- 172.21.15.116:0/3615724841 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0b0070 con 0x7f7624071ef0 2024-09-19T04:13:02.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7622ffd640 1 -- 172.21.15.116:0/3615724841 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c0ab600 con 0x7f7624071ef0 2024-09-19T04:13:02.520 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f7622ffd640 1 -- 172.21.15.116:0/3615724841 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0ab9d0 con 0x7f7624071ef0 2024-09-19T04:13:02.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- 172.21.15.116:0/3615724841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624071ef0 msgr2=0x7f7624141610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 --2- 172.21.15.116:0/3615724841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624071ef0 0x7f7624141610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f760c0097c0 tx=0x7f760c01c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.521 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- 172.21.15.116:0/3615724841 shutdown_connections 2024-09-19T04:13:02.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 --2- 172.21.15.116:0/3615724841 >> v2:172.21.15.116:3300/0 conn(0x7f7624141b50 0x7f76240d1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 --2- 172.21.15.116:0/3615724841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624071ef0 0x7f7624141610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- 172.21.15.116:0/3615724841 >> 172.21.15.116:0/3615724841 conn(0x7f762406c0c0 msgr2=0x7f762406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- 172.21.15.116:0/3615724841 shutdown_connections 2024-09-19T04:13:02.522 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.515+0000 7f762a595640 1 -- 172.21.15.116:0/3615724841 wait complete. 2024-09-19T04:13:02.524 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 Processor -- start 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 -- start start 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7624071ef0 0x7f7624155a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624141230 0x7f7624144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7624141b50 0x7f7624138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f76240d2830 con 0x7f7624141b50 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76240d26b0 con 0x7f7624071ef0 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f76240d29b0 con 0x7f7624141230 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624141230 0x7f7624144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f7623fff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7624071ef0 0x7f7624155a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624141230 0x7f7624144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:49606/0 (socket says 172.21.15.116:49606) 2024-09-19T04:13:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 -- 172.21.15.116:0/2497852120 learned_addr learned my addr 172.21.15.116:0/2497852120 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 -- 172.21.15.116:0/2497852120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7624071ef0 msgr2=0x7f7624155a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7624071ef0 0x7f7624155a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f7628b32640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7624141b50 0x7f7624138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 -- 172.21.15.116:0/2497852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7624141b50 msgr2=0x7f7624138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7624141b50 0x7f7624138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 -- 172.21.15.116:0/2497852120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7624155f40 con 0x7f7624141230 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f76237fe640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624141230 0x7f7624144240 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f760c0ae040 tx=0x7f760c01ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f7620ff9640 1 -- 172.21.15.116:0/2497852120 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c0b0050 con 0x7f7624141230 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f7620ff9640 1 -- 172.21.15.116:0/2497852120 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f760c0b1040 con 0x7f7624141230 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7624137c70 con 0x7f7624141230 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f7620ff9640 1 -- 172.21.15.116:0/2497852120 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760c006e20 con 0x7f7624141230 2024-09-19T04:13:02.526 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7624138230 con 0x7f7624141230 2024-09-19T04:13:02.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f7620ff9640 1 -- 172.21.15.116:0/2497852120 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f760c0be060 con 0x7f7624141230 2024-09-19T04:13:02.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f7620ff9640 1 -- 172.21.15.116:0/2497852120 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f760c0b5610 con 0x7f7624141230 2024-09-19T04:13:02.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.519+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7624141610 con 0x7f7624141230 2024-09-19T04:13:02.527 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.523+0000 7f7620ff9640 1 -- 172.21.15.116:0/2497852120 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f760c016090 con 0x7f7624141230 2024-09-19T04:13:02.568 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:02.570+0000 7fa5b8131180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:13:02.590 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.587+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "388909a1-37ed-4039-85ef-fe9f10af682c", "id": 4} v 0) -- 0x7f76240b68d0 con 0x7f7624141230 2024-09-19T04:13:02.598 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f7620ff9640 1 -- 172.21.15.116:0/2497852120 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "388909a1-37ed-4039-85ef-fe9f10af682c", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f760c004220 con 0x7f7624141230 2024-09-19T04:13:02.598 INFO:teuthology.orchestra.run.smithi116.stdout:4 2024-09-19T04:13:02.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624141230 msgr2=0x7f7624144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624141230 0x7f7624144240 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f760c0ae040 tx=0x7f760c01ace0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 shutdown_connections 2024-09-19T04:13:02.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7624141b50 0x7f7624138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.599 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7624141230 0x7f7624144240 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 --2- 172.21.15.116:0/2497852120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7624071ef0 0x7f7624155a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 >> 172.21.15.116:0/2497852120 conn(0x7f762406c0c0 msgr2=0x7f76240b6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 shutdown_connections 2024-09-19T04:13:02.600 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.595+0000 7f762a595640 1 -- 172.21.15.116:0/2497852120 wait complete. 2024-09-19T04:13:02.615 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new 2f3b2716-0ce3-487d-add8-be076a5138c4 5 2024-09-19T04:13:02.622 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:02.622+0000 7fa5b8131180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:13:02.699 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:02.702+0000 7fa5b8131180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:13:02.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.723+0000 7f2c9ab65640 1 Processor -- start 2024-09-19T04:13:02.728 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.723+0000 7f2c9ab65640 1 -- start start 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940b3280 0x7f2c940cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c9415d0d0 0x7f2c940b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c940cf1e0 0x7f2c9415d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2c940b3ba0 con 0x7f2c940cf1e0 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2c940ce140 con 0x7f2c9415d0d0 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2c940ce420 con 0x7f2c940b3280 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c9415d0d0 0x7f2c940b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c9415d0d0 0x7f2c940b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34582/0 (socket says 172.21.15.116:34582) 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 -- 172.21.15.116:0/2950489057 learned_addr learned my addr 172.21.15.116:0/2950489057 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c98901640 1 --2- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940b3280 0x7f2c940cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 -- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940b3280 msgr2=0x7f2c940cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 --2- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940b3280 0x7f2c940cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.730 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c99102640 1 --2- 172.21.15.116:0/2950489057 >> v2:172.21.15.76:3300/0 conn(0x7f2c940cf1e0 0x7f2c9415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 -- 172.21.15.116:0/2950489057 >> v2:172.21.15.76:3300/0 conn(0x7f2c940cf1e0 msgr2=0x7f2c9415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:02.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 --2- 172.21.15.116:0/2950489057 >> v2:172.21.15.76:3300/0 conn(0x7f2c940cf1e0 0x7f2c9415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.731 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 -- 172.21.15.116:0/2950489057 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c940ce700 con 0x7f2c9415d0d0 2024-09-19T04:13:02.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c93fff640 1 --2- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c9415d0d0 0x7f2c940b3660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c940b3f40 tx=0x7f2c8001c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9d4ae2a0c9da88a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c937fe640 1 -- 172.21.15.116:0/2950489057 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800ab620 con 0x7f2c9415d0d0 2024-09-19T04:13:02.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c937fe640 1 -- 172.21.15.116:0/2950489057 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c800b0070 con 0x7f2c9415d0d0 2024-09-19T04:13:02.732 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c937fe640 1 -- 172.21.15.116:0/2950489057 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800ab9e0 con 0x7f2c9415d0d0 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c9415d0d0 msgr2=0x7f2c940b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c9415d0d0 0x7f2c940b3660 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f2c940b3f40 tx=0x7f2c8001c460 comp rx=0 tx=0).stop 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2950489057 shutdown_connections 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2950489057 >> v2:172.21.15.76:3300/0 conn(0x7f2c940cf1e0 0x7f2c9415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c9415d0d0 0x7f2c940b3660 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2950489057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940b3280 0x7f2c940cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2950489057 >> 172.21.15.116:0/2950489057 conn(0x7f2c940b6870 msgr2=0x7f2c940b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2950489057 shutdown_connections 2024-09-19T04:13:02.733 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.727+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2950489057 wait complete. 2024-09-19T04:13:02.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 Processor -- start 2024-09-19T04:13:02.734 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 -- start start 2024-09-19T04:13:02.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c940b3280 0x7f2c94155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940cf1e0 0x7f2c94152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.736 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c9415d0d0 0x7f2c940cfa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9415a060 con 0x7f2c9415d0d0 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2c94159ee0 con 0x7f2c940b3280 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2c9415e160 con 0x7f2c940cf1e0 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c98901640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c940b3280 0x7f2c94155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c93fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940cf1e0 0x7f2c94152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c98901640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c940b3280 0x7f2c94155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34592/0 (socket says 172.21.15.116:34592) 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c98901640 1 -- 172.21.15.116:0/2080761864 learned_addr learned my addr 172.21.15.116:0/2080761864 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c99102640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c9415d0d0 0x7f2c940cfa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c93fff640 1 -- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c940b3280 msgr2=0x7f2c94155100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c93fff640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c940b3280 0x7f2c94155100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c93fff640 1 -- 172.21.15.116:0/2080761864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c9415d0d0 msgr2=0x7f2c940cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.737 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c93fff640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c9415d0d0 0x7f2c940cfa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c93fff640 1 -- 172.21.15.116:0/2080761864 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c94152be0 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c98901640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c940b3280 0x7f2c94155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c93fff640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940cf1e0 0x7f2c94152390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2c800ac000 tx=0x7f2c80019200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c917fa640 1 -- 172.21.15.116:0/2080761864 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800abd80 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c917fa640 1 -- 172.21.15.116:0/2080761864 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c800bf410 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c94154620 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c917fa640 1 -- 172.21.15.116:0/2080761864 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800bf800 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c941528d0 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c917fa640 1 -- 172.21.15.116:0/2080761864 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2c800bd020 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c917fa640 1 -- 172.21.15.116:0/2080761864 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f2c800b5a90 con 0x7f2c940cf1e0 2024-09-19T04:13:02.738 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.731+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9415d810 con 0x7f2c940cf1e0 2024-09-19T04:13:02.739 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.735+0000 7f2c917fa640 1 -- 172.21.15.116:0/2080761864 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2c800bfe20 con 0x7f2c940cf1e0 2024-09-19T04:13:02.751 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:02.754+0000 7fa5b8131180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:13:02.808 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.803+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2f3b2716-0ce3-487d-add8-be076a5138c4", "id": 5} v 0) -- 0x7f2c94151490 con 0x7f2c940cf1e0 2024-09-19T04:13:02.816 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.811+0000 7f2c917fa640 1 -- 172.21.15.116:0/2080761864 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2f3b2716-0ce3-487d-add8-be076a5138c4", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f2c800c4060 con 0x7f2c940cf1e0 2024-09-19T04:13:02.817 INFO:teuthology.orchestra.run.smithi116.stdout:5 2024-09-19T04:13:02.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940cf1e0 msgr2=0x7f2c94152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940cf1e0 0x7f2c94152390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2c800ac000 tx=0x7f2c80019200 comp rx=0 tx=0).stop 2024-09-19T04:13:02.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 shutdown_connections 2024-09-19T04:13:02.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c9415d0d0 0x7f2c940cfa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c940cf1e0 0x7f2c94152390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.818 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 --2- 172.21.15.116:0/2080761864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c940b3280 0x7f2c94155100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 >> 172.21.15.116:0/2080761864 conn(0x7f2c940b6870 msgr2=0x7f2c940b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 shutdown_connections 2024-09-19T04:13:02.819 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.815+0000 7f2c9ab65640 1 -- 172.21.15.116:0/2080761864 wait complete. 2024-09-19T04:13:02.830 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:02.830+0000 7fa5b8131180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:13:02.834 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new c4170a51-44cc-44f1-b384-ae0df43b54dc 6 2024-09-19T04:13:02.882 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:02.886+0000 7fa5b8131180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:13:02.917 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:02.915+0000 7f89675d8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T04:13:02.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.943+0000 7f01362f7640 1 Processor -- start 2024-09-19T04:13:02.949 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.943+0000 7f01362f7640 1 -- start start 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f01362f7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f013015d290 0x7f013015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f01362f7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f013015fc10 con 0x7f013015f830 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f01362f7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f013015d800 con 0x7f013006c3d0 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f01362f7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f013015da40 con 0x7f013015d290 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f0134894640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f013015d290 0x7f013015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f0134894640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f013015d290 0x7f013015e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34604/0 (socket says 172.21.15.116:34604) 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f0134894640 1 -- 172.21.15.116:0/3098638043 learned_addr learned my addr 172.21.15.116:0/3098638043 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f0134894640 1 -- 172.21.15.116:0/3098638043 >> v1:172.21.15.116:6789/0 conn(0x7f013006c3d0 legacy=0x7f013015ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.950 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f0134894640 1 -- 172.21.15.116:0/3098638043 >> v1:172.21.15.76:6789/0 conn(0x7f013015f830 legacy=0x7f013006bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.951 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f0134894640 1 -- 172.21.15.116:0/3098638043 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01300b3570 con 0x7f013015d290 2024-09-19T04:13:02.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f0134894640 1 --2- 172.21.15.116:0/3098638043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013015d290 0x7f013015e5e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0128009e50 tx=0x7f013015f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509acd4383ca394a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f012effd640 1 -- 172.21.15.116:0/3098638043 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01280a58d0 con 0x7f013015d290 2024-09-19T04:13:02.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f012effd640 1 -- 172.21.15.116:0/3098638043 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01280a5a70 con 0x7f013015d290 2024-09-19T04:13:02.953 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.947+0000 7f012effd640 1 -- 172.21.15.116:0/3098638043 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01280a5e40 con 0x7f013015d290 2024-09-19T04:13:02.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- 172.21.15.116:0/3098638043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013015d290 msgr2=0x7f013015e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 --2- 172.21.15.116:0/3098638043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013015d290 0x7f013015e5e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0128009e50 tx=0x7f013015f330 comp rx=0 tx=0).stop 2024-09-19T04:13:02.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- 172.21.15.116:0/3098638043 shutdown_connections 2024-09-19T04:13:02.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 --2- 172.21.15.116:0/3098638043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013015d290 0x7f013015e5e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- 172.21.15.116:0/3098638043 >> 172.21.15.116:0/3098638043 conn(0x7f01300b6800 msgr2=0x7f01300b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:02.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- 172.21.15.116:0/3098638043 shutdown_connections 2024-09-19T04:13:02.954 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- 172.21.15.116:0/3098638043 wait complete. 2024-09-19T04:13:02.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 Processor -- start 2024-09-19T04:13:02.955 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- start start 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013006c3d0 0x7f01300b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f013015d290 0x7f0130149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f013015f830 0x7f013014c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f013016a3a0 con 0x7f013015f830 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f013016a220 con 0x7f013006c3d0 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f01362f7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f013016a520 con 0x7f013015d290 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f013015d290 0x7f0130149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.958 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f013015d290 0x7f0130149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.116:49616/0 (socket says 172.21.15.116:49616) 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 -- 172.21.15.116:0/1197361654 learned_addr learned my addr 172.21.15.116:0/1197361654 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012ffff640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013006c3d0 0x7f01300b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f0134894640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f013015f830 0x7f013014c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 -- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013006c3d0 msgr2=0x7f01300b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013006c3d0 0x7f01300b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 -- 172.21.15.116:0/1197361654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f013015f830 msgr2=0x7f013014c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f013015f830 0x7f013014c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f012f7fe640 1 -- 172.21.15.116:0/1197361654 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0130149cc0 con 0x7f013015d290 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.951+0000 7f0134894640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f013015f830 0x7f013014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f012f7fe640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f013015d290 0x7f0130149550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f012400c3c0 tx=0x7f012400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f012cff9640 1 -- 172.21.15.116:0/1197361654 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124017070 con 0x7f013015d290 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f012cff9640 1 -- 172.21.15.116:0/1197361654 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01240052d0 con 0x7f013015d290 2024-09-19T04:13:02.959 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f012cff9640 1 -- 172.21.15.116:0/1197361654 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124003840 con 0x7f013015d290 2024-09-19T04:13:02.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0130149fa0 con 0x7f013015d290 2024-09-19T04:13:02.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f013014a560 con 0x7f013015d290 2024-09-19T04:13:02.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f012cff9640 1 -- 172.21.15.116:0/1197361654 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f012401a070 con 0x7f013015d290 2024-09-19T04:13:02.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f012cff9640 1 -- 172.21.15.116:0/1197361654 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f0124011b80 con 0x7f013015d290 2024-09-19T04:13:02.960 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.955+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013006bcc0 con 0x7f013015d290 2024-09-19T04:13:02.962 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:02.959+0000 7f012cff9640 1 -- 172.21.15.116:0/1197361654 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f0124021020 con 0x7f013015d290 2024-09-19T04:13:02.975 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:02.971+0000 7f89675d8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T04:13:03.028 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:03.023+0000 7f89675d8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T04:13:03.030 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.027+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c4170a51-44cc-44f1-b384-ae0df43b54dc", "id": 6} v 0) -- 0x7f013006e7c0 con 0x7f013015d290 2024-09-19T04:13:03.033 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:03.034+0000 7fa5b8131180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:13:03.036 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.031+0000 7f012cff9640 1 -- 172.21.15.116:0/1197361654 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c4170a51-44cc-44f1-b384-ae0df43b54dc", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f0124011590 con 0x7f013015d290 2024-09-19T04:13:03.037 INFO:teuthology.orchestra.run.smithi116.stdout:6 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f013015d290 msgr2=0x7f0130149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f013015d290 0x7f0130149550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f012400c3c0 tx=0x7f012400c880 comp rx=0 tx=0).stop 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 shutdown_connections 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f013015f830 0x7f013014c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f013015d290 0x7f0130149550 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 --2- 172.21.15.116:0/1197361654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013006c3d0 0x7f01300b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 >> 172.21.15.116:0/1197361654 conn(0x7f01300b6800 msgr2=0x7f013015a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 shutdown_connections 2024-09-19T04:13:03.038 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.035+0000 7f01362f7640 1 -- 172.21.15.116:0/1197361654 wait complete. 2024-09-19T04:13:03.054 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd new ec38642c-af4a-4d58-9af5-9200d729196d 7 2024-09-19T04:13:03.097 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:03.098+0000 7fa5b8131180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:13:03.110 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:03.107+0000 7f89675d8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T04:13:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237c124640 1 Processor -- start 2024-09-19T04:13:03.164 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237c124640 1 -- start start 2024-09-19T04:13:03.164 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f89675d8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T04:13:03.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237c124640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f237409ac60 0x7f237409ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.165 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237c124640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f23740af730 0x7f237415d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237c124640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23740afd80 con 0x7f237409b040 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237c124640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f23740afb60 con 0x7f23740af730 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237c124640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f23740b0060 con 0x7f237409ac60 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f2379ec0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f237409ac60 0x7f237409ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f23796bf640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f237409b040 0x7f23740b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57870/0 (socket says 172.21.15.116:57870) 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f23796bf640 1 -- 172.21.15.116:0/3453884159 learned_addr learned my addr 172.21.15.116:0/3453884159 (peer_addr_for_me v1:172.21.15.116:0/0) 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.159+0000 7f237a6c1640 1 --2- 172.21.15.116:0/3453884159 >> v2:172.21.15.76:3300/0 conn(0x7f23740af730 0x7f237415d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2379ec0640 1 -- 172.21.15.116:0/3453884159 >> v2:172.21.15.76:3300/0 conn(0x7f23740af730 msgr2=0x7f237415d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2379ec0640 1 --2- 172.21.15.116:0/3453884159 >> v2:172.21.15.76:3300/0 conn(0x7f23740af730 0x7f237415d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2379ec0640 1 -- 172.21.15.116:0/3453884159 >> v1:172.21.15.116:6789/0 conn(0x7f237409b040 legacy=0x7f23740b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2379ec0640 1 -- 172.21.15.116:0/3453884159 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23740b0340 con 0x7f237409ac60 2024-09-19T04:13:03.166 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2378ebe640 1 -- 172.21.15.116:0/3453884159 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4035697654 0 0) 0x7f23740afd80 con 0x7f237409b040 2024-09-19T04:13:03.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2379ec0640 1 --2- 172.21.15.116:0/3453884159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f237409ba00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f23740b7d70 tx=0x7f237001c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a036ec766afd128 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:03.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2378ebe640 1 -- 172.21.15.116:0/3453884159 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700b0070 con 0x7f237409ac60 2024-09-19T04:13:03.167 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2378ebe640 1 -- 172.21.15.116:0/3453884159 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23700ab600 con 0x7f237409ac60 2024-09-19T04:13:03.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f2378ebe640 1 -- 172.21.15.116:0/3453884159 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700ab9d0 con 0x7f237409ac60 2024-09-19T04:13:03.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 -- 172.21.15.116:0/3453884159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 msgr2=0x7f237409ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 --2- 172.21.15.116:0/3453884159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f237409ba00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f23740b7d70 tx=0x7f237001c3f0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.168 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 -- 172.21.15.116:0/3453884159 shutdown_connections 2024-09-19T04:13:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 --2- 172.21.15.116:0/3453884159 >> v2:172.21.15.76:3300/0 conn(0x7f23740af730 0x7f237415d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 --2- 172.21.15.116:0/3453884159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f237409ba00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 -- 172.21.15.116:0/3453884159 >> 172.21.15.116:0/3453884159 conn(0x7f23740b78d0 msgr2=0x7f23740b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 -- 172.21.15.116:0/3453884159 shutdown_connections 2024-09-19T04:13:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.163+0000 7f237c124640 1 -- 172.21.15.116:0/3453884159 wait complete. 2024-09-19T04:13:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 Processor -- start 2024-09-19T04:13:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 -- start start 2024-09-19T04:13:03.171 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f2374139e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f237409b040 0x7f2374154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23740af730 0x7f2374149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23740b23c0 con 0x7f237409b040 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f237415d550 con 0x7f237409ac60 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f23740b2540 con 0x7f23740af730 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f2374139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237a6c1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23740af730 0x7f2374149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f2374139e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:34618/0 (socket says 172.21.15.116:34618) 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 -- 172.21.15.116:0/3738068522 learned_addr learned my addr 172.21.15.116:0/3738068522 (peer_addr_for_me v2:172.21.15.116:0/0) 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f23796bf640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f237409b040 0x7f2374154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.172 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 -- 172.21.15.116:0/3738068522 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23740af730 msgr2=0x7f2374149c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23740af730 0x7f2374149c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 -- 172.21.15.116:0/3738068522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f237409b040 msgr2=0x7f2374154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f237409b040 0x7f2374154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 -- 172.21.15.116:0/3738068522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2374155280 con 0x7f237409ac60 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f2379ec0640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f2374139e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f237001a020 tx=0x7f237001ac30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f23667fc640 1 -- 172.21.15.116:0/3738068522 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700b0050 con 0x7f237409ac60 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f23667fc640 1 -- 172.21.15.116:0/3738068522 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23700b1040 con 0x7f237409ac60 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2374155560 con 0x7f237409ac60 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f23667fc640 1 -- 172.21.15.116:0/3738068522 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370006e20 con 0x7f237409ac60 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2374149570 con 0x7f237409ac60 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f23667fc640 1 -- 172.21.15.116:0/3738068522 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f23700be060 con 0x7f237409ac60 2024-09-19T04:13:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f23667fc640 1 -- 172.21.15.116:0/3738068522 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f2370003c80 con 0x7f237409ac60 2024-09-19T04:13:03.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.167+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237415caa0 con 0x7f237409ac60 2024-09-19T04:13:03.174 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.171+0000 7f23667fc640 1 -- 172.21.15.116:0/3738068522 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f23700d72d0 con 0x7f237409ac60 2024-09-19T04:13:03.243 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.239+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ec38642c-af4a-4d58-9af5-9200d729196d", "id": 7} v 0) -- 0x7f237415c7c0 con 0x7f237409ac60 2024-09-19T04:13:03.246 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:03.243+0000 7f89675d8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T04:13:03.250 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f23667fc640 1 -- 172.21.15.116:0/3738068522 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ec38642c-af4a-4d58-9af5-9200d729196d", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f23700c5070 con 0x7f237409ac60 2024-09-19T04:13:03.250 INFO:teuthology.orchestra.run.smithi116.stdout:7 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 msgr2=0x7f2374139e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f2374139e10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f237001a020 tx=0x7f237001ac30 comp rx=0 tx=0).stop 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 shutdown_connections 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23740af730 0x7f2374149c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f237409b040 0x7f2374154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 --2- 172.21.15.116:0/3738068522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f237409ac60 0x7f2374139e10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 >> 172.21.15.116:0/3738068522 conn(0x7f23740b78d0 msgr2=0x7f237415d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 shutdown_connections 2024-09-19T04:13:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:03.247+0000 7f237c124640 1 -- 172.21.15.116:0/3738068522 wait complete. 2024-09-19T04:13:03.267 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-19T04:13:03.268 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-19T04:13:03.270 INFO:tasks.ceph.osd.0:Started 2024-09-19T04:13:03.270 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-19T04:13:03.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-19T04:13:03.273 INFO:tasks.ceph.osd.1:Started 2024-09-19T04:13:03.273 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-19T04:13:03.273 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-19T04:13:03.275 INFO:tasks.ceph.osd.2:Started 2024-09-19T04:13:03.275 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-19T04:13:03.276 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-19T04:13:03.278 INFO:tasks.ceph.osd.3:Started 2024-09-19T04:13:03.278 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-19T04:13:03.278 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-19T04:13:03.280 INFO:tasks.ceph.osd.4:Started 2024-09-19T04:13:03.281 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-19T04:13:03.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-19T04:13:03.283 INFO:tasks.ceph.osd.5:Started 2024-09-19T04:13:03.283 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-19T04:13:03.283 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-19T04:13:03.285 INFO:tasks.ceph.osd.6:Started 2024-09-19T04:13:03.285 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-19T04:13:03.285 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-19T04:13:03.287 INFO:tasks.ceph.osd.7:Started 2024-09-19T04:13:03.288 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:03.312 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:03.307+0000 7f89675d8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T04:13:03.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.422+0000 7f693ba4f640 1 Processor -- start 2024-09-19T04:13:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.422+0000 7f693ba4f640 1 -- start start 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f693ba4f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f693415a200 0x7f693415a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f693ba4f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6934072db0 0x7f6934072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f693ba4f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072880 0x7f69340b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f693ba4f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f693415aec0 con 0x7f693415a200 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f693ba4f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f693415b1a0 con 0x7f6934072db0 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f693ba4f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f693415b480 con 0x7f6934072880 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072880 0x7f69340b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f69397eb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f693415a200 0x7f693415a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072880 0x7f69340b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:57830/0 (socket says 172.21.15.76:57830) 2024-09-19T04:13:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 -- 172.21.15.76:0/3249642261 learned_addr learned my addr 172.21.15.76:0/3249642261 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:03.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6938fea640 1 --2- 172.21.15.76:0/3249642261 >> v2:172.21.15.116:3300/0 conn(0x7f6934072db0 0x7f6934072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 -- 172.21.15.76:0/3249642261 >> v2:172.21.15.116:3300/0 conn(0x7f6934072db0 msgr2=0x7f6934072340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 --2- 172.21.15.76:0/3249642261 >> v2:172.21.15.116:3300/0 conn(0x7f6934072db0 0x7f6934072340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 -- 172.21.15.76:0/3249642261 >> v2:172.21.15.76:3300/0 conn(0x7f693415a200 msgr2=0x7f693415a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 --2- 172.21.15.76:0/3249642261 >> v2:172.21.15.76:3300/0 conn(0x7f693415a200 0x7f693415a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.426+0000 7f6939fec640 1 -- 172.21.15.76:0/3249642261 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6934071e20 con 0x7f6934072880 2024-09-19T04:13:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f6938fea640 1 --2- 172.21.15.76:0/3249642261 >> v2:172.21.15.116:3300/0 conn(0x7f6934072db0 0x7f6934072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f6939fec640 1 --2- 172.21.15.76:0/3249642261 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072880 0x7f69340b2fd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69340b5510 tx=0x7f692c021190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b7ce197998bc4da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f692bfff640 1 -- 172.21.15.76:0/3249642261 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0b5070 con 0x7f6934072880 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f692bfff640 1 -- 172.21.15.76:0/3249642261 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c0b0600 con 0x7f6934072880 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f692bfff640 1 -- 172.21.15.76:0/3249642261 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0b0a10 con 0x7f6934072880 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f693ba4f640 1 -- 172.21.15.76:0/3249642261 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072880 msgr2=0x7f69340b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f693ba4f640 1 --2- 172.21.15.76:0/3249642261 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072880 0x7f69340b2fd0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69340b5510 tx=0x7f692c021190 comp rx=0 tx=0).stop 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f693ba4f640 1 -- 172.21.15.76:0/3249642261 shutdown_connections 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f693ba4f640 1 --2- 172.21.15.76:0/3249642261 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072880 0x7f69340b2fd0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f693ba4f640 1 --2- 172.21.15.76:0/3249642261 >> v2:172.21.15.116:3300/0 conn(0x7f6934072db0 0x7f6934072340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f693ba4f640 1 --2- 172.21.15.76:0/3249642261 >> v2:172.21.15.76:3300/0 conn(0x7f693415a200 0x7f693415a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.430+0000 7f693ba4f640 1 -- 172.21.15.76:0/3249642261 >> 172.21.15.76:0/3249642261 conn(0x7f69340b1cd0 msgr2=0x7f69340b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.434+0000 7f693ba4f640 1 -- 172.21.15.76:0/3249642261 shutdown_connections 2024-09-19T04:13:03.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.434+0000 7f693ba4f640 1 -- 172.21.15.76:0/3249642261 wait complete. 2024-09-19T04:13:03.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.434+0000 7f693ba4f640 1 Processor -- start 2024-09-19T04:13:03.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.434+0000 7f693ba4f640 1 -- start start 2024-09-19T04:13:03.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6934072880 0x7f69340aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072db0 0x7f69340a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6934072880 0x7f69340aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6934072880 0x7f69340aff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40800/0 (socket says 172.21.15.76:40800) 2024-09-19T04:13:03.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 -- 172.21.15.76:0/1446990905 learned_addr learned my addr 172.21.15.76:0/1446990905 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:03.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693415a200 0x7f6934150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:03.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f6938fea640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072db0 0x7f69340a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6934166030 con 0x7f6934072880 2024-09-19T04:13:03.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6934165eb0 con 0x7f693415a200 2024-09-19T04:13:03.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f69341661b0 con 0x7f6934072db0 2024-09-19T04:13:03.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f6939fec640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693415a200 0x7f6934150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 -- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072db0 msgr2=0x7f69340a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072db0 0x7f69340a9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 -- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693415a200 msgr2=0x7f6934150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693415a200 0x7f6934150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69340aaa20 con 0x7f6934072880 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f6938fea640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072db0 0x7f69340a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f69397eb640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6934072880 0x7f69340aff20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f693000e930 tx=0x7f693000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:03.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f6929ffb640 1 -- 172.21.15.76:0/1446990905 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6930017070 con 0x7f6934072880 2024-09-19T04:13:03.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69340aa180 con 0x7f6934072880 2024-09-19T04:13:03.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.438+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69340aa740 con 0x7f6934072880 2024-09-19T04:13:03.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.442+0000 7f6929ffb640 1 -- 172.21.15.76:0/1446990905 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693000ca00 con 0x7f6934072880 2024-09-19T04:13:03.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.442+0000 7f6929ffb640 1 -- 172.21.15.76:0/1446990905 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693000cdb0 con 0x7f6934072880 2024-09-19T04:13:03.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.442+0000 7f6929ffb640 1 -- 172.21.15.76:0/1446990905 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f693001a070 con 0x7f6934072880 2024-09-19T04:13:03.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.442+0000 7f6929ffb640 1 -- 172.21.15.76:0/1446990905 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f6930006780 con 0x7f6934072880 2024-09-19T04:13:03.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.442+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69140052a0 con 0x7f6934072880 2024-09-19T04:13:03.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.446+0000 7f6929ffb640 1 -- 172.21.15.76:0/1446990905 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f6930006000 con 0x7f6934072880 2024-09-19T04:13:03.480 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:03.475+0000 7f89675d8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T04:13:03.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6914005860 con 0x7f6934072880 2024-09-19T04:13:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f6929ffb640 1 -- 172.21.15.76:0/1446990905 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f69300061e0 con 0x7f6934072880 2024-09-19T04:13:03.521 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:03.521 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":10,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:03.248507+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6934072880 msgr2=0x7f69340aff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6934072880 0x7f69340aff20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f693000e930 tx=0x7f693000edf0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 shutdown_connections 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693415a200 0x7f6934150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6934072db0 0x7f69340a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 --2- 172.21.15.76:0/1446990905 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6934072880 0x7f69340aff20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 >> 172.21.15.76:0/1446990905 conn(0x7f69340b1cd0 msgr2=0x7f69340cf790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 shutdown_connections 2024-09-19T04:13:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:03.522+0000 7f693ba4f640 1 -- 172.21.15.76:0/1446990905 wait complete. 2024-09-19T04:13:03.536 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-19T04:13:03.536 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-19T04:13:03.541 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:03.535+0000 7f89675d8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T04:13:03.736 INFO:tasks.ceph.mgr.y.smithi076.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-19T04:13:03.736 INFO:tasks.ceph.mgr.y.smithi076.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-19T04:13:03.736 INFO:tasks.ceph.mgr.y.smithi076.stderr: from numpy import show_config as show_numpy_config 2024-09-19T04:13:03.817 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:03.818+0000 7fa5b8131180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:13:04.003 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:04.006+0000 7fa5b8131180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:13:04.146 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-09-19T04:13:04.143+0000 7f3b4ba387c0 -1 Falling back to public interface 2024-09-19T04:13:04.167 INFO:tasks.ceph.mgr.x.smithi116.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-19T04:13:04.167 INFO:tasks.ceph.mgr.x.smithi116.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-19T04:13:04.167 INFO:tasks.ceph.mgr.x.smithi116.stderr: from numpy import show_config as show_numpy_config 2024-09-19T04:13:04.177 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:04.178+0000 7fa5b8131180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:13:04.186 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-09-19T04:13:04.183+0000 7f1cadf817c0 -1 Falling back to public interface 2024-09-19T04:13:04.219 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:04.215+0000 7f89675d8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T04:13:04.245 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:04.246+0000 7fa5b8131180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:13:04.342 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:04.352 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:04.347+0000 7f89675d8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:13:04.383 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:04.386+0000 7fa5b8131180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:13:04.404 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-19T04:13:04.406+0000 7f3a44fea7c0 -1 Falling back to public interface 2024-09-19T04:13:04.436 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-19T04:13:04.438+0000 7f0e4f6287c0 -1 Falling back to public interface 2024-09-19T04:13:04.451 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc84e22f640 1 Processor -- start 2024-09-19T04:13:04.451 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc84e22f640 1 -- start start 2024-09-19T04:13:04.452 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc84e22f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc848070db0 0x7fc84806f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc84e22f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8481629d0 con 0x7fc848147d30 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc84e22f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc848162cb0 con 0x7fc84806c610 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc84e22f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc84806bbc0 con 0x7fc848070db0 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc847fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc848070db0 0x7fc84806f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc8477fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc848147d30 0x7fc848161f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43796/0 (socket says 172.21.15.76:43796) 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc8477fe640 1 -- 172.21.15.76:0/3798456107 learned_addr learned my addr 172.21.15.76:0/3798456107 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc847fff640 1 -- 172.21.15.76:0/3798456107 >> v1:172.21.15.116:6789/0 conn(0x7fc84806c610 legacy=0x7fc8480706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc847fff640 1 -- 172.21.15.76:0/3798456107 >> v1:172.21.15.76:6789/0 conn(0x7fc848147d30 legacy=0x7fc848161f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc8467fc640 1 -- 172.21.15.76:0/3798456107 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1007937710 0 0) 0x7fc8481629d0 con 0x7fc848147d30 2024-09-19T04:13:04.453 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.454+0000 7fc847fff640 1 -- 172.21.15.76:0/3798456107 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84806be10 con 0x7fc848070db0 2024-09-19T04:13:04.455 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc847fff640 1 --2- 172.21.15.76:0/3798456107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc84806f010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc840009d90 tx=0x7fc84001f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8d37929043bd63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:04.455 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc8467fc640 1 -- 172.21.15.76:0/3798456107 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400b4070 con 0x7fc848070db0 2024-09-19T04:13:04.455 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc8467fc640 1 -- 172.21.15.76:0/3798456107 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8400af600 con 0x7fc848070db0 2024-09-19T04:13:04.455 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc8467fc640 1 -- 172.21.15.76:0/3798456107 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8400af9d0 con 0x7fc848070db0 2024-09-19T04:13:04.456 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- 172.21.15.76:0/3798456107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 msgr2=0x7fc84806f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:04.456 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 --2- 172.21.15.76:0/3798456107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc84806f010 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc840009d90 tx=0x7fc84001f120 comp rx=0 tx=0).stop 2024-09-19T04:13:04.456 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- 172.21.15.76:0/3798456107 shutdown_connections 2024-09-19T04:13:04.456 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 --2- 172.21.15.76:0/3798456107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc84806f010 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:04.456 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- 172.21.15.76:0/3798456107 >> 172.21.15.76:0/3798456107 conn(0x7fc8480b6690 msgr2=0x7fc8480b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:04.456 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- 172.21.15.76:0/3798456107 shutdown_connections 2024-09-19T04:13:04.456 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- 172.21.15.76:0/3798456107 wait complete. 2024-09-19T04:13:04.457 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 Processor -- start 2024-09-19T04:13:04.457 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- start start 2024-09-19T04:13:04.459 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc84806c610 0x7fc848160730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:04.459 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc848161190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc848147d30 0x7fc848140c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc848162660 con 0x7fc848070db0 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc84806ca60 con 0x7fc848147d30 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.458+0000 7fc84e22f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc8480b4340 con 0x7fc84806c610 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc848161190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc848161190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40830/0 (socket says 172.21.15.76:40830) 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 -- 172.21.15.76:0/2757775638 learned_addr learned my addr 172.21.15.76:0/2757775638 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc8477fe640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc84806c610 0x7fc848160730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc847fff640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc848147d30 0x7fc848140c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 -- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc84806c610 msgr2=0x7fc848160730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc84806c610 0x7fc848160730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:04.460 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 -- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc848147d30 msgr2=0x7fc848140c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc848147d30 0x7fc848140c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 -- 172.21.15.76:0/2757775638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84809b400 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc846ffd640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc848161190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c450 tx=0x7fc83800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc82ffff640 1 -- 172.21.15.76:0/2757775638 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838012830 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc82ffff640 1 -- 172.21.15.76:0/2757775638 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc838017070 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84809b6e0 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc82ffff640 1 -- 172.21.15.76:0/2757775638 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838012c30 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84809bca0 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc82ffff640 1 -- 172.21.15.76:0/2757775638 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc83801c460 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc82ffff640 1 -- 172.21.15.76:0/2757775638 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fc83801bd10 con 0x7fc848070db0 2024-09-19T04:13:04.461 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84809bea0 con 0x7fc848070db0 2024-09-19T04:13:04.462 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.462+0000 7fc82ffff640 1 -- 172.21.15.76:0/2757775638 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc84809bea0 con 0x7fc848070db0 2024-09-19T04:13:04.487 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-09-19T04:13:04.483+0000 7f78ed4917c0 -1 Falling back to public interface 2024-09-19T04:13:04.509 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:04.503+0000 7f89675d8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T04:13:04.521 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.522+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8480731a0 con 0x7fc848070db0 2024-09-19T04:13:04.522 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc82ffff640 1 -- 172.21.15.76:0/2757775638 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fc83801c930 con 0x7fc848070db0 2024-09-19T04:13:04.523 INFO:teuthology.misc.health.smithi076.stdout: 2024-09-19T04:13:04.523 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":10,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:03.248507+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 msgr2=0x7fc848161190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc848161190 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc83800c450 tx=0x7fc83800c910 comp rx=0 tx=0).stop 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 shutdown_connections 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc848147d30 0x7fc848140c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc848070db0 0x7fc848161190 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 --2- 172.21.15.76:0/2757775638 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc84806c610 0x7fc848160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 >> 172.21.15.76:0/2757775638 conn(0x7fc8480b6690 msgr2=0x7fc848146fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 shutdown_connections 2024-09-19T04:13:04.524 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:04.526+0000 7fc84e22f640 1 -- 172.21.15.76:0/2757775638 wait complete. 2024-09-19T04:13:04.538 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-19T04:13:04.572 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:04.567+0000 7f89675d8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T04:13:04.607 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-19T04:13:04.610+0000 7fbe7912e7c0 -1 Falling back to public interface 2024-09-19T04:13:04.635 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-19T04:13:04.638+0000 7feb756f57c0 -1 Falling back to public interface 2024-09-19T04:13:04.705 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:04.699+0000 7f89675d8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T04:13:04.887 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-09-19T04:13:04.883+0000 7f77b5d797c0 -1 Falling back to public interface 2024-09-19T04:13:05.002 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:05.006+0000 7fa5b8131180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:13:05.080 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:05.082+0000 7fa5b8131180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:13:05.263 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:05.266+0000 7fa5b8131180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:13:05.307 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:05.303+0000 7f89675d8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T04:13:05.338 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:05.342+0000 7fa5b8131180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:13:05.381 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:05.375+0000 7f89675d8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T04:13:05.562 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:05.559+0000 7f89675d8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T04:13:05.635 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:05.631+0000 7f89675d8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T04:13:05.636 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:05.638+0000 7fa5b8131180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:13:05.698 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:05.702+0000 7fa5b8131180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:13:05.931 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:05.927+0000 7f89675d8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T04:13:05.993 INFO:tasks.ceph.mgr.x.smithi116.stderr:2024-09-19T04:13:05.987+0000 7f89675d8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T04:13:06.743 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:06.746+0000 7fa55ec68640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-19T04:13:06.744 INFO:tasks.ceph.mgr.y.smithi076.stderr:2024-09-19T04:13:06.746+0000 7fa55ec68640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-19T04:13:08.235 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-09-19T04:13:08.231+0000 7f3b4ba387c0 -1 osd.4 0 log_to_monitors true 2024-09-19T04:13:08.251 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-09-19T04:13:08.247+0000 7f1cadf817c0 -1 osd.5 0 log_to_monitors true 2024-09-19T04:13:08.465 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-19T04:13:08.465+0000 7f3a44fea7c0 -1 osd.0 0 log_to_monitors true 2024-09-19T04:13:08.521 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-19T04:13:08.521+0000 7f0e4f6287c0 -1 osd.1 0 log_to_monitors true 2024-09-19T04:13:08.527 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-09-19T04:13:08.523+0000 7f78ed4917c0 -1 osd.6 0 log_to_monitors true 2024-09-19T04:13:08.687 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-19T04:13:08.689+0000 7fbe7912e7c0 -1 osd.3 0 log_to_monitors true 2024-09-19T04:13:08.934 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-09-19T04:13:08.931+0000 7f77b5d797c0 -1 osd.7 0 log_to_monitors true 2024-09-19T04:13:09.202 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-19T04:13:09.205+0000 7feb756f57c0 -1 osd.2 0 log_to_monitors true 2024-09-19T04:13:11.345 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:11.501 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9aea093640 1 Processor -- start 2024-09-19T04:13:11.501 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9aea093640 1 -- start start 2024-09-19T04:13:11.502 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9aea093640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ae415c350 0x7f9ae40cfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:11.502 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9aea093640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae40cd810 con 0x7f9ae40cf110 2024-09-19T04:13:11.502 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9aea093640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ae4073f20 con 0x7f9ae40723a0 2024-09-19T04:13:11.502 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9aea093640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9ae40b4ed0 con 0x7f9ae415c350 2024-09-19T04:13:11.502 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9ae37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9ae415c350 0x7f9ae40cfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:11.502 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9ae3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9ae40cf110 0x7f9ae40cd100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43802/0 (socket says 172.21.15.76:43802) 2024-09-19T04:13:11.502 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.501+0000 7f9ae3fff640 1 -- 172.21.15.76:0/2695920499 learned_addr learned my addr 172.21.15.76:0/2695920499 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:11.503 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.505+0000 7f9ae37fe640 1 -- 172.21.15.76:0/2695920499 >> v1:172.21.15.116:6789/0 conn(0x7f9ae40723a0 legacy=0x7f9ae40cc9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:11.503 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.505+0000 7f9ae37fe640 1 -- 172.21.15.76:0/2695920499 >> v1:172.21.15.76:6789/0 conn(0x7f9ae40cf110 legacy=0x7f9ae40cd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:11.503 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.505+0000 7f9ae37fe640 1 -- 172.21.15.76:0/2695920499 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae40b51b0 con 0x7f9ae415c350 2024-09-19T04:13:11.505 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.505+0000 7f9ae37fe640 1 --2- 172.21.15.76:0/2695920499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae40cfc90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097c0 tx=0x7f9ad401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cd03cf6039633e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:11.505 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.505+0000 7f9ae27fc640 1 -- 172.21.15.76:0/2695920499 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40b0070 con 0x7f9ae415c350 2024-09-19T04:13:11.505 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.505+0000 7f9ae27fc640 1 -- 172.21.15.76:0/2695920499 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ad40ab600 con 0x7f9ae415c350 2024-09-19T04:13:11.505 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.505+0000 7f9ae27fc640 1 -- 172.21.15.76:0/2695920499 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad40ab9d0 con 0x7f9ae415c350 2024-09-19T04:13:11.506 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- 172.21.15.76:0/2695920499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 msgr2=0x7f9ae40cfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:11.507 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 --2- 172.21.15.76:0/2695920499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae40cfc90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40097c0 tx=0x7f9ad401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.507 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- 172.21.15.76:0/2695920499 shutdown_connections 2024-09-19T04:13:11.507 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 --2- 172.21.15.76:0/2695920499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae40cfc90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.507 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- 172.21.15.76:0/2695920499 >> 172.21.15.76:0/2695920499 conn(0x7f9ae40b5790 msgr2=0x7f9ae4073250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:11.507 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- 172.21.15.76:0/2695920499 shutdown_connections 2024-09-19T04:13:11.507 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- 172.21.15.76:0/2695920499 wait complete. 2024-09-19T04:13:11.508 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 Processor -- start 2024-09-19T04:13:11.508 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- start start 2024-09-19T04:13:11.509 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ae40723a0 0x7f9ae406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ae40cf110 0x7f9ae4158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae414b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae416a730 con 0x7f9ae415c350 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae40d05e0 con 0x7f9ae40cf110 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9aea093640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae416a8b0 con 0x7f9ae40723a0 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9ae37fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ae40723a0 0x7f9ae406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9ae2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ae40cf110 0x7f9ae4158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9ae2ffd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ae40cf110 0x7f9ae4158500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:40868/0 (socket says 172.21.15.76:40868) 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9ae2ffd640 1 -- 172.21.15.76:0/4182076687 learned_addr learned my addr 172.21.15.76:0/4182076687 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:11.510 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9ae3fff640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae414b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:11.511 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9ae3fff640 1 -- 172.21.15.76:0/4182076687 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ae40723a0 msgr2=0x7f9ae406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:11.511 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.509+0000 7f9ae3fff640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ae40723a0 0x7f9ae406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.511 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9ae3fff640 1 -- 172.21.15.76:0/4182076687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ae40cf110 msgr2=0x7f9ae4158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:11.511 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9ae3fff640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ae40cf110 0x7f9ae4158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.511 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9ae3fff640 1 -- 172.21.15.76:0/4182076687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae4151ca0 con 0x7f9ae415c350 2024-09-19T04:13:11.511 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9ae2ffd640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ae40cf110 0x7f9ae4158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:11.511 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9ae3fff640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae414b100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00caa0 tx=0x7f9adc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:11.513 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9acbfff640 1 -- 172.21.15.76:0/4182076687 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc017070 con 0x7f9ae415c350 2024-09-19T04:13:11.513 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9acbfff640 1 -- 172.21.15.76:0/4182076687 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc00ac00 con 0x7f9ae415c350 2024-09-19T04:13:11.513 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9acbfff640 1 -- 172.21.15.76:0/4182076687 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc012400 con 0x7f9ae415c350 2024-09-19T04:13:11.514 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9aea093640 1 -- 172.21.15.76:0/4182076687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae4151f80 con 0x7f9ae415c350 2024-09-19T04:13:11.514 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.513+0000 7f9aea093640 1 -- 172.21.15.76:0/4182076687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae4152540 con 0x7f9ae415c350 2024-09-19T04:13:11.516 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.517+0000 7f9acbfff640 1 -- 172.21.15.76:0/4182076687 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 4) ==== 87389+0+0 (secure 0 0 0) 0x7f9adc01a070 con 0x7f9ae415c350 2024-09-19T04:13:11.516 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.517+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae40b2a90 con 0x7f9ae415c350 2024-09-19T04:13:11.517 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.517+0000 7f9acbfff640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad0067880 0x7f9ad0069d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:11.517 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.517+0000 7f9acbfff640 1 -- 172.21.15.76:0/4182076687 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f9adc08a060 con 0x7f9ae415c350 2024-09-19T04:13:11.522 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.521+0000 7f9ae37fe640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad0067880 0x7f9ad0069d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:11.523 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.525+0000 7f9ae37fe640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad0067880 0x7f9ad0069d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ad000 tx=0x7f9ad40a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:11.538 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.541+0000 7f9acbfff640 1 -- 172.21.15.76:0/4182076687 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9adc055b20 con 0x7f9ae415c350 2024-09-19T04:13:11.657 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-09-19T04:13:11.651+0000 7f3b479e1640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T04:13:11.686 INFO:tasks.ceph.osd.4.smithi116.stderr:2024-09-19T04:13:11.683+0000 7f3b3f816640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:11.689 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-09-19T04:13:11.683+0000 7f1ca9f2a640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T04:13:11.718 INFO:tasks.ceph.osd.5.smithi116.stderr:2024-09-19T04:13:11.715+0000 7f1ca1d5f640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:11.771 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-09-19T04:13:11.767+0000 7f78e943a640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T04:13:11.806 INFO:tasks.ceph.osd.6.smithi116.stderr:2024-09-19T04:13:11.803+0000 7f78e126f640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:11.806 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-19T04:13:11.805+0000 7f3a40f93640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T04:13:11.813 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-19T04:13:11.813+0000 7fbe750d7640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T04:13:11.835 INFO:tasks.ceph.osd.0.smithi076.stderr:2024-09-19T04:13:11.837+0000 7f3a38dc8640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:11.836 INFO:tasks.ceph.osd.3.smithi076.stderr:2024-09-19T04:13:11.837+0000 7fbe6cf0c640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:11.842 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.841+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9ae40735d0 con 0x7f9ae415c350 2024-09-19T04:13:11.842 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9acbfff640 1 -- 172.21.15.76:0/4182076687 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f9adc059e20 con 0x7f9ae415c350 2024-09-19T04:13:11.843 INFO:teuthology.misc.health.smithi076.stdout: 2024-09-19T04:13:11.843 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":13,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:10.756707+0000","last_up_change":"0.000000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad0067880 msgr2=0x7f9ad0069d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad0067880 0x7f9ad0069d40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9ad40ad000 tx=0x7f9ad40a6000 comp rx=0 tx=0).stop 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 msgr2=0x7f9ae414b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae414b100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00caa0 tx=0x7f9adc00cf60 comp rx=0 tx=0).stop 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 shutdown_connections 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad0067880 0x7f9ad0069d40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ae415c350 0x7f9ae414b100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.846 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ae40cf110 0x7f9ae4158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.847 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 --2- 172.21.15.76:0/4182076687 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ae40723a0 0x7f9ae406efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:11.847 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 >> 172.21.15.76:0/4182076687 conn(0x7f9ae40b5790 msgr2=0x7f9ae414c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:11.847 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 shutdown_connections 2024-09-19T04:13:11.847 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:11.845+0000 7f9ac9ffb640 1 -- 172.21.15.76:0/4182076687 wait complete. 2024-09-19T04:13:11.863 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-19T04:13:11.890 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-19T04:13:11.889+0000 7f0e4b5d1640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T04:13:11.923 INFO:tasks.ceph.osd.1.smithi076.stderr:2024-09-19T04:13:11.925+0000 7f0e43406640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:11.980 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-09-19T04:13:11.975+0000 7f77b1d22640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T04:13:12.010 INFO:tasks.ceph.osd.7.smithi116.stderr:2024-09-19T04:13:12.007+0000 7f77a9b57640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:12.065 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-19T04:13:12.065+0000 7feb71eb0640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T04:13:12.091 INFO:tasks.ceph.osd.2.smithi076.stderr:2024-09-19T04:13:12.093+0000 7feb694d3640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T04:13:18.670 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:18.795 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 Processor -- start 2024-09-19T04:13:18.796 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 -- start start 2024-09-19T04:13:18.797 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 0x7fb1640ad030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.797 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb16415c3f0 0x7fb1640b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.797 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb1640b4770 0x7fb16415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.797 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb1640cf9a0 con 0x7fb16415c3f0 2024-09-19T04:13:18.797 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb164074070 con 0x7fb1640b4770 2024-09-19T04:13:18.797 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb168b63640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb1640ced90 con 0x7fb16415c010 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb16415c3f0 0x7fb1640b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb16415c3f0 0x7fb1640b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44262/0 (socket says 172.21.15.76:44262) 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 -- 172.21.15.76:0/1135703642 learned_addr learned my addr 172.21.15.76:0/1135703642 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb16259c640 1 --2- 172.21.15.76:0/1135703642 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 0x7fb1640ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb162d9d640 1 --2- 172.21.15.76:0/1135703642 >> v2:172.21.15.116:3300/0 conn(0x7fb1640b4770 0x7fb16415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 -- 172.21.15.76:0/1135703642 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 msgr2=0x7fb1640ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 --2- 172.21.15.76:0/1135703642 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 0x7fb1640ad030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 -- 172.21.15.76:0/1135703642 >> v2:172.21.15.116:3300/0 conn(0x7fb1640b4770 msgr2=0x7fb16415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 --2- 172.21.15.76:0/1135703642 >> v2:172.21.15.116:3300/0 conn(0x7fb1640b4770 0x7fb16415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb161d9b640 1 -- 172.21.15.76:0/1135703642 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1640cf070 con 0x7fb16415c3f0 2024-09-19T04:13:18.798 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.796+0000 7fb162d9d640 1 --2- 172.21.15.76:0/1135703642 >> v2:172.21.15.116:3300/0 conn(0x7fb1640b4770 0x7fb16415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:18.800 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb161d9b640 1 --2- 172.21.15.76:0/1135703642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16415c3f0 0x7fb1640b4230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1640d10a0 tx=0x7fb15801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d98d430e2dd01c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:18.800 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb16159a640 1 -- 172.21.15.76:0/1135703642 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580ab5f0 con 0x7fb16415c3f0 2024-09-19T04:13:18.800 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb16159a640 1 -- 172.21.15.76:0/1135703642 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1580b0070 con 0x7fb16415c3f0 2024-09-19T04:13:18.800 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb16159a640 1 -- 172.21.15.76:0/1135703642 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1580ab9b0 con 0x7fb16415c3f0 2024-09-19T04:13:18.801 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- 172.21.15.76:0/1135703642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16415c3f0 msgr2=0x7fb1640b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.801 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 --2- 172.21.15.76:0/1135703642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16415c3f0 0x7fb1640b4230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1640d10a0 tx=0x7fb15801c520 comp rx=0 tx=0).stop 2024-09-19T04:13:18.801 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- 172.21.15.76:0/1135703642 shutdown_connections 2024-09-19T04:13:18.802 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 --2- 172.21.15.76:0/1135703642 >> v2:172.21.15.116:3300/0 conn(0x7fb1640b4770 0x7fb16415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.802 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 --2- 172.21.15.76:0/1135703642 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb16415c3f0 0x7fb1640b4230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.802 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 --2- 172.21.15.76:0/1135703642 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 0x7fb1640ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.802 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- 172.21.15.76:0/1135703642 >> 172.21.15.76:0/1135703642 conn(0x7fb16406bbc0 msgr2=0x7fb16406bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:18.802 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- 172.21.15.76:0/1135703642 shutdown_connections 2024-09-19T04:13:18.802 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- 172.21.15.76:0/1135703642 wait complete. 2024-09-19T04:13:18.807 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 Processor -- start 2024-09-19T04:13:18.807 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- start start 2024-09-19T04:13:18.807 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1640b4770 0x7fb16415d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.807 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 0x7fb16415f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.807 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb16415c3f0 0x7fb16415f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.808 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640d09b0 con 0x7fb1640b4770 2024-09-19T04:13:18.808 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1640d0830 con 0x7fb16415c3f0 2024-09-19T04:13:18.808 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.800+0000 7fb168b63640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb1641606c0 con 0x7fb16415c010 2024-09-19T04:13:18.808 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1640b4770 0x7fb16415d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.808 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1640b4770 0x7fb16415d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44270/0 (socket says 172.21.15.76:44270) 2024-09-19T04:13:18.809 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 -- 172.21.15.76:0/877328493 learned_addr learned my addr 172.21.15.76:0/877328493 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:18.809 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 -- 172.21.15.76:0/877328493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 msgr2=0x7fb16415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.809 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 0x7fb16415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.809 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 -- 172.21.15.76:0/877328493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb16415c3f0 msgr2=0x7fb16415f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.809 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb162d9d640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb16415c3f0 0x7fb16415f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.809 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb16415c3f0 0x7fb16415f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.810 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 -- 172.21.15.76:0/877328493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb164154210 con 0x7fb1640b4770 2024-09-19T04:13:18.810 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb16259c640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1640b4770 0x7fb16415d6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d830 tx=0x7fb15400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:18.810 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb152ffd640 1 -- 172.21.15.76:0/877328493 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15400b6c0 con 0x7fb1640b4770 2024-09-19T04:13:18.810 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb152ffd640 1 -- 172.21.15.76:0/877328493 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb154019070 con 0x7fb1640b4770 2024-09-19T04:13:18.810 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16415e120 con 0x7fb1640b4770 2024-09-19T04:13:18.810 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb152ffd640 1 -- 172.21.15.76:0/877328493 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15400bae0 con 0x7fb1640b4770 2024-09-19T04:13:18.811 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16415c990 con 0x7fb1640b4770 2024-09-19T04:13:18.811 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1640ad030 con 0x7fb1640b4770 2024-09-19T04:13:18.811 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb152ffd640 1 -- 172.21.15.76:0/877328493 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb15401e460 con 0x7fb1640b4770 2024-09-19T04:13:18.811 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb152ffd640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1440678c0 0x7fb144069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:18.811 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb161d9b640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1440678c0 0x7fb144069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:18.812 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb152ffd640 1 -- 172.21.15.76:0/877328493 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fb1540860d0 con 0x7fb1640b4770 2024-09-19T04:13:18.812 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.804+0000 7fb161d9b640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1440678c0 0x7fb144069d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1580af000 tx=0x7fb158004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:18.812 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.812+0000 7fb152ffd640 1 -- 172.21.15.76:0/877328493 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb154056ed0 con 0x7fb1640b4770 2024-09-19T04:13:18.975 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.972+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb1640ce660 con 0x7fb1640b4770 2024-09-19T04:13:18.975 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb152ffd640 1 -- 172.21.15.76:0/877328493 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12920 (secure 0 0 0) 0x7fb1540567d0 con 0x7fb1640b4770 2024-09-19T04:13:18.976 INFO:teuthology.misc.health.smithi076.stdout: 2024-09-19T04:13:18.976 INFO:teuthology.misc.health.smithi076.stdout:{"epoch":17,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:15.841203+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1440678c0 msgr2=0x7fb144069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1440678c0 0x7fb144069d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb1580af000 tx=0x7fb158004740 comp rx=0 tx=0).stop 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1640b4770 msgr2=0x7fb16415d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1640b4770 0x7fb16415d6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb15400d830 tx=0x7fb15400dcf0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 shutdown_connections 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1440678c0 0x7fb144069d80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb16415c3f0 0x7fb16415f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.978 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb16415c010 0x7fb16415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.979 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 --2- 172.21.15.76:0/877328493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1640b4770 0x7fb16415d6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:18.979 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 >> 172.21.15.76:0/877328493 conn(0x7fb16406bbc0 msgr2=0x7fb16406f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:18.979 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 shutdown_connections 2024-09-19T04:13:18.979 INFO:teuthology.misc.health.smithi076.stderr:2024-09-19T04:13:18.976+0000 7fb168b63640 1 -- 172.21.15.76:0/877328493 wait complete. 2024-09-19T04:13:18.993 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-19T04:13:18.993 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-19T04:13:18.993 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T04:13:18.994 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T04:13:18.996 INFO:tasks.ceph.mds.a:Started 2024-09-19T04:13:18.996 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T04:13:18.996 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T04:13:19.002 INFO:tasks.ceph.mds.c:Started 2024-09-19T04:13:19.002 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T04:13:19.002 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T04:13:19.005 INFO:tasks.ceph.mds.b:Started 2024-09-19T04:13:19.005 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T04:13:19.005 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T04:13:19.007 INFO:tasks.ceph.mds.d:Started 2024-09-19T04:13:19.007 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-19T04:13:19.008 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:19.066 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:13:19.066 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:13:19.092 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:13:19.096 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:13:19.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.112+0000 7f595e7ed640 1 Processor -- start 2024-09-19T04:13:19.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.112+0000 7f595e7ed640 1 -- start start 2024-09-19T04:13:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595e7ed640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 0x7f59580d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595e7ed640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59580d32b0 con 0x7f59580d2e90 2024-09-19T04:13:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595e7ed640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59580cfad0 con 0x7f59580cf620 2024-09-19T04:13:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595e7ed640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f59580d0300 con 0x7f5958161a60 2024-09-19T04:13:19.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f59577fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f59580d2e90 0x7f59580cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47568/0 (socket says 172.21.15.76:47568) 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f59577fe640 1 -- 172.21.15.76:0/1876424790 learned_addr learned my addr 172.21.15.76:0/1876424790 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2799942410 0 0) 0x7f59580d32b0 con 0x7f59580d2e90 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5940003530 con 0x7f59580d2e90 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5957fff640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 0x7f59580d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 542276821 0 0) 0x7f5940003530 con 0x7f59580d2e90 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 msgr2=0x7f59580d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 0x7f59580d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 >> v1:172.21.15.116:6789/0 conn(0x7f59580cf620 legacy=0x7f595806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:19.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580b0f60 con 0x7f59580d2e90 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f594c001680 con 0x7f59580d2e90 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 >> v1:172.21.15.76:6789/0 conn(0x7f59580d2e90 legacy=0x7f59580cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 0x7f59400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004070 0x7f5940004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5940004d50 0x7f59400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59580d0300 con 0x7f5940004d50 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f594000bcb0 con 0x7f5940004070 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f594000beb0 con 0x7f5958161a60 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595cd8a640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5940004d50 0x7f59400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5957fff640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 0x7f59400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f59577fe640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004070 0x7f5940004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595cd8a640 1 -- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 msgr2=0x7f59400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:19.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595cd8a640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5958161a60 0x7f59400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595cd8a640 1 -- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004070 msgr2=0x7f5940004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595cd8a640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004070 0x7f5940004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595cd8a640 1 -- 172.21.15.76:0/1876424790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5940003530 con 0x7f5940004d50 2024-09-19T04:13:19.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f595cd8a640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5940004d50 0x7f59400051d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5950001e70 tx=0x7f595001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40d2cfc29d7b0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950001a90 con 0x7f5940004d50 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5950003b80 con 0x7f5940004d50 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.116+0000 7f5956ffd640 1 -- 172.21.15.76:0/1876424790 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950005080 con 0x7f5940004d50 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/1876424790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5940004d50 msgr2=0x7f59400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 --2- 172.21.15.76:0/1876424790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5940004d50 0x7f59400051d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5950001e70 tx=0x7f595001c780 comp rx=0 tx=0).stop 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595cd8a640 1 -- 172.21.15.76:0/1876424790 reap_dead start 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/1876424790 shutdown_connections 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/1876424790 >> 172.21.15.76:0/1876424790 conn(0x7f595808bf90 msgr2=0x7f595809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/1876424790 shutdown_connections 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/1876424790 wait complete. 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 Processor -- start 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- start start 2024-09-19T04:13:19.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004d50 0x7f595815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59580b9d60 0x7f59580cc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f595815e0a0 0x7f59580b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f595806ceb0 con 0x7f59580b9d60 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59580d0020 con 0x7f5940004d50 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f595806d190 con 0x7f595815e0a0 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59580b9d60 0x7f59580cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59580b9d60 0x7f59580cc870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44296/0 (socket says 172.21.15.76:44296) 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 -- 172.21.15.76:0/3722577669 learned_addr learned my addr 172.21.15.76:0/3722577669 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f5957fff640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004d50 0x7f595815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595cd8a640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f595815e0a0 0x7f59580b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 -- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f595815e0a0 msgr2=0x7f59580b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f595815e0a0 0x7f59580b87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 -- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004d50 msgr2=0x7f595815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004d50 0x7f595815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 -- 172.21.15.76:0/3722577669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595815d940 con 0x7f59580b9d60 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f59577fe640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59580b9d60 0x7f59580cc870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f594c0124a0 tx=0x7f594c012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f5954ff9640 1 -- 172.21.15.76:0/3722577669 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c014810 con 0x7f59580b9d60 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f5954ff9640 1 -- 172.21.15.76:0/3722577669 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f594c01b070 con 0x7f59580b9d60 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f5954ff9640 1 -- 172.21.15.76:0/3722577669 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c014bb0 con 0x7f59580b9d60 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59580cda10 con 0x7f59580b9d60 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59580cd450 con 0x7f59580b9d60 2024-09-19T04:13:19.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.120+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59580d1630 con 0x7f59580b9d60 2024-09-19T04:13:19.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.124+0000 7f5954ff9640 1 -- 172.21.15.76:0/3722577669 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f594c016480 con 0x7f59580b9d60 2024-09-19T04:13:19.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.124+0000 7f5954ff9640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59440678c0 0x7f5944069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:19.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.124+0000 7f5954ff9640 1 -- 172.21.15.76:0/3722577669 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f594c088b70 con 0x7f59580b9d60 2024-09-19T04:13:19.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.124+0000 7f5957fff640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59440678c0 0x7f5944069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:19.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.124+0000 7f5957fff640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59440678c0 0x7f5944069d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5948003730 tx=0x7f5948008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:19.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.128+0000 7f5954ff9640 1 -- 172.21.15.76:0/3722577669 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f594c059440 con 0x7f59580b9d60 2024-09-19T04:13:19.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.324+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f595806bcc0 con 0x7f59580b9d60 2024-09-19T04:13:19.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.324+0000 7f5954ff9640 1 -- 172.21.15.76:0/3722577669 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f594c05d740 con 0x7f59580b9d60 2024-09-19T04:13:19.326 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:19.326 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":2,"btime":"2024-09-19T04:13:19:073853+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:13:19.326 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 2 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59440678c0 msgr2=0x7f5944069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59440678c0 0x7f5944069d80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5948003730 tx=0x7f5948008040 comp rx=0 tx=0).stop 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59580b9d60 msgr2=0x7f59580cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59580b9d60 0x7f59580cc870 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f594c0124a0 tx=0x7f594c012960 comp rx=0 tx=0).stop 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 shutdown_connections 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59440678c0 0x7f5944069d80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f595815e0a0 0x7f59580b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59580b9d60 0x7f59580cc870 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 --2- 172.21.15.76:0/3722577669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5940004d50 0x7f595815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 >> 172.21.15.76:0/3722577669 conn(0x7f595808bf90 msgr2=0x7f595806d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 shutdown_connections 2024-09-19T04:13:19.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:19.328+0000 7f595e7ed640 1 -- 172.21.15.76:0/3722577669 wait complete. 2024-09-19T04:13:21.345 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:21.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd0905c3640 1 Processor -- start 2024-09-19T04:13:21.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd0905c3640 1 -- start start 2024-09-19T04:13:21.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd0905c3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0881587a0 0x7fd0880b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd0905c3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0880b37d0 0x7fd08806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd0905c3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd08806ec60 con 0x7fd08806e4b0 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd0905c3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd0880b3c40 con 0x7fd0881587a0 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd0905c3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd08806f870 con 0x7fd0880b37d0 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08e35f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0881587a0 0x7fd0880b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08eb60640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd08806e4b0 0x7fd088161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47574/0 (socket says 172.21.15.76:47574) 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08eb60640 1 -- 172.21.15.76:0/2084616308 learned_addr learned my addr 172.21.15.76:0/2084616308 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08db5e640 1 --2- 172.21.15.76:0/2084616308 >> v2:172.21.15.116:3300/0 conn(0x7fd0880b37d0 0x7fd08806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08e35f640 1 -- 172.21.15.76:0/2084616308 >> v2:172.21.15.116:3300/0 conn(0x7fd0880b37d0 msgr2=0x7fd08806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08e35f640 1 --2- 172.21.15.76:0/2084616308 >> v2:172.21.15.116:3300/0 conn(0x7fd0880b37d0 0x7fd08806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08e35f640 1 -- 172.21.15.76:0/2084616308 >> v1:172.21.15.76:6789/0 conn(0x7fd08806e4b0 legacy=0x7fd088161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.476+0000 7fd08e35f640 1 -- 172.21.15.76:0/2084616308 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08806ef40 con 0x7fd0881587a0 2024-09-19T04:13:21.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 --2- 172.21.15.76:0/2084616308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0881587a0 0x7fd0880b3290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd088161d90 tx=0x7fd07c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c02048f40ec355e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08d35d640 1 -- 172.21.15.76:0/2084616308 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b0070 con 0x7fd0881587a0 2024-09-19T04:13:21.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08d35d640 1 -- 172.21.15.76:0/2084616308 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c0ab600 con 0x7fd0881587a0 2024-09-19T04:13:21.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08d35d640 1 -- 172.21.15.76:0/2084616308 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0ab9d0 con 0x7fd0881587a0 2024-09-19T04:13:21.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- 172.21.15.76:0/2084616308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0881587a0 msgr2=0x7fd0880b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 --2- 172.21.15.76:0/2084616308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0881587a0 0x7fd0880b3290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd088161d90 tx=0x7fd07c01c450 comp rx=0 tx=0).stop 2024-09-19T04:13:21.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- 172.21.15.76:0/2084616308 shutdown_connections 2024-09-19T04:13:21.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 --2- 172.21.15.76:0/2084616308 >> v2:172.21.15.116:3300/0 conn(0x7fd0880b37d0 0x7fd08806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 --2- 172.21.15.76:0/2084616308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0881587a0 0x7fd0880b3290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- 172.21.15.76:0/2084616308 >> 172.21.15.76:0/2084616308 conn(0x7fd0880b1f90 msgr2=0x7fd0880b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:21.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- 172.21.15.76:0/2084616308 shutdown_connections 2024-09-19T04:13:21.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- 172.21.15.76:0/2084616308 wait complete. 2024-09-19T04:13:21.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 Processor -- start 2024-09-19T04:13:21.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- start start 2024-09-19T04:13:21.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08806e4b0 0x7fd088137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0880b37d0 0x7fd088147420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0881587a0 0x7fd088153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd088158b80 con 0x7fd08806e4b0 2024-09-19T04:13:21.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd088073170 con 0x7fd0880b37d0 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd088153fb0 con 0x7fd0881587a0 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08806e4b0 0x7fd088137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08806e4b0 0x7fd088137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44312/0 (socket says 172.21.15.76:44312) 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 -- 172.21.15.76:0/3753161647 learned_addr learned my addr 172.21.15.76:0/3753161647 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08eb60640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0881587a0 0x7fd088153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08db5e640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0880b37d0 0x7fd088147420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 -- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0881587a0 msgr2=0x7fd088153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0881587a0 0x7fd088153a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 -- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0880b37d0 msgr2=0x7fd088147420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0880b37d0 0x7fd088147420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 -- 172.21.15.76:0/3753161647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0881382b0 con 0x7fd08806e4b0 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd08e35f640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08806e4b0 0x7fd088137cd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0ae040 tx=0x7fd07c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd07affd640 1 -- 172.21.15.76:0/3753161647 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b0040 con 0x7fd08806e4b0 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd07affd640 1 -- 172.21.15.76:0/3753161647 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c0ba070 con 0x7fd08806e4b0 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd088138590 con 0x7fd08806e4b0 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.480+0000 7fd07affd640 1 -- 172.21.15.76:0/3753161647 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0b4630 con 0x7fd08806e4b0 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.484+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08814c400 con 0x7fd08806e4b0 2024-09-19T04:13:21.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.484+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08814c1f0 con 0x7fd08806e4b0 2024-09-19T04:13:21.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.484+0000 7fd07affd640 1 -- 172.21.15.76:0/3753161647 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd07c0b47d0 con 0x7fd08806e4b0 2024-09-19T04:13:21.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.484+0000 7fd07affd640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd06c0678c0 0x7fd06c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.484+0000 7fd07affd640 1 -- 172.21.15.76:0/3753161647 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fd07c11f680 con 0x7fd08806e4b0 2024-09-19T04:13:21.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.484+0000 7fd08db5e640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd06c0678c0 0x7fd06c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.484+0000 7fd08db5e640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd06c0678c0 0x7fd06c069d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd088147e10 tx=0x7fd084008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.488+0000 7fd07affd640 1 -- 172.21.15.76:0/3753161647 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd07c016070 con 0x7fd08806e4b0 2024-09-19T04:13:21.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.684+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd088070080 con 0x7fd08806e4b0 2024-09-19T04:13:21.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.684+0000 7fd07affd640 1 -- 172.21.15.76:0/3753161647 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3906 (secure 0 0 0) 0x7fd07c0f0470 con 0x7fd08806e4b0 2024-09-19T04:13:21.687 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:21.687 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":3,"btime":"2024-09-19T04:13:20:075150+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T04:13:21.687 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 3 2024-09-19T04:13:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd06c0678c0 msgr2=0x7fd06c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd06c0678c0 0x7fd06c069d80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd088147e10 tx=0x7fd084008040 comp rx=0 tx=0).stop 2024-09-19T04:13:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08806e4b0 msgr2=0x7fd088137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08806e4b0 0x7fd088137cd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd07c0ae040 tx=0x7fd07c0191f0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 shutdown_connections 2024-09-19T04:13:21.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd06c0678c0 0x7fd06c069d80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0881587a0 0x7fd088153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0880b37d0 0x7fd088147420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 --2- 172.21.15.76:0/3753161647 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08806e4b0 0x7fd088137cd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 >> 172.21.15.76:0/3753161647 conn(0x7fd0880b1f90 msgr2=0x7fd08815e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 shutdown_connections 2024-09-19T04:13:21.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.688+0000 7fd0905c3640 1 -- 172.21.15.76:0/3753161647 wait complete. 2024-09-19T04:13:21.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T04:13:21.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.824+0000 7ff87d6c2640 1 Processor -- start 2024-09-19T04:13:21.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.824+0000 7ff87d6c2640 1 -- start start 2024-09-19T04:13:21.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff87d6c2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff8781520a0 0x7ff87806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff87d6c2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 0x7ff87815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff87d6c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff878070250 con 0x7ff87815f1d0 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff87d6c2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff878152480 con 0x7ff8781520a0 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff87d6c2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff8780aede0 con 0x7ff87806fe50 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff87815f1d0 0x7ff878071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47578/0 (socket says 172.21.15.76:47578) 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 -- 172.21.15.76:0/2328991915 learned_addr learned my addr 172.21.15.76:0/2328991915 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff876ffd640 1 --2- 172.21.15.76:0/2328991915 >> v2:172.21.15.116:3300/0 conn(0x7ff8781520a0 0x7ff87806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8767fc640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 0x7ff87815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4118817066 0 0) 0x7ff878070250 con 0x7ff87815f1d0 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff864003530 con 0x7ff87815f1d0 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2936995622 0 0) 0x7ff864003530 con 0x7ff87815f1d0 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 msgr2=0x7ff87815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 0x7ff87815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 >> v2:172.21.15.116:3300/0 conn(0x7ff8781520a0 msgr2=0x7ff87806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 --2- 172.21.15.76:0/2328991915 >> v2:172.21.15.116:3300/0 conn(0x7ff8781520a0 0x7ff87806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87815fa10 con 0x7ff87815f1d0 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7ff870001680 con 0x7ff87815f1d0 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 >> v1:172.21.15.76:6789/0 conn(0x7ff87815f1d0 legacy=0x7ff878071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8640039c0 0x7ff864003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 0x7ff864004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff864004d50 0x7ff8640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff8780aede0 con 0x7ff864004d50 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff86400b9d0 con 0x7ff8640039c0 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff86400bb50 con 0x7ff87806fe50 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff864004d50 0x7ff8640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8767fc640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 0x7ff864004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 -- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 msgr2=0x7ff864004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:21.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff87806fe50 0x7ff864004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 -- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8640039c0 msgr2=0x7ff864003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:21.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8640039c0 0x7ff864003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.828+0000 7ff8777fe640 1 -- 172.21.15.76:0/2328991915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff864003530 con 0x7ff864004d50 2024-09-19T04:13:21.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff8777fe640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff864004d50 0x7ff8640051d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c240 tx=0x7ff87001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab704877838cd616 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870007d40 con 0x7ff864004d50 2024-09-19T04:13:21.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8700a9020 con 0x7ff864004d50 2024-09-19T04:13:21.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff875ffb640 1 -- 172.21.15.76:0/2328991915 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8700a5be0 con 0x7ff864004d50 2024-09-19T04:13:21.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- 172.21.15.76:0/2328991915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff864004d50 msgr2=0x7ff8640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 --2- 172.21.15.76:0/2328991915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff864004d50 0x7ff8640051d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7ff87000c240 tx=0x7ff87001cca0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff8777fe640 1 -- 172.21.15.76:0/2328991915 reap_dead start 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- 172.21.15.76:0/2328991915 shutdown_connections 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- 172.21.15.76:0/2328991915 >> 172.21.15.76:0/2328991915 conn(0x7ff8780b1f90 msgr2=0x7ff8780b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- 172.21.15.76:0/2328991915 shutdown_connections 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- 172.21.15.76:0/2328991915 wait complete. 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 Processor -- start 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- start start 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87815b5f0 0x7ff878152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff878153500 0x7ff87815b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814a340 0x7ff87815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff878073770 con 0x7ff87815b5f0 2024-09-19T04:13:21.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff878073490 con 0x7ff87814a340 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff878073a50 con 0x7ff878153500 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87815b5f0 0x7ff878152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87815b5f0 0x7ff878152fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44326/0 (socket says 172.21.15.76:44326) 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 -- 172.21.15.76:0/72939683 learned_addr learned my addr 172.21.15.76:0/72939683 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff8777fe640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814a340 0x7ff87815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff8767fc640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff878153500 0x7ff87815b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 -- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff878153500 msgr2=0x7ff87815b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff878153500 0x7ff87815b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 -- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814a340 msgr2=0x7ff87815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814a340 0x7ff87815bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 -- 172.21.15.76:0/72939683 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8781539c0 con 0x7ff87815b5f0 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff8777fe640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814a340 0x7ff87815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:21.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff876ffd640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87815b5f0 0x7ff878152fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff860009fd0 tx=0x7ff86000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff86f7fe640 1 -- 172.21.15.76:0/72939683 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff860017070 con 0x7ff87815b5f0 2024-09-19T04:13:21.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff86f7fe640 1 -- 172.21.15.76:0/72939683 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86000c940 con 0x7ff87815b5f0 2024-09-19T04:13:21.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff878153ca0 con 0x7ff87815b5f0 2024-09-19T04:13:21.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff86f7fe640 1 -- 172.21.15.76:0/72939683 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86000eea0 con 0x7ff87815b5f0 2024-09-19T04:13:21.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.832+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff878154260 con 0x7ff87815b5f0 2024-09-19T04:13:21.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.836+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff878073d30 con 0x7ff87815b5f0 2024-09-19T04:13:21.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.836+0000 7ff86f7fe640 1 -- 172.21.15.76:0/72939683 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff86001a070 con 0x7ff87815b5f0 2024-09-19T04:13:21.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.836+0000 7ff86f7fe640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8580678c0 0x7ff858069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:21.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.836+0000 7ff86f7fe640 1 -- 172.21.15.76:0/72939683 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff860085be0 con 0x7ff87815b5f0 2024-09-19T04:13:21.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.836+0000 7ff8767fc640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8580678c0 0x7ff858069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:21.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.836+0000 7ff8767fc640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8580678c0 0x7ff858069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff878152db0 tx=0x7ff86800d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:21.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:21.840+0000 7ff86f7fe640 1 -- 172.21.15.76:0/72939683 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff860056a70 con 0x7ff87815b5f0 2024-09-19T04:13:22.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.004+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff87806e540 con 0x7ff87815b5f0 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.004+0000 7ff86f7fe640 1 -- 172.21.15.76:0/72939683 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7ff860056370 con 0x7ff87815b5f0 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:13:22.005 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:13:22.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8580678c0 msgr2=0x7ff858069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8580678c0 0x7ff858069d80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff878152db0 tx=0x7ff86800d040 comp rx=0 tx=0).stop 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87815b5f0 msgr2=0x7ff878152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87815b5f0 0x7ff878152fc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff860009fd0 tx=0x7ff86000ec50 comp rx=0 tx=0).stop 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 shutdown_connections 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8580678c0 0x7ff858069d80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814a340 0x7ff87815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff878153500 0x7ff87815b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 --2- 172.21.15.76:0/72939683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff87815b5f0 0x7ff878152fc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 >> 172.21.15.76:0/72939683 conn(0x7ff8780b1f90 msgr2=0x7ff87806efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 shutdown_connections 2024-09-19T04:13:22.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.008+0000 7ff87d6c2640 1 -- 172.21.15.76:0/72939683 wait complete. 2024-09-19T04:13:22.024 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:13:22.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:13:22.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c968aa640 1 Processor -- start 2024-09-19T04:13:22.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c968aa640 1 -- start start 2024-09-19T04:13:22.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c968aa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c90073e60 0x7f5c9006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:22.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c968aa640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c900b3d50 0x7f5c900b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:22.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c968aa640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c900b4670 con 0x7f5c9006fef0 2024-09-19T04:13:22.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c968aa640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9006ee50 con 0x7f5c90073e60 2024-09-19T04:13:22.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c968aa640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5c9006f130 con 0x7f5c900b3d50 2024-09-19T04:13:22.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c8ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c90073e60 0x7f5c9006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:22.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c8ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5c90073e60 0x7f5c9006e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44334/0 (socket says 172.21.15.76:44334) 2024-09-19T04:13:22.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c8ffff640 1 -- 172.21.15.76:0/2981049111 learned_addr learned my addr 172.21.15.76:0/2981049111 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:22.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c94e47640 1 --2- 172.21.15.76:0/2981049111 >> v2:172.21.15.116:3300/0 conn(0x7f5c900b3d50 0x7f5c900b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:22.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c8ffff640 1 -- 172.21.15.76:0/2981049111 >> v2:172.21.15.116:3300/0 conn(0x7f5c900b3d50 msgr2=0x7f5c900b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:22.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c8ffff640 1 --2- 172.21.15.76:0/2981049111 >> v2:172.21.15.116:3300/0 conn(0x7f5c900b3d50 0x7f5c900b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c8ffff640 1 -- 172.21.15.76:0/2981049111 >> v1:172.21.15.116:6789/0 conn(0x7f5c9006fef0 legacy=0x7f5c9015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:22.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.152+0000 7f5c8ffff640 1 -- 172.21.15.76:0/2981049111 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9006f410 con 0x7f5c90073e60 2024-09-19T04:13:22.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c8ffff640 1 --2- 172.21.15.76:0/2981049111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c90073e60 0x7f5c9006e800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c900b2d50 tx=0x7f5c8001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0b225a4dcda2ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:22.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c8effd640 1 -- 172.21.15.76:0/2981049111 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800b0070 con 0x7f5c90073e60 2024-09-19T04:13:22.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c8effd640 1 -- 172.21.15.76:0/2981049111 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c800ab600 con 0x7f5c90073e60 2024-09-19T04:13:22.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c8effd640 1 -- 172.21.15.76:0/2981049111 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c800ab9d0 con 0x7f5c90073e60 2024-09-19T04:13:22.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 -- 172.21.15.76:0/2981049111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c90073e60 msgr2=0x7f5c9006e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 --2- 172.21.15.76:0/2981049111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c90073e60 0x7f5c9006e800 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5c900b2d50 tx=0x7f5c8001c450 comp rx=0 tx=0).stop 2024-09-19T04:13:22.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 -- 172.21.15.76:0/2981049111 shutdown_connections 2024-09-19T04:13:22.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 --2- 172.21.15.76:0/2981049111 >> v2:172.21.15.116:3300/0 conn(0x7f5c900b3d50 0x7f5c900b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 --2- 172.21.15.76:0/2981049111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c90073e60 0x7f5c9006e800 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 -- 172.21.15.76:0/2981049111 >> 172.21.15.76:0/2981049111 conn(0x7f5c900b1f90 msgr2=0x7f5c900b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:22.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 -- 172.21.15.76:0/2981049111 shutdown_connections 2024-09-19T04:13:22.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 -- 172.21.15.76:0/2981049111 wait complete. 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 Processor -- start 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.156+0000 7f5c968aa640 1 -- start start 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9006fef0 0x7f5c9014caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90073e60 0x7f5c9009abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c900b3d50 0x7f5c9014e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900729a0 con 0x7f5c900b3d50 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90072820 con 0x7f5c90073e60 2024-09-19T04:13:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5c90072b20 con 0x7f5c9006fef0 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c900b3d50 0x7f5c9014e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c900b3d50 0x7f5c9014e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44338/0 (socket says 172.21.15.76:44338) 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 -- 172.21.15.76:0/1422262409 learned_addr learned my addr 172.21.15.76:0/1422262409 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8f7fe640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90073e60 0x7f5c9009abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8ffff640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9006fef0 0x7f5c9014caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 -- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9006fef0 msgr2=0x7f5c9014caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9006fef0 0x7f5c9014caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 -- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90073e60 msgr2=0x7f5c9009abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90073e60 0x7f5c9009abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 -- 172.21.15.76:0/1422262409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c900b0130 con 0x7f5c900b3d50 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c94e47640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c900b3d50 0x7f5c9014e720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5c88007e40 tx=0x7f5c8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:22.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8cff9640 1 -- 172.21.15.76:0/1422262409 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88014620 con 0x7f5c900b3d50 2024-09-19T04:13:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8cff9640 1 -- 172.21.15.76:0/1422262409 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c88019070 con 0x7f5c900b3d50 2024-09-19T04:13:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c900b0410 con 0x7f5c900b3d50 2024-09-19T04:13:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8cff9640 1 -- 172.21.15.76:0/1422262409 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88014a20 con 0x7f5c900b3d50 2024-09-19T04:13:22.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c900b09d0 con 0x7f5c900b3d50 2024-09-19T04:13:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c9006e800 con 0x7f5c900b3d50 2024-09-19T04:13:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8cff9640 1 -- 172.21.15.76:0/1422262409 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5c88014bc0 con 0x7f5c900b3d50 2024-09-19T04:13:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8cff9640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5c700678c0 0x7f5c70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8ffff640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5c700678c0 0x7f5c70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.160+0000 7f5c8cff9640 1 -- 172.21.15.76:0/1422262409 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5c88086250 con 0x7f5c900b3d50 2024-09-19T04:13:22.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.164+0000 7f5c8ffff640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5c700678c0 0x7f5c70069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c800ad000 tx=0x7f5c80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:22.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.168+0000 7f5c8cff9640 1 -- 172.21.15.76:0/1422262409 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5c880570e0 con 0x7f5c900b3d50 2024-09-19T04:13:22.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.324+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5c900723f0 con 0x7f5c900b3d50 2024-09-19T04:13:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.868+0000 7f5c8cff9640 1 -- 172.21.15.76:0/1422262409 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f5c880569e0 con 0x7f5c900b3d50 2024-09-19T04:13:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-09-19T04:13:22.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5c700678c0 msgr2=0x7f5c70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5c700678c0 0x7f5c70069d80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5c800ad000 tx=0x7f5c80008040 comp rx=0 tx=0).stop 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c900b3d50 msgr2=0x7f5c9014e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c900b3d50 0x7f5c9014e720 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5c88007e40 tx=0x7f5c8800e4b0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 shutdown_connections 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5c700678c0 0x7f5c70069d80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5c900b3d50 0x7f5c9014e720 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c90073e60 0x7f5c9009abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 --2- 172.21.15.76:0/1422262409 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5c9006fef0 0x7f5c9014caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 >> 172.21.15.76:0/1422262409 conn(0x7f5c900b1f90 msgr2=0x7f5c9015d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 shutdown_connections 2024-09-19T04:13:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:22.876+0000 7f5c968aa640 1 -- 172.21.15.76:0/1422262409 wait complete. 2024-09-19T04:13:22.884 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:13:22.886 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:13:22.903 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-19T04:13:23.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53e9e9640 1 Processor -- start 2024-09-19T04:13:23.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53e9e9640 1 -- start start 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53e9e9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5380d15d0 0x7ff5380cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53e9e9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5380d0350 con 0x7ff538162070 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53e9e9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5380cf070 con 0x7ff53815e4f0 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53e9e9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff5380d1c20 con 0x7ff5380d15d0 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff52ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5380d15d0 0x7ff5380cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53cf86640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff538162070 0x7ff538162480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47266/0 (socket says 172.21.15.76:47266) 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.108+0000 7ff53cf86640 1 -- 172.21.15.76:0/1304899144 learned_addr learned my addr 172.21.15.76:0/1304899144 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff52ffff640 1 -- 172.21.15.76:0/1304899144 >> v1:172.21.15.116:6789/0 conn(0x7ff53815e4f0 legacy=0x7ff5380d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff52ffff640 1 -- 172.21.15.76:0/1304899144 >> v1:172.21.15.76:6789/0 conn(0x7ff538162070 legacy=0x7ff538162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff52ffff640 1 -- 172.21.15.76:0/1304899144 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5380cf330 con 0x7ff5380d15d0 2024-09-19T04:13:23.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff5377fe640 1 -- 172.21.15.76:0/1304899144 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1593651450 0 0) 0x7ff5380d0350 con 0x7ff538162070 2024-09-19T04:13:23.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff52ffff640 1 --2- 172.21.15.76:0/1304899144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5380d15d0 0x7ff5380cfe10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff5200097c0 tx=0x7ff52001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2bad226238ce518 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:23.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff5377fe640 1 -- 172.21.15.76:0/1304899144 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200b5070 con 0x7ff5380d15d0 2024-09-19T04:13:23.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff5377fe640 1 -- 172.21.15.76:0/1304899144 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5200a8950 con 0x7ff5380d15d0 2024-09-19T04:13:23.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff5377fe640 1 -- 172.21.15.76:0/1304899144 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5200a8d20 con 0x7ff5380d15d0 2024-09-19T04:13:23.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1304899144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5380d15d0 msgr2=0x7ff5380cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1304899144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5380d15d0 0x7ff5380cfe10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff5200097c0 tx=0x7ff52001ec50 comp rx=0 tx=0).stop 2024-09-19T04:13:23.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1304899144 shutdown_connections 2024-09-19T04:13:23.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1304899144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5380d15d0 0x7ff5380cfe10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1304899144 >> 172.21.15.76:0/1304899144 conn(0x7ff53808bf90 msgr2=0x7ff5380d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:23.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1304899144 shutdown_connections 2024-09-19T04:13:23.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.112+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1304899144 wait complete. 2024-09-19T04:13:23.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 Processor -- start 2024-09-19T04:13:23.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 -- start start 2024-09-19T04:13:23.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380d15d0 0x7ff538155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff53815e4f0 0x7ff538160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff538162070 0x7ff538161010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff53815e8d0 con 0x7ff538162070 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5380cf8f0 con 0x7ff5380d15d0 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff5380b7260 con 0x7ff53815e4f0 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff538162070 0x7ff538161010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff538162070 0x7ff538161010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53412/0 (socket says 172.21.15.76:53412) 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 -- 172.21.15.76:0/1833480677 learned_addr learned my addr 172.21.15.76:0/1833480677 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff52ffff640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff53815e4f0 0x7ff538160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 -- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff53815e4f0 msgr2=0x7ff538160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff537fff640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380d15d0 0x7ff538155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff53815e4f0 0x7ff538160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 -- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380d15d0 msgr2=0x7ff538155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380d15d0 0x7ff538155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 -- 172.21.15.76:0/1833480677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff53815eaf0 con 0x7ff538162070 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff537fff640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380d15d0 0x7ff538155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53cf86640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff538162070 0x7ff538161010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff53000c780 tx=0x7ff53000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff5357fa640 1 -- 172.21.15.76:0/1833480677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff530018070 con 0x7ff538162070 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff5357fa640 1 -- 172.21.15.76:0/1833480677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff530004d70 con 0x7ff538162070 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5380ca0e0 con 0x7ff538162070 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff5357fa640 1 -- 172.21.15.76:0/1833480677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53000a880 con 0x7ff538162070 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff53815f070 con 0x7ff538162070 2024-09-19T04:13:23.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5380cfe10 con 0x7ff538162070 2024-09-19T04:13:23.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.116+0000 7ff5357fa640 1 -- 172.21.15.76:0/1833480677 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff53001b070 con 0x7ff538162070 2024-09-19T04:13:23.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.120+0000 7ff5357fa640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5180678c0 0x7ff518069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:23.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.120+0000 7ff5357fa640 1 -- 172.21.15.76:0/1833480677 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7ff530084fa0 con 0x7ff538162070 2024-09-19T04:13:23.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.120+0000 7ff537fff640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5180678c0 0x7ff518069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:23.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.120+0000 7ff537fff640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5180678c0 0x7ff518069d80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff528003e40 tx=0x7ff528009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:23.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.124+0000 7ff5357fa640 1 -- 172.21.15.76:0/1833480677 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff530055d50 con 0x7ff538162070 2024-09-19T04:13:23.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.284+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7ff5380d4b20 con 0x7ff538162070 2024-09-19T04:13:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.876+0000 7ff5357fa640 1 -- 172.21.15.76:0/1833480677 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7ff530055650 con 0x7ff538162070 2024-09-19T04:13:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-09-19T04:13:23.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5180678c0 msgr2=0x7ff518069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5180678c0 0x7ff518069d80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff528003e40 tx=0x7ff528009040 comp rx=0 tx=0).stop 2024-09-19T04:13:23.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff538162070 msgr2=0x7ff538161010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff538162070 0x7ff538161010 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff53000c780 tx=0x7ff53000cc40 comp rx=0 tx=0).stop 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 shutdown_connections 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5180678c0 0x7ff518069d80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff538162070 0x7ff538161010 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff53815e4f0 0x7ff538160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 --2- 172.21.15.76:0/1833480677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5380d15d0 0x7ff538155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 >> 172.21.15.76:0/1833480677 conn(0x7ff53808bf90 msgr2=0x7ff5380d4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 shutdown_connections 2024-09-19T04:13:23.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:23.880+0000 7ff53e9e9640 1 -- 172.21.15.76:0/1833480677 wait complete. 2024-09-19T04:13:23.913 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:13:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.036+0000 7f2e55596640 1 Processor -- start 2024-09-19T04:13:24.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.036+0000 7f2e55596640 1 -- start start 2024-09-19T04:13:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e55596640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 0x7f2e5006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e55596640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e50070250 con 0x7f2e5015f3a0 2024-09-19T04:13:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e55596640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e500aede0 con 0x7f2e5006fe50 2024-09-19T04:13:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e55596640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2e50152480 con 0x7f2e501520a0 2024-09-19T04:13:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2e5015f3a0 0x7f2e500adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47282/0 (socket says 172.21.15.76:47282) 2024-09-19T04:13:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 -- 172.21.15.76:0/2597379937 learned_addr learned my addr 172.21.15.76:0/2597379937 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4effd640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 0x7f2e5006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4031417097 0 0) 0x7f2e50070250 con 0x7f2e5015f3a0 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2e3c003530 con 0x7f2e5015f3a0 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2563593540 0 0) 0x7f2e3c003530 con 0x7f2e5015f3a0 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 msgr2=0x7f2e5006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 0x7f2e5006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 >> v1:172.21.15.116:6789/0 conn(0x7f2e5006fe50 legacy=0x7f2e5015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e5015fa10 con 0x7f2e5015f3a0 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2e48001a00 con 0x7f2e5015f3a0 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 >> v1:172.21.15.76:6789/0 conn(0x7f2e5015f3a0 legacy=0x7f2e500adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0039f0 0x7f2e3c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 0x7f2e3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e3c004d50 0x7f2e3c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c00ba80 con 0x7f2e3c004d50 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c00bc00 con 0x7f2e3c0039f0 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c00be00 con 0x7f2e501520a0 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e3c004d50 0x7f2e3c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4effd640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0039f0 0x7f2e3c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 -- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 msgr2=0x7f2e3c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e47fff640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 0x7f2e3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e501520a0 0x7f2e3c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 -- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0039f0 msgr2=0x7f2e3c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0039f0 0x7f2e3c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4f7fe640 1 -- 172.21.15.76:0/2597379937 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c003530 con 0x7f2e3c004d50 2024-09-19T04:13:24.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.040+0000 7f2e4effd640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0039f0 0x7f2e3c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e4f7fe640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e3c004d50 0x7f2e3c0051d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800a810 tx=0x7f2e4801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad5d72eaeeea45c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4800a460 con 0x7f2e3c004d50 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e480ad650 con 0x7f2e3c004d50 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e4e7fc640 1 -- 172.21.15.76:0/2597379937 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e480ada40 con 0x7f2e3c004d50 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- 172.21.15.76:0/2597379937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e3c004d50 msgr2=0x7f2e3c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 --2- 172.21.15.76:0/2597379937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e3c004d50 0x7f2e3c0051d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2e4800a810 tx=0x7f2e4801d340 comp rx=0 tx=0).stop 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e4f7fe640 1 -- 172.21.15.76:0/2597379937 reap_dead start 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- 172.21.15.76:0/2597379937 shutdown_connections 2024-09-19T04:13:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- 172.21.15.76:0/2597379937 >> 172.21.15.76:0/2597379937 conn(0x7f2e500b1f90 msgr2=0x7f2e500b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- 172.21.15.76:0/2597379937 shutdown_connections 2024-09-19T04:13:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- 172.21.15.76:0/2597379937 wait complete. 2024-09-19T04:13:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 Processor -- start 2024-09-19T04:13:24.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- start start 2024-09-19T04:13:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e3c004d50 0x7f2e500994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e50149ad0 0x7f2e50144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e501efe50 0x7f2e501449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50073770 con 0x7f2e50149ad0 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e50073490 con 0x7f2e501efe50 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e55596640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2e50073a50 con 0x7f2e3c004d50 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e50149ad0 0x7f2e50144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e50149ad0 0x7f2e50144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53430/0 (socket says 172.21.15.76:53430) 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 -- 172.21.15.76:0/150570710 learned_addr learned my addr 172.21.15.76:0/150570710 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e4effd640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e3c004d50 0x7f2e500994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e4f7fe640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e501efe50 0x7f2e501449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 -- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e3c004d50 msgr2=0x7f2e500994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e3c004d50 0x7f2e500994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 -- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e501efe50 msgr2=0x7f2e501449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e501efe50 0x7f2e501449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.044+0000 7f2e47fff640 1 -- 172.21.15.76:0/150570710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e501f0d30 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e47fff640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e50149ad0 0x7f2e50144470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800d500 tx=0x7f2e3800d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e477fe640 1 -- 172.21.15.76:0/150570710 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38003620 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e477fe640 1 -- 172.21.15.76:0/150570710 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e38017070 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e500b1c00 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e477fe640 1 -- 172.21.15.76:0/150570710 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380039c0 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e501f12a0 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e50073d30 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e477fe640 1 -- 172.21.15.76:0/150570710 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e38003b60 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e477fe640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678c0 0x7f2e30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e477fe640 1 -- 172.21.15.76:0/150570710 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f2e38085a00 con 0x7f2e50149ad0 2024-09-19T04:13:24.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e4effd640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678c0 0x7f2e30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:24.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.048+0000 7f2e4effd640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678c0 0x7f2e30069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009a30 tx=0x7f2e40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:24.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.052+0000 7f2e477fe640 1 -- 172.21.15.76:0/150570710 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2e3800f2f0 con 0x7f2e50149ad0 2024-09-19T04:13:24.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.248+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2e5006e540 con 0x7f2e50149ad0 2024-09-19T04:13:24.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.936+0000 7f2e477fe640 1 -- 172.21.15.76:0/150570710 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f2e38056630 con 0x7f2e50149ad0 2024-09-19T04:13:24.940 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T04:13:24.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.948+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678c0 msgr2=0x7f2e30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.948+0000 7f2e55596640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678c0 0x7f2e30069d80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009a30 tx=0x7f2e40008040 comp rx=0 tx=0).stop 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.948+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e50149ad0 msgr2=0x7f2e50144470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.948+0000 7f2e55596640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e50149ad0 0x7f2e50144470 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800d500 tx=0x7f2e3800d9c0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 shutdown_connections 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678c0 0x7f2e30069d80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e501efe50 0x7f2e501449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e50149ad0 0x7f2e50144470 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 --2- 172.21.15.76:0/150570710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e3c004d50 0x7f2e500994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 >> 172.21.15.76:0/150570710 conn(0x7f2e500b1f90 msgr2=0x7f2e500b3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 shutdown_connections 2024-09-19T04:13:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:24.952+0000 7f2e55596640 1 -- 172.21.15.76:0/150570710 wait complete. 2024-09-19T04:13:24.981 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-19T04:13:24.982 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-19T04:13:25.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc368d22640 1 Processor -- start 2024-09-19T04:13:25.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc368d22640 1 -- start start 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc368d22640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc36409ac60 0x7fc36409ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc368d22640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc36409b040 0x7fc3640af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc368d22640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3640afd80 con 0x7fc3640b0e20 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc368d22640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc3640b0060 con 0x7fc36409ac60 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc368d22640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc3640b0340 con 0x7fc36409b040 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc36409ac60 0x7fc36409ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc36409ac60 0x7fc36409ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53444/0 (socket says 172.21.15.76:53444) 2024-09-19T04:13:25.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 -- 172.21.15.76:0/2736484094 learned_addr learned my addr 172.21.15.76:0/2736484094 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:25.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc361d9b640 1 --2- 172.21.15.76:0/2736484094 >> v2:172.21.15.116:3300/0 conn(0x7fc36409b040 0x7fc3640af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 -- 172.21.15.76:0/2736484094 >> v2:172.21.15.116:3300/0 conn(0x7fc36409b040 msgr2=0x7fc3640af730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:25.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 --2- 172.21.15.76:0/2736484094 >> v2:172.21.15.116:3300/0 conn(0x7fc36409b040 0x7fc3640af730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 -- 172.21.15.76:0/2736484094 >> v1:172.21.15.116:6789/0 conn(0x7fc3640b0e20 legacy=0x7fc3640b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:25.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 -- 172.21.15.76:0/2736484094 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3640b0620 con 0x7fc36409ac60 2024-09-19T04:13:25.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.104+0000 7fc36259c640 1 --2- 172.21.15.76:0/2736484094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36409ba00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3640b52b0 tx=0x7fc35401ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8cfa2d5ac65a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:25.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36159a640 1 -- 172.21.15.76:0/2736484094 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540ab640 con 0x7fc36409ac60 2024-09-19T04:13:25.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36159a640 1 -- 172.21.15.76:0/2736484094 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3540b0070 con 0x7fc36409ac60 2024-09-19T04:13:25.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36159a640 1 -- 172.21.15.76:0/2736484094 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540aba00 con 0x7fc36409ac60 2024-09-19T04:13:25.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- 172.21.15.76:0/2736484094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 msgr2=0x7fc36409ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 --2- 172.21.15.76:0/2736484094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36409ba00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc3640b52b0 tx=0x7fc35401ce80 comp rx=0 tx=0).stop 2024-09-19T04:13:25.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- 172.21.15.76:0/2736484094 shutdown_connections 2024-09-19T04:13:25.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 --2- 172.21.15.76:0/2736484094 >> v2:172.21.15.116:3300/0 conn(0x7fc36409b040 0x7fc3640af730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 --2- 172.21.15.76:0/2736484094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36409ba00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- 172.21.15.76:0/2736484094 >> 172.21.15.76:0/2736484094 conn(0x7fc3640b78d0 msgr2=0x7fc3640b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:25.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- 172.21.15.76:0/2736484094 shutdown_connections 2024-09-19T04:13:25.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- 172.21.15.76:0/2736484094 wait complete. 2024-09-19T04:13:25.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 Processor -- start 2024-09-19T04:13:25.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- start start 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36414d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc36409b040 0x7fc3641553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3640b0e20 0x7fc364148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc36415d3e0 con 0x7fc36409ac60 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc36415d260 con 0x7fc3640b0e20 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc368d22640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc36415d560 con 0x7fc36409b040 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36414d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36414d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53456/0 (socket says 172.21.15.76:53456) 2024-09-19T04:13:25.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 -- 172.21.15.76:0/4187548192 learned_addr learned my addr 172.21.15.76:0/4187548192 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc361d9b640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc36409b040 0x7fc3641553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 -- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc36409b040 msgr2=0x7fc3641553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc36409b040 0x7fc3641553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc362d9d640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3640b0e20 0x7fc364148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 -- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3640b0e20 msgr2=0x7fc364148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3640b0e20 0x7fc364148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 -- 172.21.15.76:0/4187548192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36414f8f0 con 0x7fc36409ac60 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc362d9d640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3640b0e20 0x7fc364148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.108+0000 7fc36259c640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36414d4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3540ad040 tx=0x7fc35401ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc352ffd640 1 -- 172.21.15.76:0/4187548192 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540be5f0 con 0x7fc36409ac60 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc352ffd640 1 -- 172.21.15.76:0/4187548192 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3540be7b0 con 0x7fc36409ac60 2024-09-19T04:13:25.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc352ffd640 1 -- 172.21.15.76:0/4187548192 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3540beba0 con 0x7fc36409ac60 2024-09-19T04:13:25.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc36414ca00 con 0x7fc36409ac60 2024-09-19T04:13:25.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36414fb90 con 0x7fc36409ac60 2024-09-19T04:13:25.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3640b2770 con 0x7fc36409ac60 2024-09-19T04:13:25.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc352ffd640 1 -- 172.21.15.76:0/4187548192 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3540b43f0 con 0x7fc36409ac60 2024-09-19T04:13:25.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc352ffd640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3440678f0 0x7fc344069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:25.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc361d9b640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3440678f0 0x7fc344069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:25.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc352ffd640 1 -- 172.21.15.76:0/4187548192 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc3541206a0 con 0x7fc36409ac60 2024-09-19T04:13:25.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.112+0000 7fc361d9b640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3440678f0 0x7fc344069db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc36414ead0 tx=0x7fc358008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:25.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.116+0000 7fc352ffd640 1 -- 172.21.15.76:0/4187548192 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc3540160d0 con 0x7fc36409ac60 2024-09-19T04:13:25.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.288+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fc3640b40a0 con 0x7fc36409ac60 2024-09-19T04:13:25.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.908+0000 7fc352ffd640 1 -- 172.21.15.76:0/4187548192 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7fc3540f0b90 con 0x7fc36409ac60 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3440678f0 msgr2=0x7fc344069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3440678f0 0x7fc344069db0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc36414ead0 tx=0x7fc358008040 comp rx=0 tx=0).stop 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 msgr2=0x7fc36414d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36414d4c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3540ad040 tx=0x7fc35401ac60 comp rx=0 tx=0).stop 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 shutdown_connections 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3440678f0 0x7fc344069db0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3640b0e20 0x7fc364148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc36409b040 0x7fc3641553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 --2- 172.21.15.76:0/4187548192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc36409ac60 0x7fc36414d4c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 >> 172.21.15.76:0/4187548192 conn(0x7fc3640b78d0 msgr2=0x7fc364071700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 shutdown_connections 2024-09-19T04:13:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:25.912+0000 7fc368d22640 1 -- 172.21.15.76:0/4187548192 wait complete. 2024-09-19T04:13:25.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:13:26.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.067+0000 7fa21a85c640 1 Processor -- start 2024-09-19T04:13:26.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.067+0000 7fa21a85c640 1 -- start start 2024-09-19T04:13:26.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa21a85c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2141584a0 0x7fa21415b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa21a85c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa2140b3f30 0x7fa214071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa21a85c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa21406f350 con 0x7fa21406ebb0 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa21a85c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa21406ff60 con 0x7fa2141584a0 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa21a85c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa21406f630 con 0x7fa2140b3f30 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2141584a0 0x7fa21415b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa2141584a0 0x7fa21415b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53468/0 (socket says 172.21.15.76:53468) 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa2137fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa21406ebb0 0x7fa2140b3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47296/0 (socket says 172.21.15.76:47296) 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 -- 172.21.15.76:0/4212377176 learned_addr learned my addr 172.21.15.76:0/4212377176 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa218df9640 1 --2- 172.21.15.76:0/4212377176 >> v2:172.21.15.116:3300/0 conn(0x7fa2140b3f30 0x7fa214071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 -- 172.21.15.76:0/4212377176 >> v2:172.21.15.116:3300/0 conn(0x7fa2140b3f30 msgr2=0x7fa214071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:26.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 --2- 172.21.15.76:0/4212377176 >> v2:172.21.15.116:3300/0 conn(0x7fa2140b3f30 0x7fa214071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 -- 172.21.15.76:0/4212377176 >> v1:172.21.15.76:6789/0 conn(0x7fa21406ebb0 legacy=0x7fa2140b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:26.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa212ffd640 1 -- 172.21.15.76:0/4212377176 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2006358097 0 0) 0x7fa21406f350 con 0x7fa21406ebb0 2024-09-19T04:13:26.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 -- 172.21.15.76:0/4212377176 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21406f910 con 0x7fa2141584a0 2024-09-19T04:13:26.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa213fff640 1 --2- 172.21.15.76:0/4212377176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2141584a0 0x7fa21415b850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2140cff40 tx=0x7fa20c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780db3a636e5e505 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:26.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.071+0000 7fa212ffd640 1 -- 172.21.15.76:0/4212377176 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0aa5f0 con 0x7fa2141584a0 2024-09-19T04:13:26.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa212ffd640 1 -- 172.21.15.76:0/4212377176 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa20c0af070 con 0x7fa2141584a0 2024-09-19T04:13:26.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa212ffd640 1 -- 172.21.15.76:0/4212377176 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa20c0aa9b0 con 0x7fa2141584a0 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- 172.21.15.76:0/4212377176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2141584a0 msgr2=0x7fa21415b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 --2- 172.21.15.76:0/4212377176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2141584a0 0x7fa21415b850 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa2140cff40 tx=0x7fa20c01c4f0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- 172.21.15.76:0/4212377176 shutdown_connections 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 --2- 172.21.15.76:0/4212377176 >> v2:172.21.15.116:3300/0 conn(0x7fa2140b3f30 0x7fa214071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 --2- 172.21.15.76:0/4212377176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2141584a0 0x7fa21415b850 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- 172.21.15.76:0/4212377176 >> 172.21.15.76:0/4212377176 conn(0x7fa21406c380 msgr2=0x7fa2140b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- 172.21.15.76:0/4212377176 shutdown_connections 2024-09-19T04:13:26.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- 172.21.15.76:0/4212377176 wait complete. 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 Processor -- start 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- start start 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa21406ebb0 0x7fa21414b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2140b3f30 0x7fa21414acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2141584a0 0x7fa21414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2140727e0 con 0x7fa2140b3f30 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa214072660 con 0x7fa2141584a0 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa21a85c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa214072960 con 0x7fa21406ebb0 2024-09-19T04:13:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2140b3f30 0x7fa21414acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2140b3f30 0x7fa21414acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53480/0 (socket says 172.21.15.76:53480) 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 -- 172.21.15.76:0/2697374535 learned_addr learned my addr 172.21.15.76:0/2697374535 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa213fff640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa21406ebb0 0x7fa21414b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa218df9640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2141584a0 0x7fa21414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 -- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa21406ebb0 msgr2=0x7fa21414b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa21406ebb0 0x7fa21414b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 -- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2141584a0 msgr2=0x7fa21414d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2141584a0 0x7fa21414d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 -- 172.21.15.76:0/2697374535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa214159ec0 con 0x7fa2140b3f30 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa2137fe640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2140b3f30 0x7fa21414acd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2080011f0 tx=0x7fa20800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa210ff9640 1 -- 172.21.15.76:0/2697374535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208018070 con 0x7fa2140b3f30 2024-09-19T04:13:26.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa210ff9640 1 -- 172.21.15.76:0/2697374535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2080074e0 con 0x7fa2140b3f30 2024-09-19T04:13:26.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.075+0000 7fa210ff9640 1 -- 172.21.15.76:0/2697374535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa208004a10 con 0x7fa2140b3f30 2024-09-19T04:13:26.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21415a1a0 con 0x7fa2140b3f30 2024-09-19T04:13:26.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21415cc30 con 0x7fa2140b3f30 2024-09-19T04:13:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21415ce30 con 0x7fa2140b3f30 2024-09-19T04:13:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa210ff9640 1 -- 172.21.15.76:0/2697374535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa208004bb0 con 0x7fa2140b3f30 2024-09-19T04:13:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa210ff9640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2000678c0 0x7fa200069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa210ff9640 1 -- 172.21.15.76:0/2697374535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7fa208085190 con 0x7fa2140b3f30 2024-09-19T04:13:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa213fff640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2000678c0 0x7fa200069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.079+0000 7fa213fff640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2000678c0 0x7fa200069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0ae000 tx=0x7fa20c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.083+0000 7fa210ff9640 1 -- 172.21.15.76:0/2697374535 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa21415ce30 con 0x7fa2140b3f30 2024-09-19T04:13:26.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:26.247+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fa2141575c0 con 0x7fa2140b3f30 2024-09-19T04:13:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.935+0000 7fa210ff9640 1 -- 172.21.15.76:0/2697374535 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7fa208055ba0 con 0x7fa2140b3f30 2024-09-19T04:13:27.937 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data_ec' created 2024-09-19T04:13:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2000678c0 msgr2=0x7fa200069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2000678c0 0x7fa200069d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa20c0ae000 tx=0x7fa20c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:13:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2140b3f30 msgr2=0x7fa21414acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:27.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2140b3f30 0x7fa21414acd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa2080011f0 tx=0x7fa20800c9b0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 shutdown_connections 2024-09-19T04:13:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2000678c0 0x7fa200069d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2141584a0 0x7fa21414d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa2140b3f30 0x7fa21414acd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 --2- 172.21.15.76:0/2697374535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa21406ebb0 0x7fa21414b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 >> 172.21.15.76:0/2697374535 conn(0x7fa21406c380 msgr2=0x7fa214157990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:27.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 shutdown_connections 2024-09-19T04:13:27.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:27.939+0000 7fa21a85c640 1 -- 172.21.15.76:0/2697374535 wait complete. 2024-09-19T04:13:27.971 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-19T04:13:28.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f421decb640 1 Processor -- start 2024-09-19T04:13:28.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f421decb640 1 -- start start 2024-09-19T04:13:28.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f421decb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f421806fef0 0x7f421806e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:28.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f421decb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:28.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f421decb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f421806ee50 con 0x7f4218073e60 2024-09-19T04:13:28.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f421decb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f42180b46c0 con 0x7f421806fef0 2024-09-19T04:13:28.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f421decb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f421806f130 con 0x7f42180b3d50 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4217fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4216ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f421806fef0 0x7f421806e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4216ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f421806fef0 0x7f421806e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:32928/0 (socket says 172.21.15.76:32928) 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4216ffd640 1 -- 172.21.15.76:0/1980900718 learned_addr learned my addr 172.21.15.76:0/1980900718 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4216ffd640 1 -- 172.21.15.76:0/1980900718 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 msgr2=0x7f42180b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4216ffd640 1 --2- 172.21.15.76:0/1980900718 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4216ffd640 1 -- 172.21.15.76:0/1980900718 >> v1:172.21.15.116:6789/0 conn(0x7f4218073e60 legacy=0x7f421815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.095+0000 7f4216ffd640 1 -- 172.21.15.76:0/1980900718 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421806f410 con 0x7f421806fef0 2024-09-19T04:13:28.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f4216ffd640 1 --2- 172.21.15.76:0/1980900718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 0x7f421806e800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4208009870 tx=0x7f420801c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f876f71fb2dca040 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f42167fc640 1 -- 172.21.15.76:0/1980900718 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42080b0070 con 0x7f421806fef0 2024-09-19T04:13:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f42167fc640 1 -- 172.21.15.76:0/1980900718 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42080ab600 con 0x7f421806fef0 2024-09-19T04:13:28.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f42167fc640 1 -- 172.21.15.76:0/1980900718 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42080ab9d0 con 0x7f421806fef0 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 -- 172.21.15.76:0/1980900718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 msgr2=0x7f421806e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 --2- 172.21.15.76:0/1980900718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 0x7f421806e800 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4208009870 tx=0x7f420801c4d0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 -- 172.21.15.76:0/1980900718 shutdown_connections 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 --2- 172.21.15.76:0/1980900718 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 --2- 172.21.15.76:0/1980900718 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 0x7f421806e800 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 -- 172.21.15.76:0/1980900718 >> 172.21.15.76:0/1980900718 conn(0x7f42180b1f90 msgr2=0x7f42180b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 -- 172.21.15.76:0/1980900718 shutdown_connections 2024-09-19T04:13:28.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 -- 172.21.15.76:0/1980900718 wait complete. 2024-09-19T04:13:28.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 Processor -- start 2024-09-19T04:13:28.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.099+0000 7f421decb640 1 -- start start 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 0x7f42180b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4218073e60 0x7f42180ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42180729a0 con 0x7f4218073e60 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4218072820 con 0x7f421806fef0 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4218072b20 con 0x7f42180b3d50 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4218073e60 0x7f42180ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4218073e60 0x7f42180ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53490/0 (socket says 172.21.15.76:53490) 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 -- 172.21.15.76:0/1990036217 learned_addr learned my addr 172.21.15.76:0/1990036217 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4217fff640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:28.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f42177fe640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 0x7f42180b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 -- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 msgr2=0x7f42180b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 -- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 msgr2=0x7f42180b1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 0x7f42180b1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 -- 172.21.15.76:0/1990036217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421814d720 con 0x7f4218073e60 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f4216ffd640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4218073e60 0x7f42180ae5b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42080ae040 tx=0x7f4208019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f41fffff640 1 -- 172.21.15.76:0/1990036217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42080b0040 con 0x7f4218073e60 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f41fffff640 1 -- 172.21.15.76:0/1990036217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42080b4640 con 0x7f4218073e60 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42180adad0 con 0x7f4218073e60 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f41fffff640 1 -- 172.21.15.76:0/1990036217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42080b4a30 con 0x7f4218073e60 2024-09-19T04:13:28.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42180ae090 con 0x7f4218073e60 2024-09-19T04:13:28.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f421806e800 con 0x7f4218073e60 2024-09-19T04:13:28.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f41fffff640 1 -- 172.21.15.76:0/1990036217 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f42080ab4c0 con 0x7f4218073e60 2024-09-19T04:13:28.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f41fffff640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42040678f0 0x7f4204069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:28.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f41fffff640 1 -- 172.21.15.76:0/1990036217 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f420811fa40 con 0x7f4218073e60 2024-09-19T04:13:28.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.103+0000 7f42177fe640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42040678f0 0x7f4204069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:28.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.107+0000 7f42177fe640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42040678f0 0x7f4204069db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4218152db0 tx=0x7f420c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:28.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.111+0000 7f41fffff640 1 -- 172.21.15.76:0/1990036217 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4208016150 con 0x7f4218073e60 2024-09-19T04:13:28.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.271+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f4218157e30 con 0x7f4218073e60 2024-09-19T04:13:28.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.939+0000 7f41fffff640 1 -- 172.21.15.76:0/1990036217 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f42080efc10 con 0x7f4218073e60 2024-09-19T04:13:28.940 INFO:teuthology.orchestra.run.smithi076.stderr:set pool 4 allow_ec_overwrites to true 2024-09-19T04:13:28.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.939+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42040678f0 msgr2=0x7f4204069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.939+0000 7f421decb640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42040678f0 0x7f4204069db0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4218152db0 tx=0x7f420c009040 comp rx=0 tx=0).stop 2024-09-19T04:13:28.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.939+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4218073e60 msgr2=0x7f42180ae5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:28.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.939+0000 7f421decb640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4218073e60 0x7f42180ae5b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42080ae040 tx=0x7f4208019270 comp rx=0 tx=0).stop 2024-09-19T04:13:28.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 shutdown_connections 2024-09-19T04:13:28.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42040678f0 0x7f4204069db0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f42180b3d50 0x7f42180b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4218073e60 0x7f42180ae5b0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f42080ae040 tx=0x7f4208019270 comp rx=0 tx=0).stop 2024-09-19T04:13:28.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 --2- 172.21.15.76:0/1990036217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f421806fef0 0x7f42180b1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:28.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 >> 172.21.15.76:0/1990036217 conn(0x7f42180b1f90 msgr2=0x7f42181590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:28.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 shutdown_connections 2024-09-19T04:13:28.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:28.943+0000 7f421decb640 1 -- 172.21.15.76:0/1990036217 wait complete. 2024-09-19T04:13:28.986 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-19T04:13:29.077 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:13:29.075+0000 7f268488d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:13:29.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97f5097640 1 Processor -- start 2024-09-19T04:13:29.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97f5097640 1 -- start start 2024-09-19T04:13:29.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97f5097640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f015b170 0x7f97f009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97f5097640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97f00b0f00 0x7f97f015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97f5097640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97f00ae8b0 con 0x7f97f00af950 2024-09-19T04:13:29.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97f5097640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f97f00aeb90 con 0x7f97f00b0f00 2024-09-19T04:13:29.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97f5097640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f97f00aee70 con 0x7f97f015b170 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97eed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f015b170 0x7f97f009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97ef59e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97f00b0f00 0x7f97f015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97eed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f015b170 0x7f97f009aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:42362/0 (socket says 172.21.15.76:42362) 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97eed9d640 1 -- 172.21.15.76:0/1112969315 learned_addr learned my addr 172.21.15.76:0/1112969315 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97ef59e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97f00b0f00 0x7f97f015ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:32942/0 (socket says 172.21.15.76:32942) 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.107+0000 7f97ee59c640 1 --1- 172.21.15.76:0/1112969315 >> v1:172.21.15.116:6789/0 conn(0x7f97f00af950 0x7f97f00b07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:50004/0 (socket says 172.21.15.76:50004) 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97eed9d640 1 -- 172.21.15.76:0/1112969315 >> v2:172.21.15.116:3300/0 conn(0x7f97f00b0f00 msgr2=0x7f97f015ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97eed9d640 1 --2- 172.21.15.76:0/1112969315 >> v2:172.21.15.116:3300/0 conn(0x7f97f00b0f00 0x7f97f015ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97eed9d640 1 -- 172.21.15.76:0/1112969315 >> v1:172.21.15.116:6789/0 conn(0x7f97f00af950 legacy=0x7f97f00b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97eed9d640 1 -- 172.21.15.76:0/1112969315 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00af150 con 0x7f97f015b170 2024-09-19T04:13:29.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97ef59e640 1 --2- 172.21.15.76:0/1112969315 >> v2:172.21.15.116:3300/0 conn(0x7f97f00b0f00 0x7f97f015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97eed9d640 1 --2- 172.21.15.76:0/1112969315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f015b170 0x7f97f009aa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009870 tx=0x7f97e401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d2b5dbb7f82670c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97edd9b640 1 -- 172.21.15.76:0/1112969315 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40b2070 con 0x7f97f015b170 2024-09-19T04:13:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97edd9b640 1 -- 172.21.15.76:0/1112969315 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e40a9070 con 0x7f97f015b170 2024-09-19T04:13:29.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97edd9b640 1 -- 172.21.15.76:0/1112969315 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40ad880 con 0x7f97f015b170 2024-09-19T04:13:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 -- 172.21.15.76:0/1112969315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f015b170 msgr2=0x7f97f009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 --2- 172.21.15.76:0/1112969315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f015b170 0x7f97f009aa00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009870 tx=0x7f97e401c560 comp rx=0 tx=0).stop 2024-09-19T04:13:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 -- 172.21.15.76:0/1112969315 shutdown_connections 2024-09-19T04:13:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 --2- 172.21.15.76:0/1112969315 >> v2:172.21.15.116:3300/0 conn(0x7f97f00b0f00 0x7f97f015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 --2- 172.21.15.76:0/1112969315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f015b170 0x7f97f009aa00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 -- 172.21.15.76:0/1112969315 >> 172.21.15.76:0/1112969315 conn(0x7f97f00b6800 msgr2=0x7f97f00b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:29.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 -- 172.21.15.76:0/1112969315 shutdown_connections 2024-09-19T04:13:29.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.111+0000 7f97f5097640 1 -- 172.21.15.76:0/1112969315 wait complete. 2024-09-19T04:13:29.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 Processor -- start 2024-09-19T04:13:29.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 -- start start 2024-09-19T04:13:29.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97f00af950 0x7f97f01495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f00b0f00 0x7f97f0145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97f015b170 0x7f97f014b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f97f006c050 con 0x7f97f00af950 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f97f006bed0 con 0x7f97f015b170 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f97f006c1d0 con 0x7f97f00b0f00 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97f00af950 0x7f97f01495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97f00af950 0x7f97f01495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53500/0 (socket says 172.21.15.76:53500) 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 -- 172.21.15.76:0/735620059 learned_addr learned my addr 172.21.15.76:0/735620059 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97ee59c640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f00b0f00 0x7f97f0145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97ef59e640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97f015b170 0x7f97f014b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 -- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f00b0f00 msgr2=0x7f97f0145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f00b0f00 0x7f97f0145c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 -- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97f015b170 msgr2=0x7f97f014b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97f015b170 0x7f97f014b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 -- 172.21.15.76:0/735620059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f00b5260 con 0x7f97f00af950 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97ee59c640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f00b0f00 0x7f97f0145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97eed9d640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97f00af950 0x7f97f01495e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97e401a190 tx=0x7f97e401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97d77fe640 1 -- 172.21.15.76:0/735620059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40bc020 con 0x7f97f00af950 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97d77fe640 1 -- 172.21.15.76:0/735620059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e40add80 con 0x7f97f00af950 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f01461a0 con 0x7f97f00af950 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97d77fe640 1 -- 172.21.15.76:0/735620059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40b2040 con 0x7f97f00af950 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f014e410 con 0x7f97f00af950 2024-09-19T04:13:29.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f015ee20 con 0x7f97f00af950 2024-09-19T04:13:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97d77fe640 1 -- 172.21.15.76:0/735620059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f97e40bf020 con 0x7f97f00af950 2024-09-19T04:13:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97d77fe640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97dc0678c0 0x7f97dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.115+0000 7f97d77fe640 1 -- 172.21.15.76:0/735620059 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f97e4120100 con 0x7f97f00af950 2024-09-19T04:13:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.119+0000 7f97ee59c640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97dc0678c0 0x7f97dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:29.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.119+0000 7f97ee59c640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97dc0678c0 0x7f97dc069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97e0007fd0 tx=0x7f97e0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:29.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.123+0000 7f97d77fe640 1 -- 172.21.15.76:0/735620059 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f97e40c6030 con 0x7f97f00af950 2024-09-19T04:13:29.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.319+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f97f006e7c0 con 0x7f97f00af950 2024-09-19T04:13:29.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.971+0000 7f97d77fe640 1 -- 172.21.15.76:0/735620059 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7f97e4016150 con 0x7f97f00af950 2024-09-19T04:13:29.974 INFO:teuthology.orchestra.run.smithi076.stderr:added data pool 4 to fsmap 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97dc0678c0 msgr2=0x7f97dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97dc0678c0 0x7f97dc069d80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f97e0007fd0 tx=0x7f97e0009040 comp rx=0 tx=0).stop 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97f00af950 msgr2=0x7f97f01495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97f00af950 0x7f97f01495e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f97e401a190 tx=0x7f97e401ada0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 shutdown_connections 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97dc0678c0 0x7f97dc069d80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97f015b170 0x7f97f014b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97f00b0f00 0x7f97f0145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 --2- 172.21.15.76:0/735620059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97f00af950 0x7f97f01495e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:29.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 >> 172.21.15.76:0/735620059 conn(0x7f97f00b6800 msgr2=0x7f97f00b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:29.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 shutdown_connections 2024-09-19T04:13:29.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:29.975+0000 7f97f5097640 1 -- 172.21.15.76:0/735620059 wait complete. 2024-09-19T04:13:29.997 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b3e04b640 1 Processor -- start 2024-09-19T04:13:30.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b3e04b640 1 -- start start 2024-09-19T04:13:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b3e04b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b3815c2a0 0x7f3b380aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b3e04b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b380af2a0 0x7f3b3806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b3e04b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b380b3290 con 0x7f3b3815bec0 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b3e04b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3b380b3570 con 0x7f3b3815c2a0 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b3e04b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3b380b3850 con 0x7f3b380af2a0 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b36ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b3815c2a0 0x7f3b380aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b36ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b3815c2a0 0x7f3b380aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53502/0 (socket says 172.21.15.76:53502) 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b36ffd640 1 -- 172.21.15.76:0/1762595158 learned_addr learned my addr 172.21.15.76:0/1762595158 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b37fff640 1 --2- 172.21.15.76:0/1762595158 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b380af2a0 0x7f3b3806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b36ffd640 1 -- 172.21.15.76:0/1762595158 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b380af2a0 msgr2=0x7f3b3806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b36ffd640 1 --2- 172.21.15.76:0/1762595158 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b380af2a0 0x7f3b3806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b36ffd640 1 -- 172.21.15.76:0/1762595158 >> v1:172.21.15.116:6789/0 conn(0x7f3b3815bec0 legacy=0x7f3b380ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b36ffd640 1 -- 172.21.15.76:0/1762595158 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b380b3b30 con 0x7f3b3815c2a0 2024-09-19T04:13:30.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.115+0000 7f3b37fff640 1 --2- 172.21.15.76:0/1762595158 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b380af2a0 0x7f3b3806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:30.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b36ffd640 1 --2- 172.21.15.76:0/1762595158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b380aed60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009870 tx=0x7f3b2001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcd5dd708643222 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b367fc640 1 -- 172.21.15.76:0/1762595158 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b200aa610 con 0x7f3b3815c2a0 2024-09-19T04:13:30.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b367fc640 1 -- 172.21.15.76:0/1762595158 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b200af070 con 0x7f3b3815c2a0 2024-09-19T04:13:30.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b367fc640 1 -- 172.21.15.76:0/1762595158 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b200aa9d0 con 0x7f3b3815c2a0 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1762595158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 msgr2=0x7f3b380aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1762595158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b380aed60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3b20009870 tx=0x7f3b2001c4f0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1762595158 shutdown_connections 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1762595158 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b380af2a0 0x7f3b3806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1762595158 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b380aed60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1762595158 >> 172.21.15.76:0/1762595158 conn(0x7f3b380b1f90 msgr2=0x7f3b380b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1762595158 shutdown_connections 2024-09-19T04:13:30.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.119+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1762595158 wait complete. 2024-09-19T04:13:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 Processor -- start 2024-09-19T04:13:30.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 -- start start 2024-09-19T04:13:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b380af2a0 0x7f3b38164000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b3815bec0 0x7f3b381520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b3814b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b3806ed50 con 0x7f3b3815c2a0 2024-09-19T04:13:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b380af6b0 con 0x7f3b380af2a0 2024-09-19T04:13:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3b3806eed0 con 0x7f3b3815bec0 2024-09-19T04:13:30.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b3814b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b3814b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53506/0 (socket says 172.21.15.76:53506) 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 -- 172.21.15.76:0/1988388752 learned_addr learned my addr 172.21.15.76:0/1988388752 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b377fe640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b380af2a0 0x7f3b38164000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b36ffd640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b3815bec0 0x7f3b381520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 -- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b3815bec0 msgr2=0x7f3b381520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b3815bec0 0x7f3b381520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 -- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b380af2a0 msgr2=0x7f3b38164000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b380af2a0 0x7f3b38164000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 -- 172.21.15.76:0/1988388752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b38156670 con 0x7f3b3815c2a0 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b36ffd640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b3815bec0 0x7f3b381520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:30.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b377fe640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b380af2a0 0x7f3b38164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:30.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b37fff640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b3814b520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c7f0 tx=0x7f3b3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b2ffff640 1 -- 172.21.15.76:0/1988388752 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30015070 con 0x7f3b3815c2a0 2024-09-19T04:13:30.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b2ffff640 1 -- 172.21.15.76:0/1988388752 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b30007860 con 0x7f3b3815c2a0 2024-09-19T04:13:30.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b2ffff640 1 -- 172.21.15.76:0/1988388752 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b30007c50 con 0x7f3b3815c2a0 2024-09-19T04:13:30.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b38071fb0 con 0x7f3b3815c2a0 2024-09-19T04:13:30.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b38156950 con 0x7f3b3815c2a0 2024-09-19T04:13:30.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b3806e000 con 0x7f3b3815c2a0 2024-09-19T04:13:30.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b2ffff640 1 -- 172.21.15.76:0/1988388752 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b30018070 con 0x7f3b3815c2a0 2024-09-19T04:13:30.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.123+0000 7f3b2ffff640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b180678c0 0x7f3b18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.127+0000 7f3b2ffff640 1 -- 172.21.15.76:0/1988388752 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3b30086100 con 0x7f3b3815c2a0 2024-09-19T04:13:30.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.127+0000 7f3b377fe640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b180678c0 0x7f3b18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.127+0000 7f3b377fe640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b180678c0 0x7f3b18069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b280045c0 tx=0x7f3b28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.131+0000 7f3b2ffff640 1 -- 172.21.15.76:0/1988388752 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3b30056990 con 0x7f3b3815c2a0 2024-09-19T04:13:30.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.327+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b38166dd0 con 0x7f3b3815c2a0 2024-09-19T04:13:30.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b2ffff640 1 -- 172.21.15.76:0/1988388752 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5219 (secure 0 0 0) 0x7f3b30056290 con 0x7f3b3815c2a0 2024-09-19T04:13:30.332 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:30.333 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":6,"btime":"2024-09-19T04:13:29:964662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:29.326305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:13:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 6 2024-09-19T04:13:30.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b180678c0 msgr2=0x7f3b18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b180678c0 0x7f3b18069d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3b280045c0 tx=0x7f3b28009040 comp rx=0 tx=0).stop 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 msgr2=0x7f3b3814b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b3814b520 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000c7f0 tx=0x7f3b3000ccb0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 shutdown_connections 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b180678c0 0x7f3b18069d80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b3815c2a0 0x7f3b3814b520 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b3815bec0 0x7f3b381520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 --2- 172.21.15.76:0/1988388752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b380af2a0 0x7f3b38164000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 >> 172.21.15.76:0/1988388752 conn(0x7f3b380b1f90 msgr2=0x7f3b380b12f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 shutdown_connections 2024-09-19T04:13:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.331+0000 7f3b3e04b640 1 -- 172.21.15.76:0/1988388752 wait complete. 2024-09-19T04:13:30.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:30.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28d11e5640 1 Processor -- start 2024-09-19T04:13:30.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28d11e5640 1 -- start start 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28d11e5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f28cc1457a0 0x7f28cc0d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28d11e5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28cc0cf800 0x7f28cc0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28d11e5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc0d27a0 con 0x7f28cc0b4340 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28d11e5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc0d1720 con 0x7f28cc0cf800 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28d11e5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc0b4790 con 0x7f28cc1457a0 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28ca59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28cc0cf800 0x7f28cc0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28ca59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f28cc0cf800 0x7f28cc0b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53522/0 (socket says 172.21.15.76:53522) 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28ca59c640 1 -- 172.21.15.76:0/1768033849 learned_addr learned my addr 172.21.15.76:0/1768033849 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:30.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28ca59c640 1 -- 172.21.15.76:0/1768033849 >> v2:172.21.15.116:3300/0 conn(0x7f28cc1457a0 msgr2=0x7f28cc0d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:30.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28cad9d640 1 --2- 172.21.15.76:0/1768033849 >> v2:172.21.15.116:3300/0 conn(0x7f28cc1457a0 0x7f28cc0d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28ca59c640 1 --2- 172.21.15.76:0/1768033849 >> v2:172.21.15.116:3300/0 conn(0x7f28cc1457a0 0x7f28cc0d1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28ca59c640 1 -- 172.21.15.76:0/1768033849 >> v1:172.21.15.116:6789/0 conn(0x7f28cc0b4340 legacy=0x7f28cc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:30.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.459+0000 7f28ca59c640 1 -- 172.21.15.76:0/1768033849 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc0d2a80 con 0x7f28cc0cf800 2024-09-19T04:13:30.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28ca59c640 1 --2- 172.21.15.76:0/1768033849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cf800 0x7f28cc0b3e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f28cc068500 tx=0x7f28c001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e87380824c403ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28c9d9b640 1 -- 172.21.15.76:0/1768033849 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00b0070 con 0x7f28cc0cf800 2024-09-19T04:13:30.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28c9d9b640 1 -- 172.21.15.76:0/1768033849 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c00ab400 con 0x7f28cc0cf800 2024-09-19T04:13:30.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28c9d9b640 1 -- 172.21.15.76:0/1768033849 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c00ab7d0 con 0x7f28cc0cf800 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- 172.21.15.76:0/1768033849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cf800 msgr2=0x7f28cc0b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1768033849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cf800 0x7f28cc0b3e00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f28cc068500 tx=0x7f28c001c450 comp rx=0 tx=0).stop 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- 172.21.15.76:0/1768033849 shutdown_connections 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1768033849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc0cf800 0x7f28cc0b3e00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1768033849 >> v2:172.21.15.116:3300/0 conn(0x7f28cc1457a0 0x7f28cc0d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- 172.21.15.76:0/1768033849 >> 172.21.15.76:0/1768033849 conn(0x7f28cc06bcc0 msgr2=0x7f28cc06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- 172.21.15.76:0/1768033849 shutdown_connections 2024-09-19T04:13:30.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- 172.21.15.76:0/1768033849 wait complete. 2024-09-19T04:13:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 Processor -- start 2024-09-19T04:13:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- start start 2024-09-19T04:13:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f28cc0b4340 0x7f28cc13f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28cc0cf800 0x7f28cc141140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc1457a0 0x7f28cc13b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0b52f0 con 0x7f28cc1457a0 2024-09-19T04:13:30.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc0b5170 con 0x7f28cc0cf800 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc1690b0 con 0x7f28cc0b4340 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc1457a0 0x7f28cc13b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc1457a0 0x7f28cc13b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53528/0 (socket says 172.21.15.76:53528) 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 -- 172.21.15.76:0/1174448537 learned_addr learned my addr 172.21.15.76:0/1174448537 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28ca59c640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28cc0cf800 0x7f28cc141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cad9d640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f28cc0b4340 0x7f28cc13f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 -- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f28cc0b4340 msgr2=0x7f28cc13f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f28cc0b4340 0x7f28cc13f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 -- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28cc0cf800 msgr2=0x7f28cc141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28cc0cf800 0x7f28cc141140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 -- 172.21.15.76:0/1174448537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc0cbb30 con 0x7f28cc1457a0 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28ca59c640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28cc0cf800 0x7f28cc141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28cb59e640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc1457a0 0x7f28cc13b050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c780 tx=0x7f28c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28b37fe640 1 -- 172.21.15.76:0/1174448537 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4017070 con 0x7f28cc1457a0 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28b37fe640 1 -- 172.21.15.76:0/1174448537 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c4003db0 con 0x7f28cc1457a0 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28b37fe640 1 -- 172.21.15.76:0/1174448537 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40052b0 con 0x7f28cc1457a0 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc0cbe10 con 0x7f28cc1457a0 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.463+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc0cc0f0 con 0x7f28cc1457a0 2024-09-19T04:13:30.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.467+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc0b3e00 con 0x7f28cc1457a0 2024-09-19T04:13:30.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.467+0000 7f28b37fe640 1 -- 172.21.15.76:0/1174448537 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f28c401a070 con 0x7f28cc1457a0 2024-09-19T04:13:30.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.467+0000 7f28b37fe640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f28b80678c0 0x7f28b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.467+0000 7f28b37fe640 1 -- 172.21.15.76:0/1174448537 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f28c40855f0 con 0x7f28cc1457a0 2024-09-19T04:13:30.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.467+0000 7f28cad9d640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f28b80678c0 0x7f28b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.467+0000 7f28cad9d640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f28b80678c0 0x7f28b8069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00b7c0 tx=0x7f28bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.471+0000 7f28b37fe640 1 -- 172.21.15.76:0/1174448537 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f28c4055e80 con 0x7f28cc1457a0 2024-09-19T04:13:30.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.631+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28cc0ccd50 con 0x7f28cc1457a0 2024-09-19T04:13:30.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28b37fe640 1 -- 172.21.15.76:0/1174448537 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f28c4055780 con 0x7f28cc1457a0 2024-09-19T04:13:30.637 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:30.637 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f28b80678c0 msgr2=0x7f28b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f28b80678c0 0x7f28b8069d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00b7c0 tx=0x7f28bc008040 comp rx=0 tx=0).stop 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc1457a0 msgr2=0x7f28cc13b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc1457a0 0x7f28cc13b050 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c780 tx=0x7f28c400cc40 comp rx=0 tx=0).stop 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 shutdown_connections 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f28b80678c0 0x7f28b8069d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f28cc1457a0 0x7f28cc13b050 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f28cc0cf800 0x7f28cc141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 --2- 172.21.15.76:0/1174448537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f28cc0b4340 0x7f28cc13f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 >> 172.21.15.76:0/1174448537 conn(0x7f28cc06bcc0 msgr2=0x7f28cc15d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 shutdown_connections 2024-09-19T04:13:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.635+0000 7f28d11e5640 1 -- 172.21.15.76:0/1174448537 wait complete. 2024-09-19T04:13:30.657 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.771+0000 7f8cae69a640 1 Processor -- start 2024-09-19T04:13:30.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cae69a640 1 -- start start 2024-09-19T04:13:30.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cae69a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ca815e3e0 0x7f8ca815f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cae69a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ca80b3300 0x7f8ca80b3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cae69a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ca80b0770 con 0x7f8ca8072dd0 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cae69a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca80b3c90 con 0x7f8ca80b3300 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cae69a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8ca80b0a50 con 0x7f8ca815e3e0 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cacc37640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ca80b3300 0x7f8ca80b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8ca77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8ca8072dd0 0x7f8ca80aff60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47300/0 (socket says 172.21.15.76:47300) 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8ca77fe640 1 -- 172.21.15.76:0/3672756525 learned_addr learned my addr 172.21.15.76:0/3672756525 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8ca7fff640 1 --2- 172.21.15.76:0/3672756525 >> v2:172.21.15.116:3300/0 conn(0x7f8ca815e3e0 0x7f8ca815f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8ca6ffd640 1 -- 172.21.15.76:0/3672756525 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112467523 0 0) 0x7f8ca80b0770 con 0x7f8ca8072dd0 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8ca6ffd640 1 -- 172.21.15.76:0/3672756525 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c90003530 con 0x7f8ca8072dd0 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cacc37640 1 -- 172.21.15.76:0/3672756525 >> v2:172.21.15.116:3300/0 conn(0x7f8ca815e3e0 msgr2=0x7f8ca815f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:30.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cacc37640 1 --2- 172.21.15.76:0/3672756525 >> v2:172.21.15.116:3300/0 conn(0x7f8ca815e3e0 0x7f8ca815f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cacc37640 1 -- 172.21.15.76:0/3672756525 >> v1:172.21.15.76:6789/0 conn(0x7f8ca8072dd0 legacy=0x7f8ca80aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.775+0000 7f8cacc37640 1 -- 172.21.15.76:0/3672756525 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca815eb10 con 0x7f8ca80b3300 2024-09-19T04:13:30.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cacc37640 1 --2- 172.21.15.76:0/3672756525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca80b3750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ca806eee0 tx=0x7f8ca001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd4cc329401219fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8ca6ffd640 1 -- 172.21.15.76:0/3672756525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00b2070 con 0x7f8ca80b3300 2024-09-19T04:13:30.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8ca6ffd640 1 -- 172.21.15.76:0/3672756525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ca00a5870 con 0x7f8ca80b3300 2024-09-19T04:13:30.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8ca6ffd640 1 -- 172.21.15.76:0/3672756525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca00a5c20 con 0x7f8ca80b3300 2024-09-19T04:13:30.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- 172.21.15.76:0/3672756525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 msgr2=0x7f8ca80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 --2- 172.21.15.76:0/3672756525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca80b3750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8ca806eee0 tx=0x7f8ca001c8a0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- 172.21.15.76:0/3672756525 shutdown_connections 2024-09-19T04:13:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 --2- 172.21.15.76:0/3672756525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca80b3750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 --2- 172.21.15.76:0/3672756525 >> v2:172.21.15.116:3300/0 conn(0x7f8ca815e3e0 0x7f8ca815f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- 172.21.15.76:0/3672756525 >> 172.21.15.76:0/3672756525 conn(0x7f8ca80b6870 msgr2=0x7f8ca80b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- 172.21.15.76:0/3672756525 shutdown_connections 2024-09-19T04:13:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- 172.21.15.76:0/3672756525 wait complete. 2024-09-19T04:13:30.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 Processor -- start 2024-09-19T04:13:30.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- start start 2024-09-19T04:13:30.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ca8072dd0 0x7f8ca815c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca8151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ca815e3e0 0x7f8ca815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca8162a40 con 0x7f8ca80b3300 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca81628c0 con 0x7f8ca8072dd0 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8cae69a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ca816af90 con 0x7f8ca815e3e0 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca8151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8ca77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca8151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53538/0 (socket says 172.21.15.76:53538) 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.779+0000 7f8ca77fe640 1 -- 172.21.15.76:0/321520495 learned_addr learned my addr 172.21.15.76:0/321520495 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca77fe640 1 -- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ca815e3e0 msgr2=0x7f8ca815ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8cacc37640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ca815e3e0 0x7f8ca815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca7fff640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ca8072dd0 0x7f8ca815c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca77fe640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ca815e3e0 0x7f8ca815ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca77fe640 1 -- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ca8072dd0 msgr2=0x7f8ca815c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca77fe640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ca8072dd0 0x7f8ca815c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca77fe640 1 -- 172.21.15.76:0/321520495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca815c860 con 0x7f8ca80b3300 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca77fe640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca8151450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c480 tx=0x7f8c9c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca4ff9640 1 -- 172.21.15.76:0/321520495 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c017070 con 0x7f8ca80b3300 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca4ff9640 1 -- 172.21.15.76:0/321520495 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c9c005cf0 con 0x7f8ca80b3300 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca815cb40 con 0x7f8ca80b3300 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca4ff9640 1 -- 172.21.15.76:0/321520495 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c007290 con 0x7f8ca80b3300 2024-09-19T04:13:30.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca815d100 con 0x7f8ca80b3300 2024-09-19T04:13:30.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca80b3750 con 0x7f8ca80b3300 2024-09-19T04:13:30.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca4ff9640 1 -- 172.21.15.76:0/321520495 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c9c01a070 con 0x7f8ca80b3300 2024-09-19T04:13:30.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca4ff9640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c940678f0 0x7f8c94069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:30.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca4ff9640 1 -- 172.21.15.76:0/321520495 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8c9c085330 con 0x7f8ca80b3300 2024-09-19T04:13:30.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca7fff640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c940678f0 0x7f8c94069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:30.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.783+0000 7f8ca7fff640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c940678f0 0x7f8c94069db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8160f20 tx=0x7f8c98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:30.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.787+0000 7f8ca4ff9640 1 -- 172.21.15.76:0/321520495 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8c9c055bc0 con 0x7f8ca80b3300 2024-09-19T04:13:30.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.951+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ca80d03b0 con 0x7f8ca80b3300 2024-09-19T04:13:30.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8ca4ff9640 1 -- 172.21.15.76:0/321520495 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f8c9c0554c0 con 0x7f8ca80b3300 2024-09-19T04:13:30.956 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:30.956 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c940678f0 msgr2=0x7f8c94069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c940678f0 0x7f8c94069db0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8160f20 tx=0x7f8c98008040 comp rx=0 tx=0).stop 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 msgr2=0x7f8ca8151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca8151450 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00c480 tx=0x7f8c9c00c940 comp rx=0 tx=0).stop 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 shutdown_connections 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c940678f0 0x7f8c94069db0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ca815e3e0 0x7f8ca815ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ca80b3300 0x7f8ca8151450 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 --2- 172.21.15.76:0/321520495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ca8072dd0 0x7f8ca815c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 >> 172.21.15.76:0/321520495 conn(0x7f8ca80b6870 msgr2=0x7f8ca80b4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:30.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 shutdown_connections 2024-09-19T04:13:30.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:30.955+0000 7f8cae69a640 1 -- 172.21.15.76:0/321520495 wait complete. 2024-09-19T04:13:30.993 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:31.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f5470979640 1 Processor -- start 2024-09-19T04:13:31.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f5470979640 1 -- start start 2024-09-19T04:13:31.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f5470979640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f546c15bec0 0x7f546c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f5470979640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f546c0b3290 con 0x7f546c0af470 2024-09-19T04:13:31.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f5470979640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f546c0b3570 con 0x7f546c15c2a0 2024-09-19T04:13:31.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f5470979640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f546c0b3850 con 0x7f546c15bec0 2024-09-19T04:13:31.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f546c0af470 0x7f546c06e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47306/0 (socket says 172.21.15.76:47306) 2024-09-19T04:13:31.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546ad9d640 1 -- 172.21.15.76:0/2425197066 learned_addr learned my addr 172.21.15.76:0/2425197066 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1533434212 0 0) 0x7f546c0b3290 con 0x7f546c0af470 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5458003530 con 0x7f546c0af470 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546a59c640 1 --2- 172.21.15.76:0/2425197066 >> v2:172.21.15.116:3300/0 conn(0x7f546c15bec0 0x7f546c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1310430417 0 0) 0x7f5458003530 con 0x7f546c0af470 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 >> v2:172.21.15.116:3300/0 conn(0x7f546c15bec0 msgr2=0x7f546c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 --2- 172.21.15.76:0/2425197066 >> v2:172.21.15.116:3300/0 conn(0x7f546c15bec0 0x7f546c0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 >> v1:172.21.15.116:6789/0 conn(0x7f546c15c2a0 legacy=0x7f546c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c0b3b30 con 0x7f546c0af470 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f545c001680 con 0x7f546c0af470 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 >> v1:172.21.15.76:6789/0 conn(0x7f546c0af470 legacy=0x7f546c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54580039c0 0x7f5458003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5458004810 0x7f5458004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54580051d0 0x7f545800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f546c0b3850 con 0x7f54580051d0 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f545800c120 con 0x7f54580039c0 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f545800c320 con 0x7f5458004810 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.111+0000 7f546ad9d640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54580051d0 0x7f545800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546a59c640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54580039c0 0x7f5458003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546ad9d640 1 -- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5458004810 msgr2=0x7f5458004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5469d9b640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5458004810 0x7f5458004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546ad9d640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5458004810 0x7f5458004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546ad9d640 1 -- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54580039c0 msgr2=0x7f5458003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546ad9d640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54580039c0 0x7f5458003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546ad9d640 1 -- 172.21.15.76:0/2425197066 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5458003530 con 0x7f54580051d0 2024-09-19T04:13:31.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546a59c640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54580039c0 0x7f5458003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546ad9d640 1 -- 172.21.15.76:0/2425197066 reap_dead start 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546ad9d640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54580051d0 0x7f545800b9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c1c0 tx=0x7f545c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f74e921eb182bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c007d00 con 0x7f54580051d0 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f545c0a9070 con 0x7f54580051d0 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f546959a640 1 -- 172.21.15.76:0/2425197066 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c0a5dc0 con 0x7f54580051d0 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5470979640 1 -- 172.21.15.76:0/2425197066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54580051d0 msgr2=0x7f545800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5470979640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54580051d0 0x7f545800b9d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f545c00c1c0 tx=0x7f545c01cc60 comp rx=0 tx=0).stop 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5470979640 1 -- 172.21.15.76:0/2425197066 shutdown_connections 2024-09-19T04:13:31.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5470979640 1 --2- 172.21.15.76:0/2425197066 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54580051d0 0x7f545800b9d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5470979640 1 -- 172.21.15.76:0/2425197066 >> 172.21.15.76:0/2425197066 conn(0x7f546c0b1f90 msgr2=0x7f546c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:31.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5470979640 1 -- 172.21.15.76:0/2425197066 shutdown_connections 2024-09-19T04:13:31.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.115+0000 7f5470979640 1 -- 172.21.15.76:0/2425197066 wait complete. 2024-09-19T04:13:31.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 Processor -- start 2024-09-19T04:13:31.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 -- start start 2024-09-19T04:13:31.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54580051d0 0x7f546c1f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f546c1500e0 0x7f546c152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f546c146790 0x7f546c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f546c0afc90 con 0x7f546c146790 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f546c0b40f0 con 0x7f546c1500e0 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f546c0aed60 con 0x7f54580051d0 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f546c146790 0x7f546c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f546c146790 0x7f546c0cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53550/0 (socket says 172.21.15.76:53550) 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 -- 172.21.15.76:0/230423775 learned_addr learned my addr 172.21.15.76:0/230423775 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546a59c640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54580051d0 0x7f546c1f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5469d9b640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f546c1500e0 0x7f546c152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 -- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54580051d0 msgr2=0x7f546c1f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54580051d0 0x7f546c1f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 -- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f546c1500e0 msgr2=0x7f546c152e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f546c1500e0 0x7f546c152e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 -- 172.21.15.76:0/230423775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f546c15dd80 con 0x7f546c146790 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f546ad9d640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f546c146790 0x7f546c0cfc10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f545c01d500 tx=0x7f5458005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5462ffd640 1 -- 172.21.15.76:0/230423775 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c007d00 con 0x7f546c146790 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5462ffd640 1 -- 172.21.15.76:0/230423775 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f545c0b4070 con 0x7f546c146790 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f546c1f24b0 con 0x7f546c146790 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5462ffd640 1 -- 172.21.15.76:0/230423775 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f545c001c40 con 0x7f546c146790 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f546c15e020 con 0x7f546c146790 2024-09-19T04:13:31.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f546c0af0f0 con 0x7f546c146790 2024-09-19T04:13:31.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5462ffd640 1 -- 172.21.15.76:0/230423775 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f545c004bf0 con 0x7f546c146790 2024-09-19T04:13:31.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.119+0000 7f5462ffd640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f544c0678c0 0x7f544c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.123+0000 7f5462ffd640 1 -- 172.21.15.76:0/230423775 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f545c11f310 con 0x7f546c146790 2024-09-19T04:13:31.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.123+0000 7f546a59c640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f544c0678c0 0x7f544c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.123+0000 7f546a59c640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f544c0678c0 0x7f544c069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fd0 tx=0x7f546400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.127+0000 7f5462ffd640 1 -- 172.21.15.76:0/230423775 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f545c016970 con 0x7f546c146790 2024-09-19T04:13:31.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.319+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f546c06fcd0 con 0x7f546c146790 2024-09-19T04:13:31.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.319+0000 7f5462ffd640 1 -- 172.21.15.76:0/230423775 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5219 (secure 0 0 0) 0x7f545c0efb10 con 0x7f546c146790 2024-09-19T04:13:31.322 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:31.322 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":6,"btime":"2024-09-19T04:13:29:964662+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:29.326305+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:13:31.322 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 6 2024-09-19T04:13:31.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f544c0678c0 msgr2=0x7f544c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f544c0678c0 0x7f544c069d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5464009fd0 tx=0x7f546400b040 comp rx=0 tx=0).stop 2024-09-19T04:13:31.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f546c146790 msgr2=0x7f546c0cfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f546c146790 0x7f546c0cfc10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f545c01d500 tx=0x7f5458005840 comp rx=0 tx=0).stop 2024-09-19T04:13:31.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 shutdown_connections 2024-09-19T04:13:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f544c0678c0 0x7f544c069d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f546c146790 0x7f546c0cfc10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f546c1500e0 0x7f546c152e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 --2- 172.21.15.76:0/230423775 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54580051d0 0x7f546c1f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 >> 172.21.15.76:0/230423775 conn(0x7f546c0b1f90 msgr2=0x7f546c1597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 shutdown_connections 2024-09-19T04:13:31.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.323+0000 7f5470979640 1 -- 172.21.15.76:0/230423775 wait complete. 2024-09-19T04:13:31.339 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:31.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.443+0000 7f075fe7b640 1 Processor -- start 2024-09-19T04:13:31.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.443+0000 7f075fe7b640 1 -- start start 2024-09-19T04:13:31.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075fe7b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 0x7f075806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075fe7b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0758070250 con 0x7f075806fe50 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075fe7b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07580aede0 con 0x7f075815f3a0 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075fe7b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0758152480 con 0x7f07581520a0 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075d416640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f075806fe50 0x7f075815ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47312/0 (socket says 172.21.15.76:47312) 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075d416640 1 -- 172.21.15.76:0/3030198600 learned_addr learned my addr 172.21.15.76:0/3030198600 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075dc17640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 0x7f075806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3148756382 0 0) 0x7f0758070250 con 0x7f075806fe50 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0740003530 con 0x7f075806fe50 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 330527540 0 0) 0x7f0740003530 con 0x7f075806fe50 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 msgr2=0x7f075806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 0x7f075806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 >> v1:172.21.15.116:6789/0 conn(0x7f075815f3a0 legacy=0x7f07580adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f075815fa10 con 0x7f075806fe50 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f074c001680 con 0x7f075806fe50 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 >> v1:172.21.15.76:6789/0 conn(0x7f075806fe50 legacy=0x7f075815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07400039f0 0x7f0740003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 0x7f0740004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0740004d50 0x7f07400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f074000ba80 con 0x7f0740004d50 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f074000bc00 con 0x7f07400039f0 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f074000be00 con 0x7f07581520a0 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075e418640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0740004d50 0x7f07400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075d416640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 0x7f0740004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075dc17640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07400039f0 0x7f0740003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075e418640 1 -- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 msgr2=0x7f0740004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075e418640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 0x7f0740004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075e418640 1 -- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07400039f0 msgr2=0x7f0740003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075e418640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07400039f0 0x7f0740003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075e418640 1 -- 172.21.15.76:0/3030198600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0740003530 con 0x7f0740004d50 2024-09-19T04:13:31.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.447+0000 7f075d416640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07581520a0 0x7f0740004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:31.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075e418640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0740004d50 0x7f07400051d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0750001750 tx=0x7f075001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acebe58132a418da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500013a0 con 0x7f0740004d50 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07500052f0 con 0x7f0740004d50 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075cc15640 1 -- 172.21.15.76:0/3030198600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750003840 con 0x7f0740004d50 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 -- 172.21.15.76:0/3030198600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0740004d50 msgr2=0x7f07400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 --2- 172.21.15.76:0/3030198600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0740004d50 0x7f07400051d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0750001750 tx=0x7f075001c3e0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075e418640 1 -- 172.21.15.76:0/3030198600 reap_dead start 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 -- 172.21.15.76:0/3030198600 shutdown_connections 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 -- 172.21.15.76:0/3030198600 >> 172.21.15.76:0/3030198600 conn(0x7f07580b1f90 msgr2=0x7f07580b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 -- 172.21.15.76:0/3030198600 shutdown_connections 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 -- 172.21.15.76:0/3030198600 wait complete. 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 Processor -- start 2024-09-19T04:13:31.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.451+0000 7f075fe7b640 1 -- start start 2024-09-19T04:13:31.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0740004d50 0x7f0758148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f075815da90 0x7f07580ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07580cf260 0x7f075814bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0758073770 con 0x7f07580cf260 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0758073490 con 0x7f0740004d50 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0758073a50 con 0x7f075815da90 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07580cf260 0x7f075814bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07580cf260 0x7f075814bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53566/0 (socket says 172.21.15.76:53566) 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 -- 172.21.15.76:0/2020168435 learned_addr learned my addr 172.21.15.76:0/2020168435 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075dc17640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0740004d50 0x7f0758148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075d416640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f075815da90 0x7f07580ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 -- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f075815da90 msgr2=0x7f07580ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f075815da90 0x7f07580ced20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 -- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0740004d50 msgr2=0x7f0758148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0740004d50 0x7f0758148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 -- 172.21.15.76:0/2020168435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0758148ac0 con 0x7f07580cf260 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075dc17640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0740004d50 0x7f0758148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075e418640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07580cf260 0x7f075814bdf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0750005040 tx=0x7f0740005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f07567fc640 1 -- 172.21.15.76:0/2020168435 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07500013a0 con 0x7f07580cf260 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f07567fc640 1 -- 172.21.15.76:0/2020168435 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07500054a0 con 0x7f07580cf260 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f07567fc640 1 -- 172.21.15.76:0/2020168435 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750007a40 con 0x7f07580cf260 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f075815e450 con 0x7f07580cf260 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07580cdf60 con 0x7f07580cf260 2024-09-19T04:13:31.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0758073d30 con 0x7f07580cf260 2024-09-19T04:13:31.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f07567fc640 1 -- 172.21.15.76:0/2020168435 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f07500b4040 con 0x7f07580cf260 2024-09-19T04:13:31.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f07567fc640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07440678c0 0x7f0744069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:31.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f07567fc640 1 -- 172.21.15.76:0/2020168435 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f07500bc070 con 0x7f07580cf260 2024-09-19T04:13:31.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075dc17640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07440678c0 0x7f0744069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:31.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.455+0000 7f075dc17640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07440678c0 0x7f0744069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f075815e150 tx=0x7f074800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:31.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.463+0000 7f07567fc640 1 -- 172.21.15.76:0/2020168435 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f07500160f0 con 0x7f07580cf260 2024-09-19T04:13:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.627+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f075806e540 con 0x7f07580cf260 2024-09-19T04:13:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.627+0000 7f07567fc640 1 -- 172.21.15.76:0/2020168435 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f07500efb60 con 0x7f07580cf260 2024-09-19T04:13:31.632 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:31.632 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07440678c0 msgr2=0x7f0744069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07440678c0 0x7f0744069d80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f075815e150 tx=0x7f074800b040 comp rx=0 tx=0).stop 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07580cf260 msgr2=0x7f075814bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07580cf260 0x7f075814bdf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0750005040 tx=0x7f0740005840 comp rx=0 tx=0).stop 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 shutdown_connections 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07440678c0 0x7f0744069d80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07580cf260 0x7f075814bdf0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f075815da90 0x7f07580ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 --2- 172.21.15.76:0/2020168435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0740004d50 0x7f0758148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 >> 172.21.15.76:0/2020168435 conn(0x7f07580b1f90 msgr2=0x7f0758150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 shutdown_connections 2024-09-19T04:13:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:31.631+0000 7f075fe7b640 1 -- 172.21.15.76:0/2020168435 wait complete. 2024-09-19T04:13:32.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:13:24.253935+0000', 'modified': '2024-09-19T04:13:29.326305+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4260': {'gid': 4260, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3901169819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3901169819}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3901169819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4260, 'qdb_cluster': [4260]} 2024-09-19T04:13:32.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:13:32.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:13:32.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc3ed1640 1 Processor -- start 2024-09-19T04:13:32.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc3ed1640 1 -- start start 2024-09-19T04:13:32.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc3ed1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6cbc1520a0 0x7f6cbc06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc3ed1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6cbc15f3a0 0x7f6cbc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc3ed1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cbc070250 con 0x7f6cbc06fe50 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc3ed1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc152480 con 0x7f6cbc1520a0 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc3ed1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6cbc0aede0 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc1c6d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6cbc1520a0 0x7f6cbc06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc1c6d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6cbc1520a0 0x7f6cbc06f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53582/0 (socket says 172.21.15.76:53582) 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/1266994579 learned_addr learned my addr 172.21.15.76:0/1266994579 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc246e640 1 --2- 172.21.15.76:0/1266994579 >> v2:172.21.15.116:3300/0 conn(0x7f6cbc15f3a0 0x7f6cbc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/1266994579 >> v2:172.21.15.116:3300/0 conn(0x7f6cbc15f3a0 msgr2=0x7f6cbc0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.183+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/1266994579 >> v2:172.21.15.116:3300/0 conn(0x7f6cbc15f3a0 0x7f6cbc0adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/1266994579 >> v1:172.21.15.76:6789/0 conn(0x7f6cbc06fe50 legacy=0x7f6cbc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/1266994579 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc15fa10 con 0x7f6cbc1520a0 2024-09-19T04:13:32.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/1266994579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc1520a0 0x7f6cbc06f910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc155b40 tx=0x7f6cb801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881e692de89f4877 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc0c6b640 1 -- 172.21.15.76:0/1266994579 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ab610 con 0x7f6cbc1520a0 2024-09-19T04:13:32.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc0c6b640 1 -- 172.21.15.76:0/1266994579 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb80b0070 con 0x7f6cbc1520a0 2024-09-19T04:13:32.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc0c6b640 1 -- 172.21.15.76:0/1266994579 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80ab9d0 con 0x7f6cbc1520a0 2024-09-19T04:13:32.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1266994579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc1520a0 msgr2=0x7f6cbc06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1266994579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc1520a0 0x7f6cbc06f910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc155b40 tx=0x7f6cb801c520 comp rx=0 tx=0).stop 2024-09-19T04:13:32.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1266994579 shutdown_connections 2024-09-19T04:13:32.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1266994579 >> v2:172.21.15.116:3300/0 conn(0x7f6cbc15f3a0 0x7f6cbc0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1266994579 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc1520a0 0x7f6cbc06f910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1266994579 >> 172.21.15.76:0/1266994579 conn(0x7f6cbc0b1f90 msgr2=0x7f6cbc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1266994579 shutdown_connections 2024-09-19T04:13:32.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.187+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1266994579 wait complete. 2024-09-19T04:13:32.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 Processor -- start 2024-09-19T04:13:32.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 -- start start 2024-09-19T04:13:32.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cbc06fe50 0x7f6cbc0ce840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cbc1520a0 0x7f6cbc14d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc15f3a0 0x7f6cbc0b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc073d30 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc073a50 con 0x7f6cbc06fe50 2024-09-19T04:13:32.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc167f10 con 0x7f6cbc1520a0 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc15f3a0 0x7f6cbc0b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc15f3a0 0x7f6cbc0b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53584/0 (socket says 172.21.15.76:53584) 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 -- 172.21.15.76:0/1231463420 learned_addr learned my addr 172.21.15.76:0/1231463420 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cbc06fe50 0x7f6cbc0ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc146c640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cbc1520a0 0x7f6cbc14d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 -- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cbc1520a0 msgr2=0x7f6cbc14d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cbc1520a0 0x7f6cbc14d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 -- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cbc06fe50 msgr2=0x7f6cbc0ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cbc06fe50 0x7f6cbc0ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 -- 172.21.15.76:0/1231463420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cbc14da90 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cbc06fe50 0x7f6cbc0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:32.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc246e640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc15f3a0 0x7f6cbc0b0f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4007e70 tx=0x7f6cb400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cb27fc640 1 -- 172.21.15.76:0/1231463420 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4014620 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cb27fc640 1 -- 172.21.15.76:0/1231463420 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb4019070 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cbc15e140 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cb27fc640 1 -- 172.21.15.76:0/1231463420 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4014a20 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cbc0ce040 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cb27fc640 1 -- 172.21.15.76:0/1231463420 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6cb4014bc0 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cb27fc640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ca00678c0 0x7f6ca0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cb27fc640 1 -- 172.21.15.76:0/1231463420 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6cb40868f0 con 0x7f6cbc15f3a0 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.191+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ca00678c0 0x7f6ca0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ca00678c0 0x7f6ca0069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af000 tx=0x7f6cb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 Processor -- start 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- start start 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eab39f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 0x55b6eaaefd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eaaf0290 0x55b6eaaf0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea9d7b60 con 0x55b6eab39b20 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea93ec70 con 0x55b6eaaf0290 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b6ea8e4ed0 con 0x55b6eab3a440 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eab39f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eab39f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53594/0 (socket says 172.21.15.76:53594) 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/612858845 learned_addr learned my addr 172.21.15.76:0/612858845 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/612858845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 msgr2=0x55b6eaaefd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/612858845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 0x55b6eaaefd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/612858845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eaaf0290 msgr2=0x55b6eaaf0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/612858845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eaaf0290 0x55b6eaaf0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/612858845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ea8f6e50 con 0x55b6eab39b20 2024-09-19T04:13:32.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/612858845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eab39f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8008010 tx=0x7f6cb80a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4d46bcf98cfe91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6c9f7fe640 1 -- 172.21.15.76:0/612858845 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80c6070 con 0x55b6eab39b20 2024-09-19T04:13:32.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6c9f7fe640 1 -- 172.21.15.76:0/612858845 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb80b0040 con 0x55b6eab39b20 2024-09-19T04:13:32.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6c9f7fe640 1 -- 172.21.15.76:0/612858845 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80b4930 con 0x55b6eab39b20 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/612858845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 msgr2=0x55b6eab39f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/612858845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eab39f00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8008010 tx=0x7f6cb80a7000 comp rx=0 tx=0).stop 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/612858845 shutdown_connections 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/612858845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eaaf0290 0x55b6eaaf0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/612858845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 0x55b6eaaefd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/612858845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eab39f00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/612858845 >> 172.21.15.76:0/612858845 conn(0x55b6eaaf6600 msgr2=0x55b6eaaee630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/612858845 shutdown_connections 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/612858845 wait complete. 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 -- start start 2024-09-19T04:13:32.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.195+0000 7f6cc6f8d000 1 Processor -- start 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab39b20 0x55b6eaaf4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 0x55b6eaafacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eaafb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea9a0b00 con 0x55b6eaaf5330 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea919360 con 0x55b6eab39b20 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b6ea9162d0 con 0x55b6eab3a440 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 10 client.0 _ll_register_callbacks cb 0x55b6eaadb820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eaafb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eaafb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53602/0 (socket says 172.21.15.76:53602) 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 -- 172.21.15.76:0/983174751 learned_addr learned my addr 172.21.15.76:0/983174751 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 -- 172.21.15.76:0/983174751 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 msgr2=0x55b6eaafacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 0x55b6eaafacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 -- 172.21.15.76:0/983174751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab39b20 msgr2=0x55b6eaaf4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab39b20 0x55b6eaaf4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 -- 172.21.15.76:0/983174751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ea8e5120 con 0x55b6eaaf5330 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc246e640 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eaafb230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4052e50 tx=0x7f6cb40569f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd4bc30f1cc3adc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 1 -- 172.21.15.76:0/983174751 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb405d070 con 0x55b6eaaf5330 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 1 -- 172.21.15.76:0/983174751 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6cb405d210 con 0x55b6eaaf5330 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb80b0040 con 0x55b6eaaf5330 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 1 -- 172.21.15.76:0/983174751 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f6cb405f140 con 0x55b6eaaf5330 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 10 client.4383 fetch_fsmap learned FSMap version 6 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55b6ea8de450 con 0x55b6eaaf5330 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 1 -- 172.21.15.76:0/983174751 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb80b0040 con 0x55b6eaaf5330 2024-09-19T04:13:32.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 1 -- 172.21.15.76:0/983174751 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f6cb4055e40 con 0x55b6eaaf5330 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 10 client.4383 fetch_fsmap finished waiting for FSMap version 6 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 10 client.4383 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 20 client.4383 populate_metadata read hostname 'smithi076' 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab3ff50 0x55b6eab40330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 4 client.4383 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 --> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ea8d8c00 con 0x55b6eab3ff50 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab3ff50 0x55b6eab40330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab3ff50 0x55b6eab40330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=587885263efa91fa server_cookie=d0eda2757ed67d43 in_seq=0 out_seq=0 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 10 client.4383 ms_handle_connect on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 1 -- 172.21.15.76:0/983174751 <== mds.0 v2:172.21.15.76:6834/3901169819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6ea8d8c00 con 0x55b6eab3ff50 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 10 client.4383 handle_command_reply: tid=0 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 client.4383 shutdown 2024-09-19T04:13:32.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 msgr2=0x55b6eaafb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eaafb230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4052e50 tx=0x7f6cb40569f0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 shutdown_connections 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab3ff50 0x55b6eab40330 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eaafb230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab3a440 0x55b6eaafacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6c9cff9640 0 client.4383 ms_handle_reset on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/983174751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab39b20 0x55b6eaaf4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 >> 172.21.15.76:0/983174751 conn(0x55b6eaaf0290 msgr2=0x55b6eaaeed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 shutdown_connections 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/983174751 wait complete. 2024-09-19T04:13:32.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.199+0000 7f6cc6f8d000 20 client.4383 trim_cache size 0 max 16384 2024-09-19T04:13:32.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 Processor -- start 2024-09-19T04:13:32.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- start start 2024-09-19T04:13:32.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eab50850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab39b20 0x55b6eab38cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 0x55b6eab39230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc203ee0 con 0x55b6eaaf5330 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6cbc072910 con 0x55b6eab3a440 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6cbc203890 con 0x55b6eab39b20 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eab50850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eab50850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53606/0 (socket says 172.21.15.76:53606) 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/4229994400 learned_addr learned my addr 172.21.15.76:0/4229994400 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/4229994400 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab39b20 msgr2=0x55b6eab38cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/4229994400 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab39b20 0x55b6eab38cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/4229994400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 msgr2=0x55b6eab39230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/4229994400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 0x55b6eab39230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 -- 172.21.15.76:0/4229994400 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ea8c5fd0 con 0x55b6eaaf5330 2024-09-19T04:13:32.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/4229994400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eab50850 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cb801ca00 tx=0x7f6cb80046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788c52a926de6565 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6c9effd640 1 -- 172.21.15.76:0/4229994400 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80d0040 con 0x55b6eaaf5330 2024-09-19T04:13:32.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6c9effd640 1 -- 172.21.15.76:0/4229994400 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb80b0040 con 0x55b6eaaf5330 2024-09-19T04:13:32.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6c9effd640 1 -- 172.21.15.76:0/4229994400 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80c6040 con 0x55b6eaaf5330 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/4229994400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 msgr2=0x55b6eab50850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/4229994400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eab50850 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6cb801ca00 tx=0x7f6cb80046d0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/4229994400 shutdown_connections 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/4229994400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 0x55b6eab39230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/4229994400 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eab39b20 0x55b6eab38cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/4229994400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eaaf5330 0x55b6eab50850 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/4229994400 >> 172.21.15.76:0/4229994400 conn(0x55b6eaaf0290 msgr2=0x55b6eaaf4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/4229994400 shutdown_connections 2024-09-19T04:13:32.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.219+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/4229994400 wait complete. 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- start start 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 Processor -- start 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eaaf5330 0x55b6eabacfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eabad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 0x55b6eab437f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea93df80 con 0x55b6eab39b20 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b6ea8f6ca0 con 0x55b6eab3a440 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b6ea9432a0 con 0x55b6eaaf5330 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 10 client.0 _ll_register_callbacks cb 0x7f6cb4052d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eabad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eabad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53614/0 (socket says 172.21.15.76:53614) 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 -- 172.21.15.76:0/394122768 learned_addr learned my addr 172.21.15.76:0/394122768 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 -- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eaaf5330 msgr2=0x55b6eabacfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc246e640 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 0x55b6eab437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eaaf5330 0x55b6eabacfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eaaf5330 0x55b6eabacfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 -- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 msgr2=0x55b6eab437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 0x55b6eab437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 -- 172.21.15.76:0/394122768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ea9c1750 con 0x55b6eab39b20 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc146c640 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eabad500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cac001b10 tx=0x7f6cac00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de39edfee18bbd26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 1 -- 172.21.15.76:0/394122768 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cac018060 con 0x55b6eab39b20 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 1 -- 172.21.15.76:0/394122768 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6cac0050d0 con 0x55b6eab39b20 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb80b0040 con 0x55b6eab39b20 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 1 -- 172.21.15.76:0/394122768 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f6cac01e050 con 0x55b6eab39b20 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 10 client.4389 fetch_fsmap learned FSMap version 6 2024-09-19T04:13:32.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55b6ea8cca30 con 0x55b6eab39b20 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 1 -- 172.21.15.76:0/394122768 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cb80b0040 con 0x55b6eab39b20 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 1 -- 172.21.15.76:0/394122768 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 6) ==== 3510+0+0 (secure 0 0 0) 0x7f6cac021020 con 0x55b6eab39b20 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 10 client.4389 fetch_fsmap finished waiting for FSMap version 6 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 10 client.4389 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 20 client.4389 populate_metadata read hostname 'smithi076' 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab43d30 0x55b6eab44110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 4 client.4389 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 --> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] -- command(tid 0: {"prefix": "status"}) -- 0x55b6ea94c3a0 con 0x55b6eab43d30 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab43d30 0x55b6eab44110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 10 client.4389 ms_handle_connect on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc1c6d640 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab43d30 0x55b6eab44110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea87dd2dd177fa5b server_cookie=b6088f8dfff1f0f5 in_seq=0 out_seq=0 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 1 -- 172.21.15.76:0/394122768 <== mds.0 v2:172.21.15.76:6834/3901169819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55b6ea94c3a0 con 0x55b6eab43d30 2024-09-19T04:13:32.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 10 client.4389 handle_command_reply: tid=0 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 client.4389 shutdown 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 msgr2=0x55b6eabad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eabad500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6cac001b10 tx=0x7f6cac00d520 comp rx=0 tx=0).stop 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 shutdown_connections 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55b6eab43d30 0x55b6eab44110 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b6eab3a440 0x55b6eab437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6c9cff9640 0 client.4389 ms_handle_reset on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b6eab39b20 0x55b6eabad500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 --2- 172.21.15.76:0/394122768 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b6eaaf5330 0x55b6eabacfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 >> 172.21.15.76:0/394122768 conn(0x55b6eaaf0290 msgr2=0x55b6eab37ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 shutdown_connections 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 1 -- 172.21.15.76:0/394122768 wait complete. 2024-09-19T04:13:32.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc6f8d000 20 client.4389 trim_cache size 0 max 16384 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4260, 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 7.2865207009999997, 2024-09-19T04:13:32.233 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 6, 2024-09-19T04:13:32.234 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 25, 2024-09-19T04:13:32.234 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 25, 2024-09-19T04:13:32.234 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 13.15786948 2024-09-19T04:13:32.234 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ca00678c0 msgr2=0x7f6ca0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ca00678c0 0x7f6ca0069d80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cb80af000 tx=0x7f6cb8008040 comp rx=0 tx=0).stop 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc15f3a0 msgr2=0x7f6cbc0b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc15f3a0 0x7f6cbc0b0f80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4007e70 tx=0x7f6cb400e4b0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 shutdown_connections 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ca00678c0 0x7f6ca0069d80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cbc15f3a0 0x7f6cbc0b0f80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cbc1520a0 0x7f6cbc14d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 --2- 172.21.15.76:0/1231463420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cbc06fe50 0x7f6cbc0ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 >> 172.21.15.76:0/1231463420 conn(0x7f6cbc0b1f90 msgr2=0x7f6cbc15b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 shutdown_connections 2024-09-19T04:13:32.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.223+0000 7f6cc3ed1640 1 -- 172.21.15.76:0/1231463420 wait complete. 2024-09-19T04:13:32.243 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:13:32.243 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:32.243 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:13:32.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.351+0000 7f9f3ae44640 1 Processor -- start 2024-09-19T04:13:32.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.351+0000 7f9f3ae44640 1 -- start start 2024-09-19T04:13:32.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.351+0000 7f9f3ae44640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9f34156850 0x7f9f3415c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.351+0000 7f9f3ae44640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f340b66d0 0x7f9f3406e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.351+0000 7f9f3ae44640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f340ce3a0 con 0x7f9f340cdc30 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.351+0000 7f9f3ae44640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9f340ce680 con 0x7f9f340b66d0 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.351+0000 7f9f3ae44640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9f340ce960 con 0x7f9f34156850 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f340b66d0 0x7f9f3406e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f340b66d0 0x7f9f3406e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53620/0 (socket says 172.21.15.76:53620) 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 -- 172.21.15.76:0/503568512 learned_addr learned my addr 172.21.15.76:0/503568512 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f33fff640 1 --1- 172.21.15.76:0/503568512 >> v1:172.21.15.76:6789/0 conn(0x7f9f340cdc30 0x7f9f340b5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47332/0 (socket says 172.21.15.76:47332) 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f38be0640 1 --2- 172.21.15.76:0/503568512 >> v2:172.21.15.116:3300/0 conn(0x7f9f34156850 0x7f9f3415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 -- 172.21.15.76:0/503568512 >> v2:172.21.15.116:3300/0 conn(0x7f9f34156850 msgr2=0x7f9f3415c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 --2- 172.21.15.76:0/503568512 >> v2:172.21.15.116:3300/0 conn(0x7f9f34156850 0x7f9f3415c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 -- 172.21.15.76:0/503568512 >> v1:172.21.15.76:6789/0 conn(0x7f9f340cdc30 legacy=0x7f9f340b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 -- 172.21.15.76:0/503568512 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f340cec40 con 0x7f9f340b66d0 2024-09-19T04:13:32.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f337fe640 1 -- 172.21.15.76:0/503568512 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4059967825 0 0) 0x7f9f340ce3a0 con 0x7f9f340cdc30 2024-09-19T04:13:32.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f38be0640 1 --2- 172.21.15.76:0/503568512 >> v2:172.21.15.116:3300/0 conn(0x7f9f34156850 0x7f9f3415c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f393e1640 1 --2- 172.21.15.76:0/503568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f3406e860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f34074440 tx=0x7f9f2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55e91ceb1f58802 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f337fe640 1 -- 172.21.15.76:0/503568512 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0ab430 con 0x7f9f340b66d0 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f337fe640 1 -- 172.21.15.76:0/503568512 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2c0b0070 con 0x7f9f340b66d0 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f337fe640 1 -- 172.21.15.76:0/503568512 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c0ab7f0 con 0x7f9f340b66d0 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 -- 172.21.15.76:0/503568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 msgr2=0x7f9f3406e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/503568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f3406e860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f9f34074440 tx=0x7f9f2c01c520 comp rx=0 tx=0).stop 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 -- 172.21.15.76:0/503568512 shutdown_connections 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/503568512 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f3406e860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/503568512 >> v2:172.21.15.116:3300/0 conn(0x7f9f34156850 0x7f9f3415c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 -- 172.21.15.76:0/503568512 >> 172.21.15.76:0/503568512 conn(0x7f9f340736d0 msgr2=0x7f9f3406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 -- 172.21.15.76:0/503568512 shutdown_connections 2024-09-19T04:13:32.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.355+0000 7f9f3ae44640 1 -- 172.21.15.76:0/503568512 wait complete. 2024-09-19T04:13:32.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 Processor -- start 2024-09-19T04:13:32.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 -- start start 2024-09-19T04:13:32.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f34153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f340cdc30 0x7f9f341555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f34156850 0x7f9f341f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3406f940 con 0x7f9f340b66d0 2024-09-19T04:13:32.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f3406f7c0 con 0x7f9f340cdc30 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9f3406fac0 con 0x7f9f34156850 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f34153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f34153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53634/0 (socket says 172.21.15.76:53634) 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 -- 172.21.15.76:0/3103918999 learned_addr learned my addr 172.21.15.76:0/3103918999 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f393e1640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f34156850 0x7f9f341f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f33fff640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f340cdc30 0x7f9f341555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 -- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f34156850 msgr2=0x7f9f341f6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f34156850 0x7f9f341f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 -- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f340cdc30 msgr2=0x7f9f341555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f340cdc30 0x7f9f341555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 -- 172.21.15.76:0/3103918999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f34157200 con 0x7f9f340b66d0 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f38be0640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f34153670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400cf90 tx=0x7f9f2400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f317fa640 1 -- 172.21.15.76:0/3103918999 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2401a070 con 0x7f9f340b66d0 2024-09-19T04:13:32.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f317fa640 1 -- 172.21.15.76:0/3103918999 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f2400d840 con 0x7f9f340b66d0 2024-09-19T04:13:32.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f341574e0 con 0x7f9f340b66d0 2024-09-19T04:13:32.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f317fa640 1 -- 172.21.15.76:0/3103918999 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400dc10 con 0x7f9f340b66d0 2024-09-19T04:13:32.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f34157aa0 con 0x7f9f340b66d0 2024-09-19T04:13:32.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f3406e860 con 0x7f9f340b66d0 2024-09-19T04:13:32.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.359+0000 7f9f317fa640 1 -- 172.21.15.76:0/3103918999 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f2401d070 con 0x7f9f340b66d0 2024-09-19T04:13:32.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.363+0000 7f9f317fa640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f140678f0 0x7f9f14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:32.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.363+0000 7f9f317fa640 1 -- 172.21.15.76:0/3103918999 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9f24086730 con 0x7f9f340b66d0 2024-09-19T04:13:32.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.363+0000 7f9f33fff640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f140678f0 0x7f9f14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:32.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.363+0000 7f9f33fff640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f140678f0 0x7f9f14069db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f3415a460 tx=0x7f9f1c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:32.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.367+0000 7f9f317fa640 1 -- 172.21.15.76:0/3103918999 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9f24056970 con 0x7f9f340b66d0 2024-09-19T04:13:32.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.559+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9f34067a00 con 0x7f9f340b66d0 2024-09-19T04:13:32.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.559+0000 7f9f317fa640 1 -- 172.21.15.76:0/3103918999 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7f9f2405ac70 con 0x7f9f340b66d0 2024-09-19T04:13:32.564 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f140678f0 msgr2=0x7f9f14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f140678f0 0x7f9f14069db0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f3415a460 tx=0x7f9f1c009040 comp rx=0 tx=0).stop 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 msgr2=0x7f9f34153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f34153670 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9f2400cf90 tx=0x7f9f2400a7d0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 shutdown_connections 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f140678f0 0x7f9f14069db0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f34156850 0x7f9f341f6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f340cdc30 0x7f9f341555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 --2- 172.21.15.76:0/3103918999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f340b66d0 0x7f9f34153670 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:32.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 >> 172.21.15.76:0/3103918999 conn(0x7f9f340736d0 msgr2=0x7f9f340b5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:32.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 shutdown_connections 2024-09-19T04:13:32.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32.563+0000 7f9f3ae44640 1 -- 172.21.15.76:0/3103918999 wait complete. 2024-09-19T04:13:32.580 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-19T04:13:32.580 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:13:32.581 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:13:32.581 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-19T04:13:32.581 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:32.581 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-19T04:13:32.581 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:13:32.581 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:13:32.581 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T04:13:32.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:32.581 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:13:32.586 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:32.587 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42508sec preferred_lft 42508sec 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:13:32.588 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:32.589 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:13:32.589 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:32.589 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:32.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:13:32.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2024-09-19T04:13:32.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2024-09-19T04:13:32.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:13:32.590 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:32.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:32.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:32.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:32.706 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:13:32.756 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:13:32.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:32.757 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:13:32.823 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:32.824 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:13:32.825 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:32.825 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:32.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:13:32.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:13:32.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:13:32.825 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:32.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:32.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:32.935 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:13:32.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:32.980 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-09-19T04:13:33.028 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:33.028 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:33.028 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:33.029 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-19T04:13:33.029 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:33.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:33.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:33.142 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-19T04:13:33.142 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:33.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:33.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:33.283 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:33.283 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:33.283 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:13:33.283 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:13:33.283 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:33.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:33.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:33.384 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-19T04:13:33.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:33.384 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:33.432 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-19T04:13:33.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:33.432 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:33.480 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:33.537 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:33.585 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:33.585 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:33.585 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:33.585 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:33.585 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:33.585 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:33.586 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:33.587 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:33.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:33.588 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:33.636 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:13:33.637 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-19T04:13:33.682 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:33.697 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:33.745 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:33.746 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:33.747 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:33.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:33.749 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:33.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:2024-09-19T04:13:33.759+0000 7f9d3ea16580 -1 init, newargv = 0x55fe8649b690 newargc=15 2024-09-19T04:13:33.764 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[35265]: starting ceph client 2024-09-19T04:13:33.771 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[35265]: starting fuse 2024-09-19T04:13:34.870 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:34.886 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:34.933 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:34.934 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:34.935 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:34.936 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:34.936 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:34.936 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:34.936 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:34.936 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:34.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:34.937 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:34.985 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:13:34.985 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:13:34.985 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:13:34.985 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:13:34.985 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:13:34.986 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.admin")) 2024-09-19T04:13:34.987 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:13:35.110 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.admin.35265.asok 2024-09-19T04:13:35.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:35.121 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.35265.asok 2024-09-19T04:13:35.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:35.121 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.35265.asok status 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "admin", 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "35265", 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:13:35.275 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4404, 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4404 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:13:35.276 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 335109884 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 335109884 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4404 192.168.0.1:0/335109884", 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/335109884", 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 6, 2024-09-19T04:13:35.277 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 25, 2024-09-19T04:13:35.278 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:13:35.278 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:13:35.278 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:13:35.278 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:13:35.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:35.289 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:13:35.289 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:35.295 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:13:35.295 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:35.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:35.296 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:35.352 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-19T04:13:35.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T04:13:35.465 INFO:teuthology.orchestra.run.smithi076.stdout:# file: . 2024-09-19T04:13:35.465 INFO:teuthology.orchestra.run.smithi076.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-19T04:13:35.465 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:35.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:35.472 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:13:35.472 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:13:35.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:13:35.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9fb9c640 1 Processor -- start 2024-09-19T04:13:35.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9fb9c640 1 -- start start 2024-09-19T04:13:35.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9fb9c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdf98070970 0x7fdf980ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9fb9c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf980b5c90 con 0x7fdf980ae5b0 2024-09-19T04:13:35.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9fb9c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf980b5f30 con 0x7fdf98162750 2024-09-19T04:13:35.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9fb9c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fdf9806f9b0 con 0x7fdf98070970 2024-09-19T04:13:35.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdf980ae5b0 0x7fdf9806fe40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34224/0 (socket says 172.21.15.76:34224) 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 -- 172.21.15.76:0/1128397461 learned_addr learned my addr 172.21.15.76:0/1128397461 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9d137640 1 --2- 172.21.15.76:0/1128397461 >> v2:172.21.15.116:3300/0 conn(0x7fdf98070970 0x7fdf980ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3913149885 0 0) 0x7fdf980b5c90 con 0x7fdf980ae5b0 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf80003530 con 0x7fdf980ae5b0 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2307091199 0 0) 0x7fdf80003530 con 0x7fdf980ae5b0 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 >> v2:172.21.15.116:3300/0 conn(0x7fdf98070970 msgr2=0x7fdf980ae070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 --2- 172.21.15.76:0/1128397461 >> v2:172.21.15.116:3300/0 conn(0x7fdf98070970 0x7fdf980ae070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 >> v1:172.21.15.116:6789/0 conn(0x7fdf98162750 legacy=0x7fdf980b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf980b6210 con 0x7fdf980ae5b0 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fdf88001680 con 0x7fdf980ae5b0 2024-09-19T04:13:35.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 >> v1:172.21.15.76:6789/0 conn(0x7fdf980ae5b0 legacy=0x7fdf9806fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800039c0 0x7fdf80003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf80004810 0x7fdf80004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf800051d0 0x7fdf8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf9806f9b0 con 0x7fdf800051d0 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8000c170 con 0x7fdf800039c0 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdf8000c370 con 0x7fdf80004810 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf800051d0 0x7fdf8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9d137640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf80004810 0x7fdf80004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9d938640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800039c0 0x7fdf80003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 -- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf80004810 msgr2=0x7fdf80004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf80004810 0x7fdf80004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 -- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800039c0 msgr2=0x7fdf80003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:35.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800039c0 0x7fdf80003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9e139640 1 -- 172.21.15.76:0/1128397461 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf80003530 con 0x7fdf800051d0 2024-09-19T04:13:35.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.691+0000 7fdf9d137640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf80004810 0x7fdf80004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9e139640 1 -- 172.21.15.76:0/1128397461 reap_dead start 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9e139640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf800051d0 0x7fdf8000b9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c210 tx=0x7fdf8801f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81403fe23d456d73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88007d40 con 0x7fdf800051d0 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf88005b00 con 0x7fdf800051d0 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9c936640 1 -- 172.21.15.76:0/1128397461 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88007070 con 0x7fdf800051d0 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/1128397461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf800051d0 msgr2=0x7fdf8000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf800051d0 0x7fdf8000b9d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf8800c210 tx=0x7fdf8801f3f0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/1128397461 shutdown_connections 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/1128397461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf800051d0 0x7fdf8000b9d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/1128397461 >> 172.21.15.76:0/1128397461 conn(0x7fdf9809aa00 msgr2=0x7fdf980b65b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/1128397461 shutdown_connections 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/1128397461 wait complete. 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.695+0000 7fdf9fb9c640 1 Processor -- start 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 -- start start 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800051d0 0x7fdf980d1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9809d9b0 0x7fdf9806bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf9806c200 0x7fdf9814cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf98070ef0 con 0x7fdf9809d9b0 2024-09-19T04:13:35.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf98070d70 con 0x7fdf800051d0 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdf98071070 con 0x7fdf9806c200 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9809d9b0 0x7fdf9806bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9809d9b0 0x7fdf9806bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40420/0 (socket says 172.21.15.76:40420) 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 -- 172.21.15.76:0/3279012592 learned_addr learned my addr 172.21.15.76:0/3279012592 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9e139640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf9806c200 0x7fdf9814cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d938640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800051d0 0x7fdf980d1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 -- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf9806c200 msgr2=0x7fdf9814cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf9806c200 0x7fdf9814cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 -- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800051d0 msgr2=0x7fdf980d1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800051d0 0x7fdf980d1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 -- 172.21.15.76:0/3279012592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9809e170 con 0x7fdf9809d9b0 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d938640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800051d0 0x7fdf980d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:35.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9e139640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf9806c200 0x7fdf9814cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:35.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d137640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9809d9b0 0x7fdf9806bcc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0108c0 tx=0x7fdf8c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:35.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf967fc640 1 -- 172.21.15.76:0/3279012592 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00f040 con 0x7fdf9809d9b0 2024-09-19T04:13:35.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf967fc640 1 -- 172.21.15.76:0/3279012592 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf8c004220 con 0x7fdf9809d9b0 2024-09-19T04:13:35.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9809e450 con 0x7fdf9809d9b0 2024-09-19T04:13:35.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf967fc640 1 -- 172.21.15.76:0/3279012592 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c00e960 con 0x7fdf9809d9b0 2024-09-19T04:13:35.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9809ea10 con 0x7fdf9809d9b0 2024-09-19T04:13:35.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf980ae100 con 0x7fdf9809d9b0 2024-09-19T04:13:35.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf967fc640 1 -- 172.21.15.76:0/3279012592 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf8c00eb00 con 0x7fdf9809d9b0 2024-09-19T04:13:35.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf967fc640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf840678f0 0x7fdf84069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:35.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf967fc640 1 -- 172.21.15.76:0/3279012592 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdf8c0861c0 con 0x7fdf9809d9b0 2024-09-19T04:13:35.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.699+0000 7fdf9d938640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf840678f0 0x7fdf84069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:35.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.703+0000 7fdf9d938640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf840678f0 0x7fdf84069db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf90001d50 tx=0x7fdf90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:35.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.703+0000 7fdf967fc640 1 -- 172.21.15.76:0/3279012592 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdf8c018030 con 0x7fdf9809d9b0 2024-09-19T04:13:35.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.867+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdf98159010 con 0x7fdf9809d9b0 2024-09-19T04:13:35.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.867+0000 7fdf967fc640 1 -- 172.21.15.76:0/3279012592 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fdf8c056a50 con 0x7fdf9809d9b0 2024-09-19T04:13:35.870 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:13:35.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf840678f0 msgr2=0x7fdf84069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf840678f0 0x7fdf84069db0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdf90001d50 tx=0x7fdf90009040 comp rx=0 tx=0).stop 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9809d9b0 msgr2=0x7fdf9806bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9809d9b0 0x7fdf9806bcc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0108c0 tx=0x7fdf8c010d80 comp rx=0 tx=0).stop 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 shutdown_connections 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf840678f0 0x7fdf84069db0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf9806c200 0x7fdf9814cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf9809d9b0 0x7fdf9806bcc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 --2- 172.21.15.76:0/3279012592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf800051d0 0x7fdf980d1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 >> 172.21.15.76:0/3279012592 conn(0x7fdf9809aa00 msgr2=0x7fdf98163220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 shutdown_connections 2024-09-19T04:13:35.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:35.871+0000 7fdf9fb9c640 1 -- 172.21.15.76:0/3279012592 wait complete. 2024-09-19T04:13:35.887 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:13:35.888 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:13:35.893 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:13:36.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.051+0000 7f1ebf632640 1 Processor -- start 2024-09-19T04:13:36.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.051+0000 7f1ebf632640 1 -- start start 2024-09-19T04:13:36.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebf632640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 0x7f1eb80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebf632640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb80b3290 con 0x7f1eb815bec0 2024-09-19T04:13:36.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebf632640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1eb80b3570 con 0x7f1eb80af2a0 2024-09-19T04:13:36.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebf632640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb80b3850 con 0x7f1eb815c2a0 2024-09-19T04:13:36.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1eb815bec0 0x7f1eb80ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34234/0 (socket says 172.21.15.76:34234) 2024-09-19T04:13:36.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 -- 172.21.15.76:0/3022600280 learned_addr learned my addr 172.21.15.76:0/3022600280 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4062927552 0 0) 0x7f1eb80b3290 con 0x7f1eb815bec0 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebcbcd640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 0x7f1eb80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ea4003530 con 0x7f1eb815bec0 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 208166615 0 0) 0x7f1ea4003530 con 0x7f1eb815bec0 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 msgr2=0x7f1eb80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 0x7f1eb80aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 >> v1:172.21.15.116:6789/0 conn(0x7f1eb80af2a0 legacy=0x7f1eb806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb80b3b30 con 0x7f1eb815bec0 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f1eb4001680 con 0x7f1eb815bec0 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 >> v1:172.21.15.76:6789/0 conn(0x7f1eb815bec0 legacy=0x7f1eb80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea40039f0 0x7f1ea4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 0x7f1ea4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea4004d50 0x7f1ea40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80b3850 con 0x7f1ea40039f0 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea400bb60 con 0x7f1ea4004d50 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea400bd30 con 0x7f1eb815c2a0 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea40039f0 0x7f1ea4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebcbcd640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 0x7f1ea4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebdbcf640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea4004d50 0x7f1ea40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 -- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 msgr2=0x7f1ea4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb815c2a0 0x7f1ea4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 -- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea4004d50 msgr2=0x7f1ea40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea4004d50 0x7f1ea40051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 -- 172.21.15.76:0/3022600280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea4003530 con 0x7f1ea40039f0 2024-09-19T04:13:36.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1ebd3ce640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea40039f0 0x7f1ea4003e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400c1f0 tx=0x7f1eb401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e488e9341f10eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4007d00 con 0x7f1ea40039f0 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1eb40a9020 con 0x7f1ea40039f0 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.055+0000 7f1eaffff640 1 -- 172.21.15.76:0/3022600280 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4005b00 con 0x7f1ea40039f0 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- 172.21.15.76:0/3022600280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea40039f0 msgr2=0x7f1ea4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 --2- 172.21.15.76:0/3022600280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea40039f0 0x7f1ea4003e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400c1f0 tx=0x7f1eb401cc30 comp rx=0 tx=0).stop 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebdbcf640 1 -- 172.21.15.76:0/3022600280 reap_dead start 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- 172.21.15.76:0/3022600280 shutdown_connections 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- 172.21.15.76:0/3022600280 >> 172.21.15.76:0/3022600280 conn(0x7f1eb80b1f90 msgr2=0x7f1eb80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- 172.21.15.76:0/3022600280 shutdown_connections 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- 172.21.15.76:0/3022600280 wait complete. 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 Processor -- start 2024-09-19T04:13:36.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- start start 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea40039f0 0x7f1eb81f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eb8155d70 0x7f1eb813fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb81561f0 0x7f1eb8140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb806ef20 con 0x7f1eb8155d70 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb80af6b0 con 0x7f1ea40039f0 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1eb806f0a0 con 0x7f1eb81561f0 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eb8155d70 0x7f1eb813fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eb8155d70 0x7f1eb813fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40424/0 (socket says 172.21.15.76:40424) 2024-09-19T04:13:36.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 -- 172.21.15.76:0/1112628114 learned_addr learned my addr 172.21.15.76:0/1112628114 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebd3ce640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea40039f0 0x7f1eb81f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 -- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb81561f0 msgr2=0x7f1eb8140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebdbcf640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb81561f0 0x7f1eb8140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb81561f0 0x7f1eb8140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 -- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea40039f0 msgr2=0x7f1eb81f93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea40039f0 0x7f1eb81f93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 -- 172.21.15.76:0/1112628114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb81f98e0 con 0x7f1eb8155d70 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebcbcd640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eb8155d70 0x7f1eb813fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800e810 tx=0x7f1ea800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1eadffb640 1 -- 172.21.15.76:0/1112628114 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8017070 con 0x7f1eb8155d70 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1eadffb640 1 -- 172.21.15.76:0/1112628114 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ea800c6d0 con 0x7f1eb8155d70 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb8161340 con 0x7f1eb8155d70 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1eadffb640 1 -- 172.21.15.76:0/1112628114 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800f040 con 0x7f1eb8155d70 2024-09-19T04:13:36.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.059+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb8161900 con 0x7f1eb8155d70 2024-09-19T04:13:36.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.063+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1eb806e000 con 0x7f1eb8155d70 2024-09-19T04:13:36.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.063+0000 7f1eadffb640 1 -- 172.21.15.76:0/1112628114 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ea801a070 con 0x7f1eb8155d70 2024-09-19T04:13:36.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.063+0000 7f1eadffb640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e9c0678c0 0x7f1e9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.063+0000 7f1eadffb640 1 -- 172.21.15.76:0/1112628114 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ea8085b20 con 0x7f1eb8155d70 2024-09-19T04:13:36.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.063+0000 7f1ebd3ce640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e9c0678c0 0x7f1e9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.063+0000 7f1ebd3ce640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e9c0678c0 0x7f1e9c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40aa000 tx=0x7f1eb80ae5f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.067+0000 7f1eadffb640 1 -- 172.21.15.76:0/1112628114 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1eb81f98e0 con 0x7f1eb8155d70 2024-09-19T04:13:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1eb80b16a0 con 0x7f1eb8155d70 2024-09-19T04:13:36.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1eadffb640 1 -- 172.21.15.76:0/1112628114 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f1ea8056300 con 0x7f1eb8155d70 2024-09-19T04:13:36.242 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:13:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e9c0678c0 msgr2=0x7f1e9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e9c0678c0 0x7f1e9c069d80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40aa000 tx=0x7f1eb80ae5f0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eb8155d70 msgr2=0x7f1eb813fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eb8155d70 0x7f1eb813fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800e810 tx=0x7f1ea800ecd0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 shutdown_connections 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e9c0678c0 0x7f1e9c069d80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1eb81561f0 0x7f1eb8140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1eb8155d70 0x7f1eb813fd50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.239+0000 7f1ebf632640 1 --2- 172.21.15.76:0/1112628114 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea40039f0 0x7f1eb81f93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.243+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 >> 172.21.15.76:0/1112628114 conn(0x7f1eb80b1f90 msgr2=0x7f1eb80b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.243+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 shutdown_connections 2024-09-19T04:13:36.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.243+0000 7f1ebf632640 1 -- 172.21.15.76:0/1112628114 wait complete. 2024-09-19T04:13:36.260 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:13:36.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:36.260 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-19T04:13:36.278 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi076.stderr:ceph-fuse[35265]: fuse finished with error 0 and tester_r 0 2024-09-19T04:13:36.279 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:13:36.350 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:13:36.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:36.350 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-19T04:13:36.358 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:36.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.519+0000 7f9bef7e7640 1 Processor -- start 2024-09-19T04:13:36.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.519+0000 7f9bef7e7640 1 -- start start 2024-09-19T04:13:36.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bef7e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815c2a0 0x7f9be80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bef7e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9be80af2a0 0x7f9be806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bef7e7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9be80b3290 con 0x7f9be815bec0 2024-09-19T04:13:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bef7e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9be80b3570 con 0x7f9be80af2a0 2024-09-19T04:13:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bef7e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9be80b3850 con 0x7f9be815c2a0 2024-09-19T04:13:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bedd84640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9be80af2a0 0x7f9be806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bed583640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9be815bec0 0x7f9be80ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34240/0 (socket says 172.21.15.76:34240) 2024-09-19T04:13:36.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bed583640 1 -- 172.21.15.76:0/4265837841 learned_addr learned my addr 172.21.15.76:0/4265837841 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:36.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bedd84640 1 -- 172.21.15.76:0/4265837841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815c2a0 msgr2=0x7f9be80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:36.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9becd82640 1 --2- 172.21.15.76:0/4265837841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815c2a0 0x7f9be80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bedd84640 1 --2- 172.21.15.76:0/4265837841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815c2a0 0x7f9be80aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9be7fff640 1 -- 172.21.15.76:0/4265837841 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3184523198 0 0) 0x7f9be80b3290 con 0x7f9be815bec0 2024-09-19T04:13:36.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bedd84640 1 -- 172.21.15.76:0/4265837841 >> v1:172.21.15.76:6789/0 conn(0x7f9be815bec0 legacy=0x7f9be80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bedd84640 1 -- 172.21.15.76:0/4265837841 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be80b3b30 con 0x7f9be80af2a0 2024-09-19T04:13:36.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9becd82640 1 --2- 172.21.15.76:0/4265837841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815c2a0 0x7f9be80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:36.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9bedd84640 1 --2- 172.21.15.76:0/4265837841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be806e000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80097c0 tx=0x7f9bd801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedb7b94dbb2bf37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9be7fff640 1 -- 172.21.15.76:0/4265837841 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a7070 con 0x7f9be80af2a0 2024-09-19T04:13:36.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9be7fff640 1 -- 172.21.15.76:0/4265837841 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bd80b0070 con 0x7f9be80af2a0 2024-09-19T04:13:36.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.523+0000 7f9be7fff640 1 -- 172.21.15.76:0/4265837841 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80ab890 con 0x7f9be80af2a0 2024-09-19T04:13:36.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- 172.21.15.76:0/4265837841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 msgr2=0x7f9be806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/4265837841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be806e000 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80097c0 tx=0x7f9bd801c470 comp rx=0 tx=0).stop 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- 172.21.15.76:0/4265837841 shutdown_connections 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/4265837841 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be806e000 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/4265837841 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815c2a0 0x7f9be80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- 172.21.15.76:0/4265837841 >> 172.21.15.76:0/4265837841 conn(0x7f9be80b1f90 msgr2=0x7f9be80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- 172.21.15.76:0/4265837841 shutdown_connections 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- 172.21.15.76:0/4265837841 wait complete. 2024-09-19T04:13:36.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 Processor -- start 2024-09-19T04:13:36.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- start start 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be81559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815bec0 0x7f9be814c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9be815c2a0 0x7f9be8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9be806ed50 con 0x7f9be80af2a0 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9be80afc90 con 0x7f9be815c2a0 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9be806eed0 con 0x7f9be815bec0 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be81559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be81559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40440/0 (socket says 172.21.15.76:40440) 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 -- 172.21.15.76:0/2605554921 learned_addr learned my addr 172.21.15.76:0/2605554921 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bedd84640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9be815c2a0 0x7f9be8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9becd82640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815bec0 0x7f9be814c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 -- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815bec0 msgr2=0x7f9be814c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815bec0 0x7f9be814c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 -- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9be815c2a0 msgr2=0x7f9be8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9be815c2a0 0x7f9be8071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 -- 172.21.15.76:0/2605554921 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be814cdc0 con 0x7f9be80af2a0 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bed583640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be81559b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9be00011f0 tx=0x7f9be000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9be5ffb640 1 -- 172.21.15.76:0/2605554921 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0018070 con 0x7f9be80af2a0 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9be5ffb640 1 -- 172.21.15.76:0/2605554921 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be00074e0 con 0x7f9be80af2a0 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be8072760 con 0x7f9be80af2a0 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9be5ffb640 1 -- 172.21.15.76:0/2605554921 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0004a10 con 0x7f9be80af2a0 2024-09-19T04:13:36.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.527+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be8072d20 con 0x7f9be80af2a0 2024-09-19T04:13:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.531+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be8073650 con 0x7f9be80af2a0 2024-09-19T04:13:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.531+0000 7f9be5ffb640 1 -- 172.21.15.76:0/2605554921 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9be001b070 con 0x7f9be80af2a0 2024-09-19T04:13:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.531+0000 7f9be5ffb640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9bd40678c0 0x7f9bd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.531+0000 7f9be5ffb640 1 -- 172.21.15.76:0/2605554921 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9be0085370 con 0x7f9be80af2a0 2024-09-19T04:13:36.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.531+0000 7f9becd82640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9bd40678c0 0x7f9bd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.531+0000 7f9becd82640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9bd40678c0 0x7f9bd4069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9be8155ef0 tx=0x7f9bdc008430 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.535+0000 7f9be5ffb640 1 -- 172.21.15.76:0/2605554921 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9be0055b70 con 0x7f9be80af2a0 2024-09-19T04:13:36.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.695+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9be806f280 con 0x7f9be80af2a0 2024-09-19T04:13:36.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9be5ffb640 1 -- 172.21.15.76:0/2605554921 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f9be0055470 con 0x7f9be80af2a0 2024-09-19T04:13:36.701 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:36.702 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9bd40678c0 msgr2=0x7f9bd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9bd40678c0 0x7f9bd4069d80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f9be8155ef0 tx=0x7f9bdc008430 comp rx=0 tx=0).stop 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 msgr2=0x7f9be81559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be81559b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9be00011f0 tx=0x7f9be000c9b0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 shutdown_connections 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9bd40678c0 0x7f9bd4069d80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9be815c2a0 0x7f9be8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9be815bec0 0x7f9be814c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 --2- 172.21.15.76:0/2605554921 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9be80af2a0 0x7f9be81559b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 >> 172.21.15.76:0/2605554921 conn(0x7f9be80b1f90 msgr2=0x7f9be80cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:36.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 shutdown_connections 2024-09-19T04:13:36.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.699+0000 7f9bef7e7640 1 -- 172.21.15.76:0/2605554921 wait complete. 2024-09-19T04:13:36.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:36.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60ab2c6640 1 Processor -- start 2024-09-19T04:13:36.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60ab2c6640 1 -- start start 2024-09-19T04:13:36.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60ab2c6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60a40b39a0 0x7f60a406df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60ab2c6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a406ec60 con 0x7f60a41587a0 2024-09-19T04:13:36.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60ab2c6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60a406ef40 con 0x7f60a406e4b0 2024-09-19T04:13:36.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60ab2c6640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f60a406f870 con 0x7f60a40b39a0 2024-09-19T04:13:36.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60a9062640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f60a41587a0 0x7f60a40b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34248/0 (socket says 172.21.15.76:34248) 2024-09-19T04:13:36.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60a9062640 1 -- 172.21.15.76:0/3340314508 learned_addr learned my addr 172.21.15.76:0/3340314508 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:36.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f60a8861640 1 --2- 172.21.15.76:0/3340314508 >> v2:172.21.15.116:3300/0 conn(0x7f60a40b39a0 0x7f60a406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1220855065 0 0) 0x7f60a406ec60 con 0x7f60a41587a0 2024-09-19T04:13:36.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6090003530 con 0x7f60a41587a0 2024-09-19T04:13:36.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 874125031 0 0) 0x7f6090003530 con 0x7f60a41587a0 2024-09-19T04:13:36.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 >> v2:172.21.15.116:3300/0 conn(0x7f60a40b39a0 msgr2=0x7f60a406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:36.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 --2- 172.21.15.76:0/3340314508 >> v2:172.21.15.116:3300/0 conn(0x7f60a40b39a0 0x7f60a406df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 >> v1:172.21.15.116:6789/0 conn(0x7f60a406e4b0 legacy=0x7f60a4072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a406f220 con 0x7f60a41587a0 2024-09-19T04:13:36.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6094001680 con 0x7f60a41587a0 2024-09-19T04:13:36.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.835+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 >> v1:172.21.15.76:6789/0 conn(0x7f60a41587a0 legacy=0x7f60a40b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f609bfff640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60900039c0 0x7f6090003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f609bfff640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6090004810 0x7f6090004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f609bfff640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60900051d0 0x7f609000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f60a406f870 con 0x7f60900039c0 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f609000c0a0 con 0x7f60900051d0 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f609000c2a0 con 0x7f6090004810 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9062640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60900039c0 0x7f6090003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a8861640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6090004810 0x7f6090004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9863640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60900051d0 0x7f609000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9062640 1 -- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6090004810 msgr2=0x7f6090004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9062640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6090004810 0x7f6090004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9062640 1 -- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60900051d0 msgr2=0x7f609000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:36.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9062640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60900051d0 0x7f609000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9863640 1 -- 172.21.15.76:0/3340314508 reap_dead start 2024-09-19T04:13:36.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.839+0000 7f60a9062640 1 -- 172.21.15.76:0/3340314508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6090003530 con 0x7f60900039c0 2024-09-19T04:13:36.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60a9062640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60900039c0 0x7f6090003e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f609400c1c0 tx=0x7f609401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e3fba33fe85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094007d00 con 0x7f60900039c0 2024-09-19T04:13:36.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60940a9020 con 0x7f60900039c0 2024-09-19T04:13:36.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f609bfff640 1 -- 172.21.15.76:0/3340314508 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094005b20 con 0x7f60900039c0 2024-09-19T04:13:36.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60ab2c6640 1 -- 172.21.15.76:0/3340314508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60900039c0 msgr2=0x7f6090003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60900039c0 0x7f6090003e00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f609400c1c0 tx=0x7f609401cc30 comp rx=0 tx=0).stop 2024-09-19T04:13:36.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60ab2c6640 1 -- 172.21.15.76:0/3340314508 shutdown_connections 2024-09-19T04:13:36.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/3340314508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60900039c0 0x7f6090003e00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60ab2c6640 1 -- 172.21.15.76:0/3340314508 >> 172.21.15.76:0/3340314508 conn(0x7f60a40b1f90 msgr2=0x7f60a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:36.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60ab2c6640 1 -- 172.21.15.76:0/3340314508 shutdown_connections 2024-09-19T04:13:36.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.843+0000 7f60ab2c6640 1 -- 172.21.15.76:0/3340314508 wait complete. 2024-09-19T04:13:36.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 Processor -- start 2024-09-19T04:13:36.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 -- start start 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60900039c0 0x7f60a4152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60a41534e0 0x7f60a4153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60a4153e80 0x7f60a409c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f60a406ea10 con 0x7f60a41534e0 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60a406e890 con 0x7f60a4153e80 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f60a4162660 con 0x7f60900039c0 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60a41534e0 0x7f60a4153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60a41534e0 0x7f60a4153940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40456/0 (socket says 172.21.15.76:40456) 2024-09-19T04:13:36.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 -- 172.21.15.76:0/1537607027 learned_addr learned my addr 172.21.15.76:0/1537607027 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 -- 172.21.15.76:0/1537607027 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60900039c0 msgr2=0x7f60a4152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60900039c0 0x7f60a4152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 -- 172.21.15.76:0/1537607027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60a4153e80 msgr2=0x7f60a409c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60a4153e80 0x7f60a409c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 -- 172.21.15.76:0/1537607027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60a409e390 con 0x7f60a41534e0 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60a8861640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60a41534e0 0x7f60a4153940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60a00119f0 tx=0x7f60a0011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f6099ffb640 1 -- 172.21.15.76:0/1537607027 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a000fad0 con 0x7f60a41534e0 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f6099ffb640 1 -- 172.21.15.76:0/1537607027 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60a000fc70 con 0x7f60a41534e0 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60a409d810 con 0x7f60a41534e0 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f6099ffb640 1 -- 172.21.15.76:0/1537607027 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60a0014440 con 0x7f60a41534e0 2024-09-19T04:13:36.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60a409ddd0 con 0x7f60a41534e0 2024-09-19T04:13:36.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60a41542a0 con 0x7f60a41534e0 2024-09-19T04:13:36.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.847+0000 7f6099ffb640 1 -- 172.21.15.76:0/1537607027 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f60a0012040 con 0x7f60a41534e0 2024-09-19T04:13:36.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.851+0000 7f6099ffb640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60880678f0 0x7f6088069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:36.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.851+0000 7f60a9062640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60880678f0 0x7f6088069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:36.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.851+0000 7f6099ffb640 1 -- 172.21.15.76:0/1537607027 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f60a00865e0 con 0x7f60a41534e0 2024-09-19T04:13:36.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.851+0000 7f60a9062640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60880678f0 0x7f6088069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f609400c1c0 tx=0x7f60a406f110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:36.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:36.855+0000 7f6099ffb640 1 -- 172.21.15.76:0/1537607027 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f60a0019030 con 0x7f60a41534e0 2024-09-19T04:13:37.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.023+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f60a4070080 con 0x7f60a41534e0 2024-09-19T04:13:37.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f6099ffb640 1 -- 172.21.15.76:0/1537607027 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f60a0056790 con 0x7f60a41534e0 2024-09-19T04:13:37.028 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:37.029 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60880678f0 msgr2=0x7f6088069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60880678f0 0x7f6088069db0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f609400c1c0 tx=0x7f60a406f110 comp rx=0 tx=0).stop 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60a41534e0 msgr2=0x7f60a4153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60a41534e0 0x7f60a4153940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60a00119f0 tx=0x7f60a0011eb0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 shutdown_connections 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60880678f0 0x7f6088069db0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60a4153e80 0x7f60a409c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60a41534e0 0x7f60a4153940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 --2- 172.21.15.76:0/1537607027 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60900039c0 0x7f60a4152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 >> 172.21.15.76:0/1537607027 conn(0x7f60a40b1f90 msgr2=0x7f60a40b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 shutdown_connections 2024-09-19T04:13:37.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.027+0000 7f60ab2c6640 1 -- 172.21.15.76:0/1537607027 wait complete. 2024-09-19T04:13:37.046 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:13:37.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61e887640 1 Processor -- start 2024-09-19T04:13:37.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61e887640 1 -- start start 2024-09-19T04:13:37.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61e887640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa618161a60 0x7fa6180d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61e887640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6180cf620 0x7fa61806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61e887640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6180d32b0 con 0x7fa6180d2e90 2024-09-19T04:13:37.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61e887640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa6180d0300 con 0x7fa618161a60 2024-09-19T04:13:37.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61e887640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa6180cfab0 con 0x7fa6180cf620 2024-09-19T04:13:37.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61ce24640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6180cf620 0x7fa61806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa617fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa618161a60 0x7fa6180d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61ce24640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6180cf620 0x7fa61806d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:37996/0 (socket says 172.21.15.76:37996) 2024-09-19T04:13:37.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa6177fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa6180d2e90 0x7fa6180cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:46012/0 (socket says 172.21.15.76:46012) 2024-09-19T04:13:37.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa617fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa618161a60 0x7fa6180d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:55916/0 (socket says 172.21.15.76:55916) 2024-09-19T04:13:37.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa61ce24640 1 -- 172.21.15.76:0/917111082 learned_addr learned my addr 172.21.15.76:0/917111082 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:37.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa617fff640 1 -- 172.21.15.76:0/917111082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6180cf620 msgr2=0x7fa61806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:37.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa617fff640 1 --2- 172.21.15.76:0/917111082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6180cf620 0x7fa61806d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa617fff640 1 -- 172.21.15.76:0/917111082 >> v1:172.21.15.116:6789/0 conn(0x7fa6180d2e90 legacy=0x7fa6180cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.163+0000 7fa617fff640 1 -- 172.21.15.76:0/917111082 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6180b0f60 con 0x7fa618161a60 2024-09-19T04:13:37.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa617fff640 1 --2- 172.21.15.76:0/917111082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa618161a60 0x7fa6180d2950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6080097c0 tx=0x7fa60801c3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59827ec7a6ceb09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:37.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa616ffd640 1 -- 172.21.15.76:0/917111082 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6080b1070 con 0x7fa618161a60 2024-09-19T04:13:37.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa616ffd640 1 -- 172.21.15.76:0/917111082 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6080a8050 con 0x7fa618161a60 2024-09-19T04:13:37.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa616ffd640 1 -- 172.21.15.76:0/917111082 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6080ac860 con 0x7fa618161a60 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 -- 172.21.15.76:0/917111082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa618161a60 msgr2=0x7fa6180d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 --2- 172.21.15.76:0/917111082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa618161a60 0x7fa6180d2950 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa6080097c0 tx=0x7fa60801c3e0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 -- 172.21.15.76:0/917111082 shutdown_connections 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 --2- 172.21.15.76:0/917111082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6180cf620 0x7fa61806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 --2- 172.21.15.76:0/917111082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa618161a60 0x7fa6180d2950 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 -- 172.21.15.76:0/917111082 >> 172.21.15.76:0/917111082 conn(0x7fa61808bf90 msgr2=0x7fa61809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 -- 172.21.15.76:0/917111082 shutdown_connections 2024-09-19T04:13:37.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 -- 172.21.15.76:0/917111082 wait complete. 2024-09-19T04:13:37.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 Processor -- start 2024-09-19T04:13:37.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.167+0000 7fa61e887640 1 -- start start 2024-09-19T04:13:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6180cf620 0x7fa61814a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6180d2e90 0x7fa6180d0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa618161a60 0x7fa6180d1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa61806e400 con 0x7fa6180cf620 2024-09-19T04:13:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa61806d190 con 0x7fa6180d2e90 2024-09-19T04:13:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa61806e580 con 0x7fa618161a60 2024-09-19T04:13:37.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6180cf620 0x7fa61814a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6180cf620 0x7fa61814a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40464/0 (socket says 172.21.15.76:40464) 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 -- 172.21.15.76:0/101344740 learned_addr learned my addr 172.21.15.76:0/101344740 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa6177fe640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6180d2e90 0x7fa6180d0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61ce24640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa618161a60 0x7fa6180d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 -- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa618161a60 msgr2=0x7fa6180d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa618161a60 0x7fa6180d1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 -- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6180d2e90 msgr2=0x7fa6180d0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6180d2e90 0x7fa6180d0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 -- 172.21.15.76:0/101344740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6180cb570 con 0x7fa6180cf620 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61ce24640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa618161a60 0x7fa6180d1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa617fff640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6180cf620 0x7fa61814a630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa60801aa90 tx=0x7fa608019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa614ff9640 1 -- 172.21.15.76:0/101344740 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6080ba070 con 0x7fa6180cf620 2024-09-19T04:13:37.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa614ff9640 1 -- 172.21.15.76:0/101344740 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6080acdf0 con 0x7fa6180cf620 2024-09-19T04:13:37.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6180cb850 con 0x7fa6180cf620 2024-09-19T04:13:37.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa614ff9640 1 -- 172.21.15.76:0/101344740 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6080b1040 con 0x7fa6180cf620 2024-09-19T04:13:37.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61814ab70 con 0x7fa6180cf620 2024-09-19T04:13:37.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61806d6b0 con 0x7fa6180cf620 2024-09-19T04:13:37.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa614ff9640 1 -- 172.21.15.76:0/101344740 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6080bd070 con 0x7fa6180cf620 2024-09-19T04:13:37.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa614ff9640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6040678c0 0x7fa604069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:37.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa614ff9640 1 -- 172.21.15.76:0/101344740 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa608120240 con 0x7fa6180cf620 2024-09-19T04:13:37.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.171+0000 7fa6177fe640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6040678c0 0x7fa604069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:37.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.175+0000 7fa6177fe640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6040678c0 0x7fa604069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6181fa480 tx=0x7fa60c007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:37.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.175+0000 7fa614ff9640 1 -- 172.21.15.76:0/101344740 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa6080c4030 con 0x7fa6180cf620 2024-09-19T04:13:37.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.359+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa6180c7c40 con 0x7fa6180cf620 2024-09-19T04:13:38.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.995+0000 7fa614ff9640 1 -- 172.21.15.76:0/101344740 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fa608016090 con 0x7fa6180cf620 2024-09-19T04:13:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6040678c0 msgr2=0x7fa604069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6040678c0 0x7fa604069d80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa6181fa480 tx=0x7fa60c007c90 comp rx=0 tx=0).stop 2024-09-19T04:13:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6180cf620 msgr2=0x7fa61814a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6180cf620 0x7fa61814a630 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa60801aa90 tx=0x7fa608019180 comp rx=0 tx=0).stop 2024-09-19T04:13:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 shutdown_connections 2024-09-19T04:13:38.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6040678c0 0x7fa604069d80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa618161a60 0x7fa6180d1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6180d2e90 0x7fa6180d0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 --2- 172.21.15.76:0/101344740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6180cf620 0x7fa61814a630 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 >> 172.21.15.76:0/101344740 conn(0x7fa61808bf90 msgr2=0x7fa6180d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:38.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 shutdown_connections 2024-09-19T04:13:38.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:37.999+0000 7fa61e887640 1 -- 172.21.15.76:0/101344740 wait complete. 2024-09-19T04:13:38.022 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-09-19T04:13:38.022 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T04:13:38.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b33000640 1 Processor -- start 2024-09-19T04:13:38.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b33000640 1 -- start start 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b33000640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b33000640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1b2c0b39c0 0x7f1b2c06eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b33000640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b2c15e6f0 con 0x7f1b2c15e310 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b33000640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1b2c0b3e80 con 0x7f1b2c0b39c0 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b33000640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b4160 con 0x7f1b2c06d3e0 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b30d9c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1b2c15e310 0x7f1b2c06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34260/0 (socket says 172.21.15.76:34260) 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b30d9c640 1 -- 172.21.15.76:0/17541550 learned_addr learned my addr 172.21.15.76:0/17541550 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b3159d640 1 --2- 172.21.15.76:0/17541550 >> v2:172.21.15.76:3300/0 conn(0x7f1b2c0b39c0 0x7f1b2c06eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b2b7fe640 1 -- 172.21.15.76:0/17541550 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3348794787 0 0) 0x7f1b240014a0 con 0x7f1b2c15e310 2024-09-19T04:13:38.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b2bfff640 1 --2- 172.21.15.76:0/17541550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:38.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b2b7fe640 1 -- 172.21.15.76:0/17541550 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b14003530 con 0x7f1b2c15e310 2024-09-19T04:13:38.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b3159d640 1 -- 172.21.15.76:0/17541550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 msgr2=0x7f1b2c0ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:38.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b3159d640 1 --2- 172.21.15.76:0/17541550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c0ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b3159d640 1 -- 172.21.15.76:0/17541550 >> v1:172.21.15.76:6789/0 conn(0x7f1b2c15e310 legacy=0x7f1b2c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.147+0000 7f1b3159d640 1 -- 172.21.15.76:0/17541550 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c0b4440 con 0x7f1b2c0b39c0 2024-09-19T04:13:38.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b3159d640 1 --2- 172.21.15.76:0/17541550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c0b39c0 0x7f1b2c06eac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0097c0 tx=0x7f1b1c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9153630d2dcca95e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:38.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b2b7fe640 1 -- 172.21.15.76:0/17541550 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0b1070 con 0x7f1b2c0b39c0 2024-09-19T04:13:38.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b2b7fe640 1 -- 172.21.15.76:0/17541550 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0ac410 con 0x7f1b2c0b39c0 2024-09-19T04:13:38.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b2b7fe640 1 -- 172.21.15.76:0/17541550 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0ac5e0 con 0x7f1b2c0b39c0 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 -- 172.21.15.76:0/17541550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c0b39c0 msgr2=0x7f1b2c06eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 --2- 172.21.15.76:0/17541550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c0b39c0 0x7f1b2c06eac0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0097c0 tx=0x7f1b1c01c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 -- 172.21.15.76:0/17541550 shutdown_connections 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 --2- 172.21.15.76:0/17541550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c0b39c0 0x7f1b2c06eac0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 --2- 172.21.15.76:0/17541550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c0ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 -- 172.21.15.76:0/17541550 >> 172.21.15.76:0/17541550 conn(0x7f1b2c0727b0 msgr2=0x7f1b2c072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 -- 172.21.15.76:0/17541550 shutdown_connections 2024-09-19T04:13:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.151+0000 7f1b33000640 1 -- 172.21.15.76:0/17541550 wait complete. 2024-09-19T04:13:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 Processor -- start 2024-09-19T04:13:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 -- start start 2024-09-19T04:13:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c13d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b2c0b39c0 0x7f1b2c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c15e310 0x7f1b2c14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c16afe0 con 0x7f1b2c15e310 2024-09-19T04:13:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c0b5140 con 0x7f1b2c0b39c0 2024-09-19T04:13:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2c16b160 con 0x7f1b2c06d3e0 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c15e310 0x7f1b2c14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c15e310 0x7f1b2c14f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40474/0 (socket says 172.21.15.76:40474) 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 -- 172.21.15.76:0/145799040 learned_addr learned my addr 172.21.15.76:0/145799040 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b30d9c640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c13d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 -- 172.21.15.76:0/145799040 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 msgr2=0x7f1b2c13d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c13d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 -- 172.21.15.76:0/145799040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b2c0b39c0 msgr2=0x7f1b2c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b2c0b39c0 0x7f1b2c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 -- 172.21.15.76:0/145799040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c13d6f0 con 0x7f1b2c15e310 2024-09-19T04:13:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b3159d640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c15e310 0x7f1b2c14f0b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c01a0d0 tx=0x7f1b1c019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b297fa640 1 -- 172.21.15.76:0/145799040 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0b1040 con 0x7f1b2c15e310 2024-09-19T04:13:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b297fa640 1 -- 172.21.15.76:0/145799040 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1c0ba070 con 0x7f1b2c15e310 2024-09-19T04:13:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c156940 con 0x7f1b2c15e310 2024-09-19T04:13:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b297fa640 1 -- 172.21.15.76:0/145799040 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0b5670 con 0x7f1b2c15e310 2024-09-19T04:13:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c156f00 con 0x7f1b2c15e310 2024-09-19T04:13:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c0ae780 con 0x7f1b2c15e310 2024-09-19T04:13:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b297fa640 1 -- 172.21.15.76:0/145799040 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1b1c0b5810 con 0x7f1b2c15e310 2024-09-19T04:13:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b297fa640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b180678f0 0x7f1b18069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b297fa640 1 -- 172.21.15.76:0/145799040 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1b1c11fc60 con 0x7f1b2c15e310 2024-09-19T04:13:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.155+0000 7f1b30d9c640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b180678f0 0x7f1b18069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.159+0000 7f1b30d9c640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b180678f0 0x7f1b18069db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b24001640 tx=0x7f1b24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:38.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.163+0000 7f1b297fa640 1 -- 172.21.15.76:0/145799040 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1b1c016090 con 0x7f1b2c15e310 2024-09-19T04:13:38.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.359+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1b2c15a550 con 0x7f1b2c15e310 2024-09-19T04:13:39.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:38.999+0000 7f1b297fa640 1 -- 172.21.15.76:0/145799040 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f1b1c0f04f0 con 0x7f1b2c15e310 2024-09-19T04:13:39.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b180678f0 msgr2=0x7f1b18069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b180678f0 0x7f1b18069db0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f1b24001640 tx=0x7f1b24009040 comp rx=0 tx=0).stop 2024-09-19T04:13:39.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c15e310 msgr2=0x7f1b2c14f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c15e310 0x7f1b2c14f0b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c01a0d0 tx=0x7f1b1c019240 comp rx=0 tx=0).stop 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 shutdown_connections 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b180678f0 0x7f1b18069db0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b2c15e310 0x7f1b2c14f0b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b2c0b39c0 0x7f1b2c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 --2- 172.21.15.76:0/145799040 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b2c06d3e0 0x7f1b2c13d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 >> 172.21.15.76:0/145799040 conn(0x7f1b2c0727b0 msgr2=0x7f1b2c15cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 shutdown_connections 2024-09-19T04:13:39.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.003+0000 7f1b33000640 1 -- 172.21.15.76:0/145799040 wait complete. 2024-09-19T04:13:39.025 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T04:13:39.025 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:39.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.143+0000 7fbf7200e640 1 Processor -- start 2024-09-19T04:13:39.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.143+0000 7fbf7200e640 1 -- start start 2024-09-19T04:13:39.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf7200e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15bec0 0x7fbf6c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf7200e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf6c15c2a0 0x7fbf6c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf7200e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf6c0b3290 con 0x7fbf6c0af2a0 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf7200e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbf6c0b3570 con 0x7fbf6c15c2a0 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf7200e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c0b3850 con 0x7fbf6c15bec0 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf6c15c2a0 0x7fbf6c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbf6c0af2a0 0x7fbf6c15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34276/0 (socket says 172.21.15.76:34276) 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6bfff640 1 -- 172.21.15.76:0/906213831 learned_addr learned my addr 172.21.15.76:0/906213831 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6b7fe640 1 --2- 172.21.15.76:0/906213831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15bec0 0x7fbf6c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6affd640 1 -- 172.21.15.76:0/906213831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15bec0 msgr2=0x7fbf6c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6affd640 1 --2- 172.21.15.76:0/906213831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15bec0 0x7fbf6c0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6affd640 1 -- 172.21.15.76:0/906213831 >> v1:172.21.15.76:6789/0 conn(0x7fbf6c0af2a0 legacy=0x7fbf6c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6a7fc640 1 -- 172.21.15.76:0/906213831 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2919693544 0 0) 0x7fbf6c0b3290 con 0x7fbf6c0af2a0 2024-09-19T04:13:39.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6affd640 1 -- 172.21.15.76:0/906213831 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c0b3b30 con 0x7fbf6c15c2a0 2024-09-19T04:13:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.147+0000 7fbf6affd640 1 --2- 172.21.15.76:0/906213831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15c2a0 0x7fbf6c0aed60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097c0 tx=0x7fbf6001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2403d874611983c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6a7fc640 1 -- 172.21.15.76:0/906213831 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600a7070 con 0x7fbf6c15c2a0 2024-09-19T04:13:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6a7fc640 1 -- 172.21.15.76:0/906213831 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf600b0070 con 0x7fbf6c15c2a0 2024-09-19T04:13:39.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6a7fc640 1 -- 172.21.15.76:0/906213831 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600ab890 con 0x7fbf6c15c2a0 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- 172.21.15.76:0/906213831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15c2a0 msgr2=0x7fbf6c0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 --2- 172.21.15.76:0/906213831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15c2a0 0x7fbf6c0aed60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbf600097c0 tx=0x7fbf6001c470 comp rx=0 tx=0).stop 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- 172.21.15.76:0/906213831 shutdown_connections 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 --2- 172.21.15.76:0/906213831 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15c2a0 0x7fbf6c0aed60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 --2- 172.21.15.76:0/906213831 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15bec0 0x7fbf6c0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- 172.21.15.76:0/906213831 >> 172.21.15.76:0/906213831 conn(0x7fbf6c0b1f90 msgr2=0x7fbf6c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- 172.21.15.76:0/906213831 shutdown_connections 2024-09-19T04:13:39.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- 172.21.15.76:0/906213831 wait complete. 2024-09-19T04:13:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 Processor -- start 2024-09-19T04:13:39.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- start start 2024-09-19T04:13:39.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0af2a0 0x7fbf6c14e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15bec0 0x7fbf6c14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15c2a0 0x7fbf6c156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0af6f0 con 0x7fbf6c15bec0 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6c0ad740 con 0x7fbf6c0af2a0 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6c169000 con 0x7fbf6c15c2a0 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15bec0 0x7fbf6c14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15bec0 0x7fbf6c14a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40494/0 (socket says 172.21.15.76:40494) 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 -- 172.21.15.76:0/2516584299 learned_addr learned my addr 172.21.15.76:0/2516584299 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 -- 172.21.15.76:0/2516584299 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15c2a0 msgr2=0x7fbf6c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15c2a0 0x7fbf6c156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 -- 172.21.15.76:0/2516584299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0af2a0 msgr2=0x7fbf6c14e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0af2a0 0x7fbf6c14e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 -- 172.21.15.76:0/2516584299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf6c15b660 con 0x7fbf6c15bec0 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf6affd640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15bec0 0x7fbf6c14a5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbf6001ab20 tx=0x7fbf60019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf53fff640 1 -- 172.21.15.76:0/2516584299 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600abd40 con 0x7fbf6c15bec0 2024-09-19T04:13:39.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf53fff640 1 -- 172.21.15.76:0/2516584299 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf600bf450 con 0x7fbf6c15bec0 2024-09-19T04:13:39.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf6c157cd0 con 0x7fbf6c15bec0 2024-09-19T04:13:39.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf53fff640 1 -- 172.21.15.76:0/2516584299 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf600bf840 con 0x7fbf6c15bec0 2024-09-19T04:13:39.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.151+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf6c156790 con 0x7fbf6c15bec0 2024-09-19T04:13:39.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.155+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf6c0ad200 con 0x7fbf6c15bec0 2024-09-19T04:13:39.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.155+0000 7fbf53fff640 1 -- 172.21.15.76:0/2516584299 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf600bf9e0 con 0x7fbf6c15bec0 2024-09-19T04:13:39.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.155+0000 7fbf53fff640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf580678c0 0x7fbf58069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.155+0000 7fbf53fff640 1 -- 172.21.15.76:0/2516584299 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbf60120d00 con 0x7fbf6c15bec0 2024-09-19T04:13:39.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.155+0000 7fbf6b7fe640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf580678c0 0x7fbf58069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.155+0000 7fbf6b7fe640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf580678c0 0x7fbf58069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0070b0 tx=0x7fbf5c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.159+0000 7fbf53fff640 1 -- 172.21.15.76:0/2516584299 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbf600c4060 con 0x7fbf6c15bec0 2024-09-19T04:13:39.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.339+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf6c09e330 con 0x7fbf6c15bec0 2024-09-19T04:13:39.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf53fff640 1 -- 172.21.15.76:0/2516584299 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7fbf60016090 con 0x7fbf6c15bec0 2024-09-19T04:13:39.344 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:39.344 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:13:39.345 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:13:39.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf580678c0 msgr2=0x7fbf58069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf580678c0 0x7fbf58069d80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c0070b0 tx=0x7fbf5c007040 comp rx=0 tx=0).stop 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15bec0 msgr2=0x7fbf6c14a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15bec0 0x7fbf6c14a5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbf6001ab20 tx=0x7fbf60019210 comp rx=0 tx=0).stop 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 shutdown_connections 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf580678c0 0x7fbf58069d80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf6c15c2a0 0x7fbf6c156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf6c15bec0 0x7fbf6c14a5a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 --2- 172.21.15.76:0/2516584299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0af2a0 0x7fbf6c14e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 >> 172.21.15.76:0/2516584299 conn(0x7fbf6c0b1f90 msgr2=0x7fbf6c074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 shutdown_connections 2024-09-19T04:13:39.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.343+0000 7fbf7200e640 1 -- 172.21.15.76:0/2516584299 wait complete. 2024-09-19T04:13:39.362 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:39.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 Processor -- start 2024-09-19T04:13:39.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 -- start start 2024-09-19T04:13:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f45f815e5f0 0x7f45f806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f45f80ce9d0 0x7f45f80cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f80d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f45f806d9c0 con 0x7f45f815e5f0 2024-09-19T04:13:39.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f45f806dca0 con 0x7f45f80ce9d0 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45fde45640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f45f806df80 con 0x7f45f80cf920 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f45f815e5f0 0x7f45f806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f45f815e5f0 0x7f45f806d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40508/0 (socket says 172.21.15.76:40508) 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 -- 172.21.15.76:0/1337539849 learned_addr learned my addr 172.21.15.76:0/1337539849 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45effff640 1 --2- 172.21.15.76:0/1337539849 >> v2:172.21.15.116:3300/0 conn(0x7f45f80ce9d0 0x7f45f80cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f7fff640 1 --2- 172.21.15.76:0/1337539849 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f80d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 -- 172.21.15.76:0/1337539849 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 msgr2=0x7f45f80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 --2- 172.21.15.76:0/1337539849 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f80d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 -- 172.21.15.76:0/1337539849 >> v2:172.21.15.116:3300/0 conn(0x7f45f80ce9d0 msgr2=0x7f45f80cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 --2- 172.21.15.76:0/1337539849 >> v2:172.21.15.116:3300/0 conn(0x7f45f80ce9d0 0x7f45f80cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.467+0000 7f45f77fe640 1 -- 172.21.15.76:0/1337539849 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f806e260 con 0x7f45f815e5f0 2024-09-19T04:13:39.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45f77fe640 1 --2- 172.21.15.76:0/1337539849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f815e5f0 0x7f45f806d090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ce980 tx=0x7f45e801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd119229e4e8d5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45f6ffd640 1 -- 172.21.15.76:0/1337539849 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e80b0070 con 0x7f45f815e5f0 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45f6ffd640 1 -- 172.21.15.76:0/1337539849 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e80ab600 con 0x7f45f815e5f0 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45f6ffd640 1 -- 172.21.15.76:0/1337539849 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e80ab9d0 con 0x7f45f815e5f0 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- 172.21.15.76:0/1337539849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f815e5f0 msgr2=0x7f45f806d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 --2- 172.21.15.76:0/1337539849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f815e5f0 0x7f45f806d090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f45f80ce980 tx=0x7f45e801c260 comp rx=0 tx=0).stop 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- 172.21.15.76:0/1337539849 shutdown_connections 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 --2- 172.21.15.76:0/1337539849 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 --2- 172.21.15.76:0/1337539849 >> v2:172.21.15.116:3300/0 conn(0x7f45f80ce9d0 0x7f45f80cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 --2- 172.21.15.76:0/1337539849 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f815e5f0 0x7f45f806d090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- 172.21.15.76:0/1337539849 >> 172.21.15.76:0/1337539849 conn(0x7f45f8074030 msgr2=0x7f45f809d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- 172.21.15.76:0/1337539849 shutdown_connections 2024-09-19T04:13:39.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- 172.21.15.76:0/1337539849 wait complete. 2024-09-19T04:13:39.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 Processor -- start 2024-09-19T04:13:39.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- start start 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f814e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f814e9c0 0x7f45f80b9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45f80b97c0 0x7f45f815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f45f80d3740 con 0x7f45f814e9c0 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f45f80d35c0 con 0x7f45f80b97c0 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45fde45640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f45f80d38c0 con 0x7f45f80cf920 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f814e9c0 0x7f45f80b9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f814e9c0 0x7f45f80b9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40514/0 (socket says 172.21.15.76:40514) 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 -- 172.21.15.76:0/468712497 learned_addr learned my addr 172.21.15.76:0/468712497 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:39.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45f7fff640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45f80b97c0 0x7f45f815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45f77fe640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f814e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 -- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 msgr2=0x7f45f814e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f814e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 -- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45f80b97c0 msgr2=0x7f45f815a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45f80b97c0 0x7f45f815a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.471+0000 7f45effff640 1 -- 172.21.15.76:0/468712497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45f80b9c60 con 0x7f45f814e9c0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f7fff640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45f80b97c0 0x7f45f815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45effff640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f814e9c0 0x7f45f80b9280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45e000d800 tx=0x7f45e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f4ff9640 1 -- 172.21.15.76:0/468712497 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0018020 con 0x7f45f814e9c0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f4ff9640 1 -- 172.21.15.76:0/468712497 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45e00096a0 con 0x7f45f814e9c0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f4ff9640 1 -- 172.21.15.76:0/468712497 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e0009a50 con 0x7f45f814e9c0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45f80b9f40 con 0x7f45f814e9c0 2024-09-19T04:13:39.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45f8157d50 con 0x7f45f814e9c0 2024-09-19T04:13:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45f80d3c80 con 0x7f45f814e9c0 2024-09-19T04:13:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f4ff9640 1 -- 172.21.15.76:0/468712497 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f45e001b020 con 0x7f45f814e9c0 2024-09-19T04:13:39.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f4ff9640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45d80678c0 0x7f45d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f4ff9640 1 -- 172.21.15.76:0/468712497 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f45e0086520 con 0x7f45f814e9c0 2024-09-19T04:13:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f77fe640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45d80678c0 0x7f45d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.475+0000 7f45f77fe640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45d80678c0 0x7f45d8069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ad000 tx=0x7f45e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.479+0000 7f45f4ff9640 1 -- 172.21.15.76:0/468712497 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f45e0022020 con 0x7f45f814e9c0 2024-09-19T04:13:39.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.643+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45f81588e0 con 0x7f45f814e9c0 2024-09-19T04:13:39.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45f4ff9640 1 -- 172.21.15.76:0/468712497 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f45e0056db0 con 0x7f45f814e9c0 2024-09-19T04:13:39.648 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:39.648 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45d80678c0 msgr2=0x7f45d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45d80678c0 0x7f45d8069d80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f45e80ad000 tx=0x7f45e8008040 comp rx=0 tx=0).stop 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f814e9c0 msgr2=0x7f45f80b9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f814e9c0 0x7f45f80b9280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f45e000d800 tx=0x7f45e000dcc0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 shutdown_connections 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45d80678c0 0x7f45d8069d80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45f80b97c0 0x7f45f815a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45f814e9c0 0x7f45f80b9280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 --2- 172.21.15.76:0/468712497 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45f80cf920 0x7f45f814e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 >> 172.21.15.76:0/468712497 conn(0x7f45f80ce9d0 msgr2=0x7f45f80d40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 shutdown_connections 2024-09-19T04:13:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.647+0000 7f45fde45640 1 -- 172.21.15.76:0/468712497 wait complete. 2024-09-19T04:13:39.667 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:13:39.667 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:13:39.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531eea9640 1 Processor -- start 2024-09-19T04:13:39.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531eea9640 1 -- start start 2024-09-19T04:13:39.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531eea9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53180cf620 0x7f5318168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531eea9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53180d0300 con 0x7f53180d3060 2024-09-19T04:13:39.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531eea9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f531806e6c0 con 0x7f5318161a60 2024-09-19T04:13:39.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531eea9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f53180cfa90 con 0x7f53180cf620 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531d446640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53180cf620 0x7f5318168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f5317fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f53180d3060 0x7f53180cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34288/0 (socket says 172.21.15.76:34288) 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531d446640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f53180cf620 0x7f5318168f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40520/0 (socket says 172.21.15.76:40520) 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f5317fff640 1 -- 172.21.15.76:0/2702764361 learned_addr learned my addr 172.21.15.76:0/2702764361 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531d446640 1 -- 172.21.15.76:0/2702764361 >> v1:172.21.15.116:6789/0 conn(0x7f5318161a60 legacy=0x7f53180d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531d446640 1 -- 172.21.15.76:0/2702764361 >> v1:172.21.15.76:6789/0 conn(0x7f53180d3060 legacy=0x7f53180cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531d446640 1 -- 172.21.15.76:0/2702764361 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53180b0f60 con 0x7f53180cf620 2024-09-19T04:13:39.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f53177fe640 1 -- 172.21.15.76:0/2702764361 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1817121811 0 0) 0x7f5304001120 con 0x7f53180d3060 2024-09-19T04:13:39.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.779+0000 7f531d446640 1 --2- 172.21.15.76:0/2702764361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318168f10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5310009d90 tx=0x7f531001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653a1505fb8dd5d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f53177fe640 1 -- 172.21.15.76:0/2702764361 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53100b2070 con 0x7f53180cf620 2024-09-19T04:13:39.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f53177fe640 1 -- 172.21.15.76:0/2702764361 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53100a5970 con 0x7f53180cf620 2024-09-19T04:13:39.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f53177fe640 1 -- 172.21.15.76:0/2702764361 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53100a5d40 con 0x7f53180cf620 2024-09-19T04:13:39.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- 172.21.15.76:0/2702764361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 msgr2=0x7f5318168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 --2- 172.21.15.76:0/2702764361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318168f10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5310009d90 tx=0x7f531001c8b0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- 172.21.15.76:0/2702764361 shutdown_connections 2024-09-19T04:13:39.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 --2- 172.21.15.76:0/2702764361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318168f10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- 172.21.15.76:0/2702764361 >> 172.21.15.76:0/2702764361 conn(0x7f531808bf90 msgr2=0x7f531809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- 172.21.15.76:0/2702764361 shutdown_connections 2024-09-19T04:13:39.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- 172.21.15.76:0/2702764361 wait complete. 2024-09-19T04:13:39.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 Processor -- start 2024-09-19T04:13:39.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- start start 2024-09-19T04:13:39.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53180d3060 0x7f531815c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5318161a60 0x7f531815c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f531816bc70 con 0x7f53180cf620 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f531806d190 con 0x7f5318161a60 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f531816bdf0 con 0x7f53180d3060 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40528/0 (socket says 172.21.15.76:40528) 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 -- 172.21.15.76:0/269718001 learned_addr learned my addr 172.21.15.76:0/269718001 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 -- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53180d3060 msgr2=0x7f531815c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f5317fff640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53180d3060 0x7f531815c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531d446640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5318161a60 0x7f531815c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53180d3060 0x7f531815c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 -- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5318161a60 msgr2=0x7f531815c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5318161a60 0x7f531815c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 -- 172.21.15.76:0/269718001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53181611f0 con 0x7f53180cf620 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531cc45640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318159560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c3c0 tx=0x7f530c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f53157fa640 1 -- 172.21.15.76:0/269718001 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c017070 con 0x7f53180cf620 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f53157fa640 1 -- 172.21.15.76:0/269718001 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f530c0052d0 con 0x7f53180cf620 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f53157fa640 1 -- 172.21.15.76:0/269718001 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c003840 con 0x7f53180cf620 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53180b49e0 con 0x7f53180cf620 2024-09-19T04:13:39.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.783+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53180cb970 con 0x7f53180cf620 2024-09-19T04:13:39.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.787+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53180cb420 con 0x7f53180cf620 2024-09-19T04:13:39.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.787+0000 7f53157fa640 1 -- 172.21.15.76:0/269718001 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f530c01a070 con 0x7f53180cf620 2024-09-19T04:13:39.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.787+0000 7f53157fa640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52fc0678c0 0x7f52fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:39.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.787+0000 7f53157fa640 1 -- 172.21.15.76:0/269718001 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f530c085280 con 0x7f53180cf620 2024-09-19T04:13:39.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.787+0000 7f5317fff640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52fc0678c0 0x7f52fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:39.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.787+0000 7f5317fff640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52fc0678c0 0x7f52fc069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5304007fd0 tx=0x7f5304009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:39.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.791+0000 7f53157fa640 1 -- 172.21.15.76:0/269718001 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f530c005480 con 0x7f53180cf620 2024-09-19T04:13:39.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.939+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f53180cef10 con 0x7f52fc0678c0 2024-09-19T04:13:39.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.947+0000 7f53157fa640 1 -- 172.21.15.76:0/269718001 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7f53180cef10 con 0x7f52fc0678c0 2024-09-19T04:13:39.954 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:39.956 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52fc0678c0 msgr2=0x7f52fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52fc0678c0 0x7f52fc069d80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5304007fd0 tx=0x7f5304009040 comp rx=0 tx=0).stop 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 msgr2=0x7f5318159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318159560 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f530c00c3c0 tx=0x7f530c00c880 comp rx=0 tx=0).stop 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 shutdown_connections 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52fc0678c0 0x7f52fc069d80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5318161a60 0x7f531815c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53180d3060 0x7f531815c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 --2- 172.21.15.76:0/269718001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f53180cf620 0x7f5318159560 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:39.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 >> 172.21.15.76:0/269718001 conn(0x7f531808bf90 msgr2=0x7f53180b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:39.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 shutdown_connections 2024-09-19T04:13:39.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:39.951+0000 7f531eea9640 1 -- 172.21.15.76:0/269718001 wait complete. 2024-09-19T04:13:39.971 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-19T04:13:39.741662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223668,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751052,"statfs":{"total":773094113280,"available":772865077248,"internally_reserved":0,"allocated":3350528,"data_stored":2017900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003044"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062396+0000","last_change":"2024-09-19T04:13:23.917029+0000","last_active":"2024-09-19T04:13:31.062396+0000","last_peered":"2024-09-19T04:13:31.062396+0000","last_clean":"2024-09-19T04:13:31.062396+0000","last_became_active":"2024-09-19T04:13:23.916640+0000","last_became_peered":"2024-09-19T04:13:23.916640+0000","last_unstale":"2024-09-19T04:13:31.062396+0000","last_undegraded":"2024-09-19T04:13:31.062396+0000","last_fullsized":"2024-09-19T04:13:31.062396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:22.981160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301574+0000","last_change":"2024-09-19T04:13:24.947160+0000","last_active":"2024-09-19T04:13:30.301574+0000","last_peered":"2024-09-19T04:13:30.301574+0000","last_clean":"2024-09-19T04:13:30.301574+0000","last_became_active":"2024-09-19T04:13:24.946824+0000","last_became_peered":"2024-09-19T04:13:24.946824+0000","last_unstale":"2024-09-19T04:13:30.301574+0000","last_undegraded":"2024-09-19T04:13:30.301574+0000","last_fullsized":"2024-09-19T04:13:30.301574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300320+0000","last_change":"2024-09-19T04:13:29.029580+0000","last_active":"2024-09-19T04:13:30.300320+0000","last_peered":"2024-09-19T04:13:30.300320+0000","last_clean":"2024-09-19T04:13:30.300320+0000","last_became_active":"2024-09-19T04:13:29.029159+0000","last_became_peered":"2024-09-19T04:13:29.029159+0000","last_unstale":"2024-09-19T04:13:30.300320+0000","last_undegraded":"2024-09-19T04:13:30.300320+0000","last_fullsized":"2024-09-19T04:13:30.300320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301237+0000","last_change":"2024-09-19T04:13:23.910773+0000","last_active":"2024-09-19T04:13:30.301237+0000","last_peered":"2024-09-19T04:13:30.301237+0000","last_clean":"2024-09-19T04:13:30.301237+0000","last_became_active":"2024-09-19T04:13:23.910355+0000","last_became_peered":"2024-09-19T04:13:23.910355+0000","last_unstale":"2024-09-19T04:13:30.301237+0000","last_undegraded":"2024-09-19T04:13:30.301237+0000","last_fullsized":"2024-09-19T04:13:30.301237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062270+0000","last_change":"2024-09-19T04:13:24.956969+0000","last_active":"2024-09-19T04:13:31.062270+0000","last_peered":"2024-09-19T04:13:31.062270+0000","last_clean":"2024-09-19T04:13:31.062270+0000","last_became_active":"2024-09-19T04:13:24.956671+0000","last_became_peered":"2024-09-19T04:13:24.956671+0000","last_unstale":"2024-09-19T04:13:31.062270+0000","last_undegraded":"2024-09-19T04:13:31.062270+0000","last_fullsized":"2024-09-19T04:13:31.062270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061243+0000","last_change":"2024-09-19T04:13:28.991785+0000","last_active":"2024-09-19T04:13:31.061243+0000","last_peered":"2024-09-19T04:13:31.061243+0000","last_clean":"2024-09-19T04:13:31.061243+0000","last_became_active":"2024-09-19T04:13:28.987169+0000","last_became_peered":"2024-09-19T04:13:28.987169+0000","last_unstale":"2024-09-19T04:13:31.061243+0000","last_undegraded":"2024-09-19T04:13:31.061243+0000","last_fullsized":"2024-09-19T04:13:31.061243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:00.614358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062999+0000","last_change":"2024-09-19T04:13:23.924693+0000","last_active":"2024-09-19T04:13:31.062999+0000","last_peered":"2024-09-19T04:13:31.062999+0000","last_clean":"2024-09-19T04:13:31.062999+0000","last_became_active":"2024-09-19T04:13:23.924077+0000","last_became_peered":"2024-09-19T04:13:23.924077+0000","last_unstale":"2024-09-19T04:13:31.062999+0000","last_undegraded":"2024-09-19T04:13:31.062999+0000","last_fullsized":"2024-09-19T04:13:31.062999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:10.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970152+0000","last_change":"2024-09-19T04:13:24.948486+0000","last_active":"2024-09-19T04:13:29.970152+0000","last_peered":"2024-09-19T04:13:29.970152+0000","last_clean":"2024-09-19T04:13:29.970152+0000","last_became_active":"2024-09-19T04:13:24.947749+0000","last_became_peered":"2024-09-19T04:13:24.947749+0000","last_unstale":"2024-09-19T04:13:29.970152+0000","last_undegraded":"2024-09-19T04:13:29.970152+0000","last_fullsized":"2024-09-19T04:13:29.970152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:37.654967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063108+0000","last_change":"2024-09-19T04:13:28.992157+0000","last_active":"2024-09-19T04:13:31.063108+0000","last_peered":"2024-09-19T04:13:31.063108+0000","last_clean":"2024-09-19T04:13:31.063108+0000","last_became_active":"2024-09-19T04:13:28.990879+0000","last_became_peered":"2024-09-19T04:13:28.990879+0000","last_unstale":"2024-09-19T04:13:31.063108+0000","last_undegraded":"2024-09-19T04:13:31.063108+0000","last_fullsized":"2024-09-19T04:13:31.063108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:39.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062192+0000","last_change":"2024-09-19T04:13:23.923303+0000","last_active":"2024-09-19T04:13:31.062192+0000","last_peered":"2024-09-19T04:13:31.062192+0000","last_clean":"2024-09-19T04:13:31.062192+0000","last_became_active":"2024-09-19T04:13:23.920842+0000","last_became_peered":"2024-09-19T04:13:23.920842+0000","last_unstale":"2024-09-19T04:13:31.062192+0000","last_undegraded":"2024-09-19T04:13:31.062192+0000","last_fullsized":"2024-09-19T04:13:31.062192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:23.176892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971159+0000","last_change":"2024-09-19T04:13:24.948074+0000","last_active":"2024-09-19T04:13:29.971159+0000","last_peered":"2024-09-19T04:13:29.971159+0000","last_clean":"2024-09-19T04:13:29.971159+0000","last_became_active":"2024-09-19T04:13:24.946595+0000","last_became_peered":"2024-09-19T04:13:24.946595+0000","last_unstale":"2024-09-19T04:13:29.971159+0000","last_undegraded":"2024-09-19T04:13:29.971159+0000","last_fullsized":"2024-09-19T04:13:29.971159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:21.342803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062406+0000","last_change":"2024-09-19T04:13:28.985965+0000","last_active":"2024-09-19T04:13:31.062406+0000","last_peered":"2024-09-19T04:13:31.062406+0000","last_clean":"2024-09-19T04:13:31.062406+0000","last_became_active":"2024-09-19T04:13:28.979842+0000","last_became_peered":"2024-09-19T04:13:28.979842+0000","last_unstale":"2024-09-19T04:13:31.062406+0000","last_undegraded":"2024-09-19T04:13:31.062406+0000","last_fullsized":"2024-09-19T04:13:31.062406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:13.654228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971528+0000","last_change":"2024-09-19T04:13:24.964645+0000","last_active":"2024-09-19T04:13:29.971528+0000","last_peered":"2024-09-19T04:13:29.971528+0000","last_clean":"2024-09-19T04:13:29.971528+0000","last_became_active":"2024-09-19T04:13:24.964051+0000","last_became_peered":"2024-09-19T04:13:24.964051+0000","last_unstale":"2024-09-19T04:13:29.971528+0000","last_undegraded":"2024-09-19T04:13:29.971528+0000","last_fullsized":"2024-09-19T04:13:29.971528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062721+0000","last_change":"2024-09-19T04:13:23.921499+0000","last_active":"2024-09-19T04:13:31.062721+0000","last_peered":"2024-09-19T04:13:31.062721+0000","last_clean":"2024-09-19T04:13:31.062721+0000","last_became_active":"2024-09-19T04:13:23.921146+0000","last_became_peered":"2024-09-19T04:13:23.921146+0000","last_unstale":"2024-09-19T04:13:31.062721+0000","last_undegraded":"2024-09-19T04:13:31.062721+0000","last_fullsized":"2024-09-19T04:13:31.062721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:54.181515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063276+0000","last_change":"2024-09-19T04:13:28.979497+0000","last_active":"2024-09-19T04:13:31.063276+0000","last_peered":"2024-09-19T04:13:31.063276+0000","last_clean":"2024-09-19T04:13:31.063276+0000","last_became_active":"2024-09-19T04:13:28.978329+0000","last_became_peered":"2024-09-19T04:13:28.978329+0000","last_unstale":"2024-09-19T04:13:31.063276+0000","last_undegraded":"2024-09-19T04:13:31.063276+0000","last_fullsized":"2024-09-19T04:13:31.063276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:30.314159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300960+0000","last_change":"2024-09-19T04:13:24.951290+0000","last_active":"2024-09-19T04:13:30.300960+0000","last_peered":"2024-09-19T04:13:30.300960+0000","last_clean":"2024-09-19T04:13:30.300960+0000","last_became_active":"2024-09-19T04:13:24.950973+0000","last_became_peered":"2024-09-19T04:13:24.950973+0000","last_unstale":"2024-09-19T04:13:30.300960+0000","last_undegraded":"2024-09-19T04:13:30.300960+0000","last_fullsized":"2024-09-19T04:13:30.300960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131596+0000","last_change":"2024-09-19T04:13:23.918153+0000","last_active":"2024-09-19T04:13:30.131596+0000","last_peered":"2024-09-19T04:13:30.131596+0000","last_clean":"2024-09-19T04:13:30.131596+0000","last_became_active":"2024-09-19T04:13:23.917783+0000","last_became_peered":"2024-09-19T04:13:23.917783+0000","last_unstale":"2024-09-19T04:13:30.131596+0000","last_undegraded":"2024-09-19T04:13:30.131596+0000","last_fullsized":"2024-09-19T04:13:30.131596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063970+0000","last_change":"2024-09-19T04:13:28.988863+0000","last_active":"2024-09-19T04:13:31.063970+0000","last_peered":"2024-09-19T04:13:31.063970+0000","last_clean":"2024-09-19T04:13:31.063970+0000","last_became_active":"2024-09-19T04:13:28.987942+0000","last_became_peered":"2024-09-19T04:13:28.987942+0000","last_unstale":"2024-09-19T04:13:31.063970+0000","last_undegraded":"2024-09-19T04:13:31.063970+0000","last_fullsized":"2024-09-19T04:13:31.063970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300167+0000","last_change":"2024-09-19T04:13:24.946137+0000","last_active":"2024-09-19T04:13:30.300167+0000","last_peered":"2024-09-19T04:13:30.300167+0000","last_clean":"2024-09-19T04:13:30.300167+0000","last_became_active":"2024-09-19T04:13:24.945850+0000","last_became_peered":"2024-09-19T04:13:24.945850+0000","last_unstale":"2024-09-19T04:13:30.300167+0000","last_undegraded":"2024-09-19T04:13:30.300167+0000","last_fullsized":"2024-09-19T04:13:30.300167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:23.916109+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:23.915767+0000","last_became_peered":"2024-09-19T04:13:23.915767+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131755+0000","last_change":"2024-09-19T04:13:29.030707+0000","last_active":"2024-09-19T04:13:30.131755+0000","last_peered":"2024-09-19T04:13:30.131755+0000","last_clean":"2024-09-19T04:13:30.131755+0000","last_became_active":"2024-09-19T04:13:29.030294+0000","last_became_peered":"2024-09-19T04:13:29.030294+0000","last_unstale":"2024-09-19T04:13:30.131755+0000","last_undegraded":"2024-09-19T04:13:30.131755+0000","last_fullsized":"2024-09-19T04:13:30.131755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063152+0000","last_change":"2024-09-19T04:13:24.948148+0000","last_active":"2024-09-19T04:13:31.063152+0000","last_peered":"2024-09-19T04:13:31.063152+0000","last_clean":"2024-09-19T04:13:31.063152+0000","last_became_active":"2024-09-19T04:13:24.947330+0000","last_became_peered":"2024-09-19T04:13:24.947330+0000","last_unstale":"2024-09-19T04:13:31.063152+0000","last_undegraded":"2024-09-19T04:13:31.063152+0000","last_fullsized":"2024-09-19T04:13:31.063152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:49.624559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061085+0000","last_change":"2024-09-19T04:13:23.923583+0000","last_active":"2024-09-19T04:13:31.061085+0000","last_peered":"2024-09-19T04:13:31.061085+0000","last_clean":"2024-09-19T04:13:31.061085+0000","last_became_active":"2024-09-19T04:13:23.921593+0000","last_became_peered":"2024-09-19T04:13:23.921593+0000","last_unstale":"2024-09-19T04:13:31.061085+0000","last_undegraded":"2024-09-19T04:13:31.061085+0000","last_fullsized":"2024-09-19T04:13:31.061085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063591+0000","last_change":"2024-09-19T04:13:28.990856+0000","last_active":"2024-09-19T04:13:31.063591+0000","last_peered":"2024-09-19T04:13:31.063591+0000","last_clean":"2024-09-19T04:13:31.063591+0000","last_became_active":"2024-09-19T04:13:28.987830+0000","last_became_peered":"2024-09-19T04:13:28.987830+0000","last_unstale":"2024-09-19T04:13:31.063591+0000","last_undegraded":"2024-09-19T04:13:31.063591+0000","last_fullsized":"2024-09-19T04:13:31.063591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062632+0000","last_change":"2024-09-19T04:13:24.934798+0000","last_active":"2024-09-19T04:13:31.062632+0000","last_peered":"2024-09-19T04:13:31.062632+0000","last_clean":"2024-09-19T04:13:31.062632+0000","last_became_active":"2024-09-19T04:13:24.933977+0000","last_became_peered":"2024-09-19T04:13:24.933977+0000","last_unstale":"2024-09-19T04:13:31.062632+0000","last_undegraded":"2024-09-19T04:13:31.062632+0000","last_fullsized":"2024-09-19T04:13:31.062632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973271+0000","last_change":"2024-09-19T04:13:23.920837+0000","last_active":"2024-09-19T04:13:29.973271+0000","last_peered":"2024-09-19T04:13:29.973271+0000","last_clean":"2024-09-19T04:13:29.973271+0000","last_became_active":"2024-09-19T04:13:23.920245+0000","last_became_peered":"2024-09-19T04:13:23.920245+0000","last_unstale":"2024-09-19T04:13:29.973271+0000","last_undegraded":"2024-09-19T04:13:29.973271+0000","last_fullsized":"2024-09-19T04:13:29.973271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062572+0000","last_change":"2024-09-19T04:13:29.019257+0000","last_active":"2024-09-19T04:13:31.062572+0000","last_peered":"2024-09-19T04:13:31.062572+0000","last_clean":"2024-09-19T04:13:31.062572+0000","last_became_active":"2024-09-19T04:13:29.018425+0000","last_became_peered":"2024-09-19T04:13:29.018425+0000","last_unstale":"2024-09-19T04:13:31.062572+0000","last_undegraded":"2024-09-19T04:13:31.062572+0000","last_fullsized":"2024-09-19T04:13:31.062572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:42.657248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301533+0000","last_change":"2024-09-19T04:13:24.944069+0000","last_active":"2024-09-19T04:13:30.301533+0000","last_peered":"2024-09-19T04:13:30.301533+0000","last_clean":"2024-09-19T04:13:30.301533+0000","last_became_active":"2024-09-19T04:13:24.943774+0000","last_became_peered":"2024-09-19T04:13:24.943774+0000","last_unstale":"2024-09-19T04:13:30.301533+0000","last_undegraded":"2024-09-19T04:13:30.301533+0000","last_fullsized":"2024-09-19T04:13:30.301533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131771+0000","last_change":"2024-09-19T04:13:23.911708+0000","last_active":"2024-09-19T04:13:30.131771+0000","last_peered":"2024-09-19T04:13:30.131771+0000","last_clean":"2024-09-19T04:13:30.131771+0000","last_became_active":"2024-09-19T04:13:23.911135+0000","last_became_peered":"2024-09-19T04:13:23.911135+0000","last_unstale":"2024-09-19T04:13:30.131771+0000","last_undegraded":"2024-09-19T04:13:30.131771+0000","last_fullsized":"2024-09-19T04:13:30.131771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:48.393589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299979+0000","last_change":"2024-09-19T04:13:29.020657+0000","last_active":"2024-09-19T04:13:30.299979+0000","last_peered":"2024-09-19T04:13:30.299979+0000","last_clean":"2024-09-19T04:13:30.299979+0000","last_became_active":"2024-09-19T04:13:29.020172+0000","last_became_peered":"2024-09-19T04:13:29.020172+0000","last_unstale":"2024-09-19T04:13:30.299979+0000","last_undegraded":"2024-09-19T04:13:30.299979+0000","last_fullsized":"2024-09-19T04:13:30.299979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131405+0000","last_change":"2024-09-19T04:13:24.949008+0000","last_active":"2024-09-19T04:13:30.131405+0000","last_peered":"2024-09-19T04:13:30.131405+0000","last_clean":"2024-09-19T04:13:30.131405+0000","last_became_active":"2024-09-19T04:13:24.948513+0000","last_became_peered":"2024-09-19T04:13:24.948513+0000","last_unstale":"2024-09-19T04:13:30.131405+0000","last_undegraded":"2024-09-19T04:13:30.131405+0000","last_fullsized":"2024-09-19T04:13:30.131405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063250+0000","last_change":"2024-09-19T04:13:23.921473+0000","last_active":"2024-09-19T04:13:31.063250+0000","last_peered":"2024-09-19T04:13:31.063250+0000","last_clean":"2024-09-19T04:13:31.063250+0000","last_became_active":"2024-09-19T04:13:23.921052+0000","last_became_peered":"2024-09-19T04:13:23.921052+0000","last_unstale":"2024-09-19T04:13:31.063250+0000","last_undegraded":"2024-09-19T04:13:31.063250+0000","last_fullsized":"2024-09-19T04:13:31.063250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:17.114813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061949+0000","last_change":"2024-09-19T04:13:29.020022+0000","last_active":"2024-09-19T04:13:31.061949+0000","last_peered":"2024-09-19T04:13:31.061949+0000","last_clean":"2024-09-19T04:13:31.061949+0000","last_became_active":"2024-09-19T04:13:29.019585+0000","last_became_peered":"2024-09-19T04:13:29.019585+0000","last_unstale":"2024-09-19T04:13:31.061949+0000","last_undegraded":"2024-09-19T04:13:31.061949+0000","last_fullsized":"2024-09-19T04:13:31.061949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970941+0000","last_change":"2024-09-19T04:13:24.952102+0000","last_active":"2024-09-19T04:13:29.970941+0000","last_peered":"2024-09-19T04:13:29.970941+0000","last_clean":"2024-09-19T04:13:29.970941+0000","last_became_active":"2024-09-19T04:13:24.951184+0000","last_became_peered":"2024-09-19T04:13:24.951184+0000","last_unstale":"2024-09-19T04:13:29.970941+0000","last_undegraded":"2024-09-19T04:13:29.970941+0000","last_fullsized":"2024-09-19T04:13:29.970941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131508+0000","last_change":"2024-09-19T04:13:23.908259+0000","last_active":"2024-09-19T04:13:30.131508+0000","last_peered":"2024-09-19T04:13:30.131508+0000","last_clean":"2024-09-19T04:13:30.131508+0000","last_became_active":"2024-09-19T04:13:23.907717+0000","last_became_peered":"2024-09-19T04:13:23.907717+0000","last_unstale":"2024-09-19T04:13:30.131508+0000","last_undegraded":"2024-09-19T04:13:30.131508+0000","last_fullsized":"2024-09-19T04:13:30.131508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.975119+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.974476+0000","last_became_peered":"2024-09-19T04:13:28.974476+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063922+0000","last_change":"2024-09-19T04:13:24.922621+0000","last_active":"2024-09-19T04:13:31.063922+0000","last_peered":"2024-09-19T04:13:31.063922+0000","last_clean":"2024-09-19T04:13:31.063922+0000","last_became_active":"2024-09-19T04:13:24.921879+0000","last_became_peered":"2024-09-19T04:13:24.921879+0000","last_unstale":"2024-09-19T04:13:31.063922+0000","last_undegraded":"2024-09-19T04:13:31.063922+0000","last_fullsized":"2024-09-19T04:13:31.063922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:04.750679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972020+0000","last_change":"2024-09-19T04:13:23.901650+0000","last_active":"2024-09-19T04:13:29.972020+0000","last_peered":"2024-09-19T04:13:29.972020+0000","last_clean":"2024-09-19T04:13:29.972020+0000","last_became_active":"2024-09-19T04:13:23.900880+0000","last_became_peered":"2024-09-19T04:13:23.900880+0000","last_unstale":"2024-09-19T04:13:29.972020+0000","last_undegraded":"2024-09-19T04:13:29.972020+0000","last_fullsized":"2024-09-19T04:13:29.972020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131635+0000","last_change":"2024-09-19T04:13:28.980693+0000","last_active":"2024-09-19T04:13:30.131635+0000","last_peered":"2024-09-19T04:13:30.131635+0000","last_clean":"2024-09-19T04:13:30.131635+0000","last_became_active":"2024-09-19T04:13:28.979948+0000","last_became_peered":"2024-09-19T04:13:28.979948+0000","last_unstale":"2024-09-19T04:13:30.131635+0000","last_undegraded":"2024-09-19T04:13:30.131635+0000","last_fullsized":"2024-09-19T04:13:30.131635+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130985+0000","last_change":"2024-09-19T04:13:24.948867+0000","last_active":"2024-09-19T04:13:30.130985+0000","last_peered":"2024-09-19T04:13:30.130985+0000","last_clean":"2024-09-19T04:13:30.130985+0000","last_became_active":"2024-09-19T04:13:24.948471+0000","last_became_peered":"2024-09-19T04:13:24.948471+0000","last_unstale":"2024-09-19T04:13:30.130985+0000","last_undegraded":"2024-09-19T04:13:30.130985+0000","last_fullsized":"2024-09-19T04:13:30.130985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970768+0000","last_change":"2024-09-19T04:13:23.903592+0000","last_active":"2024-09-19T04:13:29.970768+0000","last_peered":"2024-09-19T04:13:29.970768+0000","last_clean":"2024-09-19T04:13:29.970768+0000","last_became_active":"2024-09-19T04:13:23.903306+0000","last_became_peered":"2024-09-19T04:13:23.903306+0000","last_unstale":"2024-09-19T04:13:29.970768+0000","last_undegraded":"2024-09-19T04:13:29.970768+0000","last_fullsized":"2024-09-19T04:13:29.970768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:13.135208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131834+0000","last_change":"2024-09-19T04:13:28.976039+0000","last_active":"2024-09-19T04:13:30.131834+0000","last_peered":"2024-09-19T04:13:30.131834+0000","last_clean":"2024-09-19T04:13:30.131834+0000","last_became_active":"2024-09-19T04:13:28.975424+0000","last_became_peered":"2024-09-19T04:13:28.975424+0000","last_unstale":"2024-09-19T04:13:30.131834+0000","last_undegraded":"2024-09-19T04:13:30.131834+0000","last_fullsized":"2024-09-19T04:13:30.131834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:27.255493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061535+0000","last_change":"2024-09-19T04:13:24.956224+0000","last_active":"2024-09-19T04:13:31.061535+0000","last_peered":"2024-09-19T04:13:31.061535+0000","last_clean":"2024-09-19T04:13:31.061535+0000","last_became_active":"2024-09-19T04:13:24.953799+0000","last_became_peered":"2024-09-19T04:13:24.953799+0000","last_unstale":"2024-09-19T04:13:31.061535+0000","last_undegraded":"2024-09-19T04:13:31.061535+0000","last_fullsized":"2024-09-19T04:13:31.061535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:33.543861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061237+0000","last_change":"2024-09-19T04:13:23.919493+0000","last_active":"2024-09-19T04:13:31.061237+0000","last_peered":"2024-09-19T04:13:31.061237+0000","last_clean":"2024-09-19T04:13:31.061237+0000","last_became_active":"2024-09-19T04:13:23.918702+0000","last_became_peered":"2024-09-19T04:13:23.918702+0000","last_unstale":"2024-09-19T04:13:31.061237+0000","last_undegraded":"2024-09-19T04:13:31.061237+0000","last_fullsized":"2024-09-19T04:13:31.061237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:42.592915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062499+0000","last_change":"2024-09-19T04:13:29.028357+0000","last_active":"2024-09-19T04:13:31.062499+0000","last_peered":"2024-09-19T04:13:31.062499+0000","last_clean":"2024-09-19T04:13:31.062499+0000","last_became_active":"2024-09-19T04:13:29.027946+0000","last_became_peered":"2024-09-19T04:13:29.027946+0000","last_unstale":"2024-09-19T04:13:31.062499+0000","last_undegraded":"2024-09-19T04:13:31.062499+0000","last_fullsized":"2024-09-19T04:13:31.062499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300424+0000","last_change":"2024-09-19T04:13:24.968375+0000","last_active":"2024-09-19T04:13:30.300424+0000","last_peered":"2024-09-19T04:13:30.300424+0000","last_clean":"2024-09-19T04:13:30.300424+0000","last_became_active":"2024-09-19T04:13:24.968097+0000","last_became_peered":"2024-09-19T04:13:24.968097+0000","last_unstale":"2024-09-19T04:13:30.300424+0000","last_undegraded":"2024-09-19T04:13:30.300424+0000","last_fullsized":"2024-09-19T04:13:30.300424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969947+0000","last_change":"2024-09-19T04:13:23.903638+0000","last_active":"2024-09-19T04:13:29.969947+0000","last_peered":"2024-09-19T04:13:29.969947+0000","last_clean":"2024-09-19T04:13:29.969947+0000","last_became_active":"2024-09-19T04:13:23.898948+0000","last_became_peered":"2024-09-19T04:13:23.898948+0000","last_unstale":"2024-09-19T04:13:29.969947+0000","last_undegraded":"2024-09-19T04:13:29.969947+0000","last_fullsized":"2024-09-19T04:13:29.969947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061278+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:13:31.061278+0000","last_peered":"2024-09-19T04:13:31.061278+0000","last_clean":"2024-09-19T04:13:31.061278+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:13:31.061278+0000","last_undegraded":"2024-09-19T04:13:31.061278+0000","last_fullsized":"2024-09-19T04:13:31.061278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972561+0000","last_change":"2024-09-19T04:13:29.020008+0000","last_active":"2024-09-19T04:13:29.972561+0000","last_peered":"2024-09-19T04:13:29.972561+0000","last_clean":"2024-09-19T04:13:29.972561+0000","last_became_active":"2024-09-19T04:13:29.018706+0000","last_became_peered":"2024-09-19T04:13:29.018706+0000","last_unstale":"2024-09-19T04:13:29.972561+0000","last_undegraded":"2024-09-19T04:13:29.972561+0000","last_fullsized":"2024-09-19T04:13:29.972561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062691+0000","last_change":"2024-09-19T04:13:24.966194+0000","last_active":"2024-09-19T04:13:31.062691+0000","last_peered":"2024-09-19T04:13:31.062691+0000","last_clean":"2024-09-19T04:13:31.062691+0000","last_became_active":"2024-09-19T04:13:24.965690+0000","last_became_peered":"2024-09-19T04:13:24.965690+0000","last_unstale":"2024-09-19T04:13:31.062691+0000","last_undegraded":"2024-09-19T04:13:31.062691+0000","last_fullsized":"2024-09-19T04:13:31.062691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:14.510098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971314+0000","last_change":"2024-09-19T04:13:23.901039+0000","last_active":"2024-09-19T04:13:29.971314+0000","last_peered":"2024-09-19T04:13:29.971314+0000","last_clean":"2024-09-19T04:13:29.971314+0000","last_became_active":"2024-09-19T04:13:23.900709+0000","last_became_peered":"2024-09-19T04:13:23.900709+0000","last_unstale":"2024-09-19T04:13:29.971314+0000","last_undegraded":"2024-09-19T04:13:29.971314+0000","last_fullsized":"2024-09-19T04:13:29.971314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:19:40.425738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969768+0000","last_change":"2024-09-19T04:13:29.019565+0000","last_active":"2024-09-19T04:13:29.969768+0000","last_peered":"2024-09-19T04:13:29.969768+0000","last_clean":"2024-09-19T04:13:29.969768+0000","last_became_active":"2024-09-19T04:13:29.018547+0000","last_became_peered":"2024-09-19T04:13:29.018547+0000","last_unstale":"2024-09-19T04:13:29.969768+0000","last_undegraded":"2024-09-19T04:13:29.969768+0000","last_fullsized":"2024-09-19T04:13:29.969768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970414+0000","last_change":"2024-09-19T04:13:24.953123+0000","last_active":"2024-09-19T04:13:29.970414+0000","last_peered":"2024-09-19T04:13:29.970414+0000","last_clean":"2024-09-19T04:13:29.970414+0000","last_became_active":"2024-09-19T04:13:24.950851+0000","last_became_peered":"2024-09-19T04:13:24.950851+0000","last_unstale":"2024-09-19T04:13:29.970414+0000","last_undegraded":"2024-09-19T04:13:29.970414+0000","last_fullsized":"2024-09-19T04:13:29.970414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131715+0000","last_change":"2024-09-19T04:13:23.902143+0000","last_active":"2024-09-19T04:13:30.131715+0000","last_peered":"2024-09-19T04:13:30.131715+0000","last_clean":"2024-09-19T04:13:30.131715+0000","last_became_active":"2024-09-19T04:13:23.901843+0000","last_became_peered":"2024-09-19T04:13:23.901843+0000","last_unstale":"2024-09-19T04:13:30.131715+0000","last_undegraded":"2024-09-19T04:13:30.131715+0000","last_fullsized":"2024-09-19T04:13:30.131715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:03.746896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062168+0000","last_change":"2024-09-19T04:13:28.994151+0000","last_active":"2024-09-19T04:13:31.062168+0000","last_peered":"2024-09-19T04:13:31.062168+0000","last_clean":"2024-09-19T04:13:31.062168+0000","last_became_active":"2024-09-19T04:13:28.993608+0000","last_became_peered":"2024-09-19T04:13:28.993608+0000","last_unstale":"2024-09-19T04:13:31.062168+0000","last_undegraded":"2024-09-19T04:13:31.062168+0000","last_fullsized":"2024-09-19T04:13:31.062168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064926+0000","last_change":"2024-09-19T04:13:24.966033+0000","last_active":"2024-09-19T04:13:31.064926+0000","last_peered":"2024-09-19T04:13:31.064926+0000","last_clean":"2024-09-19T04:13:31.064926+0000","last_became_active":"2024-09-19T04:13:24.962081+0000","last_became_peered":"2024-09-19T04:13:24.962081+0000","last_unstale":"2024-09-19T04:13:31.064926+0000","last_undegraded":"2024-09-19T04:13:31.064926+0000","last_fullsized":"2024-09-19T04:13:31.064926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:58.621676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971640+0000","last_change":"2024-09-19T04:13:23.911774+0000","last_active":"2024-09-19T04:13:29.971640+0000","last_peered":"2024-09-19T04:13:29.971640+0000","last_clean":"2024-09-19T04:13:29.971640+0000","last_became_active":"2024-09-19T04:13:23.911154+0000","last_became_peered":"2024-09-19T04:13:23.911154+0000","last_unstale":"2024-09-19T04:13:29.971640+0000","last_undegraded":"2024-09-19T04:13:29.971640+0000","last_fullsized":"2024-09-19T04:13:29.971640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969327+0000","last_change":"2024-09-19T04:13:29.020613+0000","last_active":"2024-09-19T04:13:29.969327+0000","last_peered":"2024-09-19T04:13:29.969327+0000","last_clean":"2024-09-19T04:13:29.969327+0000","last_became_active":"2024-09-19T04:13:29.019473+0000","last_became_peered":"2024-09-19T04:13:29.019473+0000","last_unstale":"2024-09-19T04:13:29.969327+0000","last_undegraded":"2024-09-19T04:13:29.969327+0000","last_fullsized":"2024-09-19T04:13:29.969327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064591+0000","last_change":"2024-09-19T04:13:24.962284+0000","last_active":"2024-09-19T04:13:31.064591+0000","last_peered":"2024-09-19T04:13:31.064591+0000","last_clean":"2024-09-19T04:13:31.064591+0000","last_became_active":"2024-09-19T04:13:24.961763+0000","last_became_peered":"2024-09-19T04:13:24.961763+0000","last_unstale":"2024-09-19T04:13:31.064591+0000","last_undegraded":"2024-09-19T04:13:31.064591+0000","last_fullsized":"2024-09-19T04:13:31.064591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:40.924861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064080+0000","last_change":"2024-09-19T04:13:23.905574+0000","last_active":"2024-09-19T04:13:31.064080+0000","last_peered":"2024-09-19T04:13:31.064080+0000","last_clean":"2024-09-19T04:13:31.064080+0000","last_became_active":"2024-09-19T04:13:23.904993+0000","last_became_peered":"2024-09-19T04:13:23.904993+0000","last_unstale":"2024-09-19T04:13:31.064080+0000","last_undegraded":"2024-09-19T04:13:31.064080+0000","last_fullsized":"2024-09-19T04:13:31.064080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:00.615690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061135+0000","last_change":"2024-09-19T04:13:28.991444+0000","last_active":"2024-09-19T04:13:31.061135+0000","last_peered":"2024-09-19T04:13:31.061135+0000","last_clean":"2024-09-19T04:13:31.061135+0000","last_became_active":"2024-09-19T04:13:28.987837+0000","last_became_peered":"2024-09-19T04:13:28.987837+0000","last_unstale":"2024-09-19T04:13:31.061135+0000","last_undegraded":"2024-09-19T04:13:31.061135+0000","last_fullsized":"2024-09-19T04:13:31.061135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:17.703055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061563+0000","last_change":"2024-09-19T04:13:24.965972+0000","last_active":"2024-09-19T04:13:31.061563+0000","last_peered":"2024-09-19T04:13:31.061563+0000","last_clean":"2024-09-19T04:13:31.061563+0000","last_became_active":"2024-09-19T04:13:24.965315+0000","last_became_peered":"2024-09-19T04:13:24.965315+0000","last_unstale":"2024-09-19T04:13:31.061563+0000","last_undegraded":"2024-09-19T04:13:31.061563+0000","last_fullsized":"2024-09-19T04:13:31.061563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:16.174335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970388+0000","last_change":"2024-09-19T04:13:23.896881+0000","last_active":"2024-09-19T04:13:29.970388+0000","last_peered":"2024-09-19T04:13:29.970388+0000","last_clean":"2024-09-19T04:13:29.970388+0000","last_became_active":"2024-09-19T04:13:23.896440+0000","last_became_peered":"2024-09-19T04:13:23.896440+0000","last_unstale":"2024-09-19T04:13:29.970388+0000","last_undegraded":"2024-09-19T04:13:29.970388+0000","last_fullsized":"2024-09-19T04:13:29.970388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:36:56.677455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062482+0000","last_change":"2024-09-19T04:13:28.985691+0000","last_active":"2024-09-19T04:13:31.062482+0000","last_peered":"2024-09-19T04:13:31.062482+0000","last_clean":"2024-09-19T04:13:31.062482+0000","last_became_active":"2024-09-19T04:13:28.984749+0000","last_became_peered":"2024-09-19T04:13:28.984749+0000","last_unstale":"2024-09-19T04:13:31.062482+0000","last_undegraded":"2024-09-19T04:13:31.062482+0000","last_fullsized":"2024-09-19T04:13:31.062482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969844+0000","last_change":"2024-09-19T04:13:24.942591+0000","last_active":"2024-09-19T04:13:29.969844+0000","last_peered":"2024-09-19T04:13:29.969844+0000","last_clean":"2024-09-19T04:13:29.969844+0000","last_became_active":"2024-09-19T04:13:24.941789+0000","last_became_peered":"2024-09-19T04:13:24.941789+0000","last_unstale":"2024-09-19T04:13:29.969844+0000","last_undegraded":"2024-09-19T04:13:29.969844+0000","last_fullsized":"2024-09-19T04:13:29.969844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970255+0000","last_change":"2024-09-19T04:13:23.916881+0000","last_active":"2024-09-19T04:13:29.970255+0000","last_peered":"2024-09-19T04:13:29.970255+0000","last_clean":"2024-09-19T04:13:29.970255+0000","last_became_active":"2024-09-19T04:13:23.916050+0000","last_became_peered":"2024-09-19T04:13:23.916050+0000","last_unstale":"2024-09-19T04:13:29.970255+0000","last_undegraded":"2024-09-19T04:13:29.970255+0000","last_fullsized":"2024-09-19T04:13:29.970255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301470+0000","last_change":"2024-09-19T04:13:29.021093+0000","last_active":"2024-09-19T04:13:30.301470+0000","last_peered":"2024-09-19T04:13:30.301470+0000","last_clean":"2024-09-19T04:13:30.301470+0000","last_became_active":"2024-09-19T04:13:29.020573+0000","last_became_peered":"2024-09-19T04:13:29.020573+0000","last_unstale":"2024-09-19T04:13:30.301470+0000","last_undegraded":"2024-09-19T04:13:30.301470+0000","last_fullsized":"2024-09-19T04:13:30.301470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:35.622467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062500+0000","last_change":"2024-09-19T04:13:24.968018+0000","last_active":"2024-09-19T04:13:31.062500+0000","last_peered":"2024-09-19T04:13:31.062500+0000","last_clean":"2024-09-19T04:13:31.062500+0000","last_became_active":"2024-09-19T04:13:24.962386+0000","last_became_peered":"2024-09-19T04:13:24.962386+0000","last_unstale":"2024-09-19T04:13:31.062500+0000","last_undegraded":"2024-09-19T04:13:31.062500+0000","last_fullsized":"2024-09-19T04:13:31.062500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:39.665883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971777+0000","last_change":"2024-09-19T04:13:23.911883+0000","last_active":"2024-09-19T04:13:29.971777+0000","last_peered":"2024-09-19T04:13:29.971777+0000","last_clean":"2024-09-19T04:13:29.971777+0000","last_became_active":"2024-09-19T04:13:23.910990+0000","last_became_peered":"2024-09-19T04:13:23.910990+0000","last_unstale":"2024-09-19T04:13:29.971777+0000","last_undegraded":"2024-09-19T04:13:29.971777+0000","last_fullsized":"2024-09-19T04:13:29.971777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301624+0000","last_change":"2024-09-19T04:13:29.017846+0000","last_active":"2024-09-19T04:13:30.301624+0000","last_peered":"2024-09-19T04:13:30.301624+0000","last_clean":"2024-09-19T04:13:30.301624+0000","last_became_active":"2024-09-19T04:13:29.017269+0000","last_became_peered":"2024-09-19T04:13:29.017269+0000","last_unstale":"2024-09-19T04:13:30.301624+0000","last_undegraded":"2024-09-19T04:13:30.301624+0000","last_fullsized":"2024-09-19T04:13:30.301624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:07.730896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061954+0000","last_change":"2024-09-19T04:13:24.944986+0000","last_active":"2024-09-19T04:13:31.061954+0000","last_peered":"2024-09-19T04:13:31.061954+0000","last_clean":"2024-09-19T04:13:31.061954+0000","last_became_active":"2024-09-19T04:13:24.944266+0000","last_became_peered":"2024-09-19T04:13:24.944266+0000","last_unstale":"2024-09-19T04:13:31.061954+0000","last_undegraded":"2024-09-19T04:13:31.061954+0000","last_fullsized":"2024-09-19T04:13:31.061954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:17.729114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062229+0000","last_change":"2024-09-19T04:13:23.902337+0000","last_active":"2024-09-19T04:13:31.062229+0000","last_peered":"2024-09-19T04:13:31.062229+0000","last_clean":"2024-09-19T04:13:31.062229+0000","last_became_active":"2024-09-19T04:13:23.901958+0000","last_became_peered":"2024-09-19T04:13:23.901958+0000","last_unstale":"2024-09-19T04:13:31.062229+0000","last_undegraded":"2024-09-19T04:13:31.062229+0000","last_fullsized":"2024-09-19T04:13:31.062229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:39.987613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970753+0000","last_change":"2024-09-19T04:13:29.022266+0000","last_active":"2024-09-19T04:13:29.970753+0000","last_peered":"2024-09-19T04:13:29.970753+0000","last_clean":"2024-09-19T04:13:29.970753+0000","last_became_active":"2024-09-19T04:13:29.021104+0000","last_became_peered":"2024-09-19T04:13:29.021104+0000","last_unstale":"2024-09-19T04:13:29.970753+0000","last_undegraded":"2024-09-19T04:13:29.970753+0000","last_fullsized":"2024-09-19T04:13:29.970753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062460+0000","last_change":"2024-09-19T04:13:24.952759+0000","last_active":"2024-09-19T04:13:31.062460+0000","last_peered":"2024-09-19T04:13:31.062460+0000","last_clean":"2024-09-19T04:13:31.062460+0000","last_became_active":"2024-09-19T04:13:24.952088+0000","last_became_peered":"2024-09-19T04:13:24.952088+0000","last_unstale":"2024-09-19T04:13:31.062460+0000","last_undegraded":"2024-09-19T04:13:31.062460+0000","last_fullsized":"2024-09-19T04:13:31.062460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:29.020474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:35.460051+0000","last_change":"2024-09-19T04:13:23.909991+0000","last_active":"2024-09-19T04:13:35.460051+0000","last_peered":"2024-09-19T04:13:35.460051+0000","last_clean":"2024-09-19T04:13:35.460051+0000","last_became_active":"2024-09-19T04:13:23.909517+0000","last_became_peered":"2024-09-19T04:13:23.909517+0000","last_unstale":"2024-09-19T04:13:35.460051+0000","last_undegraded":"2024-09-19T04:13:35.460051+0000","last_fullsized":"2024-09-19T04:13:35.460051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.984556+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.983961+0000","last_became_peered":"2024-09-19T04:13:28.983961+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064342+0000","last_change":"2024-09-19T04:13:24.943383+0000","last_active":"2024-09-19T04:13:31.064342+0000","last_peered":"2024-09-19T04:13:31.064342+0000","last_clean":"2024-09-19T04:13:31.064342+0000","last_became_active":"2024-09-19T04:13:24.942094+0000","last_became_peered":"2024-09-19T04:13:24.942094+0000","last_unstale":"2024-09-19T04:13:31.064342+0000","last_undegraded":"2024-09-19T04:13:31.064342+0000","last_fullsized":"2024-09-19T04:13:31.064342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:15.642865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131674+0000","last_change":"2024-09-19T04:13:23.911338+0000","last_active":"2024-09-19T04:13:30.131674+0000","last_peered":"2024-09-19T04:13:30.131674+0000","last_clean":"2024-09-19T04:13:30.131674+0000","last_became_active":"2024-09-19T04:13:23.910962+0000","last_became_peered":"2024-09-19T04:13:23.910962+0000","last_unstale":"2024-09-19T04:13:30.131674+0000","last_undegraded":"2024-09-19T04:13:30.131674+0000","last_fullsized":"2024-09-19T04:13:30.131674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:39.854426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063766+0000","last_change":"2024-09-19T04:13:28.988142+0000","last_active":"2024-09-19T04:13:31.063766+0000","last_peered":"2024-09-19T04:13:31.063766+0000","last_clean":"2024-09-19T04:13:31.063766+0000","last_became_active":"2024-09-19T04:13:28.979829+0000","last_became_peered":"2024-09-19T04:13:28.979829+0000","last_unstale":"2024-09-19T04:13:31.063766+0000","last_undegraded":"2024-09-19T04:13:31.063766+0000","last_fullsized":"2024-09-19T04:13:31.063766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062345+0000","last_change":"2024-09-19T04:13:24.956553+0000","last_active":"2024-09-19T04:13:31.062345+0000","last_peered":"2024-09-19T04:13:31.062345+0000","last_clean":"2024-09-19T04:13:31.062345+0000","last_became_active":"2024-09-19T04:13:24.956238+0000","last_became_peered":"2024-09-19T04:13:24.956238+0000","last_unstale":"2024-09-19T04:13:31.062345+0000","last_undegraded":"2024-09-19T04:13:31.062345+0000","last_fullsized":"2024-09-19T04:13:31.062345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970527+0000","last_change":"2024-09-19T04:13:23.918490+0000","last_active":"2024-09-19T04:13:29.970527+0000","last_peered":"2024-09-19T04:13:29.970527+0000","last_clean":"2024-09-19T04:13:29.970527+0000","last_became_active":"2024-09-19T04:13:23.917969+0000","last_became_peered":"2024-09-19T04:13:23.917969+0000","last_unstale":"2024-09-19T04:13:29.970527+0000","last_undegraded":"2024-09-19T04:13:29.970527+0000","last_fullsized":"2024-09-19T04:13:29.970527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300379+0000","last_change":"2024-09-19T04:13:28.988707+0000","last_active":"2024-09-19T04:13:30.300379+0000","last_peered":"2024-09-19T04:13:30.300379+0000","last_clean":"2024-09-19T04:13:30.300379+0000","last_became_active":"2024-09-19T04:13:28.988388+0000","last_became_peered":"2024-09-19T04:13:28.988388+0000","last_unstale":"2024-09-19T04:13:30.300379+0000","last_undegraded":"2024-09-19T04:13:30.300379+0000","last_fullsized":"2024-09-19T04:13:30.300379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:54.745179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970310+0000","last_change":"2024-09-19T04:13:24.943308+0000","last_active":"2024-09-19T04:13:29.970310+0000","last_peered":"2024-09-19T04:13:29.970310+0000","last_clean":"2024-09-19T04:13:29.970310+0000","last_became_active":"2024-09-19T04:13:24.942608+0000","last_became_peered":"2024-09-19T04:13:24.942608+0000","last_unstale":"2024-09-19T04:13:29.970310+0000","last_undegraded":"2024-09-19T04:13:29.970310+0000","last_fullsized":"2024-09-19T04:13:29.970310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972997+0000","last_change":"2024-09-19T04:13:23.918096+0000","last_active":"2024-09-19T04:13:29.972997+0000","last_peered":"2024-09-19T04:13:29.972997+0000","last_clean":"2024-09-19T04:13:29.972997+0000","last_became_active":"2024-09-19T04:13:23.917505+0000","last_became_peered":"2024-09-19T04:13:23.917505+0000","last_unstale":"2024-09-19T04:13:29.972997+0000","last_undegraded":"2024-09-19T04:13:29.972997+0000","last_fullsized":"2024-09-19T04:13:29.972997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300417+0000","last_change":"2024-09-19T04:13:29.027451+0000","last_active":"2024-09-19T04:13:30.300417+0000","last_peered":"2024-09-19T04:13:30.300417+0000","last_clean":"2024-09-19T04:13:30.300417+0000","last_became_active":"2024-09-19T04:13:29.027181+0000","last_became_peered":"2024-09-19T04:13:29.027181+0000","last_unstale":"2024-09-19T04:13:30.300417+0000","last_undegraded":"2024-09-19T04:13:30.300417+0000","last_fullsized":"2024-09-19T04:13:30.300417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:42.206935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062147+0000","last_change":"2024-09-19T04:13:29.028793+0000","last_active":"2024-09-19T04:13:31.062147+0000","last_peered":"2024-09-19T04:13:31.062147+0000","last_clean":"2024-09-19T04:13:31.062147+0000","last_became_active":"2024-09-19T04:13:29.028293+0000","last_became_peered":"2024-09-19T04:13:29.028293+0000","last_unstale":"2024-09-19T04:13:31.062147+0000","last_undegraded":"2024-09-19T04:13:31.062147+0000","last_fullsized":"2024-09-19T04:13:31.062147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130915+0000","last_change":"2024-09-19T04:13:24.947415+0000","last_active":"2024-09-19T04:13:30.130915+0000","last_peered":"2024-09-19T04:13:30.130915+0000","last_clean":"2024-09-19T04:13:30.130915+0000","last_became_active":"2024-09-19T04:13:24.947042+0000","last_became_peered":"2024-09-19T04:13:24.947042+0000","last_unstale":"2024-09-19T04:13:30.130915+0000","last_undegraded":"2024-09-19T04:13:30.130915+0000","last_fullsized":"2024-09-19T04:13:30.130915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969735+0000","last_change":"2024-09-19T04:13:23.906982+0000","last_active":"2024-09-19T04:13:29.969735+0000","last_peered":"2024-09-19T04:13:29.969735+0000","last_clean":"2024-09-19T04:13:29.969735+0000","last_became_active":"2024-09-19T04:13:23.905070+0000","last_became_peered":"2024-09-19T04:13:23.905070+0000","last_unstale":"2024-09-19T04:13:29.969735+0000","last_undegraded":"2024-09-19T04:13:29.969735+0000","last_fullsized":"2024-09-19T04:13:29.969735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972911+0000","last_change":"2024-09-19T04:13:28.979353+0000","last_active":"2024-09-19T04:13:29.972911+0000","last_peered":"2024-09-19T04:13:29.972911+0000","last_clean":"2024-09-19T04:13:29.972911+0000","last_became_active":"2024-09-19T04:13:28.976991+0000","last_became_peered":"2024-09-19T04:13:28.976991+0000","last_unstale":"2024-09-19T04:13:29.972911+0000","last_undegraded":"2024-09-19T04:13:29.972911+0000","last_fullsized":"2024-09-19T04:13:29.972911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300060+0000","last_change":"2024-09-19T04:13:24.947190+0000","last_active":"2024-09-19T04:13:30.300060+0000","last_peered":"2024-09-19T04:13:30.300060+0000","last_clean":"2024-09-19T04:13:30.300060+0000","last_became_active":"2024-09-19T04:13:24.946883+0000","last_became_peered":"2024-09-19T04:13:24.946883+0000","last_unstale":"2024-09-19T04:13:30.300060+0000","last_undegraded":"2024-09-19T04:13:30.300060+0000","last_fullsized":"2024-09-19T04:13:30.300060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300792+0000","last_change":"2024-09-19T04:13:23.912532+0000","last_active":"2024-09-19T04:13:30.300792+0000","last_peered":"2024-09-19T04:13:30.300792+0000","last_clean":"2024-09-19T04:13:30.300792+0000","last_became_active":"2024-09-19T04:13:23.911952+0000","last_became_peered":"2024-09-19T04:13:23.911952+0000","last_unstale":"2024-09-19T04:13:30.300792+0000","last_undegraded":"2024-09-19T04:13:30.300792+0000","last_fullsized":"2024-09-19T04:13:30.300792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971798+0000","last_change":"2024-09-19T04:13:23.912898+0000","last_active":"2024-09-19T04:13:29.971798+0000","last_peered":"2024-09-19T04:13:29.971798+0000","last_clean":"2024-09-19T04:13:29.971798+0000","last_became_active":"2024-09-19T04:13:23.912234+0000","last_became_peered":"2024-09-19T04:13:23.912234+0000","last_unstale":"2024-09-19T04:13:29.971798+0000","last_undegraded":"2024-09-19T04:13:29.971798+0000","last_fullsized":"2024-09-19T04:13:29.971798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062138+0000","last_change":"2024-09-19T04:13:24.953761+0000","last_active":"2024-09-19T04:13:31.062138+0000","last_peered":"2024-09-19T04:13:31.062138+0000","last_clean":"2024-09-19T04:13:31.062138+0000","last_became_active":"2024-09-19T04:13:24.953195+0000","last_became_peered":"2024-09-19T04:13:24.953195+0000","last_unstale":"2024-09-19T04:13:31.062138+0000","last_undegraded":"2024-09-19T04:13:31.062138+0000","last_fullsized":"2024-09-19T04:13:31.062138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971169+0000","last_change":"2024-09-19T04:13:28.976510+0000","last_active":"2024-09-19T04:13:29.971169+0000","last_peered":"2024-09-19T04:13:29.971169+0000","last_clean":"2024-09-19T04:13:29.971169+0000","last_became_active":"2024-09-19T04:13:28.975842+0000","last_became_peered":"2024-09-19T04:13:28.975842+0000","last_unstale":"2024-09-19T04:13:29.971169+0000","last_undegraded":"2024-09-19T04:13:29.971169+0000","last_fullsized":"2024-09-19T04:13:29.971169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:36:39.502090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301204+0000","last_change":"2024-09-19T04:13:23.920382+0000","last_active":"2024-09-19T04:13:30.301204+0000","last_peered":"2024-09-19T04:13:30.301204+0000","last_clean":"2024-09-19T04:13:30.301204+0000","last_became_active":"2024-09-19T04:13:23.920107+0000","last_became_peered":"2024-09-19T04:13:23.920107+0000","last_unstale":"2024-09-19T04:13:30.301204+0000","last_undegraded":"2024-09-19T04:13:30.301204+0000","last_fullsized":"2024-09-19T04:13:30.301204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:24.725046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062882+0000","last_change":"2024-09-19T04:13:24.933838+0000","last_active":"2024-09-19T04:13:31.062882+0000","last_peered":"2024-09-19T04:13:31.062882+0000","last_clean":"2024-09-19T04:13:31.062882+0000","last_became_active":"2024-09-19T04:13:24.932423+0000","last_became_peered":"2024-09-19T04:13:24.932423+0000","last_unstale":"2024-09-19T04:13:31.062882+0000","last_undegraded":"2024-09-19T04:13:31.062882+0000","last_fullsized":"2024-09-19T04:13:31.062882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:52.498071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301388+0000","last_change":"2024-09-19T04:13:28.982366+0000","last_active":"2024-09-19T04:13:30.301388+0000","last_peered":"2024-09-19T04:13:30.301388+0000","last_clean":"2024-09-19T04:13:30.301388+0000","last_became_active":"2024-09-19T04:13:28.981608+0000","last_became_peered":"2024-09-19T04:13:28.981608+0000","last_unstale":"2024-09-19T04:13:30.301388+0000","last_undegraded":"2024-09-19T04:13:30.301388+0000","last_fullsized":"2024-09-19T04:13:30.301388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:12.864952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969437+0000","last_change":"2024-09-19T04:13:24.939240+0000","last_active":"2024-09-19T04:13:29.969437+0000","last_peered":"2024-09-19T04:13:29.969437+0000","last_clean":"2024-09-19T04:13:29.969437+0000","last_became_active":"2024-09-19T04:13:24.938527+0000","last_became_peered":"2024-09-19T04:13:24.938527+0000","last_unstale":"2024-09-19T04:13:29.969437+0000","last_undegraded":"2024-09-19T04:13:29.969437+0000","last_fullsized":"2024-09-19T04:13:29.969437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970473+0000","last_change":"2024-09-19T04:13:24.967773+0000","last_active":"2024-09-19T04:13:29.970473+0000","last_peered":"2024-09-19T04:13:29.970473+0000","last_clean":"2024-09-19T04:13:29.970473+0000","last_became_active":"2024-09-19T04:13:24.967467+0000","last_became_peered":"2024-09-19T04:13:24.967467+0000","last_unstale":"2024-09-19T04:13:29.970473+0000","last_undegraded":"2024-09-19T04:13:29.970473+0000","last_fullsized":"2024-09-19T04:13:29.970473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:11.155011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971529+0000","last_change":"2024-09-19T04:13:24.945017+0000","last_active":"2024-09-19T04:13:29.971529+0000","last_peered":"2024-09-19T04:13:29.971529+0000","last_clean":"2024-09-19T04:13:29.971529+0000","last_became_active":"2024-09-19T04:13:24.944535+0000","last_became_peered":"2024-09-19T04:13:24.944535+0000","last_unstale":"2024-09-19T04:13:29.971529+0000","last_undegraded":"2024-09-19T04:13:29.971529+0000","last_fullsized":"2024-09-19T04:13:29.971529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:43.596741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062090+0000","last_change":"2024-09-19T04:13:24.952497+0000","last_active":"2024-09-19T04:13:31.062090+0000","last_peered":"2024-09-19T04:13:31.062090+0000","last_clean":"2024-09-19T04:13:31.062090+0000","last_became_active":"2024-09-19T04:13:24.952037+0000","last_became_peered":"2024-09-19T04:13:24.952037+0000","last_unstale":"2024-09-19T04:13:31.062090+0000","last_undegraded":"2024-09-19T04:13:31.062090+0000","last_fullsized":"2024-09-19T04:13:31.062090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301786+0000","last_change":"2024-09-19T04:13:24.966504+0000","last_active":"2024-09-19T04:13:30.301786+0000","last_peered":"2024-09-19T04:13:30.301786+0000","last_clean":"2024-09-19T04:13:30.301786+0000","last_became_active":"2024-09-19T04:13:24.965992+0000","last_became_peered":"2024-09-19T04:13:24.965992+0000","last_unstale":"2024-09-19T04:13:30.301786+0000","last_undegraded":"2024-09-19T04:13:30.301786+0000","last_fullsized":"2024-09-19T04:13:30.301786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131526+0000","last_change":"2024-09-19T04:13:24.949970+0000","last_active":"2024-09-19T04:13:30.131526+0000","last_peered":"2024-09-19T04:13:30.131526+0000","last_clean":"2024-09-19T04:13:30.131526+0000","last_became_active":"2024-09-19T04:13:24.949676+0000","last_became_peered":"2024-09-19T04:13:24.949676+0000","last_unstale":"2024-09-19T04:13:30.131526+0000","last_undegraded":"2024-09-19T04:13:30.131526+0000","last_fullsized":"2024-09-19T04:13:30.131526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973014+0000","last_change":"2024-09-19T04:13:24.939046+0000","last_active":"2024-09-19T04:13:29.973014+0000","last_peered":"2024-09-19T04:13:29.973014+0000","last_clean":"2024-09-19T04:13:29.973014+0000","last_became_active":"2024-09-19T04:13:24.938277+0000","last_became_peered":"2024-09-19T04:13:24.938277+0000","last_unstale":"2024-09-19T04:13:29.973014+0000","last_undegraded":"2024-09-19T04:13:29.973014+0000","last_fullsized":"2024-09-19T04:13:29.973014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301516+0000","last_change":"2024-09-19T04:13:24.962572+0000","last_active":"2024-09-19T04:13:30.301516+0000","last_peered":"2024-09-19T04:13:30.301516+0000","last_clean":"2024-09-19T04:13:30.301516+0000","last_became_active":"2024-09-19T04:13:24.962277+0000","last_became_peered":"2024-09-19T04:13:24.962277+0000","last_unstale":"2024-09-19T04:13:30.301516+0000","last_undegraded":"2024-09-19T04:13:30.301516+0000","last_fullsized":"2024-09-19T04:13:30.301516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969487+0000","last_change":"2024-09-19T04:13:24.951536+0000","last_active":"2024-09-19T04:13:29.969487+0000","last_peered":"2024-09-19T04:13:29.969487+0000","last_clean":"2024-09-19T04:13:29.969487+0000","last_became_active":"2024-09-19T04:13:24.950939+0000","last_became_peered":"2024-09-19T04:13:24.950939+0000","last_unstale":"2024-09-19T04:13:29.969487+0000","last_undegraded":"2024-09-19T04:13:29.969487+0000","last_fullsized":"2024-09-19T04:13:29.969487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300567+0000","last_change":"2024-09-19T04:13:24.961761+0000","last_active":"2024-09-19T04:13:30.300567+0000","last_peered":"2024-09-19T04:13:30.300567+0000","last_clean":"2024-09-19T04:13:30.300567+0000","last_became_active":"2024-09-19T04:13:24.961415+0000","last_became_peered":"2024-09-19T04:13:24.961415+0000","last_unstale":"2024-09-19T04:13:30.300567+0000","last_undegraded":"2024-09-19T04:13:30.300567+0000","last_fullsized":"2024-09-19T04:13:30.300567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973133+0000","last_change":"2024-09-19T04:13:24.939730+0000","last_active":"2024-09-19T04:13:29.973133+0000","last_peered":"2024-09-19T04:13:29.973133+0000","last_clean":"2024-09-19T04:13:29.973133+0000","last_became_active":"2024-09-19T04:13:24.938400+0000","last_became_peered":"2024-09-19T04:13:24.938400+0000","last_unstale":"2024-09-19T04:13:29.973133+0000","last_undegraded":"2024-09-19T04:13:29.973133+0000","last_fullsized":"2024-09-19T04:13:29.973133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062008+0000","last_change":"2024-09-19T04:13:24.944592+0000","last_active":"2024-09-19T04:13:31.062008+0000","last_peered":"2024-09-19T04:13:31.062008+0000","last_clean":"2024-09-19T04:13:31.062008+0000","last_became_active":"2024-09-19T04:13:24.943915+0000","last_became_peered":"2024-09-19T04:13:24.943915+0000","last_unstale":"2024-09-19T04:13:31.062008+0000","last_undegraded":"2024-09-19T04:13:31.062008+0000","last_fullsized":"2024-09-19T04:13:31.062008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972579+0000","last_change":"2024-09-19T04:13:24.967058+0000","last_active":"2024-09-19T04:13:29.972579+0000","last_peered":"2024-09-19T04:13:29.972579+0000","last_clean":"2024-09-19T04:13:29.972579+0000","last_became_active":"2024-09-19T04:13:24.964521+0000","last_became_peered":"2024-09-19T04:13:24.964521+0000","last_unstale":"2024-09-19T04:13:29.972579+0000","last_undegraded":"2024-09-19T04:13:29.972579+0000","last_fullsized":"2024-09-19T04:13:29.972579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063128+0000","last_change":"2024-09-19T04:13:24.953833+0000","last_active":"2024-09-19T04:13:31.063128+0000","last_peered":"2024-09-19T04:13:31.063128+0000","last_clean":"2024-09-19T04:13:31.063128+0000","last_became_active":"2024-09-19T04:13:24.950878+0000","last_became_peered":"2024-09-19T04:13:24.950878+0000","last_unstale":"2024-09-19T04:13:31.063128+0000","last_undegraded":"2024-09-19T04:13:31.063128+0000","last_fullsized":"2024-09-19T04:13:31.063128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:07.653424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971483+0000","last_change":"2024-09-19T04:13:24.956195+0000","last_active":"2024-09-19T04:13:29.971483+0000","last_peered":"2024-09-19T04:13:29.971483+0000","last_clean":"2024-09-19T04:13:29.971483+0000","last_became_active":"2024-09-19T04:13:24.955911+0000","last_became_peered":"2024-09-19T04:13:24.955911+0000","last_unstale":"2024-09-19T04:13:29.971483+0000","last_undegraded":"2024-09-19T04:13:29.971483+0000","last_fullsized":"2024-09-19T04:13:29.971483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:55.185297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970952+0000","last_change":"2024-09-19T04:13:24.939650+0000","last_active":"2024-09-19T04:13:29.970952+0000","last_peered":"2024-09-19T04:13:29.970952+0000","last_clean":"2024-09-19T04:13:29.970952+0000","last_became_active":"2024-09-19T04:13:24.938774+0000","last_became_peered":"2024-09-19T04:13:24.938774+0000","last_unstale":"2024-09-19T04:13:29.970952+0000","last_undegraded":"2024-09-19T04:13:29.970952+0000","last_fullsized":"2024-09-19T04:13:29.970952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:00.871440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969219+0000","last_change":"2024-09-19T04:13:24.921407+0000","last_active":"2024-09-19T04:13:29.969219+0000","last_peered":"2024-09-19T04:13:29.969219+0000","last_clean":"2024-09-19T04:13:29.969219+0000","last_became_active":"2024-09-19T04:13:24.920646+0000","last_became_peered":"2024-09-19T04:13:24.920646+0000","last_unstale":"2024-09-19T04:13:29.969219+0000","last_undegraded":"2024-09-19T04:13:29.969219+0000","last_fullsized":"2024-09-19T04:13:29.969219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:24.963230+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:24.962932+0000","last_became_peered":"2024-09-19T04:13:24.962932+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061900+0000","last_change":"2024-09-19T04:13:24.942848+0000","last_active":"2024-09-19T04:13:31.061900+0000","last_peered":"2024-09-19T04:13:31.061900+0000","last_clean":"2024-09-19T04:13:31.061900+0000","last_became_active":"2024-09-19T04:13:24.942524+0000","last_became_peered":"2024-09-19T04:13:24.942524+0000","last_unstale":"2024-09-19T04:13:31.061900+0000","last_undegraded":"2024-09-19T04:13:31.061900+0000","last_fullsized":"2024-09-19T04:13:31.061900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:26.370075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300722+0000","last_change":"2024-09-19T04:13:24.950629+0000","last_active":"2024-09-19T04:13:30.300722+0000","last_peered":"2024-09-19T04:13:30.300722+0000","last_clean":"2024-09-19T04:13:30.300722+0000","last_became_active":"2024-09-19T04:13:24.946193+0000","last_became_peered":"2024-09-19T04:13:24.946193+0000","last_unstale":"2024-09-19T04:13:30.300722+0000","last_undegraded":"2024-09-19T04:13:30.300722+0000","last_fullsized":"2024-09-19T04:13:30.300722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064210+0000","last_change":"2024-09-19T04:13:24.936924+0000","last_active":"2024-09-19T04:13:31.064210+0000","last_peered":"2024-09-19T04:13:31.064210+0000","last_clean":"2024-09-19T04:13:31.064210+0000","last_became_active":"2024-09-19T04:13:24.936053+0000","last_became_peered":"2024-09-19T04:13:24.936053+0000","last_unstale":"2024-09-19T04:13:31.064210+0000","last_undegraded":"2024-09-19T04:13:31.064210+0000","last_fullsized":"2024-09-19T04:13:31.064210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:40.858208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972557+0000","last_change":"2024-09-19T04:13:24.967121+0000","last_active":"2024-09-19T04:13:29.972557+0000","last_peered":"2024-09-19T04:13:29.972557+0000","last_clean":"2024-09-19T04:13:29.972557+0000","last_became_active":"2024-09-19T04:13:24.964161+0000","last_became_peered":"2024-09-19T04:13:24.964161+0000","last_unstale":"2024-09-19T04:13:29.972557+0000","last_undegraded":"2024-09-19T04:13:29.972557+0000","last_fullsized":"2024-09-19T04:13:29.972557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972765+0000","last_change":"2024-09-19T04:13:24.968947+0000","last_active":"2024-09-19T04:13:29.972765+0000","last_peered":"2024-09-19T04:13:29.972765+0000","last_clean":"2024-09-19T04:13:29.972765+0000","last_became_active":"2024-09-19T04:13:24.968335+0000","last_became_peered":"2024-09-19T04:13:24.968335+0000","last_unstale":"2024-09-19T04:13:29.972765+0000","last_undegraded":"2024-09-19T04:13:29.972765+0000","last_fullsized":"2024-09-19T04:13:29.972765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064806+0000","last_change":"2024-09-19T04:13:24.970001+0000","last_active":"2024-09-19T04:13:31.064806+0000","last_peered":"2024-09-19T04:13:31.064806+0000","last_clean":"2024-09-19T04:13:31.064806+0000","last_became_active":"2024-09-19T04:13:24.969462+0000","last_became_peered":"2024-09-19T04:13:24.969462+0000","last_unstale":"2024-09-19T04:13:31.064806+0000","last_undegraded":"2024-09-19T04:13:31.064806+0000","last_fullsized":"2024-09-19T04:13:31.064806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:15:32.140847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299773+0000","last_change":"2024-09-19T04:13:24.967702+0000","last_active":"2024-09-19T04:13:30.299773+0000","last_peered":"2024-09-19T04:13:30.299773+0000","last_clean":"2024-09-19T04:13:30.299773+0000","last_became_active":"2024-09-19T04:13:24.967436+0000","last_became_peered":"2024-09-19T04:13:24.967436+0000","last_unstale":"2024-09-19T04:13:30.299773+0000","last_undegraded":"2024-09-19T04:13:30.299773+0000","last_fullsized":"2024-09-19T04:13:30.299773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061448+0000","last_change":"2024-09-19T04:13:24.942637+0000","last_active":"2024-09-19T04:13:31.061448+0000","last_peered":"2024-09-19T04:13:31.061448+0000","last_clean":"2024-09-19T04:13:31.061448+0000","last_became_active":"2024-09-19T04:13:24.942164+0000","last_became_peered":"2024-09-19T04:13:24.942164+0000","last_unstale":"2024-09-19T04:13:31.061448+0000","last_undegraded":"2024-09-19T04:13:31.061448+0000","last_fullsized":"2024-09-19T04:13:31.061448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:30.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971132+0000","last_change":"2024-09-19T04:13:24.926176+0000","last_active":"2024-09-19T04:13:29.971132+0000","last_peered":"2024-09-19T04:13:29.971132+0000","last_clean":"2024-09-19T04:13:29.971132+0000","last_became_active":"2024-09-19T04:13:24.925246+0000","last_became_peered":"2024-09-19T04:13:24.925246+0000","last_unstale":"2024-09-19T04:13:29.971132+0000","last_undegraded":"2024-09-19T04:13:29.971132+0000","last_fullsized":"2024-09-19T04:13:29.971132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:52.769507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062030+0000","last_change":"2024-09-19T04:13:24.953254+0000","last_active":"2024-09-19T04:13:31.062030+0000","last_peered":"2024-09-19T04:13:31.062030+0000","last_clean":"2024-09-19T04:13:31.062030+0000","last_became_active":"2024-09-19T04:13:24.950624+0000","last_became_peered":"2024-09-19T04:13:24.950624+0000","last_unstale":"2024-09-19T04:13:31.062030+0000","last_undegraded":"2024-09-19T04:13:31.062030+0000","last_fullsized":"2024-09-19T04:13:31.062030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:06.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969906+0000","last_change":"2024-09-19T04:13:24.954966+0000","last_active":"2024-09-19T04:13:29.969906+0000","last_peered":"2024-09-19T04:13:29.969906+0000","last_clean":"2024-09-19T04:13:29.969906+0000","last_became_active":"2024-09-19T04:13:24.954234+0000","last_became_peered":"2024-09-19T04:13:24.954234+0000","last_unstale":"2024-09-19T04:13:29.969906+0000","last_undegraded":"2024-09-19T04:13:29.969906+0000","last_fullsized":"2024-09-19T04:13:29.969906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061764+0000","last_change":"2024-09-19T04:13:24.949281+0000","last_active":"2024-09-19T04:13:31.061764+0000","last_peered":"2024-09-19T04:13:31.061764+0000","last_clean":"2024-09-19T04:13:31.061764+0000","last_became_active":"2024-09-19T04:13:24.948863+0000","last_became_peered":"2024-09-19T04:13:24.948863+0000","last_unstale":"2024-09-19T04:13:31.061764+0000","last_undegraded":"2024-09-19T04:13:31.061764+0000","last_fullsized":"2024-09-19T04:13:31.061764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131694+0000","last_change":"2024-09-19T04:13:24.943099+0000","last_active":"2024-09-19T04:13:30.131694+0000","last_peered":"2024-09-19T04:13:30.131694+0000","last_clean":"2024-09-19T04:13:30.131694+0000","last_became_active":"2024-09-19T04:13:24.942776+0000","last_became_peered":"2024-09-19T04:13:24.942776+0000","last_unstale":"2024-09-19T04:13:30.131694+0000","last_undegraded":"2024-09-19T04:13:30.131694+0000","last_fullsized":"2024-09-19T04:13:30.131694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064570+0000","last_change":"2024-09-19T04:13:24.951658+0000","last_active":"2024-09-19T04:13:31.064570+0000","last_peered":"2024-09-19T04:13:31.064570+0000","last_clean":"2024-09-19T04:13:31.064570+0000","last_became_active":"2024-09-19T04:13:24.950976+0000","last_became_peered":"2024-09-19T04:13:24.950976+0000","last_unstale":"2024-09-19T04:13:31.064570+0000","last_undegraded":"2024-09-19T04:13:31.064570+0000","last_fullsized":"2024-09-19T04:13:31.064570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:39.061865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":694435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":103611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":103485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":103645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:13:39.976 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:13:40.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe72ae77640 1 Processor -- start 2024-09-19T04:13:40.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe72ae77640 1 -- start start 2024-09-19T04:13:40.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe72ae77640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7241587a0 0x7fe7240b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe72ae77640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72406ec60 con 0x7fe72406e680 2024-09-19T04:13:40.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe72ae77640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe72406ef40 con 0x7fe7240b37d0 2024-09-19T04:13:40.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe72ae77640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe7240b3c40 con 0x7fe7241587a0 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe728c13640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7241587a0 0x7fe7240b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.115+0000 7fe728c13640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7241587a0 0x7fe7240b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40534/0 (socket says 172.21.15.76:40534) 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe728c13640 1 -- 172.21.15.76:0/429677135 learned_addr learned my addr 172.21.15.76:0/429677135 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe729414640 1 --1- 172.21.15.76:0/429677135 >> v1:172.21.15.76:6789/0 conn(0x7fe72406e680 0x7fe724072160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34290/0 (socket says 172.21.15.76:34290) 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe728c13640 1 -- 172.21.15.76:0/429677135 >> v1:172.21.15.116:6789/0 conn(0x7fe7240b37d0 legacy=0x7fe72406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe728c13640 1 -- 172.21.15.76:0/429677135 >> v1:172.21.15.76:6789/0 conn(0x7fe72406e680 legacy=0x7fe724072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe7237fe640 1 -- 172.21.15.76:0/429677135 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 307928492 0 0) 0x7fe72406ec60 con 0x7fe72406e680 2024-09-19T04:13:40.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe728c13640 1 -- 172.21.15.76:0/429677135 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe72406f220 con 0x7fe7241587a0 2024-09-19T04:13:40.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe728c13640 1 --2- 172.21.15.76:0/429677135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7241587a0 0x7fe7240b3290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe714009870 tx=0x7fe71401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c351df5f5d6b31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe7237fe640 1 -- 172.21.15.76:0/429677135 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140b0070 con 0x7fe7241587a0 2024-09-19T04:13:40.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe7237fe640 1 -- 172.21.15.76:0/429677135 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7140ab600 con 0x7fe7241587a0 2024-09-19T04:13:40.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe7237fe640 1 -- 172.21.15.76:0/429677135 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140ab9d0 con 0x7fe7241587a0 2024-09-19T04:13:40.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe72ae77640 1 -- 172.21.15.76:0/429677135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7241587a0 msgr2=0x7fe7240b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe72ae77640 1 --2- 172.21.15.76:0/429677135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7241587a0 0x7fe7240b3290 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe714009870 tx=0x7fe71401c520 comp rx=0 tx=0).stop 2024-09-19T04:13:40.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe72ae77640 1 -- 172.21.15.76:0/429677135 shutdown_connections 2024-09-19T04:13:40.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe72ae77640 1 --2- 172.21.15.76:0/429677135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7241587a0 0x7fe7240b3290 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe72ae77640 1 -- 172.21.15.76:0/429677135 >> 172.21.15.76:0/429677135 conn(0x7fe7240b1f90 msgr2=0x7fe7240b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:40.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe72ae77640 1 -- 172.21.15.76:0/429677135 shutdown_connections 2024-09-19T04:13:40.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.119+0000 7fe72ae77640 1 -- 172.21.15.76:0/429677135 wait complete. 2024-09-19T04:13:40.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 Processor -- start 2024-09-19T04:13:40.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 -- start start 2024-09-19T04:13:40.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe72406e680 0x7fe72409d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7240b37d0 0x7fe72415b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7241587a0 0x7fe72409b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe72416c200 con 0x7fe72406e680 2024-09-19T04:13:40.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe724073310 con 0x7fe7241587a0 2024-09-19T04:13:40.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe72416c380 con 0x7fe7240b37d0 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe72406e680 0x7fe72409d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe72406e680 0x7fe72409d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40536/0 (socket says 172.21.15.76:40536) 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 -- 172.21.15.76:0/3264826144 learned_addr learned my addr 172.21.15.76:0/3264826144 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 -- 172.21.15.76:0/3264826144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7240b37d0 msgr2=0x7fe72415b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe729414640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7241587a0 0x7fe72409b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7240b37d0 0x7fe72415b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 -- 172.21.15.76:0/3264826144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7241587a0 msgr2=0x7fe72409b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7241587a0 0x7fe72409b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 -- 172.21.15.76:0/3264826144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe724156080 con 0x7fe72406e680 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe728c13640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe72406e680 0x7fe72409d0e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7140ae040 tx=0x7fe71401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe7217fa640 1 -- 172.21.15.76:0/3264826144 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140bd070 con 0x7fe72406e680 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe7217fa640 1 -- 172.21.15.76:0/3264826144 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe714004030 con 0x7fe72406e680 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe72415d110 con 0x7fe72406e680 2024-09-19T04:13:40.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe7217fa640 1 -- 172.21.15.76:0/3264826144 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7140b0040 con 0x7fe72406e680 2024-09-19T04:13:40.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe72415d3b0 con 0x7fe72406e680 2024-09-19T04:13:40.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7240b3290 con 0x7fe72406e680 2024-09-19T04:13:40.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe7217fa640 1 -- 172.21.15.76:0/3264826144 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7140c0070 con 0x7fe72406e680 2024-09-19T04:13:40.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe7217fa640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7100678c0 0x7fe710069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe7217fa640 1 -- 172.21.15.76:0/3264826144 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe71411f780 con 0x7fe72406e680 2024-09-19T04:13:40.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.123+0000 7fe723fff640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7100678c0 0x7fe710069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.127+0000 7fe723fff640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7100678c0 0x7fe710069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe718003cf0 tx=0x7fe7180079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.131+0000 7fe7217fa640 1 -- 172.21.15.76:0/3264826144 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe714016110 con 0x7fe72406e680 2024-09-19T04:13:40.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.274+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe724070080 con 0x7fe7100678c0 2024-09-19T04:13:40.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.282+0000 7fe7217fa640 1 -- 172.21.15.76:0/3264826144 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332558 (secure 0 0 0) 0x7fe724070080 con 0x7fe7100678c0 2024-09-19T04:13:40.289 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:40.291 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7100678c0 msgr2=0x7fe710069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7100678c0 0x7fe710069d80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe718003cf0 tx=0x7fe7180079c0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe72406e680 msgr2=0x7fe72409d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe72406e680 0x7fe72409d0e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe7140ae040 tx=0x7fe71401ad60 comp rx=0 tx=0).stop 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 shutdown_connections 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7100678c0 0x7fe710069d80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7241587a0 0x7fe72409b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7240b37d0 0x7fe72415b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 --2- 172.21.15.76:0/3264826144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe72406e680 0x7fe72409d0e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 >> 172.21.15.76:0/3264826144 conn(0x7fe7240b1f90 msgr2=0x7fe7240b1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:40.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 shutdown_connections 2024-09-19T04:13:40.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.286+0000 7fe72ae77640 1 -- 172.21.15.76:0/3264826144 wait complete. 2024-09-19T04:13:40.304 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-19T04:13:39.741662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223668,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751052,"statfs":{"total":773094113280,"available":772865077248,"internally_reserved":0,"allocated":3350528,"data_stored":2017900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003044"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062396+0000","last_change":"2024-09-19T04:13:23.917029+0000","last_active":"2024-09-19T04:13:31.062396+0000","last_peered":"2024-09-19T04:13:31.062396+0000","last_clean":"2024-09-19T04:13:31.062396+0000","last_became_active":"2024-09-19T04:13:23.916640+0000","last_became_peered":"2024-09-19T04:13:23.916640+0000","last_unstale":"2024-09-19T04:13:31.062396+0000","last_undegraded":"2024-09-19T04:13:31.062396+0000","last_fullsized":"2024-09-19T04:13:31.062396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:22.981160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301574+0000","last_change":"2024-09-19T04:13:24.947160+0000","last_active":"2024-09-19T04:13:30.301574+0000","last_peered":"2024-09-19T04:13:30.301574+0000","last_clean":"2024-09-19T04:13:30.301574+0000","last_became_active":"2024-09-19T04:13:24.946824+0000","last_became_peered":"2024-09-19T04:13:24.946824+0000","last_unstale":"2024-09-19T04:13:30.301574+0000","last_undegraded":"2024-09-19T04:13:30.301574+0000","last_fullsized":"2024-09-19T04:13:30.301574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300320+0000","last_change":"2024-09-19T04:13:29.029580+0000","last_active":"2024-09-19T04:13:30.300320+0000","last_peered":"2024-09-19T04:13:30.300320+0000","last_clean":"2024-09-19T04:13:30.300320+0000","last_became_active":"2024-09-19T04:13:29.029159+0000","last_became_peered":"2024-09-19T04:13:29.029159+0000","last_unstale":"2024-09-19T04:13:30.300320+0000","last_undegraded":"2024-09-19T04:13:30.300320+0000","last_fullsized":"2024-09-19T04:13:30.300320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301237+0000","last_change":"2024-09-19T04:13:23.910773+0000","last_active":"2024-09-19T04:13:30.301237+0000","last_peered":"2024-09-19T04:13:30.301237+0000","last_clean":"2024-09-19T04:13:30.301237+0000","last_became_active":"2024-09-19T04:13:23.910355+0000","last_became_peered":"2024-09-19T04:13:23.910355+0000","last_unstale":"2024-09-19T04:13:30.301237+0000","last_undegraded":"2024-09-19T04:13:30.301237+0000","last_fullsized":"2024-09-19T04:13:30.301237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062270+0000","last_change":"2024-09-19T04:13:24.956969+0000","last_active":"2024-09-19T04:13:31.062270+0000","last_peered":"2024-09-19T04:13:31.062270+0000","last_clean":"2024-09-19T04:13:31.062270+0000","last_became_active":"2024-09-19T04:13:24.956671+0000","last_became_peered":"2024-09-19T04:13:24.956671+0000","last_unstale":"2024-09-19T04:13:31.062270+0000","last_undegraded":"2024-09-19T04:13:31.062270+0000","last_fullsized":"2024-09-19T04:13:31.062270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061243+0000","last_change":"2024-09-19T04:13:28.991785+0000","last_active":"2024-09-19T04:13:31.061243+0000","last_peered":"2024-09-19T04:13:31.061243+0000","last_clean":"2024-09-19T04:13:31.061243+0000","last_became_active":"2024-09-19T04:13:28.987169+0000","last_became_peered":"2024-09-19T04:13:28.987169+0000","last_unstale":"2024-09-19T04:13:31.061243+0000","last_undegraded":"2024-09-19T04:13:31.061243+0000","last_fullsized":"2024-09-19T04:13:31.061243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:00.614358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062999+0000","last_change":"2024-09-19T04:13:23.924693+0000","last_active":"2024-09-19T04:13:31.062999+0000","last_peered":"2024-09-19T04:13:31.062999+0000","last_clean":"2024-09-19T04:13:31.062999+0000","last_became_active":"2024-09-19T04:13:23.924077+0000","last_became_peered":"2024-09-19T04:13:23.924077+0000","last_unstale":"2024-09-19T04:13:31.062999+0000","last_undegraded":"2024-09-19T04:13:31.062999+0000","last_fullsized":"2024-09-19T04:13:31.062999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:10.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970152+0000","last_change":"2024-09-19T04:13:24.948486+0000","last_active":"2024-09-19T04:13:29.970152+0000","last_peered":"2024-09-19T04:13:29.970152+0000","last_clean":"2024-09-19T04:13:29.970152+0000","last_became_active":"2024-09-19T04:13:24.947749+0000","last_became_peered":"2024-09-19T04:13:24.947749+0000","last_unstale":"2024-09-19T04:13:29.970152+0000","last_undegraded":"2024-09-19T04:13:29.970152+0000","last_fullsized":"2024-09-19T04:13:29.970152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:37.654967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063108+0000","last_change":"2024-09-19T04:13:28.992157+0000","last_active":"2024-09-19T04:13:31.063108+0000","last_peered":"2024-09-19T04:13:31.063108+0000","last_clean":"2024-09-19T04:13:31.063108+0000","last_became_active":"2024-09-19T04:13:28.990879+0000","last_became_peered":"2024-09-19T04:13:28.990879+0000","last_unstale":"2024-09-19T04:13:31.063108+0000","last_undegraded":"2024-09-19T04:13:31.063108+0000","last_fullsized":"2024-09-19T04:13:31.063108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:39.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062192+0000","last_change":"2024-09-19T04:13:23.923303+0000","last_active":"2024-09-19T04:13:31.062192+0000","last_peered":"2024-09-19T04:13:31.062192+0000","last_clean":"2024-09-19T04:13:31.062192+0000","last_became_active":"2024-09-19T04:13:23.920842+0000","last_became_peered":"2024-09-19T04:13:23.920842+0000","last_unstale":"2024-09-19T04:13:31.062192+0000","last_undegraded":"2024-09-19T04:13:31.062192+0000","last_fullsized":"2024-09-19T04:13:31.062192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:23.176892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971159+0000","last_change":"2024-09-19T04:13:24.948074+0000","last_active":"2024-09-19T04:13:29.971159+0000","last_peered":"2024-09-19T04:13:29.971159+0000","last_clean":"2024-09-19T04:13:29.971159+0000","last_became_active":"2024-09-19T04:13:24.946595+0000","last_became_peered":"2024-09-19T04:13:24.946595+0000","last_unstale":"2024-09-19T04:13:29.971159+0000","last_undegraded":"2024-09-19T04:13:29.971159+0000","last_fullsized":"2024-09-19T04:13:29.971159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:21.342803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062406+0000","last_change":"2024-09-19T04:13:28.985965+0000","last_active":"2024-09-19T04:13:31.062406+0000","last_peered":"2024-09-19T04:13:31.062406+0000","last_clean":"2024-09-19T04:13:31.062406+0000","last_became_active":"2024-09-19T04:13:28.979842+0000","last_became_peered":"2024-09-19T04:13:28.979842+0000","last_unstale":"2024-09-19T04:13:31.062406+0000","last_undegraded":"2024-09-19T04:13:31.062406+0000","last_fullsized":"2024-09-19T04:13:31.062406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:13.654228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971528+0000","last_change":"2024-09-19T04:13:24.964645+0000","last_active":"2024-09-19T04:13:29.971528+0000","last_peered":"2024-09-19T04:13:29.971528+0000","last_clean":"2024-09-19T04:13:29.971528+0000","last_became_active":"2024-09-19T04:13:24.964051+0000","last_became_peered":"2024-09-19T04:13:24.964051+0000","last_unstale":"2024-09-19T04:13:29.971528+0000","last_undegraded":"2024-09-19T04:13:29.971528+0000","last_fullsized":"2024-09-19T04:13:29.971528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062721+0000","last_change":"2024-09-19T04:13:23.921499+0000","last_active":"2024-09-19T04:13:31.062721+0000","last_peered":"2024-09-19T04:13:31.062721+0000","last_clean":"2024-09-19T04:13:31.062721+0000","last_became_active":"2024-09-19T04:13:23.921146+0000","last_became_peered":"2024-09-19T04:13:23.921146+0000","last_unstale":"2024-09-19T04:13:31.062721+0000","last_undegraded":"2024-09-19T04:13:31.062721+0000","last_fullsized":"2024-09-19T04:13:31.062721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:54.181515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063276+0000","last_change":"2024-09-19T04:13:28.979497+0000","last_active":"2024-09-19T04:13:31.063276+0000","last_peered":"2024-09-19T04:13:31.063276+0000","last_clean":"2024-09-19T04:13:31.063276+0000","last_became_active":"2024-09-19T04:13:28.978329+0000","last_became_peered":"2024-09-19T04:13:28.978329+0000","last_unstale":"2024-09-19T04:13:31.063276+0000","last_undegraded":"2024-09-19T04:13:31.063276+0000","last_fullsized":"2024-09-19T04:13:31.063276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:30.314159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300960+0000","last_change":"2024-09-19T04:13:24.951290+0000","last_active":"2024-09-19T04:13:30.300960+0000","last_peered":"2024-09-19T04:13:30.300960+0000","last_clean":"2024-09-19T04:13:30.300960+0000","last_became_active":"2024-09-19T04:13:24.950973+0000","last_became_peered":"2024-09-19T04:13:24.950973+0000","last_unstale":"2024-09-19T04:13:30.300960+0000","last_undegraded":"2024-09-19T04:13:30.300960+0000","last_fullsized":"2024-09-19T04:13:30.300960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131596+0000","last_change":"2024-09-19T04:13:23.918153+0000","last_active":"2024-09-19T04:13:30.131596+0000","last_peered":"2024-09-19T04:13:30.131596+0000","last_clean":"2024-09-19T04:13:30.131596+0000","last_became_active":"2024-09-19T04:13:23.917783+0000","last_became_peered":"2024-09-19T04:13:23.917783+0000","last_unstale":"2024-09-19T04:13:30.131596+0000","last_undegraded":"2024-09-19T04:13:30.131596+0000","last_fullsized":"2024-09-19T04:13:30.131596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063970+0000","last_change":"2024-09-19T04:13:28.988863+0000","last_active":"2024-09-19T04:13:31.063970+0000","last_peered":"2024-09-19T04:13:31.063970+0000","last_clean":"2024-09-19T04:13:31.063970+0000","last_became_active":"2024-09-19T04:13:28.987942+0000","last_became_peered":"2024-09-19T04:13:28.987942+0000","last_unstale":"2024-09-19T04:13:31.063970+0000","last_undegraded":"2024-09-19T04:13:31.063970+0000","last_fullsized":"2024-09-19T04:13:31.063970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300167+0000","last_change":"2024-09-19T04:13:24.946137+0000","last_active":"2024-09-19T04:13:30.300167+0000","last_peered":"2024-09-19T04:13:30.300167+0000","last_clean":"2024-09-19T04:13:30.300167+0000","last_became_active":"2024-09-19T04:13:24.945850+0000","last_became_peered":"2024-09-19T04:13:24.945850+0000","last_unstale":"2024-09-19T04:13:30.300167+0000","last_undegraded":"2024-09-19T04:13:30.300167+0000","last_fullsized":"2024-09-19T04:13:30.300167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:23.916109+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:23.915767+0000","last_became_peered":"2024-09-19T04:13:23.915767+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131755+0000","last_change":"2024-09-19T04:13:29.030707+0000","last_active":"2024-09-19T04:13:30.131755+0000","last_peered":"2024-09-19T04:13:30.131755+0000","last_clean":"2024-09-19T04:13:30.131755+0000","last_became_active":"2024-09-19T04:13:29.030294+0000","last_became_peered":"2024-09-19T04:13:29.030294+0000","last_unstale":"2024-09-19T04:13:30.131755+0000","last_undegraded":"2024-09-19T04:13:30.131755+0000","last_fullsized":"2024-09-19T04:13:30.131755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063152+0000","last_change":"2024-09-19T04:13:24.948148+0000","last_active":"2024-09-19T04:13:31.063152+0000","last_peered":"2024-09-19T04:13:31.063152+0000","last_clean":"2024-09-19T04:13:31.063152+0000","last_became_active":"2024-09-19T04:13:24.947330+0000","last_became_peered":"2024-09-19T04:13:24.947330+0000","last_unstale":"2024-09-19T04:13:31.063152+0000","last_undegraded":"2024-09-19T04:13:31.063152+0000","last_fullsized":"2024-09-19T04:13:31.063152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:49.624559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061085+0000","last_change":"2024-09-19T04:13:23.923583+0000","last_active":"2024-09-19T04:13:31.061085+0000","last_peered":"2024-09-19T04:13:31.061085+0000","last_clean":"2024-09-19T04:13:31.061085+0000","last_became_active":"2024-09-19T04:13:23.921593+0000","last_became_peered":"2024-09-19T04:13:23.921593+0000","last_unstale":"2024-09-19T04:13:31.061085+0000","last_undegraded":"2024-09-19T04:13:31.061085+0000","last_fullsized":"2024-09-19T04:13:31.061085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063591+0000","last_change":"2024-09-19T04:13:28.990856+0000","last_active":"2024-09-19T04:13:31.063591+0000","last_peered":"2024-09-19T04:13:31.063591+0000","last_clean":"2024-09-19T04:13:31.063591+0000","last_became_active":"2024-09-19T04:13:28.987830+0000","last_became_peered":"2024-09-19T04:13:28.987830+0000","last_unstale":"2024-09-19T04:13:31.063591+0000","last_undegraded":"2024-09-19T04:13:31.063591+0000","last_fullsized":"2024-09-19T04:13:31.063591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062632+0000","last_change":"2024-09-19T04:13:24.934798+0000","last_active":"2024-09-19T04:13:31.062632+0000","last_peered":"2024-09-19T04:13:31.062632+0000","last_clean":"2024-09-19T04:13:31.062632+0000","last_became_active":"2024-09-19T04:13:24.933977+0000","last_became_peered":"2024-09-19T04:13:24.933977+0000","last_unstale":"2024-09-19T04:13:31.062632+0000","last_undegraded":"2024-09-19T04:13:31.062632+0000","last_fullsized":"2024-09-19T04:13:31.062632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973271+0000","last_change":"2024-09-19T04:13:23.920837+0000","last_active":"2024-09-19T04:13:29.973271+0000","last_peered":"2024-09-19T04:13:29.973271+0000","last_clean":"2024-09-19T04:13:29.973271+0000","last_became_active":"2024-09-19T04:13:23.920245+0000","last_became_peered":"2024-09-19T04:13:23.920245+0000","last_unstale":"2024-09-19T04:13:29.973271+0000","last_undegraded":"2024-09-19T04:13:29.973271+0000","last_fullsized":"2024-09-19T04:13:29.973271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062572+0000","last_change":"2024-09-19T04:13:29.019257+0000","last_active":"2024-09-19T04:13:31.062572+0000","last_peered":"2024-09-19T04:13:31.062572+0000","last_clean":"2024-09-19T04:13:31.062572+0000","last_became_active":"2024-09-19T04:13:29.018425+0000","last_became_peered":"2024-09-19T04:13:29.018425+0000","last_unstale":"2024-09-19T04:13:31.062572+0000","last_undegraded":"2024-09-19T04:13:31.062572+0000","last_fullsized":"2024-09-19T04:13:31.062572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:42.657248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301533+0000","last_change":"2024-09-19T04:13:24.944069+0000","last_active":"2024-09-19T04:13:30.301533+0000","last_peered":"2024-09-19T04:13:30.301533+0000","last_clean":"2024-09-19T04:13:30.301533+0000","last_became_active":"2024-09-19T04:13:24.943774+0000","last_became_peered":"2024-09-19T04:13:24.943774+0000","last_unstale":"2024-09-19T04:13:30.301533+0000","last_undegraded":"2024-09-19T04:13:30.301533+0000","last_fullsized":"2024-09-19T04:13:30.301533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131771+0000","last_change":"2024-09-19T04:13:23.911708+0000","last_active":"2024-09-19T04:13:30.131771+0000","last_peered":"2024-09-19T04:13:30.131771+0000","last_clean":"2024-09-19T04:13:30.131771+0000","last_became_active":"2024-09-19T04:13:23.911135+0000","last_became_peered":"2024-09-19T04:13:23.911135+0000","last_unstale":"2024-09-19T04:13:30.131771+0000","last_undegraded":"2024-09-19T04:13:30.131771+0000","last_fullsized":"2024-09-19T04:13:30.131771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:48.393589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299979+0000","last_change":"2024-09-19T04:13:29.020657+0000","last_active":"2024-09-19T04:13:30.299979+0000","last_peered":"2024-09-19T04:13:30.299979+0000","last_clean":"2024-09-19T04:13:30.299979+0000","last_became_active":"2024-09-19T04:13:29.020172+0000","last_became_peered":"2024-09-19T04:13:29.020172+0000","last_unstale":"2024-09-19T04:13:30.299979+0000","last_undegraded":"2024-09-19T04:13:30.299979+0000","last_fullsized":"2024-09-19T04:13:30.299979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131405+0000","last_change":"2024-09-19T04:13:24.949008+0000","last_active":"2024-09-19T04:13:30.131405+0000","last_peered":"2024-09-19T04:13:30.131405+0000","last_clean":"2024-09-19T04:13:30.131405+0000","last_became_active":"2024-09-19T04:13:24.948513+0000","last_became_peered":"2024-09-19T04:13:24.948513+0000","last_unstale":"2024-09-19T04:13:30.131405+0000","last_undegraded":"2024-09-19T04:13:30.131405+0000","last_fullsized":"2024-09-19T04:13:30.131405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063250+0000","last_change":"2024-09-19T04:13:23.921473+0000","last_active":"2024-09-19T04:13:31.063250+0000","last_peered":"2024-09-19T04:13:31.063250+0000","last_clean":"2024-09-19T04:13:31.063250+0000","last_became_active":"2024-09-19T04:13:23.921052+0000","last_became_peered":"2024-09-19T04:13:23.921052+0000","last_unstale":"2024-09-19T04:13:31.063250+0000","last_undegraded":"2024-09-19T04:13:31.063250+0000","last_fullsized":"2024-09-19T04:13:31.063250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:17.114813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061949+0000","last_change":"2024-09-19T04:13:29.020022+0000","last_active":"2024-09-19T04:13:31.061949+0000","last_peered":"2024-09-19T04:13:31.061949+0000","last_clean":"2024-09-19T04:13:31.061949+0000","last_became_active":"2024-09-19T04:13:29.019585+0000","last_became_peered":"2024-09-19T04:13:29.019585+0000","last_unstale":"2024-09-19T04:13:31.061949+0000","last_undegraded":"2024-09-19T04:13:31.061949+0000","last_fullsized":"2024-09-19T04:13:31.061949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970941+0000","last_change":"2024-09-19T04:13:24.952102+0000","last_active":"2024-09-19T04:13:29.970941+0000","last_peered":"2024-09-19T04:13:29.970941+0000","last_clean":"2024-09-19T04:13:29.970941+0000","last_became_active":"2024-09-19T04:13:24.951184+0000","last_became_peered":"2024-09-19T04:13:24.951184+0000","last_unstale":"2024-09-19T04:13:29.970941+0000","last_undegraded":"2024-09-19T04:13:29.970941+0000","last_fullsized":"2024-09-19T04:13:29.970941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131508+0000","last_change":"2024-09-19T04:13:23.908259+0000","last_active":"2024-09-19T04:13:30.131508+0000","last_peered":"2024-09-19T04:13:30.131508+0000","last_clean":"2024-09-19T04:13:30.131508+0000","last_became_active":"2024-09-19T04:13:23.907717+0000","last_became_peered":"2024-09-19T04:13:23.907717+0000","last_unstale":"2024-09-19T04:13:30.131508+0000","last_undegraded":"2024-09-19T04:13:30.131508+0000","last_fullsized":"2024-09-19T04:13:30.131508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.975119+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.974476+0000","last_became_peered":"2024-09-19T04:13:28.974476+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063922+0000","last_change":"2024-09-19T04:13:24.922621+0000","last_active":"2024-09-19T04:13:31.063922+0000","last_peered":"2024-09-19T04:13:31.063922+0000","last_clean":"2024-09-19T04:13:31.063922+0000","last_became_active":"2024-09-19T04:13:24.921879+0000","last_became_peered":"2024-09-19T04:13:24.921879+0000","last_unstale":"2024-09-19T04:13:31.063922+0000","last_undegraded":"2024-09-19T04:13:31.063922+0000","last_fullsized":"2024-09-19T04:13:31.063922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:04.750679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972020+0000","last_change":"2024-09-19T04:13:23.901650+0000","last_active":"2024-09-19T04:13:29.972020+0000","last_peered":"2024-09-19T04:13:29.972020+0000","last_clean":"2024-09-19T04:13:29.972020+0000","last_became_active":"2024-09-19T04:13:23.900880+0000","last_became_peered":"2024-09-19T04:13:23.900880+0000","last_unstale":"2024-09-19T04:13:29.972020+0000","last_undegraded":"2024-09-19T04:13:29.972020+0000","last_fullsized":"2024-09-19T04:13:29.972020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131635+0000","last_change":"2024-09-19T04:13:28.980693+0000","last_active":"2024-09-19T04:13:30.131635+0000","last_peered":"2024-09-19T04:13:30.131635+0000","last_clean":"2024-09-19T04:13:30.131635+0000","last_became_active":"2024-09-19T04:13:28.979948+0000","last_became_peered":"2024-09-19T04:13:28.979948+0000","last_unstale":"2024-09-19T04:13:30.131635+0000","last_undegraded":"2024-09-19T04:13:30.131635+0000","last_fullsized":"2024-09-19T04:13:30.131635+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130985+0000","last_change":"2024-09-19T04:13:24.948867+0000","last_active":"2024-09-19T04:13:30.130985+0000","last_peered":"2024-09-19T04:13:30.130985+0000","last_clean":"2024-09-19T04:13:30.130985+0000","last_became_active":"2024-09-19T04:13:24.948471+0000","last_became_peered":"2024-09-19T04:13:24.948471+0000","last_unstale":"2024-09-19T04:13:30.130985+0000","last_undegraded":"2024-09-19T04:13:30.130985+0000","last_fullsized":"2024-09-19T04:13:30.130985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970768+0000","last_change":"2024-09-19T04:13:23.903592+0000","last_active":"2024-09-19T04:13:29.970768+0000","last_peered":"2024-09-19T04:13:29.970768+0000","last_clean":"2024-09-19T04:13:29.970768+0000","last_became_active":"2024-09-19T04:13:23.903306+0000","last_became_peered":"2024-09-19T04:13:23.903306+0000","last_unstale":"2024-09-19T04:13:29.970768+0000","last_undegraded":"2024-09-19T04:13:29.970768+0000","last_fullsized":"2024-09-19T04:13:29.970768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:13.135208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131834+0000","last_change":"2024-09-19T04:13:28.976039+0000","last_active":"2024-09-19T04:13:30.131834+0000","last_peered":"2024-09-19T04:13:30.131834+0000","last_clean":"2024-09-19T04:13:30.131834+0000","last_became_active":"2024-09-19T04:13:28.975424+0000","last_became_peered":"2024-09-19T04:13:28.975424+0000","last_unstale":"2024-09-19T04:13:30.131834+0000","last_undegraded":"2024-09-19T04:13:30.131834+0000","last_fullsized":"2024-09-19T04:13:30.131834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:27.255493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061535+0000","last_change":"2024-09-19T04:13:24.956224+0000","last_active":"2024-09-19T04:13:31.061535+0000","last_peered":"2024-09-19T04:13:31.061535+0000","last_clean":"2024-09-19T04:13:31.061535+0000","last_became_active":"2024-09-19T04:13:24.953799+0000","last_became_peered":"2024-09-19T04:13:24.953799+0000","last_unstale":"2024-09-19T04:13:31.061535+0000","last_undegraded":"2024-09-19T04:13:31.061535+0000","last_fullsized":"2024-09-19T04:13:31.061535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:33.543861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061237+0000","last_change":"2024-09-19T04:13:23.919493+0000","last_active":"2024-09-19T04:13:31.061237+0000","last_peered":"2024-09-19T04:13:31.061237+0000","last_clean":"2024-09-19T04:13:31.061237+0000","last_became_active":"2024-09-19T04:13:23.918702+0000","last_became_peered":"2024-09-19T04:13:23.918702+0000","last_unstale":"2024-09-19T04:13:31.061237+0000","last_undegraded":"2024-09-19T04:13:31.061237+0000","last_fullsized":"2024-09-19T04:13:31.061237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:42.592915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062499+0000","last_change":"2024-09-19T04:13:29.028357+0000","last_active":"2024-09-19T04:13:31.062499+0000","last_peered":"2024-09-19T04:13:31.062499+0000","last_clean":"2024-09-19T04:13:31.062499+0000","last_became_active":"2024-09-19T04:13:29.027946+0000","last_became_peered":"2024-09-19T04:13:29.027946+0000","last_unstale":"2024-09-19T04:13:31.062499+0000","last_undegraded":"2024-09-19T04:13:31.062499+0000","last_fullsized":"2024-09-19T04:13:31.062499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300424+0000","last_change":"2024-09-19T04:13:24.968375+0000","last_active":"2024-09-19T04:13:30.300424+0000","last_peered":"2024-09-19T04:13:30.300424+0000","last_clean":"2024-09-19T04:13:30.300424+0000","last_became_active":"2024-09-19T04:13:24.968097+0000","last_became_peered":"2024-09-19T04:13:24.968097+0000","last_unstale":"2024-09-19T04:13:30.300424+0000","last_undegraded":"2024-09-19T04:13:30.300424+0000","last_fullsized":"2024-09-19T04:13:30.300424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969947+0000","last_change":"2024-09-19T04:13:23.903638+0000","last_active":"2024-09-19T04:13:29.969947+0000","last_peered":"2024-09-19T04:13:29.969947+0000","last_clean":"2024-09-19T04:13:29.969947+0000","last_became_active":"2024-09-19T04:13:23.898948+0000","last_became_peered":"2024-09-19T04:13:23.898948+0000","last_unstale":"2024-09-19T04:13:29.969947+0000","last_undegraded":"2024-09-19T04:13:29.969947+0000","last_fullsized":"2024-09-19T04:13:29.969947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061278+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:13:31.061278+0000","last_peered":"2024-09-19T04:13:31.061278+0000","last_clean":"2024-09-19T04:13:31.061278+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:13:31.061278+0000","last_undegraded":"2024-09-19T04:13:31.061278+0000","last_fullsized":"2024-09-19T04:13:31.061278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972561+0000","last_change":"2024-09-19T04:13:29.020008+0000","last_active":"2024-09-19T04:13:29.972561+0000","last_peered":"2024-09-19T04:13:29.972561+0000","last_clean":"2024-09-19T04:13:29.972561+0000","last_became_active":"2024-09-19T04:13:29.018706+0000","last_became_peered":"2024-09-19T04:13:29.018706+0000","last_unstale":"2024-09-19T04:13:29.972561+0000","last_undegraded":"2024-09-19T04:13:29.972561+0000","last_fullsized":"2024-09-19T04:13:29.972561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062691+0000","last_change":"2024-09-19T04:13:24.966194+0000","last_active":"2024-09-19T04:13:31.062691+0000","last_peered":"2024-09-19T04:13:31.062691+0000","last_clean":"2024-09-19T04:13:31.062691+0000","last_became_active":"2024-09-19T04:13:24.965690+0000","last_became_peered":"2024-09-19T04:13:24.965690+0000","last_unstale":"2024-09-19T04:13:31.062691+0000","last_undegraded":"2024-09-19T04:13:31.062691+0000","last_fullsized":"2024-09-19T04:13:31.062691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:14.510098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971314+0000","last_change":"2024-09-19T04:13:23.901039+0000","last_active":"2024-09-19T04:13:29.971314+0000","last_peered":"2024-09-19T04:13:29.971314+0000","last_clean":"2024-09-19T04:13:29.971314+0000","last_became_active":"2024-09-19T04:13:23.900709+0000","last_became_peered":"2024-09-19T04:13:23.900709+0000","last_unstale":"2024-09-19T04:13:29.971314+0000","last_undegraded":"2024-09-19T04:13:29.971314+0000","last_fullsized":"2024-09-19T04:13:29.971314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:19:40.425738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969768+0000","last_change":"2024-09-19T04:13:29.019565+0000","last_active":"2024-09-19T04:13:29.969768+0000","last_peered":"2024-09-19T04:13:29.969768+0000","last_clean":"2024-09-19T04:13:29.969768+0000","last_became_active":"2024-09-19T04:13:29.018547+0000","last_became_peered":"2024-09-19T04:13:29.018547+0000","last_unstale":"2024-09-19T04:13:29.969768+0000","last_undegraded":"2024-09-19T04:13:29.969768+0000","last_fullsized":"2024-09-19T04:13:29.969768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970414+0000","last_change":"2024-09-19T04:13:24.953123+0000","last_active":"2024-09-19T04:13:29.970414+0000","last_peered":"2024-09-19T04:13:29.970414+0000","last_clean":"2024-09-19T04:13:29.970414+0000","last_became_active":"2024-09-19T04:13:24.950851+0000","last_became_peered":"2024-09-19T04:13:24.950851+0000","last_unstale":"2024-09-19T04:13:29.970414+0000","last_undegraded":"2024-09-19T04:13:29.970414+0000","last_fullsized":"2024-09-19T04:13:29.970414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131715+0000","last_change":"2024-09-19T04:13:23.902143+0000","last_active":"2024-09-19T04:13:30.131715+0000","last_peered":"2024-09-19T04:13:30.131715+0000","last_clean":"2024-09-19T04:13:30.131715+0000","last_became_active":"2024-09-19T04:13:23.901843+0000","last_became_peered":"2024-09-19T04:13:23.901843+0000","last_unstale":"2024-09-19T04:13:30.131715+0000","last_undegraded":"2024-09-19T04:13:30.131715+0000","last_fullsized":"2024-09-19T04:13:30.131715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:03.746896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062168+0000","last_change":"2024-09-19T04:13:28.994151+0000","last_active":"2024-09-19T04:13:31.062168+0000","last_peered":"2024-09-19T04:13:31.062168+0000","last_clean":"2024-09-19T04:13:31.062168+0000","last_became_active":"2024-09-19T04:13:28.993608+0000","last_became_peered":"2024-09-19T04:13:28.993608+0000","last_unstale":"2024-09-19T04:13:31.062168+0000","last_undegraded":"2024-09-19T04:13:31.062168+0000","last_fullsized":"2024-09-19T04:13:31.062168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064926+0000","last_change":"2024-09-19T04:13:24.966033+0000","last_active":"2024-09-19T04:13:31.064926+0000","last_peered":"2024-09-19T04:13:31.064926+0000","last_clean":"2024-09-19T04:13:31.064926+0000","last_became_active":"2024-09-19T04:13:24.962081+0000","last_became_peered":"2024-09-19T04:13:24.962081+0000","last_unstale":"2024-09-19T04:13:31.064926+0000","last_undegraded":"2024-09-19T04:13:31.064926+0000","last_fullsized":"2024-09-19T04:13:31.064926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:58.621676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971640+0000","last_change":"2024-09-19T04:13:23.911774+0000","last_active":"2024-09-19T04:13:29.971640+0000","last_peered":"2024-09-19T04:13:29.971640+0000","last_clean":"2024-09-19T04:13:29.971640+0000","last_became_active":"2024-09-19T04:13:23.911154+0000","last_became_peered":"2024-09-19T04:13:23.911154+0000","last_unstale":"2024-09-19T04:13:29.971640+0000","last_undegraded":"2024-09-19T04:13:29.971640+0000","last_fullsized":"2024-09-19T04:13:29.971640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969327+0000","last_change":"2024-09-19T04:13:29.020613+0000","last_active":"2024-09-19T04:13:29.969327+0000","last_peered":"2024-09-19T04:13:29.969327+0000","last_clean":"2024-09-19T04:13:29.969327+0000","last_became_active":"2024-09-19T04:13:29.019473+0000","last_became_peered":"2024-09-19T04:13:29.019473+0000","last_unstale":"2024-09-19T04:13:29.969327+0000","last_undegraded":"2024-09-19T04:13:29.969327+0000","last_fullsized":"2024-09-19T04:13:29.969327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064591+0000","last_change":"2024-09-19T04:13:24.962284+0000","last_active":"2024-09-19T04:13:31.064591+0000","last_peered":"2024-09-19T04:13:31.064591+0000","last_clean":"2024-09-19T04:13:31.064591+0000","last_became_active":"2024-09-19T04:13:24.961763+0000","last_became_peered":"2024-09-19T04:13:24.961763+0000","last_unstale":"2024-09-19T04:13:31.064591+0000","last_undegraded":"2024-09-19T04:13:31.064591+0000","last_fullsized":"2024-09-19T04:13:31.064591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:40.924861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064080+0000","last_change":"2024-09-19T04:13:23.905574+0000","last_active":"2024-09-19T04:13:31.064080+0000","last_peered":"2024-09-19T04:13:31.064080+0000","last_clean":"2024-09-19T04:13:31.064080+0000","last_became_active":"2024-09-19T04:13:23.904993+0000","last_became_peered":"2024-09-19T04:13:23.904993+0000","last_unstale":"2024-09-19T04:13:31.064080+0000","last_undegraded":"2024-09-19T04:13:31.064080+0000","last_fullsized":"2024-09-19T04:13:31.064080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:00.615690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061135+0000","last_change":"2024-09-19T04:13:28.991444+0000","last_active":"2024-09-19T04:13:31.061135+0000","last_peered":"2024-09-19T04:13:31.061135+0000","last_clean":"2024-09-19T04:13:31.061135+0000","last_became_active":"2024-09-19T04:13:28.987837+0000","last_became_peered":"2024-09-19T04:13:28.987837+0000","last_unstale":"2024-09-19T04:13:31.061135+0000","last_undegraded":"2024-09-19T04:13:31.061135+0000","last_fullsized":"2024-09-19T04:13:31.061135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:17.703055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061563+0000","last_change":"2024-09-19T04:13:24.965972+0000","last_active":"2024-09-19T04:13:31.061563+0000","last_peered":"2024-09-19T04:13:31.061563+0000","last_clean":"2024-09-19T04:13:31.061563+0000","last_became_active":"2024-09-19T04:13:24.965315+0000","last_became_peered":"2024-09-19T04:13:24.965315+0000","last_unstale":"2024-09-19T04:13:31.061563+0000","last_undegraded":"2024-09-19T04:13:31.061563+0000","last_fullsized":"2024-09-19T04:13:31.061563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:16.174335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970388+0000","last_change":"2024-09-19T04:13:23.896881+0000","last_active":"2024-09-19T04:13:29.970388+0000","last_peered":"2024-09-19T04:13:29.970388+0000","last_clean":"2024-09-19T04:13:29.970388+0000","last_became_active":"2024-09-19T04:13:23.896440+0000","last_became_peered":"2024-09-19T04:13:23.896440+0000","last_unstale":"2024-09-19T04:13:29.970388+0000","last_undegraded":"2024-09-19T04:13:29.970388+0000","last_fullsized":"2024-09-19T04:13:29.970388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:36:56.677455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062482+0000","last_change":"2024-09-19T04:13:28.985691+0000","last_active":"2024-09-19T04:13:31.062482+0000","last_peered":"2024-09-19T04:13:31.062482+0000","last_clean":"2024-09-19T04:13:31.062482+0000","last_became_active":"2024-09-19T04:13:28.984749+0000","last_became_peered":"2024-09-19T04:13:28.984749+0000","last_unstale":"2024-09-19T04:13:31.062482+0000","last_undegraded":"2024-09-19T04:13:31.062482+0000","last_fullsized":"2024-09-19T04:13:31.062482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969844+0000","last_change":"2024-09-19T04:13:24.942591+0000","last_active":"2024-09-19T04:13:29.969844+0000","last_peered":"2024-09-19T04:13:29.969844+0000","last_clean":"2024-09-19T04:13:29.969844+0000","last_became_active":"2024-09-19T04:13:24.941789+0000","last_became_peered":"2024-09-19T04:13:24.941789+0000","last_unstale":"2024-09-19T04:13:29.969844+0000","last_undegraded":"2024-09-19T04:13:29.969844+0000","last_fullsized":"2024-09-19T04:13:29.969844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970255+0000","last_change":"2024-09-19T04:13:23.916881+0000","last_active":"2024-09-19T04:13:29.970255+0000","last_peered":"2024-09-19T04:13:29.970255+0000","last_clean":"2024-09-19T04:13:29.970255+0000","last_became_active":"2024-09-19T04:13:23.916050+0000","last_became_peered":"2024-09-19T04:13:23.916050+0000","last_unstale":"2024-09-19T04:13:29.970255+0000","last_undegraded":"2024-09-19T04:13:29.970255+0000","last_fullsized":"2024-09-19T04:13:29.970255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301470+0000","last_change":"2024-09-19T04:13:29.021093+0000","last_active":"2024-09-19T04:13:30.301470+0000","last_peered":"2024-09-19T04:13:30.301470+0000","last_clean":"2024-09-19T04:13:30.301470+0000","last_became_active":"2024-09-19T04:13:29.020573+0000","last_became_peered":"2024-09-19T04:13:29.020573+0000","last_unstale":"2024-09-19T04:13:30.301470+0000","last_undegraded":"2024-09-19T04:13:30.301470+0000","last_fullsized":"2024-09-19T04:13:30.301470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:35.622467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062500+0000","last_change":"2024-09-19T04:13:24.968018+0000","last_active":"2024-09-19T04:13:31.062500+0000","last_peered":"2024-09-19T04:13:31.062500+0000","last_clean":"2024-09-19T04:13:31.062500+0000","last_became_active":"2024-09-19T04:13:24.962386+0000","last_became_peered":"2024-09-19T04:13:24.962386+0000","last_unstale":"2024-09-19T04:13:31.062500+0000","last_undegraded":"2024-09-19T04:13:31.062500+0000","last_fullsized":"2024-09-19T04:13:31.062500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:39.665883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971777+0000","last_change":"2024-09-19T04:13:23.911883+0000","last_active":"2024-09-19T04:13:29.971777+0000","last_peered":"2024-09-19T04:13:29.971777+0000","last_clean":"2024-09-19T04:13:29.971777+0000","last_became_active":"2024-09-19T04:13:23.910990+0000","last_became_peered":"2024-09-19T04:13:23.910990+0000","last_unstale":"2024-09-19T04:13:29.971777+0000","last_undegraded":"2024-09-19T04:13:29.971777+0000","last_fullsized":"2024-09-19T04:13:29.971777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301624+0000","last_change":"2024-09-19T04:13:29.017846+0000","last_active":"2024-09-19T04:13:30.301624+0000","last_peered":"2024-09-19T04:13:30.301624+0000","last_clean":"2024-09-19T04:13:30.301624+0000","last_became_active":"2024-09-19T04:13:29.017269+0000","last_became_peered":"2024-09-19T04:13:29.017269+0000","last_unstale":"2024-09-19T04:13:30.301624+0000","last_undegraded":"2024-09-19T04:13:30.301624+0000","last_fullsized":"2024-09-19T04:13:30.301624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:07.730896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061954+0000","last_change":"2024-09-19T04:13:24.944986+0000","last_active":"2024-09-19T04:13:31.061954+0000","last_peered":"2024-09-19T04:13:31.061954+0000","last_clean":"2024-09-19T04:13:31.061954+0000","last_became_active":"2024-09-19T04:13:24.944266+0000","last_became_peered":"2024-09-19T04:13:24.944266+0000","last_unstale":"2024-09-19T04:13:31.061954+0000","last_undegraded":"2024-09-19T04:13:31.061954+0000","last_fullsized":"2024-09-19T04:13:31.061954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:17.729114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062229+0000","last_change":"2024-09-19T04:13:23.902337+0000","last_active":"2024-09-19T04:13:31.062229+0000","last_peered":"2024-09-19T04:13:31.062229+0000","last_clean":"2024-09-19T04:13:31.062229+0000","last_became_active":"2024-09-19T04:13:23.901958+0000","last_became_peered":"2024-09-19T04:13:23.901958+0000","last_unstale":"2024-09-19T04:13:31.062229+0000","last_undegraded":"2024-09-19T04:13:31.062229+0000","last_fullsized":"2024-09-19T04:13:31.062229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:39.987613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970753+0000","last_change":"2024-09-19T04:13:29.022266+0000","last_active":"2024-09-19T04:13:29.970753+0000","last_peered":"2024-09-19T04:13:29.970753+0000","last_clean":"2024-09-19T04:13:29.970753+0000","last_became_active":"2024-09-19T04:13:29.021104+0000","last_became_peered":"2024-09-19T04:13:29.021104+0000","last_unstale":"2024-09-19T04:13:29.970753+0000","last_undegraded":"2024-09-19T04:13:29.970753+0000","last_fullsized":"2024-09-19T04:13:29.970753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062460+0000","last_change":"2024-09-19T04:13:24.952759+0000","last_active":"2024-09-19T04:13:31.062460+0000","last_peered":"2024-09-19T04:13:31.062460+0000","last_clean":"2024-09-19T04:13:31.062460+0000","last_became_active":"2024-09-19T04:13:24.952088+0000","last_became_peered":"2024-09-19T04:13:24.952088+0000","last_unstale":"2024-09-19T04:13:31.062460+0000","last_undegraded":"2024-09-19T04:13:31.062460+0000","last_fullsized":"2024-09-19T04:13:31.062460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:29.020474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:35.460051+0000","last_change":"2024-09-19T04:13:23.909991+0000","last_active":"2024-09-19T04:13:35.460051+0000","last_peered":"2024-09-19T04:13:35.460051+0000","last_clean":"2024-09-19T04:13:35.460051+0000","last_became_active":"2024-09-19T04:13:23.909517+0000","last_became_peered":"2024-09-19T04:13:23.909517+0000","last_unstale":"2024-09-19T04:13:35.460051+0000","last_undegraded":"2024-09-19T04:13:35.460051+0000","last_fullsized":"2024-09-19T04:13:35.460051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.984556+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.983961+0000","last_became_peered":"2024-09-19T04:13:28.983961+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064342+0000","last_change":"2024-09-19T04:13:24.943383+0000","last_active":"2024-09-19T04:13:31.064342+0000","last_peered":"2024-09-19T04:13:31.064342+0000","last_clean":"2024-09-19T04:13:31.064342+0000","last_became_active":"2024-09-19T04:13:24.942094+0000","last_became_peered":"2024-09-19T04:13:24.942094+0000","last_unstale":"2024-09-19T04:13:31.064342+0000","last_undegraded":"2024-09-19T04:13:31.064342+0000","last_fullsized":"2024-09-19T04:13:31.064342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:15.642865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131674+0000","last_change":"2024-09-19T04:13:23.911338+0000","last_active":"2024-09-19T04:13:30.131674+0000","last_peered":"2024-09-19T04:13:30.131674+0000","last_clean":"2024-09-19T04:13:30.131674+0000","last_became_active":"2024-09-19T04:13:23.910962+0000","last_became_peered":"2024-09-19T04:13:23.910962+0000","last_unstale":"2024-09-19T04:13:30.131674+0000","last_undegraded":"2024-09-19T04:13:30.131674+0000","last_fullsized":"2024-09-19T04:13:30.131674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:39.854426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063766+0000","last_change":"2024-09-19T04:13:28.988142+0000","last_active":"2024-09-19T04:13:31.063766+0000","last_peered":"2024-09-19T04:13:31.063766+0000","last_clean":"2024-09-19T04:13:31.063766+0000","last_became_active":"2024-09-19T04:13:28.979829+0000","last_became_peered":"2024-09-19T04:13:28.979829+0000","last_unstale":"2024-09-19T04:13:31.063766+0000","last_undegraded":"2024-09-19T04:13:31.063766+0000","last_fullsized":"2024-09-19T04:13:31.063766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062345+0000","last_change":"2024-09-19T04:13:24.956553+0000","last_active":"2024-09-19T04:13:31.062345+0000","last_peered":"2024-09-19T04:13:31.062345+0000","last_clean":"2024-09-19T04:13:31.062345+0000","last_became_active":"2024-09-19T04:13:24.956238+0000","last_became_peered":"2024-09-19T04:13:24.956238+0000","last_unstale":"2024-09-19T04:13:31.062345+0000","last_undegraded":"2024-09-19T04:13:31.062345+0000","last_fullsized":"2024-09-19T04:13:31.062345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970527+0000","last_change":"2024-09-19T04:13:23.918490+0000","last_active":"2024-09-19T04:13:29.970527+0000","last_peered":"2024-09-19T04:13:29.970527+0000","last_clean":"2024-09-19T04:13:29.970527+0000","last_became_active":"2024-09-19T04:13:23.917969+0000","last_became_peered":"2024-09-19T04:13:23.917969+0000","last_unstale":"2024-09-19T04:13:29.970527+0000","last_undegraded":"2024-09-19T04:13:29.970527+0000","last_fullsized":"2024-09-19T04:13:29.970527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300379+0000","last_change":"2024-09-19T04:13:28.988707+0000","last_active":"2024-09-19T04:13:30.300379+0000","last_peered":"2024-09-19T04:13:30.300379+0000","last_clean":"2024-09-19T04:13:30.300379+0000","last_became_active":"2024-09-19T04:13:28.988388+0000","last_became_peered":"2024-09-19T04:13:28.988388+0000","last_unstale":"2024-09-19T04:13:30.300379+0000","last_undegraded":"2024-09-19T04:13:30.300379+0000","last_fullsized":"2024-09-19T04:13:30.300379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:54.745179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970310+0000","last_change":"2024-09-19T04:13:24.943308+0000","last_active":"2024-09-19T04:13:29.970310+0000","last_peered":"2024-09-19T04:13:29.970310+0000","last_clean":"2024-09-19T04:13:29.970310+0000","last_became_active":"2024-09-19T04:13:24.942608+0000","last_became_peered":"2024-09-19T04:13:24.942608+0000","last_unstale":"2024-09-19T04:13:29.970310+0000","last_undegraded":"2024-09-19T04:13:29.970310+0000","last_fullsized":"2024-09-19T04:13:29.970310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972997+0000","last_change":"2024-09-19T04:13:23.918096+0000","last_active":"2024-09-19T04:13:29.972997+0000","last_peered":"2024-09-19T04:13:29.972997+0000","last_clean":"2024-09-19T04:13:29.972997+0000","last_became_active":"2024-09-19T04:13:23.917505+0000","last_became_peered":"2024-09-19T04:13:23.917505+0000","last_unstale":"2024-09-19T04:13:29.972997+0000","last_undegraded":"2024-09-19T04:13:29.972997+0000","last_fullsized":"2024-09-19T04:13:29.972997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300417+0000","last_change":"2024-09-19T04:13:29.027451+0000","last_active":"2024-09-19T04:13:30.300417+0000","last_peered":"2024-09-19T04:13:30.300417+0000","last_clean":"2024-09-19T04:13:30.300417+0000","last_became_active":"2024-09-19T04:13:29.027181+0000","last_became_peered":"2024-09-19T04:13:29.027181+0000","last_unstale":"2024-09-19T04:13:30.300417+0000","last_undegraded":"2024-09-19T04:13:30.300417+0000","last_fullsized":"2024-09-19T04:13:30.300417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:42.206935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062147+0000","last_change":"2024-09-19T04:13:29.028793+0000","last_active":"2024-09-19T04:13:31.062147+0000","last_peered":"2024-09-19T04:13:31.062147+0000","last_clean":"2024-09-19T04:13:31.062147+0000","last_became_active":"2024-09-19T04:13:29.028293+0000","last_became_peered":"2024-09-19T04:13:29.028293+0000","last_unstale":"2024-09-19T04:13:31.062147+0000","last_undegraded":"2024-09-19T04:13:31.062147+0000","last_fullsized":"2024-09-19T04:13:31.062147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130915+0000","last_change":"2024-09-19T04:13:24.947415+0000","last_active":"2024-09-19T04:13:30.130915+0000","last_peered":"2024-09-19T04:13:30.130915+0000","last_clean":"2024-09-19T04:13:30.130915+0000","last_became_active":"2024-09-19T04:13:24.947042+0000","last_became_peered":"2024-09-19T04:13:24.947042+0000","last_unstale":"2024-09-19T04:13:30.130915+0000","last_undegraded":"2024-09-19T04:13:30.130915+0000","last_fullsized":"2024-09-19T04:13:30.130915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969735+0000","last_change":"2024-09-19T04:13:23.906982+0000","last_active":"2024-09-19T04:13:29.969735+0000","last_peered":"2024-09-19T04:13:29.969735+0000","last_clean":"2024-09-19T04:13:29.969735+0000","last_became_active":"2024-09-19T04:13:23.905070+0000","last_became_peered":"2024-09-19T04:13:23.905070+0000","last_unstale":"2024-09-19T04:13:29.969735+0000","last_undegraded":"2024-09-19T04:13:29.969735+0000","last_fullsized":"2024-09-19T04:13:29.969735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972911+0000","last_change":"2024-09-19T04:13:28.979353+0000","last_active":"2024-09-19T04:13:29.972911+0000","last_peered":"2024-09-19T04:13:29.972911+0000","last_clean":"2024-09-19T04:13:29.972911+0000","last_became_active":"2024-09-19T04:13:28.976991+0000","last_became_peered":"2024-09-19T04:13:28.976991+0000","last_unstale":"2024-09-19T04:13:29.972911+0000","last_undegraded":"2024-09-19T04:13:29.972911+0000","last_fullsized":"2024-09-19T04:13:29.972911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300060+0000","last_change":"2024-09-19T04:13:24.947190+0000","last_active":"2024-09-19T04:13:30.300060+0000","last_peered":"2024-09-19T04:13:30.300060+0000","last_clean":"2024-09-19T04:13:30.300060+0000","last_became_active":"2024-09-19T04:13:24.946883+0000","last_became_peered":"2024-09-19T04:13:24.946883+0000","last_unstale":"2024-09-19T04:13:30.300060+0000","last_undegraded":"2024-09-19T04:13:30.300060+0000","last_fullsized":"2024-09-19T04:13:30.300060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300792+0000","last_change":"2024-09-19T04:13:23.912532+0000","last_active":"2024-09-19T04:13:30.300792+0000","last_peered":"2024-09-19T04:13:30.300792+0000","last_clean":"2024-09-19T04:13:30.300792+0000","last_became_active":"2024-09-19T04:13:23.911952+0000","last_became_peered":"2024-09-19T04:13:23.911952+0000","last_unstale":"2024-09-19T04:13:30.300792+0000","last_undegraded":"2024-09-19T04:13:30.300792+0000","last_fullsized":"2024-09-19T04:13:30.300792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971798+0000","last_change":"2024-09-19T04:13:23.912898+0000","last_active":"2024-09-19T04:13:29.971798+0000","last_peered":"2024-09-19T04:13:29.971798+0000","last_clean":"2024-09-19T04:13:29.971798+0000","last_became_active":"2024-09-19T04:13:23.912234+0000","last_became_peered":"2024-09-19T04:13:23.912234+0000","last_unstale":"2024-09-19T04:13:29.971798+0000","last_undegraded":"2024-09-19T04:13:29.971798+0000","last_fullsized":"2024-09-19T04:13:29.971798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062138+0000","last_change":"2024-09-19T04:13:24.953761+0000","last_active":"2024-09-19T04:13:31.062138+0000","last_peered":"2024-09-19T04:13:31.062138+0000","last_clean":"2024-09-19T04:13:31.062138+0000","last_became_active":"2024-09-19T04:13:24.953195+0000","last_became_peered":"2024-09-19T04:13:24.953195+0000","last_unstale":"2024-09-19T04:13:31.062138+0000","last_undegraded":"2024-09-19T04:13:31.062138+0000","last_fullsized":"2024-09-19T04:13:31.062138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971169+0000","last_change":"2024-09-19T04:13:28.976510+0000","last_active":"2024-09-19T04:13:29.971169+0000","last_peered":"2024-09-19T04:13:29.971169+0000","last_clean":"2024-09-19T04:13:29.971169+0000","last_became_active":"2024-09-19T04:13:28.975842+0000","last_became_peered":"2024-09-19T04:13:28.975842+0000","last_unstale":"2024-09-19T04:13:29.971169+0000","last_undegraded":"2024-09-19T04:13:29.971169+0000","last_fullsized":"2024-09-19T04:13:29.971169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:36:39.502090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301204+0000","last_change":"2024-09-19T04:13:23.920382+0000","last_active":"2024-09-19T04:13:30.301204+0000","last_peered":"2024-09-19T04:13:30.301204+0000","last_clean":"2024-09-19T04:13:30.301204+0000","last_became_active":"2024-09-19T04:13:23.920107+0000","last_became_peered":"2024-09-19T04:13:23.920107+0000","last_unstale":"2024-09-19T04:13:30.301204+0000","last_undegraded":"2024-09-19T04:13:30.301204+0000","last_fullsized":"2024-09-19T04:13:30.301204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:24.725046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062882+0000","last_change":"2024-09-19T04:13:24.933838+0000","last_active":"2024-09-19T04:13:31.062882+0000","last_peered":"2024-09-19T04:13:31.062882+0000","last_clean":"2024-09-19T04:13:31.062882+0000","last_became_active":"2024-09-19T04:13:24.932423+0000","last_became_peered":"2024-09-19T04:13:24.932423+0000","last_unstale":"2024-09-19T04:13:31.062882+0000","last_undegraded":"2024-09-19T04:13:31.062882+0000","last_fullsized":"2024-09-19T04:13:31.062882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:52.498071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301388+0000","last_change":"2024-09-19T04:13:28.982366+0000","last_active":"2024-09-19T04:13:30.301388+0000","last_peered":"2024-09-19T04:13:30.301388+0000","last_clean":"2024-09-19T04:13:30.301388+0000","last_became_active":"2024-09-19T04:13:28.981608+0000","last_became_peered":"2024-09-19T04:13:28.981608+0000","last_unstale":"2024-09-19T04:13:30.301388+0000","last_undegraded":"2024-09-19T04:13:30.301388+0000","last_fullsized":"2024-09-19T04:13:30.301388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:12.864952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969437+0000","last_change":"2024-09-19T04:13:24.939240+0000","last_active":"2024-09-19T04:13:29.969437+0000","last_peered":"2024-09-19T04:13:29.969437+0000","last_clean":"2024-09-19T04:13:29.969437+0000","last_became_active":"2024-09-19T04:13:24.938527+0000","last_became_peered":"2024-09-19T04:13:24.938527+0000","last_unstale":"2024-09-19T04:13:29.969437+0000","last_undegraded":"2024-09-19T04:13:29.969437+0000","last_fullsized":"2024-09-19T04:13:29.969437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970473+0000","last_change":"2024-09-19T04:13:24.967773+0000","last_active":"2024-09-19T04:13:29.970473+0000","last_peered":"2024-09-19T04:13:29.970473+0000","last_clean":"2024-09-19T04:13:29.970473+0000","last_became_active":"2024-09-19T04:13:24.967467+0000","last_became_peered":"2024-09-19T04:13:24.967467+0000","last_unstale":"2024-09-19T04:13:29.970473+0000","last_undegraded":"2024-09-19T04:13:29.970473+0000","last_fullsized":"2024-09-19T04:13:29.970473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:11.155011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971529+0000","last_change":"2024-09-19T04:13:24.945017+0000","last_active":"2024-09-19T04:13:29.971529+0000","last_peered":"2024-09-19T04:13:29.971529+0000","last_clean":"2024-09-19T04:13:29.971529+0000","last_became_active":"2024-09-19T04:13:24.944535+0000","last_became_peered":"2024-09-19T04:13:24.944535+0000","last_unstale":"2024-09-19T04:13:29.971529+0000","last_undegraded":"2024-09-19T04:13:29.971529+0000","last_fullsized":"2024-09-19T04:13:29.971529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:43.596741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062090+0000","last_change":"2024-09-19T04:13:24.952497+0000","last_active":"2024-09-19T04:13:31.062090+0000","last_peered":"2024-09-19T04:13:31.062090+0000","last_clean":"2024-09-19T04:13:31.062090+0000","last_became_active":"2024-09-19T04:13:24.952037+0000","last_became_peered":"2024-09-19T04:13:24.952037+0000","last_unstale":"2024-09-19T04:13:31.062090+0000","last_undegraded":"2024-09-19T04:13:31.062090+0000","last_fullsized":"2024-09-19T04:13:31.062090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301786+0000","last_change":"2024-09-19T04:13:24.966504+0000","last_active":"2024-09-19T04:13:30.301786+0000","last_peered":"2024-09-19T04:13:30.301786+0000","last_clean":"2024-09-19T04:13:30.301786+0000","last_became_active":"2024-09-19T04:13:24.965992+0000","last_became_peered":"2024-09-19T04:13:24.965992+0000","last_unstale":"2024-09-19T04:13:30.301786+0000","last_undegraded":"2024-09-19T04:13:30.301786+0000","last_fullsized":"2024-09-19T04:13:30.301786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131526+0000","last_change":"2024-09-19T04:13:24.949970+0000","last_active":"2024-09-19T04:13:30.131526+0000","last_peered":"2024-09-19T04:13:30.131526+0000","last_clean":"2024-09-19T04:13:30.131526+0000","last_became_active":"2024-09-19T04:13:24.949676+0000","last_became_peered":"2024-09-19T04:13:24.949676+0000","last_unstale":"2024-09-19T04:13:30.131526+0000","last_undegraded":"2024-09-19T04:13:30.131526+0000","last_fullsized":"2024-09-19T04:13:30.131526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973014+0000","last_change":"2024-09-19T04:13:24.939046+0000","last_active":"2024-09-19T04:13:29.973014+0000","last_peered":"2024-09-19T04:13:29.973014+0000","last_clean":"2024-09-19T04:13:29.973014+0000","last_became_active":"2024-09-19T04:13:24.938277+0000","last_became_peered":"2024-09-19T04:13:24.938277+0000","last_unstale":"2024-09-19T04:13:29.973014+0000","last_undegraded":"2024-09-19T04:13:29.973014+0000","last_fullsized":"2024-09-19T04:13:29.973014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301516+0000","last_change":"2024-09-19T04:13:24.962572+0000","last_active":"2024-09-19T04:13:30.301516+0000","last_peered":"2024-09-19T04:13:30.301516+0000","last_clean":"2024-09-19T04:13:30.301516+0000","last_became_active":"2024-09-19T04:13:24.962277+0000","last_became_peered":"2024-09-19T04:13:24.962277+0000","last_unstale":"2024-09-19T04:13:30.301516+0000","last_undegraded":"2024-09-19T04:13:30.301516+0000","last_fullsized":"2024-09-19T04:13:30.301516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969487+0000","last_change":"2024-09-19T04:13:24.951536+0000","last_active":"2024-09-19T04:13:29.969487+0000","last_peered":"2024-09-19T04:13:29.969487+0000","last_clean":"2024-09-19T04:13:29.969487+0000","last_became_active":"2024-09-19T04:13:24.950939+0000","last_became_peered":"2024-09-19T04:13:24.950939+0000","last_unstale":"2024-09-19T04:13:29.969487+0000","last_undegraded":"2024-09-19T04:13:29.969487+0000","last_fullsized":"2024-09-19T04:13:29.969487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300567+0000","last_change":"2024-09-19T04:13:24.961761+0000","last_active":"2024-09-19T04:13:30.300567+0000","last_peered":"2024-09-19T04:13:30.300567+0000","last_clean":"2024-09-19T04:13:30.300567+0000","last_became_active":"2024-09-19T04:13:24.961415+0000","last_became_peered":"2024-09-19T04:13:24.961415+0000","last_unstale":"2024-09-19T04:13:30.300567+0000","last_undegraded":"2024-09-19T04:13:30.300567+0000","last_fullsized":"2024-09-19T04:13:30.300567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973133+0000","last_change":"2024-09-19T04:13:24.939730+0000","last_active":"2024-09-19T04:13:29.973133+0000","last_peered":"2024-09-19T04:13:29.973133+0000","last_clean":"2024-09-19T04:13:29.973133+0000","last_became_active":"2024-09-19T04:13:24.938400+0000","last_became_peered":"2024-09-19T04:13:24.938400+0000","last_unstale":"2024-09-19T04:13:29.973133+0000","last_undegraded":"2024-09-19T04:13:29.973133+0000","last_fullsized":"2024-09-19T04:13:29.973133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062008+0000","last_change":"2024-09-19T04:13:24.944592+0000","last_active":"2024-09-19T04:13:31.062008+0000","last_peered":"2024-09-19T04:13:31.062008+0000","last_clean":"2024-09-19T04:13:31.062008+0000","last_became_active":"2024-09-19T04:13:24.943915+0000","last_became_peered":"2024-09-19T04:13:24.943915+0000","last_unstale":"2024-09-19T04:13:31.062008+0000","last_undegraded":"2024-09-19T04:13:31.062008+0000","last_fullsized":"2024-09-19T04:13:31.062008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972579+0000","last_change":"2024-09-19T04:13:24.967058+0000","last_active":"2024-09-19T04:13:29.972579+0000","last_peered":"2024-09-19T04:13:29.972579+0000","last_clean":"2024-09-19T04:13:29.972579+0000","last_became_active":"2024-09-19T04:13:24.964521+0000","last_became_peered":"2024-09-19T04:13:24.964521+0000","last_unstale":"2024-09-19T04:13:29.972579+0000","last_undegraded":"2024-09-19T04:13:29.972579+0000","last_fullsized":"2024-09-19T04:13:29.972579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063128+0000","last_change":"2024-09-19T04:13:24.953833+0000","last_active":"2024-09-19T04:13:31.063128+0000","last_peered":"2024-09-19T04:13:31.063128+0000","last_clean":"2024-09-19T04:13:31.063128+0000","last_became_active":"2024-09-19T04:13:24.950878+0000","last_became_peered":"2024-09-19T04:13:24.950878+0000","last_unstale":"2024-09-19T04:13:31.063128+0000","last_undegraded":"2024-09-19T04:13:31.063128+0000","last_fullsized":"2024-09-19T04:13:31.063128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:07.653424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971483+0000","last_change":"2024-09-19T04:13:24.956195+0000","last_active":"2024-09-19T04:13:29.971483+0000","last_peered":"2024-09-19T04:13:29.971483+0000","last_clean":"2024-09-19T04:13:29.971483+0000","last_became_active":"2024-09-19T04:13:24.955911+0000","last_became_peered":"2024-09-19T04:13:24.955911+0000","last_unstale":"2024-09-19T04:13:29.971483+0000","last_undegraded":"2024-09-19T04:13:29.971483+0000","last_fullsized":"2024-09-19T04:13:29.971483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:55.185297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970952+0000","last_change":"2024-09-19T04:13:24.939650+0000","last_active":"2024-09-19T04:13:29.970952+0000","last_peered":"2024-09-19T04:13:29.970952+0000","last_clean":"2024-09-19T04:13:29.970952+0000","last_became_active":"2024-09-19T04:13:24.938774+0000","last_became_peered":"2024-09-19T04:13:24.938774+0000","last_unstale":"2024-09-19T04:13:29.970952+0000","last_undegraded":"2024-09-19T04:13:29.970952+0000","last_fullsized":"2024-09-19T04:13:29.970952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:00.871440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969219+0000","last_change":"2024-09-19T04:13:24.921407+0000","last_active":"2024-09-19T04:13:29.969219+0000","last_peered":"2024-09-19T04:13:29.969219+0000","last_clean":"2024-09-19T04:13:29.969219+0000","last_became_active":"2024-09-19T04:13:24.920646+0000","last_became_peered":"2024-09-19T04:13:24.920646+0000","last_unstale":"2024-09-19T04:13:29.969219+0000","last_undegraded":"2024-09-19T04:13:29.969219+0000","last_fullsized":"2024-09-19T04:13:29.969219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:24.963230+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:24.962932+0000","last_became_peered":"2024-09-19T04:13:24.962932+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061900+0000","last_change":"2024-09-19T04:13:24.942848+0000","last_active":"2024-09-19T04:13:31.061900+0000","last_peered":"2024-09-19T04:13:31.061900+0000","last_clean":"2024-09-19T04:13:31.061900+0000","last_became_active":"2024-09-19T04:13:24.942524+0000","last_became_peered":"2024-09-19T04:13:24.942524+0000","last_unstale":"2024-09-19T04:13:31.061900+0000","last_undegraded":"2024-09-19T04:13:31.061900+0000","last_fullsized":"2024-09-19T04:13:31.061900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:26.370075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300722+0000","last_change":"2024-09-19T04:13:24.950629+0000","last_active":"2024-09-19T04:13:30.300722+0000","last_peered":"2024-09-19T04:13:30.300722+0000","last_clean":"2024-09-19T04:13:30.300722+0000","last_became_active":"2024-09-19T04:13:24.946193+0000","last_became_peered":"2024-09-19T04:13:24.946193+0000","last_unstale":"2024-09-19T04:13:30.300722+0000","last_undegraded":"2024-09-19T04:13:30.300722+0000","last_fullsized":"2024-09-19T04:13:30.300722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064210+0000","last_change":"2024-09-19T04:13:24.936924+0000","last_active":"2024-09-19T04:13:31.064210+0000","last_peered":"2024-09-19T04:13:31.064210+0000","last_clean":"2024-09-19T04:13:31.064210+0000","last_became_active":"2024-09-19T04:13:24.936053+0000","last_became_peered":"2024-09-19T04:13:24.936053+0000","last_unstale":"2024-09-19T04:13:31.064210+0000","last_undegraded":"2024-09-19T04:13:31.064210+0000","last_fullsized":"2024-09-19T04:13:31.064210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:40.858208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972557+0000","last_change":"2024-09-19T04:13:24.967121+0000","last_active":"2024-09-19T04:13:29.972557+0000","last_peered":"2024-09-19T04:13:29.972557+0000","last_clean":"2024-09-19T04:13:29.972557+0000","last_became_active":"2024-09-19T04:13:24.964161+0000","last_became_peered":"2024-09-19T04:13:24.964161+0000","last_unstale":"2024-09-19T04:13:29.972557+0000","last_undegraded":"2024-09-19T04:13:29.972557+0000","last_fullsized":"2024-09-19T04:13:29.972557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972765+0000","last_change":"2024-09-19T04:13:24.968947+0000","last_active":"2024-09-19T04:13:29.972765+0000","last_peered":"2024-09-19T04:13:29.972765+0000","last_clean":"2024-09-19T04:13:29.972765+0000","last_became_active":"2024-09-19T04:13:24.968335+0000","last_became_peered":"2024-09-19T04:13:24.968335+0000","last_unstale":"2024-09-19T04:13:29.972765+0000","last_undegraded":"2024-09-19T04:13:29.972765+0000","last_fullsized":"2024-09-19T04:13:29.972765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064806+0000","last_change":"2024-09-19T04:13:24.970001+0000","last_active":"2024-09-19T04:13:31.064806+0000","last_peered":"2024-09-19T04:13:31.064806+0000","last_clean":"2024-09-19T04:13:31.064806+0000","last_became_active":"2024-09-19T04:13:24.969462+0000","last_became_peered":"2024-09-19T04:13:24.969462+0000","last_unstale":"2024-09-19T04:13:31.064806+0000","last_undegraded":"2024-09-19T04:13:31.064806+0000","last_fullsized":"2024-09-19T04:13:31.064806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:15:32.140847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299773+0000","last_change":"2024-09-19T04:13:24.967702+0000","last_active":"2024-09-19T04:13:30.299773+0000","last_peered":"2024-09-19T04:13:30.299773+0000","last_clean":"2024-09-19T04:13:30.299773+0000","last_became_active":"2024-09-19T04:13:24.967436+0000","last_became_peered":"2024-09-19T04:13:24.967436+0000","last_unstale":"2024-09-19T04:13:30.299773+0000","last_undegraded":"2024-09-19T04:13:30.299773+0000","last_fullsized":"2024-09-19T04:13:30.299773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061448+0000","last_change":"2024-09-19T04:13:24.942637+0000","last_active":"2024-09-19T04:13:31.061448+0000","last_peered":"2024-09-19T04:13:31.061448+0000","last_clean":"2024-09-19T04:13:31.061448+0000","last_became_active":"2024-09-19T04:13:24.942164+0000","last_became_peered":"2024-09-19T04:13:24.942164+0000","last_unstale":"2024-09-19T04:13:31.061448+0000","last_undegraded":"2024-09-19T04:13:31.061448+0000","last_fullsized":"2024-09-19T04:13:31.061448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:30.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971132+0000","last_change":"2024-09-19T04:13:24.926176+0000","last_active":"2024-09-19T04:13:29.971132+0000","last_peered":"2024-09-19T04:13:29.971132+0000","last_clean":"2024-09-19T04:13:29.971132+0000","last_became_active":"2024-09-19T04:13:24.925246+0000","last_became_peered":"2024-09-19T04:13:24.925246+0000","last_unstale":"2024-09-19T04:13:29.971132+0000","last_undegraded":"2024-09-19T04:13:29.971132+0000","last_fullsized":"2024-09-19T04:13:29.971132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:52.769507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062030+0000","last_change":"2024-09-19T04:13:24.953254+0000","last_active":"2024-09-19T04:13:31.062030+0000","last_peered":"2024-09-19T04:13:31.062030+0000","last_clean":"2024-09-19T04:13:31.062030+0000","last_became_active":"2024-09-19T04:13:24.950624+0000","last_became_peered":"2024-09-19T04:13:24.950624+0000","last_unstale":"2024-09-19T04:13:31.062030+0000","last_undegraded":"2024-09-19T04:13:31.062030+0000","last_fullsized":"2024-09-19T04:13:31.062030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:06.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969906+0000","last_change":"2024-09-19T04:13:24.954966+0000","last_active":"2024-09-19T04:13:29.969906+0000","last_peered":"2024-09-19T04:13:29.969906+0000","last_clean":"2024-09-19T04:13:29.969906+0000","last_became_active":"2024-09-19T04:13:24.954234+0000","last_became_peered":"2024-09-19T04:13:24.954234+0000","last_unstale":"2024-09-19T04:13:29.969906+0000","last_undegraded":"2024-09-19T04:13:29.969906+0000","last_fullsized":"2024-09-19T04:13:29.969906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061764+0000","last_change":"2024-09-19T04:13:24.949281+0000","last_active":"2024-09-19T04:13:31.061764+0000","last_peered":"2024-09-19T04:13:31.061764+0000","last_clean":"2024-09-19T04:13:31.061764+0000","last_became_active":"2024-09-19T04:13:24.948863+0000","last_became_peered":"2024-09-19T04:13:24.948863+0000","last_unstale":"2024-09-19T04:13:31.061764+0000","last_undegraded":"2024-09-19T04:13:31.061764+0000","last_fullsized":"2024-09-19T04:13:31.061764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131694+0000","last_change":"2024-09-19T04:13:24.943099+0000","last_active":"2024-09-19T04:13:30.131694+0000","last_peered":"2024-09-19T04:13:30.131694+0000","last_clean":"2024-09-19T04:13:30.131694+0000","last_became_active":"2024-09-19T04:13:24.942776+0000","last_became_peered":"2024-09-19T04:13:24.942776+0000","last_unstale":"2024-09-19T04:13:30.131694+0000","last_undegraded":"2024-09-19T04:13:30.131694+0000","last_fullsized":"2024-09-19T04:13:30.131694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064570+0000","last_change":"2024-09-19T04:13:24.951658+0000","last_active":"2024-09-19T04:13:31.064570+0000","last_peered":"2024-09-19T04:13:31.064570+0000","last_clean":"2024-09-19T04:13:31.064570+0000","last_became_active":"2024-09-19T04:13:24.950976+0000","last_became_peered":"2024-09-19T04:13:24.950976+0000","last_unstale":"2024-09-19T04:13:31.064570+0000","last_undegraded":"2024-09-19T04:13:31.064570+0000","last_fullsized":"2024-09-19T04:13:31.064570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:39.061865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":694435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":103611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542152,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":103485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542152,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":103645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:13:40.309 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:13:40.309 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T04:13:40.309 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-19T04:13:40.309 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-19T04:13:40.309 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-19T04:13:40.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.450+0000 7f5bc36f0640 1 Processor -- start 2024-09-19T04:13:40.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.450+0000 7f5bc36f0640 1 -- start start 2024-09-19T04:13:40.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc36f0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc1520a0 0x7f5bbc06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc36f0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5bbc06fe50 0x7f5bbc15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc36f0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bbc070250 con 0x7f5bbc15f1d0 2024-09-19T04:13:40.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc36f0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5bbc152480 con 0x7f5bbc06fe50 2024-09-19T04:13:40.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc36f0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc0aede0 con 0x7f5bbc1520a0 2024-09-19T04:13:40.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc1c8d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5bbc15f1d0 0x7f5bbc071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34298/0 (socket says 172.21.15.76:34298) 2024-09-19T04:13:40.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc1c8d640 1 -- 172.21.15.76:0/2242145461 learned_addr learned my addr 172.21.15.76:0/2242145461 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc148c640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc1520a0 0x7f5bbc06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 --2- 172.21.15.76:0/2242145461 >> v2:172.21.15.116:3300/0 conn(0x7f5bbc06fe50 0x7f5bbc15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16736461 0 0) 0x7f5bbc070250 con 0x7f5bbc15f1d0 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ba4003530 con 0x7f5bbc15f1d0 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1624098321 0 0) 0x7f5ba4003530 con 0x7f5bbc15f1d0 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc1520a0 msgr2=0x7f5bbc06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc1520a0 0x7f5bbc06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 >> v2:172.21.15.116:3300/0 conn(0x7f5bbc06fe50 msgr2=0x7f5bbc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 --2- 172.21.15.76:0/2242145461 >> v2:172.21.15.116:3300/0 conn(0x7f5bbc06fe50 0x7f5bbc15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc15fa10 con 0x7f5bbc15f1d0 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5bb4001a00 con 0x7f5bbc15f1d0 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 >> v1:172.21.15.76:6789/0 conn(0x7f5bbc15f1d0 legacy=0x7f5bbc071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba40039c0 0x7f5ba4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba4004810 0x7f5ba4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc1520a0 0x7f5ba40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc0aede0 con 0x7f5ba4004810 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba400dbe0 con 0x7f5ba40039c0 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba400dd60 con 0x7f5bbc1520a0 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba4004810 0x7f5ba4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc148c640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba40039c0 0x7f5ba4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 -- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc1520a0 msgr2=0x7f5ba40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:40.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc1520a0 0x7f5ba40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 -- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba40039c0 msgr2=0x7f5ba4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:40.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba40039c0 0x7f5ba4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 -- 172.21.15.76:0/2242145461 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba4003530 con 0x7f5ba4004810 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.454+0000 7f5bc0c8b640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba4004810 0x7f5ba4004c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009ab0 tx=0x7f5bb001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=655813055d32bea4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00b2070 con 0x7f5ba4004810 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb00a55a0 con 0x7f5ba4004810 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bbbfff640 1 -- 172.21.15.76:0/2242145461 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00096b0 con 0x7f5ba4004810 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- 172.21.15.76:0/2242145461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba4004810 msgr2=0x7f5ba4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 --2- 172.21.15.76:0/2242145461 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba4004810 0x7f5ba4004c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009ab0 tx=0x7f5bb001ca90 comp rx=0 tx=0).stop 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 -- 172.21.15.76:0/2242145461 reap_dead start 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- 172.21.15.76:0/2242145461 shutdown_connections 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- 172.21.15.76:0/2242145461 >> 172.21.15.76:0/2242145461 conn(0x7f5bbc0b1f90 msgr2=0x7f5bbc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:40.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- 172.21.15.76:0/2242145461 shutdown_connections 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- 172.21.15.76:0/2242145461 wait complete. 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 Processor -- start 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- start start 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4004810 0x7f5bbc15e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc0cf840 0x7f5bbc0b07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bbc1f7dc0 0x7f5bbc159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc073770 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc073490 con 0x7f5ba4004810 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc073a50 con 0x7f5bbc0cf840 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bbc1f7dc0 0x7f5bbc159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bbc1f7dc0 0x7f5bbc159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40560/0 (socket says 172.21.15.76:40560) 2024-09-19T04:13:40.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 -- 172.21.15.76:0/3736616727 learned_addr learned my addr 172.21.15.76:0/3736616727 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc148c640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4004810 0x7f5bbc15e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc0c8b640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc0cf840 0x7f5bbc0b07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 -- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc0cf840 msgr2=0x7f5bbc0b07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc0cf840 0x7f5bbc0b07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 -- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4004810 msgr2=0x7f5bbc15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4004810 0x7f5bbc15e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 -- 172.21.15.76:0/3736616727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc0b1020 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc0c8b640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc0cf840 0x7f5bbc0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc1c8d640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bbc1f7dc0 0x7f5bbc159ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc073340 tx=0x7f5bb400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bb9ffb640 1 -- 172.21.15.76:0/3736616727 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb400d9f0 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bb9ffb640 1 -- 172.21.15.76:0/3736616727 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb400db90 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bbc0b1300 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bb9ffb640 1 -- 172.21.15.76:0/3736616727 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4017420 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.458+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bbc1f75c0 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.462+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bbc073d30 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.462+0000 7f5bb9ffb640 1 -- 172.21.15.76:0/3736616727 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5bb4014070 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.462+0000 7f5bb9ffb640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ba80678c0 0x7f5ba8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.462+0000 7f5bb9ffb640 1 -- 172.21.15.76:0/3736616727 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5bb4088c70 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.462+0000 7f5bc148c640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ba80678c0 0x7f5ba8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.462+0000 7f5bc148c640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ba80678c0 0x7f5ba8069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0b1ee0 tx=0x7f5bac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.466+0000 7f5bb9ffb640 1 -- 172.21.15.76:0/3736616727 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5bb401d060 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.650+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f5bbc06e540 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.658+0000 7f5bb9ffb640 1 -- 172.21.15.76:0/3736616727 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+166886 (secure 0 0 0) 0x7f5bb4059470 con 0x7f5bbc1f7dc0 2024-09-19T04:13:40.665 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ba80678c0 msgr2=0x7f5ba8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ba80678c0 0x7f5ba8069d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc0b1ee0 tx=0x7f5bac008040 comp rx=0 tx=0).stop 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bbc1f7dc0 msgr2=0x7f5bbc159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bbc1f7dc0 0x7f5bbc159ff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc073340 tx=0x7f5bb400ff00 comp rx=0 tx=0).stop 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 shutdown_connections 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ba80678c0 0x7f5ba8069d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bbc1f7dc0 0x7f5bbc159ff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bbc0cf840 0x7f5bbc0b07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 --2- 172.21.15.76:0/3736616727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba4004810 0x7f5bbc15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 >> 172.21.15.76:0/3736616727 conn(0x7f5bbc0b1f90 msgr2=0x7f5bbc1504b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 shutdown_connections 2024-09-19T04:13:40.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.662+0000 7f5bc36f0640 1 -- 172.21.15.76:0/3736616727 wait complete. 2024-09-19T04:13:40.682 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6832","nonce":3111825053},{"type":"v1","addr":"172.21.15.76:6833","nonce":3111825053}]},"active_addr":"172.21.15.76:6833/3111825053","active_change":"2024-09-19T04:13:05.724270+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":1058949247}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":145024566}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":4087769127}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.76:0","nonce":1418671568}]}]} 2024-09-19T04:13:40.687 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-19T04:13:40.687 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-19T04:13:40.687 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:40.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.798+0000 7f81c8115640 1 Processor -- start 2024-09-19T04:13:40.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.798+0000 7f81c8115640 1 -- start start 2024-09-19T04:13:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 0x7f81c00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81c015c2a0 0x7f81c00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81c00af2a0 0x7f81c015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00b3290 con 0x7f81c015c2a0 2024-09-19T04:13:40.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f81c00b3570 con 0x7f81c00af2a0 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f81c00b3850 con 0x7f81c015bec0 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81c015c2a0 0x7f81c00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f81c015c2a0 0x7f81c00aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40570/0 (socket says 172.21.15.76:40570) 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 -- 172.21.15.76:0/2763675296 learned_addr learned my addr 172.21.15.76:0/2763675296 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c66b2640 1 --2- 172.21.15.76:0/2763675296 >> v2:172.21.15.116:3300/0 conn(0x7f81c00af2a0 0x7f81c015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c5eb1640 1 --2- 172.21.15.76:0/2763675296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 0x7f81c00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 -- 172.21.15.76:0/2763675296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 msgr2=0x7f81c00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 --2- 172.21.15.76:0/2763675296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 0x7f81c00ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 -- 172.21.15.76:0/2763675296 >> v2:172.21.15.116:3300/0 conn(0x7f81c00af2a0 msgr2=0x7f81c015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 --2- 172.21.15.76:0/2763675296 >> v2:172.21.15.116:3300/0 conn(0x7f81c00af2a0 0x7f81c015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 -- 172.21.15.76:0/2763675296 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c00b3b30 con 0x7f81c015c2a0 2024-09-19T04:13:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c56b0640 1 --2- 172.21.15.76:0/2763675296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c015c2a0 0x7f81c00aed60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81c00ada40 tx=0x7f81b001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca8558afc6f0d174 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c4eaf640 1 -- 172.21.15.76:0/2763675296 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00b0070 con 0x7f81c015c2a0 2024-09-19T04:13:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c4eaf640 1 -- 172.21.15.76:0/2763675296 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81b00ab600 con 0x7f81c015c2a0 2024-09-19T04:13:40.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c4eaf640 1 -- 172.21.15.76:0/2763675296 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b00ab9d0 con 0x7f81c015c2a0 2024-09-19T04:13:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 -- 172.21.15.76:0/2763675296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c015c2a0 msgr2=0x7f81c00aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 --2- 172.21.15.76:0/2763675296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c015c2a0 0x7f81c00aed60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f81c00ada40 tx=0x7f81b001c230 comp rx=0 tx=0).stop 2024-09-19T04:13:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 -- 172.21.15.76:0/2763675296 shutdown_connections 2024-09-19T04:13:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 --2- 172.21.15.76:0/2763675296 >> v2:172.21.15.116:3300/0 conn(0x7f81c00af2a0 0x7f81c015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 --2- 172.21.15.76:0/2763675296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c015c2a0 0x7f81c00aed60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 --2- 172.21.15.76:0/2763675296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 0x7f81c00ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.802+0000 7f81c8115640 1 -- 172.21.15.76:0/2763675296 >> 172.21.15.76:0/2763675296 conn(0x7f81c00b1f90 msgr2=0x7f81c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- 172.21.15.76:0/2763675296 shutdown_connections 2024-09-19T04:13:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- 172.21.15.76:0/2763675296 wait complete. 2024-09-19T04:13:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 Processor -- start 2024-09-19T04:13:40.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- start start 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c00af2a0 0x7f81c0152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 0x7f81c009d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c015c2a0 0x7f81c0159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00af6b0 con 0x7f81c00af2a0 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00ad740 con 0x7f81c015c2a0 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f81c006e000 con 0x7f81c015bec0 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c00af2a0 0x7f81c0152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c00af2a0 0x7f81c0152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40582/0 (socket says 172.21.15.76:40582) 2024-09-19T04:13:40.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 -- 172.21.15.76:0/2774637241 learned_addr learned my addr 172.21.15.76:0/2774637241 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 -- 172.21.15.76:0/2774637241 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 msgr2=0x7f81c009d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 0x7f81c009d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 -- 172.21.15.76:0/2774637241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c015c2a0 msgr2=0x7f81c0159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c015c2a0 0x7f81c0159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 -- 172.21.15.76:0/2774637241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c009d900 con 0x7f81c00af2a0 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c5eb1640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c00af2a0 0x7f81c0152630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00d800 tx=0x7f81bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81b67fc640 1 -- 172.21.15.76:0/2774637241 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00b6c0 con 0x7f81c00af2a0 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81b67fc640 1 -- 172.21.15.76:0/2774637241 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81bc019070 con 0x7f81c00af2a0 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c009dbe0 con 0x7f81c00af2a0 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81b67fc640 1 -- 172.21.15.76:0/2774637241 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00bae0 con 0x7f81c00af2a0 2024-09-19T04:13:40.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c009e1a0 con 0x7f81c00af2a0 2024-09-19T04:13:40.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.806+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c00ad200 con 0x7f81c00af2a0 2024-09-19T04:13:40.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.810+0000 7f81b67fc640 1 -- 172.21.15.76:0/2774637241 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81bc00bc80 con 0x7f81c00af2a0 2024-09-19T04:13:40.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.810+0000 7f81b67fc640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81a40678f0 0x7f81a4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:40.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.810+0000 7f81b67fc640 1 -- 172.21.15.76:0/2774637241 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f81bc086e50 con 0x7f81c00af2a0 2024-09-19T04:13:40.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.810+0000 7f81c56b0640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81a40678f0 0x7f81a4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:40.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.810+0000 7f81c56b0640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81a40678f0 0x7f81a4069db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ad000 tx=0x7f81b0004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.814+0000 7f81b67fc640 1 -- 172.21.15.76:0/2774637241 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f81bc0577c0 con 0x7f81c00af2a0 2024-09-19T04:13:40.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.974+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81c006fcd0 con 0x7f81c00af2a0 2024-09-19T04:13:40.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81b67fc640 1 -- 172.21.15.76:0/2774637241 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f81bc0570c0 con 0x7f81c00af2a0 2024-09-19T04:13:40.982 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:40.982 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81a40678f0 msgr2=0x7f81a4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81a40678f0 0x7f81a4069db0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f81b00ad000 tx=0x7f81b0004740 comp rx=0 tx=0).stop 2024-09-19T04:13:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c00af2a0 msgr2=0x7f81c0152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c00af2a0 0x7f81c0152630 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00d800 tx=0x7f81bc00dcc0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 shutdown_connections 2024-09-19T04:13:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81a40678f0 0x7f81a4069db0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81c015c2a0 0x7f81c0159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f81c015bec0 0x7f81c009d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 --2- 172.21.15.76:0/2774637241 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f81c00af2a0 0x7f81c0152630 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 >> 172.21.15.76:0/2774637241 conn(0x7f81c00b1f90 msgr2=0x7f81c00aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 shutdown_connections 2024-09-19T04:13:40.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:40.978+0000 7f81c8115640 1 -- 172.21.15.76:0/2774637241 wait complete. 2024-09-19T04:13:41.000 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-19T04:13:41.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:41.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2c3b6640 1 Processor -- start 2024-09-19T04:13:41.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2c3b6640 1 -- start start 2024-09-19T04:13:41.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2c3b6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e241630c0 0x7f1e2406f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2c3b6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e240b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2c3b6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e24163bb0 con 0x7f1e2406df70 2024-09-19T04:13:41.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2c3b6640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1e2406e470 con 0x7f1e241630c0 2024-09-19T04:13:41.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2c3b6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2406e750 con 0x7f1e240b3290 2024-09-19T04:13:41.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a953640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e240b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a152640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e241630c0 0x7f1e2406f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a152640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e241630c0 0x7f1e2406f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56052/0 (socket says 172.21.15.76:56052) 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a152640 1 -- 172.21.15.76:0/2743176119 learned_addr learned my addr 172.21.15.76:0/2743176119 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a953640 1 --2- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e240b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38118/0 (socket says 172.21.15.76:38118) 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e29951640 1 --1- 172.21.15.76:0/2743176119 >> v1:172.21.15.116:6789/0 conn(0x7f1e2406df70 0x7f1e241634a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:46038/0 (socket says 172.21.15.76:46038) 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a152640 1 -- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 msgr2=0x7f1e240b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a152640 1 --2- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e240b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a152640 1 -- 172.21.15.76:0/2743176119 >> v1:172.21.15.116:6789/0 conn(0x7f1e2406df70 legacy=0x7f1e241634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a152640 1 -- 172.21.15.76:0/2743176119 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2406ea30 con 0x7f1e241630c0 2024-09-19T04:13:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.118+0000 7f1e2a953640 1 --2- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e240b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2a152640 1 --2- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e241630c0 0x7f1e2406f510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009870 tx=0x7f1e2001c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3a344b9d95f7f29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e29150640 1 -- 172.21.15.76:0/2743176119 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e200b2070 con 0x7f1e241630c0 2024-09-19T04:13:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e29150640 1 -- 172.21.15.76:0/2743176119 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e200a9070 con 0x7f1e241630c0 2024-09-19T04:13:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e29150640 1 -- 172.21.15.76:0/2743176119 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e200ad880 con 0x7f1e241630c0 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e241630c0 msgr2=0x7f1e2406f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e241630c0 0x7f1e2406f510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1e20009870 tx=0x7f1e2001c560 comp rx=0 tx=0).stop 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/2743176119 shutdown_connections 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e240b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/2743176119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e241630c0 0x7f1e2406f510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/2743176119 >> 172.21.15.76:0/2743176119 conn(0x7f1e240b1f90 msgr2=0x7f1e240b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/2743176119 shutdown_connections 2024-09-19T04:13:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/2743176119 wait complete. 2024-09-19T04:13:41.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 Processor -- start 2024-09-19T04:13:41.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- start start 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e2406df70 0x7f1e240cf340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e2414c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e241630c0 0x7f1e240cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2416f5f0 con 0x7f1e241630c0 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2406fc60 con 0x7f1e2406df70 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2c3b6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1e2416f770 con 0x7f1e240b3290 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2a953640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e241630c0 0x7f1e240cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2a953640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e241630c0 0x7f1e240cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40594/0 (socket says 172.21.15.76:40594) 2024-09-19T04:13:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.122+0000 7f1e2a953640 1 -- 172.21.15.76:0/3701199172 learned_addr learned my addr 172.21.15.76:0/3701199172 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a152640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e2406df70 0x7f1e240cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e29951640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e2414c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a953640 1 -- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 msgr2=0x7f1e2414c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a953640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e2414c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a953640 1 -- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e2406df70 msgr2=0x7f1e240cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a953640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e2406df70 0x7f1e240cf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a953640 1 -- 172.21.15.76:0/3701199172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e240cdee0 con 0x7f1e241630c0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a953640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e241630c0 0x7f1e240cd1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009f00 tx=0x7f1e1c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e1affd640 1 -- 172.21.15.76:0/3701199172 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c018030 con 0x7f1e241630c0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e1affd640 1 -- 172.21.15.76:0/3701199172 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e1c00ce50 con 0x7f1e241630c0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e1affd640 1 -- 172.21.15.76:0/3701199172 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c003bc0 con 0x7f1e241630c0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e240ccfe0 con 0x7f1e241630c0 2024-09-19T04:13:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e240cca20 con 0x7f1e241630c0 2024-09-19T04:13:41.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e240b3720 con 0x7f1e241630c0 2024-09-19T04:13:41.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e1affd640 1 -- 172.21.15.76:0/3701199172 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1e1c01b070 con 0x7f1e241630c0 2024-09-19T04:13:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e1affd640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e080678c0 0x7f1e08069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a152640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e080678c0 0x7f1e08069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e1affd640 1 -- 172.21.15.76:0/3701199172 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1e1c086a00 con 0x7f1e241630c0 2024-09-19T04:13:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.126+0000 7f1e2a152640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e080678c0 0x7f1e08069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e200af000 tx=0x7f1e200a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.130+0000 7f1e1affd640 1 -- 172.21.15.76:0/3701199172 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1e1c022070 con 0x7f1e241630c0 2024-09-19T04:13:41.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.278+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e240af1a0 con 0x7f1e241630c0 2024-09-19T04:13:41.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e1affd640 1 -- 172.21.15.76:0/3701199172 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f1e240cdee0 con 0x7f1e241630c0 2024-09-19T04:13:41.285 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:41.285 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:41.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e080678c0 msgr2=0x7f1e08069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e080678c0 0x7f1e08069d80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1e200af000 tx=0x7f1e200a7000 comp rx=0 tx=0).stop 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e241630c0 msgr2=0x7f1e240cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e241630c0 0x7f1e240cd1f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c009f00 tx=0x7f1e1c00c490 comp rx=0 tx=0).stop 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 shutdown_connections 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e080678c0 0x7f1e08069d80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1e241630c0 0x7f1e240cd1f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1e240b3290 0x7f1e2414c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 --2- 172.21.15.76:0/3701199172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e2406df70 0x7f1e240cf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 >> 172.21.15.76:0/3701199172 conn(0x7f1e240b1f90 msgr2=0x7f1e240ade10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 shutdown_connections 2024-09-19T04:13:41.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.282+0000 7f1e2c3b6640 1 -- 172.21.15.76:0/3701199172 wait complete. 2024-09-19T04:13:41.303 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-19T04:13:41.303 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-19T04:13:41.304 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-19T04:13:41.304 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-19T04:13:41.305 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-19T04:13:41.305 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-19T04:13:41.305 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-19T04:13:41.306 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-19T04:13:41.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.474+0000 7fef72c04640 1 Processor -- start 2024-09-19T04:13:41.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.474+0000 7fef72c04640 1 -- start start 2024-09-19T04:13:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef72c04640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c15a200 0x7fef6c15a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef72c04640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fef6c072db0 0x7fef6c072340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef72c04640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef6c15aec0 con 0x7fef6c072880 2024-09-19T04:13:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef72c04640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fef6c15b1a0 con 0x7fef6c072db0 2024-09-19T04:13:41.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef72c04640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c15b480 con 0x7fef6c15a200 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef709a0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c15a200 0x7fef6c15a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef6bfff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fef6c072db0 0x7fef6c072340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef709a0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c15a200 0x7fef6c15a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38138/0 (socket says 172.21.15.76:38138) 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef709a0640 1 -- 172.21.15.76:0/1145721043 learned_addr learned my addr 172.21.15.76:0/1145721043 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef6bfff640 1 -- 172.21.15.76:0/1145721043 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c15a200 msgr2=0x7fef6c15a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef6bfff640 1 --2- 172.21.15.76:0/1145721043 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c15a200 0x7fef6c15a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef6bfff640 1 -- 172.21.15.76:0/1145721043 >> v1:172.21.15.116:6789/0 conn(0x7fef6c072880 legacy=0x7fef6c0b2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fef6bfff640 1 -- 172.21.15.76:0/1145721043 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c071e20 con 0x7fef6c072db0 2024-09-19T04:13:41.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fe153e52640 1 Processor -- start 2024-09-19T04:13:41.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7fe153e52640 1 -- start start 2024-09-19T04:13:41.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7f4dc44bc640 1 Processor -- start 2024-09-19T04:13:41.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35e2d6640 1 Processor -- start 2024-09-19T04:13:41.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc686787640 1 Processor -- start 2024-09-19T04:13:41.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.478+0000 7f4dc44bc640 1 -- start start 2024-09-19T04:13:41.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc686787640 1 -- start start 2024-09-19T04:13:41.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35e2d6640 1 -- start start 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe153e52640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c161a50 0x7fe14c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe153e52640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe14c0d1e10 0x7fe14c0d41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe153e52640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe14c0d2560 con 0x7fe14c0d4710 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe153e52640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe14c0d3170 con 0x7fe14c0d1e10 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe153e52640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe14c0d4b40 con 0x7fe14c161a50 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe151bee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c161a50 0x7fe14c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe151bee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c161a50 0x7fe14c0b4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38154/0 (socket says 172.21.15.76:38154) 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe151bee640 1 -- 172.21.15.76:0/3551966821 learned_addr learned my addr 172.21.15.76:0/3551966821 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031c18640 1 Processor -- start 2024-09-19T04:13:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe1513ed640 1 --2- 172.21.15.76:0/3551966821 >> v2:172.21.15.76:3300/0 conn(0x7fe14c0d1e10 0x7fe14c0d41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe151bee640 1 -- 172.21.15.76:0/3551966821 >> v2:172.21.15.76:3300/0 conn(0x7fe14c0d1e10 msgr2=0x7fe14c0d41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe151bee640 1 --2- 172.21.15.76:0/3551966821 >> v2:172.21.15.76:3300/0 conn(0x7fe14c0d1e10 0x7fe14c0d41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe151bee640 1 -- 172.21.15.76:0/3551966821 >> v1:172.21.15.116:6789/0 conn(0x7fe14c0d4710 legacy=0x7fe14c0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef6bfff640 1 --2- 172.21.15.76:0/1145721043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c072db0 0x7fef6c072340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef600097c0 tx=0x7fef6001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edfe96f792dc104d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031c18640 1 -- start start 2024-09-19T04:13:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc44bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc06f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc44bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4dbc06df70 0x7f4dbc1634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc44bc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4dbc0b3290 0x7f4dbc0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc44bc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4dbc06e470 con 0x7f4dbc06df70 2024-09-19T04:13:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc44bc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4dbc06e750 con 0x7f4dbc0b3290 2024-09-19T04:13:41.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc44bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc06ea30 con 0x7f4dbc1630c0 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fe151bee640 1 -- 172.21.15.76:0/3551966821 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c0d2840 con 0x7fe14c161a50 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35e2d6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb350007080 0x7fb350039fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35e2d6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb350054ed0 con 0x7fb350036030 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35e2d6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb350055450 con 0x7fb350033b60 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35e2d6640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb350055170 con 0x7fb350007080 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35d2d4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb350036030 0x7fb350036410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34320/0 (socket says 172.21.15.76:34320) 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35d2d4640 1 -- 172.21.15.76:0/2490086688 learned_addr learned my addr 172.21.15.76:0/2490086688 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:41.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3698964736 0 0) 0x7fb350054ed0 con 0x7fb350036030 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb344003530 con 0x7fb350036030 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1026351896 0 0) 0x7fb344003530 con 0x7fb350036030 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 >> v2:172.21.15.116:3300/0 conn(0x7fb350007080 msgr2=0x7fb350039fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 --2- 172.21.15.76:0/2490086688 >> v2:172.21.15.116:3300/0 conn(0x7fb350007080 0x7fb350039fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 >> v1:172.21.15.116:6789/0 conn(0x7fb350033b60 legacy=0x7fb35004eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb350009cd0 con 0x7fb350036030 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb3480016d0 con 0x7fb350036030 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 >> v1:172.21.15.76:6789/0 conn(0x7fb350036030 legacy=0x7fb350036410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3440039c0 0x7fb344003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb344004810 0x7fb344004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3440053d0 0x7fb34400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb350055170 con 0x7fb3440053d0 2024-09-19T04:13:41.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb34400c2f0 con 0x7fb3440039c0 2024-09-19T04:13:41.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef6b7fe640 1 -- 172.21.15.76:0/1145721043 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef600b1070 con 0x7fef6c072db0 2024-09-19T04:13:41.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef6b7fe640 1 -- 172.21.15.76:0/1145721043 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef600ac410 con 0x7fef6c072db0 2024-09-19T04:13:41.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef72c04640 1 -- 172.21.15.76:0/1145721043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c072db0 msgr2=0x7fef6c072340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef72c04640 1 --2- 172.21.15.76:0/1145721043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c072db0 0x7fef6c072340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef600097c0 tx=0x7fef6001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031c18640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa02c141230 0x7fa02c0b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031c18640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa02c0b5460 con 0x7fa02c0b2ef0 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031c18640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa02c0b5740 con 0x7fa02c140e50 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031c18640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa02c0b5fe0 con 0x7fa02c141230 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031417640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa02c0b2ef0 0x7fa02c0b3310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34326/0 (socket says 172.21.15.76:34326) 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa031417640 1 -- 172.21.15.76:0/3700835926 learned_addr learned my addr 172.21.15.76:0/3700835926 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc686787640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6800b50c0 0x7fc6800b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc686787640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6800ae780 con 0x7fc68006d3e0 2024-09-19T04:13:41.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc686787640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc680158ce0 con 0x7fc680158900 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc686787640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc6800b5730 con 0x7fc6800b50c0 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc684d24640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6800b50c0 0x7fc6800b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc684d24640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6800b50c0 0x7fc6800b5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56096/0 (socket says 172.21.15.76:56096) 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc684d24640 1 -- 172.21.15.76:0/1182290912 learned_addr learned my addr 172.21.15.76:0/1182290912 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc684d24640 1 -- 172.21.15.76:0/1182290912 >> v1:172.21.15.116:6789/0 conn(0x7fc680158900 legacy=0x7fc68006ccd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc684d24640 1 -- 172.21.15.76:0/1182290912 >> v1:172.21.15.76:6789/0 conn(0x7fc68006d3e0 legacy=0x7fc6800b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fc684d24640 1 -- 172.21.15.76:0/1182290912 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68015e960 con 0x7fc6800b50c0 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4dbc06df70 0x7f4dbc1634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4dbc06df70 0x7f4dbc1634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40600/0 (socket says 172.21.15.76:40600) 2024-09-19T04:13:41.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 -- 172.21.15.76:0/2818173568 learned_addr learned my addr 172.21.15.76:0/2818173568 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc2a59640 1 --2- 172.21.15.76:0/2818173568 >> v2:172.21.15.116:3300/0 conn(0x7f4dbc0b3290 0x7f4dbc0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc2258640 1 --2- 172.21.15.76:0/2818173568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc06f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb34400c4f0 con 0x7fb344004810 2024-09-19T04:13:41.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa02bfff640 1 --2- 172.21.15.76:0/3700835926 >> v2:172.21.15.116:3300/0 conn(0x7fa02c141230 0x7fa02c0b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef72c04640 1 -- 172.21.15.76:0/1145721043 shutdown_connections 2024-09-19T04:13:41.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef72c04640 1 --2- 172.21.15.76:0/1145721043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c072db0 0x7fef6c072340 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fef600097c0 tx=0x7fef6001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef72c04640 1 --2- 172.21.15.76:0/1145721043 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c15a200 0x7fef6c15a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fef72c04640 1 -- 172.21.15.76:0/1145721043 >> 172.21.15.76:0/1145721043 conn(0x7fef6c0b1cd0 msgr2=0x7fef6c0b20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35dad5640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3440053d0 0x7fb34400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35cad3640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb344004810 0x7fb344004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35d2d4640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3440039c0 0x7fb344003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 -- 172.21.15.76:0/2818173568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 msgr2=0x7f4dbc06f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 --2- 172.21.15.76:0/2818173568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc06f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 -- 172.21.15.76:0/2818173568 >> v2:172.21.15.116:3300/0 conn(0x7f4dbc0b3290 msgr2=0x7f4dbc0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 --2- 172.21.15.76:0/2818173568 >> v2:172.21.15.116:3300/0 conn(0x7f4dbc0b3290 0x7f4dbc0b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc1a57640 1 -- 172.21.15.76:0/2818173568 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc06ed10 con 0x7f4dbc06df70 2024-09-19T04:13:41.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f4dc2258640 1 --2- 172.21.15.76:0/2818173568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc06f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f62ae9c0640 1 Processor -- start 2024-09-19T04:13:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa02bfff640 1 -- 172.21.15.76:0/3700835926 >> v1:172.21.15.116:6789/0 conn(0x7fa02c140e50 legacy=0x7fa02c162600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:13:41.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35dad5640 1 -- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb344004810 msgr2=0x7fb344004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35dad5640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb344004810 0x7fb344004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35dad5640 1 -- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3440039c0 msgr2=0x7fb344003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35dad5640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3440039c0 0x7fb344003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fb35dad5640 1 -- 172.21.15.76:0/2490086688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb350009cd0 con 0x7fb3440053d0 2024-09-19T04:13:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa02bfff640 1 -- 172.21.15.76:0/3700835926 >> v1:172.21.15.76:6789/0 conn(0x7fa02c0b2ef0 legacy=0x7fa02c0b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7f62ae9c0640 1 -- start start 2024-09-19T04:13:41.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.482+0000 7fa02bfff640 1 -- 172.21.15.76:0/3700835926 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c0b5a20 con 0x7fa02c141230 2024-09-19T04:13:41.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db63ff640 1 Processor -- start 2024-09-19T04:13:41.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db63ff640 1 -- start start 2024-09-19T04:13:41.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 -- 172.21.15.76:0/1145721043 shutdown_connections 2024-09-19T04:13:41.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae9c0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f62a0037170 0x7f62a0036480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae9c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f62a0004b10 0x7f62a00369c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae9c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62a0048030 con 0x7f62a00ea6c0 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae9c0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f62a00338d0 con 0x7f62a0004b10 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae9c0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f62a0047a70 con 0x7f62a0037170 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ad9be640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f62a00ea6c0 0x7f62a0006b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34332/0 (socket says 172.21.15.76:34332) 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ad9be640 1 -- 172.21.15.76:0/455812635 learned_addr learned my addr 172.21.15.76:0/455812635 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae1bf640 1 --2- 172.21.15.76:0/455812635 >> v2:172.21.15.76:3300/0 conn(0x7f62a0004b10 0x7f62a00369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ad1bd640 1 --2- 172.21.15.76:0/455812635 >> v2:172.21.15.116:3300/0 conn(0x7f62a0037170 0x7f62a0036480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db63ff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7db015c0e0 0x7f7db00af910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db63ff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7db0072450 0x7f7db0073300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc684d24640 1 --2- 172.21.15.76:0/1182290912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc6800b5b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc678009e50 tx=0x7fc67801c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4943a7e8cca6a7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc1a57640 1 --2- 172.21.15.76:0/2818173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dbc06df70 0x7f4dbc1634a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc15d2c0 tx=0x7f4dac01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b03370454178ef0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db499c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7db0073840 0x7f7db015fdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34348/0 (socket says 172.21.15.76:34348) 2024-09-19T04:13:41.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db499c640 1 -- 172.21.15.76:0/1031370197 learned_addr learned my addr 172.21.15.76:0/1031370197 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:41.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db00cfa40 con 0x7f7db0073840 2024-09-19T04:13:41.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc67effd640 1 -- 172.21.15.76:0/1182290912 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a5510 con 0x7fc6800b50c0 2024-09-19T04:13:41.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7db0073c90 con 0x7f7db015c0e0 2024-09-19T04:13:41.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7db00b2a70 con 0x7f7db0072450 2024-09-19T04:13:41.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc1256640 1 -- 172.21.15.76:0/2818173568 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0b0070 con 0x7f4dbc06df70 2024-09-19T04:13:41.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc1256640 1 -- 172.21.15.76:0/2818173568 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4dac0ab600 con 0x7f4dbc06df70 2024-09-19T04:13:41.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc1256640 1 -- 172.21.15.76:0/2818173568 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac0ab9d0 con 0x7f4dbc06df70 2024-09-19T04:13:41.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 -- 172.21.15.76:0/1145721043 wait complete. 2024-09-19T04:13:41.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 Processor -- start 2024-09-19T04:13:41.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daf7fe640 1 --2- 172.21.15.76:0/1031370197 >> v2:172.21.15.116:3300/0 conn(0x7f7db0072450 0x7f7db0073300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc67effd640 1 -- 172.21.15.76:0/1182290912 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6780a56b0 con 0x7fc6800b50c0 2024-09-19T04:13:41.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc67effd640 1 -- 172.21.15.76:0/1182290912 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780a5a80 con 0x7fc6800b50c0 2024-09-19T04:13:41.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daeffd640 1 -- 172.21.15.76:0/1031370197 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1198266273 0 0) 0x7f7db00cfa40 con 0x7f7db0073840 2024-09-19T04:13:41.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fb35dad5640 1 -- 172.21.15.76:0/2490086688 reap_dead start 2024-09-19T04:13:41.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae1bf640 1 -- 172.21.15.76:0/455812635 >> v2:172.21.15.116:3300/0 conn(0x7f62a0037170 msgr2=0x7f62a0036480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae1bf640 1 --2- 172.21.15.76:0/455812635 >> v2:172.21.15.116:3300/0 conn(0x7f62a0037170 0x7f62a0036480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae1bf640 1 -- 172.21.15.76:0/455812635 >> v1:172.21.15.76:6789/0 conn(0x7f62a00ea6c0 legacy=0x7f62a0006b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f62ae1bf640 1 -- 172.21.15.76:0/455812635 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a0047d50 con 0x7f62a0004b10 2024-09-19T04:13:41.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daeffd640 1 -- 172.21.15.76:0/1031370197 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d98003530 con 0x7f7db0073840 2024-09-19T04:13:41.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc686787640 1 -- 172.21.15.76:0/1182290912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 msgr2=0x7fc6800b5b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc686787640 1 --2- 172.21.15.76:0/1182290912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc6800b5b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc678009e50 tx=0x7fc67801c740 comp rx=0 tx=0).stop 2024-09-19T04:13:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daffff640 1 --2- 172.21.15.76:0/1031370197 >> v2:172.21.15.76:3300/0 conn(0x7f7db015c0e0 0x7f7db00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc686787640 1 -- 172.21.15.76:0/1182290912 shutdown_connections 2024-09-19T04:13:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc686787640 1 --2- 172.21.15.76:0/1182290912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc6800b5b70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc686787640 1 -- 172.21.15.76:0/1182290912 >> 172.21.15.76:0/1182290912 conn(0x7fc680072670 msgr2=0x7fc680072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc686787640 1 -- 172.21.15.76:0/1182290912 shutdown_connections 2024-09-19T04:13:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 -- 172.21.15.76:0/2818173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dbc06df70 msgr2=0x7f4dbc1634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 --2- 172.21.15.76:0/2818173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dbc06df70 0x7f4dbc1634a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc15d2c0 tx=0x7f4dac01c520 comp rx=0 tx=0).stop 2024-09-19T04:13:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daf7fe640 1 -- 172.21.15.76:0/1031370197 >> v2:172.21.15.76:3300/0 conn(0x7f7db015c0e0 msgr2=0x7f7db00af910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daf7fe640 1 --2- 172.21.15.76:0/1031370197 >> v2:172.21.15.76:3300/0 conn(0x7f7db015c0e0 0x7f7db00af910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daf7fe640 1 -- 172.21.15.76:0/1031370197 >> v1:172.21.15.76:6789/0 conn(0x7f7db0073840 legacy=0x7f7db015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daf7fe640 1 -- 172.21.15.76:0/1031370197 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db00b2740 con 0x7f7db0072450 2024-09-19T04:13:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fa02bfff640 1 --2- 172.21.15.76:0/3700835926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c141230 0x7fa02c0b4df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0b6160 tx=0x7fa02001c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9298027f293a179f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 -- 172.21.15.76:0/2818173568 shutdown_connections 2024-09-19T04:13:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 --2- 172.21.15.76:0/2818173568 >> v2:172.21.15.116:3300/0 conn(0x7f4dbc0b3290 0x7f4dbc0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 --2- 172.21.15.76:0/2818173568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dbc06df70 0x7f4dbc1634a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 --2- 172.21.15.76:0/2818173568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc06f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 -- 172.21.15.76:0/2818173568 >> 172.21.15.76:0/2818173568 conn(0x7f4dbc0b1f90 msgr2=0x7f4dbc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f7daffff640 1 --2- 172.21.15.76:0/1031370197 >> v2:172.21.15.76:3300/0 conn(0x7f7db015c0e0 0x7f7db00af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fb35dad5640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3440053d0 0x7fb34400ba10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb358051960 tx=0x7fb3580983b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22487a837fa55afa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 -- start start 2024-09-19T04:13:41.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fc686787640 1 -- 172.21.15.76:0/1182290912 wait complete. 2024-09-19T04:13:41.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c072880 0x7fef6c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef6c15a200 0x7fef6c164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c1648b0 0x7fef6c14cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c167040 con 0x7fef6c15a200 2024-09-19T04:13:41.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fef6c166ec0 con 0x7fef6c1648b0 2024-09-19T04:13:41.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fef72c04640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fef6c1671c0 con 0x7fef6c072880 2024-09-19T04:13:41.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 -- 172.21.15.76:0/2818173568 shutdown_connections 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7fa02b7fe640 1 -- 172.21.15.76:0/3700835926 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200b0070 con 0x7fa02c141230 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358050f50 con 0x7fb3440053d0 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb35806de50 con 0x7fb3440053d0 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb34ffff640 1 -- 172.21.15.76:0/2490086688 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb358127920 con 0x7fb3440053d0 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef711a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c1648b0 0x7fef6c14cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef711a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c1648b0 0x7fef6c14cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56158/0 (socket says 172.21.15.76:56158) 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef711a1640 1 -- 172.21.15.76:0/2857085003 learned_addr learned my addr 172.21.15.76:0/2857085003 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef709a0640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c072880 0x7fef6c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef6bfff640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef6c15a200 0x7fef6c164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 Processor -- start 2024-09-19T04:13:41.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef6bfff640 1 -- 172.21.15.76:0/2857085003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c072880 msgr2=0x7fef6c1590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef6bfff640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef6c072880 0x7fef6c1590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef6bfff640 1 -- 172.21.15.76:0/2857085003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c1648b0 msgr2=0x7fef6c14cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef6bfff640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef6c1648b0 0x7fef6c14cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef6bfff640 1 -- 172.21.15.76:0/2857085003 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef6c14b160 con 0x7fef6c15a200 2024-09-19T04:13:41.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fe151bee640 1 --2- 172.21.15.76:0/3551966821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c161a50 0x7fe14c0b4e70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe144009e50 tx=0x7fe14401c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9671271a63ba4305 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef6bfff640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef6c15a200 0x7fef6c164370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef6c06ebd0 tx=0x7fef60019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa02b7fe640 1 -- 172.21.15.76:0/3700835926 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0200ab600 con 0x7fa02c141230 2024-09-19T04:13:41.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa02b7fe640 1 -- 172.21.15.76:0/3700835926 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0200ab9d0 con 0x7fa02c141230 2024-09-19T04:13:41.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 -- 172.21.15.76:0/3700835926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c141230 msgr2=0x7fa02c0b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 --2- 172.21.15.76:0/3700835926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c141230 0x7fa02c0b4df0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa02c0b6160 tx=0x7fa02001c3d0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 -- 172.21.15.76:0/2490086688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3440053d0 msgr2=0x7fb34400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3440053d0 0x7fb34400ba10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb358051960 tx=0x7fb3580983b0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 -- 172.21.15.76:0/3700835926 shutdown_connections 2024-09-19T04:13:41.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 --2- 172.21.15.76:0/3700835926 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c141230 0x7fa02c0b4df0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 -- 172.21.15.76:0/3700835926 >> 172.21.15.76:0/3700835926 conn(0x7fa02c0b1bf0 msgr2=0x7fa02c0b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 -- 172.21.15.76:0/2490086688 shutdown_connections 2024-09-19T04:13:41.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 --2- 172.21.15.76:0/2490086688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3440053d0 0x7fb34400ba10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 -- 172.21.15.76:0/2490086688 >> 172.21.15.76:0/2490086688 conn(0x7fb35001d940 msgr2=0x7fb3500545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 -- 172.21.15.76:0/2490086688 shutdown_connections 2024-09-19T04:13:41.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 -- 172.21.15.76:0/2490086688 wait complete. 2024-09-19T04:13:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.486+0000 7f4dc44bc640 1 -- 172.21.15.76:0/2818173568 wait complete. 2024-09-19T04:13:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f4dc44bc640 1 Processor -- start 2024-09-19T04:13:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fe150bec640 1 -- 172.21.15.76:0/3551966821 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440a5830 con 0x7fe14c161a50 2024-09-19T04:13:41.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 -- 172.21.15.76:0/3700835926 shutdown_connections 2024-09-19T04:13:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 -- start start 2024-09-19T04:13:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc68006d3e0 0x7fc680151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc680158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc680158900 0x7fc680154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc68016bfc0 con 0x7fc68006d3e0 2024-09-19T04:13:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc68015f720 con 0x7fc6800b50c0 2024-09-19T04:13:41.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc686787640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc68016c140 con 0x7fc680158900 2024-09-19T04:13:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fe150bec640 1 -- 172.21.15.76:0/3551966821 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1440a59d0 con 0x7fe14c161a50 2024-09-19T04:13:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fe150bec640 1 -- 172.21.15.76:0/3551966821 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440a5da0 con 0x7fe14c161a50 2024-09-19T04:13:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae1bf640 1 --2- 172.21.15.76:0/455812635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0004b10 0x7f62a00369c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f62a8074530 tx=0x7f62a8098290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5826c9cb554841 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc680158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc680158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56170/0 (socket says 172.21.15.76:56170) 2024-09-19T04:13:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 -- 172.21.15.76:0/1011436518 learned_addr learned my addr 172.21.15.76:0/1011436518 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc684d24640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc680158900 0x7fc680154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ac9bc640 1 -- 172.21.15.76:0/455812635 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8127600 con 0x7f62a0004b10 2024-09-19T04:13:41.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ac9bc640 1 -- 172.21.15.76:0/455812635 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a806de50 con 0x7f62a0004b10 2024-09-19T04:13:41.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ac9bc640 1 -- 172.21.15.76:0/455812635 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8127aa0 con 0x7f62a0004b10 2024-09-19T04:13:41.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f7daf7fe640 1 --2- 172.21.15.76:0/1031370197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0072450 0x7f7db0073300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7db009e550 tx=0x7f7da401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e79fd98640883275 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 -- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc680158900 msgr2=0x7fc680154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc680158900 0x7fc680154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 -- 172.21.15.76:0/1011436518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc68006d3e0 msgr2=0x7fc680151430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc68006d3e0 0x7fc680151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 -- 172.21.15.76:0/1011436518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc680159450 con 0x7fc6800b50c0 2024-09-19T04:13:41.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f7daeffd640 1 -- 172.21.15.76:0/1031370197 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40b1070 con 0x7f7db0072450 2024-09-19T04:13:41.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67f7fe640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc680158f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b410 tx=0x7fc67000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef600b1040 con 0x7fef6c15a200 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef60003620 con 0x7fef6c15a200 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef600037d0 con 0x7fef6c15a200 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef6c14b440 con 0x7fef6c15a200 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef6c14ba00 con 0x7fef6c15a200 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f7daeffd640 1 -- 172.21.15.76:0/1031370197 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da40ac620 con 0x7f7db0072450 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f7daeffd640 1 -- 172.21.15.76:0/1031370197 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da40ac9d0 con 0x7f7db0072450 2024-09-19T04:13:41.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fe153e52640 1 -- 172.21.15.76:0/3551966821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c161a50 msgr2=0x7fe14c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fe153e52640 1 --2- 172.21.15.76:0/3551966821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c161a50 0x7fe14c0b4e70 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe144009e50 tx=0x7fe14401c990 comp rx=0 tx=0).stop 2024-09-19T04:13:41.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 -- 172.21.15.76:0/455812635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0004b10 msgr2=0x7f62a00369c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 --2- 172.21.15.76:0/455812635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0004b10 0x7f62a00369c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f62a8074530 tx=0x7f62a8098290 comp rx=0 tx=0).stop 2024-09-19T04:13:41.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f4dc44bc640 1 -- start start 2024-09-19T04:13:41.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f4dc44bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4dbc06df70 0x7f4dbc0991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 -- 172.21.15.76:0/3700835926 wait complete. 2024-09-19T04:13:41.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 Processor -- start 2024-09-19T04:13:41.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 -- 172.21.15.76:0/455812635 shutdown_connections 2024-09-19T04:13:41.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 --2- 172.21.15.76:0/455812635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0004b10 0x7f62a00369c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 --2- 172.21.15.76:0/455812635 >> v2:172.21.15.116:3300/0 conn(0x7f62a0037170 0x7f62a0036480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 -- 172.21.15.76:0/455812635 >> 172.21.15.76:0/455812635 conn(0x7f62a001dd90 msgr2=0x7f62a0054a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 -- 172.21.15.76:0/455812635 shutdown_connections 2024-09-19T04:13:41.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670015020 con 0x7fc6800b50c0 2024-09-19T04:13:41.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7f62ae9c0640 1 -- 172.21.15.76:0/455812635 wait complete. 2024-09-19T04:13:41.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef6c072340 con 0x7fef6c15a200 2024-09-19T04:13:41.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fb35e2d6640 1 Processor -- start 2024-09-19T04:13:41.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0072450 msgr2=0x7f7db0073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f7db63ff640 1 --2- 172.21.15.76:0/1031370197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0072450 0x7f7db0073300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7db009e550 tx=0x7f7da401c3c0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35e2d6640 1 -- start start 2024-09-19T04:13:41.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc44bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dbc0b3290 0x7f4dbc1567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc44bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc44bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc06fc60 con 0x7f4dbc0b3290 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc44bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4dbc166250 con 0x7f4dbc06df70 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc44bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4dbc1530a0 con 0x7f4dbc1630c0 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2258640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4dbc06df70 0x7f4dbc0991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2258640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4dbc06df70 0x7f4dbc0991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56174/0 (socket says 172.21.15.76:56174) 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2258640 1 -- 172.21.15.76:0/583453860 learned_addr learned my addr 172.21.15.76:0/583453860 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2a59640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fc686787640 1 -- 172.21.15.76:0/1011436518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6800b0060 con 0x7fc6800b50c0 2024-09-19T04:13:41.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fc686787640 1 -- 172.21.15.76:0/1011436518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6800b0620 con 0x7fc6800b50c0 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc670011830 con 0x7fc6800b50c0 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc670011c00 con 0x7fc6800b50c0 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35e2d6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3440053d0 0x7fb350032790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35e2d6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb350004a00 0x7fb350032cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35e2d6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb350033f40 0x7fb350189d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35e2d6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb350039fc0 con 0x7fb350004a00 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35e2d6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb3500e63b0 con 0x7fb350033f40 2024-09-19T04:13:41.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35e2d6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb35003a140 con 0x7fb3440053d0 2024-09-19T04:13:41.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fc686787640 1 -- 172.21.15.76:0/1011436518 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc660001040 con 0x7fc6800b50c0 2024-09-19T04:13:41.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.490+0000 7fa031c18640 1 -- start start 2024-09-19T04:13:41.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa02c0b2ef0 0x7fa02c13d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c140e50 0x7fa02c157fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa02c141230 0x7fa02c13fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fe153e52640 1 -- 172.21.15.76:0/3551966821 shutdown_connections 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fe153e52640 1 --2- 172.21.15.76:0/3551966821 >> v2:172.21.15.76:3300/0 conn(0x7fe14c0d1e10 0x7fe14c0d41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fe153e52640 1 --2- 172.21.15.76:0/3551966821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c161a50 0x7fe14c0b4e70 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fe153e52640 1 -- 172.21.15.76:0/3551966821 >> 172.21.15.76:0/3551966821 conn(0x7fe14c08bf90 msgr2=0x7fe14c09d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa030c16640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa02c0b2ef0 0x7fa02c13d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa030c16640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa02c0b2ef0 0x7fa02c13d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38226/0 (socket says 172.21.15.76:38226) 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa030c16640 1 -- 172.21.15.76:0/1455370361 learned_addr learned my addr 172.21.15.76:0/1455370361 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c0cfaa0 con 0x7fa02c141230 2024-09-19T04:13:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c0cf920 con 0x7fa02c140e50 2024-09-19T04:13:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa02c0cfc20 con 0x7fa02c0b2ef0 2024-09-19T04:13:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 Processor -- start 2024-09-19T04:13:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35d2d4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3440053d0 0x7fb350032790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35d2d4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3440053d0 0x7fb350032790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38240/0 (socket says 172.21.15.76:38240) 2024-09-19T04:13:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35d2d4640 1 -- 172.21.15.76:0/3438827144 learned_addr learned my addr 172.21.15.76:0/3438827144 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35cad3640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb350004a00 0x7fb350032cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2a59640 1 -- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4dbc06df70 msgr2=0x7f4dbc0991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2a59640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4dbc06df70 0x7f4dbc0991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2a59640 1 -- 172.21.15.76:0/583453860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dbc0b3290 msgr2=0x7f4dbc1567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2a59640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4dbc0b3290 0x7f4dbc1567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2a59640 1 -- 172.21.15.76:0/583453860 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dbc09a380 con 0x7f4dbc1630c0 2024-09-19T04:13:41.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031417640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa02c141230 0x7fa02c13fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa02bfff640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c140e50 0x7fa02c157fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35cad3640 1 -- 172.21.15.76:0/3438827144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3440053d0 msgr2=0x7fb350032790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35cad3640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3440053d0 0x7fb350032790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35cad3640 1 -- 172.21.15.76:0/3438827144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb350033f40 msgr2=0x7fb350189d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T04:13:41.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35cad3640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb350033f40 0x7fb350189d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35cad3640 1 -- 172.21.15.76:0/3438827144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb350004fc0 con 0x7fb350004a00 2024-09-19T04:13:41.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc2a59640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc156d20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4db400ef00 tx=0x7f4db400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb35cad3640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb350004a00 0x7fb350032cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e440 tx=0x7fb35400e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031417640 1 -- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa02c0b2ef0 msgr2=0x7fa02c13d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031417640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa02c0b2ef0 0x7fa02c13d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031417640 1 -- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c140e50 msgr2=0x7fa02c157fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4018020 con 0x7f4dbc1630c0 2024-09-19T04:13:41.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031417640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02c140e50 0x7fa02c157fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031417640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c13de40 con 0x7fa02c141230 2024-09-19T04:13:41.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031417640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa02c141230 0x7fa02c13fe00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00c3c0 tx=0x7fa01c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fef600bd020 con 0x7fef6c15a200 2024-09-19T04:13:41.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef697fa640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef580679c0 0x7fef58069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fef6011fb30 con 0x7fef6c15a200 2024-09-19T04:13:41.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef697fa640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fef58071140 0x7fef58073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 --> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef58073c50 con 0x7fef58071140 2024-09-19T04:13:41.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fef6011ff20 con 0x7fef6c15a200 2024-09-19T04:13:41.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef711a1640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fef58071140 0x7fef58073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c017070 con 0x7fa02c141230 2024-09-19T04:13:41.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa01c005cf0 con 0x7fa02c141230 2024-09-19T04:13:41.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa01c0072d0 con 0x7fa02c141230 2024-09-19T04:13:41.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02c144e40 con 0x7fa02c141230 2024-09-19T04:13:41.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa02c145400 con 0x7fa02c141230 2024-09-19T04:13:41.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef709a0640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef580679c0 0x7fef58069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc44bc640 1 -- 172.21.15.76:0/583453860 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dbc15ab00 con 0x7f4dbc1630c0 2024-09-19T04:13:41.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4dc44bc640 1 -- 172.21.15.76:0/583453860 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dbc09a920 con 0x7f4dbc1630c0 2024-09-19T04:13:41.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4db4014400 con 0x7f4dbc1630c0 2024-09-19T04:13:41.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4009e60 con 0x7f4dbc1630c0 2024-09-19T04:13:41.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fef711a1640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fef58071140 0x7fef58073580 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fe153e52640 1 -- 172.21.15.76:0/3551966821 shutdown_connections 2024-09-19T04:13:41.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fe153e52640 1 -- 172.21.15.76:0/3551966821 wait complete. 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fe153e52640 1 Processor -- start 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 -- start start 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0004b10 0x7f62a00dd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62a0037170 0x7f62a0050930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62a00ea6c0 0x7f62a00e4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f62a00f3bc0 con 0x7f62a0004b10 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f62a000b680 con 0x7f62a0037170 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae9c0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f62a00f3d40 con 0x7f62a00ea6c0 2024-09-19T04:13:41.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae1bf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62a00ea6c0 0x7f62a00e4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae1bf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62a00ea6c0 0x7f62a00e4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38242/0 (socket says 172.21.15.76:38242) 2024-09-19T04:13:41.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ae1bf640 1 -- 172.21.15.76:0/229507791 learned_addr learned my addr 172.21.15.76:0/229507791 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7f62ad1bd640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62a0037170 0x7f62a0050930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa00c001040 con 0x7fa02c141230 2024-09-19T04:13:41.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354019070 con 0x7fb350004a00 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3540052e0 con 0x7fb350004a00 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.494+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb354003820 con 0x7fb350004a00 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3500052a0 con 0x7fb350004a00 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb350189820 con 0x7fb350004a00 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ad1bd640 1 -- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62a00ea6c0 msgr2=0x7f62a00e4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ad1bd640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62a00ea6c0 0x7f62a00e4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ad1bd640 1 -- 172.21.15.76:0/229507791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0004b10 msgr2=0x7f62a00dd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ad1bd640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62a0004b10 0x7f62a00dd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ad1bd640 1 -- 172.21.15.76:0/229507791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a0051310 con 0x7f62a0037170 2024-09-19T04:13:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 shutdown_connections 2024-09-19T04:13:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 --2- 172.21.15.76:0/1031370197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0072450 0x7f7db0073300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 --2- 172.21.15.76:0/1031370197 >> v2:172.21.15.76:3300/0 conn(0x7f7db015c0e0 0x7f7db00af910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 >> 172.21.15.76:0/1031370197 conn(0x7f7db006c740 msgr2=0x7f7db00b1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 shutdown_connections 2024-09-19T04:13:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc670018020 con 0x7fc6800b50c0 2024-09-19T04:13:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67cff9640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc66c0678c0 0x7fc66c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc6700865b0 con 0x7fc6800b50c0 2024-09-19T04:13:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67cff9640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7fc66c071040 0x7fc66c073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 --> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc670006740 con 0x7fc66c071040 2024-09-19T04:13:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc6700869a0 con 0x7fc6800b50c0 2024-09-19T04:13:41.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f4dc44bc640 1 -- 172.21.15.76:0/583453860 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4da4001040 con 0x7f4dbc1630c0 2024-09-19T04:13:41.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fef709a0640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef580679c0 0x7fef58069e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef640072b0 tx=0x7fef640076c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb350054ed0 con 0x7fb350004a00 2024-09-19T04:13:41.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== osd.2 v2:172.21.15.76:6824/1496429724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef58073c50 con 0x7fef58071140 2024-09-19T04:13:41.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 -- 172.21.15.76:0/1031370197 wait complete. 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ad1bd640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62a0037170 0x7f62a0050930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f629800d5a0 tx=0x7f629800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe153e52640 1 -- start start 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe153e52640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c0d1e10 0x7fe14c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe153e52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe14c0d4710 0x7fe14c0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe153e52640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14c161a50 0x7fe14c0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe153e52640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c169100 con 0x7fe14c0d4710 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe153e52640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe14c168f80 con 0x7fe14c161a50 2024-09-19T04:13:41.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe153e52640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe14c169280 con 0x7fe14c0d1e10 2024-09-19T04:13:41.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c0d1e10 0x7fe14c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c0d1e10 0x7fe14c0cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38244/0 (socket says 172.21.15.76:38244) 2024-09-19T04:13:41.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 -- 172.21.15.76:0/3743955994 learned_addr learned my addr 172.21.15.76:0/3743955994 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67ffff640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc66c0678c0 0x7fc66c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc67ffff640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc66c0678c0 0x7fc66c069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6740096f0 tx=0x7fc674006c50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fc684d24640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7fc66c071040 0x7fc66c073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe1523ef640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14c161a50 0x7fe14c0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa01c01a070 con 0x7fa02c141230 2024-09-19T04:13:41.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 -- 172.21.15.76:0/3743955994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14c161a50 msgr2=0x7fe14c0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe14c161a50 0x7fe14c0b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 -- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe14c0d4710 msgr2=0x7fe14c0b1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe14c0d4710 0x7fe14c0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 -- 172.21.15.76:0/3743955994 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14c1588f0 con 0x7fe14c0d1e10 2024-09-19T04:13:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 Processor -- start 2024-09-19T04:13:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa0297fa640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa018067990 0x7fa018069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa01c085850 con 0x7fa02c141230 2024-09-19T04:13:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa030c16640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa018067990 0x7fa018069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa0297fa640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7fa018071100 0x7fa018073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa01c01fe50 con 0x7fa018071100 2024-09-19T04:13:41.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa01c085c40 con 0x7fa02c141230 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62980125f0 con 0x7f62a0037170 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6298019070 con 0x7f62a0037170 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6298012990 con 0x7f62a0037170 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a00525a0 con 0x7f62a0037170 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a0052de0 con 0x7f62a0037170 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb35401c070 con 0x7fb350004a00 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb34dffb640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb33c067990 0x7fb33c069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb354085ec0 con 0x7fb350004a00 2024-09-19T04:13:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb34dffb640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7fb33c071100 0x7fb33c073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb354009e50 con 0x7fb33c071100 2024-09-19T04:13:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb3540862b0 con 0x7fb350004a00 2024-09-19T04:13:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fb35d2d4640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb33c067990 0x7fb33c069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fb35dad5640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7fb33c071100 0x7fb33c073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7fe151bee640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c0d1e10 0x7fe14c0cb430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe14401a5c0 tx=0x7fe144019730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fa030c16640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa018067990 0x7fa018069e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0240055f0 tx=0x7fa024009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.498+0000 7f7db63ff640 1 -- start start 2024-09-19T04:13:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fb35dad5640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7fb33c071100 0x7fb33c073540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fb35d2d4640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb33c067990 0x7fb33c069e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb348001140 tx=0x7fb348000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440bc020 con 0x7fe14c0d1e10 2024-09-19T04:13:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1440b8670 con 0x7fe14c0d1e10 2024-09-19T04:13:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe14c0c7d70 con 0x7fe14c0d1e10 2024-09-19T04:13:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4db401b020 con 0x7f4dbc1630c0 2024-09-19T04:13:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4db2ffd640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4da00678c0 0x7f4da0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4db4087290 con 0x7f4dbc1630c0 2024-09-19T04:13:41.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7db0072450 0x7f7db0152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0073840 0x7f7db015f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db015c0e0 0x7f7db0154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7db015c640 con 0x7f7db015c0e0 2024-09-19T04:13:41.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7db015c4c0 con 0x7f7db0073840 2024-09-19T04:13:41.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7db00b42a0 con 0x7f7db0072450 2024-09-19T04:13:41.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe14c0c8330 con 0x7fe14c0d1e10 2024-09-19T04:13:41.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4db2ffd640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7f4da0071040 0x7f4da0073480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4da0073b50 con 0x7f4da0071040 2024-09-19T04:13:41.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4db4087680 con 0x7f4dbc1630c0 2024-09-19T04:13:41.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db015c0e0 0x7f7db0154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7daf7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0073840 0x7f7db015f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db015c0e0 0x7f7db0154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40706/0 (socket says 172.21.15.76:40706) 2024-09-19T04:13:41.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 -- 172.21.15.76:0/1492246332 learned_addr learned my addr 172.21.15.76:0/1492246332 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4dc2258640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4da00678c0 0x7f4da0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4dc1a57640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7f4da0071040 0x7f4da0073480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7daffff640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7db0072450 0x7f7db0152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fa02bfff640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7fa018071100 0x7fa018073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6290001040 con 0x7f62a0037170 2024-09-19T04:13:41.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1440b8a20 con 0x7fe14c0d1e10 2024-09-19T04:13:41.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb354009e50 con 0x7fb33c071100 2024-09-19T04:13:41.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 -- 172.21.15.76:0/1492246332 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7db0072450 msgr2=0x7f7db0152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7db0072450 0x7f7db0152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 -- 172.21.15.76:0/1492246332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0073840 msgr2=0x7f7db015f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db0073840 0x7f7db015f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 -- 172.21.15.76:0/1492246332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db0155130 con 0x7f7db015c0e0 2024-09-19T04:13:41.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe134001040 con 0x7fe14c0d1e10 2024-09-19T04:13:41.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fc684d24640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7fc66c071040 0x7fc66c073480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4dc2258640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4da00678c0 0x7f4da0069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4db800bb70 tx=0x7f4db8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db499c640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db015c0e0 0x7f7db0154bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7da800cb10 tx=0x7f7da800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017070 con 0x7f7db015c0e0 2024-09-19T04:13:41.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f4dc1a57640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7f4da0071040 0x7f4da0073480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db01553b0 con 0x7f7db015c0e0 2024-09-19T04:13:41.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da800ac70 con 0x7f7db015c0e0 2024-09-19T04:13:41.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7fa02bfff640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7fa018071100 0x7fa018073540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8012620 con 0x7f7db015c0e0 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db0155970 con 0x7f7db015c0e0 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6298012b30 con 0x7f62a0037170 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f629e7fc640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f628c067990 0x7f628c069e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6298086410 con 0x7f62a0037170 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f629e7fc640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f628c071110 0x7f628c073550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6298013e50 con 0x7f628c071110 2024-09-19T04:13:41.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6298086800 con 0x7f62a0037170 2024-09-19T04:13:41.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7db015f720 con 0x7f7db015c0e0 2024-09-19T04:13:41.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f62ad9be640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f628c067990 0x7f628c069e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.502+0000 7f62ae1bf640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f628c071110 0x7f628c073550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f62ad9be640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f628c067990 0x7f628c069e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62a4007fd0 tx=0x7f62a4007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f62ae1bf640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f628c071110 0x7f628c073550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe1440bf020 con 0x7fe14c0d1e10 2024-09-19T04:13:41.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1427fc640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe1300678f0 0x7fe130069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe144121870 con 0x7fe14c0d1e10 2024-09-19T04:13:41.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1513ed640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe1300678f0 0x7fe130069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== osd.4 v2:172.21.15.116:6800/3265669393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa01c01fe50 con 0x7fa018071100 2024-09-19T04:13:41.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1427fc640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7fe130070ff0 0x7fe130073430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1440b9e80 con 0x7fe130070ff0 2024-09-19T04:13:41.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe144121d00 con 0x7fe14c0d1e10 2024-09-19T04:13:41.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7da801a070 con 0x7f7db015c0e0 2024-09-19T04:13:41.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1523ef640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7fe130070ff0 0x7fe130073430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1513ed640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe1300678f0 0x7fe130069db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0b3410 tx=0x7fe13c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7dacff9640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7d9c0678c0 0x7f7d9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fe1523ef640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7fe130070ff0 0x7fe130073430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7da8085dd0 con 0x7f7db015c0e0 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7daffff640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7d9c0678c0 0x7f7d9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7dacff9640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7f7d9c070f40 0x7f7d9c073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7daf7fe640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7f7d9c070f40 0x7f7d9c073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7daffff640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7d9c0678c0 0x7f7d9c069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7da0006fd0 tx=0x7f7da0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7daf7fe640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7f7d9c070f40 0x7f7d9c073380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 --> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7da8014ec0 con 0x7f7d9c070f40 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== osd.3 v2:172.21.15.76:6816/1627524440 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7da8014ec0 con 0x7f7d9c070f40 2024-09-19T04:13:41.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== osd.5 v2:172.21.15.116:6808/3134608166 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc670006740 con 0x7fc66c071040 2024-09-19T04:13:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.506+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== osd.6 v2:172.21.15.116:6816/2407229160 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4da0073b50 con 0x7f4da0071040 2024-09-19T04:13:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.514+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== osd.1 v2:172.21.15.76:6808/1010487110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe1440b9e80 con 0x7fe130070ff0 2024-09-19T04:13:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.514+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7da8052100 con 0x7f7db015c0e0 2024-09-19T04:13:41.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.518+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== osd.7 v2:172.21.15.116:6824/2361616500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6298013e50 con 0x7f628c071110 2024-09-19T04:13:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 --> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa00c002de0 con 0x7fa018071100 2024-09-19T04:13:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa0297fa640 1 -- 172.21.15.76:0/1455370361 <== osd.4 v2:172.21.15.116:6800/3265669393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa00c002de0 con 0x7fa018071100 2024-09-19T04:13:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 --> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef6c14af10 con 0x7fef58071140 2024-09-19T04:13:41.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb3500dff70 con 0x7fb33c071100 2024-09-19T04:13:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef697fa640 1 -- 172.21.15.76:0/2857085003 <== osd.2 v2:172.21.15.76:6824/1496429724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fef6c14af10 con 0x7fef58071140 2024-09-19T04:13:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4dc44bc640 1 -- 172.21.15.76:0/583453860 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4da4002d60 con 0x7f4da0071040 2024-09-19T04:13:41.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fef58071140 msgr2=0x7fef58073580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fef58071140 0x7fef58073580 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fb34dffb640 1 -- 172.21.15.76:0/3438827144 <== osd.0 v2:172.21.15.76:6800/2890813412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb3500dff70 con 0x7fb33c071100 2024-09-19T04:13:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef580679c0 msgr2=0x7fef58069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef580679c0 0x7fef58069e80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fef640072b0 tx=0x7fef640076c0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef6c15a200 msgr2=0x7fef6c164370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 --2- 172.21.15.76:0/2857085003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef6c15a200 0x7fef6c164370 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fef6c06ebd0 tx=0x7fef60019240 comp rx=0 tx=0).stop 2024-09-19T04:13:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db2ffd640 1 -- 172.21.15.76:0/583453860 <== osd.6 v2:172.21.15.116:6816/2407229160 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4da4002d60 con 0x7f4da0071040 2024-09-19T04:13:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7fa018071100 msgr2=0x7fa018073540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7fa018071100 0x7fa018073540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 -- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7f4da0071040 msgr2=0x7f4da0073480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7f4da0071040 0x7f4da0073480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef711a1640 1 -- 172.21.15.76:0/2857085003 reap_dead start 2024-09-19T04:13:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 -- 172.21.15.76:0/583453860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4da00678c0 msgr2=0x7f4da0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4da00678c0 0x7f4da0069d80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4db800bb70 tx=0x7f4db8008040 comp rx=0 tx=0).stop 2024-09-19T04:13:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa018067990 msgr2=0x7fa018069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa018067990 0x7fa018069e50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0240055f0 tx=0x7fa024009040 comp rx=0 tx=0).stop 2024-09-19T04:13:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 -- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 msgr2=0x7f4dbc156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 --2- 172.21.15.76:0/583453860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4dbc1630c0 0x7f4dbc156d20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4db400ef00 tx=0x7f4db400b480 comp rx=0 tx=0).stop 2024-09-19T04:13:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4dc2a59640 1 -- 172.21.15.76:0/583453860 reap_dead start 2024-09-19T04:13:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 -- 172.21.15.76:0/583453860 shutdown_connections 2024-09-19T04:13:41.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 -- 172.21.15.76:0/583453860 >> 172.21.15.76:0/583453860 conn(0x7f4dbc0b1f90 msgr2=0x7f4dbc165540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 shutdown_connections 2024-09-19T04:13:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 >> 172.21.15.76:0/2857085003 conn(0x7fef6c0b1cd0 msgr2=0x7fef6c1496a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 -- 172.21.15.76:0/583453860 shutdown_connections 2024-09-19T04:13:41.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7f4db0ff9640 1 -- 172.21.15.76:0/583453860 wait complete. 2024-09-19T04:13:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa02c141230 msgr2=0x7fa02c13fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 shutdown_connections 2024-09-19T04:13:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 --2- 172.21.15.76:0/1455370361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa02c141230 0x7fa02c13fe00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00c3c0 tx=0x7fa01c00c880 comp rx=0 tx=0).stop 2024-09-19T04:13:41.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fef72c04640 1 -- 172.21.15.76:0/2857085003 wait complete. 2024-09-19T04:13:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031417640 1 -- 172.21.15.76:0/1455370361 reap_dead start 2024-09-19T04:13:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 shutdown_connections 2024-09-19T04:13:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.530+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 >> 172.21.15.76:0/1455370361 conn(0x7fa02c0b1bf0 msgr2=0x7fa02c160c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 shutdown_connections 2024-09-19T04:13:41.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fa031c18640 1 -- 172.21.15.76:0/1455370361 wait complete. 2024-09-19T04:13:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7fb33c071100 msgr2=0x7fb33c073540 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc686787640 1 -- 172.21.15.76:0/1011436518 --> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc660002de0 con 0x7fc66c071040 2024-09-19T04:13:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7fb33c071100 0x7fb33c073540 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb33c067990 msgr2=0x7fb33c069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb33c067990 0x7fb33c069e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb348001140 tx=0x7fb348000ff0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb350004a00 msgr2=0x7fb350032cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 --2- 172.21.15.76:0/3438827144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb350004a00 0x7fb350032cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb35400e440 tx=0x7fb35400e900 comp rx=0 tx=0).stop 2024-09-19T04:13:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35dad5640 1 -- 172.21.15.76:0/3438827144 reap_dead start 2024-09-19T04:13:41.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc67cff9640 1 -- 172.21.15.76:0/1011436518 <== osd.5 v2:172.21.15.116:6808/3134608166 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc660002de0 con 0x7fc66c071040 2024-09-19T04:13:41.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 shutdown_connections 2024-09-19T04:13:41.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 >> 172.21.15.76:0/3438827144 conn(0x7fb35001d940 msgr2=0x7fb350050da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 shutdown_connections 2024-09-19T04:13:41.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fb35e2d6640 1 -- 172.21.15.76:0/3438827144 wait complete. 2024-09-19T04:13:41.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 -- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7fc66c071040 msgr2=0x7fc66c073480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7fc66c071040 0x7fc66c073480 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 -- 172.21.15.76:0/1011436518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc66c0678c0 msgr2=0x7fc66c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc66c0678c0 0x7fc66c069d80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6740096f0 tx=0x7fc674006c50 comp rx=0 tx=0).stop 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 -- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 msgr2=0x7fc680158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 --2- 172.21.15.76:0/1011436518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6800b50c0 0x7fc680158f10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc67000b410 tx=0x7fc67000b8d0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc684d24640 1 -- 172.21.15.76:0/1011436518 reap_dead start 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 -- 172.21.15.76:0/1011436518 shutdown_connections 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7fc6667fc640 1 -- 172.21.15.76:0/1011436518 >> 172.21.15.76:0/1011436518 conn(0x7fc680072670 msgr2=0x7fc68015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.534+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 --> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7db0073300 con 0x7f7d9c070f40 2024-09-19T04:13:41.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.538+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6290002de0 con 0x7f628c071110 2024-09-19T04:13:41.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.538+0000 7f7dacff9640 1 -- 172.21.15.76:0/1492246332 <== osd.3 v2:172.21.15.76:6816/1627524440 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7db0073300 con 0x7f7d9c070f40 2024-09-19T04:13:41.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.538+0000 7f629e7fc640 1 -- 172.21.15.76:0/229507791 <== osd.7 v2:172.21.15.116:6824/2361616500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6290002de0 con 0x7f628c071110 2024-09-19T04:13:41.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.538+0000 7fc6667fc640 1 -- 172.21.15.76:0/1011436518 shutdown_connections 2024-09-19T04:13:41.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.538+0000 7fc6667fc640 1 -- 172.21.15.76:0/1011436518 wait complete. 2024-09-19T04:13:41.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.538+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f628c071110 msgr2=0x7f628c073550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.538+0000 7f62ae9c0640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f628c071110 0x7f628c073550 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe134002de0 con 0x7fe130070ff0 2024-09-19T04:13:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f628c067990 msgr2=0x7f628c069e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7f62ae9c0640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f628c067990 0x7f628c069e50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f62a4007fd0 tx=0x7f62a4007900 comp rx=0 tx=0).stop 2024-09-19T04:13:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62a0037170 msgr2=0x7f62a0050930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7f62ae9c0640 1 --2- 172.21.15.76:0/229507791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62a0037170 0x7f62a0050930 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f629800d5a0 tx=0x7f629800da60 comp rx=0 tx=0).stop 2024-09-19T04:13:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7fe1427fc640 1 -- 172.21.15.76:0/3743955994 <== osd.1 v2:172.21.15.76:6808/1010487110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe134002de0 con 0x7fe130070ff0 2024-09-19T04:13:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7fe130070ff0 msgr2=0x7fe130073430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.542+0000 7fe153e52640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7fe130070ff0 0x7fe130073430 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe1300678f0 msgr2=0x7fe130069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe1300678f0 0x7fe130069db0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0b3410 tx=0x7fe13c008040 comp rx=0 tx=0).stop 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c0d1e10 msgr2=0x7fe14c0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 --2- 172.21.15.76:0/3743955994 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe14c0d1e10 0x7fe14c0cb430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe14401a5c0 tx=0x7fe144019730 comp rx=0 tx=0).stop 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe1523ef640 1 -- 172.21.15.76:0/3743955994 reap_dead start 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 shutdown_connections 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 >> 172.21.15.76:0/3743955994 conn(0x7fe14c08bf90 msgr2=0x7fe14c0b5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7f62ae1bf640 1 -- 172.21.15.76:0/229507791 reap_dead start 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 shutdown_connections 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 >> 172.21.15.76:0/229507791 conn(0x7f62a001dd90 msgr2=0x7f62a00e5750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 shutdown_connections 2024-09-19T04:13:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7f62ae9c0640 1 -- 172.21.15.76:0/229507791 wait complete. 2024-09-19T04:13:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 shutdown_connections 2024-09-19T04:13:41.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.546+0000 7fe153e52640 1 -- 172.21.15.76:0/3743955994 wait complete. 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7f7d9c070f40 msgr2=0x7f7d9c073380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7f7d9c070f40 0x7f7d9c073380 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7d9c0678c0 msgr2=0x7f7d9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7d9c0678c0 0x7f7d9c069d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7da0006fd0 tx=0x7f7da0008040 comp rx=0 tx=0).stop 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db015c0e0 msgr2=0x7f7db0154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 --2- 172.21.15.76:0/1492246332 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7db015c0e0 0x7f7db0154bf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7da800cb10 tx=0x7f7da800a060 comp rx=0 tx=0).stop 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db499c640 1 -- 172.21.15.76:0/1492246332 reap_dead start 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 shutdown_connections 2024-09-19T04:13:41.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 >> 172.21.15.76:0/1492246332 conn(0x7f7db006c740 msgr2=0x7f7db00ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 shutdown_connections 2024-09-19T04:13:41.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.562+0000 7f7db63ff640 1 -- 172.21.15.76:0/1492246332 wait complete. 2024-09-19T04:13:41.591 INFO:teuthology.orchestra.run.smithi076.stdout:60129542152 2024-09-19T04:13:41.592 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-19T04:13:41.593 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:41.593 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-19T04:13:41.594 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:41.594 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-19T04:13:41.594 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:41.595 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:41.596 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-19T04:13:41.596 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-19T04:13:41.597 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:41.597 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-19T04:13:41.598 INFO:teuthology.orchestra.run.smithi076.stdout:60129542152 2024-09-19T04:13:41.598 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-19T04:13:41.599 INFO:teuthology.orchestra.run.smithi076.stdout:60129542152 2024-09-19T04:13:41.599 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-19T04:13:41.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.802+0000 7f480a570640 1 Processor -- start 2024-09-19T04:13:41.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.802+0000 7f5e8cc88640 1 Processor -- start 2024-09-19T04:13:41.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.802+0000 7f5e8cc88640 1 -- start start 2024-09-19T04:13:41.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.802+0000 7f63521a4640 1 Processor -- start 2024-09-19T04:13:41.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffaa5493640 1 Processor -- start 2024-09-19T04:13:41.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 Processor -- start 2024-09-19T04:13:41.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.802+0000 7f480a570640 1 -- start start 2024-09-19T04:13:41.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f480a570640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f480415b290 0x7f4804072cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f480a570640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48040b4c80 con 0x7f480415aeb0 2024-09-19T04:13:41.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffaa5493640 1 -- start start 2024-09-19T04:13:41.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f480a570640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48040b4f50 con 0x7f4804073210 2024-09-19T04:13:41.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f480a570640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4804073680 con 0x7f480415b290 2024-09-19T04:13:41.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 -- start start 2024-09-19T04:13:41.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f48037fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f480415b290 0x7f4804072cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f48037fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f480415b290 0x7f4804072cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40708/0 (socket says 172.21.15.76:40708) 2024-09-19T04:13:41.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f48037fe640 1 -- 172.21.15.76:0/4185132897 learned_addr learned my addr 172.21.15.76:0/4185132897 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e8cc88640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5e88147a10 0x7f5e88161ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e8cc88640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e880b3310 0x7f5e88073310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e8cc88640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e881623a0 con 0x7f5e880ad870 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e8cc88640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5e88162c40 con 0x7f5e880b3310 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e8cc88640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5e88070310 con 0x7f5e88147a10 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e87fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e880b3310 0x7f5e88073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e87fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e880b3310 0x7f5e88073310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40712/0 (socket says 172.21.15.76:40712) 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e87fff640 1 -- 172.21.15.76:0/2055149925 learned_addr learned my addr 172.21.15.76:0/2055149925 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.802+0000 7f63521a4640 1 -- start start 2024-09-19T04:13:41.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f63521a4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f634404eba0 0x7f63440475b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f63521a4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63440338f0 0x7f634404ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f63521a4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6344005aa0 0x7f6344005f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f63521a4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f634401ded0 con 0x7f634404eba0 2024-09-19T04:13:41.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f63521a4640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f634400baf0 con 0x7f63440338f0 2024-09-19T04:13:41.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f63521a4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f634404f4c0 con 0x7f6344005aa0 2024-09-19T04:13:41.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5ad9de1640 1 Processor -- start 2024-09-19T04:13:41.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e877fe640 1 --2- 172.21.15.76:0/2055149925 >> v2:172.21.15.116:3300/0 conn(0x7f5e88147a10 0x7f5e88161ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e867fc640 1 -- 172.21.15.76:0/2055149925 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2004938763 0 0) 0x7f5e881623a0 con 0x7f5e880ad870 2024-09-19T04:13:41.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e87fff640 1 -- 172.21.15.76:0/2055149925 >> v2:172.21.15.116:3300/0 conn(0x7f5e88147a10 msgr2=0x7f5e88161ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e87fff640 1 --2- 172.21.15.76:0/2055149925 >> v2:172.21.15.116:3300/0 conn(0x7f5e88147a10 0x7f5e88161ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e87fff640 1 -- 172.21.15.76:0/2055149925 >> v1:172.21.15.116:6789/0 conn(0x7f5e880ad870 legacy=0x7f5e880b2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5e87fff640 1 -- 172.21.15.76:0/2055149925 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e88162680 con 0x7f5e880b3310 2024-09-19T04:13:41.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63440338f0 0x7f634404ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634affd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63440338f0 0x7f634404ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56218/0 (socket says 172.21.15.76:56218) 2024-09-19T04:13:41.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634affd640 1 -- 172.21.15.76:0/653446455 learned_addr learned my addr 172.21.15.76:0/653446455 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f48037fe640 1 -- 172.21.15.76:0/4185132897 >> v1:172.21.15.116:6789/0 conn(0x7f4804073210 legacy=0x7f480415ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f48037fe640 1 -- 172.21.15.76:0/4185132897 >> v1:172.21.15.76:6789/0 conn(0x7f480415aeb0 legacy=0x7f48040b4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634b7fe640 1 --2- 172.21.15.76:0/653446455 >> v2:172.21.15.76:3300/0 conn(0x7f634404eba0 0x7f63440475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634bfff640 1 --2- 172.21.15.76:0/653446455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6344005aa0 0x7f6344005f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffaa5493640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffaa015b170 0x7ffaa009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffaa5493640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00b0f00 0x7ffaa015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f3f0a1c4640 1 Processor -- start 2024-09-19T04:13:41.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffaa5493640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffaa00ae8b0 con 0x7ffaa00af950 2024-09-19T04:13:41.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffaa5493640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ffaa00aeb90 con 0x7ffaa015b170 2024-09-19T04:13:41.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9d740df930 0x7f9d74056140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffaa5493640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa00aee70 con 0x7ffaa00b0f00 2024-09-19T04:13:41.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7400c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d74008170 0x7f9d74008600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9d74034c40 con 0x7f9d740df930 2024-09-19T04:13:41.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9d74008df0 con 0x7f9d74008170 2024-09-19T04:13:41.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82d18640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d74009090 con 0x7f9d740343f0 2024-09-19T04:13:41.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f4803fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f480415aeb0 0x7f48040b4570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34350/0 (socket says 172.21.15.76:34350) 2024-09-19T04:13:41.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f4802ffd640 1 -- 172.21.15.76:0/4185132897 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3681407662 0 0) 0x7f48040b4c80 con 0x7f480415aeb0 2024-09-19T04:13:41.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5ad9de1640 1 -- start start 2024-09-19T04:13:41.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5ad9de1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad40b5c80 0x7f5ad40b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5ad9de1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad40b71f0 con 0x7f5ad40d3cc0 2024-09-19T04:13:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5ad9de1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ad40b74d0 con 0x7f5ad406ce90 2024-09-19T04:13:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f5ad9de1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad40b7e00 con 0x7f5ad40b5c80 2024-09-19T04:13:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d74008170 0x7f9d74008600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d74008170 0x7f9d74008600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56222/0 (socket says 172.21.15.76:56222) 2024-09-19T04:13:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 -- 172.21.15.76:0/3234188068 learned_addr learned my addr 172.21.15.76:0/3234188068 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffaa015b170 0x7ffaa009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffaa015b170 0x7ffaa009aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40738/0 (socket says 172.21.15.76:40738) 2024-09-19T04:13:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9effd640 1 -- 172.21.15.76:0/315402722 learned_addr learned my addr 172.21.15.76:0/315402722 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9e7fc640 1 --1- 172.21.15.76:0/315402722 >> v1:172.21.15.116:6789/0 conn(0x7ffaa00af950 0x7ffaa00b07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:46114/0 (socket says 172.21.15.76:46114) 2024-09-19T04:13:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d81515640 1 --2- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7400c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f3f0a1c4640 1 -- start start 2024-09-19T04:13:41.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d81d16640 1 --2- 172.21.15.76:0/3234188068 >> v2:172.21.15.76:3300/0 conn(0x7f9d740df930 0x7f9d74056140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9effd640 1 -- 172.21.15.76:0/315402722 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00b0f00 msgr2=0x7ffaa015ee20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:13:41.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9effd640 1 --2- 172.21.15.76:0/315402722 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00b0f00 0x7ffaa015ee20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9effd640 1 -- 172.21.15.76:0/315402722 >> v1:172.21.15.116:6789/0 conn(0x7ffaa00af950 legacy=0x7ffaa00b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 -- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 msgr2=0x7f9d7400c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 --2- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7400c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 -- 172.21.15.76:0/3234188068 >> v2:172.21.15.76:3300/0 conn(0x7f9d740df930 msgr2=0x7f9d74056140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 --2- 172.21.15.76:0/3234188068 >> v2:172.21.15.76:3300/0 conn(0x7f9d740df930 0x7f9d74056140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f9d82517640 1 -- 172.21.15.76:0/3234188068 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d740dfd10 con 0x7f9d74008170 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f3f0a1c4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f041627a0 0x7f3f040b2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f3f0a1c4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f040b3c20 con 0x7f3f0406e2a0 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f3f0a1c4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f040b3f00 con 0x7f3f041623c0 2024-09-19T04:13:41.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f3f0a1c4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3f04071f40 con 0x7f3f041627a0 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634b7fe640 1 -- 172.21.15.76:0/653446455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6344005aa0 msgr2=0x7f6344005f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634b7fe640 1 --2- 172.21.15.76:0/653446455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6344005aa0 0x7f6344005f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634b7fe640 1 -- 172.21.15.76:0/653446455 >> v2:172.21.15.116:3300/0 conn(0x7f63440338f0 msgr2=0x7f634404ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634b7fe640 1 --2- 172.21.15.76:0/653446455 >> v2:172.21.15.116:3300/0 conn(0x7f63440338f0 0x7f634404ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f634b7fe640 1 -- 172.21.15.76:0/653446455 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634401d110 con 0x7f634404eba0 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad8ddf640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5ad40d3cc0 0x7f5ad409cb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34366/0 (socket says 172.21.15.76:34366) 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad8ddf640 1 -- 172.21.15.76:0/3197115200 learned_addr learned my addr 172.21.15.76:0/3197115200 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:41.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad40b5c80 0x7f5ad40b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f3f02ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f041627a0 0x7f3f040b2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f3f02ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f041627a0 0x7f3f040b2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40742/0 (socket says 172.21.15.76:40742) 2024-09-19T04:13:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f3f02ffd640 1 -- 172.21.15.76:0/4195009213 learned_addr learned my addr 172.21.15.76:0/4195009213 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7f48037fe640 1 -- 172.21.15.76:0/4185132897 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48040b5230 con 0x7f480415b290 2024-09-19T04:13:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e87fff640 1 --2- 172.21.15.76:0/2055149925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e880b3310 0x7f5e88073310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e88074ca0 tx=0x7f5e7801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3121366a3272ae55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f3f02ffd640 1 -- 172.21.15.76:0/4195009213 >> v1:172.21.15.116:6789/0 conn(0x7f3f041623c0 legacy=0x7f3f0406db90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T04:13:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f3f02ffd640 1 -- 172.21.15.76:0/4195009213 >> v1:172.21.15.76:6789/0 conn(0x7f3f0406e2a0 legacy=0x7f3f040cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f3f02ffd640 1 -- 172.21.15.76:0/4195009213 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f040b41e0 con 0x7f3f041627a0 2024-09-19T04:13:41.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2844268027 0 0) 0x7f5ad40b71f0 con 0x7f5ad40d3cc0 2024-09-19T04:13:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5abc003530 con 0x7f5ad40d3cc0 2024-09-19T04:13:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 218491443 0 0) 0x7f5ad40b74d0 con 0x7f5ad406ce90 2024-09-19T04:13:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ad40b71f0 con 0x7f5ad406ce90 2024-09-19T04:13:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e867fc640 1 -- 172.21.15.76:0/2055149925 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780ab430 con 0x7f5e880b3310 2024-09-19T04:13:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e867fc640 1 -- 172.21.15.76:0/2055149925 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e780b0070 con 0x7f5e880b3310 2024-09-19T04:13:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e867fc640 1 -- 172.21.15.76:0/2055149925 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e780ab7f0 con 0x7f5e880b3310 2024-09-19T04:13:41.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 <== mon.1 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3106110879 0 0) 0x7f5ad40b71f0 con 0x7f5ad406ce90 2024-09-19T04:13:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad40b5c80 msgr2=0x7f5ad40b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad40b5c80 0x7f5ad40b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 >> v1:172.21.15.76:6789/0 conn(0x7f5ad40d3cc0 legacy=0x7f5ad409cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 -- 172.21.15.76:0/2055149925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e880b3310 msgr2=0x7f5e88073310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/2055149925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e880b3310 0x7f5e88073310 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5e88074ca0 tx=0x7f5e7801c540 comp rx=0 tx=0).stop 2024-09-19T04:13:41.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f634b7fe640 1 --2- 172.21.15.76:0/653446455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f634404eba0 0x7f63440475b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f634c074530 tx=0x7f634c0982c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1c096d42ef2842 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40b77b0 con 0x7f5ad406ce90 2024-09-19T04:13:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 <== mon.1 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5ac8001590 con 0x7f5ad406ce90 2024-09-19T04:13:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 >> v1:172.21.15.116:6789/0 conn(0x7f5ad406ce90 legacy=0x7f5ad406d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5abc004be0 0x7f5abc004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5abc0037b0 0x7f5abc003c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f48037fe640 1 --2- 172.21.15.76:0/4185132897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f480415b290 0x7f4804072cd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009870 tx=0x7f47f401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5fa50ee71c8dc6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5abc004170 0x7f5abc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40b7e00 con 0x7f5abc0037b0 2024-09-19T04:13:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 -- 172.21.15.76:0/2055149925 shutdown_connections 2024-09-19T04:13:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/2055149925 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e880b3310 0x7f5e88073310 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/2055149925 >> v2:172.21.15.116:3300/0 conn(0x7f5e88147a10 0x7f5e88161ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 -- 172.21.15.76:0/2055149925 >> 172.21.15.76:0/2055149925 conn(0x7f5e8806ee40 msgr2=0x7f5e8806f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5abc00c050 con 0x7f5abc004170 2024-09-19T04:13:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5abc00c250 con 0x7f5abc004be0 2024-09-19T04:13:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.806+0000 7ffa9effd640 1 -- 172.21.15.76:0/315402722 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa00af150 con 0x7ffaa015b170 2024-09-19T04:13:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 -- 172.21.15.76:0/2055149925 shutdown_connections 2024-09-19T04:13:41.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5e8cc88640 1 -- 172.21.15.76:0/2055149925 wait complete. 2024-09-19T04:13:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f634a7fc640 1 -- 172.21.15.76:0/653446455 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c127600 con 0x7f634404eba0 2024-09-19T04:13:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f634a7fc640 1 -- 172.21.15.76:0/653446455 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f634c06de50 con 0x7f634404eba0 2024-09-19T04:13:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f634a7fc640 1 -- 172.21.15.76:0/653446455 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634c127aa0 con 0x7f634404eba0 2024-09-19T04:13:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad8ddf640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5abc004be0 0x7f5abc004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f4802ffd640 1 -- 172.21.15.76:0/4185132897 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f40aa600 con 0x7f480415b290 2024-09-19T04:13:41.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f4802ffd640 1 -- 172.21.15.76:0/4185132897 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f40af070 con 0x7f480415b290 2024-09-19T04:13:41.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f4802ffd640 1 -- 172.21.15.76:0/4185132897 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f40aa9c0 con 0x7f480415b290 2024-09-19T04:13:41.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7ffa9effd640 1 --2- 172.21.15.76:0/315402722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa015b170 0x7ffaa009aa00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000d150 tx=0x7ffa9001ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caf9db3a87817b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5abc0037b0 0x7f5abc003c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad3fff640 1 -- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5abc004be0 msgr2=0x7f5abc004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5abc004be0 0x7f5abc004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f9d82517640 1 --2- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 0x7f9d74008600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400eb80 tx=0x7f9d6c01ea10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60b631232fd0c890 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad3fff640 1 -- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5abc004170 msgr2=0x7f5abc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5abc004170 0x7f5abc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f5ad3fff640 1 -- 172.21.15.76:0/3197115200 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40b77b0 con 0x7f5abc0037b0 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f480a570640 1 -- 172.21.15.76:0/4185132897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f480415b290 msgr2=0x7f4804072cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f480a570640 1 --2- 172.21.15.76:0/4185132897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f480415b290 0x7f4804072cd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009870 tx=0x7f47f401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f480a570640 1 -- 172.21.15.76:0/4185132897 shutdown_connections 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f480a570640 1 --2- 172.21.15.76:0/4185132897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f480415b290 0x7f4804072cd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f480a570640 1 -- 172.21.15.76:0/4185132897 >> 172.21.15.76:0/4185132897 conn(0x7f480406cdd0 msgr2=0x7f480406d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f480a570640 1 -- 172.21.15.76:0/4185132897 shutdown_connections 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f480a570640 1 -- 172.21.15.76:0/4185132897 wait complete. 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f9d80d14640 1 -- 172.21.15.76:0/3234188068 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0b4070 con 0x7f9d74008170 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f9d80d14640 1 -- 172.21.15.76:0/3234188068 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d6c0a78b0 con 0x7f9d74008170 2024-09-19T04:13:41.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.810+0000 7f9d80d14640 1 -- 172.21.15.76:0/3234188068 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0a7c80 con 0x7f9d74008170 2024-09-19T04:13:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f02ffd640 1 --2- 172.21.15.76:0/4195009213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041627a0 0x7f3f040b2fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2559249569aa9dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffa9dffb640 1 -- 172.21.15.76:0/315402722 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000b2d0 con 0x7ffaa015b170 2024-09-19T04:13:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffa9dffb640 1 -- 172.21.15.76:0/315402722 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa9000b470 con 0x7ffaa015b170 2024-09-19T04:13:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffa9dffb640 1 -- 172.21.15.76:0/315402722 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa900a7b40 con 0x7ffaa015b170 2024-09-19T04:13:41.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- 172.21.15.76:0/653446455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f634404eba0 msgr2=0x7f63440475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 --2- 172.21.15.76:0/653446455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f634404eba0 0x7f63440475b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f634c074530 tx=0x7f634c0982c0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 Processor -- start 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 -- start start 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e880ad870 0x7f5e88140480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e880b3310 0x7f5e8815da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e88147a10 0x7f5e880b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e88071340 con 0x7f5e88147a10 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5e880711c0 con 0x7f5e880ad870 2024-09-19T04:13:41.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5e880714c0 con 0x7f5e880b3310 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 -- 172.21.15.76:0/315402722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa015b170 msgr2=0x7ffaa009aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 --2- 172.21.15.76:0/315402722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa015b170 0x7ffaa009aa00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ffa9000d150 tx=0x7ffa9001ec50 comp rx=0 tx=0).stop 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- 172.21.15.76:0/653446455 shutdown_connections 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 --2- 172.21.15.76:0/653446455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6344005aa0 0x7f6344005f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 --2- 172.21.15.76:0/653446455 >> v2:172.21.15.116:3300/0 conn(0x7f63440338f0 0x7f634404ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 --2- 172.21.15.76:0/653446455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f634404eba0 0x7f63440475b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- 172.21.15.76:0/653446455 >> 172.21.15.76:0/653446455 conn(0x7f63440360d0 msgr2=0x7f63440364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- 172.21.15.76:0/653446455 shutdown_connections 2024-09-19T04:13:41.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- 172.21.15.76:0/653446455 wait complete. 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f027fc640 1 -- 172.21.15.76:0/4195009213 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40ab680 con 0x7f3f041627a0 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f027fc640 1 -- 172.21.15.76:0/4195009213 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ef40b0070 con 0x7f3f041627a0 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f027fc640 1 -- 172.21.15.76:0/4195009213 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef40aba40 con 0x7f3f041627a0 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/4195009213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041627a0 msgr2=0x7f3f040b2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/4195009213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041627a0 0x7f3f040b2fd0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3ef4009870 tx=0x7f3ef401c460 comp rx=0 tx=0).stop 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/4195009213 shutdown_connections 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/4195009213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041627a0 0x7f3f040b2fd0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/4195009213 >> 172.21.15.76:0/4195009213 conn(0x7f3f0406f420 msgr2=0x7f3f0406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 Processor -- start 2024-09-19T04:13:41.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 -- start start 2024-09-19T04:13:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e880ad870 0x7f5e88140480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e880ad870 0x7f5e88140480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56244/0 (socket says 172.21.15.76:56244) 2024-09-19T04:13:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 -- 172.21.15.76:0/3565357284 learned_addr learned my addr 172.21.15.76:0/3565357284 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e86ffd640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e880b3310 0x7f5e8815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 -- 172.21.15.76:0/315402722 shutdown_connections 2024-09-19T04:13:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 --2- 172.21.15.76:0/315402722 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00b0f00 0x7ffaa015ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 --2- 172.21.15.76:0/315402722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa015b170 0x7ffaa009aa00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 -- 172.21.15.76:0/315402722 >> 172.21.15.76:0/315402722 conn(0x7ffaa00b6800 msgr2=0x7ffaa00b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 -- 172.21.15.76:0/315402722 shutdown_connections 2024-09-19T04:13:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7ffaa5493640 1 -- 172.21.15.76:0/315402722 wait complete. 2024-09-19T04:13:41.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 -- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e880b3310 msgr2=0x7f5e8815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e880b3310 0x7f5e8815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 -- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e88147a10 msgr2=0x7f5e880b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e88147a10 0x7f5e880b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 -- 172.21.15.76:0/3565357284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e880b44b0 con 0x7f5e880ad870 2024-09-19T04:13:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 Processor -- start 2024-09-19T04:13:41.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5abc0037b0 0x7f5abc003c30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40aaca0 tx=0x7f5ac401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4fb98ccf009391 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4804073210 0x7f4804206550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f480415aeb0 0x7f480406ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f480415b290 0x7f480406f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f480415cb20 con 0x7f4804073210 2024-09-19T04:13:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f480415c9a0 con 0x7f480415b290 2024-09-19T04:13:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f480a570640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f480415cca0 con 0x7f480415aeb0 2024-09-19T04:13:41.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f9d82d18640 1 -- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 msgr2=0x7f9d74008600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 0x7f9d74008600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400eb80 tx=0x7f9d6c01ea10 comp rx=0 tx=0).stop 2024-09-19T04:13:41.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e877fe640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e880ad870 0x7f5e88140480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000efc0 tx=0x7f5e8000c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f9d82d18640 1 -- 172.21.15.76:0/3234188068 shutdown_connections 2024-09-19T04:13:41.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 0x7f9d74008600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3234188068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7400c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3234188068 >> v2:172.21.15.76:3300/0 conn(0x7f9d740df930 0x7f9d74056140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f9d82d18640 1 -- 172.21.15.76:0/3234188068 >> 172.21.15.76:0/3234188068 conn(0x7f9d7401ea50 msgr2=0x7f9d740556c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f480415aeb0 0x7f480406ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f480415aeb0 0x7f480406ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38312/0 (socket says 172.21.15.76:38312) 2024-09-19T04:13:41.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 -- 172.21.15.76:0/2146311000 learned_addr learned my addr 172.21.15.76:0/2146311000 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f4808b0d640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f480415b290 0x7f480406f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e6ffff640 1 -- 172.21.15.76:0/3565357284 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80018020 con 0x7f5e880ad870 2024-09-19T04:13:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e88141de0 con 0x7f5e880ad870 2024-09-19T04:13:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e881423a0 con 0x7f5e880ad870 2024-09-19T04:13:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e6ffff640 1 -- 172.21.15.76:0/3565357284 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e80014480 con 0x7f5e880ad870 2024-09-19T04:13:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5e6ffff640 1 -- 172.21.15.76:0/3565357284 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e80014850 con 0x7f5e880ad870 2024-09-19T04:13:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac40b1070 con 0x7f5abc0037b0 2024-09-19T04:13:41.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ac40a7070 con 0x7f5abc0037b0 2024-09-19T04:13:41.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5ad37fe640 1 -- 172.21.15.76:0/3197115200 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac400b540 con 0x7f5abc0037b0 2024-09-19T04:13:41.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3197115200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5abc0037b0 msgr2=0x7f5abc003c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5ad9de1640 1 --2- 172.21.15.76:0/3197115200 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5abc0037b0 0x7f5abc003c30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5ad40aaca0 tx=0x7f5ac401c540 comp rx=0 tx=0).stop 2024-09-19T04:13:41.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f5ad95e0640 1 -- 172.21.15.76:0/3197115200 reap_dead start 2024-09-19T04:13:41.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 Processor -- start 2024-09-19T04:13:41.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 -- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f480415b290 msgr2=0x7f480406f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f480415b290 0x7f480406f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 -- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4804073210 msgr2=0x7f4804206550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4804073210 0x7f4804206550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f48037fe640 1 -- 172.21.15.76:0/2146311000 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4804153940 con 0x7f480415aeb0 2024-09-19T04:13:41.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- start start 2024-09-19T04:13:41.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6344005aa0 0x7f634404b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63440338f0 0x7f6344038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f634404eba0 0x7f634404a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63440072c0 con 0x7f63440338f0 2024-09-19T04:13:41.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f634401d9b0 con 0x7f6344005aa0 2024-09-19T04:13:41.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.814+0000 7f63521a4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6344007440 con 0x7f634404eba0 2024-09-19T04:13:41.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3197115200 shutdown_connections 2024-09-19T04:13:41.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3197115200 >> 172.21.15.76:0/3197115200 conn(0x7f5ad40d0c80 msgr2=0x7f5ad40d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 -- start start 2024-09-19T04:13:41.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f634404eba0 0x7f634404a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6344005aa0 0x7f634404b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f634404eba0 0x7f634404a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38326/0 (socket says 172.21.15.76:38326) 2024-09-19T04:13:41.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 -- 172.21.15.76:0/643070377 learned_addr learned my addr 172.21.15.76:0/643070377 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3197115200 shutdown_connections 2024-09-19T04:13:41.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f48037fe640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f480415aeb0 0x7f480406ecc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f47f40ac040 tx=0x7f47f401ace0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 -- 172.21.15.76:0/3234188068 shutdown_connections 2024-09-19T04:13:41.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/4195009213 shutdown_connections 2024-09-19T04:13:41.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634affd640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63440338f0 0x7f6344038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/4195009213 wait complete. 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00af950 0x7ffaa0151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa00b0f00 0x7ffaa0148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa015b170 0x7ffaa0150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa006c090 con 0x7ffaa00b0f00 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa006bf10 con 0x7ffaa015b170 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffaa006c210 con 0x7ffaa00af950 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa00b0f00 0x7ffaa0148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa00b0f00 0x7ffaa0148c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40792/0 (socket says 172.21.15.76:40792) 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 -- 172.21.15.76:0/1240284759 learned_addr learned my addr 172.21.15.76:0/1240284759 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9effd640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00af950 0x7ffaa0151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9f7fe640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa015b170 0x7ffaa0150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 -- 172.21.15.76:0/643070377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6344005aa0 msgr2=0x7f634404b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6344005aa0 0x7f634404b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 -- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63440338f0 msgr2=0x7f6344038ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63440338f0 0x7f6344038ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 -- 172.21.15.76:0/643070377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6344052ff0 con 0x7f634404eba0 2024-09-19T04:13:41.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 -- 172.21.15.76:0/3234188068 wait complete. 2024-09-19T04:13:41.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 Processor -- start 2024-09-19T04:13:41.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f4800ff9640 1 -- 172.21.15.76:0/2146311000 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f40b3800 con 0x7f480415aeb0 2024-09-19T04:13:41.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f4800ff9640 1 -- 172.21.15.76:0/2146311000 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47f40b39c0 con 0x7f480415aeb0 2024-09-19T04:13:41.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f480a570640 1 -- 172.21.15.76:0/2146311000 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4804153c20 con 0x7f480415aeb0 2024-09-19T04:13:41.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f4800ff9640 1 -- 172.21.15.76:0/2146311000 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f40b3db0 con 0x7f480415aeb0 2024-09-19T04:13:41.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f480a570640 1 -- 172.21.15.76:0/2146311000 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4804144ae0 con 0x7f480415aeb0 2024-09-19T04:13:41.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 -- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00af950 msgr2=0x7ffaa0151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00af950 0x7ffaa0151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 -- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa015b170 msgr2=0x7ffaa0150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa015b170 0x7ffaa0150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 -- 172.21.15.76:0/1240284759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa01545a0 con 0x7ffaa00b0f00 2024-09-19T04:13:41.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 Processor -- start 2024-09-19T04:13:41.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 -- start start 2024-09-19T04:13:41.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa9e7fc640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa00b0f00 0x7ffaa0148c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c490 tx=0x7ffa9800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3197115200 wait complete. 2024-09-19T04:13:41.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f5ad9de1640 1 Processor -- start 2024-09-19T04:13:41.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f634bfff640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f634404eba0 0x7f634404a940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f633800e7f0 tx=0x7f633800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f6337fff640 1 -- 172.21.15.76:0/643070377 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338017070 con 0x7f634404eba0 2024-09-19T04:13:41.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f6337fff640 1 -- 172.21.15.76:0/643070377 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f633800c940 con 0x7f634404eba0 2024-09-19T04:13:41.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f6337fff640 1 -- 172.21.15.76:0/643070377 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f633800eea0 con 0x7f634404eba0 2024-09-19T04:13:41.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffa977fe640 1 -- 172.21.15.76:0/1240284759 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98018070 con 0x7ffaa00b0f00 2024-09-19T04:13:41.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6344032ad0 con 0x7f634404eba0 2024-09-19T04:13:41.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f63440d6af0 con 0x7f634404eba0 2024-09-19T04:13:41.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 -- start start 2024-09-19T04:13:41.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa01542c0 con 0x7ffaa00b0f00 2024-09-19T04:13:41.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa0152810 con 0x7ffaa00b0f00 2024-09-19T04:13:41.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f0406e2a0 0x7f3f04144440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041623c0 0x7f3f0415a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f041627a0 0x7f3f0408b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3f041702a0 con 0x7f3f041623c0 2024-09-19T04:13:41.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f040b39e0 con 0x7f3f0406e2a0 2024-09-19T04:13:41.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f3f0a1c4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3f04170420 con 0x7f3f041627a0 2024-09-19T04:13:41.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 0x7f9d740d8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7404e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740df930 0x7f9d740dcc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7404fe00 con 0x7f9d740df930 2024-09-19T04:13:41.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7404fc80 con 0x7f9d74008170 2024-09-19T04:13:41.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82d18640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d7403a370 con 0x7f9d740343f0 2024-09-19T04:13:41.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f5ad9de1640 1 -- start start 2024-09-19T04:13:41.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f0406e2a0 0x7f3f04144440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f02ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041623c0 0x7f3f0415a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f0406e2a0 0x7f3f04144440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56274/0 (socket says 172.21.15.76:56274) 2024-09-19T04:13:41.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 -- 172.21.15.76:0/2099329515 learned_addr learned my addr 172.21.15.76:0/2099329515 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82517640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740df930 0x7f9d740dcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d81515640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7404e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f9d82517640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740df930 0x7f9d740dcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40818/0 (socket says 172.21.15.76:40818) 2024-09-19T04:13:41.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d82517640 1 -- 172.21.15.76:0/3493612524 learned_addr learned my addr 172.21.15.76:0/3493612524 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f634404ef80 con 0x7f634404eba0 2024-09-19T04:13:41.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d81d16640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 0x7f9d740d8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d82517640 1 -- 172.21.15.76:0/3493612524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 msgr2=0x7f9d7404e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d82517640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7404e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d82517640 1 -- 172.21.15.76:0/3493612524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 msgr2=0x7f9d740d8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d82517640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 0x7f9d740d8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d82517640 1 -- 172.21.15.76:0/3493612524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d740d8c70 con 0x7f9d740df930 2024-09-19T04:13:41.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaa015ee20 con 0x7ffaa00b0f00 2024-09-19T04:13:41.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad9de1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad41f8180 0x7f5ad409e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad9de1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad4073b40 0x7f5ad409e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad9de1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ad4160f80 0x7f5ad415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad9de1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40b3a10 con 0x7f5ad41f8180 2024-09-19T04:13:41.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad9de1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40b3890 con 0x7f5ad4160f80 2024-09-19T04:13:41.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad9de1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad40b60c0 con 0x7f5ad4073b40 2024-09-19T04:13:41.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 -- 172.21.15.76:0/2099329515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f041627a0 msgr2=0x7f3f0408b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f041627a0 0x7f3f0408b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 -- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041623c0 msgr2=0x7f3f0415a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041623c0 0x7f3f0415a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 -- 172.21.15.76:0/2099329515 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f04145910 con 0x7f3f0406e2a0 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f02ffd640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041623c0 0x7f3f0415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad41f8180 0x7f5ad409e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad41f8180 0x7f5ad409e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40832/0 (socket says 172.21.15.76:40832) 2024-09-19T04:13:41.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 -- 172.21.15.76:0/3224233753 learned_addr learned my addr 172.21.15.76:0/3224233753 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f9d82517640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740df930 0x7f9d740dcc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00b2c0 tx=0x7f9d6c01b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad95e0640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ad4160f80 0x7f5ad415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5e6ffff640 1 -- 172.21.15.76:0/3565357284 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5e8001b020 con 0x7f5e880ad870 2024-09-19T04:13:41.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5e6ffff640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e740678c0 0x7f5e74069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5e6ffff640 1 -- 172.21.15.76:0/3565357284 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5e80086da0 con 0x7f5e880ad870 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad4073b40 0x7f5ad409e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 -- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad4073b40 msgr2=0x7f5ad409e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad4073b40 0x7f5ad409e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 -- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ad4160f80 msgr2=0x7f5ad415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ad4160f80 0x7f5ad415e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 -- 172.21.15.76:0/3224233753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad40a1f90 con 0x7f5ad41f8180 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad4073b40 0x7f5ad409e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:41.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5ad8ddf640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad41f8180 0x7f5ad409e2a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5acc001ad0 tx=0x7f5acc00a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f037fe640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f0406e2a0 0x7f3f04144440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00b4d0 tx=0x7f3efc00b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f649d358640 1 Processor -- start 2024-09-19T04:13:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.818+0000 7f480a570640 1 -- 172.21.15.76:0/2146311000 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4804068930 con 0x7f480415aeb0 2024-09-19T04:13:41.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f649d358640 1 -- start start 2024-09-19T04:13:41.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5e86ffd640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e740678c0 0x7f5e74069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f6337fff640 1 -- 172.21.15.76:0/643070377 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f633801a070 con 0x7f634404eba0 2024-09-19T04:13:41.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3efbfff640 1 -- 172.21.15.76:0/2099329515 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc014070 con 0x7f3f0406e2a0 2024-09-19T04:13:41.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f04144d00 con 0x7f3f0406e2a0 2024-09-19T04:13:41.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f6337fff640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6330067940 0x7f6330069e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f6337fff640 1 -- 172.21.15.76:0/643070377 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f63380863d0 con 0x7f634404eba0 2024-09-19T04:13:41.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f5e86ffd640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e740678c0 0x7f5e74069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e880b0280 tx=0x7f5e7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f649d358640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f64900df9f0 0x7f649003a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3efbfff640 1 -- 172.21.15.76:0/2099329515 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3efc010890 con 0x7f3f0406e2a0 2024-09-19T04:13:41.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3efbfff640 1 -- 172.21.15.76:0/2099329515 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3efc010c80 con 0x7f3f0406e2a0 2024-09-19T04:13:41.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f041452c0 con 0x7f3f0406e2a0 2024-09-19T04:13:41.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f040b2fd0 con 0x7f3f0406e2a0 2024-09-19T04:13:41.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f649d358640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6490034a40 0x7f649003a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f649d358640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6490008750 con 0x7f6490034e90 2024-09-19T04:13:41.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f649d358640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f64900dfdd0 con 0x7f6490034a40 2024-09-19T04:13:41.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f649d358640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f64900368c0 con 0x7f64900df9f0 2024-09-19T04:13:41.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f6497fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f64900df9f0 0x7f649003a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f6497fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f64900df9f0 0x7f649003a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:56308/0 (socket says 172.21.15.76:56308) 2024-09-19T04:13:41.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.822+0000 7f6497fff640 1 -- 172.21.15.76:0/336254606 learned_addr learned my addr 172.21.15.76:0/336254606 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7ffa977fe640 1 -- 172.21.15.76:0/1240284759 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa98007480 con 0x7ffaa00b0f00 2024-09-19T04:13:41.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f4800ff9640 1 -- 172.21.15.76:0/2146311000 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47f40a9070 con 0x7f480415aeb0 2024-09-19T04:13:41.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f634b7fe640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6330067940 0x7f6330069e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f4800ff9640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f47f00678f0 0x7f47f0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f4800ff9640 1 -- 172.21.15.76:0/2146311000 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f47f41207a0 con 0x7f480415aeb0 2024-09-19T04:13:41.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f4803fff640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f47f00678f0 0x7f47f0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f634b7fe640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6330067940 0x7f6330069e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f634401e0a0 tx=0x7f634c122000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f5ad17fa640 1 -- 172.21.15.76:0/3224233753 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc017020 con 0x7f5ad41f8180 2024-09-19T04:13:41.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad406dcf0 con 0x7f5ad41f8180 2024-09-19T04:13:41.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad40a0f60 con 0x7f5ad41f8180 2024-09-19T04:13:41.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f3efbfff640 1 -- 172.21.15.76:0/2099329515 <== mon.1 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3efc017070 con 0x7f3f0406e2a0 2024-09-19T04:13:41.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f3efbfff640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ee40678c0 0x7f3ee4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f3f02ffd640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ee40678c0 0x7f3ee4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f3efbfff640 1 -- 172.21.15.76:0/2099329515 <== mon.1 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3efc086270 con 0x7f3f0406e2a0 2024-09-19T04:13:41.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f4803fff640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f47f00678f0 0x7f47f0069db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47f8000c10 tx=0x7f47f8007910 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f3f02ffd640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ee40678c0 0x7f3ee4069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40af000 tx=0x7f3ef40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f9d727fc640 1 -- 172.21.15.76:0/3493612524 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0bf070 con 0x7f9d740df930 2024-09-19T04:13:41.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f6497fff640 1 -- 172.21.15.76:0/336254606 >> v2:172.21.15.76:3300/0 conn(0x7f6490034a40 msgr2=0x7f649003a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:41.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7404ee10 con 0x7f9d740df930 2024-09-19T04:13:41.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7404f3c0 con 0x7f9d740df930 2024-09-19T04:13:41.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad406da80 con 0x7f5ad41f8180 2024-09-19T04:13:41.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f64977fe640 1 --2- 172.21.15.76:0/336254606 >> v2:172.21.15.76:3300/0 conn(0x7f6490034a40 0x7f649003a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7ffa977fe640 1 -- 172.21.15.76:0/1240284759 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98004a10 con 0x7ffaa00b0f00 2024-09-19T04:13:41.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f6497fff640 1 --2- 172.21.15.76:0/336254606 >> v2:172.21.15.76:3300/0 conn(0x7f6490034a40 0x7f649003a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7ffa977fe640 1 -- 172.21.15.76:0/1240284759 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffa9801b070 con 0x7ffaa00b0f00 2024-09-19T04:13:41.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f6497fff640 1 -- 172.21.15.76:0/336254606 >> v1:172.21.15.76:6789/0 conn(0x7f6490034e90 legacy=0x7f64900532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f6497fff640 1 -- 172.21.15.76:0/336254606 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6490036af0 con 0x7f64900df9f0 2024-09-19T04:13:41.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7ffa977fe640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffa800679c0 0x7ffa80069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f64977fe640 1 --2- 172.21.15.76:0/336254606 >> v2:172.21.15.76:3300/0 conn(0x7f6490034a40 0x7f649003a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:41.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d740491f0 con 0x7f9d740df930 2024-09-19T04:13:41.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7ffa9effd640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffa800679c0 0x7ffa80069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7ffa977fe640 1 -- 172.21.15.76:0/1240284759 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffa980853f0 con 0x7ffaa00b0f00 2024-09-19T04:13:41.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e680052a0 con 0x7f5e880ad870 2024-09-19T04:13:41.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.826+0000 7ffa9effd640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffa800679c0 0x7ffa80069e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa900b0000 tx=0x7ffa900a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.830+0000 7f6497fff640 1 --2- 172.21.15.76:0/336254606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64900df9f0 0x7f649003a3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f649000a580 tx=0x7f648c01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29733f8f74d77bcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.830+0000 7f9d727fc640 1 -- 172.21.15.76:0/3493612524 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d6c000e50 con 0x7f9d740df930 2024-09-19T04:13:41.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.830+0000 7f9d727fc640 1 -- 172.21.15.76:0/3493612524 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c0b4040 con 0x7f9d740df930 2024-09-19T04:13:41.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.830+0000 7f5ad17fa640 1 -- 172.21.15.76:0/3224233753 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5acc00ad70 con 0x7f5ad41f8180 2024-09-19T04:13:41.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.830+0000 7f5ad17fa640 1 -- 172.21.15.76:0/3224233753 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5acc026640 con 0x7f5ad41f8180 2024-09-19T04:13:41.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.830+0000 7ffa977fe640 1 -- 172.21.15.76:0/1240284759 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffa98055580 con 0x7ffaa00b0f00 2024-09-19T04:13:41.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f5ad17fa640 1 -- 172.21.15.76:0/3224233753 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5acc01a020 con 0x7f5ad41f8180 2024-09-19T04:13:41.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f5ad17fa640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ac00678c0 0x7f5ac0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f9d727fc640 1 -- 172.21.15.76:0/3493612524 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9d6c0af640 con 0x7f9d740df930 2024-09-19T04:13:41.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f9d727fc640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d600678c0 0x7f9d60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f9d727fc640 1 -- 172.21.15.76:0/3493612524 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9d6c120110 con 0x7f9d740df930 2024-09-19T04:13:41.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f5ad17fa640 1 -- 172.21.15.76:0/3224233753 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5acc087100 con 0x7f5ad41f8180 2024-09-19T04:13:41.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f9d81d16640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d600678c0 0x7f9d60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ac00678c0 0x7f5ac0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f9d81d16640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d600678c0 0x7f9d60069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d78006fd0 tx=0x7f9d78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f5ad3fff640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ac00678c0 0x7f5ac0069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40ae000 tx=0x7f5ac40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.834+0000 7f6337fff640 1 -- 172.21.15.76:0/643070377 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f63380566f0 con 0x7f634404eba0 2024-09-19T04:13:41.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.838+0000 7f9d727fc640 1 -- 172.21.15.76:0/3493612524 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9d6c0c9080 con 0x7f9d740df930 2024-09-19T04:13:41.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.838+0000 7f3efbfff640 1 -- 172.21.15.76:0/2099329515 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3efc056a50 con 0x7f3f0406e2a0 2024-09-19T04:13:41.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.842+0000 7f5ad17fa640 1 -- 172.21.15.76:0/3224233753 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5acc021020 con 0x7f5ad41f8180 2024-09-19T04:13:41.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.846+0000 7f4800ff9640 1 -- 172.21.15.76:0/2146311000 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f47f4016150 con 0x7f480415aeb0 2024-09-19T04:13:41.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.850+0000 7f5e6ffff640 1 -- 172.21.15.76:0/3565357284 <== mon.1 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5e80022030 con 0x7f5e880ad870 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.866+0000 7f6496ffd640 1 -- 172.21.15.76:0/336254606 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0ab650 con 0x7f64900df9f0 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.866+0000 7f6496ffd640 1 -- 172.21.15.76:0/336254606 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c0b0070 con 0x7f64900df9f0 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.866+0000 7f6496ffd640 1 -- 172.21.15.76:0/336254606 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0aba10 con 0x7f64900df9f0 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 -- 172.21.15.76:0/336254606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64900df9f0 msgr2=0x7f649003a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 --2- 172.21.15.76:0/336254606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64900df9f0 0x7f649003a3a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f649000a580 tx=0x7f648c01c540 comp rx=0 tx=0).stop 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 -- 172.21.15.76:0/336254606 shutdown_connections 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 --2- 172.21.15.76:0/336254606 >> v2:172.21.15.76:3300/0 conn(0x7f6490034a40 0x7f649003a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 --2- 172.21.15.76:0/336254606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64900df9f0 0x7f649003a3a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 -- 172.21.15.76:0/336254606 >> 172.21.15.76:0/336254606 conn(0x7f649001ea50 msgr2=0x7f64900556f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 -- 172.21.15.76:0/336254606 shutdown_connections 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.870+0000 7f649d358640 1 -- 172.21.15.76:0/336254606 wait complete. 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 Processor -- start 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 -- start start 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6490034a40 0x7f649004c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6490034a40 0x7f649004c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6490034a40 0x7f649004c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40866/0 (socket says 172.21.15.76:40866) 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 -- 172.21.15.76:0/2851255793 learned_addr learned my addr 172.21.15.76:0/2851255793 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6490034e90 0x7f6490033770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64900df9f0 0x7f64900dd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f64900ec630 con 0x7f6490034a40 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64900ec4b0 con 0x7f6490034e90 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f64977fe640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6490034e90 0x7f6490033770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f64900ec7b0 con 0x7f64900df9f0 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f649cb57640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64900df9f0 0x7f64900dd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 -- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64900df9f0 msgr2=0x7f64900dd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64900df9f0 0x7f64900dd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 -- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6490034e90 msgr2=0x7f6490033770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6490034e90 0x7f6490033770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.874+0000 7f6497fff640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649004d0a0 con 0x7f6490034a40 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.878+0000 7f6497fff640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6490034a40 0x7f649004c7c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f648c0ad040 tx=0x7f648c01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.878+0000 7f6494ff9640 1 -- 172.21.15.76:0/2851255793 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0b4800 con 0x7f6490034a40 2024-09-19T04:13:41.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.878+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649004d380 con 0x7f6490034a40 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.878+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f649004d940 con 0x7f6490034a40 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.878+0000 7f6494ff9640 1 -- 172.21.15.76:0/2851255793 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648c0b49c0 con 0x7f6490034a40 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.882+0000 7f6494ff9640 1 -- 172.21.15.76:0/2851255793 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648c0b4db0 con 0x7f6490034a40 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.882+0000 7f6494ff9640 1 -- 172.21.15.76:0/2851255793 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f648c0b3380 con 0x7f6490034a40 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.882+0000 7f6494ff9640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f647c0679c0 0x7f647c069e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.882+0000 7f6494ff9640 1 -- 172.21.15.76:0/2851255793 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f648c1207d0 con 0x7f6490034a40 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.882+0000 7f64977fe640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f647c0679c0 0x7f647c069e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:41.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.882+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64780052a0 con 0x7f6490034a40 2024-09-19T04:13:41.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.890+0000 7f6494ff9640 1 -- 172.21.15.76:0/2851255793 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f648c016150 con 0x7f6490034a40 2024-09-19T04:13:41.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:41.894+0000 7f64977fe640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f647c0679c0 0x7f647c069e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f649004be50 tx=0x7f6484008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.154+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9d74009bf0 con 0x7f9d740df930 2024-09-19T04:13:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.154+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ffaa006fa40 con 0x7ffaa00b0f00 2024-09-19T04:13:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.154+0000 7f9d727fc640 1 -- 172.21.15.76:0/3493612524 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d740d8c70 con 0x7f9d740df930 2024-09-19T04:13:42.159 INFO:teuthology.orchestra.run.smithi076.stdout:60129542152 2024-09-19T04:13:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.154+0000 7ffa977fe640 1 -- 172.21.15.76:0/1240284759 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa98059880 con 0x7ffaa00b0f00 2024-09-19T04:13:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.154+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3f04075420 con 0x7f3f0406e2a0 2024-09-19T04:13:42.160 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f3efbfff640 1 -- 172.21.15.76:0/2099329515 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3efc010e20 con 0x7f3f0406e2a0 2024-09-19T04:13:42.160 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d600678c0 msgr2=0x7f9d60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d600678c0 0x7f9d60069d80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9d78006fd0 tx=0x7f9d78008040 comp rx=0 tx=0).stop 2024-09-19T04:13:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740df930 msgr2=0x7f9d740dcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740df930 0x7f9d740dcc20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c00b2c0 tx=0x7f9d6c01b7b0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 shutdown_connections 2024-09-19T04:13:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d600678c0 0x7f9d60069d80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d740df930 0x7f9d740dcc20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d740343f0 0x7f9d7404e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 --2- 172.21.15.76:0/3493612524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d74008170 0x7f9d740d8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 >> 172.21.15.76:0/3493612524 conn(0x7f9d7401ea50 msgr2=0x7f9d740556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 shutdown_connections 2024-09-19T04:13:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7f9d82d18640 1 -- 172.21.15.76:0/3493612524 wait complete. 2024-09-19T04:13:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffa800679c0 msgr2=0x7ffa80069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffa800679c0 0x7ffa80069e80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffa900b0000 tx=0x7ffa900a9070 comp rx=0 tx=0).stop 2024-09-19T04:13:42.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa00b0f00 msgr2=0x7ffaa0148c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa00b0f00 0x7ffaa0148c20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800c490 tx=0x7ffa9800c950 comp rx=0 tx=0).stop 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 shutdown_connections 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffa800679c0 0x7ffa80069e80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa015b170 0x7ffaa0150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffaa00b0f00 0x7ffaa0148c20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 --2- 172.21.15.76:0/1240284759 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffaa00af950 0x7ffaa0151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 >> 172.21.15.76:0/1240284759 conn(0x7ffaa00b6800 msgr2=0x7ffaa00b1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 shutdown_connections 2024-09-19T04:13:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.158+0000 7ffaa5493640 1 -- 172.21.15.76:0/1240284759 wait complete. 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ee40678c0 msgr2=0x7f3ee4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ee40678c0 0x7f3ee4069d80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40af000 tx=0x7f3ef40a7000 comp rx=0 tx=0).stop 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f0406e2a0 msgr2=0x7f3f04144440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f0406e2a0 0x7f3f04144440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3efc00b4d0 tx=0x7f3efc00b990 comp rx=0 tx=0).stop 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 shutdown_connections 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ee40678c0 0x7f3ee4069d80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f041627a0 0x7f3f0408b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f041623c0 0x7f3f0415a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 --2- 172.21.15.76:0/2099329515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f0406e2a0 0x7f3f04144440 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 >> 172.21.15.76:0/2099329515 conn(0x7f3f0406f420 msgr2=0x7f3f04073540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 shutdown_connections 2024-09-19T04:13:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.162+0000 7f3f0a1c4640 1 -- 172.21.15.76:0/2099329515 wait complete. 2024-09-19T04:13:42.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.166+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f5ad40b71f0 con 0x7f5ad41f8180 2024-09-19T04:13:42.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.166+0000 7f5ad17fa640 1 -- 172.21.15.76:0/3224233753 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5acc057860 con 0x7f5ad41f8180 2024-09-19T04:13:42.172 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:42.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f63440db380 con 0x7f634404eba0 2024-09-19T04:13:42.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f6337fff640 1 -- 172.21.15.76:0/643070377 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f633805a9f0 con 0x7f634404eba0 2024-09-19T04:13:42.174 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ac00678c0 msgr2=0x7f5ac0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ac00678c0 0x7f5ac0069d80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40ae000 tx=0x7f5ac40046d0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad41f8180 msgr2=0x7f5ad409e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad41f8180 0x7f5ad409e2a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5acc001ad0 tx=0x7f5acc00a330 comp rx=0 tx=0).stop 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 shutdown_connections 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ac00678c0 0x7f5ac0069d80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ad4160f80 0x7f5ad415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ad4073b40 0x7f5ad409e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 --2- 172.21.15.76:0/3224233753 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ad41f8180 0x7f5ad409e2a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 >> 172.21.15.76:0/3224233753 conn(0x7f5ad40d0c80 msgr2=0x7f5ad40d21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 shutdown_connections 2024-09-19T04:13:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.170+0000 7f5ad9de1640 1 -- 172.21.15.76:0/3224233753 wait complete. 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6330067940 msgr2=0x7f6330069e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6330067940 0x7f6330069e00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f634401e0a0 tx=0x7f634c122000 comp rx=0 tx=0).stop 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f634404eba0 msgr2=0x7f634404a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f634404eba0 0x7f634404a940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f633800e7f0 tx=0x7f633800ecb0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 shutdown_connections 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6330067940 0x7f6330069e00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f634404eba0 0x7f634404a940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63440338f0 0x7f6344038ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 --2- 172.21.15.76:0/643070377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6344005aa0 0x7f634404b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.174+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 >> 172.21.15.76:0/643070377 conn(0x7f63440360d0 msgr2=0x7f6344052120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.178+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 shutdown_connections 2024-09-19T04:13:42.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.178+0000 7f63521a4640 1 -- 172.21.15.76:0/643070377 wait complete. 2024-09-19T04:13:42.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.178+0000 7f480a570640 1 -- 172.21.15.76:0/2146311000 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4804144ce0 con 0x7f480415aeb0 2024-09-19T04:13:42.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.182+0000 7f4800ff9640 1 -- 172.21.15.76:0/2146311000 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4804144ce0 con 0x7f480415aeb0 2024-09-19T04:13:42.188 INFO:teuthology.orchestra.run.smithi076.stdout:60129542152 2024-09-19T04:13:42.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.186+0000 7f47ea7fc640 1 -- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f47f00678f0 msgr2=0x7f47f0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.186+0000 7f47ea7fc640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f47f00678f0 0x7f47f0069db0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47f8000c10 tx=0x7f47f8007910 comp rx=0 tx=0).stop 2024-09-19T04:13:42.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.186+0000 7f47ea7fc640 1 -- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f480415aeb0 msgr2=0x7f480406ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.186+0000 7f47ea7fc640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f480415aeb0 0x7f480406ecc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f47f40ac040 tx=0x7f47f401ace0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.6 2024-09-19T04:13:42.190 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 -- 172.21.15.76:0/2146311000 shutdown_connections 2024-09-19T04:13:42.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f47f00678f0 0x7f47f0069db0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f480415b290 0x7f480406f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f480415aeb0 0x7f480406ecc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 --2- 172.21.15.76:0/2146311000 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4804073210 0x7f4804206550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 -- 172.21.15.76:0/2146311000 >> 172.21.15.76:0/2146311000 conn(0x7f480406cdd0 msgr2=0x7f4804143b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 -- 172.21.15.76:0/2146311000 shutdown_connections 2024-09-19T04:13:42.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f47ea7fc640 1 -- 172.21.15.76:0/2146311000 wait complete. 2024-09-19T04:13:42.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f5e68005590 con 0x7f5e880ad870 2024-09-19T04:13:42.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.7 2024-09-19T04:13:42.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.190+0000 7f5e6ffff640 1 -- 172.21.15.76:0/3565357284 <== mon.1 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5e80010040 con 0x7f5e880ad870 2024-09-19T04:13:42.195 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.195 INFO:teuthology.orchestra.run.smithi076.stdout:60129542152 2024-09-19T04:13:42.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.0 2024-09-19T04:13:42.198 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e740678c0 msgr2=0x7f5e74069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e740678c0 0x7f5e74069d80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5e880b0280 tx=0x7f5e7c009040 comp rx=0 tx=0).stop 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e880ad870 msgr2=0x7f5e88140480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e880ad870 0x7f5e88140480 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e8000efc0 tx=0x7f5e8000c490 comp rx=0 tx=0).stop 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 shutdown_connections 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e740678c0 0x7f5e74069d80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e88147a10 0x7f5e880b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e880b3310 0x7f5e8815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 --2- 172.21.15.76:0/3565357284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e880ad870 0x7f5e88140480 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 >> 172.21.15.76:0/3565357284 conn(0x7f5e8806ee40 msgr2=0x7f5e88146b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 shutdown_connections 2024-09-19T04:13:42.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.194+0000 7f5e8cc88640 1 -- 172.21.15.76:0/3565357284 wait complete. 2024-09-19T04:13:42.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.5 2024-09-19T04:13:42.207 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.2 2024-09-19T04:13:42.209 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.206+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f6478005590 con 0x7f6490034a40 2024-09-19T04:13:42.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.206+0000 7f6494ff9640 1 -- 172.21.15.76:0/2851255793 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f648c0f0960 con 0x7f6490034a40 2024-09-19T04:13:42.211 INFO:teuthology.orchestra.run.smithi076.stdout:60129542153 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f647c0679c0 msgr2=0x7f647c069e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f647c0679c0 0x7f647c069e80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f649004be50 tx=0x7f6484008040 comp rx=0 tx=0).stop 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6490034a40 msgr2=0x7f649004c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6490034a40 0x7f649004c7c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f648c0ad040 tx=0x7f648c01ad80 comp rx=0 tx=0).stop 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 shutdown_connections 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f647c0679c0 0x7f647c069e80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64900df9f0 0x7f64900dd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6490034e90 0x7f6490033770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 --2- 172.21.15.76:0/2851255793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6490034a40 0x7f649004c7c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 >> 172.21.15.76:0/2851255793 conn(0x7f649001ea50 msgr2=0x7f64900d8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 shutdown_connections 2024-09-19T04:13:42.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.210+0000 7f649d358640 1 -- 172.21.15.76:0/2851255793 wait complete. 2024-09-19T04:13:42.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.3 2024-09-19T04:13:42.215 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542152 for osd.4 2024-09-19T04:13:42.221 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542153 for osd.1 2024-09-19T04:13:42.229 DEBUG:teuthology.parallel:result is None 2024-09-19T04:13:42.229 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:13:42.229 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:13:42.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.374+0000 7fbfc29fe640 1 Processor -- start 2024-09-19T04:13:42.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.374+0000 7fbfc29fe640 1 -- start start 2024-09-19T04:13:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbfbc073e60 0x7fbfbc06e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbfbc06fef0 0x7fbfbc15b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc0b3d50 0x7fbfbc0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbfbc0b46b0 con 0x7fbfbc06fef0 2024-09-19T04:13:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbfbc15bda0 con 0x7fbfbc073e60 2024-09-19T04:13:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc06ee50 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbfbc06fef0 0x7fbfbc15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbfbc06fef0 0x7fbfbc15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40878/0 (socket says 172.21.15.76:40878) 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 -- 172.21.15.76:0/2333296573 learned_addr learned my addr 172.21.15.76:0/2333296573 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 -- 172.21.15.76:0/2333296573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc0b3d50 msgr2=0x7fbfbc0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc0f9b640 1 --2- 172.21.15.76:0/2333296573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc0b3d50 0x7fbfbc0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 --2- 172.21.15.76:0/2333296573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc0b3d50 0x7fbfbc0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 -- 172.21.15.76:0/2333296573 >> v2:172.21.15.116:3300/0 conn(0x7fbfbc073e60 msgr2=0x7fbfbc06e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbbfff640 1 --2- 172.21.15.76:0/2333296573 >> v2:172.21.15.116:3300/0 conn(0x7fbfbc073e60 0x7fbfbc06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 --2- 172.21.15.76:0/2333296573 >> v2:172.21.15.116:3300/0 conn(0x7fbfbc073e60 0x7fbfbc06e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 -- 172.21.15.76:0/2333296573 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc06f130 con 0x7fbfbc06fef0 2024-09-19T04:13:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbb7fe640 1 --2- 172.21.15.76:0/2333296573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc06fef0 0x7fbfbc15b860 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc070340 tx=0x7fbfa401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245adcaa812fb160 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbaffd640 1 -- 172.21.15.76:0/2333296573 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40b0070 con 0x7fbfbc06fef0 2024-09-19T04:13:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbaffd640 1 -- 172.21.15.76:0/2333296573 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfa40ab600 con 0x7fbfbc06fef0 2024-09-19T04:13:42.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfbaffd640 1 -- 172.21.15.76:0/2333296573 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa40ab9d0 con 0x7fbfbc06fef0 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- 172.21.15.76:0/2333296573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc06fef0 msgr2=0x7fbfbc15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/2333296573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc06fef0 0x7fbfbc15b860 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc070340 tx=0x7fbfa401c490 comp rx=0 tx=0).stop 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- 172.21.15.76:0/2333296573 shutdown_connections 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/2333296573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc0b3d50 0x7fbfbc0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/2333296573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc06fef0 0x7fbfbc15b860 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/2333296573 >> v2:172.21.15.116:3300/0 conn(0x7fbfbc073e60 0x7fbfbc06e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- 172.21.15.76:0/2333296573 >> 172.21.15.76:0/2333296573 conn(0x7fbfbc0b1f90 msgr2=0x7fbfbc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- 172.21.15.76:0/2333296573 shutdown_connections 2024-09-19T04:13:42.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.378+0000 7fbfc29fe640 1 -- 172.21.15.76:0/2333296573 wait complete. 2024-09-19T04:13:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 Processor -- start 2024-09-19T04:13:42.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 -- start start 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfbc06fef0 0x7fbfbc155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc073e60 0x7fbfbc157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc0b3d50 0x7fbfbc0ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc15e580 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbfbc15e400 con 0x7fbfbc06fef0 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbfbc15e700 con 0x7fbfbc073e60 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc0b3d50 0x7fbfbc0ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc0b3d50 0x7fbfbc0ae9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:40894/0 (socket says 172.21.15.76:40894) 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 -- 172.21.15.76:0/1087857940 learned_addr learned my addr 172.21.15.76:0/1087857940 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:42.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfbbfff640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfbc06fef0 0x7fbfbc155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfbb7fe640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc073e60 0x7fbfbc157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 -- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc073e60 msgr2=0x7fbfbc157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc073e60 0x7fbfbc157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 -- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfbc06fef0 msgr2=0x7fbfbc155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfbc06fef0 0x7fbfbc155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 -- 172.21.15.76:0/1087857940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfbc0ae4e0 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc0f9b640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc0b3d50 0x7fbfbc0ae9a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400cda0 tx=0x7fbfb400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfb8ff9640 1 -- 172.21.15.76:0/1087857940 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4003ba0 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfb8ff9640 1 -- 172.21.15.76:0/1087857940 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbfb4003d40 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfbc154020 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfb8ff9640 1 -- 172.21.15.76:0/1087857940 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb4004240 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfbc099e10 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.382+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfbc15b860 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.386+0000 7fbfb8ff9640 1 -- 172.21.15.76:0/1087857940 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbfb40043e0 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.386+0000 7fbfb8ff9640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf9c0678c0 0x7fbf9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.386+0000 7fbfb8ff9640 1 -- 172.21.15.76:0/1087857940 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbfb4086ab0 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.386+0000 7fbfbbfff640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf9c0678c0 0x7fbf9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.386+0000 7fbfbbfff640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf9c0678c0 0x7fbf9c069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbfac006fd0 tx=0x7fbfac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.390+0000 7fbfb8ff9640 1 -- 172.21.15.76:0/1087857940 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbfb4057340 con 0x7fbfbc0b3d50 2024-09-19T04:13:42.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.534+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbfbc0723f0 con 0x7fbf9c0678c0 2024-09-19T04:13:42.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.546+0000 7fbfb8ff9640 1 -- 172.21.15.76:0/1087857940 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fbfbc0723f0 con 0x7fbf9c0678c0 2024-09-19T04:13:42.554 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:42.557 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf9c0678c0 msgr2=0x7fbf9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf9c0678c0 0x7fbf9c069d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbfac006fd0 tx=0x7fbfac008040 comp rx=0 tx=0).stop 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc0b3d50 msgr2=0x7fbfbc0ae9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc0b3d50 0x7fbfbc0ae9a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbfb400cda0 tx=0x7fbfb400a660 comp rx=0 tx=0).stop 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 shutdown_connections 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf9c0678c0 0x7fbf9c069d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbfbc0b3d50 0x7fbfbc0ae9a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbfbc073e60 0x7fbfbc157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 --2- 172.21.15.76:0/1087857940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbfbc06fef0 0x7fbfbc155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 >> 172.21.15.76:0/1087857940 conn(0x7fbfbc0b1f90 msgr2=0x7fbfbc074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 shutdown_connections 2024-09-19T04:13:42.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.554+0000 7fbfc29fe640 1 -- 172.21.15.76:0/1087857940 wait complete. 2024-09-19T04:13:42.572 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-19T04:13:41.742492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223668,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751052,"statfs":{"total":773094113280,"available":772865077248,"internally_reserved":0,"allocated":3350528,"data_stored":2017900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.755961"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062396+0000","last_change":"2024-09-19T04:13:23.917029+0000","last_active":"2024-09-19T04:13:31.062396+0000","last_peered":"2024-09-19T04:13:31.062396+0000","last_clean":"2024-09-19T04:13:31.062396+0000","last_became_active":"2024-09-19T04:13:23.916640+0000","last_became_peered":"2024-09-19T04:13:23.916640+0000","last_unstale":"2024-09-19T04:13:31.062396+0000","last_undegraded":"2024-09-19T04:13:31.062396+0000","last_fullsized":"2024-09-19T04:13:31.062396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:22.981160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301574+0000","last_change":"2024-09-19T04:13:24.947160+0000","last_active":"2024-09-19T04:13:30.301574+0000","last_peered":"2024-09-19T04:13:30.301574+0000","last_clean":"2024-09-19T04:13:30.301574+0000","last_became_active":"2024-09-19T04:13:24.946824+0000","last_became_peered":"2024-09-19T04:13:24.946824+0000","last_unstale":"2024-09-19T04:13:30.301574+0000","last_undegraded":"2024-09-19T04:13:30.301574+0000","last_fullsized":"2024-09-19T04:13:30.301574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300320+0000","last_change":"2024-09-19T04:13:29.029580+0000","last_active":"2024-09-19T04:13:30.300320+0000","last_peered":"2024-09-19T04:13:30.300320+0000","last_clean":"2024-09-19T04:13:30.300320+0000","last_became_active":"2024-09-19T04:13:29.029159+0000","last_became_peered":"2024-09-19T04:13:29.029159+0000","last_unstale":"2024-09-19T04:13:30.300320+0000","last_undegraded":"2024-09-19T04:13:30.300320+0000","last_fullsized":"2024-09-19T04:13:30.300320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301237+0000","last_change":"2024-09-19T04:13:23.910773+0000","last_active":"2024-09-19T04:13:30.301237+0000","last_peered":"2024-09-19T04:13:30.301237+0000","last_clean":"2024-09-19T04:13:30.301237+0000","last_became_active":"2024-09-19T04:13:23.910355+0000","last_became_peered":"2024-09-19T04:13:23.910355+0000","last_unstale":"2024-09-19T04:13:30.301237+0000","last_undegraded":"2024-09-19T04:13:30.301237+0000","last_fullsized":"2024-09-19T04:13:30.301237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062270+0000","last_change":"2024-09-19T04:13:24.956969+0000","last_active":"2024-09-19T04:13:31.062270+0000","last_peered":"2024-09-19T04:13:31.062270+0000","last_clean":"2024-09-19T04:13:31.062270+0000","last_became_active":"2024-09-19T04:13:24.956671+0000","last_became_peered":"2024-09-19T04:13:24.956671+0000","last_unstale":"2024-09-19T04:13:31.062270+0000","last_undegraded":"2024-09-19T04:13:31.062270+0000","last_fullsized":"2024-09-19T04:13:31.062270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061243+0000","last_change":"2024-09-19T04:13:28.991785+0000","last_active":"2024-09-19T04:13:31.061243+0000","last_peered":"2024-09-19T04:13:31.061243+0000","last_clean":"2024-09-19T04:13:31.061243+0000","last_became_active":"2024-09-19T04:13:28.987169+0000","last_became_peered":"2024-09-19T04:13:28.987169+0000","last_unstale":"2024-09-19T04:13:31.061243+0000","last_undegraded":"2024-09-19T04:13:31.061243+0000","last_fullsized":"2024-09-19T04:13:31.061243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:00.614358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062999+0000","last_change":"2024-09-19T04:13:23.924693+0000","last_active":"2024-09-19T04:13:31.062999+0000","last_peered":"2024-09-19T04:13:31.062999+0000","last_clean":"2024-09-19T04:13:31.062999+0000","last_became_active":"2024-09-19T04:13:23.924077+0000","last_became_peered":"2024-09-19T04:13:23.924077+0000","last_unstale":"2024-09-19T04:13:31.062999+0000","last_undegraded":"2024-09-19T04:13:31.062999+0000","last_fullsized":"2024-09-19T04:13:31.062999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:10.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970152+0000","last_change":"2024-09-19T04:13:24.948486+0000","last_active":"2024-09-19T04:13:29.970152+0000","last_peered":"2024-09-19T04:13:29.970152+0000","last_clean":"2024-09-19T04:13:29.970152+0000","last_became_active":"2024-09-19T04:13:24.947749+0000","last_became_peered":"2024-09-19T04:13:24.947749+0000","last_unstale":"2024-09-19T04:13:29.970152+0000","last_undegraded":"2024-09-19T04:13:29.970152+0000","last_fullsized":"2024-09-19T04:13:29.970152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:37.654967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063108+0000","last_change":"2024-09-19T04:13:28.992157+0000","last_active":"2024-09-19T04:13:31.063108+0000","last_peered":"2024-09-19T04:13:31.063108+0000","last_clean":"2024-09-19T04:13:31.063108+0000","last_became_active":"2024-09-19T04:13:28.990879+0000","last_became_peered":"2024-09-19T04:13:28.990879+0000","last_unstale":"2024-09-19T04:13:31.063108+0000","last_undegraded":"2024-09-19T04:13:31.063108+0000","last_fullsized":"2024-09-19T04:13:31.063108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:39.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062192+0000","last_change":"2024-09-19T04:13:23.923303+0000","last_active":"2024-09-19T04:13:31.062192+0000","last_peered":"2024-09-19T04:13:31.062192+0000","last_clean":"2024-09-19T04:13:31.062192+0000","last_became_active":"2024-09-19T04:13:23.920842+0000","last_became_peered":"2024-09-19T04:13:23.920842+0000","last_unstale":"2024-09-19T04:13:31.062192+0000","last_undegraded":"2024-09-19T04:13:31.062192+0000","last_fullsized":"2024-09-19T04:13:31.062192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:23.176892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971159+0000","last_change":"2024-09-19T04:13:24.948074+0000","last_active":"2024-09-19T04:13:29.971159+0000","last_peered":"2024-09-19T04:13:29.971159+0000","last_clean":"2024-09-19T04:13:29.971159+0000","last_became_active":"2024-09-19T04:13:24.946595+0000","last_became_peered":"2024-09-19T04:13:24.946595+0000","last_unstale":"2024-09-19T04:13:29.971159+0000","last_undegraded":"2024-09-19T04:13:29.971159+0000","last_fullsized":"2024-09-19T04:13:29.971159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:21.342803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062406+0000","last_change":"2024-09-19T04:13:28.985965+0000","last_active":"2024-09-19T04:13:31.062406+0000","last_peered":"2024-09-19T04:13:31.062406+0000","last_clean":"2024-09-19T04:13:31.062406+0000","last_became_active":"2024-09-19T04:13:28.979842+0000","last_became_peered":"2024-09-19T04:13:28.979842+0000","last_unstale":"2024-09-19T04:13:31.062406+0000","last_undegraded":"2024-09-19T04:13:31.062406+0000","last_fullsized":"2024-09-19T04:13:31.062406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:13.654228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971528+0000","last_change":"2024-09-19T04:13:24.964645+0000","last_active":"2024-09-19T04:13:29.971528+0000","last_peered":"2024-09-19T04:13:29.971528+0000","last_clean":"2024-09-19T04:13:29.971528+0000","last_became_active":"2024-09-19T04:13:24.964051+0000","last_became_peered":"2024-09-19T04:13:24.964051+0000","last_unstale":"2024-09-19T04:13:29.971528+0000","last_undegraded":"2024-09-19T04:13:29.971528+0000","last_fullsized":"2024-09-19T04:13:29.971528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062721+0000","last_change":"2024-09-19T04:13:23.921499+0000","last_active":"2024-09-19T04:13:31.062721+0000","last_peered":"2024-09-19T04:13:31.062721+0000","last_clean":"2024-09-19T04:13:31.062721+0000","last_became_active":"2024-09-19T04:13:23.921146+0000","last_became_peered":"2024-09-19T04:13:23.921146+0000","last_unstale":"2024-09-19T04:13:31.062721+0000","last_undegraded":"2024-09-19T04:13:31.062721+0000","last_fullsized":"2024-09-19T04:13:31.062721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:54.181515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063276+0000","last_change":"2024-09-19T04:13:28.979497+0000","last_active":"2024-09-19T04:13:31.063276+0000","last_peered":"2024-09-19T04:13:31.063276+0000","last_clean":"2024-09-19T04:13:31.063276+0000","last_became_active":"2024-09-19T04:13:28.978329+0000","last_became_peered":"2024-09-19T04:13:28.978329+0000","last_unstale":"2024-09-19T04:13:31.063276+0000","last_undegraded":"2024-09-19T04:13:31.063276+0000","last_fullsized":"2024-09-19T04:13:31.063276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:30.314159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300960+0000","last_change":"2024-09-19T04:13:24.951290+0000","last_active":"2024-09-19T04:13:30.300960+0000","last_peered":"2024-09-19T04:13:30.300960+0000","last_clean":"2024-09-19T04:13:30.300960+0000","last_became_active":"2024-09-19T04:13:24.950973+0000","last_became_peered":"2024-09-19T04:13:24.950973+0000","last_unstale":"2024-09-19T04:13:30.300960+0000","last_undegraded":"2024-09-19T04:13:30.300960+0000","last_fullsized":"2024-09-19T04:13:30.300960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131596+0000","last_change":"2024-09-19T04:13:23.918153+0000","last_active":"2024-09-19T04:13:30.131596+0000","last_peered":"2024-09-19T04:13:30.131596+0000","last_clean":"2024-09-19T04:13:30.131596+0000","last_became_active":"2024-09-19T04:13:23.917783+0000","last_became_peered":"2024-09-19T04:13:23.917783+0000","last_unstale":"2024-09-19T04:13:30.131596+0000","last_undegraded":"2024-09-19T04:13:30.131596+0000","last_fullsized":"2024-09-19T04:13:30.131596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063970+0000","last_change":"2024-09-19T04:13:28.988863+0000","last_active":"2024-09-19T04:13:31.063970+0000","last_peered":"2024-09-19T04:13:31.063970+0000","last_clean":"2024-09-19T04:13:31.063970+0000","last_became_active":"2024-09-19T04:13:28.987942+0000","last_became_peered":"2024-09-19T04:13:28.987942+0000","last_unstale":"2024-09-19T04:13:31.063970+0000","last_undegraded":"2024-09-19T04:13:31.063970+0000","last_fullsized":"2024-09-19T04:13:31.063970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300167+0000","last_change":"2024-09-19T04:13:24.946137+0000","last_active":"2024-09-19T04:13:30.300167+0000","last_peered":"2024-09-19T04:13:30.300167+0000","last_clean":"2024-09-19T04:13:30.300167+0000","last_became_active":"2024-09-19T04:13:24.945850+0000","last_became_peered":"2024-09-19T04:13:24.945850+0000","last_unstale":"2024-09-19T04:13:30.300167+0000","last_undegraded":"2024-09-19T04:13:30.300167+0000","last_fullsized":"2024-09-19T04:13:30.300167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:23.916109+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:23.915767+0000","last_became_peered":"2024-09-19T04:13:23.915767+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131755+0000","last_change":"2024-09-19T04:13:29.030707+0000","last_active":"2024-09-19T04:13:30.131755+0000","last_peered":"2024-09-19T04:13:30.131755+0000","last_clean":"2024-09-19T04:13:30.131755+0000","last_became_active":"2024-09-19T04:13:29.030294+0000","last_became_peered":"2024-09-19T04:13:29.030294+0000","last_unstale":"2024-09-19T04:13:30.131755+0000","last_undegraded":"2024-09-19T04:13:30.131755+0000","last_fullsized":"2024-09-19T04:13:30.131755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063152+0000","last_change":"2024-09-19T04:13:24.948148+0000","last_active":"2024-09-19T04:13:31.063152+0000","last_peered":"2024-09-19T04:13:31.063152+0000","last_clean":"2024-09-19T04:13:31.063152+0000","last_became_active":"2024-09-19T04:13:24.947330+0000","last_became_peered":"2024-09-19T04:13:24.947330+0000","last_unstale":"2024-09-19T04:13:31.063152+0000","last_undegraded":"2024-09-19T04:13:31.063152+0000","last_fullsized":"2024-09-19T04:13:31.063152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:49.624559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061085+0000","last_change":"2024-09-19T04:13:23.923583+0000","last_active":"2024-09-19T04:13:31.061085+0000","last_peered":"2024-09-19T04:13:31.061085+0000","last_clean":"2024-09-19T04:13:31.061085+0000","last_became_active":"2024-09-19T04:13:23.921593+0000","last_became_peered":"2024-09-19T04:13:23.921593+0000","last_unstale":"2024-09-19T04:13:31.061085+0000","last_undegraded":"2024-09-19T04:13:31.061085+0000","last_fullsized":"2024-09-19T04:13:31.061085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063591+0000","last_change":"2024-09-19T04:13:28.990856+0000","last_active":"2024-09-19T04:13:31.063591+0000","last_peered":"2024-09-19T04:13:31.063591+0000","last_clean":"2024-09-19T04:13:31.063591+0000","last_became_active":"2024-09-19T04:13:28.987830+0000","last_became_peered":"2024-09-19T04:13:28.987830+0000","last_unstale":"2024-09-19T04:13:31.063591+0000","last_undegraded":"2024-09-19T04:13:31.063591+0000","last_fullsized":"2024-09-19T04:13:31.063591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062632+0000","last_change":"2024-09-19T04:13:24.934798+0000","last_active":"2024-09-19T04:13:31.062632+0000","last_peered":"2024-09-19T04:13:31.062632+0000","last_clean":"2024-09-19T04:13:31.062632+0000","last_became_active":"2024-09-19T04:13:24.933977+0000","last_became_peered":"2024-09-19T04:13:24.933977+0000","last_unstale":"2024-09-19T04:13:31.062632+0000","last_undegraded":"2024-09-19T04:13:31.062632+0000","last_fullsized":"2024-09-19T04:13:31.062632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973271+0000","last_change":"2024-09-19T04:13:23.920837+0000","last_active":"2024-09-19T04:13:29.973271+0000","last_peered":"2024-09-19T04:13:29.973271+0000","last_clean":"2024-09-19T04:13:29.973271+0000","last_became_active":"2024-09-19T04:13:23.920245+0000","last_became_peered":"2024-09-19T04:13:23.920245+0000","last_unstale":"2024-09-19T04:13:29.973271+0000","last_undegraded":"2024-09-19T04:13:29.973271+0000","last_fullsized":"2024-09-19T04:13:29.973271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062572+0000","last_change":"2024-09-19T04:13:29.019257+0000","last_active":"2024-09-19T04:13:31.062572+0000","last_peered":"2024-09-19T04:13:31.062572+0000","last_clean":"2024-09-19T04:13:31.062572+0000","last_became_active":"2024-09-19T04:13:29.018425+0000","last_became_peered":"2024-09-19T04:13:29.018425+0000","last_unstale":"2024-09-19T04:13:31.062572+0000","last_undegraded":"2024-09-19T04:13:31.062572+0000","last_fullsized":"2024-09-19T04:13:31.062572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:42.657248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301533+0000","last_change":"2024-09-19T04:13:24.944069+0000","last_active":"2024-09-19T04:13:30.301533+0000","last_peered":"2024-09-19T04:13:30.301533+0000","last_clean":"2024-09-19T04:13:30.301533+0000","last_became_active":"2024-09-19T04:13:24.943774+0000","last_became_peered":"2024-09-19T04:13:24.943774+0000","last_unstale":"2024-09-19T04:13:30.301533+0000","last_undegraded":"2024-09-19T04:13:30.301533+0000","last_fullsized":"2024-09-19T04:13:30.301533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131771+0000","last_change":"2024-09-19T04:13:23.911708+0000","last_active":"2024-09-19T04:13:30.131771+0000","last_peered":"2024-09-19T04:13:30.131771+0000","last_clean":"2024-09-19T04:13:30.131771+0000","last_became_active":"2024-09-19T04:13:23.911135+0000","last_became_peered":"2024-09-19T04:13:23.911135+0000","last_unstale":"2024-09-19T04:13:30.131771+0000","last_undegraded":"2024-09-19T04:13:30.131771+0000","last_fullsized":"2024-09-19T04:13:30.131771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:48.393589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299979+0000","last_change":"2024-09-19T04:13:29.020657+0000","last_active":"2024-09-19T04:13:30.299979+0000","last_peered":"2024-09-19T04:13:30.299979+0000","last_clean":"2024-09-19T04:13:30.299979+0000","last_became_active":"2024-09-19T04:13:29.020172+0000","last_became_peered":"2024-09-19T04:13:29.020172+0000","last_unstale":"2024-09-19T04:13:30.299979+0000","last_undegraded":"2024-09-19T04:13:30.299979+0000","last_fullsized":"2024-09-19T04:13:30.299979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131405+0000","last_change":"2024-09-19T04:13:24.949008+0000","last_active":"2024-09-19T04:13:30.131405+0000","last_peered":"2024-09-19T04:13:30.131405+0000","last_clean":"2024-09-19T04:13:30.131405+0000","last_became_active":"2024-09-19T04:13:24.948513+0000","last_became_peered":"2024-09-19T04:13:24.948513+0000","last_unstale":"2024-09-19T04:13:30.131405+0000","last_undegraded":"2024-09-19T04:13:30.131405+0000","last_fullsized":"2024-09-19T04:13:30.131405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063250+0000","last_change":"2024-09-19T04:13:23.921473+0000","last_active":"2024-09-19T04:13:31.063250+0000","last_peered":"2024-09-19T04:13:31.063250+0000","last_clean":"2024-09-19T04:13:31.063250+0000","last_became_active":"2024-09-19T04:13:23.921052+0000","last_became_peered":"2024-09-19T04:13:23.921052+0000","last_unstale":"2024-09-19T04:13:31.063250+0000","last_undegraded":"2024-09-19T04:13:31.063250+0000","last_fullsized":"2024-09-19T04:13:31.063250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:17.114813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061949+0000","last_change":"2024-09-19T04:13:29.020022+0000","last_active":"2024-09-19T04:13:31.061949+0000","last_peered":"2024-09-19T04:13:31.061949+0000","last_clean":"2024-09-19T04:13:31.061949+0000","last_became_active":"2024-09-19T04:13:29.019585+0000","last_became_peered":"2024-09-19T04:13:29.019585+0000","last_unstale":"2024-09-19T04:13:31.061949+0000","last_undegraded":"2024-09-19T04:13:31.061949+0000","last_fullsized":"2024-09-19T04:13:31.061949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970941+0000","last_change":"2024-09-19T04:13:24.952102+0000","last_active":"2024-09-19T04:13:29.970941+0000","last_peered":"2024-09-19T04:13:29.970941+0000","last_clean":"2024-09-19T04:13:29.970941+0000","last_became_active":"2024-09-19T04:13:24.951184+0000","last_became_peered":"2024-09-19T04:13:24.951184+0000","last_unstale":"2024-09-19T04:13:29.970941+0000","last_undegraded":"2024-09-19T04:13:29.970941+0000","last_fullsized":"2024-09-19T04:13:29.970941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131508+0000","last_change":"2024-09-19T04:13:23.908259+0000","last_active":"2024-09-19T04:13:30.131508+0000","last_peered":"2024-09-19T04:13:30.131508+0000","last_clean":"2024-09-19T04:13:30.131508+0000","last_became_active":"2024-09-19T04:13:23.907717+0000","last_became_peered":"2024-09-19T04:13:23.907717+0000","last_unstale":"2024-09-19T04:13:30.131508+0000","last_undegraded":"2024-09-19T04:13:30.131508+0000","last_fullsized":"2024-09-19T04:13:30.131508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.975119+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.974476+0000","last_became_peered":"2024-09-19T04:13:28.974476+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063922+0000","last_change":"2024-09-19T04:13:24.922621+0000","last_active":"2024-09-19T04:13:31.063922+0000","last_peered":"2024-09-19T04:13:31.063922+0000","last_clean":"2024-09-19T04:13:31.063922+0000","last_became_active":"2024-09-19T04:13:24.921879+0000","last_became_peered":"2024-09-19T04:13:24.921879+0000","last_unstale":"2024-09-19T04:13:31.063922+0000","last_undegraded":"2024-09-19T04:13:31.063922+0000","last_fullsized":"2024-09-19T04:13:31.063922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:04.750679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972020+0000","last_change":"2024-09-19T04:13:23.901650+0000","last_active":"2024-09-19T04:13:29.972020+0000","last_peered":"2024-09-19T04:13:29.972020+0000","last_clean":"2024-09-19T04:13:29.972020+0000","last_became_active":"2024-09-19T04:13:23.900880+0000","last_became_peered":"2024-09-19T04:13:23.900880+0000","last_unstale":"2024-09-19T04:13:29.972020+0000","last_undegraded":"2024-09-19T04:13:29.972020+0000","last_fullsized":"2024-09-19T04:13:29.972020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131635+0000","last_change":"2024-09-19T04:13:28.980693+0000","last_active":"2024-09-19T04:13:30.131635+0000","last_peered":"2024-09-19T04:13:30.131635+0000","last_clean":"2024-09-19T04:13:30.131635+0000","last_became_active":"2024-09-19T04:13:28.979948+0000","last_became_peered":"2024-09-19T04:13:28.979948+0000","last_unstale":"2024-09-19T04:13:30.131635+0000","last_undegraded":"2024-09-19T04:13:30.131635+0000","last_fullsized":"2024-09-19T04:13:30.131635+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130985+0000","last_change":"2024-09-19T04:13:24.948867+0000","last_active":"2024-09-19T04:13:30.130985+0000","last_peered":"2024-09-19T04:13:30.130985+0000","last_clean":"2024-09-19T04:13:30.130985+0000","last_became_active":"2024-09-19T04:13:24.948471+0000","last_became_peered":"2024-09-19T04:13:24.948471+0000","last_unstale":"2024-09-19T04:13:30.130985+0000","last_undegraded":"2024-09-19T04:13:30.130985+0000","last_fullsized":"2024-09-19T04:13:30.130985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970768+0000","last_change":"2024-09-19T04:13:23.903592+0000","last_active":"2024-09-19T04:13:29.970768+0000","last_peered":"2024-09-19T04:13:29.970768+0000","last_clean":"2024-09-19T04:13:29.970768+0000","last_became_active":"2024-09-19T04:13:23.903306+0000","last_became_peered":"2024-09-19T04:13:23.903306+0000","last_unstale":"2024-09-19T04:13:29.970768+0000","last_undegraded":"2024-09-19T04:13:29.970768+0000","last_fullsized":"2024-09-19T04:13:29.970768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:13.135208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131834+0000","last_change":"2024-09-19T04:13:28.976039+0000","last_active":"2024-09-19T04:13:30.131834+0000","last_peered":"2024-09-19T04:13:30.131834+0000","last_clean":"2024-09-19T04:13:30.131834+0000","last_became_active":"2024-09-19T04:13:28.975424+0000","last_became_peered":"2024-09-19T04:13:28.975424+0000","last_unstale":"2024-09-19T04:13:30.131834+0000","last_undegraded":"2024-09-19T04:13:30.131834+0000","last_fullsized":"2024-09-19T04:13:30.131834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:27.255493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061535+0000","last_change":"2024-09-19T04:13:24.956224+0000","last_active":"2024-09-19T04:13:31.061535+0000","last_peered":"2024-09-19T04:13:31.061535+0000","last_clean":"2024-09-19T04:13:31.061535+0000","last_became_active":"2024-09-19T04:13:24.953799+0000","last_became_peered":"2024-09-19T04:13:24.953799+0000","last_unstale":"2024-09-19T04:13:31.061535+0000","last_undegraded":"2024-09-19T04:13:31.061535+0000","last_fullsized":"2024-09-19T04:13:31.061535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:33.543861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061237+0000","last_change":"2024-09-19T04:13:23.919493+0000","last_active":"2024-09-19T04:13:31.061237+0000","last_peered":"2024-09-19T04:13:31.061237+0000","last_clean":"2024-09-19T04:13:31.061237+0000","last_became_active":"2024-09-19T04:13:23.918702+0000","last_became_peered":"2024-09-19T04:13:23.918702+0000","last_unstale":"2024-09-19T04:13:31.061237+0000","last_undegraded":"2024-09-19T04:13:31.061237+0000","last_fullsized":"2024-09-19T04:13:31.061237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:42.592915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062499+0000","last_change":"2024-09-19T04:13:29.028357+0000","last_active":"2024-09-19T04:13:31.062499+0000","last_peered":"2024-09-19T04:13:31.062499+0000","last_clean":"2024-09-19T04:13:31.062499+0000","last_became_active":"2024-09-19T04:13:29.027946+0000","last_became_peered":"2024-09-19T04:13:29.027946+0000","last_unstale":"2024-09-19T04:13:31.062499+0000","last_undegraded":"2024-09-19T04:13:31.062499+0000","last_fullsized":"2024-09-19T04:13:31.062499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300424+0000","last_change":"2024-09-19T04:13:24.968375+0000","last_active":"2024-09-19T04:13:30.300424+0000","last_peered":"2024-09-19T04:13:30.300424+0000","last_clean":"2024-09-19T04:13:30.300424+0000","last_became_active":"2024-09-19T04:13:24.968097+0000","last_became_peered":"2024-09-19T04:13:24.968097+0000","last_unstale":"2024-09-19T04:13:30.300424+0000","last_undegraded":"2024-09-19T04:13:30.300424+0000","last_fullsized":"2024-09-19T04:13:30.300424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969947+0000","last_change":"2024-09-19T04:13:23.903638+0000","last_active":"2024-09-19T04:13:29.969947+0000","last_peered":"2024-09-19T04:13:29.969947+0000","last_clean":"2024-09-19T04:13:29.969947+0000","last_became_active":"2024-09-19T04:13:23.898948+0000","last_became_peered":"2024-09-19T04:13:23.898948+0000","last_unstale":"2024-09-19T04:13:29.969947+0000","last_undegraded":"2024-09-19T04:13:29.969947+0000","last_fullsized":"2024-09-19T04:13:29.969947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061278+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:13:31.061278+0000","last_peered":"2024-09-19T04:13:31.061278+0000","last_clean":"2024-09-19T04:13:31.061278+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:13:31.061278+0000","last_undegraded":"2024-09-19T04:13:31.061278+0000","last_fullsized":"2024-09-19T04:13:31.061278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972561+0000","last_change":"2024-09-19T04:13:29.020008+0000","last_active":"2024-09-19T04:13:29.972561+0000","last_peered":"2024-09-19T04:13:29.972561+0000","last_clean":"2024-09-19T04:13:29.972561+0000","last_became_active":"2024-09-19T04:13:29.018706+0000","last_became_peered":"2024-09-19T04:13:29.018706+0000","last_unstale":"2024-09-19T04:13:29.972561+0000","last_undegraded":"2024-09-19T04:13:29.972561+0000","last_fullsized":"2024-09-19T04:13:29.972561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062691+0000","last_change":"2024-09-19T04:13:24.966194+0000","last_active":"2024-09-19T04:13:31.062691+0000","last_peered":"2024-09-19T04:13:31.062691+0000","last_clean":"2024-09-19T04:13:31.062691+0000","last_became_active":"2024-09-19T04:13:24.965690+0000","last_became_peered":"2024-09-19T04:13:24.965690+0000","last_unstale":"2024-09-19T04:13:31.062691+0000","last_undegraded":"2024-09-19T04:13:31.062691+0000","last_fullsized":"2024-09-19T04:13:31.062691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:14.510098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971314+0000","last_change":"2024-09-19T04:13:23.901039+0000","last_active":"2024-09-19T04:13:29.971314+0000","last_peered":"2024-09-19T04:13:29.971314+0000","last_clean":"2024-09-19T04:13:29.971314+0000","last_became_active":"2024-09-19T04:13:23.900709+0000","last_became_peered":"2024-09-19T04:13:23.900709+0000","last_unstale":"2024-09-19T04:13:29.971314+0000","last_undegraded":"2024-09-19T04:13:29.971314+0000","last_fullsized":"2024-09-19T04:13:29.971314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:19:40.425738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969768+0000","last_change":"2024-09-19T04:13:29.019565+0000","last_active":"2024-09-19T04:13:29.969768+0000","last_peered":"2024-09-19T04:13:29.969768+0000","last_clean":"2024-09-19T04:13:29.969768+0000","last_became_active":"2024-09-19T04:13:29.018547+0000","last_became_peered":"2024-09-19T04:13:29.018547+0000","last_unstale":"2024-09-19T04:13:29.969768+0000","last_undegraded":"2024-09-19T04:13:29.969768+0000","last_fullsized":"2024-09-19T04:13:29.969768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970414+0000","last_change":"2024-09-19T04:13:24.953123+0000","last_active":"2024-09-19T04:13:29.970414+0000","last_peered":"2024-09-19T04:13:29.970414+0000","last_clean":"2024-09-19T04:13:29.970414+0000","last_became_active":"2024-09-19T04:13:24.950851+0000","last_became_peered":"2024-09-19T04:13:24.950851+0000","last_unstale":"2024-09-19T04:13:29.970414+0000","last_undegraded":"2024-09-19T04:13:29.970414+0000","last_fullsized":"2024-09-19T04:13:29.970414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131715+0000","last_change":"2024-09-19T04:13:23.902143+0000","last_active":"2024-09-19T04:13:30.131715+0000","last_peered":"2024-09-19T04:13:30.131715+0000","last_clean":"2024-09-19T04:13:30.131715+0000","last_became_active":"2024-09-19T04:13:23.901843+0000","last_became_peered":"2024-09-19T04:13:23.901843+0000","last_unstale":"2024-09-19T04:13:30.131715+0000","last_undegraded":"2024-09-19T04:13:30.131715+0000","last_fullsized":"2024-09-19T04:13:30.131715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:03.746896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062168+0000","last_change":"2024-09-19T04:13:28.994151+0000","last_active":"2024-09-19T04:13:31.062168+0000","last_peered":"2024-09-19T04:13:31.062168+0000","last_clean":"2024-09-19T04:13:31.062168+0000","last_became_active":"2024-09-19T04:13:28.993608+0000","last_became_peered":"2024-09-19T04:13:28.993608+0000","last_unstale":"2024-09-19T04:13:31.062168+0000","last_undegraded":"2024-09-19T04:13:31.062168+0000","last_fullsized":"2024-09-19T04:13:31.062168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064926+0000","last_change":"2024-09-19T04:13:24.966033+0000","last_active":"2024-09-19T04:13:31.064926+0000","last_peered":"2024-09-19T04:13:31.064926+0000","last_clean":"2024-09-19T04:13:31.064926+0000","last_became_active":"2024-09-19T04:13:24.962081+0000","last_became_peered":"2024-09-19T04:13:24.962081+0000","last_unstale":"2024-09-19T04:13:31.064926+0000","last_undegraded":"2024-09-19T04:13:31.064926+0000","last_fullsized":"2024-09-19T04:13:31.064926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:58.621676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971640+0000","last_change":"2024-09-19T04:13:23.911774+0000","last_active":"2024-09-19T04:13:29.971640+0000","last_peered":"2024-09-19T04:13:29.971640+0000","last_clean":"2024-09-19T04:13:29.971640+0000","last_became_active":"2024-09-19T04:13:23.911154+0000","last_became_peered":"2024-09-19T04:13:23.911154+0000","last_unstale":"2024-09-19T04:13:29.971640+0000","last_undegraded":"2024-09-19T04:13:29.971640+0000","last_fullsized":"2024-09-19T04:13:29.971640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969327+0000","last_change":"2024-09-19T04:13:29.020613+0000","last_active":"2024-09-19T04:13:29.969327+0000","last_peered":"2024-09-19T04:13:29.969327+0000","last_clean":"2024-09-19T04:13:29.969327+0000","last_became_active":"2024-09-19T04:13:29.019473+0000","last_became_peered":"2024-09-19T04:13:29.019473+0000","last_unstale":"2024-09-19T04:13:29.969327+0000","last_undegraded":"2024-09-19T04:13:29.969327+0000","last_fullsized":"2024-09-19T04:13:29.969327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064591+0000","last_change":"2024-09-19T04:13:24.962284+0000","last_active":"2024-09-19T04:13:31.064591+0000","last_peered":"2024-09-19T04:13:31.064591+0000","last_clean":"2024-09-19T04:13:31.064591+0000","last_became_active":"2024-09-19T04:13:24.961763+0000","last_became_peered":"2024-09-19T04:13:24.961763+0000","last_unstale":"2024-09-19T04:13:31.064591+0000","last_undegraded":"2024-09-19T04:13:31.064591+0000","last_fullsized":"2024-09-19T04:13:31.064591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:40.924861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064080+0000","last_change":"2024-09-19T04:13:23.905574+0000","last_active":"2024-09-19T04:13:31.064080+0000","last_peered":"2024-09-19T04:13:31.064080+0000","last_clean":"2024-09-19T04:13:31.064080+0000","last_became_active":"2024-09-19T04:13:23.904993+0000","last_became_peered":"2024-09-19T04:13:23.904993+0000","last_unstale":"2024-09-19T04:13:31.064080+0000","last_undegraded":"2024-09-19T04:13:31.064080+0000","last_fullsized":"2024-09-19T04:13:31.064080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:00.615690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061135+0000","last_change":"2024-09-19T04:13:28.991444+0000","last_active":"2024-09-19T04:13:31.061135+0000","last_peered":"2024-09-19T04:13:31.061135+0000","last_clean":"2024-09-19T04:13:31.061135+0000","last_became_active":"2024-09-19T04:13:28.987837+0000","last_became_peered":"2024-09-19T04:13:28.987837+0000","last_unstale":"2024-09-19T04:13:31.061135+0000","last_undegraded":"2024-09-19T04:13:31.061135+0000","last_fullsized":"2024-09-19T04:13:31.061135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:17.703055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061563+0000","last_change":"2024-09-19T04:13:24.965972+0000","last_active":"2024-09-19T04:13:31.061563+0000","last_peered":"2024-09-19T04:13:31.061563+0000","last_clean":"2024-09-19T04:13:31.061563+0000","last_became_active":"2024-09-19T04:13:24.965315+0000","last_became_peered":"2024-09-19T04:13:24.965315+0000","last_unstale":"2024-09-19T04:13:31.061563+0000","last_undegraded":"2024-09-19T04:13:31.061563+0000","last_fullsized":"2024-09-19T04:13:31.061563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:16.174335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970388+0000","last_change":"2024-09-19T04:13:23.896881+0000","last_active":"2024-09-19T04:13:29.970388+0000","last_peered":"2024-09-19T04:13:29.970388+0000","last_clean":"2024-09-19T04:13:29.970388+0000","last_became_active":"2024-09-19T04:13:23.896440+0000","last_became_peered":"2024-09-19T04:13:23.896440+0000","last_unstale":"2024-09-19T04:13:29.970388+0000","last_undegraded":"2024-09-19T04:13:29.970388+0000","last_fullsized":"2024-09-19T04:13:29.970388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:36:56.677455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062482+0000","last_change":"2024-09-19T04:13:28.985691+0000","last_active":"2024-09-19T04:13:31.062482+0000","last_peered":"2024-09-19T04:13:31.062482+0000","last_clean":"2024-09-19T04:13:31.062482+0000","last_became_active":"2024-09-19T04:13:28.984749+0000","last_became_peered":"2024-09-19T04:13:28.984749+0000","last_unstale":"2024-09-19T04:13:31.062482+0000","last_undegraded":"2024-09-19T04:13:31.062482+0000","last_fullsized":"2024-09-19T04:13:31.062482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969844+0000","last_change":"2024-09-19T04:13:24.942591+0000","last_active":"2024-09-19T04:13:29.969844+0000","last_peered":"2024-09-19T04:13:29.969844+0000","last_clean":"2024-09-19T04:13:29.969844+0000","last_became_active":"2024-09-19T04:13:24.941789+0000","last_became_peered":"2024-09-19T04:13:24.941789+0000","last_unstale":"2024-09-19T04:13:29.969844+0000","last_undegraded":"2024-09-19T04:13:29.969844+0000","last_fullsized":"2024-09-19T04:13:29.969844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970255+0000","last_change":"2024-09-19T04:13:23.916881+0000","last_active":"2024-09-19T04:13:29.970255+0000","last_peered":"2024-09-19T04:13:29.970255+0000","last_clean":"2024-09-19T04:13:29.970255+0000","last_became_active":"2024-09-19T04:13:23.916050+0000","last_became_peered":"2024-09-19T04:13:23.916050+0000","last_unstale":"2024-09-19T04:13:29.970255+0000","last_undegraded":"2024-09-19T04:13:29.970255+0000","last_fullsized":"2024-09-19T04:13:29.970255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301470+0000","last_change":"2024-09-19T04:13:29.021093+0000","last_active":"2024-09-19T04:13:30.301470+0000","last_peered":"2024-09-19T04:13:30.301470+0000","last_clean":"2024-09-19T04:13:30.301470+0000","last_became_active":"2024-09-19T04:13:29.020573+0000","last_became_peered":"2024-09-19T04:13:29.020573+0000","last_unstale":"2024-09-19T04:13:30.301470+0000","last_undegraded":"2024-09-19T04:13:30.301470+0000","last_fullsized":"2024-09-19T04:13:30.301470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:35.622467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062500+0000","last_change":"2024-09-19T04:13:24.968018+0000","last_active":"2024-09-19T04:13:31.062500+0000","last_peered":"2024-09-19T04:13:31.062500+0000","last_clean":"2024-09-19T04:13:31.062500+0000","last_became_active":"2024-09-19T04:13:24.962386+0000","last_became_peered":"2024-09-19T04:13:24.962386+0000","last_unstale":"2024-09-19T04:13:31.062500+0000","last_undegraded":"2024-09-19T04:13:31.062500+0000","last_fullsized":"2024-09-19T04:13:31.062500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:39.665883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971777+0000","last_change":"2024-09-19T04:13:23.911883+0000","last_active":"2024-09-19T04:13:29.971777+0000","last_peered":"2024-09-19T04:13:29.971777+0000","last_clean":"2024-09-19T04:13:29.971777+0000","last_became_active":"2024-09-19T04:13:23.910990+0000","last_became_peered":"2024-09-19T04:13:23.910990+0000","last_unstale":"2024-09-19T04:13:29.971777+0000","last_undegraded":"2024-09-19T04:13:29.971777+0000","last_fullsized":"2024-09-19T04:13:29.971777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301624+0000","last_change":"2024-09-19T04:13:29.017846+0000","last_active":"2024-09-19T04:13:30.301624+0000","last_peered":"2024-09-19T04:13:30.301624+0000","last_clean":"2024-09-19T04:13:30.301624+0000","last_became_active":"2024-09-19T04:13:29.017269+0000","last_became_peered":"2024-09-19T04:13:29.017269+0000","last_unstale":"2024-09-19T04:13:30.301624+0000","last_undegraded":"2024-09-19T04:13:30.301624+0000","last_fullsized":"2024-09-19T04:13:30.301624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:07.730896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061954+0000","last_change":"2024-09-19T04:13:24.944986+0000","last_active":"2024-09-19T04:13:31.061954+0000","last_peered":"2024-09-19T04:13:31.061954+0000","last_clean":"2024-09-19T04:13:31.061954+0000","last_became_active":"2024-09-19T04:13:24.944266+0000","last_became_peered":"2024-09-19T04:13:24.944266+0000","last_unstale":"2024-09-19T04:13:31.061954+0000","last_undegraded":"2024-09-19T04:13:31.061954+0000","last_fullsized":"2024-09-19T04:13:31.061954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:17.729114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062229+0000","last_change":"2024-09-19T04:13:23.902337+0000","last_active":"2024-09-19T04:13:31.062229+0000","last_peered":"2024-09-19T04:13:31.062229+0000","last_clean":"2024-09-19T04:13:31.062229+0000","last_became_active":"2024-09-19T04:13:23.901958+0000","last_became_peered":"2024-09-19T04:13:23.901958+0000","last_unstale":"2024-09-19T04:13:31.062229+0000","last_undegraded":"2024-09-19T04:13:31.062229+0000","last_fullsized":"2024-09-19T04:13:31.062229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:39.987613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970753+0000","last_change":"2024-09-19T04:13:29.022266+0000","last_active":"2024-09-19T04:13:29.970753+0000","last_peered":"2024-09-19T04:13:29.970753+0000","last_clean":"2024-09-19T04:13:29.970753+0000","last_became_active":"2024-09-19T04:13:29.021104+0000","last_became_peered":"2024-09-19T04:13:29.021104+0000","last_unstale":"2024-09-19T04:13:29.970753+0000","last_undegraded":"2024-09-19T04:13:29.970753+0000","last_fullsized":"2024-09-19T04:13:29.970753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062460+0000","last_change":"2024-09-19T04:13:24.952759+0000","last_active":"2024-09-19T04:13:31.062460+0000","last_peered":"2024-09-19T04:13:31.062460+0000","last_clean":"2024-09-19T04:13:31.062460+0000","last_became_active":"2024-09-19T04:13:24.952088+0000","last_became_peered":"2024-09-19T04:13:24.952088+0000","last_unstale":"2024-09-19T04:13:31.062460+0000","last_undegraded":"2024-09-19T04:13:31.062460+0000","last_fullsized":"2024-09-19T04:13:31.062460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:29.020474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:35.460051+0000","last_change":"2024-09-19T04:13:23.909991+0000","last_active":"2024-09-19T04:13:35.460051+0000","last_peered":"2024-09-19T04:13:35.460051+0000","last_clean":"2024-09-19T04:13:35.460051+0000","last_became_active":"2024-09-19T04:13:23.909517+0000","last_became_peered":"2024-09-19T04:13:23.909517+0000","last_unstale":"2024-09-19T04:13:35.460051+0000","last_undegraded":"2024-09-19T04:13:35.460051+0000","last_fullsized":"2024-09-19T04:13:35.460051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.984556+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.983961+0000","last_became_peered":"2024-09-19T04:13:28.983961+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064342+0000","last_change":"2024-09-19T04:13:24.943383+0000","last_active":"2024-09-19T04:13:31.064342+0000","last_peered":"2024-09-19T04:13:31.064342+0000","last_clean":"2024-09-19T04:13:31.064342+0000","last_became_active":"2024-09-19T04:13:24.942094+0000","last_became_peered":"2024-09-19T04:13:24.942094+0000","last_unstale":"2024-09-19T04:13:31.064342+0000","last_undegraded":"2024-09-19T04:13:31.064342+0000","last_fullsized":"2024-09-19T04:13:31.064342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:15.642865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131674+0000","last_change":"2024-09-19T04:13:23.911338+0000","last_active":"2024-09-19T04:13:30.131674+0000","last_peered":"2024-09-19T04:13:30.131674+0000","last_clean":"2024-09-19T04:13:30.131674+0000","last_became_active":"2024-09-19T04:13:23.910962+0000","last_became_peered":"2024-09-19T04:13:23.910962+0000","last_unstale":"2024-09-19T04:13:30.131674+0000","last_undegraded":"2024-09-19T04:13:30.131674+0000","last_fullsized":"2024-09-19T04:13:30.131674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:39.854426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063766+0000","last_change":"2024-09-19T04:13:28.988142+0000","last_active":"2024-09-19T04:13:31.063766+0000","last_peered":"2024-09-19T04:13:31.063766+0000","last_clean":"2024-09-19T04:13:31.063766+0000","last_became_active":"2024-09-19T04:13:28.979829+0000","last_became_peered":"2024-09-19T04:13:28.979829+0000","last_unstale":"2024-09-19T04:13:31.063766+0000","last_undegraded":"2024-09-19T04:13:31.063766+0000","last_fullsized":"2024-09-19T04:13:31.063766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062345+0000","last_change":"2024-09-19T04:13:24.956553+0000","last_active":"2024-09-19T04:13:31.062345+0000","last_peered":"2024-09-19T04:13:31.062345+0000","last_clean":"2024-09-19T04:13:31.062345+0000","last_became_active":"2024-09-19T04:13:24.956238+0000","last_became_peered":"2024-09-19T04:13:24.956238+0000","last_unstale":"2024-09-19T04:13:31.062345+0000","last_undegraded":"2024-09-19T04:13:31.062345+0000","last_fullsized":"2024-09-19T04:13:31.062345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970527+0000","last_change":"2024-09-19T04:13:23.918490+0000","last_active":"2024-09-19T04:13:29.970527+0000","last_peered":"2024-09-19T04:13:29.970527+0000","last_clean":"2024-09-19T04:13:29.970527+0000","last_became_active":"2024-09-19T04:13:23.917969+0000","last_became_peered":"2024-09-19T04:13:23.917969+0000","last_unstale":"2024-09-19T04:13:29.970527+0000","last_undegraded":"2024-09-19T04:13:29.970527+0000","last_fullsized":"2024-09-19T04:13:29.970527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300379+0000","last_change":"2024-09-19T04:13:28.988707+0000","last_active":"2024-09-19T04:13:30.300379+0000","last_peered":"2024-09-19T04:13:30.300379+0000","last_clean":"2024-09-19T04:13:30.300379+0000","last_became_active":"2024-09-19T04:13:28.988388+0000","last_became_peered":"2024-09-19T04:13:28.988388+0000","last_unstale":"2024-09-19T04:13:30.300379+0000","last_undegraded":"2024-09-19T04:13:30.300379+0000","last_fullsized":"2024-09-19T04:13:30.300379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:54.745179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970310+0000","last_change":"2024-09-19T04:13:24.943308+0000","last_active":"2024-09-19T04:13:29.970310+0000","last_peered":"2024-09-19T04:13:29.970310+0000","last_clean":"2024-09-19T04:13:29.970310+0000","last_became_active":"2024-09-19T04:13:24.942608+0000","last_became_peered":"2024-09-19T04:13:24.942608+0000","last_unstale":"2024-09-19T04:13:29.970310+0000","last_undegraded":"2024-09-19T04:13:29.970310+0000","last_fullsized":"2024-09-19T04:13:29.970310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972997+0000","last_change":"2024-09-19T04:13:23.918096+0000","last_active":"2024-09-19T04:13:29.972997+0000","last_peered":"2024-09-19T04:13:29.972997+0000","last_clean":"2024-09-19T04:13:29.972997+0000","last_became_active":"2024-09-19T04:13:23.917505+0000","last_became_peered":"2024-09-19T04:13:23.917505+0000","last_unstale":"2024-09-19T04:13:29.972997+0000","last_undegraded":"2024-09-19T04:13:29.972997+0000","last_fullsized":"2024-09-19T04:13:29.972997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300417+0000","last_change":"2024-09-19T04:13:29.027451+0000","last_active":"2024-09-19T04:13:30.300417+0000","last_peered":"2024-09-19T04:13:30.300417+0000","last_clean":"2024-09-19T04:13:30.300417+0000","last_became_active":"2024-09-19T04:13:29.027181+0000","last_became_peered":"2024-09-19T04:13:29.027181+0000","last_unstale":"2024-09-19T04:13:30.300417+0000","last_undegraded":"2024-09-19T04:13:30.300417+0000","last_fullsized":"2024-09-19T04:13:30.300417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:42.206935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062147+0000","last_change":"2024-09-19T04:13:29.028793+0000","last_active":"2024-09-19T04:13:31.062147+0000","last_peered":"2024-09-19T04:13:31.062147+0000","last_clean":"2024-09-19T04:13:31.062147+0000","last_became_active":"2024-09-19T04:13:29.028293+0000","last_became_peered":"2024-09-19T04:13:29.028293+0000","last_unstale":"2024-09-19T04:13:31.062147+0000","last_undegraded":"2024-09-19T04:13:31.062147+0000","last_fullsized":"2024-09-19T04:13:31.062147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130915+0000","last_change":"2024-09-19T04:13:24.947415+0000","last_active":"2024-09-19T04:13:30.130915+0000","last_peered":"2024-09-19T04:13:30.130915+0000","last_clean":"2024-09-19T04:13:30.130915+0000","last_became_active":"2024-09-19T04:13:24.947042+0000","last_became_peered":"2024-09-19T04:13:24.947042+0000","last_unstale":"2024-09-19T04:13:30.130915+0000","last_undegraded":"2024-09-19T04:13:30.130915+0000","last_fullsized":"2024-09-19T04:13:30.130915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969735+0000","last_change":"2024-09-19T04:13:23.906982+0000","last_active":"2024-09-19T04:13:29.969735+0000","last_peered":"2024-09-19T04:13:29.969735+0000","last_clean":"2024-09-19T04:13:29.969735+0000","last_became_active":"2024-09-19T04:13:23.905070+0000","last_became_peered":"2024-09-19T04:13:23.905070+0000","last_unstale":"2024-09-19T04:13:29.969735+0000","last_undegraded":"2024-09-19T04:13:29.969735+0000","last_fullsized":"2024-09-19T04:13:29.969735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972911+0000","last_change":"2024-09-19T04:13:28.979353+0000","last_active":"2024-09-19T04:13:29.972911+0000","last_peered":"2024-09-19T04:13:29.972911+0000","last_clean":"2024-09-19T04:13:29.972911+0000","last_became_active":"2024-09-19T04:13:28.976991+0000","last_became_peered":"2024-09-19T04:13:28.976991+0000","last_unstale":"2024-09-19T04:13:29.972911+0000","last_undegraded":"2024-09-19T04:13:29.972911+0000","last_fullsized":"2024-09-19T04:13:29.972911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300060+0000","last_change":"2024-09-19T04:13:24.947190+0000","last_active":"2024-09-19T04:13:30.300060+0000","last_peered":"2024-09-19T04:13:30.300060+0000","last_clean":"2024-09-19T04:13:30.300060+0000","last_became_active":"2024-09-19T04:13:24.946883+0000","last_became_peered":"2024-09-19T04:13:24.946883+0000","last_unstale":"2024-09-19T04:13:30.300060+0000","last_undegraded":"2024-09-19T04:13:30.300060+0000","last_fullsized":"2024-09-19T04:13:30.300060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300792+0000","last_change":"2024-09-19T04:13:23.912532+0000","last_active":"2024-09-19T04:13:30.300792+0000","last_peered":"2024-09-19T04:13:30.300792+0000","last_clean":"2024-09-19T04:13:30.300792+0000","last_became_active":"2024-09-19T04:13:23.911952+0000","last_became_peered":"2024-09-19T04:13:23.911952+0000","last_unstale":"2024-09-19T04:13:30.300792+0000","last_undegraded":"2024-09-19T04:13:30.300792+0000","last_fullsized":"2024-09-19T04:13:30.300792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971798+0000","last_change":"2024-09-19T04:13:23.912898+0000","last_active":"2024-09-19T04:13:29.971798+0000","last_peered":"2024-09-19T04:13:29.971798+0000","last_clean":"2024-09-19T04:13:29.971798+0000","last_became_active":"2024-09-19T04:13:23.912234+0000","last_became_peered":"2024-09-19T04:13:23.912234+0000","last_unstale":"2024-09-19T04:13:29.971798+0000","last_undegraded":"2024-09-19T04:13:29.971798+0000","last_fullsized":"2024-09-19T04:13:29.971798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062138+0000","last_change":"2024-09-19T04:13:24.953761+0000","last_active":"2024-09-19T04:13:31.062138+0000","last_peered":"2024-09-19T04:13:31.062138+0000","last_clean":"2024-09-19T04:13:31.062138+0000","last_became_active":"2024-09-19T04:13:24.953195+0000","last_became_peered":"2024-09-19T04:13:24.953195+0000","last_unstale":"2024-09-19T04:13:31.062138+0000","last_undegraded":"2024-09-19T04:13:31.062138+0000","last_fullsized":"2024-09-19T04:13:31.062138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971169+0000","last_change":"2024-09-19T04:13:28.976510+0000","last_active":"2024-09-19T04:13:29.971169+0000","last_peered":"2024-09-19T04:13:29.971169+0000","last_clean":"2024-09-19T04:13:29.971169+0000","last_became_active":"2024-09-19T04:13:28.975842+0000","last_became_peered":"2024-09-19T04:13:28.975842+0000","last_unstale":"2024-09-19T04:13:29.971169+0000","last_undegraded":"2024-09-19T04:13:29.971169+0000","last_fullsized":"2024-09-19T04:13:29.971169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:36:39.502090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301204+0000","last_change":"2024-09-19T04:13:23.920382+0000","last_active":"2024-09-19T04:13:30.301204+0000","last_peered":"2024-09-19T04:13:30.301204+0000","last_clean":"2024-09-19T04:13:30.301204+0000","last_became_active":"2024-09-19T04:13:23.920107+0000","last_became_peered":"2024-09-19T04:13:23.920107+0000","last_unstale":"2024-09-19T04:13:30.301204+0000","last_undegraded":"2024-09-19T04:13:30.301204+0000","last_fullsized":"2024-09-19T04:13:30.301204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:24.725046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062882+0000","last_change":"2024-09-19T04:13:24.933838+0000","last_active":"2024-09-19T04:13:31.062882+0000","last_peered":"2024-09-19T04:13:31.062882+0000","last_clean":"2024-09-19T04:13:31.062882+0000","last_became_active":"2024-09-19T04:13:24.932423+0000","last_became_peered":"2024-09-19T04:13:24.932423+0000","last_unstale":"2024-09-19T04:13:31.062882+0000","last_undegraded":"2024-09-19T04:13:31.062882+0000","last_fullsized":"2024-09-19T04:13:31.062882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:52.498071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301388+0000","last_change":"2024-09-19T04:13:28.982366+0000","last_active":"2024-09-19T04:13:30.301388+0000","last_peered":"2024-09-19T04:13:30.301388+0000","last_clean":"2024-09-19T04:13:30.301388+0000","last_became_active":"2024-09-19T04:13:28.981608+0000","last_became_peered":"2024-09-19T04:13:28.981608+0000","last_unstale":"2024-09-19T04:13:30.301388+0000","last_undegraded":"2024-09-19T04:13:30.301388+0000","last_fullsized":"2024-09-19T04:13:30.301388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:12.864952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969437+0000","last_change":"2024-09-19T04:13:24.939240+0000","last_active":"2024-09-19T04:13:29.969437+0000","last_peered":"2024-09-19T04:13:29.969437+0000","last_clean":"2024-09-19T04:13:29.969437+0000","last_became_active":"2024-09-19T04:13:24.938527+0000","last_became_peered":"2024-09-19T04:13:24.938527+0000","last_unstale":"2024-09-19T04:13:29.969437+0000","last_undegraded":"2024-09-19T04:13:29.969437+0000","last_fullsized":"2024-09-19T04:13:29.969437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970473+0000","last_change":"2024-09-19T04:13:24.967773+0000","last_active":"2024-09-19T04:13:29.970473+0000","last_peered":"2024-09-19T04:13:29.970473+0000","last_clean":"2024-09-19T04:13:29.970473+0000","last_became_active":"2024-09-19T04:13:24.967467+0000","last_became_peered":"2024-09-19T04:13:24.967467+0000","last_unstale":"2024-09-19T04:13:29.970473+0000","last_undegraded":"2024-09-19T04:13:29.970473+0000","last_fullsized":"2024-09-19T04:13:29.970473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:11.155011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971529+0000","last_change":"2024-09-19T04:13:24.945017+0000","last_active":"2024-09-19T04:13:29.971529+0000","last_peered":"2024-09-19T04:13:29.971529+0000","last_clean":"2024-09-19T04:13:29.971529+0000","last_became_active":"2024-09-19T04:13:24.944535+0000","last_became_peered":"2024-09-19T04:13:24.944535+0000","last_unstale":"2024-09-19T04:13:29.971529+0000","last_undegraded":"2024-09-19T04:13:29.971529+0000","last_fullsized":"2024-09-19T04:13:29.971529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:43.596741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062090+0000","last_change":"2024-09-19T04:13:24.952497+0000","last_active":"2024-09-19T04:13:31.062090+0000","last_peered":"2024-09-19T04:13:31.062090+0000","last_clean":"2024-09-19T04:13:31.062090+0000","last_became_active":"2024-09-19T04:13:24.952037+0000","last_became_peered":"2024-09-19T04:13:24.952037+0000","last_unstale":"2024-09-19T04:13:31.062090+0000","last_undegraded":"2024-09-19T04:13:31.062090+0000","last_fullsized":"2024-09-19T04:13:31.062090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301786+0000","last_change":"2024-09-19T04:13:24.966504+0000","last_active":"2024-09-19T04:13:30.301786+0000","last_peered":"2024-09-19T04:13:30.301786+0000","last_clean":"2024-09-19T04:13:30.301786+0000","last_became_active":"2024-09-19T04:13:24.965992+0000","last_became_peered":"2024-09-19T04:13:24.965992+0000","last_unstale":"2024-09-19T04:13:30.301786+0000","last_undegraded":"2024-09-19T04:13:30.301786+0000","last_fullsized":"2024-09-19T04:13:30.301786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131526+0000","last_change":"2024-09-19T04:13:24.949970+0000","last_active":"2024-09-19T04:13:30.131526+0000","last_peered":"2024-09-19T04:13:30.131526+0000","last_clean":"2024-09-19T04:13:30.131526+0000","last_became_active":"2024-09-19T04:13:24.949676+0000","last_became_peered":"2024-09-19T04:13:24.949676+0000","last_unstale":"2024-09-19T04:13:30.131526+0000","last_undegraded":"2024-09-19T04:13:30.131526+0000","last_fullsized":"2024-09-19T04:13:30.131526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973014+0000","last_change":"2024-09-19T04:13:24.939046+0000","last_active":"2024-09-19T04:13:29.973014+0000","last_peered":"2024-09-19T04:13:29.973014+0000","last_clean":"2024-09-19T04:13:29.973014+0000","last_became_active":"2024-09-19T04:13:24.938277+0000","last_became_peered":"2024-09-19T04:13:24.938277+0000","last_unstale":"2024-09-19T04:13:29.973014+0000","last_undegraded":"2024-09-19T04:13:29.973014+0000","last_fullsized":"2024-09-19T04:13:29.973014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301516+0000","last_change":"2024-09-19T04:13:24.962572+0000","last_active":"2024-09-19T04:13:30.301516+0000","last_peered":"2024-09-19T04:13:30.301516+0000","last_clean":"2024-09-19T04:13:30.301516+0000","last_became_active":"2024-09-19T04:13:24.962277+0000","last_became_peered":"2024-09-19T04:13:24.962277+0000","last_unstale":"2024-09-19T04:13:30.301516+0000","last_undegraded":"2024-09-19T04:13:30.301516+0000","last_fullsized":"2024-09-19T04:13:30.301516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969487+0000","last_change":"2024-09-19T04:13:24.951536+0000","last_active":"2024-09-19T04:13:29.969487+0000","last_peered":"2024-09-19T04:13:29.969487+0000","last_clean":"2024-09-19T04:13:29.969487+0000","last_became_active":"2024-09-19T04:13:24.950939+0000","last_became_peered":"2024-09-19T04:13:24.950939+0000","last_unstale":"2024-09-19T04:13:29.969487+0000","last_undegraded":"2024-09-19T04:13:29.969487+0000","last_fullsized":"2024-09-19T04:13:29.969487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300567+0000","last_change":"2024-09-19T04:13:24.961761+0000","last_active":"2024-09-19T04:13:30.300567+0000","last_peered":"2024-09-19T04:13:30.300567+0000","last_clean":"2024-09-19T04:13:30.300567+0000","last_became_active":"2024-09-19T04:13:24.961415+0000","last_became_peered":"2024-09-19T04:13:24.961415+0000","last_unstale":"2024-09-19T04:13:30.300567+0000","last_undegraded":"2024-09-19T04:13:30.300567+0000","last_fullsized":"2024-09-19T04:13:30.300567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973133+0000","last_change":"2024-09-19T04:13:24.939730+0000","last_active":"2024-09-19T04:13:29.973133+0000","last_peered":"2024-09-19T04:13:29.973133+0000","last_clean":"2024-09-19T04:13:29.973133+0000","last_became_active":"2024-09-19T04:13:24.938400+0000","last_became_peered":"2024-09-19T04:13:24.938400+0000","last_unstale":"2024-09-19T04:13:29.973133+0000","last_undegraded":"2024-09-19T04:13:29.973133+0000","last_fullsized":"2024-09-19T04:13:29.973133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062008+0000","last_change":"2024-09-19T04:13:24.944592+0000","last_active":"2024-09-19T04:13:31.062008+0000","last_peered":"2024-09-19T04:13:31.062008+0000","last_clean":"2024-09-19T04:13:31.062008+0000","last_became_active":"2024-09-19T04:13:24.943915+0000","last_became_peered":"2024-09-19T04:13:24.943915+0000","last_unstale":"2024-09-19T04:13:31.062008+0000","last_undegraded":"2024-09-19T04:13:31.062008+0000","last_fullsized":"2024-09-19T04:13:31.062008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972579+0000","last_change":"2024-09-19T04:13:24.967058+0000","last_active":"2024-09-19T04:13:29.972579+0000","last_peered":"2024-09-19T04:13:29.972579+0000","last_clean":"2024-09-19T04:13:29.972579+0000","last_became_active":"2024-09-19T04:13:24.964521+0000","last_became_peered":"2024-09-19T04:13:24.964521+0000","last_unstale":"2024-09-19T04:13:29.972579+0000","last_undegraded":"2024-09-19T04:13:29.972579+0000","last_fullsized":"2024-09-19T04:13:29.972579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063128+0000","last_change":"2024-09-19T04:13:24.953833+0000","last_active":"2024-09-19T04:13:31.063128+0000","last_peered":"2024-09-19T04:13:31.063128+0000","last_clean":"2024-09-19T04:13:31.063128+0000","last_became_active":"2024-09-19T04:13:24.950878+0000","last_became_peered":"2024-09-19T04:13:24.950878+0000","last_unstale":"2024-09-19T04:13:31.063128+0000","last_undegraded":"2024-09-19T04:13:31.063128+0000","last_fullsized":"2024-09-19T04:13:31.063128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:07.653424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971483+0000","last_change":"2024-09-19T04:13:24.956195+0000","last_active":"2024-09-19T04:13:29.971483+0000","last_peered":"2024-09-19T04:13:29.971483+0000","last_clean":"2024-09-19T04:13:29.971483+0000","last_became_active":"2024-09-19T04:13:24.955911+0000","last_became_peered":"2024-09-19T04:13:24.955911+0000","last_unstale":"2024-09-19T04:13:29.971483+0000","last_undegraded":"2024-09-19T04:13:29.971483+0000","last_fullsized":"2024-09-19T04:13:29.971483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:55.185297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970952+0000","last_change":"2024-09-19T04:13:24.939650+0000","last_active":"2024-09-19T04:13:29.970952+0000","last_peered":"2024-09-19T04:13:29.970952+0000","last_clean":"2024-09-19T04:13:29.970952+0000","last_became_active":"2024-09-19T04:13:24.938774+0000","last_became_peered":"2024-09-19T04:13:24.938774+0000","last_unstale":"2024-09-19T04:13:29.970952+0000","last_undegraded":"2024-09-19T04:13:29.970952+0000","last_fullsized":"2024-09-19T04:13:29.970952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:00.871440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969219+0000","last_change":"2024-09-19T04:13:24.921407+0000","last_active":"2024-09-19T04:13:29.969219+0000","last_peered":"2024-09-19T04:13:29.969219+0000","last_clean":"2024-09-19T04:13:29.969219+0000","last_became_active":"2024-09-19T04:13:24.920646+0000","last_became_peered":"2024-09-19T04:13:24.920646+0000","last_unstale":"2024-09-19T04:13:29.969219+0000","last_undegraded":"2024-09-19T04:13:29.969219+0000","last_fullsized":"2024-09-19T04:13:29.969219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:24.963230+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:24.962932+0000","last_became_peered":"2024-09-19T04:13:24.962932+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061900+0000","last_change":"2024-09-19T04:13:24.942848+0000","last_active":"2024-09-19T04:13:31.061900+0000","last_peered":"2024-09-19T04:13:31.061900+0000","last_clean":"2024-09-19T04:13:31.061900+0000","last_became_active":"2024-09-19T04:13:24.942524+0000","last_became_peered":"2024-09-19T04:13:24.942524+0000","last_unstale":"2024-09-19T04:13:31.061900+0000","last_undegraded":"2024-09-19T04:13:31.061900+0000","last_fullsized":"2024-09-19T04:13:31.061900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:26.370075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300722+0000","last_change":"2024-09-19T04:13:24.950629+0000","last_active":"2024-09-19T04:13:30.300722+0000","last_peered":"2024-09-19T04:13:30.300722+0000","last_clean":"2024-09-19T04:13:30.300722+0000","last_became_active":"2024-09-19T04:13:24.946193+0000","last_became_peered":"2024-09-19T04:13:24.946193+0000","last_unstale":"2024-09-19T04:13:30.300722+0000","last_undegraded":"2024-09-19T04:13:30.300722+0000","last_fullsized":"2024-09-19T04:13:30.300722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064210+0000","last_change":"2024-09-19T04:13:24.936924+0000","last_active":"2024-09-19T04:13:31.064210+0000","last_peered":"2024-09-19T04:13:31.064210+0000","last_clean":"2024-09-19T04:13:31.064210+0000","last_became_active":"2024-09-19T04:13:24.936053+0000","last_became_peered":"2024-09-19T04:13:24.936053+0000","last_unstale":"2024-09-19T04:13:31.064210+0000","last_undegraded":"2024-09-19T04:13:31.064210+0000","last_fullsized":"2024-09-19T04:13:31.064210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:40.858208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972557+0000","last_change":"2024-09-19T04:13:24.967121+0000","last_active":"2024-09-19T04:13:29.972557+0000","last_peered":"2024-09-19T04:13:29.972557+0000","last_clean":"2024-09-19T04:13:29.972557+0000","last_became_active":"2024-09-19T04:13:24.964161+0000","last_became_peered":"2024-09-19T04:13:24.964161+0000","last_unstale":"2024-09-19T04:13:29.972557+0000","last_undegraded":"2024-09-19T04:13:29.972557+0000","last_fullsized":"2024-09-19T04:13:29.972557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972765+0000","last_change":"2024-09-19T04:13:24.968947+0000","last_active":"2024-09-19T04:13:29.972765+0000","last_peered":"2024-09-19T04:13:29.972765+0000","last_clean":"2024-09-19T04:13:29.972765+0000","last_became_active":"2024-09-19T04:13:24.968335+0000","last_became_peered":"2024-09-19T04:13:24.968335+0000","last_unstale":"2024-09-19T04:13:29.972765+0000","last_undegraded":"2024-09-19T04:13:29.972765+0000","last_fullsized":"2024-09-19T04:13:29.972765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064806+0000","last_change":"2024-09-19T04:13:24.970001+0000","last_active":"2024-09-19T04:13:31.064806+0000","last_peered":"2024-09-19T04:13:31.064806+0000","last_clean":"2024-09-19T04:13:31.064806+0000","last_became_active":"2024-09-19T04:13:24.969462+0000","last_became_peered":"2024-09-19T04:13:24.969462+0000","last_unstale":"2024-09-19T04:13:31.064806+0000","last_undegraded":"2024-09-19T04:13:31.064806+0000","last_fullsized":"2024-09-19T04:13:31.064806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:15:32.140847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299773+0000","last_change":"2024-09-19T04:13:24.967702+0000","last_active":"2024-09-19T04:13:30.299773+0000","last_peered":"2024-09-19T04:13:30.299773+0000","last_clean":"2024-09-19T04:13:30.299773+0000","last_became_active":"2024-09-19T04:13:24.967436+0000","last_became_peered":"2024-09-19T04:13:24.967436+0000","last_unstale":"2024-09-19T04:13:30.299773+0000","last_undegraded":"2024-09-19T04:13:30.299773+0000","last_fullsized":"2024-09-19T04:13:30.299773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061448+0000","last_change":"2024-09-19T04:13:24.942637+0000","last_active":"2024-09-19T04:13:31.061448+0000","last_peered":"2024-09-19T04:13:31.061448+0000","last_clean":"2024-09-19T04:13:31.061448+0000","last_became_active":"2024-09-19T04:13:24.942164+0000","last_became_peered":"2024-09-19T04:13:24.942164+0000","last_unstale":"2024-09-19T04:13:31.061448+0000","last_undegraded":"2024-09-19T04:13:31.061448+0000","last_fullsized":"2024-09-19T04:13:31.061448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:30.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971132+0000","last_change":"2024-09-19T04:13:24.926176+0000","last_active":"2024-09-19T04:13:29.971132+0000","last_peered":"2024-09-19T04:13:29.971132+0000","last_clean":"2024-09-19T04:13:29.971132+0000","last_became_active":"2024-09-19T04:13:24.925246+0000","last_became_peered":"2024-09-19T04:13:24.925246+0000","last_unstale":"2024-09-19T04:13:29.971132+0000","last_undegraded":"2024-09-19T04:13:29.971132+0000","last_fullsized":"2024-09-19T04:13:29.971132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:52.769507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062030+0000","last_change":"2024-09-19T04:13:24.953254+0000","last_active":"2024-09-19T04:13:31.062030+0000","last_peered":"2024-09-19T04:13:31.062030+0000","last_clean":"2024-09-19T04:13:31.062030+0000","last_became_active":"2024-09-19T04:13:24.950624+0000","last_became_peered":"2024-09-19T04:13:24.950624+0000","last_unstale":"2024-09-19T04:13:31.062030+0000","last_undegraded":"2024-09-19T04:13:31.062030+0000","last_fullsized":"2024-09-19T04:13:31.062030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:06.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969906+0000","last_change":"2024-09-19T04:13:24.954966+0000","last_active":"2024-09-19T04:13:29.969906+0000","last_peered":"2024-09-19T04:13:29.969906+0000","last_clean":"2024-09-19T04:13:29.969906+0000","last_became_active":"2024-09-19T04:13:24.954234+0000","last_became_peered":"2024-09-19T04:13:24.954234+0000","last_unstale":"2024-09-19T04:13:29.969906+0000","last_undegraded":"2024-09-19T04:13:29.969906+0000","last_fullsized":"2024-09-19T04:13:29.969906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061764+0000","last_change":"2024-09-19T04:13:24.949281+0000","last_active":"2024-09-19T04:13:31.061764+0000","last_peered":"2024-09-19T04:13:31.061764+0000","last_clean":"2024-09-19T04:13:31.061764+0000","last_became_active":"2024-09-19T04:13:24.948863+0000","last_became_peered":"2024-09-19T04:13:24.948863+0000","last_unstale":"2024-09-19T04:13:31.061764+0000","last_undegraded":"2024-09-19T04:13:31.061764+0000","last_fullsized":"2024-09-19T04:13:31.061764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131694+0000","last_change":"2024-09-19T04:13:24.943099+0000","last_active":"2024-09-19T04:13:30.131694+0000","last_peered":"2024-09-19T04:13:30.131694+0000","last_clean":"2024-09-19T04:13:30.131694+0000","last_became_active":"2024-09-19T04:13:24.942776+0000","last_became_peered":"2024-09-19T04:13:24.942776+0000","last_unstale":"2024-09-19T04:13:30.131694+0000","last_undegraded":"2024-09-19T04:13:30.131694+0000","last_fullsized":"2024-09-19T04:13:30.131694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064570+0000","last_change":"2024-09-19T04:13:24.951658+0000","last_active":"2024-09-19T04:13:31.064570+0000","last_peered":"2024-09-19T04:13:31.064570+0000","last_clean":"2024-09-19T04:13:31.064570+0000","last_became_active":"2024-09-19T04:13:24.950976+0000","last_became_peered":"2024-09-19T04:13:24.950976+0000","last_unstale":"2024-09-19T04:13:31.064570+0000","last_undegraded":"2024-09-19T04:13:31.064570+0000","last_fullsized":"2024-09-19T04:13:31.064570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:39.061865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":694435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":103611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":103485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":103645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:13:42.577 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:13:42.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.726+0000 7fb09ac4d640 1 Processor -- start 2024-09-19T04:13:42.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb09ac4d640 1 -- start start 2024-09-19T04:13:42.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb09ac4d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb09415f3a0 0x7fb094165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb09ac4d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0940aede0 con 0x7fb0941520a0 2024-09-19T04:13:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb09ac4d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb09415fa10 con 0x7fb094070020 2024-09-19T04:13:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb09ac4d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb094152480 con 0x7fb09415f3a0 2024-09-19T04:13:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0989e9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb0941520a0 0x7fb09406f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37512/0 (socket says 172.21.15.76:37512) 2024-09-19T04:13:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0989e9640 1 -- 172.21.15.76:0/3989995465 learned_addr learned my addr 172.21.15.76:0/3989995465 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0991ea640 1 --2- 172.21.15.76:0/3989995465 >> v2:172.21.15.116:3300/0 conn(0x7fb09415f3a0 0x7fb094165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3745827582 0 0) 0x7fb0940aede0 con 0x7fb0941520a0 2024-09-19T04:13:42.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb084005740 con 0x7fb0941520a0 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2362007738 0 0) 0x7fb084005740 con 0x7fb0941520a0 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 >> v2:172.21.15.116:3300/0 conn(0x7fb09415f3a0 msgr2=0x7fb094165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 --2- 172.21.15.76:0/3989995465 >> v2:172.21.15.116:3300/0 conn(0x7fb09415f3a0 0x7fb094165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 >> v1:172.21.15.116:6789/0 conn(0x7fb094070020 legacy=0x7fb09415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb094074250 con 0x7fb0941520a0 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb080001680 con 0x7fb0941520a0 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 >> v1:172.21.15.76:6789/0 conn(0x7fb0941520a0 legacy=0x7fb09406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb084005bd0 0x7fb084005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb084006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0840073e0 0x7fb08400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb08400e250 con 0x7fb084006a20 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb08400e400 con 0x7fb0840073e0 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb08400e600 con 0x7fb084005bd0 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb093fff640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb084006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0991ea640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0840073e0 0x7fb08400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0989e9640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb084005bd0 0x7fb084005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb093fff640 1 -- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb084005bd0 msgr2=0x7fb084005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb093fff640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb084005bd0 0x7fb084005ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb093fff640 1 -- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0840073e0 msgr2=0x7fb08400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb093fff640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0840073e0 0x7fb08400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb093fff640 1 -- 172.21.15.76:0/3989995465 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb084005740 con 0x7fb084006a20 2024-09-19T04:13:42.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0991ea640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0840073e0 0x7fb08400dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:42.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.730+0000 7fb0991ea640 1 -- 172.21.15.76:0/3989995465 reap_dead start 2024-09-19T04:13:42.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb093fff640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb084006ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb088001750 tx=0x7fb08801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b470b723d892b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880013a0 con 0x7fb084006a20 2024-09-19T04:13:42.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0880052f0 con 0x7fb084006a20 2024-09-19T04:13:42.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0937fe640 1 -- 172.21.15.76:0/3989995465 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb088003840 con 0x7fb084006a20 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- 172.21.15.76:0/3989995465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 msgr2=0x7fb084006ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb084006ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb088001750 tx=0x7fb08801c3c0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- 172.21.15.76:0/3989995465 shutdown_connections 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/3989995465 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb084006ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- 172.21.15.76:0/3989995465 >> 172.21.15.76:0/3989995465 conn(0x7fb0940b1f90 msgr2=0x7fb0940b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- 172.21.15.76:0/3989995465 shutdown_connections 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- 172.21.15.76:0/3989995465 wait complete. 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 Processor -- start 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- start start 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb09414f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb094145a30 0x7fb094154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb094145eb0 0x7fb0941549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb094073a50 con 0x7fb084006a20 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb094073770 con 0x7fb094145a30 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb09ac4d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb094073d30 con 0x7fb094145eb0 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb09414f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb09414f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32880/0 (socket says 172.21.15.76:32880) 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 -- 172.21.15.76:0/1810766469 learned_addr learned my addr 172.21.15.76:0/1810766469 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0991ea640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb094145eb0 0x7fb0941549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb093fff640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb094145a30 0x7fb094154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 -- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb094145eb0 msgr2=0x7fb0941549d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb094145eb0 0x7fb0941549d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 -- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb094145a30 msgr2=0x7fb094154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb094145a30 0x7fb094154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 -- 172.21.15.76:0/1810766469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09414fba0 con 0x7fb084006a20 2024-09-19T04:13:42.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb093fff640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb094145a30 0x7fb094154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:42.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.734+0000 7fb0989e9640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb09414f660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb080004880 tx=0x7fb08000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb0917fa640 1 -- 172.21.15.76:0/1810766469 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08001b070 con 0x7fb084006a20 2024-09-19T04:13:42.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb0917fa640 1 -- 172.21.15.76:0/1810766469 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb080011040 con 0x7fb084006a20 2024-09-19T04:13:42.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09414e5c0 con 0x7fb084006a20 2024-09-19T04:13:42.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb0917fa640 1 -- 172.21.15.76:0/1810766469 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0800143a0 con 0x7fb084006a20 2024-09-19T04:13:42.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb09414eb80 con 0x7fb084006a20 2024-09-19T04:13:42.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb094068e50 con 0x7fb084006a20 2024-09-19T04:13:42.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb0917fa640 1 -- 172.21.15.76:0/1810766469 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb080014540 con 0x7fb084006a20 2024-09-19T04:13:42.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb0917fa640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0780678c0 0x7fb078069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:42.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb0917fa640 1 -- 172.21.15.76:0/1810766469 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb080088a20 con 0x7fb084006a20 2024-09-19T04:13:42.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb093fff640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0780678c0 0x7fb078069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:42.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.738+0000 7fb093fff640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0780678c0 0x7fb078069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb094070420 tx=0x7fb0880011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:42.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.742+0000 7fb0917fa640 1 -- 172.21.15.76:0/1810766469 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb080058b50 con 0x7fb084006a20 2024-09-19T04:13:42.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.890+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb09406e540 con 0x7fb0780678c0 2024-09-19T04:13:42.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.910+0000 7fb0917fa640 1 -- 172.21.15.76:0/1810766469 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fb08810c310 con 0x7fb0780678c0 2024-09-19T04:13:42.919 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:42.921 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0780678c0 msgr2=0x7fb078069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0780678c0 0x7fb078069d80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb094070420 tx=0x7fb0880011a0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 msgr2=0x7fb09414f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb09414f660 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb080004880 tx=0x7fb08000e4d0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 shutdown_connections 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0780678c0 0x7fb078069d80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb094145eb0 0x7fb0941549d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb094145a30 0x7fb094154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 --2- 172.21.15.76:0/1810766469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb084006a20 0x7fb09414f660 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 >> 172.21.15.76:0/1810766469 conn(0x7fb0940b1f90 msgr2=0x7fb0941511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 shutdown_connections 2024-09-19T04:13:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:42.918+0000 7fb09ac4d640 1 -- 172.21.15.76:0/1810766469 wait complete. 2024-09-19T04:13:42.936 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-19T04:13:41.742492+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595010,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223668,"kb_used_data":3272,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754751052,"statfs":{"total":773094113280,"available":772865077248,"internally_reserved":0,"allocated":3350528,"data_stored":2017900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2232,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.755961"},"pg_stats":[{"pgid":"2.1f","version":"20'4","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062396+0000","last_change":"2024-09-19T04:13:23.917029+0000","last_active":"2024-09-19T04:13:31.062396+0000","last_peered":"2024-09-19T04:13:31.062396+0000","last_clean":"2024-09-19T04:13:31.062396+0000","last_became_active":"2024-09-19T04:13:23.916640+0000","last_became_peered":"2024-09-19T04:13:23.916640+0000","last_unstale":"2024-09-19T04:13:31.062396+0000","last_undegraded":"2024-09-19T04:13:31.062396+0000","last_fullsized":"2024-09-19T04:13:31.062396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:17:22.981160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301574+0000","last_change":"2024-09-19T04:13:24.947160+0000","last_active":"2024-09-19T04:13:30.301574+0000","last_peered":"2024-09-19T04:13:30.301574+0000","last_clean":"2024-09-19T04:13:30.301574+0000","last_became_active":"2024-09-19T04:13:24.946824+0000","last_became_peered":"2024-09-19T04:13:24.946824+0000","last_unstale":"2024-09-19T04:13:30.301574+0000","last_undegraded":"2024-09-19T04:13:30.301574+0000","last_fullsized":"2024-09-19T04:13:30.301574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300320+0000","last_change":"2024-09-19T04:13:29.029580+0000","last_active":"2024-09-19T04:13:30.300320+0000","last_peered":"2024-09-19T04:13:30.300320+0000","last_clean":"2024-09-19T04:13:30.300320+0000","last_became_active":"2024-09-19T04:13:29.029159+0000","last_became_peered":"2024-09-19T04:13:29.029159+0000","last_unstale":"2024-09-19T04:13:30.300320+0000","last_undegraded":"2024-09-19T04:13:30.300320+0000","last_fullsized":"2024-09-19T04:13:30.300320+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301237+0000","last_change":"2024-09-19T04:13:23.910773+0000","last_active":"2024-09-19T04:13:30.301237+0000","last_peered":"2024-09-19T04:13:30.301237+0000","last_clean":"2024-09-19T04:13:30.301237+0000","last_became_active":"2024-09-19T04:13:23.910355+0000","last_became_peered":"2024-09-19T04:13:23.910355+0000","last_unstale":"2024-09-19T04:13:30.301237+0000","last_undegraded":"2024-09-19T04:13:30.301237+0000","last_fullsized":"2024-09-19T04:13:30.301237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062270+0000","last_change":"2024-09-19T04:13:24.956969+0000","last_active":"2024-09-19T04:13:31.062270+0000","last_peered":"2024-09-19T04:13:31.062270+0000","last_clean":"2024-09-19T04:13:31.062270+0000","last_became_active":"2024-09-19T04:13:24.956671+0000","last_became_peered":"2024-09-19T04:13:24.956671+0000","last_unstale":"2024-09-19T04:13:31.062270+0000","last_undegraded":"2024-09-19T04:13:31.062270+0000","last_fullsized":"2024-09-19T04:13:31.062270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061243+0000","last_change":"2024-09-19T04:13:28.991785+0000","last_active":"2024-09-19T04:13:31.061243+0000","last_peered":"2024-09-19T04:13:31.061243+0000","last_clean":"2024-09-19T04:13:31.061243+0000","last_became_active":"2024-09-19T04:13:28.987169+0000","last_became_peered":"2024-09-19T04:13:28.987169+0000","last_unstale":"2024-09-19T04:13:31.061243+0000","last_undegraded":"2024-09-19T04:13:31.061243+0000","last_fullsized":"2024-09-19T04:13:31.061243+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:16:00.614358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062999+0000","last_change":"2024-09-19T04:13:23.924693+0000","last_active":"2024-09-19T04:13:31.062999+0000","last_peered":"2024-09-19T04:13:31.062999+0000","last_clean":"2024-09-19T04:13:31.062999+0000","last_became_active":"2024-09-19T04:13:23.924077+0000","last_became_peered":"2024-09-19T04:13:23.924077+0000","last_unstale":"2024-09-19T04:13:31.062999+0000","last_undegraded":"2024-09-19T04:13:31.062999+0000","last_fullsized":"2024-09-19T04:13:31.062999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:10.151229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970152+0000","last_change":"2024-09-19T04:13:24.948486+0000","last_active":"2024-09-19T04:13:29.970152+0000","last_peered":"2024-09-19T04:13:29.970152+0000","last_clean":"2024-09-19T04:13:29.970152+0000","last_became_active":"2024-09-19T04:13:24.947749+0000","last_became_peered":"2024-09-19T04:13:24.947749+0000","last_unstale":"2024-09-19T04:13:29.970152+0000","last_undegraded":"2024-09-19T04:13:29.970152+0000","last_fullsized":"2024-09-19T04:13:29.970152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:37.654967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063108+0000","last_change":"2024-09-19T04:13:28.992157+0000","last_active":"2024-09-19T04:13:31.063108+0000","last_peered":"2024-09-19T04:13:31.063108+0000","last_clean":"2024-09-19T04:13:31.063108+0000","last_became_active":"2024-09-19T04:13:28.990879+0000","last_became_peered":"2024-09-19T04:13:28.990879+0000","last_unstale":"2024-09-19T04:13:31.063108+0000","last_undegraded":"2024-09-19T04:13:31.063108+0000","last_fullsized":"2024-09-19T04:13:31.063108+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:39.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062192+0000","last_change":"2024-09-19T04:13:23.923303+0000","last_active":"2024-09-19T04:13:31.062192+0000","last_peered":"2024-09-19T04:13:31.062192+0000","last_clean":"2024-09-19T04:13:31.062192+0000","last_became_active":"2024-09-19T04:13:23.920842+0000","last_became_peered":"2024-09-19T04:13:23.920842+0000","last_unstale":"2024-09-19T04:13:31.062192+0000","last_undegraded":"2024-09-19T04:13:31.062192+0000","last_fullsized":"2024-09-19T04:13:31.062192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:23.176892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971159+0000","last_change":"2024-09-19T04:13:24.948074+0000","last_active":"2024-09-19T04:13:29.971159+0000","last_peered":"2024-09-19T04:13:29.971159+0000","last_clean":"2024-09-19T04:13:29.971159+0000","last_became_active":"2024-09-19T04:13:24.946595+0000","last_became_peered":"2024-09-19T04:13:24.946595+0000","last_unstale":"2024-09-19T04:13:29.971159+0000","last_undegraded":"2024-09-19T04:13:29.971159+0000","last_fullsized":"2024-09-19T04:13:29.971159+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:21.342803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062406+0000","last_change":"2024-09-19T04:13:28.985965+0000","last_active":"2024-09-19T04:13:31.062406+0000","last_peered":"2024-09-19T04:13:31.062406+0000","last_clean":"2024-09-19T04:13:31.062406+0000","last_became_active":"2024-09-19T04:13:28.979842+0000","last_became_peered":"2024-09-19T04:13:28.979842+0000","last_unstale":"2024-09-19T04:13:31.062406+0000","last_undegraded":"2024-09-19T04:13:31.062406+0000","last_fullsized":"2024-09-19T04:13:31.062406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:13.654228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971528+0000","last_change":"2024-09-19T04:13:24.964645+0000","last_active":"2024-09-19T04:13:29.971528+0000","last_peered":"2024-09-19T04:13:29.971528+0000","last_clean":"2024-09-19T04:13:29.971528+0000","last_became_active":"2024-09-19T04:13:24.964051+0000","last_became_peered":"2024-09-19T04:13:24.964051+0000","last_unstale":"2024-09-19T04:13:29.971528+0000","last_undegraded":"2024-09-19T04:13:29.971528+0000","last_fullsized":"2024-09-19T04:13:29.971528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062721+0000","last_change":"2024-09-19T04:13:23.921499+0000","last_active":"2024-09-19T04:13:31.062721+0000","last_peered":"2024-09-19T04:13:31.062721+0000","last_clean":"2024-09-19T04:13:31.062721+0000","last_became_active":"2024-09-19T04:13:23.921146+0000","last_became_peered":"2024-09-19T04:13:23.921146+0000","last_unstale":"2024-09-19T04:13:31.062721+0000","last_undegraded":"2024-09-19T04:13:31.062721+0000","last_fullsized":"2024-09-19T04:13:31.062721+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:54.181515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063276+0000","last_change":"2024-09-19T04:13:28.979497+0000","last_active":"2024-09-19T04:13:31.063276+0000","last_peered":"2024-09-19T04:13:31.063276+0000","last_clean":"2024-09-19T04:13:31.063276+0000","last_became_active":"2024-09-19T04:13:28.978329+0000","last_became_peered":"2024-09-19T04:13:28.978329+0000","last_unstale":"2024-09-19T04:13:31.063276+0000","last_undegraded":"2024-09-19T04:13:31.063276+0000","last_fullsized":"2024-09-19T04:13:31.063276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:30.314159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300960+0000","last_change":"2024-09-19T04:13:24.951290+0000","last_active":"2024-09-19T04:13:30.300960+0000","last_peered":"2024-09-19T04:13:30.300960+0000","last_clean":"2024-09-19T04:13:30.300960+0000","last_became_active":"2024-09-19T04:13:24.950973+0000","last_became_peered":"2024-09-19T04:13:24.950973+0000","last_unstale":"2024-09-19T04:13:30.300960+0000","last_undegraded":"2024-09-19T04:13:30.300960+0000","last_fullsized":"2024-09-19T04:13:30.300960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131596+0000","last_change":"2024-09-19T04:13:23.918153+0000","last_active":"2024-09-19T04:13:30.131596+0000","last_peered":"2024-09-19T04:13:30.131596+0000","last_clean":"2024-09-19T04:13:30.131596+0000","last_became_active":"2024-09-19T04:13:23.917783+0000","last_became_peered":"2024-09-19T04:13:23.917783+0000","last_unstale":"2024-09-19T04:13:30.131596+0000","last_undegraded":"2024-09-19T04:13:30.131596+0000","last_fullsized":"2024-09-19T04:13:30.131596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063970+0000","last_change":"2024-09-19T04:13:28.988863+0000","last_active":"2024-09-19T04:13:31.063970+0000","last_peered":"2024-09-19T04:13:31.063970+0000","last_clean":"2024-09-19T04:13:31.063970+0000","last_became_active":"2024-09-19T04:13:28.987942+0000","last_became_peered":"2024-09-19T04:13:28.987942+0000","last_unstale":"2024-09-19T04:13:31.063970+0000","last_undegraded":"2024-09-19T04:13:31.063970+0000","last_fullsized":"2024-09-19T04:13:31.063970+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300167+0000","last_change":"2024-09-19T04:13:24.946137+0000","last_active":"2024-09-19T04:13:30.300167+0000","last_peered":"2024-09-19T04:13:30.300167+0000","last_clean":"2024-09-19T04:13:30.300167+0000","last_became_active":"2024-09-19T04:13:24.945850+0000","last_became_peered":"2024-09-19T04:13:24.945850+0000","last_unstale":"2024-09-19T04:13:30.300167+0000","last_undegraded":"2024-09-19T04:13:30.300167+0000","last_fullsized":"2024-09-19T04:13:30.300167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:23.916109+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:23.915767+0000","last_became_peered":"2024-09-19T04:13:23.915767+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131755+0000","last_change":"2024-09-19T04:13:29.030707+0000","last_active":"2024-09-19T04:13:30.131755+0000","last_peered":"2024-09-19T04:13:30.131755+0000","last_clean":"2024-09-19T04:13:30.131755+0000","last_became_active":"2024-09-19T04:13:29.030294+0000","last_became_peered":"2024-09-19T04:13:29.030294+0000","last_unstale":"2024-09-19T04:13:30.131755+0000","last_undegraded":"2024-09-19T04:13:30.131755+0000","last_fullsized":"2024-09-19T04:13:30.131755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063152+0000","last_change":"2024-09-19T04:13:24.948148+0000","last_active":"2024-09-19T04:13:31.063152+0000","last_peered":"2024-09-19T04:13:31.063152+0000","last_clean":"2024-09-19T04:13:31.063152+0000","last_became_active":"2024-09-19T04:13:24.947330+0000","last_became_peered":"2024-09-19T04:13:24.947330+0000","last_unstale":"2024-09-19T04:13:31.063152+0000","last_undegraded":"2024-09-19T04:13:31.063152+0000","last_fullsized":"2024-09-19T04:13:31.063152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:49.624559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061085+0000","last_change":"2024-09-19T04:13:23.923583+0000","last_active":"2024-09-19T04:13:31.061085+0000","last_peered":"2024-09-19T04:13:31.061085+0000","last_clean":"2024-09-19T04:13:31.061085+0000","last_became_active":"2024-09-19T04:13:23.921593+0000","last_became_peered":"2024-09-19T04:13:23.921593+0000","last_unstale":"2024-09-19T04:13:31.061085+0000","last_undegraded":"2024-09-19T04:13:31.061085+0000","last_fullsized":"2024-09-19T04:13:31.061085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063591+0000","last_change":"2024-09-19T04:13:28.990856+0000","last_active":"2024-09-19T04:13:31.063591+0000","last_peered":"2024-09-19T04:13:31.063591+0000","last_clean":"2024-09-19T04:13:31.063591+0000","last_became_active":"2024-09-19T04:13:28.987830+0000","last_became_peered":"2024-09-19T04:13:28.987830+0000","last_unstale":"2024-09-19T04:13:31.063591+0000","last_undegraded":"2024-09-19T04:13:31.063591+0000","last_fullsized":"2024-09-19T04:13:31.063591+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062632+0000","last_change":"2024-09-19T04:13:24.934798+0000","last_active":"2024-09-19T04:13:31.062632+0000","last_peered":"2024-09-19T04:13:31.062632+0000","last_clean":"2024-09-19T04:13:31.062632+0000","last_became_active":"2024-09-19T04:13:24.933977+0000","last_became_peered":"2024-09-19T04:13:24.933977+0000","last_unstale":"2024-09-19T04:13:31.062632+0000","last_undegraded":"2024-09-19T04:13:31.062632+0000","last_fullsized":"2024-09-19T04:13:31.062632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973271+0000","last_change":"2024-09-19T04:13:23.920837+0000","last_active":"2024-09-19T04:13:29.973271+0000","last_peered":"2024-09-19T04:13:29.973271+0000","last_clean":"2024-09-19T04:13:29.973271+0000","last_became_active":"2024-09-19T04:13:23.920245+0000","last_became_peered":"2024-09-19T04:13:23.920245+0000","last_unstale":"2024-09-19T04:13:29.973271+0000","last_undegraded":"2024-09-19T04:13:29.973271+0000","last_fullsized":"2024-09-19T04:13:29.973271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062572+0000","last_change":"2024-09-19T04:13:29.019257+0000","last_active":"2024-09-19T04:13:31.062572+0000","last_peered":"2024-09-19T04:13:31.062572+0000","last_clean":"2024-09-19T04:13:31.062572+0000","last_became_active":"2024-09-19T04:13:29.018425+0000","last_became_peered":"2024-09-19T04:13:29.018425+0000","last_unstale":"2024-09-19T04:13:31.062572+0000","last_undegraded":"2024-09-19T04:13:31.062572+0000","last_fullsized":"2024-09-19T04:13:31.062572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:42.657248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301533+0000","last_change":"2024-09-19T04:13:24.944069+0000","last_active":"2024-09-19T04:13:30.301533+0000","last_peered":"2024-09-19T04:13:30.301533+0000","last_clean":"2024-09-19T04:13:30.301533+0000","last_became_active":"2024-09-19T04:13:24.943774+0000","last_became_peered":"2024-09-19T04:13:24.943774+0000","last_unstale":"2024-09-19T04:13:30.301533+0000","last_undegraded":"2024-09-19T04:13:30.301533+0000","last_fullsized":"2024-09-19T04:13:30.301533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131771+0000","last_change":"2024-09-19T04:13:23.911708+0000","last_active":"2024-09-19T04:13:30.131771+0000","last_peered":"2024-09-19T04:13:30.131771+0000","last_clean":"2024-09-19T04:13:30.131771+0000","last_became_active":"2024-09-19T04:13:23.911135+0000","last_became_peered":"2024-09-19T04:13:23.911135+0000","last_unstale":"2024-09-19T04:13:30.131771+0000","last_undegraded":"2024-09-19T04:13:30.131771+0000","last_fullsized":"2024-09-19T04:13:30.131771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:48.393589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299979+0000","last_change":"2024-09-19T04:13:29.020657+0000","last_active":"2024-09-19T04:13:30.299979+0000","last_peered":"2024-09-19T04:13:30.299979+0000","last_clean":"2024-09-19T04:13:30.299979+0000","last_became_active":"2024-09-19T04:13:29.020172+0000","last_became_peered":"2024-09-19T04:13:29.020172+0000","last_unstale":"2024-09-19T04:13:30.299979+0000","last_undegraded":"2024-09-19T04:13:30.299979+0000","last_fullsized":"2024-09-19T04:13:30.299979+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131405+0000","last_change":"2024-09-19T04:13:24.949008+0000","last_active":"2024-09-19T04:13:30.131405+0000","last_peered":"2024-09-19T04:13:30.131405+0000","last_clean":"2024-09-19T04:13:30.131405+0000","last_became_active":"2024-09-19T04:13:24.948513+0000","last_became_peered":"2024-09-19T04:13:24.948513+0000","last_unstale":"2024-09-19T04:13:30.131405+0000","last_undegraded":"2024-09-19T04:13:30.131405+0000","last_fullsized":"2024-09-19T04:13:30.131405+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063250+0000","last_change":"2024-09-19T04:13:23.921473+0000","last_active":"2024-09-19T04:13:31.063250+0000","last_peered":"2024-09-19T04:13:31.063250+0000","last_clean":"2024-09-19T04:13:31.063250+0000","last_became_active":"2024-09-19T04:13:23.921052+0000","last_became_peered":"2024-09-19T04:13:23.921052+0000","last_unstale":"2024-09-19T04:13:31.063250+0000","last_undegraded":"2024-09-19T04:13:31.063250+0000","last_fullsized":"2024-09-19T04:13:31.063250+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:37:17.114813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061949+0000","last_change":"2024-09-19T04:13:29.020022+0000","last_active":"2024-09-19T04:13:31.061949+0000","last_peered":"2024-09-19T04:13:31.061949+0000","last_clean":"2024-09-19T04:13:31.061949+0000","last_became_active":"2024-09-19T04:13:29.019585+0000","last_became_peered":"2024-09-19T04:13:29.019585+0000","last_unstale":"2024-09-19T04:13:31.061949+0000","last_undegraded":"2024-09-19T04:13:31.061949+0000","last_fullsized":"2024-09-19T04:13:31.061949+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970941+0000","last_change":"2024-09-19T04:13:24.952102+0000","last_active":"2024-09-19T04:13:29.970941+0000","last_peered":"2024-09-19T04:13:29.970941+0000","last_clean":"2024-09-19T04:13:29.970941+0000","last_became_active":"2024-09-19T04:13:24.951184+0000","last_became_peered":"2024-09-19T04:13:24.951184+0000","last_unstale":"2024-09-19T04:13:29.970941+0000","last_undegraded":"2024-09-19T04:13:29.970941+0000","last_fullsized":"2024-09-19T04:13:29.970941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:00.982972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131508+0000","last_change":"2024-09-19T04:13:23.908259+0000","last_active":"2024-09-19T04:13:30.131508+0000","last_peered":"2024-09-19T04:13:30.131508+0000","last_clean":"2024-09-19T04:13:30.131508+0000","last_became_active":"2024-09-19T04:13:23.907717+0000","last_became_peered":"2024-09-19T04:13:23.907717+0000","last_unstale":"2024-09-19T04:13:30.131508+0000","last_undegraded":"2024-09-19T04:13:30.131508+0000","last_fullsized":"2024-09-19T04:13:30.131508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.975119+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.974476+0000","last_became_peered":"2024-09-19T04:13:28.974476+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063922+0000","last_change":"2024-09-19T04:13:24.922621+0000","last_active":"2024-09-19T04:13:31.063922+0000","last_peered":"2024-09-19T04:13:31.063922+0000","last_clean":"2024-09-19T04:13:31.063922+0000","last_became_active":"2024-09-19T04:13:24.921879+0000","last_became_peered":"2024-09-19T04:13:24.921879+0000","last_unstale":"2024-09-19T04:13:31.063922+0000","last_undegraded":"2024-09-19T04:13:31.063922+0000","last_fullsized":"2024-09-19T04:13:31.063922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:04.750679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972020+0000","last_change":"2024-09-19T04:13:23.901650+0000","last_active":"2024-09-19T04:13:29.972020+0000","last_peered":"2024-09-19T04:13:29.972020+0000","last_clean":"2024-09-19T04:13:29.972020+0000","last_became_active":"2024-09-19T04:13:23.900880+0000","last_became_peered":"2024-09-19T04:13:23.900880+0000","last_unstale":"2024-09-19T04:13:29.972020+0000","last_undegraded":"2024-09-19T04:13:29.972020+0000","last_fullsized":"2024-09-19T04:13:29.972020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131635+0000","last_change":"2024-09-19T04:13:28.980693+0000","last_active":"2024-09-19T04:13:30.131635+0000","last_peered":"2024-09-19T04:13:30.131635+0000","last_clean":"2024-09-19T04:13:30.131635+0000","last_became_active":"2024-09-19T04:13:28.979948+0000","last_became_peered":"2024-09-19T04:13:28.979948+0000","last_unstale":"2024-09-19T04:13:30.131635+0000","last_undegraded":"2024-09-19T04:13:30.131635+0000","last_fullsized":"2024-09-19T04:13:30.131635+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130985+0000","last_change":"2024-09-19T04:13:24.948867+0000","last_active":"2024-09-19T04:13:30.130985+0000","last_peered":"2024-09-19T04:13:30.130985+0000","last_clean":"2024-09-19T04:13:30.130985+0000","last_became_active":"2024-09-19T04:13:24.948471+0000","last_became_peered":"2024-09-19T04:13:24.948471+0000","last_unstale":"2024-09-19T04:13:30.130985+0000","last_undegraded":"2024-09-19T04:13:30.130985+0000","last_fullsized":"2024-09-19T04:13:30.130985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970768+0000","last_change":"2024-09-19T04:13:23.903592+0000","last_active":"2024-09-19T04:13:29.970768+0000","last_peered":"2024-09-19T04:13:29.970768+0000","last_clean":"2024-09-19T04:13:29.970768+0000","last_became_active":"2024-09-19T04:13:23.903306+0000","last_became_peered":"2024-09-19T04:13:23.903306+0000","last_unstale":"2024-09-19T04:13:29.970768+0000","last_undegraded":"2024-09-19T04:13:29.970768+0000","last_fullsized":"2024-09-19T04:13:29.970768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:13.135208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131834+0000","last_change":"2024-09-19T04:13:28.976039+0000","last_active":"2024-09-19T04:13:30.131834+0000","last_peered":"2024-09-19T04:13:30.131834+0000","last_clean":"2024-09-19T04:13:30.131834+0000","last_became_active":"2024-09-19T04:13:28.975424+0000","last_became_peered":"2024-09-19T04:13:28.975424+0000","last_unstale":"2024-09-19T04:13:30.131834+0000","last_undegraded":"2024-09-19T04:13:30.131834+0000","last_fullsized":"2024-09-19T04:13:30.131834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:51:27.255493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061535+0000","last_change":"2024-09-19T04:13:24.956224+0000","last_active":"2024-09-19T04:13:31.061535+0000","last_peered":"2024-09-19T04:13:31.061535+0000","last_clean":"2024-09-19T04:13:31.061535+0000","last_became_active":"2024-09-19T04:13:24.953799+0000","last_became_peered":"2024-09-19T04:13:24.953799+0000","last_unstale":"2024-09-19T04:13:31.061535+0000","last_undegraded":"2024-09-19T04:13:31.061535+0000","last_fullsized":"2024-09-19T04:13:31.061535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:33.543861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061237+0000","last_change":"2024-09-19T04:13:23.919493+0000","last_active":"2024-09-19T04:13:31.061237+0000","last_peered":"2024-09-19T04:13:31.061237+0000","last_clean":"2024-09-19T04:13:31.061237+0000","last_became_active":"2024-09-19T04:13:23.918702+0000","last_became_peered":"2024-09-19T04:13:23.918702+0000","last_unstale":"2024-09-19T04:13:31.061237+0000","last_undegraded":"2024-09-19T04:13:31.061237+0000","last_fullsized":"2024-09-19T04:13:31.061237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:47:42.592915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062499+0000","last_change":"2024-09-19T04:13:29.028357+0000","last_active":"2024-09-19T04:13:31.062499+0000","last_peered":"2024-09-19T04:13:31.062499+0000","last_clean":"2024-09-19T04:13:31.062499+0000","last_became_active":"2024-09-19T04:13:29.027946+0000","last_became_peered":"2024-09-19T04:13:29.027946+0000","last_unstale":"2024-09-19T04:13:31.062499+0000","last_undegraded":"2024-09-19T04:13:31.062499+0000","last_fullsized":"2024-09-19T04:13:31.062499+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:57:13.944765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300424+0000","last_change":"2024-09-19T04:13:24.968375+0000","last_active":"2024-09-19T04:13:30.300424+0000","last_peered":"2024-09-19T04:13:30.300424+0000","last_clean":"2024-09-19T04:13:30.300424+0000","last_became_active":"2024-09-19T04:13:24.968097+0000","last_became_peered":"2024-09-19T04:13:24.968097+0000","last_unstale":"2024-09-19T04:13:30.300424+0000","last_undegraded":"2024-09-19T04:13:30.300424+0000","last_fullsized":"2024-09-19T04:13:30.300424+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969947+0000","last_change":"2024-09-19T04:13:23.903638+0000","last_active":"2024-09-19T04:13:29.969947+0000","last_peered":"2024-09-19T04:13:29.969947+0000","last_clean":"2024-09-19T04:13:29.969947+0000","last_became_active":"2024-09-19T04:13:23.898948+0000","last_became_peered":"2024-09-19T04:13:23.898948+0000","last_unstale":"2024-09-19T04:13:29.969947+0000","last_undegraded":"2024-09-19T04:13:29.969947+0000","last_fullsized":"2024-09-19T04:13:29.969947+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061278+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:13:31.061278+0000","last_peered":"2024-09-19T04:13:31.061278+0000","last_clean":"2024-09-19T04:13:31.061278+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:13:31.061278+0000","last_undegraded":"2024-09-19T04:13:31.061278+0000","last_fullsized":"2024-09-19T04:13:31.061278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972561+0000","last_change":"2024-09-19T04:13:29.020008+0000","last_active":"2024-09-19T04:13:29.972561+0000","last_peered":"2024-09-19T04:13:29.972561+0000","last_clean":"2024-09-19T04:13:29.972561+0000","last_became_active":"2024-09-19T04:13:29.018706+0000","last_became_peered":"2024-09-19T04:13:29.018706+0000","last_unstale":"2024-09-19T04:13:29.972561+0000","last_undegraded":"2024-09-19T04:13:29.972561+0000","last_fullsized":"2024-09-19T04:13:29.972561+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062691+0000","last_change":"2024-09-19T04:13:24.966194+0000","last_active":"2024-09-19T04:13:31.062691+0000","last_peered":"2024-09-19T04:13:31.062691+0000","last_clean":"2024-09-19T04:13:31.062691+0000","last_became_active":"2024-09-19T04:13:24.965690+0000","last_became_peered":"2024-09-19T04:13:24.965690+0000","last_unstale":"2024-09-19T04:13:31.062691+0000","last_undegraded":"2024-09-19T04:13:31.062691+0000","last_fullsized":"2024-09-19T04:13:31.062691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:02:14.510098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971314+0000","last_change":"2024-09-19T04:13:23.901039+0000","last_active":"2024-09-19T04:13:29.971314+0000","last_peered":"2024-09-19T04:13:29.971314+0000","last_clean":"2024-09-19T04:13:29.971314+0000","last_became_active":"2024-09-19T04:13:23.900709+0000","last_became_peered":"2024-09-19T04:13:23.900709+0000","last_unstale":"2024-09-19T04:13:29.971314+0000","last_undegraded":"2024-09-19T04:13:29.971314+0000","last_fullsized":"2024-09-19T04:13:29.971314+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:19:40.425738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969768+0000","last_change":"2024-09-19T04:13:29.019565+0000","last_active":"2024-09-19T04:13:29.969768+0000","last_peered":"2024-09-19T04:13:29.969768+0000","last_clean":"2024-09-19T04:13:29.969768+0000","last_became_active":"2024-09-19T04:13:29.018547+0000","last_became_peered":"2024-09-19T04:13:29.018547+0000","last_unstale":"2024-09-19T04:13:29.969768+0000","last_undegraded":"2024-09-19T04:13:29.969768+0000","last_fullsized":"2024-09-19T04:13:29.969768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:41.712259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970414+0000","last_change":"2024-09-19T04:13:24.953123+0000","last_active":"2024-09-19T04:13:29.970414+0000","last_peered":"2024-09-19T04:13:29.970414+0000","last_clean":"2024-09-19T04:13:29.970414+0000","last_became_active":"2024-09-19T04:13:24.950851+0000","last_became_peered":"2024-09-19T04:13:24.950851+0000","last_unstale":"2024-09-19T04:13:29.970414+0000","last_undegraded":"2024-09-19T04:13:29.970414+0000","last_fullsized":"2024-09-19T04:13:29.970414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:17:26.256866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131715+0000","last_change":"2024-09-19T04:13:23.902143+0000","last_active":"2024-09-19T04:13:30.131715+0000","last_peered":"2024-09-19T04:13:30.131715+0000","last_clean":"2024-09-19T04:13:30.131715+0000","last_became_active":"2024-09-19T04:13:23.901843+0000","last_became_peered":"2024-09-19T04:13:23.901843+0000","last_unstale":"2024-09-19T04:13:30.131715+0000","last_undegraded":"2024-09-19T04:13:30.131715+0000","last_fullsized":"2024-09-19T04:13:30.131715+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:03.746896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062168+0000","last_change":"2024-09-19T04:13:28.994151+0000","last_active":"2024-09-19T04:13:31.062168+0000","last_peered":"2024-09-19T04:13:31.062168+0000","last_clean":"2024-09-19T04:13:31.062168+0000","last_became_active":"2024-09-19T04:13:28.993608+0000","last_became_peered":"2024-09-19T04:13:28.993608+0000","last_unstale":"2024-09-19T04:13:31.062168+0000","last_undegraded":"2024-09-19T04:13:31.062168+0000","last_fullsized":"2024-09-19T04:13:31.062168+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064926+0000","last_change":"2024-09-19T04:13:24.966033+0000","last_active":"2024-09-19T04:13:31.064926+0000","last_peered":"2024-09-19T04:13:31.064926+0000","last_clean":"2024-09-19T04:13:31.064926+0000","last_became_active":"2024-09-19T04:13:24.962081+0000","last_became_peered":"2024-09-19T04:13:24.962081+0000","last_unstale":"2024-09-19T04:13:31.064926+0000","last_undegraded":"2024-09-19T04:13:31.064926+0000","last_fullsized":"2024-09-19T04:13:31.064926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:58.621676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971640+0000","last_change":"2024-09-19T04:13:23.911774+0000","last_active":"2024-09-19T04:13:29.971640+0000","last_peered":"2024-09-19T04:13:29.971640+0000","last_clean":"2024-09-19T04:13:29.971640+0000","last_became_active":"2024-09-19T04:13:23.911154+0000","last_became_peered":"2024-09-19T04:13:23.911154+0000","last_unstale":"2024-09-19T04:13:29.971640+0000","last_undegraded":"2024-09-19T04:13:29.971640+0000","last_fullsized":"2024-09-19T04:13:29.971640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:37:08.314261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969327+0000","last_change":"2024-09-19T04:13:29.020613+0000","last_active":"2024-09-19T04:13:29.969327+0000","last_peered":"2024-09-19T04:13:29.969327+0000","last_clean":"2024-09-19T04:13:29.969327+0000","last_became_active":"2024-09-19T04:13:29.019473+0000","last_became_peered":"2024-09-19T04:13:29.019473+0000","last_unstale":"2024-09-19T04:13:29.969327+0000","last_undegraded":"2024-09-19T04:13:29.969327+0000","last_fullsized":"2024-09-19T04:13:29.969327+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:50.437481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064591+0000","last_change":"2024-09-19T04:13:24.962284+0000","last_active":"2024-09-19T04:13:31.064591+0000","last_peered":"2024-09-19T04:13:31.064591+0000","last_clean":"2024-09-19T04:13:31.064591+0000","last_became_active":"2024-09-19T04:13:24.961763+0000","last_became_peered":"2024-09-19T04:13:24.961763+0000","last_unstale":"2024-09-19T04:13:31.064591+0000","last_undegraded":"2024-09-19T04:13:31.064591+0000","last_fullsized":"2024-09-19T04:13:31.064591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:44:40.924861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064080+0000","last_change":"2024-09-19T04:13:23.905574+0000","last_active":"2024-09-19T04:13:31.064080+0000","last_peered":"2024-09-19T04:13:31.064080+0000","last_clean":"2024-09-19T04:13:31.064080+0000","last_became_active":"2024-09-19T04:13:23.904993+0000","last_became_peered":"2024-09-19T04:13:23.904993+0000","last_unstale":"2024-09-19T04:13:31.064080+0000","last_undegraded":"2024-09-19T04:13:31.064080+0000","last_fullsized":"2024-09-19T04:13:31.064080+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:00.615690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061135+0000","last_change":"2024-09-19T04:13:28.991444+0000","last_active":"2024-09-19T04:13:31.061135+0000","last_peered":"2024-09-19T04:13:31.061135+0000","last_clean":"2024-09-19T04:13:31.061135+0000","last_became_active":"2024-09-19T04:13:28.987837+0000","last_became_peered":"2024-09-19T04:13:28.987837+0000","last_unstale":"2024-09-19T04:13:31.061135+0000","last_undegraded":"2024-09-19T04:13:31.061135+0000","last_fullsized":"2024-09-19T04:13:31.061135+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:17.703055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061563+0000","last_change":"2024-09-19T04:13:24.965972+0000","last_active":"2024-09-19T04:13:31.061563+0000","last_peered":"2024-09-19T04:13:31.061563+0000","last_clean":"2024-09-19T04:13:31.061563+0000","last_became_active":"2024-09-19T04:13:24.965315+0000","last_became_peered":"2024-09-19T04:13:24.965315+0000","last_unstale":"2024-09-19T04:13:31.061563+0000","last_undegraded":"2024-09-19T04:13:31.061563+0000","last_fullsized":"2024-09-19T04:13:31.061563+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:26:16.174335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970388+0000","last_change":"2024-09-19T04:13:23.896881+0000","last_active":"2024-09-19T04:13:29.970388+0000","last_peered":"2024-09-19T04:13:29.970388+0000","last_clean":"2024-09-19T04:13:29.970388+0000","last_became_active":"2024-09-19T04:13:23.896440+0000","last_became_peered":"2024-09-19T04:13:23.896440+0000","last_unstale":"2024-09-19T04:13:29.970388+0000","last_undegraded":"2024-09-19T04:13:29.970388+0000","last_fullsized":"2024-09-19T04:13:29.970388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:36:56.677455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062482+0000","last_change":"2024-09-19T04:13:28.985691+0000","last_active":"2024-09-19T04:13:31.062482+0000","last_peered":"2024-09-19T04:13:31.062482+0000","last_clean":"2024-09-19T04:13:31.062482+0000","last_became_active":"2024-09-19T04:13:28.984749+0000","last_became_peered":"2024-09-19T04:13:28.984749+0000","last_unstale":"2024-09-19T04:13:31.062482+0000","last_undegraded":"2024-09-19T04:13:31.062482+0000","last_fullsized":"2024-09-19T04:13:31.062482+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:05.040265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969844+0000","last_change":"2024-09-19T04:13:24.942591+0000","last_active":"2024-09-19T04:13:29.969844+0000","last_peered":"2024-09-19T04:13:29.969844+0000","last_clean":"2024-09-19T04:13:29.969844+0000","last_became_active":"2024-09-19T04:13:24.941789+0000","last_became_peered":"2024-09-19T04:13:24.941789+0000","last_unstale":"2024-09-19T04:13:29.969844+0000","last_undegraded":"2024-09-19T04:13:29.969844+0000","last_fullsized":"2024-09-19T04:13:29.969844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970255+0000","last_change":"2024-09-19T04:13:23.916881+0000","last_active":"2024-09-19T04:13:29.970255+0000","last_peered":"2024-09-19T04:13:29.970255+0000","last_clean":"2024-09-19T04:13:29.970255+0000","last_became_active":"2024-09-19T04:13:23.916050+0000","last_became_peered":"2024-09-19T04:13:23.916050+0000","last_unstale":"2024-09-19T04:13:29.970255+0000","last_undegraded":"2024-09-19T04:13:29.970255+0000","last_fullsized":"2024-09-19T04:13:29.970255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:46.823394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301470+0000","last_change":"2024-09-19T04:13:29.021093+0000","last_active":"2024-09-19T04:13:30.301470+0000","last_peered":"2024-09-19T04:13:30.301470+0000","last_clean":"2024-09-19T04:13:30.301470+0000","last_became_active":"2024-09-19T04:13:29.020573+0000","last_became_peered":"2024-09-19T04:13:29.020573+0000","last_unstale":"2024-09-19T04:13:30.301470+0000","last_undegraded":"2024-09-19T04:13:30.301470+0000","last_fullsized":"2024-09-19T04:13:30.301470+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:35.622467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062500+0000","last_change":"2024-09-19T04:13:24.968018+0000","last_active":"2024-09-19T04:13:31.062500+0000","last_peered":"2024-09-19T04:13:31.062500+0000","last_clean":"2024-09-19T04:13:31.062500+0000","last_became_active":"2024-09-19T04:13:24.962386+0000","last_became_peered":"2024-09-19T04:13:24.962386+0000","last_unstale":"2024-09-19T04:13:31.062500+0000","last_undegraded":"2024-09-19T04:13:31.062500+0000","last_fullsized":"2024-09-19T04:13:31.062500+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:39.665883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971777+0000","last_change":"2024-09-19T04:13:23.911883+0000","last_active":"2024-09-19T04:13:29.971777+0000","last_peered":"2024-09-19T04:13:29.971777+0000","last_clean":"2024-09-19T04:13:29.971777+0000","last_became_active":"2024-09-19T04:13:23.910990+0000","last_became_peered":"2024-09-19T04:13:23.910990+0000","last_unstale":"2024-09-19T04:13:29.971777+0000","last_undegraded":"2024-09-19T04:13:29.971777+0000","last_fullsized":"2024-09-19T04:13:29.971777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301624+0000","last_change":"2024-09-19T04:13:29.017846+0000","last_active":"2024-09-19T04:13:30.301624+0000","last_peered":"2024-09-19T04:13:30.301624+0000","last_clean":"2024-09-19T04:13:30.301624+0000","last_became_active":"2024-09-19T04:13:29.017269+0000","last_became_peered":"2024-09-19T04:13:29.017269+0000","last_unstale":"2024-09-19T04:13:30.301624+0000","last_undegraded":"2024-09-19T04:13:30.301624+0000","last_fullsized":"2024-09-19T04:13:30.301624+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:07.730896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061954+0000","last_change":"2024-09-19T04:13:24.944986+0000","last_active":"2024-09-19T04:13:31.061954+0000","last_peered":"2024-09-19T04:13:31.061954+0000","last_clean":"2024-09-19T04:13:31.061954+0000","last_became_active":"2024-09-19T04:13:24.944266+0000","last_became_peered":"2024-09-19T04:13:24.944266+0000","last_unstale":"2024-09-19T04:13:31.061954+0000","last_undegraded":"2024-09-19T04:13:31.061954+0000","last_fullsized":"2024-09-19T04:13:31.061954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:17.729114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062229+0000","last_change":"2024-09-19T04:13:23.902337+0000","last_active":"2024-09-19T04:13:31.062229+0000","last_peered":"2024-09-19T04:13:31.062229+0000","last_clean":"2024-09-19T04:13:31.062229+0000","last_became_active":"2024-09-19T04:13:23.901958+0000","last_became_peered":"2024-09-19T04:13:23.901958+0000","last_unstale":"2024-09-19T04:13:31.062229+0000","last_undegraded":"2024-09-19T04:13:31.062229+0000","last_fullsized":"2024-09-19T04:13:31.062229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:39.987613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970753+0000","last_change":"2024-09-19T04:13:29.022266+0000","last_active":"2024-09-19T04:13:29.970753+0000","last_peered":"2024-09-19T04:13:29.970753+0000","last_clean":"2024-09-19T04:13:29.970753+0000","last_became_active":"2024-09-19T04:13:29.021104+0000","last_became_peered":"2024-09-19T04:13:29.021104+0000","last_unstale":"2024-09-19T04:13:29.970753+0000","last_undegraded":"2024-09-19T04:13:29.970753+0000","last_fullsized":"2024-09-19T04:13:29.970753+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:31:55.880154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062460+0000","last_change":"2024-09-19T04:13:24.952759+0000","last_active":"2024-09-19T04:13:31.062460+0000","last_peered":"2024-09-19T04:13:31.062460+0000","last_clean":"2024-09-19T04:13:31.062460+0000","last_became_active":"2024-09-19T04:13:24.952088+0000","last_became_peered":"2024-09-19T04:13:24.952088+0000","last_unstale":"2024-09-19T04:13:31.062460+0000","last_undegraded":"2024-09-19T04:13:31.062460+0000","last_fullsized":"2024-09-19T04:13:31.062460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:14:29.020474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:35.460051+0000","last_change":"2024-09-19T04:13:23.909991+0000","last_active":"2024-09-19T04:13:35.460051+0000","last_peered":"2024-09-19T04:13:35.460051+0000","last_clean":"2024-09-19T04:13:35.460051+0000","last_became_active":"2024-09-19T04:13:23.909517+0000","last_became_peered":"2024-09-19T04:13:23.909517+0000","last_unstale":"2024-09-19T04:13:35.460051+0000","last_undegraded":"2024-09-19T04:13:35.460051+0000","last_fullsized":"2024-09-19T04:13:35.460051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:14.639083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3844,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131391+0000","last_change":"2024-09-19T04:13:28.984556+0000","last_active":"2024-09-19T04:13:30.131391+0000","last_peered":"2024-09-19T04:13:30.131391+0000","last_clean":"2024-09-19T04:13:30.131391+0000","last_became_active":"2024-09-19T04:13:28.983961+0000","last_became_peered":"2024-09-19T04:13:28.983961+0000","last_unstale":"2024-09-19T04:13:30.131391+0000","last_undegraded":"2024-09-19T04:13:30.131391+0000","last_fullsized":"2024-09-19T04:13:30.131391+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:32:25.400096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064342+0000","last_change":"2024-09-19T04:13:24.943383+0000","last_active":"2024-09-19T04:13:31.064342+0000","last_peered":"2024-09-19T04:13:31.064342+0000","last_clean":"2024-09-19T04:13:31.064342+0000","last_became_active":"2024-09-19T04:13:24.942094+0000","last_became_peered":"2024-09-19T04:13:24.942094+0000","last_unstale":"2024-09-19T04:13:31.064342+0000","last_undegraded":"2024-09-19T04:13:31.064342+0000","last_fullsized":"2024-09-19T04:13:31.064342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:15.642865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131674+0000","last_change":"2024-09-19T04:13:23.911338+0000","last_active":"2024-09-19T04:13:30.131674+0000","last_peered":"2024-09-19T04:13:30.131674+0000","last_clean":"2024-09-19T04:13:30.131674+0000","last_became_active":"2024-09-19T04:13:23.910962+0000","last_became_peered":"2024-09-19T04:13:23.910962+0000","last_unstale":"2024-09-19T04:13:30.131674+0000","last_undegraded":"2024-09-19T04:13:30.131674+0000","last_fullsized":"2024-09-19T04:13:30.131674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:39.854426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063766+0000","last_change":"2024-09-19T04:13:28.988142+0000","last_active":"2024-09-19T04:13:31.063766+0000","last_peered":"2024-09-19T04:13:31.063766+0000","last_clean":"2024-09-19T04:13:31.063766+0000","last_became_active":"2024-09-19T04:13:28.979829+0000","last_became_peered":"2024-09-19T04:13:28.979829+0000","last_unstale":"2024-09-19T04:13:31.063766+0000","last_undegraded":"2024-09-19T04:13:31.063766+0000","last_fullsized":"2024-09-19T04:13:31.063766+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:47.654034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062345+0000","last_change":"2024-09-19T04:13:24.956553+0000","last_active":"2024-09-19T04:13:31.062345+0000","last_peered":"2024-09-19T04:13:31.062345+0000","last_clean":"2024-09-19T04:13:31.062345+0000","last_became_active":"2024-09-19T04:13:24.956238+0000","last_became_peered":"2024-09-19T04:13:24.956238+0000","last_unstale":"2024-09-19T04:13:31.062345+0000","last_undegraded":"2024-09-19T04:13:31.062345+0000","last_fullsized":"2024-09-19T04:13:31.062345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970527+0000","last_change":"2024-09-19T04:13:23.918490+0000","last_active":"2024-09-19T04:13:29.970527+0000","last_peered":"2024-09-19T04:13:29.970527+0000","last_clean":"2024-09-19T04:13:29.970527+0000","last_became_active":"2024-09-19T04:13:23.917969+0000","last_became_peered":"2024-09-19T04:13:23.917969+0000","last_unstale":"2024-09-19T04:13:29.970527+0000","last_undegraded":"2024-09-19T04:13:29.970527+0000","last_fullsized":"2024-09-19T04:13:29.970527+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:51.494289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300379+0000","last_change":"2024-09-19T04:13:28.988707+0000","last_active":"2024-09-19T04:13:30.300379+0000","last_peered":"2024-09-19T04:13:30.300379+0000","last_clean":"2024-09-19T04:13:30.300379+0000","last_became_active":"2024-09-19T04:13:28.988388+0000","last_became_peered":"2024-09-19T04:13:28.988388+0000","last_unstale":"2024-09-19T04:13:30.300379+0000","last_undegraded":"2024-09-19T04:13:30.300379+0000","last_fullsized":"2024-09-19T04:13:30.300379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:54.745179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970310+0000","last_change":"2024-09-19T04:13:24.943308+0000","last_active":"2024-09-19T04:13:29.970310+0000","last_peered":"2024-09-19T04:13:29.970310+0000","last_clean":"2024-09-19T04:13:29.970310+0000","last_became_active":"2024-09-19T04:13:24.942608+0000","last_became_peered":"2024-09-19T04:13:24.942608+0000","last_unstale":"2024-09-19T04:13:29.970310+0000","last_undegraded":"2024-09-19T04:13:29.970310+0000","last_fullsized":"2024-09-19T04:13:29.970310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972997+0000","last_change":"2024-09-19T04:13:23.918096+0000","last_active":"2024-09-19T04:13:29.972997+0000","last_peered":"2024-09-19T04:13:29.972997+0000","last_clean":"2024-09-19T04:13:29.972997+0000","last_became_active":"2024-09-19T04:13:23.917505+0000","last_became_peered":"2024-09-19T04:13:23.917505+0000","last_unstale":"2024-09-19T04:13:29.972997+0000","last_undegraded":"2024-09-19T04:13:29.972997+0000","last_fullsized":"2024-09-19T04:13:29.972997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:48.620777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300417+0000","last_change":"2024-09-19T04:13:29.027451+0000","last_active":"2024-09-19T04:13:30.300417+0000","last_peered":"2024-09-19T04:13:30.300417+0000","last_clean":"2024-09-19T04:13:30.300417+0000","last_became_active":"2024-09-19T04:13:29.027181+0000","last_became_peered":"2024-09-19T04:13:29.027181+0000","last_unstale":"2024-09-19T04:13:30.300417+0000","last_undegraded":"2024-09-19T04:13:30.300417+0000","last_fullsized":"2024-09-19T04:13:30.300417+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:42.206935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062147+0000","last_change":"2024-09-19T04:13:29.028793+0000","last_active":"2024-09-19T04:13:31.062147+0000","last_peered":"2024-09-19T04:13:31.062147+0000","last_clean":"2024-09-19T04:13:31.062147+0000","last_became_active":"2024-09-19T04:13:29.028293+0000","last_became_peered":"2024-09-19T04:13:29.028293+0000","last_unstale":"2024-09-19T04:13:31.062147+0000","last_undegraded":"2024-09-19T04:13:31.062147+0000","last_fullsized":"2024-09-19T04:13:31.062147+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:56.826800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.130915+0000","last_change":"2024-09-19T04:13:24.947415+0000","last_active":"2024-09-19T04:13:30.130915+0000","last_peered":"2024-09-19T04:13:30.130915+0000","last_clean":"2024-09-19T04:13:30.130915+0000","last_became_active":"2024-09-19T04:13:24.947042+0000","last_became_peered":"2024-09-19T04:13:24.947042+0000","last_unstale":"2024-09-19T04:13:30.130915+0000","last_undegraded":"2024-09-19T04:13:30.130915+0000","last_fullsized":"2024-09-19T04:13:30.130915+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969735+0000","last_change":"2024-09-19T04:13:23.906982+0000","last_active":"2024-09-19T04:13:29.969735+0000","last_peered":"2024-09-19T04:13:29.969735+0000","last_clean":"2024-09-19T04:13:29.969735+0000","last_became_active":"2024-09-19T04:13:23.905070+0000","last_became_peered":"2024-09-19T04:13:23.905070+0000","last_unstale":"2024-09-19T04:13:29.969735+0000","last_undegraded":"2024-09-19T04:13:29.969735+0000","last_fullsized":"2024-09-19T04:13:29.969735+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972911+0000","last_change":"2024-09-19T04:13:28.979353+0000","last_active":"2024-09-19T04:13:29.972911+0000","last_peered":"2024-09-19T04:13:29.972911+0000","last_clean":"2024-09-19T04:13:29.972911+0000","last_became_active":"2024-09-19T04:13:28.976991+0000","last_became_peered":"2024-09-19T04:13:28.976991+0000","last_unstale":"2024-09-19T04:13:29.972911+0000","last_undegraded":"2024-09-19T04:13:29.972911+0000","last_fullsized":"2024-09-19T04:13:29.972911+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:04.928733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300060+0000","last_change":"2024-09-19T04:13:24.947190+0000","last_active":"2024-09-19T04:13:30.300060+0000","last_peered":"2024-09-19T04:13:30.300060+0000","last_clean":"2024-09-19T04:13:30.300060+0000","last_became_active":"2024-09-19T04:13:24.946883+0000","last_became_peered":"2024-09-19T04:13:24.946883+0000","last_unstale":"2024-09-19T04:13:30.300060+0000","last_undegraded":"2024-09-19T04:13:30.300060+0000","last_fullsized":"2024-09-19T04:13:30.300060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300792+0000","last_change":"2024-09-19T04:13:23.912532+0000","last_active":"2024-09-19T04:13:30.300792+0000","last_peered":"2024-09-19T04:13:30.300792+0000","last_clean":"2024-09-19T04:13:30.300792+0000","last_became_active":"2024-09-19T04:13:23.911952+0000","last_became_peered":"2024-09-19T04:13:23.911952+0000","last_unstale":"2024-09-19T04:13:30.300792+0000","last_undegraded":"2024-09-19T04:13:30.300792+0000","last_fullsized":"2024-09-19T04:13:30.300792+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971798+0000","last_change":"2024-09-19T04:13:23.912898+0000","last_active":"2024-09-19T04:13:29.971798+0000","last_peered":"2024-09-19T04:13:29.971798+0000","last_clean":"2024-09-19T04:13:29.971798+0000","last_became_active":"2024-09-19T04:13:23.912234+0000","last_became_peered":"2024-09-19T04:13:23.912234+0000","last_unstale":"2024-09-19T04:13:29.971798+0000","last_undegraded":"2024-09-19T04:13:29.971798+0000","last_fullsized":"2024-09-19T04:13:29.971798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:59.362344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062138+0000","last_change":"2024-09-19T04:13:24.953761+0000","last_active":"2024-09-19T04:13:31.062138+0000","last_peered":"2024-09-19T04:13:31.062138+0000","last_clean":"2024-09-19T04:13:31.062138+0000","last_became_active":"2024-09-19T04:13:24.953195+0000","last_became_peered":"2024-09-19T04:13:24.953195+0000","last_unstale":"2024-09-19T04:13:31.062138+0000","last_undegraded":"2024-09-19T04:13:31.062138+0000","last_fullsized":"2024-09-19T04:13:31.062138+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971169+0000","last_change":"2024-09-19T04:13:28.976510+0000","last_active":"2024-09-19T04:13:29.971169+0000","last_peered":"2024-09-19T04:13:29.971169+0000","last_clean":"2024-09-19T04:13:29.971169+0000","last_became_active":"2024-09-19T04:13:28.975842+0000","last_became_peered":"2024-09-19T04:13:28.975842+0000","last_unstale":"2024-09-19T04:13:29.971169+0000","last_undegraded":"2024-09-19T04:13:29.971169+0000","last_fullsized":"2024-09-19T04:13:29.971169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:36:39.502090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301204+0000","last_change":"2024-09-19T04:13:23.920382+0000","last_active":"2024-09-19T04:13:30.301204+0000","last_peered":"2024-09-19T04:13:30.301204+0000","last_clean":"2024-09-19T04:13:30.301204+0000","last_became_active":"2024-09-19T04:13:23.920107+0000","last_became_peered":"2024-09-19T04:13:23.920107+0000","last_unstale":"2024-09-19T04:13:30.301204+0000","last_undegraded":"2024-09-19T04:13:30.301204+0000","last_fullsized":"2024-09-19T04:13:30.301204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:22.869628+0000","last_clean_scrub_stamp":"2024-09-19T04:13:22.869628+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:32:24.725046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062882+0000","last_change":"2024-09-19T04:13:24.933838+0000","last_active":"2024-09-19T04:13:31.062882+0000","last_peered":"2024-09-19T04:13:31.062882+0000","last_clean":"2024-09-19T04:13:31.062882+0000","last_became_active":"2024-09-19T04:13:24.932423+0000","last_became_peered":"2024-09-19T04:13:24.932423+0000","last_unstale":"2024-09-19T04:13:31.062882+0000","last_undegraded":"2024-09-19T04:13:31.062882+0000","last_fullsized":"2024-09-19T04:13:31.062882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:53:52.498071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301388+0000","last_change":"2024-09-19T04:13:28.982366+0000","last_active":"2024-09-19T04:13:30.301388+0000","last_peered":"2024-09-19T04:13:30.301388+0000","last_clean":"2024-09-19T04:13:30.301388+0000","last_became_active":"2024-09-19T04:13:28.981608+0000","last_became_peered":"2024-09-19T04:13:28.981608+0000","last_unstale":"2024-09-19T04:13:30.301388+0000","last_undegraded":"2024-09-19T04:13:30.301388+0000","last_fullsized":"2024-09-19T04:13:30.301388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:27.930703+0000","last_clean_scrub_stamp":"2024-09-19T04:13:27.930703+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:10:12.864952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969437+0000","last_change":"2024-09-19T04:13:24.939240+0000","last_active":"2024-09-19T04:13:29.969437+0000","last_peered":"2024-09-19T04:13:29.969437+0000","last_clean":"2024-09-19T04:13:29.969437+0000","last_became_active":"2024-09-19T04:13:24.938527+0000","last_became_peered":"2024-09-19T04:13:24.938527+0000","last_unstale":"2024-09-19T04:13:29.969437+0000","last_undegraded":"2024-09-19T04:13:29.969437+0000","last_fullsized":"2024-09-19T04:13:29.969437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970473+0000","last_change":"2024-09-19T04:13:24.967773+0000","last_active":"2024-09-19T04:13:29.970473+0000","last_peered":"2024-09-19T04:13:29.970473+0000","last_clean":"2024-09-19T04:13:29.970473+0000","last_became_active":"2024-09-19T04:13:24.967467+0000","last_became_peered":"2024-09-19T04:13:24.967467+0000","last_unstale":"2024-09-19T04:13:29.970473+0000","last_undegraded":"2024-09-19T04:13:29.970473+0000","last_fullsized":"2024-09-19T04:13:29.970473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:54:11.155011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971529+0000","last_change":"2024-09-19T04:13:24.945017+0000","last_active":"2024-09-19T04:13:29.971529+0000","last_peered":"2024-09-19T04:13:29.971529+0000","last_clean":"2024-09-19T04:13:29.971529+0000","last_became_active":"2024-09-19T04:13:24.944535+0000","last_became_peered":"2024-09-19T04:13:24.944535+0000","last_unstale":"2024-09-19T04:13:29.971529+0000","last_undegraded":"2024-09-19T04:13:29.971529+0000","last_fullsized":"2024-09-19T04:13:29.971529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:43.596741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062090+0000","last_change":"2024-09-19T04:13:24.952497+0000","last_active":"2024-09-19T04:13:31.062090+0000","last_peered":"2024-09-19T04:13:31.062090+0000","last_clean":"2024-09-19T04:13:31.062090+0000","last_became_active":"2024-09-19T04:13:24.952037+0000","last_became_peered":"2024-09-19T04:13:24.952037+0000","last_unstale":"2024-09-19T04:13:31.062090+0000","last_undegraded":"2024-09-19T04:13:31.062090+0000","last_fullsized":"2024-09-19T04:13:31.062090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:19.238015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301786+0000","last_change":"2024-09-19T04:13:24.966504+0000","last_active":"2024-09-19T04:13:30.301786+0000","last_peered":"2024-09-19T04:13:30.301786+0000","last_clean":"2024-09-19T04:13:30.301786+0000","last_became_active":"2024-09-19T04:13:24.965992+0000","last_became_peered":"2024-09-19T04:13:24.965992+0000","last_unstale":"2024-09-19T04:13:30.301786+0000","last_undegraded":"2024-09-19T04:13:30.301786+0000","last_fullsized":"2024-09-19T04:13:30.301786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131526+0000","last_change":"2024-09-19T04:13:24.949970+0000","last_active":"2024-09-19T04:13:30.131526+0000","last_peered":"2024-09-19T04:13:30.131526+0000","last_clean":"2024-09-19T04:13:30.131526+0000","last_became_active":"2024-09-19T04:13:24.949676+0000","last_became_peered":"2024-09-19T04:13:24.949676+0000","last_unstale":"2024-09-19T04:13:30.131526+0000","last_undegraded":"2024-09-19T04:13:30.131526+0000","last_fullsized":"2024-09-19T04:13:30.131526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973014+0000","last_change":"2024-09-19T04:13:24.939046+0000","last_active":"2024-09-19T04:13:29.973014+0000","last_peered":"2024-09-19T04:13:29.973014+0000","last_clean":"2024-09-19T04:13:29.973014+0000","last_became_active":"2024-09-19T04:13:24.938277+0000","last_became_peered":"2024-09-19T04:13:24.938277+0000","last_unstale":"2024-09-19T04:13:29.973014+0000","last_undegraded":"2024-09-19T04:13:29.973014+0000","last_fullsized":"2024-09-19T04:13:29.973014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.301516+0000","last_change":"2024-09-19T04:13:24.962572+0000","last_active":"2024-09-19T04:13:30.301516+0000","last_peered":"2024-09-19T04:13:30.301516+0000","last_clean":"2024-09-19T04:13:30.301516+0000","last_became_active":"2024-09-19T04:13:24.962277+0000","last_became_peered":"2024-09-19T04:13:24.962277+0000","last_unstale":"2024-09-19T04:13:30.301516+0000","last_undegraded":"2024-09-19T04:13:30.301516+0000","last_fullsized":"2024-09-19T04:13:30.301516+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969487+0000","last_change":"2024-09-19T04:13:24.951536+0000","last_active":"2024-09-19T04:13:29.969487+0000","last_peered":"2024-09-19T04:13:29.969487+0000","last_clean":"2024-09-19T04:13:29.969487+0000","last_became_active":"2024-09-19T04:13:24.950939+0000","last_became_peered":"2024-09-19T04:13:24.950939+0000","last_unstale":"2024-09-19T04:13:29.969487+0000","last_undegraded":"2024-09-19T04:13:29.969487+0000","last_fullsized":"2024-09-19T04:13:29.969487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:56.557066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300567+0000","last_change":"2024-09-19T04:13:24.961761+0000","last_active":"2024-09-19T04:13:30.300567+0000","last_peered":"2024-09-19T04:13:30.300567+0000","last_clean":"2024-09-19T04:13:30.300567+0000","last_became_active":"2024-09-19T04:13:24.961415+0000","last_became_peered":"2024-09-19T04:13:24.961415+0000","last_unstale":"2024-09-19T04:13:30.300567+0000","last_undegraded":"2024-09-19T04:13:30.300567+0000","last_fullsized":"2024-09-19T04:13:30.300567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.973133+0000","last_change":"2024-09-19T04:13:24.939730+0000","last_active":"2024-09-19T04:13:29.973133+0000","last_peered":"2024-09-19T04:13:29.973133+0000","last_clean":"2024-09-19T04:13:29.973133+0000","last_became_active":"2024-09-19T04:13:24.938400+0000","last_became_peered":"2024-09-19T04:13:24.938400+0000","last_unstale":"2024-09-19T04:13:29.973133+0000","last_undegraded":"2024-09-19T04:13:29.973133+0000","last_fullsized":"2024-09-19T04:13:29.973133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:57:35.405665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062008+0000","last_change":"2024-09-19T04:13:24.944592+0000","last_active":"2024-09-19T04:13:31.062008+0000","last_peered":"2024-09-19T04:13:31.062008+0000","last_clean":"2024-09-19T04:13:31.062008+0000","last_became_active":"2024-09-19T04:13:24.943915+0000","last_became_peered":"2024-09-19T04:13:24.943915+0000","last_unstale":"2024-09-19T04:13:31.062008+0000","last_undegraded":"2024-09-19T04:13:31.062008+0000","last_fullsized":"2024-09-19T04:13:31.062008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:18:30.226492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972579+0000","last_change":"2024-09-19T04:13:24.967058+0000","last_active":"2024-09-19T04:13:29.972579+0000","last_peered":"2024-09-19T04:13:29.972579+0000","last_clean":"2024-09-19T04:13:29.972579+0000","last_became_active":"2024-09-19T04:13:24.964521+0000","last_became_peered":"2024-09-19T04:13:24.964521+0000","last_unstale":"2024-09-19T04:13:29.972579+0000","last_undegraded":"2024-09-19T04:13:29.972579+0000","last_fullsized":"2024-09-19T04:13:29.972579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.063128+0000","last_change":"2024-09-19T04:13:24.953833+0000","last_active":"2024-09-19T04:13:31.063128+0000","last_peered":"2024-09-19T04:13:31.063128+0000","last_clean":"2024-09-19T04:13:31.063128+0000","last_became_active":"2024-09-19T04:13:24.950878+0000","last_became_peered":"2024-09-19T04:13:24.950878+0000","last_unstale":"2024-09-19T04:13:31.063128+0000","last_undegraded":"2024-09-19T04:13:31.063128+0000","last_fullsized":"2024-09-19T04:13:31.063128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:40:07.653424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971483+0000","last_change":"2024-09-19T04:13:24.956195+0000","last_active":"2024-09-19T04:13:29.971483+0000","last_peered":"2024-09-19T04:13:29.971483+0000","last_clean":"2024-09-19T04:13:29.971483+0000","last_became_active":"2024-09-19T04:13:24.955911+0000","last_became_peered":"2024-09-19T04:13:24.955911+0000","last_unstale":"2024-09-19T04:13:29.971483+0000","last_undegraded":"2024-09-19T04:13:29.971483+0000","last_fullsized":"2024-09-19T04:13:29.971483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:55.185297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.970952+0000","last_change":"2024-09-19T04:13:24.939650+0000","last_active":"2024-09-19T04:13:29.970952+0000","last_peered":"2024-09-19T04:13:29.970952+0000","last_clean":"2024-09-19T04:13:29.970952+0000","last_became_active":"2024-09-19T04:13:24.938774+0000","last_became_peered":"2024-09-19T04:13:24.938774+0000","last_unstale":"2024-09-19T04:13:29.970952+0000","last_undegraded":"2024-09-19T04:13:29.970952+0000","last_fullsized":"2024-09-19T04:13:29.970952+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:20:00.871440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969219+0000","last_change":"2024-09-19T04:13:24.921407+0000","last_active":"2024-09-19T04:13:29.969219+0000","last_peered":"2024-09-19T04:13:29.969219+0000","last_clean":"2024-09-19T04:13:29.969219+0000","last_became_active":"2024-09-19T04:13:24.920646+0000","last_became_peered":"2024-09-19T04:13:24.920646+0000","last_unstale":"2024-09-19T04:13:29.969219+0000","last_undegraded":"2024-09-19T04:13:29.969219+0000","last_fullsized":"2024-09-19T04:13:29.969219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131047+0000","last_change":"2024-09-19T04:13:24.963230+0000","last_active":"2024-09-19T04:13:30.131047+0000","last_peered":"2024-09-19T04:13:30.131047+0000","last_clean":"2024-09-19T04:13:30.131047+0000","last_became_active":"2024-09-19T04:13:24.962932+0000","last_became_peered":"2024-09-19T04:13:24.962932+0000","last_unstale":"2024-09-19T04:13:30.131047+0000","last_undegraded":"2024-09-19T04:13:30.131047+0000","last_fullsized":"2024-09-19T04:13:30.131047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:11.490736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061900+0000","last_change":"2024-09-19T04:13:24.942848+0000","last_active":"2024-09-19T04:13:31.061900+0000","last_peered":"2024-09-19T04:13:31.061900+0000","last_clean":"2024-09-19T04:13:31.061900+0000","last_became_active":"2024-09-19T04:13:24.942524+0000","last_became_peered":"2024-09-19T04:13:24.942524+0000","last_unstale":"2024-09-19T04:13:31.061900+0000","last_undegraded":"2024-09-19T04:13:31.061900+0000","last_fullsized":"2024-09-19T04:13:31.061900+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:25:26.370075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.300722+0000","last_change":"2024-09-19T04:13:24.950629+0000","last_active":"2024-09-19T04:13:30.300722+0000","last_peered":"2024-09-19T04:13:30.300722+0000","last_clean":"2024-09-19T04:13:30.300722+0000","last_became_active":"2024-09-19T04:13:24.946193+0000","last_became_peered":"2024-09-19T04:13:24.946193+0000","last_unstale":"2024-09-19T04:13:30.300722+0000","last_undegraded":"2024-09-19T04:13:30.300722+0000","last_fullsized":"2024-09-19T04:13:30.300722+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064210+0000","last_change":"2024-09-19T04:13:24.936924+0000","last_active":"2024-09-19T04:13:31.064210+0000","last_peered":"2024-09-19T04:13:31.064210+0000","last_clean":"2024-09-19T04:13:31.064210+0000","last_became_active":"2024-09-19T04:13:24.936053+0000","last_became_peered":"2024-09-19T04:13:24.936053+0000","last_unstale":"2024-09-19T04:13:31.064210+0000","last_undegraded":"2024-09-19T04:13:31.064210+0000","last_fullsized":"2024-09-19T04:13:31.064210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:51:40.858208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972557+0000","last_change":"2024-09-19T04:13:24.967121+0000","last_active":"2024-09-19T04:13:29.972557+0000","last_peered":"2024-09-19T04:13:29.972557+0000","last_clean":"2024-09-19T04:13:29.972557+0000","last_became_active":"2024-09-19T04:13:24.964161+0000","last_became_peered":"2024-09-19T04:13:24.964161+0000","last_unstale":"2024-09-19T04:13:29.972557+0000","last_undegraded":"2024-09-19T04:13:29.972557+0000","last_fullsized":"2024-09-19T04:13:29.972557+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.972765+0000","last_change":"2024-09-19T04:13:24.968947+0000","last_active":"2024-09-19T04:13:29.972765+0000","last_peered":"2024-09-19T04:13:29.972765+0000","last_clean":"2024-09-19T04:13:29.972765+0000","last_became_active":"2024-09-19T04:13:24.968335+0000","last_became_peered":"2024-09-19T04:13:24.968335+0000","last_unstale":"2024-09-19T04:13:29.972765+0000","last_undegraded":"2024-09-19T04:13:29.972765+0000","last_fullsized":"2024-09-19T04:13:29.972765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:13.645762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064806+0000","last_change":"2024-09-19T04:13:24.970001+0000","last_active":"2024-09-19T04:13:31.064806+0000","last_peered":"2024-09-19T04:13:31.064806+0000","last_clean":"2024-09-19T04:13:31.064806+0000","last_became_active":"2024-09-19T04:13:24.969462+0000","last_became_peered":"2024-09-19T04:13:24.969462+0000","last_unstale":"2024-09-19T04:13:31.064806+0000","last_undegraded":"2024-09-19T04:13:31.064806+0000","last_fullsized":"2024-09-19T04:13:31.064806+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:15:32.140847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.299773+0000","last_change":"2024-09-19T04:13:24.967702+0000","last_active":"2024-09-19T04:13:30.299773+0000","last_peered":"2024-09-19T04:13:30.299773+0000","last_clean":"2024-09-19T04:13:30.299773+0000","last_became_active":"2024-09-19T04:13:24.967436+0000","last_became_peered":"2024-09-19T04:13:24.967436+0000","last_unstale":"2024-09-19T04:13:30.299773+0000","last_undegraded":"2024-09-19T04:13:30.299773+0000","last_fullsized":"2024-09-19T04:13:30.299773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:47.952059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061448+0000","last_change":"2024-09-19T04:13:24.942637+0000","last_active":"2024-09-19T04:13:31.061448+0000","last_peered":"2024-09-19T04:13:31.061448+0000","last_clean":"2024-09-19T04:13:31.061448+0000","last_became_active":"2024-09-19T04:13:24.942164+0000","last_became_peered":"2024-09-19T04:13:24.942164+0000","last_unstale":"2024-09-19T04:13:31.061448+0000","last_undegraded":"2024-09-19T04:13:31.061448+0000","last_fullsized":"2024-09-19T04:13:31.061448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:29:30.804442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.971132+0000","last_change":"2024-09-19T04:13:24.926176+0000","last_active":"2024-09-19T04:13:29.971132+0000","last_peered":"2024-09-19T04:13:29.971132+0000","last_clean":"2024-09-19T04:13:29.971132+0000","last_became_active":"2024-09-19T04:13:24.925246+0000","last_became_peered":"2024-09-19T04:13:24.925246+0000","last_unstale":"2024-09-19T04:13:29.971132+0000","last_undegraded":"2024-09-19T04:13:29.971132+0000","last_fullsized":"2024-09-19T04:13:29.971132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:52.769507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.062030+0000","last_change":"2024-09-19T04:13:24.953254+0000","last_active":"2024-09-19T04:13:31.062030+0000","last_peered":"2024-09-19T04:13:31.062030+0000","last_clean":"2024-09-19T04:13:31.062030+0000","last_became_active":"2024-09-19T04:13:24.950624+0000","last_became_peered":"2024-09-19T04:13:24.950624+0000","last_unstale":"2024-09-19T04:13:31.062030+0000","last_undegraded":"2024-09-19T04:13:31.062030+0000","last_fullsized":"2024-09-19T04:13:31.062030+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:31:06.070557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:29.969906+0000","last_change":"2024-09-19T04:13:24.954966+0000","last_active":"2024-09-19T04:13:29.969906+0000","last_peered":"2024-09-19T04:13:29.969906+0000","last_clean":"2024-09-19T04:13:29.969906+0000","last_became_active":"2024-09-19T04:13:24.954234+0000","last_became_peered":"2024-09-19T04:13:24.954234+0000","last_unstale":"2024-09-19T04:13:29.969906+0000","last_undegraded":"2024-09-19T04:13:29.969906+0000","last_fullsized":"2024-09-19T04:13:29.969906+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:38.599955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":19,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.061764+0000","last_change":"2024-09-19T04:13:24.949281+0000","last_active":"2024-09-19T04:13:31.061764+0000","last_peered":"2024-09-19T04:13:31.061764+0000","last_clean":"2024-09-19T04:13:31.061764+0000","last_became_active":"2024-09-19T04:13:24.948863+0000","last_became_peered":"2024-09-19T04:13:24.948863+0000","last_unstale":"2024-09-19T04:13:31.061764+0000","last_undegraded":"2024-09-19T04:13:31.061764+0000","last_fullsized":"2024-09-19T04:13:31.061764+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:46.674435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:30.131694+0000","last_change":"2024-09-19T04:13:24.943099+0000","last_active":"2024-09-19T04:13:30.131694+0000","last_peered":"2024-09-19T04:13:30.131694+0000","last_clean":"2024-09-19T04:13:30.131694+0000","last_became_active":"2024-09-19T04:13:24.942776+0000","last_became_peered":"2024-09-19T04:13:24.942776+0000","last_unstale":"2024-09-19T04:13:30.131694+0000","last_undegraded":"2024-09-19T04:13:30.131694+0000","last_fullsized":"2024-09-19T04:13:30.131694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:09.596935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-19T04:13:31.064570+0000","last_change":"2024-09-19T04:13:24.951658+0000","last_active":"2024-09-19T04:13:31.064570+0000","last_peered":"2024-09-19T04:13:31.064570+0000","last_clean":"2024-09-19T04:13:31.064570+0000","last_became_active":"2024-09-19T04:13:24.950976+0000","last_became_peered":"2024-09-19T04:13:24.950976+0000","last_unstale":"2024-09-19T04:13:31.064570+0000","last_undegraded":"2024-09-19T04:13:31.064570+0000","last_fullsized":"2024-09-19T04:13:31.064570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:23.873411+0000","last_clean_scrub_stamp":"2024-09-19T04:13:23.873411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:39.061865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4642,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32032,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339808,"statfs":{"total":96636764160,"available":96603963392,"internally_reserved":0,"allocated":860160,"data_stored":694435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542153,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":274432,"data_stored":103611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":274432,"data_stored":107329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":103485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542153,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":103645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542153,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":868352,"data_stored":694491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:13:42.941 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:13:42.941 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T04:13:42.941 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-19T04:13:42.942 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-19T04:13:43.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa46d588640 1 Processor -- start 2024-09-19T04:13:43.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa46d588640 1 -- start start 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa46d588640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4680b5880 0x7fa46806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa46d588640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4680b3aa0 0x7fa46806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa46d588640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa46815ccb0 con 0x7fa46815c1c0 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa46d588640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa4680cd5b0 con 0x7fa4680b5880 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa46d588640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa4680cd890 con 0x7fa4680b3aa0 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4680b5880 0x7fa46806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4680b5880 0x7fa46806ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32888/0 (socket says 172.21.15.76:32888) 2024-09-19T04:13:43.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 -- 172.21.15.76:0/2820127899 learned_addr learned my addr 172.21.15.76:0/2820127899 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:43.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 -- 172.21.15.76:0/2820127899 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4680b3aa0 msgr2=0x7fa46806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:43.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 --2- 172.21.15.76:0/2820127899 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4680b3aa0 0x7fa46806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 -- 172.21.15.76:0/2820127899 >> v1:172.21.15.116:6789/0 conn(0x7fa46815c1c0 legacy=0x7fa46815c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:43.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 -- 172.21.15.76:0/2820127899 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4680cdb70 con 0x7fa4680b5880 2024-09-19T04:13:43.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.086+0000 7fa4667fc640 1 --2- 172.21.15.76:0/2820127899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b5880 0x7fa46806ccd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009870 tx=0x7fa45c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce14479b7284a953 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa465ffb640 1 -- 172.21.15.76:0/2820127899 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c0ab620 con 0x7fa4680b5880 2024-09-19T04:13:43.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa465ffb640 1 -- 172.21.15.76:0/2820127899 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa45c0b0070 con 0x7fa4680b5880 2024-09-19T04:13:43.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa465ffb640 1 -- 172.21.15.76:0/2820127899 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45c0ab9e0 con 0x7fa4680b5880 2024-09-19T04:13:43.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- 172.21.15.76:0/2820127899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b5880 msgr2=0x7fa46806ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 --2- 172.21.15.76:0/2820127899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b5880 0x7fa46806ccd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa45c009870 tx=0x7fa45c01c460 comp rx=0 tx=0).stop 2024-09-19T04:13:43.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- 172.21.15.76:0/2820127899 shutdown_connections 2024-09-19T04:13:43.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 --2- 172.21.15.76:0/2820127899 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4680b3aa0 0x7fa46806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 --2- 172.21.15.76:0/2820127899 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b5880 0x7fa46806ccd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- 172.21.15.76:0/2820127899 >> 172.21.15.76:0/2820127899 conn(0x7fa468072670 msgr2=0x7fa468072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:43.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- 172.21.15.76:0/2820127899 shutdown_connections 2024-09-19T04:13:43.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- 172.21.15.76:0/2820127899 wait complete. 2024-09-19T04:13:43.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 Processor -- start 2024-09-19T04:13:43.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- start start 2024-09-19T04:13:43.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b3aa0 0x7fa46814f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4680b5880 0x7fa46813ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa46815c1c0 0x7fa4681553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4680cefe0 con 0x7fa4680b3aa0 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa4680cee60 con 0x7fa4680b5880 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa46d588640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa468168df0 con 0x7fa46815c1c0 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b3aa0 0x7fa46814f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b3aa0 0x7fa46814f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32892/0 (socket says 172.21.15.76:32892) 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 -- 172.21.15.76:0/2671225876 learned_addr learned my addr 172.21.15.76:0/2671225876 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa4667fc640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4680b5880 0x7fa46813ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa4677fe640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa46815c1c0 0x7fa4681553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 -- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa46815c1c0 msgr2=0x7fa4681553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa46815c1c0 0x7fa4681553b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 -- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4680b5880 msgr2=0x7fa46813ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4680b5880 0x7fa46813ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 -- 172.21.15.76:0/2671225876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46813bd80 con 0x7fa4680b3aa0 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa4677fe640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa46815c1c0 0x7fa4681553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.090+0000 7fa466ffd640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b3aa0 0x7fa46814f440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c4c0 tx=0x7fa460001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa44f7fe640 1 -- 172.21.15.76:0/2671225876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa460017070 con 0x7fa4680b3aa0 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa44f7fe640 1 -- 172.21.15.76:0/2671225876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa460003600 con 0x7fa4680b3aa0 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46813b4e0 con 0x7fa4680b3aa0 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa44f7fe640 1 -- 172.21.15.76:0/2671225876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4600039f0 con 0x7fa4680b3aa0 2024-09-19T04:13:43.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46813baa0 con 0x7fa4680b3aa0 2024-09-19T04:13:43.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa468152320 con 0x7fa4680b3aa0 2024-09-19T04:13:43.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa44f7fe640 1 -- 172.21.15.76:0/2671225876 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa46001a070 con 0x7fa4680b3aa0 2024-09-19T04:13:43.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa44f7fe640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4540678c0 0x7fa454069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa4667fc640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4540678c0 0x7fa454069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa44f7fe640 1 -- 172.21.15.76:0/2671225876 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4600852d0 con 0x7fa4680b3aa0 2024-09-19T04:13:43.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.094+0000 7fa4667fc640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4540678c0 0x7fa454069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0af000 tx=0x7fa45c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.098+0000 7fa44f7fe640 1 -- 172.21.15.76:0/2671225876 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa460055b60 con 0x7fa4680b3aa0 2024-09-19T04:13:43.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa4681561c0 con 0x7fa4680b3aa0 2024-09-19T04:13:43.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa44f7fe640 1 -- 172.21.15.76:0/2671225876 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fa460055460 con 0x7fa4680b3aa0 2024-09-19T04:13:43.285 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:43.286 INFO:teuthology.orchestra.run.smithi076.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T04:13:43.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4540678c0 msgr2=0x7fa454069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4540678c0 0x7fa454069d80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa45c0af000 tx=0x7fa45c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b3aa0 msgr2=0x7fa46814f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b3aa0 0x7fa46814f440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa46000c4c0 tx=0x7fa460001e80 comp rx=0 tx=0).stop 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 shutdown_connections 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4540678c0 0x7fa454069d80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa46815c1c0 0x7fa4681553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4680b5880 0x7fa46813ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 --2- 172.21.15.76:0/2671225876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4680b3aa0 0x7fa46814f440 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 >> 172.21.15.76:0/2671225876 conn(0x7fa468072670 msgr2=0x7fa468159150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 shutdown_connections 2024-09-19T04:13:43.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.282+0000 7fa46d588640 1 -- 172.21.15.76:0/2671225876 wait complete. 2024-09-19T04:13:43.303 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-19T04:13:43.303 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:43.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.402+0000 7fa50a3c3640 1 Processor -- start 2024-09-19T04:13:43.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.402+0000 7fa50a3c3640 1 -- start start 2024-09-19T04:13:43.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa50a3c3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa504161a60 0x7fa5040d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa50a3c3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5040cf620 0x7fa50406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa50a3c3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5040d32b0 con 0x7fa5040d2e90 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa50a3c3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa5040d0300 con 0x7fa504161a60 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa50a3c3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa5040cfa90 con 0x7fa5040cf620 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa504161a60 0x7fa5040d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa504161a60 0x7fa5040d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32908/0 (socket says 172.21.15.76:32908) 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 -- 172.21.15.76:0/1029945183 learned_addr learned my addr 172.21.15.76:0/1029945183 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa508960640 1 --2- 172.21.15.76:0/1029945183 >> v2:172.21.15.116:3300/0 conn(0x7fa5040cf620 0x7fa50406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 -- 172.21.15.76:0/1029945183 >> v2:172.21.15.116:3300/0 conn(0x7fa5040cf620 msgr2=0x7fa50406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 --2- 172.21.15.76:0/1029945183 >> v2:172.21.15.116:3300/0 conn(0x7fa5040cf620 0x7fa50406d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 -- 172.21.15.76:0/1029945183 >> v1:172.21.15.116:6789/0 conn(0x7fa5040d2e90 legacy=0x7fa5040cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:43.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 -- 172.21.15.76:0/1029945183 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5040b0f60 con 0x7fa504161a60 2024-09-19T04:13:43.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa508960640 1 --2- 172.21.15.76:0/1029945183 >> v2:172.21.15.116:3300/0 conn(0x7fa5040cf620 0x7fa50406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa503fff640 1 --2- 172.21.15.76:0/1029945183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5040d2950 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ce950 tx=0x7fa4f401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65da6406b251ee19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa502ffd640 1 -- 172.21.15.76:0/1029945183 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40ab620 con 0x7fa504161a60 2024-09-19T04:13:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa502ffd640 1 -- 172.21.15.76:0/1029945183 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4f40b0070 con 0x7fa504161a60 2024-09-19T04:13:43.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.406+0000 7fa502ffd640 1 -- 172.21.15.76:0/1029945183 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4f40ab9e0 con 0x7fa504161a60 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- 172.21.15.76:0/1029945183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 msgr2=0x7fa5040d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/1029945183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5040d2950 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa5040ce950 tx=0x7fa4f401c420 comp rx=0 tx=0).stop 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- 172.21.15.76:0/1029945183 shutdown_connections 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/1029945183 >> v2:172.21.15.116:3300/0 conn(0x7fa5040cf620 0x7fa50406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/1029945183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5040d2950 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- 172.21.15.76:0/1029945183 >> 172.21.15.76:0/1029945183 conn(0x7fa50408bf90 msgr2=0x7fa50409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- 172.21.15.76:0/1029945183 shutdown_connections 2024-09-19T04:13:43.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- 172.21.15.76:0/1029945183 wait complete. 2024-09-19T04:13:43.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 Processor -- start 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- start start 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5040cf620 0x7fa504157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5040d2e90 0x7fa504151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5041518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa50406e400 con 0x7fa504161a60 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa50406d190 con 0x7fa5040cf620 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa50406e580 con 0x7fa5040d2e90 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5041518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5041518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32912/0 (socket says 172.21.15.76:32912) 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 -- 172.21.15.76:0/3243735643 learned_addr learned my addr 172.21.15.76:0/3243735643 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:43.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa503fff640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5040cf620 0x7fa504157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa5037fe640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5040d2e90 0x7fa504151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 -- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5040d2e90 msgr2=0x7fa504151380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5040d2e90 0x7fa504151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 -- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5040cf620 msgr2=0x7fa504157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5040cf620 0x7fa504157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 -- 172.21.15.76:0/3243735643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa504160430 con 0x7fa504161a60 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa508960640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5041518c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00cd50 tx=0x7fa4fc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa500ff9640 1 -- 172.21.15.76:0/3243735643 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00f680 con 0x7fa504161a60 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa500ff9640 1 -- 172.21.15.76:0/3243735643 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4fc00f840 con 0x7fa504161a60 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa504160710 con 0x7fa504161a60 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa500ff9640 1 -- 172.21.15.76:0/3243735643 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4fc00fc30 con 0x7fa504161a60 2024-09-19T04:13:43.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.410+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5041578c0 con 0x7fa504161a60 2024-09-19T04:13:43.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.414+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa50406d6b0 con 0x7fa504161a60 2024-09-19T04:13:43.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.414+0000 7fa500ff9640 1 -- 172.21.15.76:0/3243735643 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4fc0213f0 con 0x7fa504161a60 2024-09-19T04:13:43.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.414+0000 7fa500ff9640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4f00678c0 0x7fa4f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.414+0000 7fa503fff640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4f00678c0 0x7fa4f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.414+0000 7fa500ff9640 1 -- 172.21.15.76:0/3243735643 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4fc086db0 con 0x7fa504161a60 2024-09-19T04:13:43.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.414+0000 7fa503fff640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4f00678c0 0x7fa4f0069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40af000 tx=0x7fa4f40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.418+0000 7fa500ff9640 1 -- 172.21.15.76:0/3243735643 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa4fc014030 con 0x7fa504161a60 2024-09-19T04:13:43.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5040d1270 con 0x7fa504161a60 2024-09-19T04:13:43.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa500ff9640 1 -- 172.21.15.76:0/3243735643 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7fa4fc0575b0 con 0x7fa504161a60 2024-09-19T04:13:43.605 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:43.605 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:13:43.605 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:13:43.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4f00678c0 msgr2=0x7fa4f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4f00678c0 0x7fa4f0069d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa4f40af000 tx=0x7fa4f40a7000 comp rx=0 tx=0).stop 2024-09-19T04:13:43.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 msgr2=0x7fa5041518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5041518c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc00cd50 tx=0x7fa4fc00a650 comp rx=0 tx=0).stop 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 shutdown_connections 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa4f00678c0 0x7fa4f0069d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa504161a60 0x7fa5041518c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5040d2e90 0x7fa504151380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 --2- 172.21.15.76:0/3243735643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5040cf620 0x7fa504157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 >> 172.21.15.76:0/3243735643 conn(0x7fa50408bf90 msgr2=0x7fa5040d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 shutdown_connections 2024-09-19T04:13:43.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.602+0000 7fa50a3c3640 1 -- 172.21.15.76:0/3243735643 wait complete. 2024-09-19T04:13:43.623 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:43.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.730+0000 7fae34177640 1 Processor -- start 2024-09-19T04:13:43.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.730+0000 7fae34177640 1 -- start start 2024-09-19T04:13:43.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c15e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae2c15ecb0 0x7fae2c06dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae2c15f180 con 0x7fae2c06e180 2024-09-19T04:13:43.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fae2c15f460 con 0x7fae2c15ecb0 2024-09-19T04:13:43.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fae2c06d440 con 0x7fae2c0b5cf0 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae2c15ecb0 0x7fae2c06dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae2c15ecb0 0x7fae2c06dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32926/0 (socket says 172.21.15.76:32926) 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 -- 172.21.15.76:0/3993401826 learned_addr learned my addr 172.21.15.76:0/3993401826 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae32714640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fae2c06e180 0x7fae2c15f8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37518/0 (socket says 172.21.15.76:37518) 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31f13640 1 --2- 172.21.15.76:0/3993401826 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c15e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 -- 172.21.15.76:0/3993401826 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 msgr2=0x7fae2c15e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 --2- 172.21.15.76:0/3993401826 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c15e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 -- 172.21.15.76:0/3993401826 >> v1:172.21.15.76:6789/0 conn(0x7fae2c06e180 legacy=0x7fae2c15f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae30f11640 1 -- 172.21.15.76:0/3993401826 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4196680581 0 0) 0x7fae2c15f180 con 0x7fae2c06e180 2024-09-19T04:13:43.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 -- 172.21.15.76:0/3993401826 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c0b60d0 con 0x7fae2c15ecb0 2024-09-19T04:13:43.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae31712640 1 --2- 172.21.15.76:0/3993401826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c06dc40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae20009870 tx=0x7fae2001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5249af5afe099c4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae30f11640 1 -- 172.21.15.76:0/3993401826 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae200b0070 con 0x7fae2c15ecb0 2024-09-19T04:13:43.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae30f11640 1 -- 172.21.15.76:0/3993401826 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae200ab600 con 0x7fae2c15ecb0 2024-09-19T04:13:43.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae30f11640 1 -- 172.21.15.76:0/3993401826 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae200ab9d0 con 0x7fae2c15ecb0 2024-09-19T04:13:43.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 -- 172.21.15.76:0/3993401826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 msgr2=0x7fae2c06dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 --2- 172.21.15.76:0/3993401826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c06dc40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fae20009870 tx=0x7fae2001c4d0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 -- 172.21.15.76:0/3993401826 shutdown_connections 2024-09-19T04:13:43.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 --2- 172.21.15.76:0/3993401826 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c06dc40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 --2- 172.21.15.76:0/3993401826 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c15e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.734+0000 7fae34177640 1 -- 172.21.15.76:0/3993401826 >> 172.21.15.76:0/3993401826 conn(0x7fae2c06c0c0 msgr2=0x7fae2c06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:43.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- 172.21.15.76:0/3993401826 shutdown_connections 2024-09-19T04:13:43.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- 172.21.15.76:0/3993401826 wait complete. 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 Processor -- start 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- start start 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae2c06e180 0x7fae2c1654a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c16b130 con 0x7fae2c15ecb0 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae2c072900 con 0x7fae2c06e180 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fae2c16b2b0 con 0x7fae2c0b5cf0 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c1527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32928/0 (socket says 172.21.15.76:32928) 2024-09-19T04:13:43.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 -- 172.21.15.76:0/2886478820 learned_addr learned my addr 172.21.15.76:0/2886478820 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae31712640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae31f13640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae2c06e180 0x7fae2c1654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 -- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 msgr2=0x7fae2c163bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c163bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 -- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae2c06e180 msgr2=0x7fae2c1654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae2c06e180 0x7fae2c1654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 -- 172.21.15.76:0/2886478820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae2c15b500 con 0x7fae2c15ecb0 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae32714640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c1527d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00c780 tx=0x7fae1c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae2a7fc640 1 -- 172.21.15.76:0/2886478820 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c018070 con 0x7fae2c15ecb0 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae2a7fc640 1 -- 172.21.15.76:0/2886478820 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae1c00a640 con 0x7fae2c15ecb0 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae2a7fc640 1 -- 172.21.15.76:0/2886478820 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae1c00aab0 con 0x7fae2c15ecb0 2024-09-19T04:13:43.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae2c163110 con 0x7fae2c15ecb0 2024-09-19T04:13:43.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae2c1636d0 con 0x7fae2c15ecb0 2024-09-19T04:13:43.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.738+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae2c15e770 con 0x7fae2c15ecb0 2024-09-19T04:13:43.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.742+0000 7fae2a7fc640 1 -- 172.21.15.76:0/2886478820 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fae1c01b070 con 0x7fae2c15ecb0 2024-09-19T04:13:43.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.742+0000 7fae2a7fc640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae180678c0 0x7fae18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:43.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.742+0000 7fae31f13640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae180678c0 0x7fae18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:43.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.742+0000 7fae2a7fc640 1 -- 172.21.15.76:0/2886478820 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fae1c085620 con 0x7fae2c15ecb0 2024-09-19T04:13:43.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.742+0000 7fae31f13640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae180678c0 0x7fae18069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae2400b800 tx=0x7fae24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:43.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.746+0000 7fae2a7fc640 1 -- 172.21.15.76:0/2886478820 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fae1c055e20 con 0x7fae2c15ecb0 2024-09-19T04:13:43.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.926+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae2c069170 con 0x7fae2c15ecb0 2024-09-19T04:13:43.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.926+0000 7fae2a7fc640 1 -- 172.21.15.76:0/2886478820 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7fae1c055720 con 0x7fae2c15ecb0 2024-09-19T04:13:43.931 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:43.931 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:13:43.931 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae180678c0 msgr2=0x7fae18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae180678c0 0x7fae18069d80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fae2400b800 tx=0x7fae24008040 comp rx=0 tx=0).stop 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 msgr2=0x7fae2c1527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c1527d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae1c00c780 tx=0x7fae1c00cc40 comp rx=0 tx=0).stop 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 shutdown_connections 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae180678c0 0x7fae18069d80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae2c15ecb0 0x7fae2c1527d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae2c0b5cf0 0x7fae2c163bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 --2- 172.21.15.76:0/2886478820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae2c06e180 0x7fae2c1654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 >> 172.21.15.76:0/2886478820 conn(0x7fae2c06c0c0 msgr2=0x7fae2c1626c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:43.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 shutdown_connections 2024-09-19T04:13:43.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:43.930+0000 7fae34177640 1 -- 172.21.15.76:0/2886478820 wait complete. 2024-09-19T04:13:43.948 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:44.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.058+0000 7f0565c7e640 1 Processor -- start 2024-09-19T04:13:44.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.058+0000 7f0565c7e640 1 -- start start 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f056015c2a0 0x7f05600aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f05600af2a0 0x7f056006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05600b3290 con 0x7f056015bec0 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f05600b3570 con 0x7f05600af2a0 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f05600b3850 con 0x7f056015c2a0 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f056015bec0 0x7f05600ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37524/0 (socket says 172.21.15.76:37524) 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055f7fe640 1 -- 172.21.15.76:0/489974891 learned_addr learned my addr 172.21.15.76:0/489974891 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:44.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055ffff640 1 --2- 172.21.15.76:0/489974891 >> v2:172.21.15.76:3300/0 conn(0x7f05600af2a0 0x7f056006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055ffff640 1 -- 172.21.15.76:0/489974891 >> v2:172.21.15.116:3300/0 conn(0x7f056015c2a0 msgr2=0x7f05600aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055ffff640 1 --2- 172.21.15.76:0/489974891 >> v2:172.21.15.116:3300/0 conn(0x7f056015c2a0 0x7f05600aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055ffff640 1 -- 172.21.15.76:0/489974891 >> v1:172.21.15.76:6789/0 conn(0x7f056015bec0 legacy=0x7f05600ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055ffff640 1 -- 172.21.15.76:0/489974891 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600b3b30 con 0x7f05600af2a0 2024-09-19T04:13:44.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055e7fc640 1 -- 172.21.15.76:0/489974891 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2149338705 0 0) 0x7f05600b3290 con 0x7f056015bec0 2024-09-19T04:13:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055ffff640 1 --2- 172.21.15.76:0/489974891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05600af2a0 0x7f056006e000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f05600ae510 tx=0x7f055801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8af11fed8e7ad3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055e7fc640 1 -- 172.21.15.76:0/489974891 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580ab650 con 0x7f05600af2a0 2024-09-19T04:13:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055e7fc640 1 -- 172.21.15.76:0/489974891 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05580b0070 con 0x7f05600af2a0 2024-09-19T04:13:44.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f055e7fc640 1 -- 172.21.15.76:0/489974891 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05580aba10 con 0x7f05600af2a0 2024-09-19T04:13:44.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- 172.21.15.76:0/489974891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05600af2a0 msgr2=0x7f056006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 --2- 172.21.15.76:0/489974891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05600af2a0 0x7f056006e000 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f05600ae510 tx=0x7f055801c450 comp rx=0 tx=0).stop 2024-09-19T04:13:44.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- 172.21.15.76:0/489974891 shutdown_connections 2024-09-19T04:13:44.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 --2- 172.21.15.76:0/489974891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f05600af2a0 0x7f056006e000 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 --2- 172.21.15.76:0/489974891 >> v2:172.21.15.116:3300/0 conn(0x7f056015c2a0 0x7f05600aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- 172.21.15.76:0/489974891 >> 172.21.15.76:0/489974891 conn(0x7f05600b1f90 msgr2=0x7f05600b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:44.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- 172.21.15.76:0/489974891 shutdown_connections 2024-09-19T04:13:44.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.062+0000 7f0565c7e640 1 -- 172.21.15.76:0/489974891 wait complete. 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 Processor -- start 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 -- start start 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f05600af2a0 0x7f056014d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056015bec0 0x7f056014ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f056015c2a0 0x7f056014f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f056006ed50 con 0x7f056015bec0 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f05600af710 con 0x7f056015c2a0 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f056006eed0 con 0x7f05600af2a0 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056015bec0 0x7f056014ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056015bec0 0x7f056014ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32958/0 (socket says 172.21.15.76:32958) 2024-09-19T04:13:44.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 -- 172.21.15.76:0/1820850005 learned_addr learned my addr 172.21.15.76:0/1820850005 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 -- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f05600af2a0 msgr2=0x7f056014d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055f7fe640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f05600af2a0 0x7f056014d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055ffff640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f056015c2a0 0x7f056014f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f05600af2a0 0x7f056014d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 -- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f056015c2a0 msgr2=0x7f056014f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f056015c2a0 0x7f056014f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 -- 172.21.15.76:0/1820850005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0560153c00 con 0x7f056015bec0 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055f7fe640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f05600af2a0 0x7f056014d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f055effd640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056015bec0 0x7f056014ef10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f054800b420 tx=0x7f054800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0557fff640 1 -- 172.21.15.76:0/1820850005 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548011830 con 0x7f056015bec0 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0557fff640 1 -- 172.21.15.76:0/1820850005 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0548016070 con 0x7f056015bec0 2024-09-19T04:13:44.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0560153ee0 con 0x7f056015bec0 2024-09-19T04:13:44.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0557fff640 1 -- 172.21.15.76:0/1820850005 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0548011c10 con 0x7f056015bec0 2024-09-19T04:13:44.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05601544a0 con 0x7f056015bec0 2024-09-19T04:13:44.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f056006f1d0 con 0x7f056015bec0 2024-09-19T04:13:44.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.066+0000 7f0557fff640 1 -- 172.21.15.76:0/1820850005 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f054801b460 con 0x7f056015bec0 2024-09-19T04:13:44.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.070+0000 7f0557fff640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f05400678c0 0x7f0540069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.070+0000 7f0557fff640 1 -- 172.21.15.76:0/1820850005 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0548085cb0 con 0x7f056015bec0 2024-09-19T04:13:44.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.070+0000 7f055f7fe640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f05400678c0 0x7f0540069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.070+0000 7f055f7fe640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f05400678c0 0x7f0540069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0550007fd0 tx=0x7f0550009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.074+0000 7f0557fff640 1 -- 172.21.15.76:0/1820850005 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f05480564b0 con 0x7f056015bec0 2024-09-19T04:13:44.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.238+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05600b5170 con 0x7f056015bec0 2024-09-19T04:13:44.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.238+0000 7f0557fff640 1 -- 172.21.15.76:0/1820850005 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f0548055db0 con 0x7f056015bec0 2024-09-19T04:13:44.243 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:44.244 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f05400678c0 msgr2=0x7f0540069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f05400678c0 0x7f0540069d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0550007fd0 tx=0x7f0550009040 comp rx=0 tx=0).stop 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056015bec0 msgr2=0x7f056014ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056015bec0 0x7f056014ef10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f054800b420 tx=0x7f054800b8e0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 shutdown_connections 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f05400678c0 0x7f0540069d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f056015c2a0 0x7f056014f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f056015bec0 0x7f056014ef10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 --2- 172.21.15.76:0/1820850005 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f05600af2a0 0x7f056014d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 >> 172.21.15.76:0/1820850005 conn(0x7f05600b1f90 msgr2=0x7f05600b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:44.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 shutdown_connections 2024-09-19T04:13:44.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.242+0000 7f0565c7e640 1 -- 172.21.15.76:0/1820850005 wait complete. 2024-09-19T04:13:44.263 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:13:44.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4764572640 1 Processor -- start 2024-09-19T04:13:44.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4764572640 1 -- start start 2024-09-19T04:13:44.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4764572640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 0x7f475c166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4764572640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475c0b3290 con 0x7f475c15c2a0 2024-09-19T04:13:44.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4764572640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f475c0b3570 con 0x7f475c15bec0 2024-09-19T04:13:44.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4764572640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f475c0b3850 con 0x7f475c06f010 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4761b0d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f475c15c2a0 0x7f475c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37528/0 (socket says 172.21.15.76:37528) 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4761b0d640 1 -- 172.21.15.76:0/2939922349 learned_addr learned my addr 172.21.15.76:0/2939922349 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4762b0f640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 0x7f475c166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4217493468 0 0) 0x7f475c0b3290 con 0x7f475c15c2a0 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4748003530 con 0x7f475c15c2a0 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 598227310 0 0) 0x7f4748003530 con 0x7f475c15c2a0 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 msgr2=0x7f475c166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 0x7f475c166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 >> v1:172.21.15.116:6789/0 conn(0x7f475c15bec0 legacy=0x7f475c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c0b3b30 con 0x7f475c15c2a0 2024-09-19T04:13:44.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f4750001680 con 0x7f475c15c2a0 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 >> v1:172.21.15.76:6789/0 conn(0x7f475c15c2a0 legacy=0x7f475c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f4748003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 0x7f47480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f475c0b3850 con 0x7f47480039f0 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f474800ba80 con 0x7f4748004810 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f474800bc80 con 0x7f475c06f010 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476230e640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f4748003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4762b0f640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 0x7f47480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4761b0d640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476230e640 1 -- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 msgr2=0x7f47480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:44.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476230e640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c06f010 0x7f47480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476230e640 1 -- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4748004810 msgr2=0x7f4748004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:44.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476230e640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f476230e640 1 -- 172.21.15.76:0/2939922349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4748003530 con 0x7f47480039f0 2024-09-19T04:13:44.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.374+0000 7f4761b0d640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4748004810 0x7f4748004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:44.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f476230e640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f4748003e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4758001750 tx=0x7f475801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59520692cb4d6fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47580013a0 con 0x7f47480039f0 2024-09-19T04:13:44.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47580a4e50 con 0x7f47480039f0 2024-09-19T04:13:44.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f476130c640 1 -- 172.21.15.76:0/2939922349 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758003600 con 0x7f47480039f0 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 -- 172.21.15.76:0/2939922349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 msgr2=0x7f4748003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 --2- 172.21.15.76:0/2939922349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f4748003e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4758001750 tx=0x7f475801c390 comp rx=0 tx=0).stop 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4762b0f640 1 -- 172.21.15.76:0/2939922349 reap_dead start 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 -- 172.21.15.76:0/2939922349 shutdown_connections 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 -- 172.21.15.76:0/2939922349 >> 172.21.15.76:0/2939922349 conn(0x7f475c0b1f90 msgr2=0x7f475c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 -- 172.21.15.76:0/2939922349 shutdown_connections 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 -- 172.21.15.76:0/2939922349 wait complete. 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 Processor -- start 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.378+0000 7f4764572640 1 -- start start 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f475c1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f475c15f090 0x7f475c14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c14f960 0x7f475c158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f475c167980 con 0x7f47480039f0 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f475c167800 con 0x7f475c15f090 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f475c167b00 con 0x7f475c14f960 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f475c1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f475c1f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32968/0 (socket says 172.21.15.76:32968) 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 -- 172.21.15.76:0/3698187245 learned_addr learned my addr 172.21.15.76:0/3698187245 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4761b0d640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f475c15f090 0x7f475c14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4762b0f640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c14f960 0x7f475c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 -- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c14f960 msgr2=0x7f475c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c14f960 0x7f475c158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 -- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f475c15f090 msgr2=0x7f475c14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f475c15f090 0x7f475c14f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 -- 172.21.15.76:0/3698187245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f475c15e8c0 con 0x7f47480039f0 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4762b0f640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c14f960 0x7f475c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4761b0d640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f475c15f090 0x7f475c14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f476230e640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f475c1f5dc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4758001750 tx=0x7f4748005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f474effd640 1 -- 172.21.15.76:0/3698187245 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758000ee0 con 0x7f47480039f0 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f474effd640 1 -- 172.21.15.76:0/3698187245 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4758007040 con 0x7f47480039f0 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f474effd640 1 -- 172.21.15.76:0/3698187245 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47580071f0 con 0x7f47480039f0 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f475c1f8910 con 0x7f47480039f0 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f475c1f8ed0 con 0x7f47480039f0 2024-09-19T04:13:44.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f475c0ad200 con 0x7f47480039f0 2024-09-19T04:13:44.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f474effd640 1 -- 172.21.15.76:0/3698187245 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f47580bd020 con 0x7f47480039f0 2024-09-19T04:13:44.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f474effd640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4740067a60 0x7f4740069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:44.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4761b0d640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4740067a60 0x7f4740069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:44.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f474effd640 1 -- 172.21.15.76:0/3698187245 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f475811f870 con 0x7f47480039f0 2024-09-19T04:13:44.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.382+0000 7f4761b0d640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4740067a60 0x7f4740069f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b0ed0 tx=0x7f4750009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:44.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.386+0000 7f474effd640 1 -- 172.21.15.76:0/3698187245 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f47580c4020 con 0x7f47480039f0 2024-09-19T04:13:44.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.582+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f475c06fcd0 con 0x7f47480039f0 2024-09-19T04:13:44.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f474effd640 1 -- 172.21.15.76:0/3698187245 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7f47580160d0 con 0x7f47480039f0 2024-09-19T04:13:44.589 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:44.589 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:13:44.589 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:13:44.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4740067a60 msgr2=0x7f4740069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4740067a60 0x7f4740069f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f475c0b0ed0 tx=0x7f4750009040 comp rx=0 tx=0).stop 2024-09-19T04:13:44.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 msgr2=0x7f475c1f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:44.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f475c1f5dc0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4758001750 tx=0x7f4748005840 comp rx=0 tx=0).stop 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 shutdown_connections 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4740067a60 0x7f4740069f20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f475c14f960 0x7f475c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f475c15f090 0x7f475c14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 --2- 172.21.15.76:0/3698187245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f47480039f0 0x7f475c1f5dc0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 >> 172.21.15.76:0/3698187245 conn(0x7f475c0b1f90 msgr2=0x7f475c0b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 shutdown_connections 2024-09-19T04:13:44.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:44.586+0000 7f4764572640 1 -- 172.21.15.76:0/3698187245 wait complete. 2024-09-19T04:13:45.009 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:13:24.253935+0000', 'modified': '2024-09-19T04:13:38.364534+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4260}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4260': {'gid': 4260, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3901169819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3901169819}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3901169819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4260, 'qdb_cluster': [4260]} 2024-09-19T04:13:45.010 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:13:45.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:13:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.138+0000 7fd7a98fb640 1 Processor -- start 2024-09-19T04:13:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.138+0000 7fd7a98fb640 1 -- start start 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.138+0000 7fd7a98fb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7a415bec0 0x7fd7a40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.138+0000 7fd7a98fb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 0x7fd7a40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.138+0000 7fd7a98fb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7a40b3290 con 0x7fd7a40af2a0 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.138+0000 7fd7a98fb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a40b3570 con 0x7fd7a415bec0 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.138+0000 7fd7a98fb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a40b3850 con 0x7fd7a415c2a0 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd7a40af2a0 0x7fd7a415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37540/0 (socket says 172.21.15.76:37540) 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a37fe640 1 -- 172.21.15.76:0/1483869881 learned_addr learned my addr 172.21.15.76:0/1483869881 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 0x7fd7a40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/1483869881 >> v2:172.21.15.116:3300/0 conn(0x7fd7a415bec0 0x7fd7a40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3865248167 0 0) 0x7fd7a40b3290 con 0x7fd7a40af2a0 2024-09-19T04:13:45.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd78c003530 con 0x7fd7a40af2a0 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2400249966 0 0) 0x7fd78c003530 con 0x7fd7a40af2a0 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 msgr2=0x7fd7a40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 0x7fd7a40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 >> v2:172.21.15.116:3300/0 conn(0x7fd7a415bec0 msgr2=0x7fd7a40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 --2- 172.21.15.76:0/1483869881 >> v2:172.21.15.116:3300/0 conn(0x7fd7a415bec0 0x7fd7a40ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a40b3b30 con 0x7fd7a40af2a0 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fd79c001a00 con 0x7fd7a40af2a0 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 >> v1:172.21.15.76:6789/0 conn(0x7fd7a40af2a0 legacy=0x7fd7a415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78c0039c0 0x7fd78c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd78c004810 0x7fd78c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 0x7fd78c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a40b3570 con 0x7fd78c0039c0 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a40b3850 con 0x7fd78c004810 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd78c00b9d0 con 0x7fd7a415c2a0 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78c0039c0 0x7fd78c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 0x7fd78c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd78c004810 0x7fd78c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 -- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 msgr2=0x7fd78c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7a415c2a0 0x7fd78c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 -- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd78c004810 msgr2=0x7fd78c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd78c004810 0x7fd78c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 -- 172.21.15.76:0/1483869881 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78c003530 con 0x7fd78c0039c0 2024-09-19T04:13:45.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd78c004810 0x7fd78c004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:45.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78c0039c0 0x7fd78c003de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd794009ae0 tx=0x7fd79401ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3382d85ed88be099 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940b2070 con 0x7fd78c0039c0 2024-09-19T04:13:45.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7940a5570 con 0x7fd78c0039c0 2024-09-19T04:13:45.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.142+0000 7fd7a1ffb640 1 -- 172.21.15.76:0/1483869881 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7940096b0 con 0x7fd78c0039c0 2024-09-19T04:13:45.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- 172.21.15.76:0/1483869881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78c0039c0 msgr2=0x7fd78c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 --2- 172.21.15.76:0/1483869881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd78c0039c0 0x7fd78c003de0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd794009ae0 tx=0x7fd79401ca80 comp rx=0 tx=0).stop 2024-09-19T04:13:45.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 -- 172.21.15.76:0/1483869881 reap_dead start 2024-09-19T04:13:45.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- 172.21.15.76:0/1483869881 shutdown_connections 2024-09-19T04:13:45.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- 172.21.15.76:0/1483869881 >> 172.21.15.76:0/1483869881 conn(0x7fd7a40b1f90 msgr2=0x7fd7a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- 172.21.15.76:0/1483869881 shutdown_connections 2024-09-19T04:13:45.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- 172.21.15.76:0/1483869881 wait complete. 2024-09-19T04:13:45.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 Processor -- start 2024-09-19T04:13:45.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- start start 2024-09-19T04:13:45.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd78c0039c0 0x7fd7a41667c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a4099bd0 0x7fd7a4153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7a4151980 0x7fd7a41544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a40af6d0 con 0x7fd7a4151980 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a40ad740 con 0x7fd7a4099bd0 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a4166ff0 con 0x7fd78c0039c0 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7a4151980 0x7fd7a41544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7a4151980 0x7fd7a41544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:32992/0 (socket says 172.21.15.76:32992) 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 -- 172.21.15.76:0/2565020584 learned_addr learned my addr 172.21.15.76:0/2565020584 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd78c0039c0 0x7fd7a41667c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 -- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd78c0039c0 msgr2=0x7fd7a41667c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd78c0039c0 0x7fd7a41667c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a4099bd0 0x7fd7a4153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 -- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a4099bd0 msgr2=0x7fd7a4153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a4099bd0 0x7fd7a4153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 -- 172.21.15.76:0/2565020584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a41597f0 con 0x7fd7a4151980 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7a4151980 0x7fd7a41544d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40b4e40 tx=0x7fd79c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd78bfff640 1 -- 172.21.15.76:0/2565020584 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00b7d0 con 0x7fd7a4151980 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd78bfff640 1 -- 172.21.15.76:0/2565020584 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79c00b990 con 0x7fd7a4151980 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a4159a90 con 0x7fd7a4151980 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd78bfff640 1 -- 172.21.15.76:0/2565020584 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c00bd80 con 0x7fd7a4151980 2024-09-19T04:13:45.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.146+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a41f54f0 con 0x7fd7a4151980 2024-09-19T04:13:45.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd78bfff640 1 -- 172.21.15.76:0/2565020584 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd79c01b020 con 0x7fd7a4151980 2024-09-19T04:13:45.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd78bfff640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7900678c0 0x7fd790069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7900678c0 0x7fd790069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd78bfff640 1 -- 172.21.15.76:0/2565020584 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd79c086000 con 0x7fd7a4151980 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7900678c0 0x7fd790069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7940af000 tx=0x7fd7940046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 Processor -- start 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 -- start start 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cb7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126cb7690 0x55d126c6d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 0x55d126c6da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d126b50b60 con 0x55d126cb7690 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d126ab7c70 con 0x55d126cb6d70 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d126a5ded0 con 0x55d126c6d6a0 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126cb7690 0x55d126c6d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126cb7690 0x55d126c6d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33000/0 (socket says 172.21.15.76:33000) 2024-09-19T04:13:45.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2350273640 learned_addr learned my addr 172.21.15.76:0/2350273640 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cb7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 0x55d126c6da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 msgr2=0x55d126c6da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 0x55d126c6da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 msgr2=0x55d126cb7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cb7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2350273640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d126a6fe50 con 0x55d126cb7690 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cb7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126cb7690 0x55d126c6d160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd798009a80 tx=0x7fd798009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13146318729a83ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7897fa640 1 -- 172.21.15.76:0/2350273640 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798019070 con 0x55d126cb7690 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7897fa640 1 -- 172.21.15.76:0/2350273640 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798011b60 con 0x55d126cb7690 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.150+0000 7fd7897fa640 1 -- 172.21.15.76:0/2350273640 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798015460 con 0x55d126cb7690 2024-09-19T04:13:45.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2350273640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126cb7690 msgr2=0x55d126c6d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126cb7690 0x55d126c6d160 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd798009a80 tx=0x7fd798009190 comp rx=0 tx=0).stop 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2350273640 shutdown_connections 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 0x55d126c6da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126cb7690 0x55d126c6d160 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2350273640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cb7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2350273640 >> 172.21.15.76:0/2350273640 conn(0x55d126c6f600 msgr2=0x55d126c67630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2350273640 shutdown_connections 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2350273640 wait complete. 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- start start 2024-09-19T04:13:45.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 Processor -- start 2024-09-19T04:13:45.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cbceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb7690 0x55d126cbd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ca3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d126b19b00 con 0x55d126ca3490 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d126a92360 con 0x55d126cb6d70 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d126a8f2d0 con 0x55d126cb7690 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 10 client.0 _ll_register_callbacks cb 0x55d126c54820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ca3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ca3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33016/0 (socket says 172.21.15.76:33016) 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 -- 172.21.15.76:0/3146195837 learned_addr learned my addr 172.21.15.76:0/3146195837 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cbceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb7690 0x55d126cbd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 -- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb7690 msgr2=0x55d126cbd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb7690 0x55d126cbd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 -- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 msgr2=0x55d126cbceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cbceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 -- 172.21.15.76:0/3146195837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d126a5e120 con 0x55d126ca3490 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a37fe640 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ca3870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd79c05e7a0 tx=0x7fd79c0560f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95bbd956e886bd48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7827fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7827fc640 1 -- 172.21.15.76:0/3146195837 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c05ecd0 con 0x55d126ca3490 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd798011b60 con 0x55d126ca3490 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7827fc640 1 -- 172.21.15.76:0/3146195837 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd79c05e840 con 0x55d126ca3490 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7827fc640 1 -- 172.21.15.76:0/3146195837 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fd79c05ea40 con 0x55d126ca3490 2024-09-19T04:13:45.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 10 client.4623 fetch_fsmap learned FSMap version 8 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55d126a57450 con 0x55d126ca3490 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7827fc640 1 -- 172.21.15.76:0/3146195837 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798011b60 con 0x55d126ca3490 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7827fc640 1 -- 172.21.15.76:0/3146195837 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7fd79c07db40 con 0x55d126ca3490 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 10 client.4623 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 10 client.4623 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 20 client.4623 populate_metadata read hostname 'smithi076' 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126ca4280 0x55d126ca4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 4 client.4623 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 --> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d126a51c00 con 0x55d126ca4280 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126ca4280 0x55d126ca4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126ca4280 0x55d126ca4660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26d5f647b6e0724d server_cookie=fe3a1cfa39a466d2 in_seq=0 out_seq=0 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.154+0000 7fd7827fc640 10 client.4623 ms_handle_connect on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7827fc640 1 -- 172.21.15.76:0/3146195837 <== mds.0 v2:172.21.15.76:6834/3901169819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d126a51c00 con 0x55d126ca4280 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7827fc640 10 client.4623 handle_command_reply: tid=0 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 client.4623 shutdown 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 msgr2=0x55d126ca3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ca3870 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd79c05e7a0 tx=0x7fd79c0560f0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 shutdown_connections 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126ca4280 0x55d126ca4660 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ca3870 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb7690 0x55d126cbd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7827fc640 0 client.4623 ms_handle_reset on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:45.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/3146195837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cbceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 >> 172.21.15.76:0/3146195837 conn(0x55d126c6d6a0 msgr2=0x55d126c6e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 shutdown_connections 2024-09-19T04:13:45.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/3146195837 wait complete. 2024-09-19T04:13:45.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.158+0000 7fd7ac9b7000 20 client.4623 trim_cache size 0 max 16384 2024-09-19T04:13:45.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 Processor -- start 2024-09-19T04:13:45.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 -- start start 2024-09-19T04:13:45.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 0x55d126ccd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ccd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cbbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a42007c0 con 0x55d126ca3490 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a4162560 con 0x55d126cb6d70 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a4073e60 con 0x55d126c6d6a0 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ccd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ccd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33018/0 (socket says 172.21.15.76:33018) 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.174+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2380453587 learned_addr learned my addr 172.21.15.76:0/2380453587 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2380453587 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 msgr2=0x55d126ccd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/2380453587 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 0x55d126ccd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2380453587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 msgr2=0x55d126cbbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/2380453587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cbbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 -- 172.21.15.76:0/2380453587 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d126a3efd0 con 0x55d126ca3490 2024-09-19T04:13:45.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/2380453587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ccd7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7980099d0 tx=0x7fd79800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe070a518854c869 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd788ff9640 1 -- 172.21.15.76:0/2380453587 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79802e070 con 0x55d126ca3490 2024-09-19T04:13:45.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd788ff9640 1 -- 172.21.15.76:0/2380453587 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798011d70 con 0x55d126ca3490 2024-09-19T04:13:45.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd788ff9640 1 -- 172.21.15.76:0/2380453587 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd798019040 con 0x55d126ca3490 2024-09-19T04:13:45.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2380453587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 msgr2=0x55d126ccd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2380453587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ccd7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd7980099d0 tx=0x7fd79800f000 comp rx=0 tx=0).stop 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2380453587 shutdown_connections 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2380453587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126cb6d70 0x55d126cbbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2380453587 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126ccd7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/2380453587 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126c6d6a0 0x55d126ccd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2380453587 >> 172.21.15.76:0/2380453587 conn(0x55d126ca2bd0 msgr2=0x55d126c67ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2380453587 shutdown_connections 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/2380453587 wait complete. 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- start start 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 Processor -- start 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126c6d6a0 0x55d126d2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126d2b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb6d70 0x55d126d2b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d126ab6f80 con 0x55d126ca3490 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d126a6fca0 con 0x55d126c6d6a0 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d126abc2a0 con 0x55d126cb6d70 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 10 client.0 _ll_register_callbacks cb 0x7fd79c001980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126d2b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126d2b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33030/0 (socket says 172.21.15.76:33030) 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 -- 172.21.15.76:0/662802780 learned_addr learned my addr 172.21.15.76:0/662802780 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126c6d6a0 0x55d126d2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 -- 172.21.15.76:0/662802780 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb6d70 msgr2=0x55d126d2b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb6d70 0x55d126d2b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 -- 172.21.15.76:0/662802780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126c6d6a0 msgr2=0x55d126d2ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126c6d6a0 0x55d126d2ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 -- 172.21.15.76:0/662802780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d126b3a750 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a27fc640 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126d2b3a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd798011630 tx=0x7fd7980046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c121db1626ec841 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7827fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7827fc640 1 -- 172.21.15.76:0/662802780 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79802e040 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd798011d70 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7827fc640 1 -- 172.21.15.76:0/662802780 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd798011b50 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7827fc640 1 -- 172.21.15.76:0/662802780 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fd79803f050 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 10 client.4629 fetch_fsmap learned FSMap version 8 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55d126a45a30 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7827fc640 1 -- 172.21.15.76:0/662802780 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd798011d70 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7827fc640 1 -- 172.21.15.76:0/662802780 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 8) ==== 3510+0+0 (secure 0 0 0) 0x7fd798042020 con 0x55d126ca3490 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 10 client.4629 fetch_fsmap finished waiting for FSMap version 8 2024-09-19T04:13:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 10 client.4629 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 20 client.4629 populate_metadata read hostname 'smithi076' 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126cb7690 0x55d126cc1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 4 client.4629 mds_command: new command op to 4260 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 --> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] -- command(tid 0: {"prefix": "status"}) -- 0x55d126ac53a0 con 0x55d126cb7690 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.178+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126cb7690 0x55d126cc1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7827fc640 10 client.4629 ms_handle_connect on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a2ffd640 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126cb7690 0x55d126cc1ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=264624bbf5137ce7 server_cookie=85d6f91fb1a1766f in_seq=0 out_seq=0 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7827fc640 1 -- 172.21.15.76:0/662802780 <== mds.0 v2:172.21.15.76:6834/3901169819 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55d126ac53a0 con 0x55d126cb7690 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7827fc640 10 client.4629 handle_command_reply: tid=0 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 client.4629 shutdown 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 msgr2=0x55d126d2b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126d2b3a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd798011630 tx=0x7fd7980046d0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 shutdown_connections 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:6834/3901169819,v1:172.21.15.76:6837/3901169819] conn(0x55d126cb7690 0x55d126cc1ed0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d126cb6d70 0x55d126d2b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d126ca3490 0x55d126d2b3a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7827fc640 0 client.4629 ms_handle_reset on v2:172.21.15.76:6834/3901169819 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 --2- 172.21.15.76:0/662802780 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d126c6d6a0 0x55d126d2ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 >> 172.21.15.76:0/662802780 conn(0x55d126ca2bd0 msgr2=0x55d126ccc250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 shutdown_connections 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 1 -- 172.21.15.76:0/662802780 wait complete. 2024-09-19T04:13:45.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7ac9b7000 20 client.4629 trim_cache size 0 max 16384 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4260, 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 20.243850167000002, 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 8, 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 25, 2024-09-19T04:13:45.193 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 25, 2024-09-19T04:13:45.194 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 26.115200894000001 2024-09-19T04:13:45.194 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7900678c0 msgr2=0x7fd790069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7900678c0 0x7fd790069d80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd7940af000 tx=0x7fd7940046d0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7a4151980 msgr2=0x7fd7a41544d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7a4151980 0x7fd7a41544d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7a40b4e40 tx=0x7fd79c00df70 comp rx=0 tx=0).stop 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 shutdown_connections 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7900678c0 0x7fd790069d80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7a4151980 0x7fd7a41544d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7a4099bd0 0x7fd7a4153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 --2- 172.21.15.76:0/2565020584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd78c0039c0 0x7fd7a41667c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 >> 172.21.15.76:0/2565020584 conn(0x7fd7a40b1f90 msgr2=0x7fd7a40d0590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 shutdown_connections 2024-09-19T04:13:45.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.182+0000 7fd7a98fb640 1 -- 172.21.15.76:0/2565020584 wait complete. 2024-09-19T04:13:45.200 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T04:13:45.219 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T04:13:45.219 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T04:13:45.220 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T04:13:45.220 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-19T04:13:45.236 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-19T04:13:45.236 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-19T04:13:45.236 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-19T04:13:45.236 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:13:45.236 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:45.236 INFO:tasks.ceph_fuse:client.2 config is {} 2024-09-19T04:13:45.236 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:13:45.236 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:45.237 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-19T04:13:45.237 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:13:45.237 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:45.237 INFO:tasks.ceph_fuse:client.3 config is {} 2024-09-19T04:13:45.237 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T04:13:45.237 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:45.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:45.237 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-09-19T04:13:45.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:45.245 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-09-19T04:13:45.303 INFO:teuthology.orchestra.run.smithi116.stderr:Cannot find device "ceph-brx" 2024-09-19T04:13:45.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:13:45.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:45.305 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:13:45.311 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-19T04:13:45.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:45.311 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-19T04:13:45.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:45.365 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-09-19T04:13:45.558 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-19T04:13:45.558 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T04:13:45.558 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:13:45.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.678+0000 7f0a5b0f3640 1 Processor -- start 2024-09-19T04:13:45.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a5b0f3640 1 -- start start 2024-09-19T04:13:45.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a5b0f3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a54156720 0x7f0a540b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a5b0f3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a540b5b50 0x7f0a54140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a5b0f3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a5413fb30 con 0x7f0a54141110 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a5b0f3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0a54141560 con 0x7f0a54156720 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a5b0f3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0a54140740 con 0x7f0a540b5b50 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a58e8f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a54156720 0x7f0a540b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a59690640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0a54141110 0x7f0a540b4410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37544/0 (socket says 172.21.15.76:37544) 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a58e8f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0a54156720 0x7f0a540b5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33046/0 (socket says 172.21.15.76:33046) 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a59690640 1 -- 172.21.15.76:0/4054279840 learned_addr learned my addr 172.21.15.76:0/4054279840 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a53fff640 1 --2- 172.21.15.76:0/4054279840 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a540b5b50 0x7f0a54140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a58e8f640 1 -- 172.21.15.76:0/4054279840 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a540b5b50 msgr2=0x7f0a54140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a58e8f640 1 --2- 172.21.15.76:0/4054279840 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a540b5b50 0x7f0a54140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a58e8f640 1 -- 172.21.15.76:0/4054279840 >> v1:172.21.15.76:6789/0 conn(0x7f0a54141110 legacy=0x7f0a540b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a537fe640 1 -- 172.21.15.76:0/4054279840 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1945063471 0 0) 0x7f0a5413fb30 con 0x7f0a54141110 2024-09-19T04:13:45.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a58e8f640 1 -- 172.21.15.76:0/4054279840 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5413fe10 con 0x7f0a54156720 2024-09-19T04:13:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a58e8f640 1 --2- 172.21.15.76:0/4054279840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a54156720 0x7f0a540b5610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a440097c0 tx=0x7f0a4401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a262e10389c2f28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a537fe640 1 -- 172.21.15.76:0/4054279840 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440b2070 con 0x7f0a54156720 2024-09-19T04:13:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a537fe640 1 -- 172.21.15.76:0/4054279840 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a440ad640 con 0x7f0a54156720 2024-09-19T04:13:45.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.682+0000 7f0a537fe640 1 -- 172.21.15.76:0/4054279840 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440ada10 con 0x7f0a54156720 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/4054279840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a54156720 msgr2=0x7f0a540b5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/4054279840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a54156720 0x7f0a540b5610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0a440097c0 tx=0x7f0a4401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/4054279840 shutdown_connections 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/4054279840 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a540b5b50 0x7f0a54140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/4054279840 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a54156720 0x7f0a540b5610 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/4054279840 >> 172.21.15.76:0/4054279840 conn(0x7f0a5406c0c0 msgr2=0x7f0a5406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/4054279840 shutdown_connections 2024-09-19T04:13:45.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/4054279840 wait complete. 2024-09-19T04:13:45.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 Processor -- start 2024-09-19T04:13:45.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- start start 2024-09-19T04:13:45.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a540b5b50 0x7f0a5413aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a54141110 0x7f0a5413ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a54156720 0x7f0a5415dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5413f860 con 0x7f0a540b5b50 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5413f6e0 con 0x7f0a54141110 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540b4b20 con 0x7f0a54156720 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a540b5b50 0x7f0a5413aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a540b5b50 0x7f0a5413aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33058/0 (socket says 172.21.15.76:33058) 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 -- 172.21.15.76:0/3184268377 learned_addr learned my addr 172.21.15.76:0/3184268377 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 -- 172.21.15.76:0/3184268377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a54156720 msgr2=0x7f0a5415dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a54156720 0x7f0a5415dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 -- 172.21.15.76:0/3184268377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a54141110 msgr2=0x7f0a5413ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a54141110 0x7f0a5413ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 -- 172.21.15.76:0/3184268377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a54156ec0 con 0x7f0a540b5b50 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a58e8f640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a540b5b50 0x7f0a5413aca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a440b0040 tx=0x7f0a44019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a517fa640 1 -- 172.21.15.76:0/3184268377 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440b2040 con 0x7f0a540b5b50 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a517fa640 1 -- 172.21.15.76:0/3184268377 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a440bd070 con 0x7f0a540b5b50 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a541571a0 con 0x7f0a540b5b50 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a517fa640 1 -- 172.21.15.76:0/3184268377 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440b6890 con 0x7f0a540b5b50 2024-09-19T04:13:45.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.686+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a54157760 con 0x7f0a540b5b50 2024-09-19T04:13:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.690+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a54140bd0 con 0x7f0a540b5b50 2024-09-19T04:13:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.690+0000 7f0a517fa640 1 -- 172.21.15.76:0/3184268377 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0a440b6a30 con 0x7f0a540b5b50 2024-09-19T04:13:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.690+0000 7f0a517fa640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0a400678c0 0x7f0a40069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.690+0000 7f0a517fa640 1 -- 172.21.15.76:0/3184268377 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0a44120130 con 0x7f0a540b5b50 2024-09-19T04:13:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.690+0000 7f0a53fff640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0a400678c0 0x7f0a40069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:45.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.690+0000 7f0a53fff640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0a400678c0 0x7f0a40069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a541f7dd0 tx=0x7f0a48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:45.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.694+0000 7f0a517fa640 1 -- 172.21.15.76:0/3184268377 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0a44016090 con 0x7f0a540b5b50 2024-09-19T04:13:45.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.894+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0a54069170 con 0x7f0a540b5b50 2024-09-19T04:13:45.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.894+0000 7f0a517fa640 1 -- 172.21.15.76:0/3184268377 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f0a440f0ae0 con 0x7f0a540b5b50 2024-09-19T04:13:45.899 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T04:13:45.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0a400678c0 msgr2=0x7f0a40069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0a400678c0 0x7f0a40069d80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a541f7dd0 tx=0x7f0a48008040 comp rx=0 tx=0).stop 2024-09-19T04:13:45.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a540b5b50 msgr2=0x7f0a5413aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a540b5b50 0x7f0a5413aca0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0a440b0040 tx=0x7f0a44019240 comp rx=0 tx=0).stop 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 shutdown_connections 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0a400678c0 0x7f0a40069d80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0a54156720 0x7f0a5415dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a54141110 0x7f0a5413ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 --2- 172.21.15.76:0/3184268377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0a540b5b50 0x7f0a5413aca0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 >> 172.21.15.76:0/3184268377 conn(0x7f0a5406c0c0 msgr2=0x7f0a540d0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 shutdown_connections 2024-09-19T04:13:45.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:45.898+0000 7f0a5b0f3640 1 -- 172.21.15.76:0/3184268377 wait complete. 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:13:45.919 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:13:45.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:45.920 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:45.926 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42495sec preferred_lft 42495sec 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:45.927 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:13:45.928 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:45.928 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:13:45.928 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:45.928 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:13:45.928 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:45.928 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:45.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:13:45.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr flush dev ceph-brx 2024-09-19T04:13:45.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set ceph-brx up 2024-09-19T04:13:45.929 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:13:45.929 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:46.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:46.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:46.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:46.053 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:13:46.105 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:13:46.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:46.107 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:13:46.160 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:13:46.160 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:13:46.160 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:13:46.160 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:13:46.160 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:13:46.160 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:13:46.161 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:13:46.161 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:46.161 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:46.161 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:46.161 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:13:46.161 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:46.161 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:46.161 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:13:46.161 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:13:46.161 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:13:46.162 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:46.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:46.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:46.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:46.281 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:13:46.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:46.329 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-09-19T04:13:46.376 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:46.377 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:46.377 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:46.377 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T04:13:46.377 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:46.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:46.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:46.494 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T04:13:46.494 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:46.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:46.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:46.635 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:46.635 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:46.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:13:46.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:13:46.635 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:46.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:46.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:46.695 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T04:13:46.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:46.695 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:46.740 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:13:46.740 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:46.740 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:46.787 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:46.846 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:46.895 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:46.896 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:46.897 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:46.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:46.898 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:46.945 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:13:46.945 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T04:13:46.990 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:47.004 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:47.053 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:47.053 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:47.053 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:47.053 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:47.054 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:47.055 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:47.056 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:47.056 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:47.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:47.057 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:47.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-09-19T04:13:47.070+0000 7ff4ded56580 -1 init, newargv = 0x55650cff8120 newargc=15 2024-09-19T04:13:47.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[36450]: starting ceph client 2024-09-19T04:13:47.079 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[36450]: starting fuse 2024-09-19T04:13:48.178 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:48.195 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:48.242 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:48.242 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:48.242 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:48.242 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:48.242 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:48.242 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:48.242 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:48.243 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:48.244 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:48.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:48.246 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:48.293 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:13:48.294 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:13:48.294 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:13:48.295 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:13:48.296 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:13:48.296 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:48.296 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-09-19T04:13:48.296 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:13:48.422 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.36450.asok 2024-09-19T04:13:48.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:48.433 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36450.asok 2024-09-19T04:13:48.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:48.434 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36450.asok status 2024-09-19T04:13:48.586 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "36450", 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:48.587 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4644, 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4644 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3238217339 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:48.588 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 3238217339 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4644 192.168.0.1:0/3238217339", 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/3238217339", 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 8, 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 25, 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:13:48.589 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:13:48.601 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T04:13:48.601 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:13:48.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29d4a51640 1 Processor -- start 2024-09-19T04:13:48.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29d4a51640 1 -- start start 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29d4a51640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29d0161a60 0x7f29d00d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29d4a51640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d00d32b0 con 0x7f29d00d2e90 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29d4a51640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29d00d0300 con 0x7f29d00cf620 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29d4a51640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f29d00cfa90 con 0x7f29d0161a60 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cdd9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f29d00d2e90 0x7f29d00cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37552/0 (socket says 172.21.15.76:37552) 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cdd9b640 1 -- 172.21.15.76:0/609527739 learned_addr learned my addr 172.21.15.76:0/609527739 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 452890262 0 0) 0x7f29d00d32b0 con 0x7f29d00d2e90 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29b8003530 con 0x7f29d00d2e90 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ce59c640 1 --2- 172.21.15.76:0/609527739 >> v2:172.21.15.116:3300/0 conn(0x7f29d0161a60 0x7f29d00d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2677344117 0 0) 0x7f29b8003530 con 0x7f29d00d2e90 2024-09-19T04:13:48.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 >> v2:172.21.15.116:3300/0 conn(0x7f29d0161a60 msgr2=0x7f29d00d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 --2- 172.21.15.76:0/609527739 >> v2:172.21.15.116:3300/0 conn(0x7f29d0161a60 0x7f29d00d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 >> v1:172.21.15.116:6789/0 conn(0x7f29d00cf620 legacy=0x7f29d006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d00b0f60 con 0x7f29d00d2e90 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f29c4001680 con 0x7f29d00d2e90 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 >> v1:172.21.15.76:6789/0 conn(0x7f29d00d2e90 legacy=0x7f29d00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29b8004810 0x7f29b8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29b80052f0 0x7f29b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29d00cfa90 con 0x7f29b80039c0 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f29b800c1c0 con 0x7f29b80052f0 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f29b800c3c0 con 0x7f29b8004810 2024-09-19T04:13:48.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ce59c640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ced9d640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29b80052f0 0x7f29b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cdd9b640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29b8004810 0x7f29b8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ce59c640 1 -- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29b8004810 msgr2=0x7f29b8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ce59c640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29b8004810 0x7f29b8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ce59c640 1 -- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29b80052f0 msgr2=0x7f29b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ce59c640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29b80052f0 0x7f29b800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ce59c640 1 -- 172.21.15.76:0/609527739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b8003530 con 0x7f29b80039c0 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ced9d640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29b80052f0 0x7f29b800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29ced9d640 1 -- 172.21.15.76:0/609527739 reap_dead start 2024-09-19T04:13:48.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.714+0000 7f29cdd9b640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29b8004810 0x7f29b8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29ce59c640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29b8003de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29c000ba00 tx=0x7f29c001c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5b5677cda7cf19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c00b2070 con 0x7f29b80039c0 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c00ad630 con 0x7f29b80039c0 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29cd59a640 1 -- 172.21.15.76:0/609527739 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c000b600 con 0x7f29b80039c0 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 -- 172.21.15.76:0/609527739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 msgr2=0x7f29b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29b8003de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f29c000ba00 tx=0x7f29c001c630 comp rx=0 tx=0).stop 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 -- 172.21.15.76:0/609527739 shutdown_connections 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 --2- 172.21.15.76:0/609527739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29b8003de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 -- 172.21.15.76:0/609527739 >> 172.21.15.76:0/609527739 conn(0x7f29d008bf90 msgr2=0x7f29d009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:48.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 -- 172.21.15.76:0/609527739 shutdown_connections 2024-09-19T04:13:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 -- 172.21.15.76:0/609527739 wait complete. 2024-09-19T04:13:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 Processor -- start 2024-09-19T04:13:48.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.718+0000 7f29d4a51640 1 -- start start 2024-09-19T04:13:48.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29d00cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29d01f7cd0 0x7f29d015f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29d01573a0 0x7f29d0157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29d00cf320 con 0x7f29b80039c0 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f29d006d190 con 0x7f29d01f7cd0 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f29d00cf4a0 con 0x7f29d01573a0 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29d00cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29d00cbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33082/0 (socket says 172.21.15.76:33082) 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 -- 172.21.15.76:0/3366355088 learned_addr learned my addr 172.21.15.76:0/3366355088 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ced9d640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29d01573a0 0x7f29d0157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 -- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29d01573a0 msgr2=0x7f29d0157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29cdd9b640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29d01f7cd0 0x7f29d015f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29d01573a0 0x7f29d0157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 -- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29d01f7cd0 msgr2=0x7f29d015f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29d01f7cd0 0x7f29d015f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 -- 172.21.15.76:0/3366355088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29d01f8ca0 con 0x7f29b80039c0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ced9d640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29d01573a0 0x7f29d0157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29ce59c640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29d00cbec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29c001c460 tx=0x7f29b8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29beffd640 1 -- 172.21.15.76:0/3366355088 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c00b2040 con 0x7f29b80039c0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29beffd640 1 -- 172.21.15.76:0/3366355088 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29c00ad430 con 0x7f29b80039c0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29d01f8150 con 0x7f29b80039c0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29beffd640 1 -- 172.21.15.76:0/3366355088 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29c0004240 con 0x7f29b80039c0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29d01f86e0 con 0x7f29b80039c0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29d00d12b0 con 0x7f29b80039c0 2024-09-19T04:13:48.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29beffd640 1 -- 172.21.15.76:0/3366355088 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f29c00bf020 con 0x7f29b80039c0 2024-09-19T04:13:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29beffd640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29b00678c0 0x7f29b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29beffd640 1 -- 172.21.15.76:0/3366355088 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f29c0120220 con 0x7f29b80039c0 2024-09-19T04:13:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29cdd9b640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29b00678c0 0x7f29b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:48.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.722+0000 7f29cdd9b640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29b00678c0 0x7f29b0069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29c4001ec0 tx=0x7f29c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:48.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.726+0000 7f29beffd640 1 -- 172.21.15.76:0/3366355088 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f29c0016370 con 0x7f29b80039c0 2024-09-19T04:13:48.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.918+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f29d0157d90 con 0x7f29b80039c0 2024-09-19T04:13:48.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.918+0000 7f29beffd640 1 -- 172.21.15.76:0/3366355088 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f29c00ad600 con 0x7f29b80039c0 2024-09-19T04:13:48.924 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T04:13:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29b00678c0 msgr2=0x7f29b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29b00678c0 0x7f29b0069d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f29c4001ec0 tx=0x7f29c4009040 comp rx=0 tx=0).stop 2024-09-19T04:13:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 msgr2=0x7f29d00cbec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29d00cbec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f29c001c460 tx=0x7f29b8005840 comp rx=0 tx=0).stop 2024-09-19T04:13:48.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 shutdown_connections 2024-09-19T04:13:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29b00678c0 0x7f29b0069d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29d01573a0 0x7f29d0157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29d01f7cd0 0x7f29d015f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 --2- 172.21.15.76:0/3366355088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29b80039c0 0x7f29d00cbec0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 >> 172.21.15.76:0/3366355088 conn(0x7f29d008bf90 msgr2=0x7f29d006d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 shutdown_connections 2024-09-19T04:13:48.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:48.922+0000 7f29d4a51640 1 -- 172.21.15.76:0/3366355088 wait complete. 2024-09-19T04:13:48.941 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:13:48.941 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:13:48.942 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:13:48.942 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-19T04:13:48.942 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:48.942 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:13:48.942 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:13:48.942 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:13:48.942 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-19T04:13:48.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:48.942 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:13:48.948 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:48.948 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:48.948 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:48.948 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:13:48.949 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42492sec preferred_lft 42492sec 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:13:48.950 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:48.951 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:13:48.951 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:48.951 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:13:48.951 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:48.951 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:13:48.951 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:48.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:48.951 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:13:49.002 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:13:49.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:49.003 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:13:49.052 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:13:49.052 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:13:49.052 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:13:49.052 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:13:49.052 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:13:49.053 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:13:49.053 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:13:49.053 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:49.053 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:49.053 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:49.053 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:13:49.053 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:49.053 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:49.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:13:49.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:13:49.054 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:13:49.054 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:49.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:49.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:49.172 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:13:49.221 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:13:49.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:49.222 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list-id 2024-09-19T04:13:49.269 INFO:teuthology.orchestra.run.smithi076.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-19T04:13:49.269 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:49.269 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:49.269 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:13:49.270 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-19T04:13:49.270 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:49.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:49.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:49.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:49.386 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:13:49.398 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 62:72:49:16:e9:3f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T04:13:49.399 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T04:13:49.399 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.399 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::6072:49ff:fe16:e93f/64 scope link 2024-09-19T04:13:49.399 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:49.400 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-19T04:13:49.460 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout:2: veth0@if10: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 62:72:49:16:e9:3f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::6072:49ff:fe16:e93f/64 scope link 2024-09-19T04:13:49.461 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:49.462 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-19T04:13:49.463 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:49.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:49.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:49.607 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:49.607 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:13:49.607 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.1 up 2024-09-19T04:13:49.607 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set dev brx.1 master ceph-brx 2024-09-19T04:13:49.607 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:13:49.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:49.709 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-09-19T04:13:49.709 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:49.709 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:49.757 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-19T04:13:49.757 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:49.757 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:49.804 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:49.862 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:49.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:49.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:13:49.912 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:13:49.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:49.913 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:49.961 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:13:49.961 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-19T04:13:49.961 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-09-19T04:13:50.006 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:50.019 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:50.066 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:50.067 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:13:50.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:50.069 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:50.087 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:2024-09-19T04:13:50.082+0000 7f977338d580 -1 init, newargv = 0x559ea3907680 newargc=15 2024-09-19T04:13:50.088 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:ceph-fuse[36595]: starting ceph client 2024-09-19T04:13:50.092 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:13:50.093 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi076.stderr:ceph-fuse[36595]: starting fuse 2024-09-19T04:13:51.194 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:13:51.210 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:13:51.259 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:13:51.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:51.261 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:13:51.262 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:13:51.262 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:51.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:51.262 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:13:51.305 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:13:51.305 INFO:teuthology.orchestra.run.smithi076.stdout:46 2024-09-19T04:13:51.306 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-19T04:13:51.306 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:13:51.306 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:13:51.306 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:13:51.306 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:13:51.306 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:13:51.307 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:13:51.308 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:13:51.309 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:13:51.309 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.2")) 2024-09-19T04:13:51.309 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:13:51.430 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.2.36595.asok 2024-09-19T04:13:51.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:51.442 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.36595.asok 2024-09-19T04:13:51.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:51.443 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.36595.asok status 2024-09-19T04:13:51.594 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "2", 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "36595", 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:13:51.595 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4656, 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 4656 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.2:0", 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1848325475 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:13:51.596 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.2:0", 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1848325475 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.4656 192.168.0.2:0/1848325475", 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.2:0/1848325475", 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 8, 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 25, 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:13:51.597 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:13:51.609 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T04:13:51.609 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:13:51.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.726+0000 7f4bdd656640 1 Processor -- start 2024-09-19T04:13:51.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.726+0000 7f4bdd656640 1 -- start start 2024-09-19T04:13:51.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bdd656640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd8161a60 0x7f4bd80d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bdd656640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4bd80d2e90 0x7f4bd80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bdd656640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4bd80cf450 0x7f4bd80d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bdd656640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd80d32b0 con 0x7f4bd80cf450 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bdd656640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd80d0300 con 0x7f4bd80d2e90 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bdd656640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd80b0f60 con 0x7f4bd8161a60 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4bd80cf450 0x7f4bd80d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4bd80cf450 0x7f4bd80d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33094/0 (socket says 172.21.15.76:33094) 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 -- 172.21.15.76:0/3687346329 learned_addr learned my addr 172.21.15.76:0/3687346329 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 -- 172.21.15.76:0/3687346329 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd8161a60 msgr2=0x7f4bd80d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 --2- 172.21.15.76:0/3687346329 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd8161a60 0x7f4bd80d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 -- 172.21.15.76:0/3687346329 >> v2:172.21.15.116:3300/0 conn(0x7f4bd80d2e90 msgr2=0x7f4bd80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:51.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd67fc640 1 --2- 172.21.15.76:0/3687346329 >> v2:172.21.15.116:3300/0 conn(0x7f4bd80d2e90 0x7f4bd80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 --2- 172.21.15.76:0/3687346329 >> v2:172.21.15.116:3300/0 conn(0x7f4bd80d2e90 0x7f4bd80cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 -- 172.21.15.76:0/3687346329 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd80cfd40 con 0x7f4bd80cf450 2024-09-19T04:13:51.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd77fe640 1 --2- 172.21.15.76:0/3687346329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80cf450 0x7f4bd80d0a10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80d1ef0 tx=0x7f4bd001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146863785113c0eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:51.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd5ffb640 1 -- 172.21.15.76:0/3687346329 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00a57b0 con 0x7f4bd80cf450 2024-09-19T04:13:51.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd5ffb640 1 -- 172.21.15.76:0/3687346329 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd00a5950 con 0x7f4bd80cf450 2024-09-19T04:13:51.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.730+0000 7f4bd5ffb640 1 -- 172.21.15.76:0/3687346329 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00a5d20 con 0x7f4bd80cf450 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- 172.21.15.76:0/3687346329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80cf450 msgr2=0x7f4bd80d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 --2- 172.21.15.76:0/3687346329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80cf450 0x7f4bd80d0a10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4bd80d1ef0 tx=0x7f4bd001c930 comp rx=0 tx=0).stop 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- 172.21.15.76:0/3687346329 shutdown_connections 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 --2- 172.21.15.76:0/3687346329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80cf450 0x7f4bd80d0a10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 --2- 172.21.15.76:0/3687346329 >> v2:172.21.15.116:3300/0 conn(0x7f4bd80d2e90 0x7f4bd80cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 --2- 172.21.15.76:0/3687346329 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd8161a60 0x7f4bd80d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- 172.21.15.76:0/3687346329 >> 172.21.15.76:0/3687346329 conn(0x7f4bd808bf90 msgr2=0x7f4bd809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:51.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- 172.21.15.76:0/3687346329 shutdown_connections 2024-09-19T04:13:51.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- 172.21.15.76:0/3687346329 wait complete. 2024-09-19T04:13:51.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 Processor -- start 2024-09-19T04:13:51.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- start start 2024-09-19T04:13:51.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd80cf450 0x7f4bd80b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80d2e90 0x7f4bd80b9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bd8161a60 0x7f4bd8157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd80cfa80 con 0x7f4bd80d2e90 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd80cf900 con 0x7f4bd8161a60 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd806d6b0 con 0x7f4bd80cf450 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80d2e90 0x7f4bd80b9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80d2e90 0x7f4bd80b9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33104/0 (socket says 172.21.15.76:33104) 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 -- 172.21.15.76:0/886190852 learned_addr learned my addr 172.21.15.76:0/886190852 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd6ffd640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd80cf450 0x7f4bd80b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd77fe640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bd8161a60 0x7f4bd8157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 -- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd80cf450 msgr2=0x7f4bd80b35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd80cf450 0x7f4bd80b35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 -- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bd8161a60 msgr2=0x7f4bd8157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bd8161a60 0x7f4bd8157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 -- 172.21.15.76:0/886190852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd80b77c0 con 0x7f4bd80d2e90 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd6ffd640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd80cf450 0x7f4bd80b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd77fe640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bd8161a60 0x7f4bd8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bd67fc640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80d2e90 0x7f4bd80b9a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00c820 tx=0x7f4bcc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bbf7fe640 1 -- 172.21.15.76:0/886190852 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc004230 con 0x7f4bd80d2e90 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bbf7fe640 1 -- 172.21.15.76:0/886190852 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bcc0043d0 con 0x7f4bd80d2e90 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd81592e0 con 0x7f4bd80d2e90 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bbf7fe640 1 -- 172.21.15.76:0/886190852 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bcc007a10 con 0x7f4bd80d2e90 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.734+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd80b7cc0 con 0x7f4bd80d2e90 2024-09-19T04:13:51.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.738+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd80cef10 con 0x7f4bd80d2e90 2024-09-19T04:13:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.738+0000 7f4bbf7fe640 1 -- 172.21.15.76:0/886190852 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4bcc007bb0 con 0x7f4bd80d2e90 2024-09-19T04:13:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.738+0000 7f4bbf7fe640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4bc40678c0 0x7f4bc4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.738+0000 7f4bd6ffd640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4bc40678c0 0x7f4bc4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.738+0000 7f4bbf7fe640 1 -- 172.21.15.76:0/886190852 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4bcc085f50 con 0x7f4bd80d2e90 2024-09-19T04:13:51.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.738+0000 7f4bd6ffd640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4bc40678c0 0x7f4bc4069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800a870 tx=0x7f4bc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:51.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.742+0000 7f4bbf7fe640 1 -- 172.21.15.76:0/886190852 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4bcc015020 con 0x7f4bd80d2e90 2024-09-19T04:13:51.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.942+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4bd80b3e70 con 0x7f4bd80d2e90 2024-09-19T04:13:51.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.942+0000 7f4bbf7fe640 1 -- 172.21.15.76:0/886190852 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f4bcc056740 con 0x7f4bd80d2e90 2024-09-19T04:13:51.948 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T04:13:51.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4bc40678c0 msgr2=0x7f4bc4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4bc40678c0 0x7f4bc4069d80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800a870 tx=0x7f4bc8008040 comp rx=0 tx=0).stop 2024-09-19T04:13:51.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80d2e90 msgr2=0x7f4bd80b9a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:51.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80d2e90 0x7f4bd80b9a00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc00c820 tx=0x7f4bcc00cce0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 shutdown_connections 2024-09-19T04:13:51.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4bc40678c0 0x7f4bc4069d80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bd8161a60 0x7f4bd8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bd80d2e90 0x7f4bd80b9a00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 --2- 172.21.15.76:0/886190852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bd80cf450 0x7f4bd80b35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 >> 172.21.15.76:0/886190852 conn(0x7f4bd808bf90 msgr2=0x7f4bd815f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 shutdown_connections 2024-09-19T04:13:51.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:51.946+0000 7f4bdd656640 1 -- 172.21.15.76:0/886190852 wait complete. 2024-09-19T04:13:51.967 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:13:51.967 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:13:51.967 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-09-19T04:13:51.967 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-19T04:13:51.967 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:51.967 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:13:51.967 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:13:51.968 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:13:51.968 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-19T04:13:51.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:51.968 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:51.975 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42471sec preferred_lft 42471sec 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:51.976 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:13:51.977 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:7f:10:a5:b6 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:51.977 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:13:51.977 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:51.977 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T04:13:51.977 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:51.977 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:51.977 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add name ceph-brx type bridge 2024-09-19T04:13:51.977 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr flush dev ceph-brx 2024-09-19T04:13:51.977 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set ceph-brx up 2024-09-19T04:13:51.977 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T04:13:51.977 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:52.064 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:52.100 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:52.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:52.107 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:13:52.116 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-09-19T04:13:52.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:52.118 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-09-19T04:13:52.195 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-09-19T04:13:52.195 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:13:52.195 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:13:52.195 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:13:52.196 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:13:52.196 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:13:52.196 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:13:52.196 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:52.196 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:52.196 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:52.196 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:13:52.196 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:52.196 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:52.196 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:13:52.197 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:13:52.197 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:13:52.197 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:52.284 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:52.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:52.323 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-09-19T04:13:52.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:52.329 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-09-19T04:13:52.374 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:52.375 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:52.375 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:52.375 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-19T04:13:52.375 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:52.463 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:52.484 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:52.491 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-19T04:13:52.491 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:52.492 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:52.492 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-19T04:13:52.492 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T04:13:52.492 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-19T04:13:52.492 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-19T04:13:52.492 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-19T04:13:52.492 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:52.535 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:52.612 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:52.618 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:52.618 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:52.618 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2024-09-19T04:13:52.618 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T04:13:52.618 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:52.699 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:52.718 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:52.724 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-19T04:13:52.724 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:52.724 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:52.770 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-19T04:13:52.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:52.770 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:52.818 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-09-19T04:13:52.879 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-09-19T04:13:52.929 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:52.930 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:52.931 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:52.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:52.932 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-09-19T04:13:52.978 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:13:52.979 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-19T04:13:53.023 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-09-19T04:13:53.037 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:53.052 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:13:53.054 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-09-19T04:13:53.083 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-09-19T04:13:53.084 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:53.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:53.086 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-09-19T04:13:53.103 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi116.stderr:2024-09-19T04:13:53.098+0000 7f4b4fa0b580 -1 init, newargv = 0x5624a0d24130 newargc=15 2024-09-19T04:13:53.103 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi116.stderr:ceph-fuse[34558]: starting ceph client 2024-09-19T04:13:53.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi116.stderr:ceph-fuse[34558]: starting fuse 2024-09-19T04:13:54.208 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-09-19T04:13:54.224 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-09-19T04:13:54.277 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:54.278 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:54.279 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:54.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:54.280 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-09-19T04:13:54.322 INFO:teuthology.orchestra.run.smithi116.stdout:45 2024-09-19T04:13:54.322 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-09-19T04:13:54.323 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.1")) 2024-09-19T04:13:54.324 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-09-19T04:13:54.452 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.1.34558.asok 2024-09-19T04:13:54.455 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:54.462 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.34558.asok 2024-09-19T04:13:54.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:54.463 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.34558.asok status 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "1", 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "34558", 2024-09-19T04:13:54.614 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4528, 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4528 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:13:54.615 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3068770657 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3068770657 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4528 192.168.0.1:0/3068770657", 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.1:0/3068770657", 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 8, 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 25, 2024-09-19T04:13:54.616 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:13:54.617 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-09-19T04:13:54.617 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-09-19T04:13:54.617 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-19T04:13:54.628 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-19T04:13:54.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:13:54.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.754+0000 7fb1225b0640 1 Processor -- start 2024-09-19T04:13:54.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.754+0000 7fb1225b0640 1 -- start start 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb1225b0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 0x7fb11c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb1225b0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb11c06e7a0 con 0x7fb11c06e170 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb1225b0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb11c0d0f00 con 0x7fb11c0b4510 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb1225b0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c0d1550 con 0x7fb11c15e650 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb11c06e170 0x7fb11c0b3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50716/0 (socket says 172.21.15.76:50716) 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11b7fe640 1 -- 172.21.15.76:0/3314969955 learned_addr learned my addr 172.21.15.76:0/3314969955 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11bfff640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 0x7fb11c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 674650018 0 0) 0x7fb11c06e7a0 con 0x7fb11c06e170 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb104003530 con 0x7fb11c06e170 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 549380181 0 0) 0x7fb104003530 con 0x7fb11c06e170 2024-09-19T04:13:54.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 msgr2=0x7fb11c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 0x7fb11c06dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 >> v1:172.21.15.116:6789/0 conn(0x7fb11c0b4510 legacy=0x7fb11c0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c0d11e0 con 0x7fb11c06e170 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb110001680 con 0x7fb11c06e170 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 >> v1:172.21.15.76:6789/0 conn(0x7fb11c06e170 legacy=0x7fb11c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1040039f0 0x7fb104003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb104004810 0x7fb104004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 0x7fb1040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0d1550 con 0x7fb1040039f0 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb10400ba80 con 0x7fb104004810 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb10400bc80 con 0x7fb11c15e650 2024-09-19T04:13:54.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11bfff640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1040039f0 0x7fb104003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11b7fe640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb104004810 0x7fb104004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb120b4d640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 0x7fb1040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11bfff640 1 -- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 msgr2=0x7fb1040051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:54.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11bfff640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c15e650 0x7fb1040051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11bfff640 1 -- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb104004810 msgr2=0x7fb104004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:54.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11bfff640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb104004810 0x7fb104004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.758+0000 7fb11bfff640 1 -- 172.21.15.76:0/3314969955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb104003530 con 0x7fb1040039f0 2024-09-19T04:13:54.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb11bfff640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1040039f0 0x7fb104003e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009ab0 tx=0x7fb10c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=443d0a2df5dbc7e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:54.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0096b0 con 0x7fb1040039f0 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb10c0a6410 con 0x7fb1040039f0 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb11affd640 1 -- 172.21.15.76:0/3314969955 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0a6800 con 0x7fb1040039f0 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- 172.21.15.76:0/3314969955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1040039f0 msgr2=0x7fb104003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 --2- 172.21.15.76:0/3314969955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1040039f0 0x7fb104003e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009ab0 tx=0x7fb10c01ca30 comp rx=0 tx=0).stop 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 -- 172.21.15.76:0/3314969955 reap_dead start 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- 172.21.15.76:0/3314969955 shutdown_connections 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- 172.21.15.76:0/3314969955 >> 172.21.15.76:0/3314969955 conn(0x7fb11c06bcc0 msgr2=0x7fb11c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- 172.21.15.76:0/3314969955 shutdown_connections 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- 172.21.15.76:0/3314969955 wait complete. 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 Processor -- start 2024-09-19T04:13:54.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- start start 2024-09-19T04:13:54.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040039f0 0x7fb11c169940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c150ac0 0x7fb11c152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb11c152770 0x7fb11c161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0d2c90 con 0x7fb11c152770 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb11c0d2b10 con 0x7fb1040039f0 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb1225b0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb11c16a280 con 0x7fb11c150ac0 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb11c152770 0x7fb11c161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb11c152770 0x7fb11c161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36572/0 (socket says 172.21.15.76:36572) 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 -- 172.21.15.76:0/2869371641 learned_addr learned my addr 172.21.15.76:0/2869371641 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb11bfff640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040039f0 0x7fb11c169940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb11b7fe640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c150ac0 0x7fb11c152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 -- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c150ac0 msgr2=0x7fb11c152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c150ac0 0x7fb11c152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 -- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040039f0 msgr2=0x7fb11c169940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040039f0 0x7fb11c169940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 -- 172.21.15.76:0/2869371641 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb11c152ca0 con 0x7fb11c152770 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb120b4d640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb11c152770 0x7fb11c161350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0b4f00 tx=0x7fb11400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb118ff9640 1 -- 172.21.15.76:0/2869371641 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114003980 con 0x7fb11c152770 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.762+0000 7fb118ff9640 1 -- 172.21.15.76:0/2869371641 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb114018070 con 0x7fb11c152770 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb11c0b5ec0 con 0x7fb11c152770 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb118ff9640 1 -- 172.21.15.76:0/2869371641 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb114003d60 con 0x7fb11c152770 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb11c0b6480 con 0x7fb11c152770 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb11c0b6680 con 0x7fb11c152770 2024-09-19T04:13:54.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb118ff9640 1 -- 172.21.15.76:0/2869371641 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb1140050d0 con 0x7fb11c152770 2024-09-19T04:13:54.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb118ff9640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1080678c0 0x7fb108069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:54.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb118ff9640 1 -- 172.21.15.76:0/2869371641 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb114085be0 con 0x7fb11c152770 2024-09-19T04:13:54.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb11bfff640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1080678c0 0x7fb108069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:54.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.766+0000 7fb11bfff640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1080678c0 0x7fb108069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0ad000 tx=0x7fb10c0b2040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:54.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.770+0000 7fb118ff9640 1 -- 172.21.15.76:0/2869371641 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb11c0b6680 con 0x7fb11c152770 2024-09-19T04:13:54.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.970+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb11c0b3e00 con 0x7fb11c152770 2024-09-19T04:13:54.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.970+0000 7fb118ff9640 1 -- 172.21.15.76:0/2869371641 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7fb114056470 con 0x7fb11c152770 2024-09-19T04:13:54.975 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1080678c0 msgr2=0x7fb108069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1080678c0 0x7fb108069d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb10c0ad000 tx=0x7fb10c0b2040 comp rx=0 tx=0).stop 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb11c152770 msgr2=0x7fb11c161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb11c152770 0x7fb11c161350 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb11c0b4f00 tx=0x7fb11400ec10 comp rx=0 tx=0).stop 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 shutdown_connections 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1080678c0 0x7fb108069d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb11c152770 0x7fb11c161350 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb11c150ac0 0x7fb11c152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 --2- 172.21.15.76:0/2869371641 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040039f0 0x7fb11c169940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 >> 172.21.15.76:0/2869371641 conn(0x7fb11c06bcc0 msgr2=0x7fb11c155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:54.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 shutdown_connections 2024-09-19T04:13:54.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:54.974+0000 7fb1225b0640 1 -- 172.21.15.76:0/2869371641 wait complete. 2024-09-19T04:13:54.994 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:13:54.994 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:13:54.994 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2024-09-19T04:13:54.994 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-19T04:13:54.994 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:54.995 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:13:54.995 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:13:54.995 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:13:54.995 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-19T04:13:54.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:54.995 DEBUG:teuthology.orchestra.run.smithi116:> ip addr 2024-09-19T04:13:55.002 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:55.002 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f0 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: altname enp7s0f1 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2024-09-19T04:13:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:7f:10:a5:b6 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:13:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.006 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:13:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:13:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:55.006 DEBUG:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:13:55.057 INFO:teuthology.orchestra.run.smithi116.stdout:1 2024-09-19T04:13:55.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:55.059 DEBUG:teuthology.orchestra.run.smithi116:> route 2024-09-19T04:13:55.115 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:13:55.117 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:13:55.117 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:55.117 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:55.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:13:55.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:13:55.117 DEBUG:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:13:55.117 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:55.204 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:55.234 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:55.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:55.242 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-09-19T04:13:55.248 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T04:13:55.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:55.249 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list-id 2024-09-19T04:13:55.298 INFO:teuthology.orchestra.run.smithi116.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-19T04:13:55.298 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:55.298 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:55.298 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:13:55.298 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-19T04:13:55.298 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:55.389 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:55.408 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:55.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:55.416 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-19T04:13:55.426 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:55.426 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 72:f8:cc:b3:26:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::70f8:ccff:feb3:26db/64 scope link 2024-09-19T04:13:55.427 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:55.428 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-19T04:13:55.480 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:13:55.480 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:13:55.480 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:13:55.480 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.480 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2024-09-19T04:13:55.480 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.480 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:13:55.481 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 72:f8:cc:b3:26:db brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-19T04:13:55.481 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-19T04:13:55.481 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.481 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::70f8:ccff:feb3:26db/64 scope link 2024-09-19T04:13:55.481 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:13:55.482 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-19T04:13:55.482 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:55.482 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:55.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-19T04:13:55.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-19T04:13:55.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-19T04:13:55.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-19T04:13:55.482 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-19T04:13:55.483 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:55.567 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:55.614 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:55.621 DEBUG:teuthology.orchestra.run.smithi116:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:13:55.621 DEBUG:teuthology.orchestra.run.smithi116:> set -e 2024-09-19T04:13:55.621 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2024-09-19T04:13:55.621 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.1 master ceph-brx 2024-09-19T04:13:55.621 DEBUG:teuthology.orchestra.run.smithi116:> ') 2024-09-19T04:13:55.704 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:13:55.722 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:55.729 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-09-19T04:13:55.729 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:55.729 DEBUG:teuthology.orchestra.run.smithi116:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:55.773 INFO:teuthology.orchestra.run.smithi116.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-19T04:13:55.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:13:55.774 DEBUG:teuthology.orchestra.run.smithi116:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:55.827 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-09-19T04:13:55.884 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-09-19T04:13:55.933 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:55.934 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:13:55.935 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:13:55.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:55.936 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-09-19T04:13:55.982 INFO:teuthology.orchestra.run.smithi116.stdout:45 2024-09-19T04:13:55.982 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-09-19T04:13:55.983 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-09-19T04:13:56.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-09-19T04:13:56.041 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:56.091 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-09-19T04:13:56.091 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-09-19T04:13:56.091 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-09-19T04:13:56.091 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-09-19T04:13:56.091 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-09-19T04:13:56.091 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-09-19T04:13:56.092 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-09-19T04:13:56.093 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:56.094 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:56.094 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:56.094 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:13:56.094 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:13:56.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:56.094 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-09-19T04:13:56.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi116.stderr:2024-09-19T04:13:56.102+0000 7f5b88e3e580 -1 init, newargv = 0x55d48ee0c090 newargc=15 2024-09-19T04:13:56.109 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi116.stderr:ceph-fuse[34679]: starting ceph client 2024-09-19T04:13:56.112 INFO:teuthology.orchestra.run.smithi116.stdout:45 2024-09-19T04:13:56.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi116.stderr:ceph-fuse[34679]: starting fuse 2024-09-19T04:13:57.214 DEBUG:teuthology.orchestra.run.smithi116:> sudo modprobe fuse 2024-09-19T04:13:57.231 DEBUG:teuthology.orchestra.run.smithi116:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/sys 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/proc 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/dev 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/pts 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/run 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/ 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/security 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/shm 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/run/lock 2024-09-19T04:13:57.283 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/cgroup 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/pstore 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/bpf 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/hugepages 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/mqueue 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/debug 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/tracing 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/proc/fs/nfsd 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/fs/fuse/connections 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/sys/kernel/config 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/run/rpc_pipefs 2024-09-19T04:13:57.284 INFO:teuthology.orchestra.run.smithi116.stdout:/run/user/1000 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-4 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-5 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-6 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/var/lib/ceph/osd/ceph-7 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:13:57.285 INFO:teuthology.orchestra.run.smithi116.stdout:/home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:57.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.286 DEBUG:teuthology.orchestra.run.smithi116:> ls /sys/fs/fuse/connections 2024-09-19T04:13:57.334 INFO:teuthology.orchestra.run.smithi116.stdout:45 2024-09-19T04:13:57.335 INFO:teuthology.orchestra.run.smithi116.stdout:46 2024-09-19T04:13:57.335 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> import glob 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> import re 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> import os 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> import subprocess 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> def _find_admin_socket(client_name): 2024-09-19T04:13:57.335 DEBUG:teuthology.orchestra.run.smithi116:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> files = glob.glob(asok_path) 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> # Given a non-glob path, it better be there 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> if "*" not in asok_path: 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> assert(len(files) == 1) 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> return files[0] 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> for f in files: 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> contents = proc_f.read() 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> if mountpoint in contents: 2024-09-19T04:13:57.336 DEBUG:teuthology.orchestra.run.smithi116:> return f 2024-09-19T04:13:57.337 DEBUG:teuthology.orchestra.run.smithi116:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:13:57.337 DEBUG:teuthology.orchestra.run.smithi116:> 2024-09-19T04:13:57.337 DEBUG:teuthology.orchestra.run.smithi116:> print(_find_admin_socket("client.3")) 2024-09-19T04:13:57.337 DEBUG:teuthology.orchestra.run.smithi116:> ' 2024-09-19T04:13:57.463 INFO:teuthology.orchestra.run.smithi116.stdout:/var/run/ceph/ceph-client.3.34679.asok 2024-09-19T04:13:57.467 INFO:teuthology.orchestra.run.smithi116.stderr:2024-09-19T04:13:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:13:57.475 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.34679.asok 2024-09-19T04:13:57.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.475 DEBUG:teuthology.orchestra.run.smithi116:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.34679.asok status 2024-09-19T04:13:57.629 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2024-09-19T04:13:57.629 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": { 2024-09-19T04:13:57.629 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "entity_id": "3", 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "hostname": "smithi116", 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "pid": "34679", 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "root": "/" 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_count": 0, 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "dentry_pinned_count": 0, 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "id": 4406, 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "inst": { 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "name": { 2024-09-19T04:13:57.630 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "client", 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "num": 4406 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.2:0", 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3059998569 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: } 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": { 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "192.168.0.2:0", 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 3059998569 2024-09-19T04:13:57.631 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "inst_str": "client.4406 192.168.0.2:0/3059998569", 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "addr_str": "192.168.0.2:0/3059998569", 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "inode_count": 1, 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "mds_epoch": 8, 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch": 25, 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklisted": false, 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout: "fs_name": "cephfs" 2024-09-19T04:13:57.632 INFO:teuthology.orchestra.run.smithi116.stdout:} 2024-09-19T04:13:57.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.644 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:13:57.644 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:57.652 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:13:57.653 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:57.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.653 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:13:57.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.710 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:13:57.710 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:57.758 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:13:57.758 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:57.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.759 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-19T04:13:57.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.818 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-09-19T04:13:57.818 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:57.825 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-09-19T04:13:57.825 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:57.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.825 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-19T04:13:57.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.879 DEBUG:teuthology.orchestra.run.smithi116:> stat --file-system '--printf=%T 2024-09-19T04:13:57.879 DEBUG:teuthology.orchestra.run.smithi116:> ' -- /home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:57.927 INFO:teuthology.orchestra.run.smithi116.stdout:fuseblk 2024-09-19T04:13:57.927 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:57.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:13:57.928 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-19T04:13:57.982 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-19T04:13:58.010 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-19T04:13:58.010 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:13:58.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a92118640 1 Processor -- start 2024-09-19T04:13:58.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a92118640 1 -- start start 2024-09-19T04:13:58.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a92118640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a8c0d3060 0x7f7a8c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a92118640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8c0d0300 con 0x7f7a8c0cf450 2024-09-19T04:13:58.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a92118640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a8c0b0f60 con 0x7f7a8c161a60 2024-09-19T04:13:58.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a92118640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7a8c0cfd40 con 0x7f7a8c0d3060 2024-09-19T04:13:58.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a8affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a8c0d3060 0x7f7a8c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a8affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a8c0d3060 0x7f7a8c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36586/0 (socket says 172.21.15.76:36586) 2024-09-19T04:13:58.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a8affd640 1 -- 172.21.15.76:0/4129002707 learned_addr learned my addr 172.21.15.76:0/4129002707 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:58.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a8bfff640 1 --1- 172.21.15.76:0/4129002707 >> v1:172.21.15.76:6789/0 conn(0x7f7a8c0cf450 0x7f7a8c06d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50724/0 (socket says 172.21.15.76:50724) 2024-09-19T04:13:58.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a8affd640 1 -- 172.21.15.76:0/4129002707 >> v1:172.21.15.116:6789/0 conn(0x7f7a8c161a60 legacy=0x7f7a8c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a8affd640 1 -- 172.21.15.76:0/4129002707 >> v1:172.21.15.76:6789/0 conn(0x7f7a8c0cf450 legacy=0x7f7a8c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.142+0000 7f7a8affd640 1 -- 172.21.15.76:0/4129002707 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c0d0020 con 0x7f7a8c0d3060 2024-09-19T04:13:58.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8affd640 1 --2- 172.21.15.76:0/4129002707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c0cef10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009870 tx=0x7f7a8001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb1ac8b00aa7512 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8a7fc640 1 -- 172.21.15.76:0/4129002707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800b0070 con 0x7f7a8c0d3060 2024-09-19T04:13:58.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8a7fc640 1 -- 172.21.15.76:0/4129002707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a800ab600 con 0x7f7a8c0d3060 2024-09-19T04:13:58.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8a7fc640 1 -- 172.21.15.76:0/4129002707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800ab9d0 con 0x7f7a8c0d3060 2024-09-19T04:13:58.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- 172.21.15.76:0/4129002707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 msgr2=0x7f7a8c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 --2- 172.21.15.76:0/4129002707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c0cef10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009870 tx=0x7f7a8001c360 comp rx=0 tx=0).stop 2024-09-19T04:13:58.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- 172.21.15.76:0/4129002707 shutdown_connections 2024-09-19T04:13:58.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 --2- 172.21.15.76:0/4129002707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c0cef10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- 172.21.15.76:0/4129002707 >> 172.21.15.76:0/4129002707 conn(0x7f7a8c08bf90 msgr2=0x7f7a8c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- 172.21.15.76:0/4129002707 shutdown_connections 2024-09-19T04:13:58.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- 172.21.15.76:0/4129002707 wait complete. 2024-09-19T04:13:58.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 Processor -- start 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- start start 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a8c0cf450 0x7f7a8c0b8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c161a60 0x7f7a8c0c9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c0cf8c0 con 0x7f7a8c0d3060 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c06d190 con 0x7f7a8c161a60 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a92118640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7a8c0cfa40 con 0x7f7a8c0cf450 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36594/0 (socket says 172.21.15.76:36594) 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8affd640 1 -- 172.21.15.76:0/3076956556 learned_addr learned my addr 172.21.15.76:0/3076956556 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8bfff640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c161a60 0x7f7a8c0c9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.146+0000 7f7a8affd640 1 -- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a8c0cf450 msgr2=0x7f7a8c0b8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8b7fe640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a8c0cf450 0x7f7a8c0b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8affd640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a8c0cf450 0x7f7a8c0b8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8affd640 1 -- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c161a60 msgr2=0x7f7a8c0c9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8affd640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c161a60 0x7f7a8c0c9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8affd640 1 -- 172.21.15.76:0/3076956556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c0b9ff0 con 0x7f7a8c0d3060 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8b7fe640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a8c0cf450 0x7f7a8c0b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8bfff640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c161a60 0x7f7a8c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:58.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8affd640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c152770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a800ae040 tx=0x7f7a80019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a73fff640 1 -- 172.21.15.76:0/3076956556 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800b0040 con 0x7f7a8c0d3060 2024-09-19T04:13:58.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a73fff640 1 -- 172.21.15.76:0/3076956556 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a800b9070 con 0x7f7a8c0d3060 2024-09-19T04:13:58.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c0b72d0 con 0x7f7a8c0d3060 2024-09-19T04:13:58.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a73fff640 1 -- 172.21.15.76:0/3076956556 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a800aab90 con 0x7f7a8c0d3060 2024-09-19T04:13:58.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c0b7890 con 0x7f7a8c0d3060 2024-09-19T04:13:58.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c0cef10 con 0x7f7a8c0d3060 2024-09-19T04:13:58.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a73fff640 1 -- 172.21.15.76:0/3076956556 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a800be460 con 0x7f7a8c0d3060 2024-09-19T04:13:58.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a73fff640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a780678c0 0x7f7a78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a73fff640 1 -- 172.21.15.76:0/3076956556 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7a80120110 con 0x7f7a8c0d3060 2024-09-19T04:13:58.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8b7fe640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a780678c0 0x7f7a78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.150+0000 7f7a8b7fe640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a780678c0 0x7f7a78069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c001650 tx=0x7f7a7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.154+0000 7f7a73fff640 1 -- 172.21.15.76:0/3076956556 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7a80016070 con 0x7f7a8c0d3060 2024-09-19T04:13:58.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.314+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7a8c09d8a0 con 0x7f7a8c0d3060 2024-09-19T04:13:58.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a73fff640 1 -- 172.21.15.76:0/3076956556 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f7a800f09a0 con 0x7f7a8c0d3060 2024-09-19T04:13:58.317 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:13:58.318 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:13:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a780678c0 msgr2=0x7f7a78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a780678c0 0x7f7a78069d80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c001650 tx=0x7f7a7c009040 comp rx=0 tx=0).stop 2024-09-19T04:13:58.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 msgr2=0x7f7a8c152770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c152770 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7a800ae040 tx=0x7f7a80019100 comp rx=0 tx=0).stop 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 shutdown_connections 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a780678c0 0x7f7a78069d80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a8c161a60 0x7f7a8c0c9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a8c0d3060 0x7f7a8c152770 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 --2- 172.21.15.76:0/3076956556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a8c0cf450 0x7f7a8c0b8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 >> 172.21.15.76:0/3076956556 conn(0x7f7a8c08bf90 msgr2=0x7f7a8c0b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.318+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 shutdown_connections 2024-09-19T04:13:58.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.322+0000 7f7a92118640 1 -- 172.21.15.76:0/3076956556 wait complete. 2024-09-19T04:13:58.337 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:13.812655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T04:13:22.367404+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T04:13:23.326658+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-19T04:13:26.974324+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-19T04:13:58.337 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:13:58.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646c11f640 1 Processor -- start 2024-09-19T04:13:58.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646c11f640 1 -- start start 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646c11f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f64640ce0a0 0x7f64640d0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646c11f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f646415b5a0 0x7f646415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646c11f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646415c040 con 0x7f64640b4e50 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646c11f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f64640cfe00 con 0x7f646415b5a0 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646c11f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f646415c320 con 0x7f64640ce0a0 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f64696ba640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f646415b5a0 0x7f646415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646a6bc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f64640b4e50 0x7f646415c600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50728/0 (socket says 172.21.15.76:50728) 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f646a6bc640 1 -- 172.21.15.76:0/4097298801 learned_addr learned my addr 172.21.15.76:0/4097298801 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f6469ebb640 1 --2- 172.21.15.76:0/4097298801 >> v2:172.21.15.116:3300/0 conn(0x7f64640ce0a0 0x7f64640d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f64696ba640 1 -- 172.21.15.76:0/4097298801 >> v2:172.21.15.116:3300/0 conn(0x7f64640ce0a0 msgr2=0x7f64640d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f64696ba640 1 --2- 172.21.15.76:0/4097298801 >> v2:172.21.15.116:3300/0 conn(0x7f64640ce0a0 0x7f64640d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f64696ba640 1 -- 172.21.15.76:0/4097298801 >> v1:172.21.15.76:6789/0 conn(0x7f64640b4e50 legacy=0x7f646415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f64696ba640 1 -- 172.21.15.76:0/4097298801 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64640cfa90 con 0x7f646415b5a0 2024-09-19T04:13:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f6468eb9640 1 -- 172.21.15.76:0/4097298801 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938344180 0 0) 0x7f646415c040 con 0x7f64640b4e50 2024-09-19T04:13:58.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.466+0000 7f6469ebb640 1 --2- 172.21.15.76:0/4097298801 >> v2:172.21.15.116:3300/0 conn(0x7f64640ce0a0 0x7f64640d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 --2- 172.21.15.76:0/4097298801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646415b5a0 0x7f646415ba10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64540097c0 tx=0x7f645401cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433b5ea3d0985152 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f6468eb9640 1 -- 172.21.15.76:0/4097298801 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64540ab600 con 0x7f646415b5a0 2024-09-19T04:13:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f6468eb9640 1 -- 172.21.15.76:0/4097298801 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64540b0070 con 0x7f646415b5a0 2024-09-19T04:13:58.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f6468eb9640 1 -- 172.21.15.76:0/4097298801 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64540ab9c0 con 0x7f646415b5a0 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- 172.21.15.76:0/4097298801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646415b5a0 msgr2=0x7f646415ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 --2- 172.21.15.76:0/4097298801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646415b5a0 0x7f646415ba10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f64540097c0 tx=0x7f645401cec0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- 172.21.15.76:0/4097298801 shutdown_connections 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 --2- 172.21.15.76:0/4097298801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646415b5a0 0x7f646415ba10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 --2- 172.21.15.76:0/4097298801 >> v2:172.21.15.116:3300/0 conn(0x7f64640ce0a0 0x7f64640d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- 172.21.15.76:0/4097298801 >> 172.21.15.76:0/4097298801 conn(0x7f6464073580 msgr2=0x7f646406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- 172.21.15.76:0/4097298801 shutdown_connections 2024-09-19T04:13:58.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- 172.21.15.76:0/4097298801 wait complete. 2024-09-19T04:13:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 Processor -- start 2024-09-19T04:13:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- start start 2024-09-19T04:13:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64640b4e50 0x7f6464158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64640ce0a0 0x7f6464144270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646415b5a0 0x7f646415a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6464067d50 con 0x7f64640ce0a0 2024-09-19T04:13:58.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64640b56b0 con 0x7f64640b4e50 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646c11f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6464067ed0 con 0x7f646415b5a0 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64640ce0a0 0x7f6464144270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64640ce0a0 0x7f6464144270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36610/0 (socket says 172.21.15.76:36610) 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 -- 172.21.15.76:0/2845090485 learned_addr learned my addr 172.21.15.76:0/2845090485 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f646a6bc640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646415b5a0 0x7f646415a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f6469ebb640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64640b4e50 0x7f6464158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 -- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646415b5a0 msgr2=0x7f646415a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646415b5a0 0x7f646415a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 -- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64640b4e50 msgr2=0x7f6464158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64640b4e50 0x7f6464158750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 -- 172.21.15.76:0/2845090485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6464157bd0 con 0x7f64640ce0a0 2024-09-19T04:13:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f64696ba640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64640ce0a0 0x7f6464144270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64540ad040 tx=0x7f6454019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.470+0000 7f645a7fc640 1 -- 172.21.15.76:0/2845090485 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6454004220 con 0x7f64640ce0a0 2024-09-19T04:13:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f645a7fc640 1 -- 172.21.15.76:0/2845090485 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64540043c0 con 0x7f64640ce0a0 2024-09-19T04:13:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6464156e10 con 0x7f64640ce0a0 2024-09-19T04:13:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f645a7fc640 1 -- 172.21.15.76:0/2845090485 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64540aba40 con 0x7f64640ce0a0 2024-09-19T04:13:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64641573d0 con 0x7f64640ce0a0 2024-09-19T04:13:58.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64640d0290 con 0x7f64640ce0a0 2024-09-19T04:13:58.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f645a7fc640 1 -- 172.21.15.76:0/2845090485 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f64540aa070 con 0x7f64640ce0a0 2024-09-19T04:13:58.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f645a7fc640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64480678c0 0x7f6448069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f645a7fc640 1 -- 172.21.15.76:0/2845090485 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f645411fb60 con 0x7f64640ce0a0 2024-09-19T04:13:58.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f6469ebb640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64480678c0 0x7f6448069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.474+0000 7f6469ebb640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64480678c0 0x7f6448069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.478+0000 7f645a7fc640 1 -- 172.21.15.76:0/2845090485 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f64540b9070 con 0x7f64640ce0a0 2024-09-19T04:13:58.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.642+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6464158c90 con 0x7f64640ce0a0 2024-09-19T04:13:58.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.642+0000 7f645a7fc640 1 -- 172.21.15.76:0/2845090485 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f6454016050 con 0x7f64640ce0a0 2024-09-19T04:13:58.643 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64480678c0 msgr2=0x7f6448069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64480678c0 0x7f6448069d80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6460006fd0 tx=0x7f6460008040 comp rx=0 tx=0).stop 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64640ce0a0 msgr2=0x7f6464144270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64640ce0a0 0x7f6464144270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f64540ad040 tx=0x7f6454019200 comp rx=0 tx=0).stop 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 shutdown_connections 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64480678c0 0x7f6448069d80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646415b5a0 0x7f646415a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64640ce0a0 0x7f6464144270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 --2- 172.21.15.76:0/2845090485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64640b4e50 0x7f6464158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 >> 172.21.15.76:0/2845090485 conn(0x7f6464073580 msgr2=0x7f646415f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 shutdown_connections 2024-09-19T04:13:58.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.646+0000 7f646c11f640 1 -- 172.21.15.76:0/2845090485 wait complete. 2024-09-19T04:13:58.662 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T04:13:58.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc804b3640 1 Processor -- start 2024-09-19T04:13:58.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc804b3640 1 -- start start 2024-09-19T04:13:58.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc804b3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdc781520a0 0x7fdc7806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc804b3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7815f3a0 0x7fdc780adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc804b3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc78070250 con 0x7fdc7806fe50 2024-09-19T04:13:58.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc804b3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fdc78152480 con 0x7fdc781520a0 2024-09-19T04:13:58.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc804b3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdc780aede0 con 0x7fdc7815f3a0 2024-09-19T04:13:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7ea50640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7815f3a0 0x7fdc780adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7e24f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdc781520a0 0x7fdc7806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7da4e640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdc7806fe50 0x7fdc7815ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:36188/0 (socket says 172.21.15.76:36188) 2024-09-19T04:13:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7da4e640 1 -- 172.21.15.76:0/2211398729 learned_addr learned my addr 172.21.15.76:0/2211398729 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7e24f640 1 -- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7815f3a0 msgr2=0x7fdc780adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:58.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7e24f640 1 --2- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7815f3a0 0x7fdc780adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7e24f640 1 -- 172.21.15.76:0/2211398729 >> v1:172.21.15.116:6789/0 conn(0x7fdc7806fe50 legacy=0x7fdc7815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7e24f640 1 -- 172.21.15.76:0/2211398729 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7815fa10 con 0x7fdc781520a0 2024-09-19T04:13:58.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.786+0000 7fdc7ea50640 1 --2- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7815f3a0 0x7fdc780adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:13:58.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc7e24f640 1 --2- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc781520a0 0x7fdc7806f910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc700097c0 tx=0x7fdc7001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24a545a4aa69f7f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc7d24d640 1 -- 172.21.15.76:0/2211398729 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700b2070 con 0x7fdc781520a0 2024-09-19T04:13:58.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc7d24d640 1 -- 172.21.15.76:0/2211398729 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc700ad640 con 0x7fdc781520a0 2024-09-19T04:13:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc7d24d640 1 -- 172.21.15.76:0/2211398729 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700ada10 con 0x7fdc781520a0 2024-09-19T04:13:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 -- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc781520a0 msgr2=0x7fdc7806f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 --2- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc781520a0 0x7fdc7806f910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc700097c0 tx=0x7fdc7001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 -- 172.21.15.76:0/2211398729 shutdown_connections 2024-09-19T04:13:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 --2- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7815f3a0 0x7fdc780adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 --2- 172.21.15.76:0/2211398729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc781520a0 0x7fdc7806f910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 -- 172.21.15.76:0/2211398729 >> 172.21.15.76:0/2211398729 conn(0x7fdc780b1f90 msgr2=0x7fdc780b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 -- 172.21.15.76:0/2211398729 shutdown_connections 2024-09-19T04:13:58.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 -- 172.21.15.76:0/2211398729 wait complete. 2024-09-19T04:13:58.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 Processor -- start 2024-09-19T04:13:58.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.790+0000 7fdc804b3640 1 -- start start 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7806fe50 0x7fdc7809c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc781520a0 0x7fdc7815a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc7815f3a0 0x7fdc7814ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78073d30 con 0x7fdc781520a0 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdc78073a50 con 0x7fdc7815f3a0 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdc78167f10 con 0x7fdc7806fe50 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc781520a0 0x7fdc7815a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc781520a0 0x7fdc7815a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36620/0 (socket says 172.21.15.76:36620) 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 -- 172.21.15.76:0/3375154423 learned_addr learned my addr 172.21.15.76:0/3375154423 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7ea50640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc7815f3a0 0x7fdc7814ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7e24f640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7806fe50 0x7fdc7809c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 -- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7806fe50 msgr2=0x7fdc7809c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7806fe50 0x7fdc7809c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 -- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc7815f3a0 msgr2=0x7fdc7814ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc7815f3a0 0x7fdc7814ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 -- 172.21.15.76:0/3375154423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc7815ade0 con 0x7fdc781520a0 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7e24f640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7806fe50 0x7fdc7809c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7ea50640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc7815f3a0 0x7fdc7814ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7da4e640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc781520a0 0x7fdc7815a8a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c370 tx=0x7fdc6c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc76ffd640 1 -- 172.21.15.76:0/3375154423 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c017070 con 0x7fdc781520a0 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc76ffd640 1 -- 172.21.15.76:0/3375154423 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc6c005cf0 con 0x7fdc781520a0 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc7814e0f0 con 0x7fdc781520a0 2024-09-19T04:13:58.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc76ffd640 1 -- 172.21.15.76:0/3375154423 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c0072f0 con 0x7fdc781520a0 2024-09-19T04:13:58.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc7814e6b0 con 0x7fdc781520a0 2024-09-19T04:13:58.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc7806f910 con 0x7fdc781520a0 2024-09-19T04:13:58.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc76ffd640 1 -- 172.21.15.76:0/3375154423 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc6c01a070 con 0x7fdc781520a0 2024-09-19T04:13:58.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc76ffd640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc640678f0 0x7fdc64069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:58.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc76ffd640 1 -- 172.21.15.76:0/3375154423 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdc6c085330 con 0x7fdc781520a0 2024-09-19T04:13:58.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7e24f640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc640678f0 0x7fdc64069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:58.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.794+0000 7fdc7e24f640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc640678f0 0x7fdc64069db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc700af000 tx=0x7fdc70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:58.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.802+0000 7fdc76ffd640 1 -- 172.21.15.76:0/3375154423 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdc7815ade0 con 0x7fdc781520a0 2024-09-19T04:13:58.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.950+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc7806e540 con 0x7fdc781520a0 2024-09-19T04:13:58.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc76ffd640 1 -- 172.21.15.76:0/3375154423 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fdc6c055b30 con 0x7fdc781520a0 2024-09-19T04:13:58.953 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-09-19T04:13:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc640678f0 msgr2=0x7fdc64069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc640678f0 0x7fdc64069db0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fdc700af000 tx=0x7fdc70008040 comp rx=0 tx=0).stop 2024-09-19T04:13:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc781520a0 msgr2=0x7fdc7815a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc781520a0 0x7fdc7815a8a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c370 tx=0x7fdc6c00c830 comp rx=0 tx=0).stop 2024-09-19T04:13:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 shutdown_connections 2024-09-19T04:13:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc640678f0 0x7fdc64069db0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc7815f3a0 0x7fdc7814ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc781520a0 0x7fdc7815a8a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 --2- 172.21.15.76:0/3375154423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc7806fe50 0x7fdc7809c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:58.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 >> 172.21.15.76:0/3375154423 conn(0x7fdc780b1f90 msgr2=0x7fdc7814d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:58.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 shutdown_connections 2024-09-19T04:13:58.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:58.954+0000 7fdc804b3640 1 -- 172.21.15.76:0/3375154423 wait complete. 2024-09-19T04:13:58.970 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T04:13:59.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeec301640 1 Processor -- start 2024-09-19T04:13:59.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeec301640 1 -- start start 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeec301640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbee4154cd0 0x7fbee40d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeec301640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee40cf4f0 con 0x7fbee40704c0 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeec301640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbee40cf7d0 con 0x7fbee40cee50 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeec301640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbee40cf2c0 con 0x7fbee4154cd0 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea89e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbee40704c0 0x7fbee40b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50740/0 (socket says 172.21.15.76:50740) 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea89e640 1 -- 172.21.15.76:0/3577482886 learned_addr learned my addr 172.21.15.76:0/3577482886 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea09d640 1 --2- 172.21.15.76:0/3577482886 >> v2:172.21.15.116:3300/0 conn(0x7fbee4154cd0 0x7fbee40d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2708312973 0 0) 0x7fbee40cf4f0 con 0x7fbee40704c0 2024-09-19T04:13:59.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbed0003530 con 0x7fbee40704c0 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 159573532 0 0) 0x7fbed0003530 con 0x7fbee40704c0 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 >> v2:172.21.15.116:3300/0 conn(0x7fbee4154cd0 msgr2=0x7fbee40d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 --2- 172.21.15.76:0/3577482886 >> v2:172.21.15.116:3300/0 conn(0x7fbee4154cd0 0x7fbee40d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 >> v1:172.21.15.116:6789/0 conn(0x7fbee40cee50 legacy=0x7fbee415b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee40cfab0 con 0x7fbee40704c0 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fbedc001a00 con 0x7fbee40704c0 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 >> v1:172.21.15.76:6789/0 conn(0x7fbee40704c0 legacy=0x7fbee40b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed00039c0 0x7fbed0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed0004810 0x7fbed0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00051d0 0x7fbed000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbed000c040 con 0x7fbed00039c0 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbed000c1f0 con 0x7fbed0004810 2024-09-19T04:13:59.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbed000c3f0 con 0x7fbed00051d0 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea09d640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed00039c0 0x7fbed0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee989c640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed0004810 0x7fbed0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea09d640 1 -- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00051d0 msgr2=0x7fbed000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea09d640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00051d0 0x7fbed000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea09d640 1 -- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed0004810 msgr2=0x7fbed0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea09d640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed0004810 0x7fbed0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea09d640 1 -- 172.21.15.76:0/3577482886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed0003530 con 0x7fbed00039c0 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbee989c640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed0004810 0x7fbed0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:59.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.090+0000 7fbeea89e640 1 -- 172.21.15.76:0/3577482886 reap_dead start 2024-09-19T04:13:59.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeea09d640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed00039c0 0x7fbed0003de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009ae0 tx=0x7fbee001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ff6629e31e8186 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00b2070 con 0x7fbed00039c0 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbee00a5540 con 0x7fbed00039c0 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbee909b640 1 -- 172.21.15.76:0/3577482886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee00096b0 con 0x7fbed00039c0 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 -- 172.21.15.76:0/3577482886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed00039c0 msgr2=0x7fbed0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed00039c0 0x7fbed0003de0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbee0009ae0 tx=0x7fbee001ca60 comp rx=0 tx=0).stop 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 -- 172.21.15.76:0/3577482886 shutdown_connections 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 --2- 172.21.15.76:0/3577482886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbed00039c0 0x7fbed0003de0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 -- 172.21.15.76:0/3577482886 >> 172.21.15.76:0/3577482886 conn(0x7fbee4073580 msgr2=0x7fbee406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 -- 172.21.15.76:0/3577482886 shutdown_connections 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 -- 172.21.15.76:0/3577482886 wait complete. 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 Processor -- start 2024-09-19T04:13:59.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.094+0000 7fbeec301640 1 -- start start 2024-09-19T04:13:59.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00039c0 0x7fbee41444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee414d580 0x7fbee414b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbee414b7b0 0x7fbee4151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbee4155230 con 0x7fbee414d580 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbee41550b0 con 0x7fbee414b7b0 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbee40708a0 con 0x7fbed00039c0 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee414d580 0x7fbee414b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee414d580 0x7fbee414b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36638/0 (socket says 172.21.15.76:36638) 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 -- 172.21.15.76:0/2865094635 learned_addr learned my addr 172.21.15.76:0/2865094635 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeea09d640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00039c0 0x7fbee41444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeea89e640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbee414b7b0 0x7fbee4151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 -- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00039c0 msgr2=0x7fbee41444a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00039c0 0x7fbee41444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 -- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbee414b7b0 msgr2=0x7fbee4151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbee414b7b0 0x7fbee4151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 -- 172.21.15.76:0/2865094635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee414eaa0 con 0x7fbee414d580 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeea89e640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbee414b7b0 0x7fbee4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbee989c640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee414d580 0x7fbee414b270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbed400d770 tx=0x7fbed400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbedaffd640 1 -- 172.21.15.76:0/2865094635 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4019070 con 0x7fbee414d580 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbedaffd640 1 -- 172.21.15.76:0/2865094635 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed4005270 con 0x7fbee414d580 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee414ed60 con 0x7fbee414d580 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbedaffd640 1 -- 172.21.15.76:0/2865094635 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4007030 con 0x7fbee414d580 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee414f320 con 0x7fbee414d580 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee414bb90 con 0x7fbee414d580 2024-09-19T04:13:59.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbedaffd640 1 -- 172.21.15.76:0/2865094635 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbed401c070 con 0x7fbee414d580 2024-09-19T04:13:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbedaffd640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbec80678f0 0x7fbec8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeea09d640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbec80678f0 0x7fbec8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbedaffd640 1 -- 172.21.15.76:0/2865094635 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbed40857f0 con 0x7fbee414d580 2024-09-19T04:13:59.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.098+0000 7fbeea09d640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbec80678f0 0x7fbec8069db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbee00af000 tx=0x7fbee0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.102+0000 7fbedaffd640 1 -- 172.21.15.76:0/2865094635 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbed4055ff0 con 0x7fbee414d580 2024-09-19T04:13:59.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.254+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbee415b0a0 con 0x7fbee414d580 2024-09-19T04:13:59.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.254+0000 7fbedaffd640 1 -- 172.21.15.76:0/2865094635 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7fbed40558f0 con 0x7fbee414d580 2024-09-19T04:13:59.256 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 64 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbec80678f0 msgr2=0x7fbec8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbec80678f0 0x7fbec8069db0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbee00af000 tx=0x7fbee0008040 comp rx=0 tx=0).stop 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee414d580 msgr2=0x7fbee414b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee414d580 0x7fbee414b270 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbed400d770 tx=0x7fbed400dc30 comp rx=0 tx=0).stop 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 shutdown_connections 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbec80678f0 0x7fbec8069db0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbee414b7b0 0x7fbee4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbee414d580 0x7fbee414b270 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 --2- 172.21.15.76:0/2865094635 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbed00039c0 0x7fbee41444a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 >> 172.21.15.76:0/2865094635 conn(0x7fbee4073580 msgr2=0x7fbee40b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:59.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 shutdown_connections 2024-09-19T04:13:59.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.258+0000 7fbeec301640 1 -- 172.21.15.76:0/2865094635 wait complete. 2024-09-19T04:13:59.273 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-19T04:13:59.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9ab65f640 1 Processor -- start 2024-09-19T04:13:59.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9ab65f640 1 -- start start 2024-09-19T04:13:59.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9ab65f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9a41520a0 0x7fd9a406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9ab65f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9a415f3a0 0x7fd9a40adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9ab65f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9a4070250 con 0x7fd9a406fe50 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9ab65f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a4152480 con 0x7fd9a415f3a0 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9ab65f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd9a40aede0 con 0x7fd9a41520a0 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9a415f3a0 0x7fd9a40adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd9a415f3a0 0x7fd9a40adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36644/0 (socket says 172.21.15.76:36644) 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 -- 172.21.15.76:0/1501354372 learned_addr learned my addr 172.21.15.76:0/1501354372 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a93fb640 1 --2- 172.21.15.76:0/1501354372 >> v2:172.21.15.116:3300/0 conn(0x7fd9a41520a0 0x7fd9a406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 -- 172.21.15.76:0/1501354372 >> v2:172.21.15.116:3300/0 conn(0x7fd9a41520a0 msgr2=0x7fd9a406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 --2- 172.21.15.76:0/1501354372 >> v2:172.21.15.116:3300/0 conn(0x7fd9a41520a0 0x7fd9a406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 -- 172.21.15.76:0/1501354372 >> v1:172.21.15.76:6789/0 conn(0x7fd9a406fe50 legacy=0x7fd9a415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 -- 172.21.15.76:0/1501354372 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a415fa10 con 0x7fd9a415f3a0 2024-09-19T04:13:59.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd99bfff640 1 -- 172.21.15.76:0/1501354372 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1039123091 0 0) 0x7fd9a4070250 con 0x7fd9a406fe50 2024-09-19T04:13:59.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a93fb640 1 --2- 172.21.15.76:0/1501354372 >> v2:172.21.15.116:3300/0 conn(0x7fd9a41520a0 0x7fd9a406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd9a9bfc640 1 --2- 172.21.15.76:0/1501354372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a415f3a0 0x7fd9a40adc80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4071710 tx=0x7fd99c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877f1406b1fd73e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.394+0000 7fd99bfff640 1 -- 172.21.15.76:0/1501354372 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a57a0 con 0x7fd9a415f3a0 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd99bfff640 1 -- 172.21.15.76:0/1501354372 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd99c0a5940 con 0x7fd9a415f3a0 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd99bfff640 1 -- 172.21.15.76:0/1501354372 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99c0a5d10 con 0x7fd9a415f3a0 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- 172.21.15.76:0/1501354372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a415f3a0 msgr2=0x7fd9a40adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/1501354372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a415f3a0 0x7fd9a40adc80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4071710 tx=0x7fd99c01c9f0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- 172.21.15.76:0/1501354372 shutdown_connections 2024-09-19T04:13:59.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/1501354372 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a415f3a0 0x7fd9a40adc80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/1501354372 >> v2:172.21.15.116:3300/0 conn(0x7fd9a41520a0 0x7fd9a406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- 172.21.15.76:0/1501354372 >> 172.21.15.76:0/1501354372 conn(0x7fd9a40b1f90 msgr2=0x7fd9a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- 172.21.15.76:0/1501354372 shutdown_connections 2024-09-19T04:13:59.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- 172.21.15.76:0/1501354372 wait complete. 2024-09-19T04:13:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 Processor -- start 2024-09-19T04:13:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- start start 2024-09-19T04:13:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a406fe50 0x7fd9a4146120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a41520a0 0x7fd9a4146660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a415f3a0 0x7fd9a414fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4073d30 con 0x7fd9a41520a0 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a4073a50 con 0x7fd9a415f3a0 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a4167f10 con 0x7fd9a406fe50 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a41520a0 0x7fd9a4146660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a41520a0 0x7fd9a4146660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36654/0 (socket says 172.21.15.76:36654) 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 -- 172.21.15.76:0/3066453226 learned_addr learned my addr 172.21.15.76:0/3066453226 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a9bfc640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a415f3a0 0x7fd9a414fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a93fb640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a406fe50 0x7fd9a4146120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 -- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a406fe50 msgr2=0x7fd9a4146120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a406fe50 0x7fd9a4146120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 -- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a415f3a0 msgr2=0x7fd9a414fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a415f3a0 0x7fd9a414fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 -- 172.21.15.76:0/3066453226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a4150dd0 con 0x7fd9a41520a0 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9a8bfa640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a41520a0 0x7fd9a4146660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd994001ec0 tx=0x7fd99400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd999ffb640 1 -- 172.21.15.76:0/3066453226 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994017070 con 0x7fd9a41520a0 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd999ffb640 1 -- 172.21.15.76:0/3066453226 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd994003600 con 0x7fd9a41520a0 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a415c010 con 0x7fd9a41520a0 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd999ffb640 1 -- 172.21.15.76:0/3066453226 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9940039f0 con 0x7fd9a41520a0 2024-09-19T04:13:59.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.398+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a40ccd50 con 0x7fd9a41520a0 2024-09-19T04:13:59.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.402+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a406f910 con 0x7fd9a41520a0 2024-09-19T04:13:59.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.402+0000 7fd999ffb640 1 -- 172.21.15.76:0/3066453226 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd99401a070 con 0x7fd9a41520a0 2024-09-19T04:13:59.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.402+0000 7fd999ffb640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9880678f0 0x7fd988069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.402+0000 7fd999ffb640 1 -- 172.21.15.76:0/3066453226 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd9940853d0 con 0x7fd9a41520a0 2024-09-19T04:13:59.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.402+0000 7fd9a93fb640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9880678f0 0x7fd988069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.402+0000 7fd9a93fb640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9880678f0 0x7fd988069db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.406+0000 7fd999ffb640 1 -- 172.21.15.76:0/3066453226 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd994055bd0 con 0x7fd9a41520a0 2024-09-19T04:13:59.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.566+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fd9a406e540 con 0x7fd9a41520a0 2024-09-19T04:13:59.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.566+0000 7fd999ffb640 1 -- 172.21.15.76:0/3066453226 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fd9940554d0 con 0x7fd9a41520a0 2024-09-19T04:13:59.567 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 32 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9880678f0 msgr2=0x7fd988069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9880678f0 0x7fd988069db0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0008040 comp rx=0 tx=0).stop 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a41520a0 msgr2=0x7fd9a4146660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a41520a0 0x7fd9a4146660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd994001ec0 tx=0x7fd99400c8e0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 shutdown_connections 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9880678f0 0x7fd988069db0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a415f3a0 0x7fd9a414fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a41520a0 0x7fd9a4146660 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 --2- 172.21.15.76:0/3066453226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a406fe50 0x7fd9a4146120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 >> 172.21.15.76:0/3066453226 conn(0x7fd9a40b1f90 msgr2=0x7fd9a41511b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 shutdown_connections 2024-09-19T04:13:59.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.570+0000 7fd9ab65f640 1 -- 172.21.15.76:0/3066453226 wait complete. 2024-09-19T04:13:59.586 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-09-19T04:13:59.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f608b5a4640 1 Processor -- start 2024-09-19T04:13:59.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f608b5a4640 1 -- start start 2024-09-19T04:13:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f608b5a4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f608415bec0 0x7f60840ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f608b5a4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f608415c2a0 0x7f60840aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f608b5a4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60840b3290 con 0x7f60840af2a0 2024-09-19T04:13:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f608b5a4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f60840b3570 con 0x7f608415bec0 2024-09-19T04:13:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f608b5a4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f60840b3850 con 0x7f608415c2a0 2024-09-19T04:13:59.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089340640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f608415bec0 0x7f60840ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089340640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f608415bec0 0x7f60840ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36670/0 (socket says 172.21.15.76:36670) 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089b41640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f60840af2a0 0x7f608415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50768/0 (socket says 172.21.15.76:50768) 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089340640 1 -- 172.21.15.76:0/979308610 learned_addr learned my addr 172.21.15.76:0/979308610 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6088b3f640 1 --2- 172.21.15.76:0/979308610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f608415c2a0 0x7f60840aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089340640 1 -- 172.21.15.76:0/979308610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f608415c2a0 msgr2=0x7f60840aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089340640 1 --2- 172.21.15.76:0/979308610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f608415c2a0 0x7f60840aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089340640 1 -- 172.21.15.76:0/979308610 >> v1:172.21.15.76:6789/0 conn(0x7f60840af2a0 legacy=0x7f608415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f607bfff640 1 -- 172.21.15.76:0/979308610 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 150167213 0 0) 0x7f60840b3290 con 0x7f60840af2a0 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6089340640 1 -- 172.21.15.76:0/979308610 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840b3b30 con 0x7f608415bec0 2024-09-19T04:13:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.710+0000 7f6088b3f640 1 --2- 172.21.15.76:0/979308610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f608415c2a0 0x7f60840aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:13:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f6089340640 1 --2- 172.21.15.76:0/979308610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f60840ad200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6080009870 tx=0x7f608001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f66fec428264f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f607bfff640 1 -- 172.21.15.76:0/979308610 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800a95f0 con 0x7f608415bec0 2024-09-19T04:13:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f607bfff640 1 -- 172.21.15.76:0/979308610 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60800a9790 con 0x7f608415bec0 2024-09-19T04:13:59.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f607bfff640 1 -- 172.21.15.76:0/979308610 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60800a9b60 con 0x7f608415bec0 2024-09-19T04:13:59.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 -- 172.21.15.76:0/979308610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 msgr2=0x7f60840ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 --2- 172.21.15.76:0/979308610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f60840ad200 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6080009870 tx=0x7f608001c530 comp rx=0 tx=0).stop 2024-09-19T04:13:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 -- 172.21.15.76:0/979308610 shutdown_connections 2024-09-19T04:13:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 --2- 172.21.15.76:0/979308610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f608415c2a0 0x7f60840aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 --2- 172.21.15.76:0/979308610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f60840ad200 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 -- 172.21.15.76:0/979308610 >> 172.21.15.76:0/979308610 conn(0x7f60840b1f90 msgr2=0x7f60840b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:13:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 -- 172.21.15.76:0/979308610 shutdown_connections 2024-09-19T04:13:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.714+0000 7f608b5a4640 1 -- 172.21.15.76:0/979308610 wait complete. 2024-09-19T04:13:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 Processor -- start 2024-09-19T04:13:59.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 -- start start 2024-09-19T04:13:59.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60840af2a0 0x7f60841553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f6084150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f608415c2a0 0x7f6084151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f60840af6f0 con 0x7f608415bec0 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f60840ad740 con 0x7f608415c2a0 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6084169000 con 0x7f60840af2a0 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f6084150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f6084150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36674/0 (socket says 172.21.15.76:36674) 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 -- 172.21.15.76:0/128452271 learned_addr learned my addr 172.21.15.76:0/128452271 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6089340640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60840af2a0 0x7f60841553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6089b41640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f608415c2a0 0x7f6084151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 -- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60840af2a0 msgr2=0x7f60841553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60840af2a0 0x7f60841553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 -- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f608415c2a0 msgr2=0x7f6084151230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f608415c2a0 0x7f6084151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 -- 172.21.15.76:0/128452271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608409af30 con 0x7f608415bec0 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6088b3f640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f6084150cf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f607400c7f0 tx=0x7f607400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6079ffb640 1 -- 172.21.15.76:0/128452271 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074015070 con 0x7f608415bec0 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6079ffb640 1 -- 172.21.15.76:0/128452271 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6074007820 con 0x7f608415bec0 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608409b210 con 0x7f608415bec0 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6079ffb640 1 -- 172.21.15.76:0/128452271 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074007c10 con 0x7f608415bec0 2024-09-19T04:13:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608409df40 con 0x7f608415bec0 2024-09-19T04:13:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60840ad200 con 0x7f608415bec0 2024-09-19T04:13:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6079ffb640 1 -- 172.21.15.76:0/128452271 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6074018070 con 0x7f608415bec0 2024-09-19T04:13:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.718+0000 7f6079ffb640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60680678c0 0x7f6068069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:13:59.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.722+0000 7f6089340640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60680678c0 0x7f6068069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:13:59.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.722+0000 7f6079ffb640 1 -- 172.21.15.76:0/128452271 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6074086170 con 0x7f608415bec0 2024-09-19T04:13:59.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.722+0000 7f6089340640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60680678c0 0x7f6068069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f60800ae000 tx=0x7f60800b3040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:13:59.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.726+0000 7f6079ffb640 1 -- 172.21.15.76:0/128452271 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6074056970 con 0x7f608415bec0 2024-09-19T04:13:59.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:13:59.942+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7f608406fcd0 con 0x7f608415bec0 2024-09-19T04:14:00.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.814+0000 7f6079ffb640 1 -- 172.21.15.76:0/128452271 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v55) ==== 139+0+0 (secure 0 0 0) 0x7f6074056270 con 0x7f608415bec0 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60680678c0 msgr2=0x7f6068069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60680678c0 0x7f6068069d80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f60800ae000 tx=0x7f60800b3040 comp rx=0 tx=0).stop 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 msgr2=0x7f6084150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f6084150cf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f607400c7f0 tx=0x7f607400ccb0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 shutdown_connections 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60680678c0 0x7f6068069d80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f608415c2a0 0x7f6084151230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f608415bec0 0x7f6084150cf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 --2- 172.21.15.76:0/128452271 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60840af2a0 0x7f60841553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 >> 172.21.15.76:0/128452271 conn(0x7f60840b1f90 msgr2=0x7f60840b1310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:00.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 shutdown_connections 2024-09-19T04:14:00.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.818+0000 7f608b5a4640 1 -- 172.21.15.76:0/128452271 wait complete. 2024-09-19T04:14:00.838 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T04:14:00.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf7301640 1 Processor -- start 2024-09-19T04:14:00.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf7301640 1 -- start start 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf7301640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 0x7f5bf00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf7301640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf00b3290 con 0x7f5bf015bec0 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf7301640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bf00b3570 con 0x7f5bf00af2a0 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf7301640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf00b3850 con 0x7f5bf015c2a0 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf509d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5bf015bec0 0x7f5bf00ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50774/0 (socket says 172.21.15.76:50774) 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf509d640 1 -- 172.21.15.76:0/1953657051 learned_addr learned my addr 172.21.15.76:0/1953657051 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214977807 0 0) 0x7f5bf00b3290 con 0x7f5bf015bec0 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bdc003530 con 0x7f5bf015bec0 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 0x7f5bf00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1826436439 0 0) 0x7f5bdc003530 con 0x7f5bf015bec0 2024-09-19T04:14:00.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 msgr2=0x7f5bf00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 0x7f5bf00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 >> v1:172.21.15.116:6789/0 conn(0x7f5bf00af2a0 legacy=0x7f5bf006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf00b3b30 con 0x7f5bf015bec0 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5bec001680 con 0x7f5bf015bec0 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 >> v1:172.21.15.76:6789/0 conn(0x7f5bf015bec0 legacy=0x7f5bf00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 0x7f5bdc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bdc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bdc004d50 0x7f5bdc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00b3850 con 0x7f5bdc004020 2024-09-19T04:14:00.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc00bb90 con 0x7f5bdc004d50 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc00bd90 con 0x7f5bf015c2a0 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bdc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf509d640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 0x7f5bdc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf489c640 1 -- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 msgr2=0x7f5bdc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf015c2a0 0x7f5bdc0039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf489c640 1 -- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bdc004d50 msgr2=0x7f5bdc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf589e640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bdc004d50 0x7f5bdc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bdc004d50 0x7f5bdc0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.966+0000 7f5bf489c640 1 -- 172.21.15.76:0/1953657051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc003530 con 0x7f5bdc004020 2024-09-19T04:14:00.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bdc004810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5be000ba30 tx=0x7f5be001c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c014fe88446da26 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:00.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be00ae070 con 0x7f5bdc004020 2024-09-19T04:14:00.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5be00a9680 con 0x7f5bdc004020 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5be7fff640 1 -- 172.21.15.76:0/1953657051 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be000b600 con 0x7f5bdc004020 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- 172.21.15.76:0/1953657051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 msgr2=0x7f5bdc004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 --2- 172.21.15.76:0/1953657051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bdc004810 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5be000ba30 tx=0x7f5be001c6a0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf589e640 1 -- 172.21.15.76:0/1953657051 reap_dead start 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- 172.21.15.76:0/1953657051 shutdown_connections 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- 172.21.15.76:0/1953657051 >> 172.21.15.76:0/1953657051 conn(0x7f5bf00b1f90 msgr2=0x7f5bf00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- 172.21.15.76:0/1953657051 shutdown_connections 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- 172.21.15.76:0/1953657051 wait complete. 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 Processor -- start 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- start start 2024-09-19T04:14:00.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bf0152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bf00999b0 0x7f5bf01504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf0150a00 0x7f5bf015a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00af730 con 0x7f5bdc004020 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00b40f0 con 0x7f5bf00999b0 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf7301640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf00afc90 con 0x7f5bf0150a00 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf509d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bf0152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf509d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bf0152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36698/0 (socket says 172.21.15.76:36698) 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.970+0000 7f5bf509d640 1 -- 172.21.15.76:0/1817932788 learned_addr learned my addr 172.21.15.76:0/1817932788 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bf00999b0 0x7f5bf01504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf589e640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf0150a00 0x7f5bf015a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf509d640 1 -- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf0150a00 msgr2=0x7f5bf015a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf509d640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf0150a00 0x7f5bf015a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf509d640 1 -- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bf00999b0 msgr2=0x7f5bf01504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf509d640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bf00999b0 0x7f5bf01504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf509d640 1 -- 172.21.15.76:0/1817932788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf0150f80 con 0x7f5bdc004020 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf589e640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf0150a00 0x7f5bf015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf509d640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bf0152020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5bec000fc0 tx=0x7f5bec00e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5be5ffb640 1 -- 172.21.15.76:0/1817932788 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec01b070 con 0x7f5bdc004020 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5be5ffb640 1 -- 172.21.15.76:0/1817932788 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bec011040 con 0x7f5bdc004020 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf0151260 con 0x7f5bdc004020 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5be5ffb640 1 -- 172.21.15.76:0/1817932788 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec0143a0 con 0x7f5bdc004020 2024-09-19T04:14:00.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf0151820 con 0x7f5bdc004020 2024-09-19T04:14:00.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf015b680 con 0x7f5bdc004020 2024-09-19T04:14:00.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5be5ffb640 1 -- 172.21.15.76:0/1817932788 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5bec01e070 con 0x7f5bdc004020 2024-09-19T04:14:00.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5be5ffb640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5bd40678f0 0x7f5bd4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:00.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5be5ffb640 1 -- 172.21.15.76:0/1817932788 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5bec088c50 con 0x7f5bdc004020 2024-09-19T04:14:00.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5bd40678f0 0x7f5bd4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:00.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.974+0000 7f5bf489c640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5bd40678f0 0x7f5bd4069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5be00b0000 tx=0x7f5be00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:00.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:00.978+0000 7f5be5ffb640 1 -- 172.21.15.76:0/1817932788 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5bec0593b0 con 0x7f5bdc004020 2024-09-19T04:14:01.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.138+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5bf006fcd0 con 0x7f5bdc004020 2024-09-19T04:14:01.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.142+0000 7f5be5ffb640 1 -- 172.21.15.76:0/1817932788 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f5bec058cb0 con 0x7f5bdc004020 2024-09-19T04:14:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5bd40678f0 msgr2=0x7f5bd4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5bd40678f0 0x7f5bd4069db0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5be00b0000 tx=0x7f5be00046d0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 msgr2=0x7f5bf0152020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bf0152020 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5bec000fc0 tx=0x7f5bec00e4d0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 shutdown_connections 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5bd40678f0 0x7f5bd4069db0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5bf0150a00 0x7f5bf015a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bf00999b0 0x7f5bf01504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 --2- 172.21.15.76:0/1817932788 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5bdc004020 0x7f5bf0152020 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 >> 172.21.15.76:0/1817932788 conn(0x7f5bf00b1f90 msgr2=0x7f5bf0157510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:01.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 shutdown_connections 2024-09-19T04:14:01.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.146+0000 7f5bf7301640 1 -- 172.21.15.76:0/1817932788 wait complete. 2024-09-19T04:14:01.161 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:14:01.161 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:14:01.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:14:01.311 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:14:01.312 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:14:01.312 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:01.317 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:14:01.317 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:01.366 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:01.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.530+0000 7fb0fbba7640 1 Processor -- start 2024-09-19T04:14:01.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.530+0000 7fb0fbba7640 1 -- start start 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.530+0000 7fb0fbba7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb0f415bec0 0x7fb0f40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.530+0000 7fb0fbba7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 0x7fb0f406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.530+0000 7fb0fbba7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0f40b3290 con 0x7fb0f415c2a0 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.530+0000 7fb0fbba7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb0f40b3570 con 0x7fb0f415bec0 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.530+0000 7fb0fbba7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f40b3850 con 0x7fb0f40af470 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb0f415c2a0 0x7fb0f40aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50786/0 (socket says 172.21.15.76:50786) 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 -- 172.21.15.76:0/3309310861 learned_addr learned my addr 172.21.15.76:0/3309310861 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:01.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491678509 0 0) 0x7fb0f40b3290 con 0x7fb0f415c2a0 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9943640 1 --2- 172.21.15.76:0/3309310861 >> v2:172.21.15.116:3300/0 conn(0x7fb0f415bec0 0x7fb0f40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0fa144640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 0x7fb0f406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb0e0003530 con 0x7fb0f415c2a0 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2292366556 0 0) 0x7fb0e0003530 con 0x7fb0f415c2a0 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 msgr2=0x7fb0f406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 0x7fb0f406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 >> v2:172.21.15.116:3300/0 conn(0x7fb0f415bec0 msgr2=0x7fb0f40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 --2- 172.21.15.76:0/3309310861 >> v2:172.21.15.116:3300/0 conn(0x7fb0f415bec0 0x7fb0f40ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f40b3b30 con 0x7fb0f415c2a0 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb0e4001680 con 0x7fb0f415c2a0 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 >> v1:172.21.15.76:6789/0 conn(0x7fb0f415c2a0 legacy=0x7fb0f40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 0x7fb0e00039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e0004d50 0x7fb0e00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40b3570 con 0x7fb0e0003f40 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40b3850 con 0x7fb0e0004d50 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb0e000ba20 con 0x7fb0f40af470 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9943640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 0x7fb0e00039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0fa144640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e0004d50 0x7fb0e00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 -- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 msgr2=0x7fb0e00039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f40af470 0x7fb0e00039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 -- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e0004d50 msgr2=0x7fb0e00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e0004d50 0x7fb0e00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 -- 172.21.15.76:0/3309310861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0e0003530 con 0x7fb0e0003f40 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f9142640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0e0004810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c240 tx=0x7fb0e401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf20fd47a127e00e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4007d40 con 0x7fb0e0003f40 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0e40a9020 con 0x7fb0e0003f40 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.534+0000 7fb0f8941640 1 -- 172.21.15.76:0/3309310861 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4005b00 con 0x7fb0e0003f40 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/3309310861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 msgr2=0x7fb0e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 --2- 172.21.15.76:0/3309310861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0e0004810 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400c240 tx=0x7fb0e401cc30 comp rx=0 tx=0).stop 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fa144640 1 -- 172.21.15.76:0/3309310861 reap_dead start 2024-09-19T04:14:01.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/3309310861 shutdown_connections 2024-09-19T04:14:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/3309310861 >> 172.21.15.76:0/3309310861 conn(0x7fb0f40b1f90 msgr2=0x7fb0f40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/3309310861 shutdown_connections 2024-09-19T04:14:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/3309310861 wait complete. 2024-09-19T04:14:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 Processor -- start 2024-09-19T04:14:01.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- start start 2024-09-19T04:14:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0f41557b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f414f110 0x7fb0f414f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0f41fa5a0 0x7fb0f41faa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f406e810 con 0x7fb0e0003f40 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f40ad740 con 0x7fb0f41fa5a0 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb0f406e990 con 0x7fb0f414f110 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0f41557b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0f41557b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36710/0 (socket says 172.21.15.76:36710) 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 -- 172.21.15.76:0/346922930 learned_addr learned my addr 172.21.15.76:0/346922930 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9142640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f414f110 0x7fb0f414f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fa144640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0f41fa5a0 0x7fb0f41faa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 -- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f414f110 msgr2=0x7fb0f414f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f414f110 0x7fb0f414f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 -- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0f41fa5a0 msgr2=0x7fb0f41faa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0f41fa5a0 0x7fb0f41faa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 -- 172.21.15.76:0/346922930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f415d910 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0f9943640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0f41557b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00115e0 tx=0x7fb0f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0ea7fc640 1 -- 172.21.15.76:0/346922930 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000f8f0 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0ea7fc640 1 -- 172.21.15.76:0/346922930 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0f000fa90 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f414fbd0 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0ea7fc640 1 -- 172.21.15.76:0/346922930 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000fe40 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f4150190 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.538+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f40ad200 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.542+0000 7fb0ea7fc640 1 -- 172.21.15.76:0/346922930 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb0f0015460 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.542+0000 7fb0ea7fc640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0d80678f0 0x7fb0d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.542+0000 7fb0ea7fc640 1 -- 172.21.15.76:0/346922930 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb0f00869c0 con 0x7fb0e0003f40 2024-09-19T04:14:01.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.542+0000 7fb0f9142640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0d80678f0 0x7fb0d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.542+0000 7fb0f9142640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0d80678f0 0x7fb0d8069db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40aa000 tx=0x7fb0f40b3460 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.546+0000 7fb0ea7fc640 1 -- 172.21.15.76:0/346922930 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb0f0019030 con 0x7fb0e0003f40 2024-09-19T04:14:01.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb0f406fcd0 con 0x7fb0e0003f40 2024-09-19T04:14:01.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0ea7fc640 1 -- 172.21.15.76:0/346922930 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fb0f00571b0 con 0x7fb0e0003f40 2024-09-19T04:14:01.721 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:01.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0d80678f0 msgr2=0x7fb0d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0d80678f0 0x7fb0d8069db0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb0e40aa000 tx=0x7fb0f40b3460 comp rx=0 tx=0).stop 2024-09-19T04:14:01.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 msgr2=0x7fb0f41557b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0f41557b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb0f00115e0 tx=0x7fb0f0011aa0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 shutdown_connections 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb0d80678f0 0x7fb0d8069db0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0f41fa5a0 0x7fb0f41faa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb0f414f110 0x7fb0f414f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 --2- 172.21.15.76:0/346922930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb0e0003f40 0x7fb0f41557b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 >> 172.21.15.76:0/346922930 conn(0x7fb0f40b1f90 msgr2=0x7fb0f41510e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 shutdown_connections 2024-09-19T04:14:01.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.722+0000 7fb0fbba7640 1 -- 172.21.15.76:0/346922930 wait complete. 2024-09-19T04:14:01.740 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:14:01.740 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:01.746 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:01.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 Processor -- start 2024-09-19T04:14:01.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 -- start start 2024-09-19T04:14:01.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dc015c2a0 0x7f1dc00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1dc00af2a0 0x7f1dc015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc00b3290 con 0x7f1dc015c2a0 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1dc00b3570 con 0x7f1dc00af2a0 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dc4dd0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc00b3850 con 0x7f1dc015bec0 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dbdd9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dc015c2a0 0x7f1dc00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dbdd9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dc015c2a0 0x7f1dc00aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36712/0 (socket says 172.21.15.76:36712) 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dbdd9b640 1 -- 172.21.15.76:0/885872397 learned_addr learned my addr 172.21.15.76:0/885872397 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbe59c640 1 --2- 172.21.15.76:0/885872397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.906+0000 7f1dbed9d640 1 --2- 172.21.15.76:0/885872397 >> v2:172.21.15.116:3300/0 conn(0x7f1dc00af2a0 0x7f1dc015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbdd9b640 1 -- 172.21.15.76:0/885872397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 msgr2=0x7f1dc00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbdd9b640 1 --2- 172.21.15.76:0/885872397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc00ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbdd9b640 1 -- 172.21.15.76:0/885872397 >> v2:172.21.15.116:3300/0 conn(0x7f1dc00af2a0 msgr2=0x7f1dc015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:01.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbdd9b640 1 --2- 172.21.15.76:0/885872397 >> v2:172.21.15.116:3300/0 conn(0x7f1dc00af2a0 0x7f1dc015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbdd9b640 1 -- 172.21.15.76:0/885872397 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc00b3b30 con 0x7f1dc015c2a0 2024-09-19T04:14:01.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbed9d640 1 --2- 172.21.15.76:0/885872397 >> v2:172.21.15.116:3300/0 conn(0x7f1dc00af2a0 0x7f1dc015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:01.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbdd9b640 1 --2- 172.21.15.76:0/885872397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc00aed60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00ada40 tx=0x7f1db401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895dbc752c8ab144 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbd59a640 1 -- 172.21.15.76:0/885872397 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40b0070 con 0x7f1dc015c2a0 2024-09-19T04:14:01.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbd59a640 1 -- 172.21.15.76:0/885872397 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db40ab600 con 0x7f1dc015c2a0 2024-09-19T04:14:01.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dbd59a640 1 -- 172.21.15.76:0/885872397 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db40ab9d0 con 0x7f1dc015c2a0 2024-09-19T04:14:01.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/885872397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 msgr2=0x7f1dc00aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/885872397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc00aed60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1dc00ada40 tx=0x7f1db401c230 comp rx=0 tx=0).stop 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/885872397 shutdown_connections 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/885872397 >> v2:172.21.15.116:3300/0 conn(0x7f1dc00af2a0 0x7f1dc015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/885872397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc00aed60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/885872397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc00ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/885872397 >> 172.21.15.76:0/885872397 conn(0x7f1dc00b1f90 msgr2=0x7f1dc00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/885872397 shutdown_connections 2024-09-19T04:14:01.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.910+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/885872397 wait complete. 2024-09-19T04:14:01.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 Processor -- start 2024-09-19T04:14:01.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 -- start start 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dc00af2a0 0x7f1dc015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc009d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc0154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00af6b0 con 0x7f1dc015c2a0 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1dc00ad740 con 0x7f1dc00af2a0 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1dc006e000 con 0x7f1dc015bec0 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc0154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc0154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36716/0 (socket says 172.21.15.76:36716) 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 -- 172.21.15.76:0/1341729650 learned_addr learned my addr 172.21.15.76:0/1341729650 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbe59c640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dc00af2a0 0x7f1dc015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbdd9b640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc009d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 -- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 msgr2=0x7f1dc009d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc009d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 -- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dc00af2a0 msgr2=0x7f1dc015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:01.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dc00af2a0 0x7f1dc015dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:01.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 -- 172.21.15.76:0/1341729650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dc009da00 con 0x7f1dc015c2a0 2024-09-19T04:14:01.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbdd9b640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc009d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:01.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dbed9d640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc0154990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1db800cf20 tx=0x7f1db800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1daeffd640 1 -- 172.21.15.76:0/1341729650 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db801a070 con 0x7f1dc015c2a0 2024-09-19T04:14:01.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1daeffd640 1 -- 172.21.15.76:0/1341729650 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1db800d840 con 0x7f1dc015c2a0 2024-09-19T04:14:01.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dc009dce0 con 0x7f1dc015c2a0 2024-09-19T04:14:01.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1daeffd640 1 -- 172.21.15.76:0/1341729650 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1db800dc30 con 0x7f1dc015c2a0 2024-09-19T04:14:01.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dc0151560 con 0x7f1dc015c2a0 2024-09-19T04:14:01.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dc009e0d0 con 0x7f1dc015c2a0 2024-09-19T04:14:01.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1daeffd640 1 -- 172.21.15.76:0/1341729650 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1db801d070 con 0x7f1dc015c2a0 2024-09-19T04:14:01.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.914+0000 7f1daeffd640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1da00678c0 0x7f1da0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:01.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.918+0000 7f1daeffd640 1 -- 172.21.15.76:0/1341729650 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1db8086880 con 0x7f1dc015c2a0 2024-09-19T04:14:01.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.918+0000 7f1dbe59c640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1da00678c0 0x7f1da0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:01.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.918+0000 7f1dbe59c640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1da00678c0 0x7f1da0069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0155c90 tx=0x7f1db0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:01.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:01.922+0000 7f1daeffd640 1 -- 172.21.15.76:0/1341729650 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1db8057080 con 0x7f1dc015c2a0 2024-09-19T04:14:02.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.090+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1dc00ad200 con 0x7f1dc015c2a0 2024-09-19T04:14:02.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.090+0000 7f1daeffd640 1 -- 172.21.15.76:0/1341729650 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f1db8056980 con 0x7f1dc015c2a0 2024-09-19T04:14:02.091 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1da00678c0 msgr2=0x7f1da0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1da00678c0 0x7f1da0069d80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0155c90 tx=0x7f1db0008040 comp rx=0 tx=0).stop 2024-09-19T04:14:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 msgr2=0x7f1dc0154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc0154990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1db800cf20 tx=0x7f1db800a7d0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 shutdown_connections 2024-09-19T04:14:02.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1da00678c0 0x7f1da0069d80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dc015c2a0 0x7f1dc0154990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dc015bec0 0x7f1dc009d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 --2- 172.21.15.76:0/1341729650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dc00af2a0 0x7f1dc015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 >> 172.21.15.76:0/1341729650 conn(0x7f1dc00b1f90 msgr2=0x7f1dc00b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 shutdown_connections 2024-09-19T04:14:02.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.094+0000 7f1dc4dd0640 1 -- 172.21.15.76:0/1341729650 wait complete. 2024-09-19T04:14:02.109 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:14:02.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:02.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:02.121 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:02.122 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:14:02.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:02.122 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:14:02.169 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:14:02.169 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:02.175 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:14:02.176 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:02.223 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:02.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.346+0000 7f85f2c0c640 1 Processor -- start 2024-09-19T04:14:02.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.346+0000 7f85f2c0c640 1 -- start start 2024-09-19T04:14:02.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f2c0c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f85ec15c2a0 0x7f85ec0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f2c0c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec0af2a0 0x7f85ec06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f2c0c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85ec0b3290 con 0x7f85ec15bec0 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f2c0c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f85ec0b3570 con 0x7f85ec15c2a0 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f2c0c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec0b3850 con 0x7f85ec0af2a0 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f85ec15c2a0 0x7f85ec0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f85ec15c2a0 0x7f85ec0aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36730/0 (socket says 172.21.15.76:36730) 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 -- 172.21.15.76:0/1523330560 learned_addr learned my addr 172.21.15.76:0/1523330560 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f11a9640 1 --2- 172.21.15.76:0/1523330560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec0af2a0 0x7f85ec06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 -- 172.21.15.76:0/1523330560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec0af2a0 msgr2=0x7f85ec06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 --2- 172.21.15.76:0/1523330560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec0af2a0 0x7f85ec06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 -- 172.21.15.76:0/1523330560 >> v1:172.21.15.116:6789/0 conn(0x7f85ec15bec0 legacy=0x7f85ec0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 -- 172.21.15.76:0/1523330560 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec0b3b30 con 0x7f85ec15c2a0 2024-09-19T04:14:02.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f11a9640 1 --2- 172.21.15.76:0/1523330560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec0af2a0 0x7f85ec06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:02.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85ebfff640 1 --2- 172.21.15.76:0/1523330560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec15c2a0 0x7f85ec0aed60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009870 tx=0x7f85e001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd9c587e508761b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:02.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85eb7fe640 1 -- 172.21.15.76:0/1523330560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00a95f0 con 0x7f85ec15c2a0 2024-09-19T04:14:02.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85eb7fe640 1 -- 172.21.15.76:0/1523330560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e00a9790 con 0x7f85ec15c2a0 2024-09-19T04:14:02.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85eb7fe640 1 -- 172.21.15.76:0/1523330560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00a9b60 con 0x7f85ec15c2a0 2024-09-19T04:14:02.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1523330560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec15c2a0 msgr2=0x7f85ec0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.350+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1523330560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec15c2a0 0x7f85ec0aed60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f85e0009870 tx=0x7f85e001c530 comp rx=0 tx=0).stop 2024-09-19T04:14:02.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1523330560 shutdown_connections 2024-09-19T04:14:02.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1523330560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec0af2a0 0x7f85ec06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1523330560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec15c2a0 0x7f85ec0aed60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1523330560 >> 172.21.15.76:0/1523330560 conn(0x7f85ec0b1f90 msgr2=0x7f85ec0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:02.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1523330560 shutdown_connections 2024-09-19T04:14:02.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1523330560 wait complete. 2024-09-19T04:14:02.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 Processor -- start 2024-09-19T04:14:02.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- start start 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec0af2a0 0x7f85ec152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85ec15bec0 0x7f85ec1497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec155f40 0x7f85ec0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec06ed50 con 0x7f85ec0af2a0 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec0af6b0 con 0x7f85ec15bec0 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f85ec06eed0 con 0x7f85ec155f40 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec0af2a0 0x7f85ec152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec0af2a0 0x7f85ec152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36736/0 (socket says 172.21.15.76:36736) 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 -- 172.21.15.76:0/1413025701 learned_addr learned my addr 172.21.15.76:0/1413025701 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f11a9640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec155f40 0x7f85ec0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85ebfff640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85ec15bec0 0x7f85ec1497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 -- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec155f40 msgr2=0x7f85ec0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec155f40 0x7f85ec0b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 -- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85ec15bec0 msgr2=0x7f85ec1497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85ec15bec0 0x7f85ec1497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 -- 172.21.15.76:0/1413025701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec157ae0 con 0x7f85ec0af2a0 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f09a8640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec0af2a0 0x7f85ec152e40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00c370 tx=0x7f85dc00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85e97fa640 1 -- 172.21.15.76:0/1413025701 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc0052d0 con 0x7f85ec0af2a0 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85e97fa640 1 -- 172.21.15.76:0/1413025701 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85dc005470 con 0x7f85ec0af2a0 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec156490 con 0x7f85ec0af2a0 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85e97fa640 1 -- 172.21.15.76:0/1413025701 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc007260 con 0x7f85ec0af2a0 2024-09-19T04:14:02.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec156a50 con 0x7f85ec0af2a0 2024-09-19T04:14:02.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.354+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85ec0aed60 con 0x7f85ec0af2a0 2024-09-19T04:14:02.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.358+0000 7f85e97fa640 1 -- 172.21.15.76:0/1413025701 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f85dc007400 con 0x7f85ec0af2a0 2024-09-19T04:14:02.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.358+0000 7f85e97fa640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f85d80678f0 0x7f85d8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.358+0000 7f85ebfff640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f85d80678f0 0x7f85d8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.358+0000 7f85e97fa640 1 -- 172.21.15.76:0/1413025701 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f85dc085370 con 0x7f85ec0af2a0 2024-09-19T04:14:02.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.358+0000 7f85ebfff640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f85d80678f0 0x7f85d8069db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f85e00ae000 tx=0x7f85e00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:02.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.362+0000 7f85e97fa640 1 -- 172.21.15.76:0/1413025701 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f85ec157ae0 con 0x7f85ec0af2a0 2024-09-19T04:14:02.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.534+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f85ec153380 con 0x7f85ec0af2a0 2024-09-19T04:14:02.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.534+0000 7f85e97fa640 1 -- 172.21.15.76:0/1413025701 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f85dc055520 con 0x7f85ec0af2a0 2024-09-19T04:14:02.535 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:02.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f85d80678f0 msgr2=0x7f85d8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f85d80678f0 0x7f85d8069db0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f85e00ae000 tx=0x7f85e00a7000 comp rx=0 tx=0).stop 2024-09-19T04:14:02.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec0af2a0 msgr2=0x7f85ec152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec0af2a0 0x7f85ec152e40 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00c370 tx=0x7f85dc00c830 comp rx=0 tx=0).stop 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 shutdown_connections 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f85d80678f0 0x7f85d8069db0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f85ec155f40 0x7f85ec0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85ec15bec0 0x7f85ec1497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 --2- 172.21.15.76:0/1413025701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f85ec0af2a0 0x7f85ec152e40 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 >> 172.21.15.76:0/1413025701 conn(0x7f85ec0b1f90 msgr2=0x7f85ec14f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 shutdown_connections 2024-09-19T04:14:02.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.538+0000 7f85f2c0c640 1 -- 172.21.15.76:0/1413025701 wait complete. 2024-09-19T04:14:02.555 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:14:02.555 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:02.562 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:02.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f5304e32640 1 Processor -- start 2024-09-19T04:14:02.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f5304e32640 1 -- start start 2024-09-19T04:14:02.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f5304e32640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f530015bec0 0x7f53000ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f5304e32640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f530015c2a0 0x7f53000aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f5304e32640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53000b3290 con 0x7f53000af2a0 2024-09-19T04:14:02.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f5304e32640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f53000b3570 con 0x7f530015bec0 2024-09-19T04:14:02.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f5304e32640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f53000b3850 con 0x7f530015c2a0 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fe59c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f530015bec0 0x7f53000ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fdd9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f530015c2a0 0x7f53000aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fdd9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f530015c2a0 0x7f53000aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:36470/0 (socket says 172.21.15.76:36470) 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fdd9b640 1 -- 172.21.15.76:0/3295062934 learned_addr learned my addr 172.21.15.76:0/3295062934 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fed9d640 1 --1- 172.21.15.76:0/3295062934 >> v1:172.21.15.116:6789/0 conn(0x7f53000af2a0 0x7f530015fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:35266/0 (socket says 172.21.15.76:35266) 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fdd9b640 1 -- 172.21.15.76:0/3295062934 >> v2:172.21.15.116:3300/0 conn(0x7f530015bec0 msgr2=0x7f53000ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fdd9b640 1 --2- 172.21.15.76:0/3295062934 >> v2:172.21.15.116:3300/0 conn(0x7f530015bec0 0x7f53000ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fdd9b640 1 -- 172.21.15.76:0/3295062934 >> v1:172.21.15.116:6789/0 conn(0x7f53000af2a0 legacy=0x7f530015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fdd9b640 1 -- 172.21.15.76:0/3295062934 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53000b3b30 con 0x7f530015c2a0 2024-09-19T04:14:02.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.686+0000 7f52fe59c640 1 --2- 172.21.15.76:0/3295062934 >> v2:172.21.15.116:3300/0 conn(0x7f530015bec0 0x7f53000ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:02.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f52fdd9b640 1 --2- 172.21.15.76:0/3295062934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f530015c2a0 0x7f53000aed60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009870 tx=0x7f52f401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb7aafb27968163 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f52fd59a640 1 -- 172.21.15.76:0/3295062934 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40a95f0 con 0x7f530015c2a0 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f52fd59a640 1 -- 172.21.15.76:0/3295062934 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f40a9790 con 0x7f530015c2a0 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f52fd59a640 1 -- 172.21.15.76:0/3295062934 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f40a9b60 con 0x7f530015c2a0 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 -- 172.21.15.76:0/3295062934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f530015c2a0 msgr2=0x7f53000aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 --2- 172.21.15.76:0/3295062934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f530015c2a0 0x7f53000aed60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f52f4009870 tx=0x7f52f401c530 comp rx=0 tx=0).stop 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 -- 172.21.15.76:0/3295062934 shutdown_connections 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 --2- 172.21.15.76:0/3295062934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f530015c2a0 0x7f53000aed60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 --2- 172.21.15.76:0/3295062934 >> v2:172.21.15.116:3300/0 conn(0x7f530015bec0 0x7f53000ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 -- 172.21.15.76:0/3295062934 >> 172.21.15.76:0/3295062934 conn(0x7f53000b1f90 msgr2=0x7f53000b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:02.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 -- 172.21.15.76:0/3295062934 shutdown_connections 2024-09-19T04:14:02.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.690+0000 7f5304e32640 1 -- 172.21.15.76:0/3295062934 wait complete. 2024-09-19T04:14:02.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 Processor -- start 2024-09-19T04:14:02.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 -- start start 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53000af2a0 0x7f53000d0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530015bec0 0x7f53001525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f530015c2a0 0x7f5300152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f53000af6b0 con 0x7f530015c2a0 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f53000ad740 con 0x7f530015bec0 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5300169000 con 0x7f53000af2a0 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f530015c2a0 0x7f5300152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f530015c2a0 0x7f5300152b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52800/0 (socket says 172.21.15.76:52800) 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 -- 172.21.15.76:0/734285399 learned_addr learned my addr 172.21.15.76:0/734285399 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fe59c640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53000af2a0 0x7f53000d0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fdd9b640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530015bec0 0x7f53001525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 -- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53000af2a0 msgr2=0x7f53000d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53000af2a0 0x7f53000d0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 -- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530015bec0 msgr2=0x7f53001525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530015bec0 0x7f53001525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 -- 172.21.15.76:0/734285399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5300153230 con 0x7f530015c2a0 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fdd9b640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530015bec0 0x7f53001525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52fed9d640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f530015c2a0 0x7f5300152b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f52f800c7a0 tx=0x7f52f800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52eeffd640 1 -- 172.21.15.76:0/734285399 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f8017070 con 0x7f530015c2a0 2024-09-19T04:14:02.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52eeffd640 1 -- 172.21.15.76:0/734285399 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52f800a640 con 0x7f530015c2a0 2024-09-19T04:14:02.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53001534d0 con 0x7f530015c2a0 2024-09-19T04:14:02.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52eeffd640 1 -- 172.21.15.76:0/734285399 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52f800a9f0 con 0x7f530015c2a0 2024-09-19T04:14:02.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5300072e90 con 0x7f530015c2a0 2024-09-19T04:14:02.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53000ad200 con 0x7f530015c2a0 2024-09-19T04:14:02.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.694+0000 7f52eeffd640 1 -- 172.21.15.76:0/734285399 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52f801a070 con 0x7f530015c2a0 2024-09-19T04:14:02.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.698+0000 7f52eeffd640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52e00678c0 0x7f52e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:02.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.698+0000 7f52eeffd640 1 -- 172.21.15.76:0/734285399 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f52f80854b0 con 0x7f530015c2a0 2024-09-19T04:14:02.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.698+0000 7f52fe59c640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52e00678c0 0x7f52e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:02.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.698+0000 7f52fe59c640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52e00678c0 0x7f52e0069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009580 tx=0x7f52f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:02.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.702+0000 7f52eeffd640 1 -- 172.21.15.76:0/734285399 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f52f80556f0 con 0x7f530015c2a0 2024-09-19T04:14:02.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f530015f500 con 0x7f530015c2a0 2024-09-19T04:14:02.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f52eeffd640 1 -- 172.21.15.76:0/734285399 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f52f80599f0 con 0x7f530015c2a0 2024-09-19T04:14:02.865 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52e00678c0 msgr2=0x7f52e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52e00678c0 0x7f52e0069d80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f52f0009580 tx=0x7f52f0008040 comp rx=0 tx=0).stop 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f530015c2a0 msgr2=0x7f5300152b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f530015c2a0 0x7f5300152b20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f52f800c7a0 tx=0x7f52f800cc60 comp rx=0 tx=0).stop 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 shutdown_connections 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52e00678c0 0x7f52e0069d80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f530015c2a0 0x7f5300152b20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530015bec0 0x7f53001525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 --2- 172.21.15.76:0/734285399 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f53000af2a0 0x7f53000d0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 >> 172.21.15.76:0/734285399 conn(0x7f53000b1f90 msgr2=0x7f53001597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:02.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 shutdown_connections 2024-09-19T04:14:02.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:02.866+0000 7f5304e32640 1 -- 172.21.15.76:0/734285399 wait complete. 2024-09-19T04:14:02.883 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-09-19T04:14:02.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:02.883 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-09-19T04:14:02.906 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:02.906 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:14:02.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:02.907 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T04:14:02.962 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:14:02.962 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:02.968 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:14:02.968 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:03.018 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:03.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 Processor -- start 2024-09-19T04:14:03.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 -- start start 2024-09-19T04:14:03.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f48c4153030 0x7f48c415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48c406ccd0 0x7f48c406d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c40af4c0 0x7f48c40af930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f48c406d680 con 0x7f48c4153030 2024-09-19T04:14:03.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f48c415d5e0 con 0x7f48c406ccd0 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c9764640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f48c40b3e20 con 0x7f48c40af4c0 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f48c4153030 0x7f48c415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f48c4153030 0x7f48c415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52816/0 (socket says 172.21.15.76:52816) 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 -- 172.21.15.76:0/2745215139 learned_addr learned my addr 172.21.15.76:0/2745215139 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c37fe640 1 --2- 172.21.15.76:0/2745215139 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c40af4c0 0x7f48c40af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c27fc640 1 --2- 172.21.15.76:0/2745215139 >> v2:172.21.15.116:3300/0 conn(0x7f48c406ccd0 0x7f48c406d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 -- 172.21.15.76:0/2745215139 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c40af4c0 msgr2=0x7f48c40af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 --2- 172.21.15.76:0/2745215139 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c40af4c0 0x7f48c40af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 -- 172.21.15.76:0/2745215139 >> v2:172.21.15.116:3300/0 conn(0x7f48c406ccd0 msgr2=0x7f48c406d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 --2- 172.21.15.76:0/2745215139 >> v2:172.21.15.116:3300/0 conn(0x7f48c406ccd0 0x7f48c406d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.185+0000 7f48c2ffd640 1 -- 172.21.15.76:0/2745215139 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c40b4100 con 0x7f48c4153030 2024-09-19T04:14:03.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c2ffd640 1 --2- 172.21.15.76:0/2745215139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c4153030 0x7f48c415d0a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48c4068030 tx=0x7f48b401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86ce5cc6915d329 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:03.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c1ffb640 1 -- 172.21.15.76:0/2745215139 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b40b1070 con 0x7f48c4153030 2024-09-19T04:14:03.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c1ffb640 1 -- 172.21.15.76:0/2745215139 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b40ac630 con 0x7f48c4153030 2024-09-19T04:14:03.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c1ffb640 1 -- 172.21.15.76:0/2745215139 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b40a8050 con 0x7f48c4153030 2024-09-19T04:14:03.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- 172.21.15.76:0/2745215139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c4153030 msgr2=0x7f48c415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 --2- 172.21.15.76:0/2745215139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c4153030 0x7f48c415d0a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f48c4068030 tx=0x7f48b401c520 comp rx=0 tx=0).stop 2024-09-19T04:14:03.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- 172.21.15.76:0/2745215139 shutdown_connections 2024-09-19T04:14:03.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 --2- 172.21.15.76:0/2745215139 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c40af4c0 0x7f48c40af930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 --2- 172.21.15.76:0/2745215139 >> v2:172.21.15.116:3300/0 conn(0x7f48c406ccd0 0x7f48c406d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 --2- 172.21.15.76:0/2745215139 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c4153030 0x7f48c415d0a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- 172.21.15.76:0/2745215139 >> 172.21.15.76:0/2745215139 conn(0x7f48c4072690 msgr2=0x7f48c4072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:03.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- 172.21.15.76:0/2745215139 shutdown_connections 2024-09-19T04:14:03.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- 172.21.15.76:0/2745215139 wait complete. 2024-09-19T04:14:03.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 Processor -- start 2024-09-19T04:14:03.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- start start 2024-09-19T04:14:03.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c406ccd0 0x7f48c4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c40af4c0 0x7f48c414c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48c4153030 0x7f48c41586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40b5a30 con 0x7f48c40af4c0 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f48c40b3b30 con 0x7f48c4153030 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c9764640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f48c40b5bb0 con 0x7f48c406ccd0 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c40af4c0 0x7f48c414c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c40af4c0 0x7f48c414c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52818/0 (socket says 172.21.15.76:52818) 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 -- 172.21.15.76:0/3684191955 learned_addr learned my addr 172.21.15.76:0/3684191955 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:03.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c2ffd640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c406ccd0 0x7f48c4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c37fe640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48c4153030 0x7f48c41586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 -- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c406ccd0 msgr2=0x7f48c4151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c406ccd0 0x7f48c4151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 -- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48c4153030 msgr2=0x7f48c41586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48c4153030 0x7f48c41586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c27fc640 1 -- 172.21.15.76:0/3684191955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c414cc70 con 0x7f48c40af4c0 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.189+0000 7f48c37fe640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48c4153030 0x7f48c41586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48c27fc640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c40af4c0 0x7f48c414c730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c7f0 tx=0x7f48b800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48ab7fe640 1 -- 172.21.15.76:0/3684191955 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8015070 con 0x7f48c40af4c0 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48ab7fe640 1 -- 172.21.15.76:0/3684191955 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48b8004220 con 0x7f48c40af4c0 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c4159110 con 0x7f48c40af4c0 2024-09-19T04:14:03.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48ab7fe640 1 -- 172.21.15.76:0/3684191955 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48b8003620 con 0x7f48c40af4c0 2024-09-19T04:14:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c4150f60 con 0x7f48c40af4c0 2024-09-19T04:14:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c4153410 con 0x7f48c40af4c0 2024-09-19T04:14:03.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48ab7fe640 1 -- 172.21.15.76:0/3684191955 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f48b80039d0 con 0x7f48c40af4c0 2024-09-19T04:14:03.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48ab7fe640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48b00678c0 0x7f48b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48c2ffd640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48b00678c0 0x7f48b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48ab7fe640 1 -- 172.21.15.76:0/3684191955 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48b8085fd0 con 0x7f48c40af4c0 2024-09-19T04:14:03.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.193+0000 7f48c2ffd640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48b00678c0 0x7f48b0069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f48b40ae000 tx=0x7f48b40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:03.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.197+0000 7f48ab7fe640 1 -- 172.21.15.76:0/3684191955 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f48b8056860 con 0x7f48c40af4c0 2024-09-19T04:14:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.369+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f48c406d800 con 0x7f48c40af4c0 2024-09-19T04:14:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48ab7fe640 1 -- 172.21.15.76:0/3684191955 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f48b8056160 con 0x7f48c40af4c0 2024-09-19T04:14:03.373 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48b00678c0 msgr2=0x7f48b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48b00678c0 0x7f48b0069d80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f48b40ae000 tx=0x7f48b40a7000 comp rx=0 tx=0).stop 2024-09-19T04:14:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c40af4c0 msgr2=0x7f48c414c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c40af4c0 0x7f48c414c730 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f48b800c7f0 tx=0x7f48b800ccb0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 shutdown_connections 2024-09-19T04:14:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48b00678c0 0x7f48b0069d80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48c4153030 0x7f48c41586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48c40af4c0 0x7f48c414c730 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 --2- 172.21.15.76:0/3684191955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48c406ccd0 0x7f48c4151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 >> 172.21.15.76:0/3684191955 conn(0x7f48c4072690 msgr2=0x7f48c40cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 shutdown_connections 2024-09-19T04:14:03.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.373+0000 7f48c9764640 1 -- 172.21.15.76:0/3684191955 wait complete. 2024-09-19T04:14:03.391 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:14:03.391 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:03.398 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:03.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f40f03640 1 Processor -- start 2024-09-19T04:14:03.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f40f03640 1 -- start start 2024-09-19T04:14:03.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f40f03640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 0x7f4f3c166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f40f03640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f3c0b3290 con 0x7f4f3c15c2a0 2024-09-19T04:14:03.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f40f03640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f3c0b3570 con 0x7f4f3c15bec0 2024-09-19T04:14:03.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f40f03640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0b3850 con 0x7f4f3c06f010 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4f3c15c2a0 0x7f4f3c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47538/0 (socket says 172.21.15.76:47538) 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 -- 172.21.15.76:0/984193612 learned_addr learned my addr 172.21.15.76:0/984193612 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3ad9d640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 0x7f4f3c166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3385113110 0 0) 0x7f4f3c0b3290 con 0x7f4f3c15c2a0 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f24003530 con 0x7f4f3c15c2a0 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1709062754 0 0) 0x7f4f24003530 con 0x7f4f3c15c2a0 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 msgr2=0x7f4f3c166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 0x7f4f3c166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 >> v1:172.21.15.116:6789/0 conn(0x7f4f3c15bec0 legacy=0x7f4f3c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c0b3b30 con 0x7f4f3c15c2a0 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f4f30001680 con 0x7f4f3c15c2a0 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 >> v1:172.21.15.76:6789/0 conn(0x7f4f3c15c2a0 legacy=0x7f4f3c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f240039f0 0x7f4f24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 0x7f4f240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c0b3850 con 0x7f4f24004810 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2400ba80 con 0x7f4f240039f0 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2400bc80 con 0x7f4f3c06f010 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3a59c640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f240039f0 0x7f4f24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3ad9d640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 0x7f4f240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 -- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 msgr2=0x7f4f240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:03.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c06f010 0x7f4f240051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 -- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f240039f0 msgr2=0x7f4f24003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:03.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f240039f0 0x7f4f24003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f39d9b640 1 -- 172.21.15.76:0/984193612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f24003530 con 0x7f4f24004810 2024-09-19T04:14:03.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.565+0000 7f4f3a59c640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f240039f0 0x7f4f24003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:03.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f24004c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c15c6b0 tx=0x7f4f3001f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a524f917a8d75a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:03.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30007d00 con 0x7f4f24004810 2024-09-19T04:14:03.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f30005b00 con 0x7f4f24004810 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f3959a640 1 -- 172.21.15.76:0/984193612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30007070 con 0x7f4f24004810 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 -- 172.21.15.76:0/984193612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 msgr2=0x7f4f24004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 --2- 172.21.15.76:0/984193612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f24004c90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c15c6b0 tx=0x7f4f3001f350 comp rx=0 tx=0).stop 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f3ad9d640 1 -- 172.21.15.76:0/984193612 reap_dead start 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 -- 172.21.15.76:0/984193612 shutdown_connections 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 -- 172.21.15.76:0/984193612 >> 172.21.15.76:0/984193612 conn(0x7f4f3c0b1f90 msgr2=0x7f4f3c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 -- 172.21.15.76:0/984193612 shutdown_connections 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 -- 172.21.15.76:0/984193612 wait complete. 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 Processor -- start 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.569+0000 7f4f40f03640 1 -- start start 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f3c14c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f3c155830 0x7f4f3c09a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c0aff10 0x7f4f3c0b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c0afc90 con 0x7f4f24004810 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3c0b40f0 con 0x7f4f3c155830 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4f3c0af470 con 0x7f4f3c0aff10 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f3c14c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f3c14c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52834/0 (socket says 172.21.15.76:52834) 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 -- 172.21.15.76:0/4216527650 learned_addr learned my addr 172.21.15.76:0/4216527650 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3ad9d640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c0aff10 0x7f4f3c0b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f3c155830 0x7f4f3c09a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 -- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c0aff10 msgr2=0x7f4f3c0b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c0aff10 0x7f4f3c0b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 -- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f3c155830 msgr2=0x7f4f3c09a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f3c155830 0x7f4f3c09a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 -- 172.21.15.76:0/4216527650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f3c09ab80 con 0x7f4f24004810 2024-09-19T04:14:03.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f3c155830 0x7f4f3c09a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f3a59c640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f3c14c680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00d720 tx=0x7f4f2c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f2affd640 1 -- 172.21.15.76:0/4216527650 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c005270 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f2affd640 1 -- 172.21.15.76:0/4216527650 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f2c017070 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f3c09ae60 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f2affd640 1 -- 172.21.15.76:0/4216527650 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c00b6b0 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f3c09b420 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f3c0aed60 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f2affd640 1 -- 172.21.15.76:0/4216527650 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4f2c00b850 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f2affd640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f1c0678f0 0x7f4f1c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f2affd640 1 -- 172.21.15.76:0/4216527650 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4f2c0859d0 con 0x7f4f24004810 2024-09-19T04:14:03.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f1c0678f0 0x7f4f1c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:03.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.573+0000 7f4f39d9b640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f1c0678f0 0x7f4f1c069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4f300ae000 tx=0x7f4f3c0b3460 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:03.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.577+0000 7f4f2affd640 1 -- 172.21.15.76:0/4216527650 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4f2c056260 con 0x7f4f24004810 2024-09-19T04:14:03.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f3c06fcd0 con 0x7f4f24004810 2024-09-19T04:14:03.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f2affd640 1 -- 172.21.15.76:0/4216527650 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f4f2c055b60 con 0x7f4f24004810 2024-09-19T04:14:03.754 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:03.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f1c0678f0 msgr2=0x7f4f1c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f1c0678f0 0x7f4f1c069db0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4f300ae000 tx=0x7f4f3c0b3460 comp rx=0 tx=0).stop 2024-09-19T04:14:03.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 msgr2=0x7f4f3c14c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f3c14c680 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00d720 tx=0x7f4f2c00dbe0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 shutdown_connections 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f1c0678f0 0x7f4f1c069db0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f3c0aff10 0x7f4f3c0b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f3c155830 0x7f4f3c09a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 --2- 172.21.15.76:0/4216527650 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f24004810 0x7f4f3c14c680 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 >> 172.21.15.76:0/4216527650 conn(0x7f4f3c0b1f90 msgr2=0x7f4f3c159f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 shutdown_connections 2024-09-19T04:14:03.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:03.753+0000 7f4f40f03640 1 -- 172.21.15.76:0/4216527650 wait complete. 2024-09-19T04:14:03.772 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:14:03.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:03.772 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-09-19T04:14:03.784 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:03.784 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:14:03.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:03.784 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T04:14:03.833 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:14:03.833 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:03.840 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:14:03.840 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:03.887 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:04.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbf23ea640 1 Processor -- start 2024-09-19T04:14:04.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbf23ea640 1 -- start start 2024-09-19T04:14:04.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbf23ea640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec15bec0 0x7fdbec0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbf23ea640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbec0b3290 con 0x7fdbec15c2a0 2024-09-19T04:14:04.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbf23ea640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdbec0b3570 con 0x7fdbec0af470 2024-09-19T04:14:04.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbf23ea640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdbec0b3850 con 0x7fdbec15bec0 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbeb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdbec15c2a0 0x7fdbec0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47554/0 (socket says 172.21.15.76:47554) 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbeb7fe640 1 -- 172.21.15.76:0/4185906120 learned_addr learned my addr 172.21.15.76:0/4185906120 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbebfff640 1 --2- 172.21.15.76:0/4185906120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec15bec0 0x7fdbec0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbeaffd640 1 -- 172.21.15.76:0/4185906120 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3125241631 0 0) 0x7fdbec0b3290 con 0x7fdbec15c2a0 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.009+0000 7fdbeaffd640 1 -- 172.21.15.76:0/4185906120 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdbd4003530 con 0x7fdbec15c2a0 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbebfff640 1 -- 172.21.15.76:0/4185906120 >> v1:172.21.15.116:6789/0 conn(0x7fdbec0af470 legacy=0x7fdbec06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbebfff640 1 -- 172.21.15.76:0/4185906120 >> v1:172.21.15.76:6789/0 conn(0x7fdbec15c2a0 legacy=0x7fdbec0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbebfff640 1 -- 172.21.15.76:0/4185906120 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec0b3b30 con 0x7fdbec15bec0 2024-09-19T04:14:04.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbebfff640 1 --2- 172.21.15.76:0/4185906120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec15bec0 0x7fdbec0ad200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0097f0 tx=0x7fdbdc01c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f679ebd0a2edaaba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbeaffd640 1 -- 172.21.15.76:0/4185906120 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0aa610 con 0x7fdbec15bec0 2024-09-19T04:14:04.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbeaffd640 1 -- 172.21.15.76:0/4185906120 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbdc0af070 con 0x7fdbec15bec0 2024-09-19T04:14:04.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbeaffd640 1 -- 172.21.15.76:0/4185906120 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc0aa9b0 con 0x7fdbec15bec0 2024-09-19T04:14:04.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbf23ea640 1 -- 172.21.15.76:0/4185906120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec15bec0 msgr2=0x7fdbec0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/4185906120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec15bec0 0x7fdbec0ad200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0097f0 tx=0x7fdbdc01c4b0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbf23ea640 1 -- 172.21.15.76:0/4185906120 shutdown_connections 2024-09-19T04:14:04.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/4185906120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec15bec0 0x7fdbec0ad200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbf23ea640 1 -- 172.21.15.76:0/4185906120 >> 172.21.15.76:0/4185906120 conn(0x7fdbec0b1f90 msgr2=0x7fdbec0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbf23ea640 1 -- 172.21.15.76:0/4185906120 shutdown_connections 2024-09-19T04:14:04.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.013+0000 7fdbf23ea640 1 -- 172.21.15.76:0/4185906120 wait complete. 2024-09-19T04:14:04.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 Processor -- start 2024-09-19T04:14:04.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 -- start start 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec0af470 0x7fdbec099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbec15bec0 0x7fdbec146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbec15c2a0 0x7fdbec1469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec06ef20 con 0x7fdbec15bec0 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec0ad740 con 0x7fdbec15c2a0 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdbec06f0a0 con 0x7fdbec0af470 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbec15bec0 0x7fdbec146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbec15bec0 0x7fdbec146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52838/0 (socket says 172.21.15.76:52838) 2024-09-19T04:14:04.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 -- 172.21.15.76:0/314996318 learned_addr learned my addr 172.21.15.76:0/314996318 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf0987640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbec15c2a0 0x7fdbec1469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbebfff640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec0af470 0x7fdbec099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 -- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec0af470 msgr2=0x7fdbec099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec0af470 0x7fdbec099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 -- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbec15c2a0 msgr2=0x7fdbec1469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbec15c2a0 0x7fdbec1469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 -- 172.21.15.76:0/314996318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec14dc30 con 0x7fdbec15bec0 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbeb7fe640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbec15bec0 0x7fdbec146490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c460 tx=0x7fdbe000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbe8ff9640 1 -- 172.21.15.76:0/314996318 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0017070 con 0x7fdbec15bec0 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbe8ff9640 1 -- 172.21.15.76:0/314996318 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdbe0005d10 con 0x7fdbec15bec0 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbe8ff9640 1 -- 172.21.15.76:0/314996318 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe0007310 con 0x7fdbec15bec0 2024-09-19T04:14:04.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec14d0b0 con 0x7fdbec15bec0 2024-09-19T04:14:04.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec14d670 con 0x7fdbec15bec0 2024-09-19T04:14:04.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.017+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec0ad200 con 0x7fdbec15bec0 2024-09-19T04:14:04.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.021+0000 7fdbe8ff9640 1 -- 172.21.15.76:0/314996318 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdbe00074b0 con 0x7fdbec15bec0 2024-09-19T04:14:04.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.021+0000 7fdbe8ff9640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdbd80678c0 0x7fdbd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.021+0000 7fdbe8ff9640 1 -- 172.21.15.76:0/314996318 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdbe00853a0 con 0x7fdbec15bec0 2024-09-19T04:14:04.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.021+0000 7fdbebfff640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdbd80678c0 0x7fdbd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.021+0000 7fdbebfff640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdbd80678c0 0x7fdbd8069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0ae000 tx=0x7fdbdc0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.025+0000 7fdbe8ff9640 1 -- 172.21.15.76:0/314996318 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdbe0055ba0 con 0x7fdbec15bec0 2024-09-19T04:14:04.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.197+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdbec06fcd0 con 0x7fdbec15bec0 2024-09-19T04:14:04.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.197+0000 7fdbe8ff9640 1 -- 172.21.15.76:0/314996318 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fdbe00554a0 con 0x7fdbec15bec0 2024-09-19T04:14:04.199 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:04.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdbd80678c0 msgr2=0x7fdbd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdbd80678c0 0x7fdbd8069d80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc0ae000 tx=0x7fdbdc0a7000 comp rx=0 tx=0).stop 2024-09-19T04:14:04.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbec15bec0 msgr2=0x7fdbec146490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbec15bec0 0x7fdbec146490 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fdbe000c460 tx=0x7fdbe000c920 comp rx=0 tx=0).stop 2024-09-19T04:14:04.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 shutdown_connections 2024-09-19T04:14:04.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdbd80678c0 0x7fdbd8069d80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdbec15c2a0 0x7fdbec1469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdbec15bec0 0x7fdbec146490 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 --2- 172.21.15.76:0/314996318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdbec0af470 0x7fdbec099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 >> 172.21.15.76:0/314996318 conn(0x7fdbec0b1f90 msgr2=0x7fdbec0b13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 shutdown_connections 2024-09-19T04:14:04.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.201+0000 7fdbf23ea640 1 -- 172.21.15.76:0/314996318 wait complete. 2024-09-19T04:14:04.219 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:14:04.219 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:14:04.229 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:14:04.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09ffb3640 1 Processor -- start 2024-09-19T04:14:04.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09ffb3640 1 -- start start 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09ffb3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09815e450 0x7fc09809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09ffb3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc0980b40c0 0x7fc09815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09ffb3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0980b5c90 con 0x7fc09806bcc0 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09ffb3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc0980b4860 con 0x7fc0980b40c0 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09ffb3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc0980b4580 con 0x7fc09815e450 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09d54e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc0980b40c0 0x7fc09815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09e550640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc09806bcc0 0x7fc098162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47556/0 (socket says 172.21.15.76:47556) 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09d54e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc0980b40c0 0x7fc09815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52840/0 (socket says 172.21.15.76:52840) 2024-09-19T04:14:04.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09e550640 1 -- 172.21.15.76:0/3941306875 learned_addr learned my addr 172.21.15.76:0/3941306875 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09dd4f640 1 --2- 172.21.15.76:0/3941306875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09815e450 0x7fc09809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09d54e640 1 -- 172.21.15.76:0/3941306875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09815e450 msgr2=0x7fc09809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09d54e640 1 --2- 172.21.15.76:0/3941306875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09815e450 0x7fc09809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09d54e640 1 -- 172.21.15.76:0/3941306875 >> v1:172.21.15.76:6789/0 conn(0x7fc09806bcc0 legacy=0x7fc098162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09d54e640 1 -- 172.21.15.76:0/3941306875 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0980b5f10 con 0x7fc0980b40c0 2024-09-19T04:14:04.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.349+0000 7fc09dd4f640 1 --2- 172.21.15.76:0/3941306875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09815e450 0x7fc09809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09d54e640 1 --2- 172.21.15.76:0/3941306875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0980b40c0 0x7fc09815fc60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0097c0 tx=0x7fc08c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3656f63035544f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09cd4d640 1 -- 172.21.15.76:0/3941306875 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0b0070 con 0x7fc0980b40c0 2024-09-19T04:14:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09cd4d640 1 -- 172.21.15.76:0/3941306875 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc08c0ab600 con 0x7fc0980b40c0 2024-09-19T04:14:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09cd4d640 1 -- 172.21.15.76:0/3941306875 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0ab9d0 con 0x7fc0980b40c0 2024-09-19T04:14:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- 172.21.15.76:0/3941306875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0980b40c0 msgr2=0x7fc09815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/3941306875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0980b40c0 0x7fc09815fc60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc08c0097c0 tx=0x7fc08c01c410 comp rx=0 tx=0).stop 2024-09-19T04:14:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- 172.21.15.76:0/3941306875 shutdown_connections 2024-09-19T04:14:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/3941306875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc0980b40c0 0x7fc09815fc60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/3941306875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09815e450 0x7fc09809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- 172.21.15.76:0/3941306875 >> 172.21.15.76:0/3941306875 conn(0x7fc0980b6800 msgr2=0x7fc0980b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- 172.21.15.76:0/3941306875 shutdown_connections 2024-09-19T04:14:04.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- 172.21.15.76:0/3941306875 wait complete. 2024-09-19T04:14:04.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 Processor -- start 2024-09-19T04:14:04.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- start start 2024-09-19T04:14:04.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09806bcc0 0x7fc098157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980b40c0 0x7fc0981553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc09815e450 0x7fc0981558f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc09806c4e0 con 0x7fc09815e450 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc098162e90 con 0x7fc0980b40c0 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09ffb3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc09814dc20 con 0x7fc09806bcc0 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09e550640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc09815e450 0x7fc0981558f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09e550640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc09815e450 0x7fc0981558f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52842/0 (socket says 172.21.15.76:52842) 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09e550640 1 -- 172.21.15.76:0/4052427462 learned_addr learned my addr 172.21.15.76:0/4052427462 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.353+0000 7fc09dd4f640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09806bcc0 0x7fc098157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09e550640 1 -- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09806bcc0 msgr2=0x7fc098157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09e550640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09806bcc0 0x7fc098157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09e550640 1 -- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980b40c0 msgr2=0x7fc0981553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09d54e640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980b40c0 0x7fc0981553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09e550640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980b40c0 0x7fc0981553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09e550640 1 -- 172.21.15.76:0/4052427462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc09814df00 con 0x7fc09815e450 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09dd4f640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09806bcc0 0x7fc098157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09e550640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc09815e450 0x7fc0981558f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc08800c400 tx=0x7fc08800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc0967fc640 1 -- 172.21.15.76:0/4052427462 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc088017070 con 0x7fc09815e450 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc0967fc640 1 -- 172.21.15.76:0/4052427462 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc088005cf0 con 0x7fc09815e450 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc09814e1e0 con 0x7fc09815e450 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc0967fc640 1 -- 172.21.15.76:0/4052427462 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0880072d0 con 0x7fc09815e450 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc09814f2d0 con 0x7fc09815e450 2024-09-19T04:14:04.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc09815fc60 con 0x7fc09815e450 2024-09-19T04:14:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc0967fc640 1 -- 172.21.15.76:0/4052427462 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc08801a070 con 0x7fc09815e450 2024-09-19T04:14:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc0967fc640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc0840678c0 0x7fc084069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc0967fc640 1 -- 172.21.15.76:0/4052427462 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc088085180 con 0x7fc09815e450 2024-09-19T04:14:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09dd4f640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc0840678c0 0x7fc084069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.357+0000 7fc09dd4f640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc0840678c0 0x7fc084069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc090007f40 tx=0x7fc090007490 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.361+0000 7fc0967fc640 1 -- 172.21.15.76:0/4052427462 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc0980b5c90 con 0x7fc09815e450 2024-09-19T04:14:04.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.537+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc09806e7c0 con 0x7fc09815e450 2024-09-19T04:14:04.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.537+0000 7fc0967fc640 1 -- 172.21.15.76:0/4052427462 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fc0880553c0 con 0x7fc09815e450 2024-09-19T04:14:04.538 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc0840678c0 msgr2=0x7fc084069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc0840678c0 0x7fc084069d80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc090007f40 tx=0x7fc090007490 comp rx=0 tx=0).stop 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc09815e450 msgr2=0x7fc0981558f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc09815e450 0x7fc0981558f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc08800c400 tx=0x7fc08800c8c0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 shutdown_connections 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc0840678c0 0x7fc084069d80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc09815e450 0x7fc0981558f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0980b40c0 0x7fc0981553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 --2- 172.21.15.76:0/4052427462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc09806bcc0 0x7fc098157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 >> 172.21.15.76:0/4052427462 conn(0x7fc0980b6800 msgr2=0x7fc0981602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 shutdown_connections 2024-09-19T04:14:04.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.541+0000 7fc09ffb3640 1 -- 172.21.15.76:0/4052427462 wait complete. 2024-09-19T04:14:04.557 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi116.front.sepia.ceph.com... 2024-09-19T04:14:04.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:04.557 DEBUG:teuthology.orchestra.run.smithi116:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-09-19T04:14:04.572 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:04.572 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:14:04.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:04.572 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T04:14:04.623 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T04:14:04.623 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:04.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.745+0000 7f6cfedcd640 1 Processor -- start 2024-09-19T04:14:04.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.745+0000 7f6cfedcd640 1 -- start start 2024-09-19T04:14:04.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfedcd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6cf81520a0 0x7f6cf806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfedcd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf80adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfedcd640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf8070250 con 0x7f6cf806fe50 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfedcd640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf8152480 con 0x7f6cf81520a0 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfedcd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf80aede0 con 0x7f6cf815f3a0 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfcb69640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6cf81520a0 0x7f6cf806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfd36a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf80adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfcb69640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6cf81520a0 0x7f6cf806f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:39304/0 (socket says 172.21.15.76:39304) 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfd36a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf80adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:36570/0 (socket says 172.21.15.76:36570) 2024-09-19T04:14:04.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfcb69640 1 -- 172.21.15.76:0/1078914108 learned_addr learned my addr 172.21.15.76:0/1078914108 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:04.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfcb69640 1 -- 172.21.15.76:0/1078914108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 msgr2=0x7f6cf80adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:04.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfcb69640 1 --2- 172.21.15.76:0/1078914108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf80adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfcb69640 1 -- 172.21.15.76:0/1078914108 >> v1:172.21.15.116:6789/0 conn(0x7f6cf806fe50 legacy=0x7f6cf815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.749+0000 7f6cfcb69640 1 -- 172.21.15.76:0/1078914108 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf815fa10 con 0x7f6cf81520a0 2024-09-19T04:14:04.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfcb69640 1 --2- 172.21.15.76:0/1078914108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf81520a0 0x7f6cf806f910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009870 tx=0x7f6ce801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef22ae6c5f4bb84b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf77fe640 1 -- 172.21.15.76:0/1078914108 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80ab650 con 0x7f6cf81520a0 2024-09-19T04:14:04.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf77fe640 1 -- 172.21.15.76:0/1078914108 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce80b0070 con 0x7f6cf81520a0 2024-09-19T04:14:04.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf77fe640 1 -- 172.21.15.76:0/1078914108 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80aba10 con 0x7f6cf81520a0 2024-09-19T04:14:04.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- 172.21.15.76:0/1078914108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf81520a0 msgr2=0x7f6cf806f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/1078914108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf81520a0 0x7f6cf806f910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8009870 tx=0x7f6ce801c560 comp rx=0 tx=0).stop 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- 172.21.15.76:0/1078914108 shutdown_connections 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/1078914108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf80adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/1078914108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf81520a0 0x7f6cf806f910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- 172.21.15.76:0/1078914108 >> 172.21.15.76:0/1078914108 conn(0x7f6cf80b1f90 msgr2=0x7f6cf80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- 172.21.15.76:0/1078914108 shutdown_connections 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- 172.21.15.76:0/1078914108 wait complete. 2024-09-19T04:14:04.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 Processor -- start 2024-09-19T04:14:04.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- start start 2024-09-19T04:14:04.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf806fe50 0x7f6cf81670a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf81520a0 0x7f6cf81675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf8167b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf8073d30 con 0x7f6cf81520a0 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf8073a50 con 0x7f6cf806fe50 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfedcd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf8165f00 con 0x7f6cf815f3a0 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf81520a0 0x7f6cf81675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf81520a0 0x7f6cf81675e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52858/0 (socket says 172.21.15.76:52858) 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 -- 172.21.15.76:0/2949005485 learned_addr learned my addr 172.21.15.76:0/2949005485 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfcb69640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf806fe50 0x7f6cf81670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfd36a640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf8167b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 -- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 msgr2=0x7f6cf8167b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf8167b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 -- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf806fe50 msgr2=0x7f6cf81670a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf806fe50 0x7f6cf81670a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 -- 172.21.15.76:0/2949005485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf8149190 con 0x7f6cf81520a0 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cfd36a640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf8167b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.753+0000 7f6cf7fff640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf81520a0 0x7f6cf81675e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c3a0 tx=0x7f6cec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cf57fa640 1 -- 172.21.15.76:0/2949005485 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0052d0 con 0x7f6cf81520a0 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cf57fa640 1 -- 172.21.15.76:0/2949005485 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cec005470 con 0x7f6cf81520a0 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf8164b80 con 0x7f6cf81520a0 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cf57fa640 1 -- 172.21.15.76:0/2949005485 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cec007260 con 0x7f6cf81520a0 2024-09-19T04:14:04.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf8165140 con 0x7f6cf81520a0 2024-09-19T04:14:04.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf80adc80 con 0x7f6cf81520a0 2024-09-19T04:14:04.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cf57fa640 1 -- 172.21.15.76:0/2949005485 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6cec007400 con 0x7f6cf81520a0 2024-09-19T04:14:04.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cf57fa640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ce40678c0 0x7f6ce4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:04.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cf57fa640 1 -- 172.21.15.76:0/2949005485 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6cec085310 con 0x7f6cf81520a0 2024-09-19T04:14:04.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cfcb69640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ce40678c0 0x7f6ce4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:04.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.757+0000 7f6cfcb69640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ce40678c0 0x7f6ce4069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80af000 tx=0x7f6ce80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:04.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.761+0000 7f6cf57fa640 1 -- 172.21.15.76:0/2949005485 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6cec055550 con 0x7f6cf81520a0 2024-09-19T04:14:04.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.961+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6cf80b3dd0 con 0x7f6cf81520a0 2024-09-19T04:14:04.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.961+0000 7f6cf57fa640 1 -- 172.21.15.76:0/2949005485 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7f6cec059850 con 0x7f6cf81520a0 2024-09-19T04:14:04.965 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:04.966 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ce40678c0 msgr2=0x7f6ce4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ce40678c0 0x7f6ce4069d80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80af000 tx=0x7f6ce80a7000 comp rx=0 tx=0).stop 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf81520a0 msgr2=0x7f6cf81675e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf81520a0 0x7f6cf81675e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6cec00c3a0 tx=0x7f6cec00c860 comp rx=0 tx=0).stop 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 shutdown_connections 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ce40678c0 0x7f6ce4069d80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf815f3a0 0x7f6cf8167b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf81520a0 0x7f6cf81675e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 --2- 172.21.15.76:0/2949005485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf806fe50 0x7f6cf81670a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 >> 172.21.15.76:0/2949005485 conn(0x7f6cf80b1f90 msgr2=0x7f6cf806f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:04.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 shutdown_connections 2024-09-19T04:14:04.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:04.965+0000 7f6cfedcd640 1 -- 172.21.15.76:0/2949005485 wait complete. 2024-09-19T04:14:04.982 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f4507b25640 1 Processor -- start 2024-09-19T04:14:05.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f4507b25640 1 -- start start 2024-09-19T04:14:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f4507b25640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f45001520a0 0x7f450006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f4507b25640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4500070250 con 0x7f450006fe50 2024-09-19T04:14:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f4507b25640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45000aede0 con 0x7f450015f3a0 2024-09-19T04:14:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f4507b25640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4500152480 con 0x7f45001520a0 2024-09-19T04:14:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45050c0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f450006fe50 0x7f450015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47570/0 (socket says 172.21.15.76:47570) 2024-09-19T04:14:05.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45050c0640 1 -- 172.21.15.76:0/1739642480 learned_addr learned my addr 172.21.15.76:0/1739642480 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:05.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45058c1640 1 --2- 172.21.15.76:0/1739642480 >> v2:172.21.15.116:3300/0 conn(0x7f45001520a0 0x7f450006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 688709143 0 0) 0x7f4500070250 con 0x7f450006fe50 2024-09-19T04:14:05.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44ec003530 con 0x7f450006fe50 2024-09-19T04:14:05.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2265260865 0 0) 0x7f44ec003530 con 0x7f450006fe50 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 >> v2:172.21.15.116:3300/0 conn(0x7f45001520a0 msgr2=0x7f450006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 --2- 172.21.15.76:0/1739642480 >> v2:172.21.15.116:3300/0 conn(0x7f45001520a0 0x7f450006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 >> v1:172.21.15.116:6789/0 conn(0x7f450015f3a0 legacy=0x7f45000adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450015fa10 con 0x7f450006fe50 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f44f0001680 con 0x7f450006fe50 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 >> v1:172.21.15.76:6789/0 conn(0x7f450006fe50 legacy=0x7f450015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44ec0039c0 0x7f44ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44ec004810 0x7f44ec004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0051d0 0x7f44ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec00c0f0 con 0x7f44ec0039c0 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f44ec00c2a0 con 0x7f44ec004810 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f44ec00c4a0 con 0x7f44ec0051d0 2024-09-19T04:14:05.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45058c1640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44ec0039c0 0x7f44ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45050c0640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44ec004810 0x7f44ec004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45058c1640 1 -- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0051d0 msgr2=0x7f44ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45060c2640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0051d0 0x7f44ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45058c1640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0051d0 0x7f44ec00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45058c1640 1 -- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44ec004810 msgr2=0x7f44ec004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45058c1640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44ec004810 0x7f44ec004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45058c1640 1 -- 172.21.15.76:0/1739642480 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44ec003530 con 0x7f44ec0039c0 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45060c2640 1 -- 172.21.15.76:0/1739642480 reap_dead start 2024-09-19T04:14:05.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.097+0000 7f45050c0640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44ec004810 0x7f44ec004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:05.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f45058c1640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44ec0039c0 0x7f44ec003de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00ba30 tx=0x7f44fc01c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0c47aef83db33f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc00b600 con 0x7f44ec0039c0 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc0b0070 con 0x7f44ec0039c0 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f45048bf640 1 -- 172.21.15.76:0/1739642480 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0ab870 con 0x7f44ec0039c0 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 -- 172.21.15.76:0/1739642480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44ec0039c0 msgr2=0x7f44ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44ec0039c0 0x7f44ec003de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00ba30 tx=0x7f44fc01c6a0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 -- 172.21.15.76:0/1739642480 shutdown_connections 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 --2- 172.21.15.76:0/1739642480 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f44ec0039c0 0x7f44ec003de0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 -- 172.21.15.76:0/1739642480 >> 172.21.15.76:0/1739642480 conn(0x7f45000b1f90 msgr2=0x7f45000b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 -- 172.21.15.76:0/1739642480 shutdown_connections 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 -- 172.21.15.76:0/1739642480 wait complete. 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 Processor -- start 2024-09-19T04:14:05.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 -- start start 2024-09-19T04:14:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.101+0000 7f4507b25640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0039c0 0x7f45001f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45000c9fd0 0x7f45000cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45000cb540 0x7f45000cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4500073a50 con 0x7f45000c9fd0 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4500073770 con 0x7f45000cb540 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4500073d30 con 0x7f44ec0039c0 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45000c9fd0 0x7f45000cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45000c9fd0 0x7f45000cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52882/0 (socket says 172.21.15.76:52882) 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 -- 172.21.15.76:0/2728349664 learned_addr learned my addr 172.21.15.76:0/2728349664 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45058c1640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0039c0 0x7f45001f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45060c2640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45000cb540 0x7f45000cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 -- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0039c0 msgr2=0x7f45001f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0039c0 0x7f45001f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 -- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45000cb540 msgr2=0x7f45000cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45000cb540 0x7f45000cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 -- 172.21.15.76:0/2728349664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45000ca590 con 0x7f45000c9fd0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45060c2640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45000cb540 0x7f45000cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45050c0640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45000c9fd0 0x7f45000cf7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44f000d7a0 tx=0x7f44f000dba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f44f67fc640 1 -- 172.21.15.76:0/2728349664 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0019070 con 0x7f45000c9fd0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f44f67fc640 1 -- 172.21.15.76:0/2728349664 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44f00072d0 con 0x7f45000c9fd0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45000ca870 con 0x7f45000c9fd0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f44f67fc640 1 -- 172.21.15.76:0/2728349664 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0007680 con 0x7f45000c9fd0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45000cae30 con 0x7f45000c9fd0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450015f0a0 con 0x7f45000c9fd0 2024-09-19T04:14:05.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f44f67fc640 1 -- 172.21.15.76:0/2728349664 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f44f001c070 con 0x7f45000c9fd0 2024-09-19T04:14:05.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f44f67fc640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f44e40678f0 0x7f44e4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45058c1640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f44e40678f0 0x7f44e4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f44f67fc640 1 -- 172.21.15.76:0/2728349664 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f44f0085890 con 0x7f45000c9fd0 2024-09-19T04:14:05.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.105+0000 7f45058c1640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f44e40678f0 0x7f44e4069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0af000 tx=0x7f44fc0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.109+0000 7f44f67fc640 1 -- 172.21.15.76:0/2728349664 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f44f0056090 con 0x7f45000c9fd0 2024-09-19T04:14:05.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.305+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45000cb320 con 0x7f45000c9fd0 2024-09-19T04:14:05.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.305+0000 7f44f67fc640 1 -- 172.21.15.76:0/2728349664 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7f44f0055990 con 0x7f45000c9fd0 2024-09-19T04:14:05.306 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:05.307 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:14:05.307 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f44e40678f0 msgr2=0x7f44e4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f44e40678f0 0x7f44e4069db0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f44fc0af000 tx=0x7f44fc0046d0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45000c9fd0 msgr2=0x7f45000cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45000c9fd0 0x7f45000cf7b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44f000d7a0 tx=0x7f44f000dba0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 shutdown_connections 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f44e40678f0 0x7f44e4069db0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45000cb540 0x7f45000cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45000c9fd0 0x7f45000cf7b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 --2- 172.21.15.76:0/2728349664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f44ec0039c0 0x7f45001f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 >> 172.21.15.76:0/2728349664 conn(0x7f45000b1f90 msgr2=0x7f45000b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 shutdown_connections 2024-09-19T04:14:05.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.309+0000 7f4507b25640 1 -- 172.21.15.76:0/2728349664 wait complete. 2024-09-19T04:14:05.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:14:05.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.433+0000 7f9b1535a640 1 Processor -- start 2024-09-19T04:14:05.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.433+0000 7f9b1535a640 1 -- start start 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.433+0000 7f9b1535a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9b10161a60 0x7f9b100d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.433+0000 7f9b1535a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b1006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.433+0000 7f9b1535a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b100d32b0 con 0x7f9b100d2e90 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9b100d0300 con 0x7f9b10161a60 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9b100cfab0 con 0x7f9b100cf620 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9b10161a60 0x7f9b100d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9b10161a60 0x7f9b100d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52894/0 (socket says 172.21.15.76:52894) 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 -- 172.21.15.76:0/3161033462 learned_addr learned my addr 172.21.15.76:0/3161033462 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0f7fe640 1 --2- 172.21.15.76:0/3161033462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b1006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 -- 172.21.15.76:0/3161033462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 msgr2=0x7f9b1006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 --2- 172.21.15.76:0/3161033462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b1006d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 -- 172.21.15.76:0/3161033462 >> v1:172.21.15.116:6789/0 conn(0x7f9b100d2e90 legacy=0x7f9b100cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 -- 172.21.15.76:0/3161033462 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b100b0f60 con 0x7f9b10161a60 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0effd640 1 --2- 172.21.15.76:0/3161033462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b10161a60 0x7f9b100d2950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009e50 tx=0x7f9b0801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c72516e58b8cced server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0dffb640 1 -- 172.21.15.76:0/3161033462 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080a5830 con 0x7f9b10161a60 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0dffb640 1 -- 172.21.15.76:0/3161033462 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b080a59d0 con 0x7f9b10161a60 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b0dffb640 1 -- 172.21.15.76:0/3161033462 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b080a5da0 con 0x7f9b10161a60 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 -- 172.21.15.76:0/3161033462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b10161a60 msgr2=0x7f9b100d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 --2- 172.21.15.76:0/3161033462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b10161a60 0x7f9b100d2950 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9b08009e50 tx=0x7f9b0801c970 comp rx=0 tx=0).stop 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 -- 172.21.15.76:0/3161033462 shutdown_connections 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 --2- 172.21.15.76:0/3161033462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b1006d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 --2- 172.21.15.76:0/3161033462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b10161a60 0x7f9b100d2950 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 -- 172.21.15.76:0/3161033462 >> 172.21.15.76:0/3161033462 conn(0x7f9b1008bf90 msgr2=0x7f9b1009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 -- 172.21.15.76:0/3161033462 shutdown_connections 2024-09-19T04:14:05.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.437+0000 7f9b1535a640 1 -- 172.21.15.76:0/3161033462 wait complete. 2024-09-19T04:14:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 Processor -- start 2024-09-19T04:14:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 -- start start 2024-09-19T04:14:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b100c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b100d2e90 0x7f9b10158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b10161a60 0x7f9b101589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1006e400 con 0x7f9b100d2e90 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1006d190 con 0x7f9b10161a60 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1006e580 con 0x7f9b100cf620 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b100d2e90 0x7f9b10158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b100d2e90 0x7f9b10158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52900/0 (socket says 172.21.15.76:52900) 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 -- 172.21.15.76:0/2699334030 learned_addr learned my addr 172.21.15.76:0/2699334030 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0effd640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b100c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0f7fe640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b10161a60 0x7f9b101589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 -- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 msgr2=0x7f9b100c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b100c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 -- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b10161a60 msgr2=0x7f9b101589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b10161a60 0x7f9b101589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 -- 172.21.15.76:0/2699334030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b100b9eb0 con 0x7f9b100d2e90 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0effd640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b100c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:05.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0f7fe640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b10161a60 0x7f9b101589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b0e7fc640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b100d2e90 0x7f9b10158480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000c3a0 tx=0x7f9b0000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b077fe640 1 -- 172.21.15.76:0/2699334030 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b000052d0 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b077fe640 1 -- 172.21.15.76:0/2699334030 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b00005470 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b100ca2f0 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b077fe640 1 -- 172.21.15.76:0/2699334030 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b00007260 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b100ca8b0 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.441+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1006d6b0 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.445+0000 7f9b077fe640 1 -- 172.21.15.76:0/2699334030 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9b00007400 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.445+0000 7f9b077fe640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9af00678c0 0x7f9af0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.445+0000 7f9b077fe640 1 -- 172.21.15.76:0/2699334030 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9b00085900 con 0x7f9b100d2e90 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.445+0000 7f9b0effd640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9af00678c0 0x7f9af0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.445+0000 7f9b0effd640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9af00678c0 0x7f9af0069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9b080ae000 tx=0x7f9b080a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.449+0000 7f9b077fe640 1 -- 172.21.15.76:0/2699334030 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9b00056170 con 0x7f9b100d2e90 2024-09-19T04:14:05.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.597+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b100ce420 con 0x7f9b100d2e90 2024-09-19T04:14:05.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b077fe640 1 -- 172.21.15.76:0/2699334030 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18622 (secure 0 0 0) 0x7f9b00055a70 con 0x7f9b100d2e90 2024-09-19T04:14:05.601 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:05.601 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":25,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:13:29.325925+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:14:05.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9af00678c0 msgr2=0x7f9af0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9af00678c0 0x7f9af0069d80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9b080ae000 tx=0x7f9b080a7070 comp rx=0 tx=0).stop 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b100d2e90 msgr2=0x7f9b10158480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b100d2e90 0x7f9b10158480 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000c3a0 tx=0x7f9b0000c860 comp rx=0 tx=0).stop 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 shutdown_connections 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9af00678c0 0x7f9af0069d80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b10161a60 0x7f9b101589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b100d2e90 0x7f9b10158480 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 --2- 172.21.15.76:0/2699334030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b100cf620 0x7f9b100c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 >> 172.21.15.76:0/2699334030 conn(0x7f9b1008bf90 msgr2=0x7f9b10075130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 shutdown_connections 2024-09-19T04:14:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.601+0000 7f9b1535a640 1 -- 172.21.15.76:0/2699334030 wait complete. 2024-09-19T04:14:05.619 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:14:05.619 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:05.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.737+0000 7f9910cd0640 1 Processor -- start 2024-09-19T04:14:05.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.737+0000 7f9910cd0640 1 -- start start 2024-09-19T04:14:05.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9910cd0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f990c15c2a0 0x7f990c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9910cd0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f990c0af2a0 0x7f990c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9910cd0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f990c0b3290 con 0x7f990c15bec0 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9910cd0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f990c0b3570 con 0x7f990c0af2a0 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9910cd0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f990c0b3850 con 0x7f990c15c2a0 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f990c0af2a0 0x7f990c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f990c0af2a0 0x7f990c06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52908/0 (socket says 172.21.15.76:52908) 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 -- 172.21.15.76:0/1627348486 learned_addr learned my addr 172.21.15.76:0/1627348486 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9909d9b640 1 --2- 172.21.15.76:0/1627348486 >> v2:172.21.15.116:3300/0 conn(0x7f990c15c2a0 0x7f990c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 -- 172.21.15.76:0/1627348486 >> v2:172.21.15.116:3300/0 conn(0x7f990c15c2a0 msgr2=0x7f990c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 --2- 172.21.15.76:0/1627348486 >> v2:172.21.15.116:3300/0 conn(0x7f990c15c2a0 0x7f990c0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 -- 172.21.15.76:0/1627348486 >> v1:172.21.15.116:6789/0 conn(0x7f990c15bec0 legacy=0x7f990c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:05.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 -- 172.21.15.76:0/1627348486 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c0b3b30 con 0x7f990c0af2a0 2024-09-19T04:14:05.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990ad9d640 1 --2- 172.21.15.76:0/1627348486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c0af2a0 0x7f990c06e000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f990c0ae510 tx=0x7f990401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5778c3a095050e37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990959a640 1 -- 172.21.15.76:0/1627348486 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040a5840 con 0x7f990c0af2a0 2024-09-19T04:14:05.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990959a640 1 -- 172.21.15.76:0/1627348486 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99040a59e0 con 0x7f990c0af2a0 2024-09-19T04:14:05.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f990959a640 1 -- 172.21.15.76:0/1627348486 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040a5db0 con 0x7f990c0af2a0 2024-09-19T04:14:05.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9910cd0640 1 -- 172.21.15.76:0/1627348486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c0af2a0 msgr2=0x7f990c06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.741+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1627348486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c0af2a0 0x7f990c06e000 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f990c0ae510 tx=0x7f990401c980 comp rx=0 tx=0).stop 2024-09-19T04:14:05.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- 172.21.15.76:0/1627348486 shutdown_connections 2024-09-19T04:14:05.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1627348486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c0af2a0 0x7f990c06e000 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1627348486 >> v2:172.21.15.116:3300/0 conn(0x7f990c15c2a0 0x7f990c0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- 172.21.15.76:0/1627348486 >> 172.21.15.76:0/1627348486 conn(0x7f990c0b1f90 msgr2=0x7f990c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:05.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- 172.21.15.76:0/1627348486 shutdown_connections 2024-09-19T04:14:05.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- 172.21.15.76:0/1627348486 wait complete. 2024-09-19T04:14:05.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 Processor -- start 2024-09-19T04:14:05.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- start start 2024-09-19T04:14:05.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f990c0af2a0 0x7f990c1640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f990c15bec0 0x7f990c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c15c2a0 0x7f990c154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f990c06ed50 con 0x7f990c15c2a0 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f990c0af710 con 0x7f990c0af2a0 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f990c06eed0 con 0x7f990c15bec0 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c15c2a0 0x7f990c154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c15c2a0 0x7f990c154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52914/0 (socket says 172.21.15.76:52914) 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 -- 172.21.15.76:0/1858483393 learned_addr learned my addr 172.21.15.76:0/1858483393 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 -- 172.21.15.76:0/1858483393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f990c15bec0 msgr2=0x7f990c156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f990c15bec0 0x7f990c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 -- 172.21.15.76:0/1858483393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f990c0af2a0 msgr2=0x7f990c1640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f990c0af2a0 0x7f990c1640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 -- 172.21.15.76:0/1858483393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f990c14a4f0 con 0x7f990c15c2a0 2024-09-19T04:14:05.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f990ad9d640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c15c2a0 0x7f990c154f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f990401a5b0 tx=0x7f9904019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9902ffd640 1 -- 172.21.15.76:0/1858483393 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040b8670 con 0x7f990c15c2a0 2024-09-19T04:14:05.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9902ffd640 1 -- 172.21.15.76:0/1858483393 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99040be070 con 0x7f990c15c2a0 2024-09-19T04:14:05.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f990c153bb0 con 0x7f990c15c2a0 2024-09-19T04:14:05.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9902ffd640 1 -- 172.21.15.76:0/1858483393 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040b8a50 con 0x7f990c15c2a0 2024-09-19T04:14:05.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.745+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f990c154170 con 0x7f990c15c2a0 2024-09-19T04:14:05.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.749+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f990c06f260 con 0x7f990c15c2a0 2024-09-19T04:14:05.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.749+0000 7f9902ffd640 1 -- 172.21.15.76:0/1858483393 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f99040b8bf0 con 0x7f990c15c2a0 2024-09-19T04:14:05.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.749+0000 7f9902ffd640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98ec0678c0 0x7f98ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:05.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.749+0000 7f9902ffd640 1 -- 172.21.15.76:0/1858483393 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9904121260 con 0x7f990c15c2a0 2024-09-19T04:14:05.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.749+0000 7f990a59c640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98ec0678c0 0x7f98ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:05.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.749+0000 7f990a59c640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98ec0678c0 0x7f98ec069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f990c0740f0 tx=0x7f98fc0079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:05.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.753+0000 7f9902ffd640 1 -- 172.21.15.76:0/1858483393 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f990c14a4f0 con 0x7f990c15c2a0 2024-09-19T04:14:05.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.961+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f990c06fcd0 con 0x7f990c15c2a0 2024-09-19T04:14:05.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.961+0000 7f9902ffd640 1 -- 172.21.15.76:0/1858483393 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7f99040165a0 con 0x7f990c15c2a0 2024-09-19T04:14:05.962 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:05.962 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:14:05.963 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98ec0678c0 msgr2=0x7f98ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98ec0678c0 0x7f98ec069d80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f990c0740f0 tx=0x7f98fc0079c0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c15c2a0 msgr2=0x7f990c154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c15c2a0 0x7f990c154f30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f990401a5b0 tx=0x7f9904019720 comp rx=0 tx=0).stop 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 shutdown_connections 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98ec0678c0 0x7f98ec069d80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f990c15c2a0 0x7f990c154f30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f990c15bec0 0x7f990c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 --2- 172.21.15.76:0/1858483393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f990c0af2a0 0x7f990c1640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:05.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 >> 172.21.15.76:0/1858483393 conn(0x7f990c0b1f90 msgr2=0x7f990c0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:05.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 shutdown_connections 2024-09-19T04:14:05.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:05.965+0000 7f9910cd0640 1 -- 172.21.15.76:0/1858483393 wait complete. 2024-09-19T04:14:05.982 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:06.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.097+0000 7fdabb2b5640 1 Processor -- start 2024-09-19T04:14:06.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdabb2b5640 1 -- start start 2024-09-19T04:14:06.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdabb2b5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab4070020 0x7fdab415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdabb2b5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdab415f1d0 0x7fdab40adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdabb2b5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdab40aede0 con 0x7fdab41520a0 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdabb2b5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fdab4152480 con 0x7fdab415f1d0 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdabb2b5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdab415fa10 con 0x7fdab4070020 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab9051640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdab41520a0 0x7fdab406f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:35346/0 (socket says 172.21.15.76:35346) 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab9051640 1 -- 172.21.15.76:0/2135047733 learned_addr learned my addr 172.21.15.76:0/2135047733 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab8850640 1 --2- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab4070020 0x7fdab415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab9852640 1 --2- 172.21.15.76:0/2135047733 >> v2:172.21.15.116:3300/0 conn(0x7fdab415f1d0 0x7fdab40adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdaabfff640 1 -- 172.21.15.76:0/2135047733 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462861724 0 0) 0x7fdab40aede0 con 0x7fdab41520a0 2024-09-19T04:14:06.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdaabfff640 1 -- 172.21.15.76:0/2135047733 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaa0003530 con 0x7fdab41520a0 2024-09-19T04:14:06.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab9852640 1 -- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab4070020 msgr2=0x7fdab415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:06.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab9852640 1 --2- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab4070020 0x7fdab415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab9852640 1 -- 172.21.15.76:0/2135047733 >> v1:172.21.15.116:6789/0 conn(0x7fdab41520a0 legacy=0x7fdab406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.101+0000 7fdab9852640 1 -- 172.21.15.76:0/2135047733 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab4074250 con 0x7fdab415f1d0 2024-09-19T04:14:06.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9852640 1 --2- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab415f1d0 0x7fdab40adc80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009dc0 tx=0x7fdaac01c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fe4e6e1ba5280d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdaabfff640 1 -- 172.21.15.76:0/2135047733 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0b2070 con 0x7fdab415f1d0 2024-09-19T04:14:06.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdaabfff640 1 -- 172.21.15.76:0/2135047733 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac0a59e0 con 0x7fdab415f1d0 2024-09-19T04:14:06.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdaabfff640 1 -- 172.21.15.76:0/2135047733 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0a5d90 con 0x7fdab415f1d0 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab415f1d0 msgr2=0x7fdab40adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab415f1d0 0x7fdab40adc80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdaac009dc0 tx=0x7fdaac01c920 comp rx=0 tx=0).stop 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- 172.21.15.76:0/2135047733 shutdown_connections 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab415f1d0 0x7fdab40adc80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/2135047733 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab4070020 0x7fdab415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- 172.21.15.76:0/2135047733 >> 172.21.15.76:0/2135047733 conn(0x7fdab40b1f90 msgr2=0x7fdab40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- 172.21.15.76:0/2135047733 shutdown_connections 2024-09-19T04:14:06.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- 172.21.15.76:0/2135047733 wait complete. 2024-09-19T04:14:06.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 Processor -- start 2024-09-19T04:14:06.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- start start 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab4070020 0x7fdab4165550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab41520a0 0x7fdab409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdab415f1d0 0x7fdab413d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdab415f680 con 0x7fdab415f1d0 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdab4073d30 con 0x7fdab4070020 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdabb2b5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdab4167f10 con 0x7fdab41520a0 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9852640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdab415f1d0 0x7fdab413d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9852640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdab415f1d0 0x7fdab413d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52924/0 (socket says 172.21.15.76:52924) 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9852640 1 -- 172.21.15.76:0/3520127394 learned_addr learned my addr 172.21.15.76:0/3520127394 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9051640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab4070020 0x7fdab4165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab8850640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab41520a0 0x7fdab409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9852640 1 -- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab41520a0 msgr2=0x7fdab409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9852640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab41520a0 0x7fdab409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.105+0000 7fdab9852640 1 -- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab4070020 msgr2=0x7fdab4165550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdab9852640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab4070020 0x7fdab4165550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdab9852640 1 -- 172.21.15.76:0/3520127394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab409b590 con 0x7fdab415f1d0 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdab9852640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdab415f1d0 0x7fdab413d380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdaac01a550 tx=0x7fdaac0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdaa9ffb640 1 -- 172.21.15.76:0/3520127394 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac0b2050 con 0x7fdab415f1d0 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdaa9ffb640 1 -- 172.21.15.76:0/3520127394 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaac0be070 con 0x7fdab415f1d0 2024-09-19T04:14:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdaa9ffb640 1 -- 172.21.15.76:0/3520127394 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaac004040 con 0x7fdab415f1d0 2024-09-19T04:14:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab409b870 con 0x7fdab415f1d0 2024-09-19T04:14:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab409be30 con 0x7fdab415f1d0 2024-09-19T04:14:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab415ec90 con 0x7fdab415f1d0 2024-09-19T04:14:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdaa9ffb640 1 -- 172.21.15.76:0/3520127394 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdaac0041e0 con 0x7fdab415f1d0 2024-09-19T04:14:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdaa9ffb640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda980678c0 0x7fda98069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdaa9ffb640 1 -- 172.21.15.76:0/3520127394 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7fdaac121190 con 0x7fdab415f1d0 2024-09-19T04:14:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdab9051640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda980678c0 0x7fda98069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.109+0000 7fdab9051640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda980678c0 0x7fda98069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4007fd0 tx=0x7fdaa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.113+0000 7fdaa9ffb640 1 -- 172.21.15.76:0/3520127394 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdaac016540 con 0x7fdab415f1d0 2024-09-19T04:14:06.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.293+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab406ced0 con 0x7fdab415f1d0 2024-09-19T04:14:06.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.293+0000 7fdaa9ffb640 1 -- 172.21.15.76:0/3520127394 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5219 (secure 0 0 0) 0x7fdaac0f18a0 con 0x7fdab415f1d0 2024-09-19T04:14:06.296 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:06.296 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":8,"btime":"2024-09-19T04:13:39:001660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/335713697","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":335713697},{"type":"v1","addr":"172.21.15.116:6833","nonce":335713697}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1416862326","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1416862326},{"type":"v1","addr":"172.21.15.116:6835","nonce":1416862326}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6836/963703872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6835","nonce":963703872},{"type":"v1","addr":"172.21.15.76:6836","nonce":963703872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:13:24.253935+0000","modified":"2024-09-19T04:13:38.364534+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4260},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4260":{"gid":4260,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3901169819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3901169819},{"type":"v1","addr":"172.21.15.76:6837","nonce":3901169819}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4260,"qdb_cluster":[4260]},"id":1}]} 2024-09-19T04:14:06.297 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 8 2024-09-19T04:14:06.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda980678c0 msgr2=0x7fda98069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda980678c0 0x7fda98069d80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4007fd0 tx=0x7fdaa4009040 comp rx=0 tx=0).stop 2024-09-19T04:14:06.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdab415f1d0 msgr2=0x7fdab413d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdab415f1d0 0x7fdab413d380 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdaac01a550 tx=0x7fdaac0196c0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 shutdown_connections 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda980678c0 0x7fda98069d80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdab415f1d0 0x7fdab413d380 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdab41520a0 0x7fdab409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 --2- 172.21.15.76:0/3520127394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdab4070020 0x7fdab4165550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 >> 172.21.15.76:0/3520127394 conn(0x7fdab40b1f90 msgr2=0x7fdab406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 shutdown_connections 2024-09-19T04:14:06.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.297+0000 7fdabb2b5640 1 -- 172.21.15.76:0/3520127394 wait complete. 2024-09-19T04:14:06.313 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:14:06.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.421+0000 7f716b3df640 1 Processor -- start 2024-09-19T04:14:06.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.421+0000 7f716b3df640 1 -- start start 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f716415c2a0 0x7f71640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71640b3290 con 0x7f716415bec0 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71640b3570 con 0x7f71640af2a0 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f71640b3850 con 0x7f716415c2a0 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716897a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f716415c2a0 0x7f71640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716917b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f716415bec0 0x7f71640ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47586/0 (socket says 172.21.15.76:47586) 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716917b640 1 -- 172.21.15.76:0/3968828974 learned_addr learned my addr 172.21.15.76:0/3968828974 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716897a640 1 -- 172.21.15.76:0/3968828974 >> v1:172.21.15.116:6789/0 conn(0x7f71640af2a0 legacy=0x7f716406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716897a640 1 -- 172.21.15.76:0/3968828974 >> v1:172.21.15.76:6789/0 conn(0x7f716415bec0 legacy=0x7f71640ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716897a640 1 -- 172.21.15.76:0/3968828974 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71640b3b30 con 0x7f716415c2a0 2024-09-19T04:14:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f715bfff640 1 -- 172.21.15.76:0/3968828974 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2201337754 0 0) 0x7f71640b3290 con 0x7f716415bec0 2024-09-19T04:14:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716897a640 1 --2- 172.21.15.76:0/3968828974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f71640aed60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71540097c0 tx=0x7f715401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5804a284508228af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f715bfff640 1 -- 172.21.15.76:0/3968828974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540ab430 con 0x7f716415c2a0 2024-09-19T04:14:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f715bfff640 1 -- 172.21.15.76:0/3968828974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71540b0070 con 0x7f716415c2a0 2024-09-19T04:14:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f715bfff640 1 -- 172.21.15.76:0/3968828974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540ab7f0 con 0x7f716415c2a0 2024-09-19T04:14:06.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- 172.21.15.76:0/3968828974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 msgr2=0x7f71640aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 --2- 172.21.15.76:0/3968828974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f71640aed60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71540097c0 tx=0x7f715401c430 comp rx=0 tx=0).stop 2024-09-19T04:14:06.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- 172.21.15.76:0/3968828974 shutdown_connections 2024-09-19T04:14:06.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 --2- 172.21.15.76:0/3968828974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f71640aed60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- 172.21.15.76:0/3968828974 >> 172.21.15.76:0/3968828974 conn(0x7f71640b1f90 msgr2=0x7f71640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- 172.21.15.76:0/3968828974 shutdown_connections 2024-09-19T04:14:06.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.425+0000 7f716b3df640 1 -- 172.21.15.76:0/3968828974 wait complete. 2024-09-19T04:14:06.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 Processor -- start 2024-09-19T04:14:06.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 -- start start 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f71640af2a0 0x7f716415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716415bec0 0x7f71641550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f716414d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f716406ef20 con 0x7f716415c2a0 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f71640af710 con 0x7f716415bec0 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f716406f0a0 con 0x7f71640af2a0 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f716414d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f716414d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52944/0 (socket says 172.21.15.76:52944) 2024-09-19T04:14:06.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 -- 172.21.15.76:0/3321229494 learned_addr learned my addr 172.21.15.76:0/3321229494 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716917b640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f71640af2a0 0x7f716415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716897a640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716415bec0 0x7f71641550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 -- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f71640af2a0 msgr2=0x7f716415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f71640af2a0 0x7f716415c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 -- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716415bec0 msgr2=0x7f71641550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716415bec0 0x7f71641550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 -- 172.21.15.76:0/3321229494 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f716414c1d0 con 0x7f716415c2a0 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716917b640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f71640af2a0 0x7f716415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716897a640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716415bec0 0x7f71641550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716997c640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f716414d250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c780 tx=0x7f715c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f7159ffb640 1 -- 172.21.15.76:0/3321229494 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c004b70 con 0x7f716415c2a0 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f7159ffb640 1 -- 172.21.15.76:0/3321229494 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715c004d10 con 0x7f716415c2a0 2024-09-19T04:14:06.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f7159ffb640 1 -- 172.21.15.76:0/3321229494 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715c00aa90 con 0x7f716415c2a0 2024-09-19T04:14:06.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f716414c4b0 con 0x7f716415c2a0 2024-09-19T04:14:06.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f716414ca70 con 0x7f716415c2a0 2024-09-19T04:14:06.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71640aed60 con 0x7f716415c2a0 2024-09-19T04:14:06.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.429+0000 7f7159ffb640 1 -- 172.21.15.76:0/3321229494 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f715c00ac30 con 0x7f716415c2a0 2024-09-19T04:14:06.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.433+0000 7f7159ffb640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f71480678c0 0x7f7148069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.433+0000 7f716917b640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f71480678c0 0x7f7148069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.433+0000 7f7159ffb640 1 -- 172.21.15.76:0/3321229494 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f715c086320 con 0x7f716415c2a0 2024-09-19T04:14:06.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.433+0000 7f716917b640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f71480678c0 0x7f7148069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f70 tx=0x7f7160009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.437+0000 7f7159ffb640 1 -- 172.21.15.76:0/3321229494 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f715c020070 con 0x7f716415c2a0 2024-09-19T04:14:06.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.597+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f716406fcd0 con 0x7f716415c2a0 2024-09-19T04:14:06.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.601+0000 7f7159ffb640 1 -- 172.21.15.76:0/3321229494 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) ==== 74+0+19059 (secure 0 0 0) 0x7f715c07de20 con 0x7f716415c2a0 2024-09-19T04:14:06.603 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:06.603 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":26,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:14:05.829583+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:13:22.367404+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T04:13:23.326658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-19T04:13:26.974324+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.14","mappings":[{"from":4,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1c","mappings":[{"from":5,"to":2}]},{"pgid":"3.23","mappings":[{"from":3,"to":1}]},{"pgid":"3.38","mappings":[{"from":5,"to":1}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]},{"pgid":"4.1f","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:14:06.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f71480678c0 msgr2=0x7f7148069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f71480678c0 0x7f7148069d80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7160000f70 tx=0x7f7160009040 comp rx=0 tx=0).stop 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 msgr2=0x7f716414d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f716414d250 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f715c00c780 tx=0x7f715c00cc40 comp rx=0 tx=0).stop 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 shutdown_connections 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f71480678c0 0x7f7148069d80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f716415c2a0 0x7f716414d250 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f716415bec0 0x7f71641550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 --2- 172.21.15.76:0/3321229494 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f71640af2a0 0x7f716415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 >> 172.21.15.76:0/3321229494 conn(0x7f71640b1f90 msgr2=0x7f71640b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 shutdown_connections 2024-09-19T04:14:06.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.605+0000 7f716b3df640 1 -- 172.21.15.76:0/3321229494 wait complete. 2024-09-19T04:14:06.623 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T04:14:06.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b2ba5640 1 Processor -- start 2024-09-19T04:14:06.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b2ba5640 1 -- start start 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b2ba5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63ac1520a0 0x7f63ac06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b2ba5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac06fe50 0x7f63ac15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b2ba5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ac070250 con 0x7f63ac15f1d0 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b2ba5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f63ac152480 con 0x7f63ac1520a0 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b2ba5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac0aede0 con 0x7f63ac06fe50 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b0941640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f63ac1520a0 0x7f63ac06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b1142640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f63ac15f1d0 0x7f63ac071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47602/0 (socket says 172.21.15.76:47602) 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b1142640 1 -- 172.21.15.76:0/2500844058 learned_addr learned my addr 172.21.15.76:0/2500844058 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63abfff640 1 --2- 172.21.15.76:0/2500844058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac06fe50 0x7f63ac15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b0941640 1 -- 172.21.15.76:0/2500844058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac06fe50 msgr2=0x7f63ac15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:06.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b0941640 1 --2- 172.21.15.76:0/2500844058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac06fe50 0x7f63ac15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b0941640 1 -- 172.21.15.76:0/2500844058 >> v1:172.21.15.76:6789/0 conn(0x7f63ac15f1d0 legacy=0x7f63ac071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63ab7fe640 1 -- 172.21.15.76:0/2500844058 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1343977310 0 0) 0x7f63ac070250 con 0x7f63ac15f1d0 2024-09-19T04:14:06.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.741+0000 7f63b0941640 1 -- 172.21.15.76:0/2500844058 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac15fa10 con 0x7f63ac1520a0 2024-09-19T04:14:06.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b0941640 1 --2- 172.21.15.76:0/2500844058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac1520a0 0x7f63ac06f910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f639c0097c0 tx=0x7f639c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8264332cf8b5da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63ab7fe640 1 -- 172.21.15.76:0/2500844058 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c0b1070 con 0x7f63ac1520a0 2024-09-19T04:14:06.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63ab7fe640 1 -- 172.21.15.76:0/2500844058 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f639c0ac600 con 0x7f63ac1520a0 2024-09-19T04:14:06.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63ab7fe640 1 -- 172.21.15.76:0/2500844058 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639c0ac9d0 con 0x7f63ac1520a0 2024-09-19T04:14:06.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- 172.21.15.76:0/2500844058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac1520a0 msgr2=0x7f63ac06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/2500844058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac1520a0 0x7f63ac06f910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f639c0097c0 tx=0x7f639c01c480 comp rx=0 tx=0).stop 2024-09-19T04:14:06.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- 172.21.15.76:0/2500844058 shutdown_connections 2024-09-19T04:14:06.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/2500844058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac06fe50 0x7f63ac15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/2500844058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac1520a0 0x7f63ac06f910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- 172.21.15.76:0/2500844058 >> 172.21.15.76:0/2500844058 conn(0x7f63ac0b1f90 msgr2=0x7f63ac0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- 172.21.15.76:0/2500844058 shutdown_connections 2024-09-19T04:14:06.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- 172.21.15.76:0/2500844058 wait complete. 2024-09-19T04:14:06.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 Processor -- start 2024-09-19T04:14:06.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- start start 2024-09-19T04:14:06.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ac06fe50 0x7f63ac151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac1520a0 0x7f63ac1f71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac15f1d0 0x7f63ac146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac073d30 con 0x7f63ac15f1d0 2024-09-19T04:14:06.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63ac073a50 con 0x7f63ac06fe50 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b2ba5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f63ac15f640 con 0x7f63ac1520a0 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b1142640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac15f1d0 0x7f63ac146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b1142640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac15f1d0 0x7f63ac146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52966/0 (socket says 172.21.15.76:52966) 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.745+0000 7f63b1142640 1 -- 172.21.15.76:0/1489094625 learned_addr learned my addr 172.21.15.76:0/1489094625 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b0941640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ac06fe50 0x7f63ac151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63abfff640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac1520a0 0x7f63ac1f71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b1142640 1 -- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac1520a0 msgr2=0x7f63ac1f71e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b1142640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac1520a0 0x7f63ac1f71e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b1142640 1 -- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ac06fe50 msgr2=0x7f63ac151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b1142640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ac06fe50 0x7f63ac151b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b1142640 1 -- 172.21.15.76:0/1489094625 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ac165b90 con 0x7f63ac15f1d0 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b0941640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ac06fe50 0x7f63ac151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:06.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63abfff640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac1520a0 0x7f63ac1f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b1142640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac15f1d0 0x7f63ac146400 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c7a0 tx=0x7f63a400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63a97fa640 1 -- 172.21.15.76:0/1489094625 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a4018070 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63a97fa640 1 -- 172.21.15.76:0/1489094625 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63a400ce50 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ac145920 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63a97fa640 1 -- 172.21.15.76:0/1489094625 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a400a820 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ac146940 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ac15ec90 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63a97fa640 1 -- 172.21.15.76:0/1489094625 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f63a401b070 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63a97fa640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63980678c0 0x7f6398069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63a97fa640 1 -- 172.21.15.76:0/1489094625 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6764+0+0 (secure 0 0 0) 0x7f63a40855a0 con 0x7f63ac15f1d0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b0941640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63980678c0 0x7f6398069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:06.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.749+0000 7f63b0941640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63980678c0 0x7f6398069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f639c0ae000 tx=0x7f639c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:06.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.753+0000 7f63a97fa640 1 -- 172.21.15.76:0/1489094625 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f63a4055d50 con 0x7f63ac15f1d0 2024-09-19T04:14:06.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.961+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f63ac0721f0 con 0x7f63ac15f1d0 2024-09-19T04:14:06.962 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:14:06.961+0000 7fa674384640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:14:06.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63a97fa640 1 -- 172.21.15.76:0/1489094625 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f63a4055650 con 0x7f63ac15f1d0 2024-09-19T04:14:06.964 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:14:06.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63980678c0 msgr2=0x7f6398069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63980678c0 0x7f6398069d80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f639c0ae000 tx=0x7f639c008040 comp rx=0 tx=0).stop 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac15f1d0 msgr2=0x7f63ac146400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac15f1d0 0x7f63ac146400 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f63a400c7a0 tx=0x7f63a400cc60 comp rx=0 tx=0).stop 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 shutdown_connections 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63980678c0 0x7f6398069d80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63ac15f1d0 0x7f63ac146400 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63ac1520a0 0x7f63ac1f71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 --2- 172.21.15.76:0/1489094625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63ac06fe50 0x7f63ac151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 >> 172.21.15.76:0/1489094625 conn(0x7f63ac0b1f90 msgr2=0x7f63ac0b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 shutdown_connections 2024-09-19T04:14:06.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:06.965+0000 7f63b2ba5640 1 -- 172.21.15.76:0/1489094625 wait complete. 2024-09-19T04:14:06.976 INFO:tasks.ceph.mds.a.smithi076.stderr: -3355> 2024-09-19T04:13:29.075+0000 7f268488d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:14:06.984 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:14:07.026 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:14:07.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841f911640 1 Processor -- start 2024-09-19T04:14:07.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841f911640 1 -- start start 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841f911640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 0x7f84181634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841f911640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841806e470 con 0x7f84181630c0 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841f911640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f841806e750 con 0x7f84180b3290 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841f911640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f841806fc20 con 0x7f841806df70 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841d6ad640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f84181630c0 0x7f841806f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47622/0 (socket says 172.21.15.76:47622) 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841d6ad640 1 -- 172.21.15.76:0/1574569123 learned_addr learned my addr 172.21.15.76:0/1574569123 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f841ceac640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 0x7f84181634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 278165125 0 0) 0x7f841806e470 con 0x7f84181630c0 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.101+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8404003530 con 0x7f84181630c0 2024-09-19T04:14:07.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1849527836 0 0) 0x7f8404003530 con 0x7f84181630c0 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 msgr2=0x7f84181634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 0x7f84181634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 >> v1:172.21.15.116:6789/0 conn(0x7f84180b3290 legacy=0x7f84180ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841806ea30 con 0x7f84181630c0 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f8408001680 con 0x7f84181630c0 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 >> v1:172.21.15.76:6789/0 conn(0x7f84181630c0 legacy=0x7f841806f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 0x7f84040039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8404003f70 0x7f8404004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8404004d50 0x7f84040051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f841806fc20 con 0x7f8404004d50 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f840400bab0 con 0x7f8404003f70 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f840400bcb0 con 0x7f841806df70 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8404004d50 0x7f84040051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841ceac640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8404003f70 0x7f8404004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841d6ad640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 0x7f84040039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 -- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 msgr2=0x7f84040039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 0x7f84040039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 -- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8404003f70 msgr2=0x7f8404004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8404003f70 0x7f8404004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 -- 172.21.15.76:0/1574569123 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8404003530 con 0x7f8404004d50 2024-09-19T04:14:07.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841d6ad640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f841806df70 0x7f84040039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:07.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8404004d50 0x7f84040051d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8410001e70 tx=0x7f841001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3271a983be9a4333 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:07.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410001a90 con 0x7f8404004d50 2024-09-19T04:14:07.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8410003b80 con 0x7f8404004d50 2024-09-19T04:14:07.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f840ffff640 1 -- 172.21.15.76:0/1574569123 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410005080 con 0x7f8404004d50 2024-09-19T04:14:07.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841f911640 1 -- 172.21.15.76:0/1574569123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8404004d50 msgr2=0x7f84040051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841f911640 1 --2- 172.21.15.76:0/1574569123 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8404004d50 0x7f84040051d0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8410001e70 tx=0x7f841001c780 comp rx=0 tx=0).stop 2024-09-19T04:14:07.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841deae640 1 -- 172.21.15.76:0/1574569123 reap_dead start 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841f911640 1 -- 172.21.15.76:0/1574569123 shutdown_connections 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841f911640 1 -- 172.21.15.76:0/1574569123 >> 172.21.15.76:0/1574569123 conn(0x7f84180b1f90 msgr2=0x7f84180b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841f911640 1 -- 172.21.15.76:0/1574569123 shutdown_connections 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.105+0000 7f841f911640 1 -- 172.21.15.76:0/1574569123 wait complete. 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 Processor -- start 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 -- start start 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8404004d50 0x7f8418155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84180adde0 0x7f84180ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84180ae7a0 0x7f84180cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84180ad910 con 0x7f84180adde0 2024-09-19T04:14:07.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f841806eff0 con 0x7f84180ae7a0 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f84180ada90 con 0x7f8404004d50 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84180adde0 0x7f84180ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84180adde0 0x7f84180ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52992/0 (socket says 172.21.15.76:52992) 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 -- 172.21.15.76:0/4228683015 learned_addr learned my addr 172.21.15.76:0/4228683015 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841d6ad640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8404004d50 0x7f8418155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841deae640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84180ae7a0 0x7f84180cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 -- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8404004d50 msgr2=0x7f8418155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8404004d50 0x7f8418155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 -- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84180ae7a0 msgr2=0x7f84180cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84180ae7a0 0x7f84180cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 -- 172.21.15.76:0/4228683015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8418158fe0 con 0x7f84180adde0 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841d6ad640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8404004d50 0x7f8418155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841ceac640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84180adde0 0x7f84180ae260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f84140115e0 tx=0x7f8414011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f840dffb640 1 -- 172.21.15.76:0/4228683015 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841400f8f0 con 0x7f84180adde0 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f840dffb640 1 -- 172.21.15.76:0/4228683015 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f841400fa90 con 0x7f84180adde0 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84180ccc60 con 0x7f84180adde0 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f840dffb640 1 -- 172.21.15.76:0/4228683015 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841400fe40 con 0x7f84180adde0 2024-09-19T04:14:07.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84180cd220 con 0x7f84180adde0 2024-09-19T04:14:07.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841806e3d0 con 0x7f84180adde0 2024-09-19T04:14:07.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.109+0000 7f840dffb640 1 -- 172.21.15.76:0/4228683015 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8414015460 con 0x7f84180adde0 2024-09-19T04:14:07.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.113+0000 7f840dffb640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83fc0678f0 0x7f83fc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:07.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.113+0000 7f840dffb640 1 -- 172.21.15.76:0/4228683015 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6850+0+0 (secure 0 0 0) 0x7f8414086a40 con 0x7f84180adde0 2024-09-19T04:14:07.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.113+0000 7f841d6ad640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83fc0678f0 0x7f83fc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:07.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.113+0000 7f841d6ad640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83fc0678f0 0x7f83fc069db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f84080017f0 tx=0x7f8408009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:07.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.117+0000 7f840dffb640 1 -- 172.21.15.76:0/4228683015 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8414019030 con 0x7f84180adde0 2024-09-19T04:14:07.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.309+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8418070080 con 0x7f84180adde0 2024-09-19T04:14:07.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.965+0000 7f840dffb640 1 -- 172.21.15.76:0/4228683015 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f8414057190 con 0x7f84180adde0 2024-09-19T04:14:07.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83fc0678f0 msgr2=0x7f83fc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83fc0678f0 0x7f83fc069db0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f84080017f0 tx=0x7f8408009040 comp rx=0 tx=0).stop 2024-09-19T04:14:07.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84180adde0 msgr2=0x7f84180ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:07.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84180adde0 0x7f84180ae260 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f84140115e0 tx=0x7f8414011aa0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 shutdown_connections 2024-09-19T04:14:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83fc0678f0 0x7f83fc069db0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84180ae7a0 0x7f84180cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84180adde0 0x7f84180ae260 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 --2- 172.21.15.76:0/4228683015 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8404004d50 0x7f8418155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 >> 172.21.15.76:0/4228683015 conn(0x7f84180b1f90 msgr2=0x7f84180cfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 shutdown_connections 2024-09-19T04:14:07.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:07.969+0000 7f841f911640 1 -- 172.21.15.76:0/4228683015 wait complete. 2024-09-19T04:14:07.992 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:14:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22f5800640 1 Processor -- start 2024-09-19T04:14:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22f5800640 1 -- start start 2024-09-19T04:14:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22f5800640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f22f015bec0 0x7f22f00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22f5800640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22f5800640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22f00b3290 con 0x7f22f00af2a0 2024-09-19T04:14:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22f5800640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f22f00b3570 con 0x7f22f015bec0 2024-09-19T04:14:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22f5800640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f22f00b3850 con 0x7f22f015c2a0 2024-09-19T04:14:08.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22eeffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f22f015bec0 0x7f22f00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:08.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22eeffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f22f015bec0 0x7f22f00ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:39426/0 (socket says 172.21.15.76:39426) 2024-09-19T04:14:08.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22eeffd640 1 -- 172.21.15.76:0/3351834777 learned_addr learned my addr 172.21.15.76:0/3351834777 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:08.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22ee7fc640 1 --2- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:08.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22eeffd640 1 -- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 msgr2=0x7f22f00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:08.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22eeffd640 1 --2- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22eeffd640 1 -- 172.21.15.76:0/3351834777 >> v1:172.21.15.116:6789/0 conn(0x7f22f00af2a0 legacy=0x7f22f015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:08.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.165+0000 7f22eeffd640 1 -- 172.21.15.76:0/3351834777 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f00b3b30 con 0x7f22f015bec0 2024-09-19T04:14:08.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22eeffd640 1 --2- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 0x7f22f00ad200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009e50 tx=0x7f22e001ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbd5220bdc0ac954 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:08.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22edffb640 1 -- 172.21.15.76:0/3351834777 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00a58d0 con 0x7f22f015bec0 2024-09-19T04:14:08.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22edffb640 1 -- 172.21.15.76:0/3351834777 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e00a5a70 con 0x7f22f015bec0 2024-09-19T04:14:08.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22edffb640 1 -- 172.21.15.76:0/3351834777 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00a5e40 con 0x7f22f015bec0 2024-09-19T04:14:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 -- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 msgr2=0x7f22f00ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 --2- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 0x7f22f00ad200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009e50 tx=0x7f22e001ca00 comp rx=0 tx=0).stop 2024-09-19T04:14:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 -- 172.21.15.76:0/3351834777 shutdown_connections 2024-09-19T04:14:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 --2- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 --2- 172.21.15.76:0/3351834777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 0x7f22f00ad200 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 -- 172.21.15.76:0/3351834777 >> 172.21.15.76:0/3351834777 conn(0x7f22f00b1f90 msgr2=0x7f22f00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 -- 172.21.15.76:0/3351834777 shutdown_connections 2024-09-19T04:14:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 -- 172.21.15.76:0/3351834777 wait complete. 2024-09-19T04:14:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.169+0000 7f22f5800640 1 Processor -- start 2024-09-19T04:14:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 -- start start 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f00af2a0 0x7f22f015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 0x7f22f0150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f009d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00af6b0 con 0x7f22f00af2a0 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f22f00ad740 con 0x7f22f015bec0 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f22f0169000 con 0x7f22f015c2a0 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f00af2a0 0x7f22f015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f00af2a0 0x7f22f015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53004/0 (socket says 172.21.15.76:53004) 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 -- 172.21.15.76:0/1137988511 learned_addr learned my addr 172.21.15.76:0/1137988511 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22ee7fc640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 0x7f22f0150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22ef7fe640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f009d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 -- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 msgr2=0x7f22f009d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f009d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 -- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 msgr2=0x7f22f0150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 0x7f22f0150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 -- 172.21.15.76:0/1137988511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22f009da90 con 0x7f22f00af2a0 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22ef7fe640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f009d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:08.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22eeffd640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f00af2a0 0x7f22f015de60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22e001a630 tx=0x7f22e00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22d77fe640 1 -- 172.21.15.76:0/1137988511 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e00a58d0 con 0x7f22f00af2a0 2024-09-19T04:14:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22d77fe640 1 -- 172.21.15.76:0/1137988511 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22e00b8470 con 0x7f22f00af2a0 2024-09-19T04:14:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22f009dd70 con 0x7f22f00af2a0 2024-09-19T04:14:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22d77fe640 1 -- 172.21.15.76:0/1137988511 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22e0009510 con 0x7f22f00af2a0 2024-09-19T04:14:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22f009e330 con 0x7f22f00af2a0 2024-09-19T04:14:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22f00ad200 con 0x7f22f00af2a0 2024-09-19T04:14:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22d77fe640 1 -- 172.21.15.76:0/1137988511 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f22e00c0020 con 0x7f22f00af2a0 2024-09-19T04:14:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22d77fe640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f22dc0678c0 0x7f22dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22ee7fc640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f22dc0678c0 0x7f22dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.173+0000 7f22d77fe640 1 -- 172.21.15.76:0/1137988511 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6850+0+0 (secure 0 0 0) 0x7f22e0120cc0 con 0x7f22f00af2a0 2024-09-19T04:14:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.177+0000 7f22ee7fc640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f22dc0678c0 0x7f22dc069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22f01f8650 tx=0x7f22e4009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:08.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.181+0000 7f22d77fe640 1 -- 172.21.15.76:0/1137988511 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f22e0016620 con 0x7f22f00af2a0 2024-09-19T04:14:08.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.345+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22f006fcd0 con 0x7f22f00af2a0 2024-09-19T04:14:08.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.977+0000 7f22d77fe640 1 -- 172.21.15.76:0/1137988511 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7f22e00f1380 con 0x7f22f00af2a0 2024-09-19T04:14:08.977 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f22dc0678c0 msgr2=0x7f22dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f22dc0678c0 0x7f22dc069d80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22f01f8650 tx=0x7f22e4009190 comp rx=0 tx=0).stop 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f00af2a0 msgr2=0x7f22f015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f00af2a0 0x7f22f015de60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22e001a630 tx=0x7f22e00197a0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 shutdown_connections 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f22dc0678c0 0x7f22dc069d80 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f22f01f8650 tx=0x7f22e4009190 comp rx=0 tx=0).stop 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f22f015c2a0 0x7f22f009d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22f015bec0 0x7f22f0150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 --2- 172.21.15.76:0/1137988511 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f22f00af2a0 0x7f22f015de60 secure :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f22e001a630 tx=0x7f22e00197a0 comp rx=0 tx=0).stop 2024-09-19T04:14:08.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 >> 172.21.15.76:0/1137988511 conn(0x7f22f00b1f90 msgr2=0x7f22f00cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:08.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 shutdown_connections 2024-09-19T04:14:08.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:08.981+0000 7f22f5800640 1 -- 172.21.15.76:0/1137988511 wait complete. 2024-09-19T04:14:09.006 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:14:09.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.133+0000 7f4584428640 1 Processor -- start 2024-09-19T04:14:09.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.133+0000 7f4584428640 1 -- start start 2024-09-19T04:14:09.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.133+0000 7f4584428640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.133+0000 7f4584428640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f457c155450 0x7f457c0cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.133+0000 7f4584428640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f457c0af300 con 0x7f457c0aee70 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.133+0000 7f4584428640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f457c0af5e0 con 0x7f457c155450 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.133+0000 7f4584428640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f457c0af8c0 con 0x7f457c0adad0 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45829c5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f457c155450 0x7f457c0cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c15f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:36734/0 (socket says 172.21.15.76:36734) 2024-09-19T04:14:09.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 -- 172.21.15.76:0/2407722454 learned_addr learned my addr 172.21.15.76:0/2407722454 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:09.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 -- 172.21.15.76:0/2407722454 >> v2:172.21.15.116:3300/0 conn(0x7f457c155450 msgr2=0x7f457c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:09.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 --2- 172.21.15.76:0/2407722454 >> v2:172.21.15.116:3300/0 conn(0x7f457c155450 0x7f457c0cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 -- 172.21.15.76:0/2407722454 >> v1:172.21.15.116:6789/0 conn(0x7f457c0aee70 legacy=0x7f457c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 -- 172.21.15.76:0/2407722454 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c0afba0 con 0x7f457c0adad0 2024-09-19T04:14:09.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45829c5640 1 --2- 172.21.15.76:0/2407722454 >> v2:172.21.15.116:3300/0 conn(0x7f457c155450 0x7f457c0cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:09.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45821c4640 1 --2- 172.21.15.76:0/2407722454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c15f320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45780098a0 tx=0x7f457801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb6b32eb880f0e9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:09.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45811c2640 1 -- 172.21.15.76:0/2407722454 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780b0070 con 0x7f457c0adad0 2024-09-19T04:14:09.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.137+0000 7f45811c2640 1 -- 172.21.15.76:0/2407722454 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45780ab600 con 0x7f457c0adad0 2024-09-19T04:14:09.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45811c2640 1 -- 172.21.15.76:0/2407722454 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45780ab9d0 con 0x7f457c0adad0 2024-09-19T04:14:09.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- 172.21.15.76:0/2407722454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 msgr2=0x7f457c15f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 --2- 172.21.15.76:0/2407722454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c15f320 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f45780098a0 tx=0x7f457801c540 comp rx=0 tx=0).stop 2024-09-19T04:14:09.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- 172.21.15.76:0/2407722454 shutdown_connections 2024-09-19T04:14:09.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 --2- 172.21.15.76:0/2407722454 >> v2:172.21.15.116:3300/0 conn(0x7f457c155450 0x7f457c0cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 --2- 172.21.15.76:0/2407722454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c15f320 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- 172.21.15.76:0/2407722454 >> 172.21.15.76:0/2407722454 conn(0x7f457c0736d0 msgr2=0x7f457c06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:09.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- 172.21.15.76:0/2407722454 shutdown_connections 2024-09-19T04:14:09.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- 172.21.15.76:0/2407722454 wait complete. 2024-09-19T04:14:09.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 Processor -- start 2024-09-19T04:14:09.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- start start 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f457c155450 0x7f457c147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f457c148020 0x7f457c0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0d0790 con 0x7f457c155450 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f457c0d0610 con 0x7f457c148020 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f457c06f070 con 0x7f457c0adad0 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f457c155450 0x7f457c147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f457c155450 0x7f457c147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53012/0 (socket says 172.21.15.76:53012) 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 -- 172.21.15.76:0/642586225 learned_addr learned my addr 172.21.15.76:0/642586225 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45829c5640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f457c148020 0x7f457c0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 -- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 msgr2=0x7f457c14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 -- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f457c148020 msgr2=0x7f457c0b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f457c148020 0x7f457c0b1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 -- 172.21.15.76:0/642586225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457c14e0c0 con 0x7f457c155450 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45829c5640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f457c148020 0x7f457c0b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f45819c3640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f457c155450 0x7f457c147ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c460 tx=0x7f456c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4572ffd640 1 -- 172.21.15.76:0/642586225 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c016020 con 0x7f457c155450 2024-09-19T04:14:09.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4572ffd640 1 -- 172.21.15.76:0/642586225 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f456c0052d0 con 0x7f457c155450 2024-09-19T04:14:09.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457c14ce70 con 0x7f457c155450 2024-09-19T04:14:09.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4572ffd640 1 -- 172.21.15.76:0/642586225 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c007260 con 0x7f457c155450 2024-09-19T04:14:09.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.141+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f457c147000 con 0x7f457c155450 2024-09-19T04:14:09.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.145+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457c06f3c0 con 0x7f457c155450 2024-09-19T04:14:09.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.145+0000 7f4572ffd640 1 -- 172.21.15.76:0/642586225 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f456c007400 con 0x7f457c155450 2024-09-19T04:14:09.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.145+0000 7f4572ffd640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45600678f0 0x7f4560069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:09.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.145+0000 7f4572ffd640 1 -- 172.21.15.76:0/642586225 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6412+0+0 (secure 0 0 0) 0x7f456c085840 con 0x7f457c155450 2024-09-19T04:14:09.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.145+0000 7f45821c4640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45600678f0 0x7f4560069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:09.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.145+0000 7f45821c4640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45600678f0 0x7f4560069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f457c0744a0 tx=0x7f4578004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:09.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.149+0000 7f4572ffd640 1 -- 172.21.15.76:0/642586225 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f456c020020 con 0x7f457c155450 2024-09-19T04:14:09.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.317+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f457c0d0970 con 0x7f457c155450 2024-09-19T04:14:09.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.977+0000 7f4572ffd640 1 -- 172.21.15.76:0/642586225 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f456c005480 con 0x7f457c155450 2024-09-19T04:14:09.979 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-09-19T04:14:09.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45600678f0 msgr2=0x7f4560069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45600678f0 0x7f4560069db0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f457c0744a0 tx=0x7f4578004740 comp rx=0 tx=0).stop 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f457c155450 msgr2=0x7f457c147ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f457c155450 0x7f457c147ae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f456c00c460 tx=0x7f456c00c920 comp rx=0 tx=0).stop 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 shutdown_connections 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45600678f0 0x7f4560069db0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f457c148020 0x7f457c0b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f457c155450 0x7f457c147ae0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 --2- 172.21.15.76:0/642586225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f457c0adad0 0x7f457c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 >> 172.21.15.76:0/642586225 conn(0x7f457c0736d0 msgr2=0x7f457c0ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 shutdown_connections 2024-09-19T04:14:09.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:09.981+0000 7f4584428640 1 -- 172.21.15.76:0/642586225 wait complete. 2024-09-19T04:14:10.009 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-19T04:14:10.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fc1687640 1 Processor -- start 2024-09-19T04:14:10.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fc1687640 1 -- start start 2024-09-19T04:14:10.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fc1687640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2fbc0d3060 0x7f2fbc0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fc1687640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fbc0d0300 con 0x7f2fbc161a60 2024-09-19T04:14:10.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fc1687640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fbc0b0f60 con 0x7f2fbc0cf450 2024-09-19T04:14:10.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fc1687640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2fbc0cfd40 con 0x7f2fbc0d3060 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fbaffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2fbc161a60 0x7f2fbc0d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47632/0 (socket says 172.21.15.76:47632) 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fbaffd640 1 -- 172.21.15.76:0/1175069928 learned_addr learned my addr 172.21.15.76:0/1175069928 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fba7fc640 1 --2- 172.21.15.76:0/1175069928 >> v2:172.21.15.116:3300/0 conn(0x7f2fbc0d3060 0x7f2fbc0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1013955634 0 0) 0x7f2fb40014a0 con 0x7f2fbc161a60 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2fa8003530 con 0x7f2fbc161a60 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4170759480 0 0) 0x7f2fa8003530 con 0x7f2fbc161a60 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 >> v2:172.21.15.116:3300/0 conn(0x7f2fbc0d3060 msgr2=0x7f2fbc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 --2- 172.21.15.76:0/1175069928 >> v2:172.21.15.116:3300/0 conn(0x7f2fbc0d3060 0x7f2fbc0cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 >> v1:172.21.15.116:6789/0 conn(0x7f2fbc0cf450 legacy=0x7f2fbc06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fbc0d0020 con 0x7f2fbc161a60 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2fb4001bd0 con 0x7f2fbc161a60 2024-09-19T04:14:10.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 >> v1:172.21.15.76:6789/0 conn(0x7f2fbc161a60 legacy=0x7f2fbc0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fa80039c0 0x7f2fa8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa80051d0 0x7f2fa800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc0cfd40 con 0x7f2fa8004810 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa800c170 con 0x7f2fa80051d0 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2fa800c370 con 0x7f2fa80039c0 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.133+0000 7f2fba7fc640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fbb7fe640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa80051d0 0x7f2fa800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fba7fc640 1 -- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fa80039c0 msgr2=0x7f2fa8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:10.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fba7fc640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fa80039c0 0x7f2fa8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:10.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fba7fc640 1 -- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa80051d0 msgr2=0x7f2fa800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:10.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fba7fc640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa80051d0 0x7f2fa800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:10.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fba7fc640 1 -- 172.21.15.76:0/1175069928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa8003530 con 0x7f2fa8004810 2024-09-19T04:14:10.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fbb7fe640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fa80051d0 0x7f2fa800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:10.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fbb7fe640 1 -- 172.21.15.76:0/1175069928 reap_dead start 2024-09-19T04:14:10.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fba7fc640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009ae0 tx=0x7f2fa401ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffbcea6198b5f47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:10.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40b2070 con 0x7f2fa8004810 2024-09-19T04:14:10.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fa40a5610 con 0x7f2fa8004810 2024-09-19T04:14:10.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fb9ffb640 1 -- 172.21.15.76:0/1175069928 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40096b0 con 0x7f2fa8004810 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fc1687640 1 -- 172.21.15.76:0/1175069928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 msgr2=0x7f2fa8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fc1687640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009ae0 tx=0x7f2fa401ca40 comp rx=0 tx=0).stop 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fc1687640 1 -- 172.21.15.76:0/1175069928 shutdown_connections 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fc1687640 1 --2- 172.21.15.76:0/1175069928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fa8004c90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fc1687640 1 -- 172.21.15.76:0/1175069928 >> 172.21.15.76:0/1175069928 conn(0x7f2fbc08bf90 msgr2=0x7f2fbc09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fc1687640 1 -- 172.21.15.76:0/1175069928 shutdown_connections 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.137+0000 7f2fc1687640 1 -- 172.21.15.76:0/1175069928 wait complete. 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 Processor -- start 2024-09-19T04:14:10.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 -- start start 2024-09-19T04:14:10.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fbc15d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fbc0b98b0 0x7f2fbc15d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fbc15c400 0x7f2fbc15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc0d2ce0 con 0x7f2fa8004810 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fbc0d2b60 con 0x7f2fbc15c400 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2fbc0d2e60 con 0x7f2fbc0b98b0 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fbc15d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fbc15d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53022/0 (socket says 172.21.15.76:53022) 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 -- 172.21.15.76:0/555977605 learned_addr learned my addr 172.21.15.76:0/555977605 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 -- 172.21.15.76:0/555977605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fbc0b98b0 msgr2=0x7f2fbc15d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fbc0b98b0 0x7f2fbc15d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 -- 172.21.15.76:0/555977605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fbc15c400 msgr2=0x7f2fbc15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:10.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbb7fe640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fbc15c400 0x7f2fbc15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fbc15c400 0x7f2fbc15c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 -- 172.21.15.76:0/555977605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fbc157bc0 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fbaffd640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fbc15d200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400caa0 tx=0x7f2fb400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fb37fe640 1 -- 172.21.15.76:0/555977605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4018070 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fb37fe640 1 -- 172.21.15.76:0/555977605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb400a790 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fb37fe640 1 -- 172.21.15.76:0/555977605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb400ab40 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fbc157ea0 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fbc158460 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fbc0d1220 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.141+0000 7f2fb37fe640 1 -- 172.21.15.76:0/555977605 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2fb400ace0 con 0x7f2fa8004810 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.145+0000 7f2fb37fe640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2f9c0678f0 0x7f2f9c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:10.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.145+0000 7f2fba7fc640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2f9c0678f0 0x7f2f9c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:10.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.145+0000 7f2fb37fe640 1 -- 172.21.15.76:0/555977605 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f2fb4085530 con 0x7f2fa8004810 2024-09-19T04:14:10.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.145+0000 7f2fba7fc640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2f9c0678f0 0x7f2f9c069db0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40af000 tx=0x7f2fa4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:10.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.149+0000 7f2fb37fe640 1 -- 172.21.15.76:0/555977605 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2fbc0d0300 con 0x7f2fa8004810 2024-09-19T04:14:10.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.309+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2fbc15cdf0 con 0x7f2fa8004810 2024-09-19T04:14:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:10.997+0000 7f2fb37fe640 1 -- 172.21.15.76:0/555977605 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f2fb4055f50 con 0x7f2fa8004810 2024-09-19T04:14:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data_ec' removed 2024-09-19T04:14:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2f9c0678f0 msgr2=0x7f2f9c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2f9c0678f0 0x7f2f9c069db0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40af000 tx=0x7f2fa4008040 comp rx=0 tx=0).stop 2024-09-19T04:14:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 msgr2=0x7f2fbc15d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fbc15d200 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400caa0 tx=0x7f2fb400cf60 comp rx=0 tx=0).stop 2024-09-19T04:14:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 shutdown_connections 2024-09-19T04:14:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2f9c0678f0 0x7f2f9c069db0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fbc15c400 0x7f2fbc15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fbc0b98b0 0x7f2fbc15d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 --2- 172.21.15.76:0/555977605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fa8004810 0x7f2fbc15d200 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.001+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 >> 172.21.15.76:0/555977605 conn(0x7f2fbc08bf90 msgr2=0x7f2fbc06d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.005+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 shutdown_connections 2024-09-19T04:14:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.005+0000 7f2fc1687640 1 -- 172.21.15.76:0/555977605 wait complete. 2024-09-19T04:14:11.026 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T04:14:11.026 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T04:14:11.026 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T04:14:11.026 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:11.027 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T04:14:11.027 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T04:14:11.027 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T04:14:11.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:11.027 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T04:14:11.027 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T04:14:11.027 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T04:14:11.027 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:11.028 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T04:14:11.028 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T04:14:11.028 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T04:14:11.028 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:14:11.037 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T04:14:11.037 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T04:14:11.039 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T04:14:11.039 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T04:14:11.041 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T04:14:11.041 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T04:14:11.042 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T04:14:11.042 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T04:14:11.081 INFO:tasks.ceph.mds.a:Started 2024-09-19T04:14:11.082 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:11.083 INFO:tasks.ceph.mds.c:Started 2024-09-19T04:14:11.083 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:11.084 INFO:tasks.ceph.mds.d:Started 2024-09-19T04:14:11.084 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:11.085 INFO:tasks.ceph.mds.b:Started 2024-09-19T04:14:11.085 DEBUG:teuthology.parallel:result is None 2024-09-19T04:14:11.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T04:14:11.146 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:14:11.154 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:14:11.156 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:14:11.159 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:14:11.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa9048f9640 1 Processor -- start 2024-09-19T04:14:11.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa9048f9640 1 -- start start 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa9048f9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa9000cfaf0 0x7fa900168f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa9048f9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90006d9c0 con 0x7fa9000ce9d0 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa9048f9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa90006dca0 con 0x7fa90015e5f0 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa9048f9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa90006d7a0 con 0x7fa9000cfaf0 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8fed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa9000cfaf0 0x7fa900168f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8fed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa9000cfaf0 0x7fa900168f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53044/0 (socket says 172.21.15.76:53044) 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8fed9d640 1 -- 172.21.15.76:0/2599983351 learned_addr learned my addr 172.21.15.76:0/2599983351 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8f7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa9000ce9d0 0x7fa9000cf3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47642/0 (socket says 172.21.15.76:47642) 2024-09-19T04:14:11.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8fed9d640 1 -- 172.21.15.76:0/2599983351 >> v1:172.21.15.116:6789/0 conn(0x7fa90015e5f0 legacy=0x7fa90006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8fed9d640 1 -- 172.21.15.76:0/2599983351 >> v1:172.21.15.76:6789/0 conn(0x7fa9000ce9d0 legacy=0x7fa9000cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8fed9d640 1 -- 172.21.15.76:0/2599983351 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa90006df80 con 0x7fa9000cfaf0 2024-09-19T04:14:11.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.201+0000 7fa8fdd9b640 1 -- 172.21.15.76:0/2599983351 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3147175465 0 0) 0x7fa90006d9c0 con 0x7fa9000ce9d0 2024-09-19T04:14:11.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fed9d640 1 --2- 172.21.15.76:0/2599983351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000cfaf0 0x7fa900168f90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009e50 tx=0x7fa8f801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2efb735a40ebed2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fdd9b640 1 -- 172.21.15.76:0/2599983351 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a5810 con 0x7fa9000cfaf0 2024-09-19T04:14:11.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fdd9b640 1 -- 172.21.15.76:0/2599983351 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f80a59b0 con 0x7fa9000cfaf0 2024-09-19T04:14:11.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fdd9b640 1 -- 172.21.15.76:0/2599983351 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a5d80 con 0x7fa9000cfaf0 2024-09-19T04:14:11.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- 172.21.15.76:0/2599983351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000cfaf0 msgr2=0x7fa900168f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 --2- 172.21.15.76:0/2599983351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000cfaf0 0x7fa900168f90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009e50 tx=0x7fa8f801ca30 comp rx=0 tx=0).stop 2024-09-19T04:14:11.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- 172.21.15.76:0/2599983351 shutdown_connections 2024-09-19T04:14:11.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 --2- 172.21.15.76:0/2599983351 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000cfaf0 0x7fa900168f90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- 172.21.15.76:0/2599983351 >> 172.21.15.76:0/2599983351 conn(0x7fa900074030 msgr2=0x7fa90009d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:11.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- 172.21.15.76:0/2599983351 shutdown_connections 2024-09-19T04:14:11.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- 172.21.15.76:0/2599983351 wait complete. 2024-09-19T04:14:11.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 Processor -- start 2024-09-19T04:14:11.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- start start 2024-09-19T04:14:11.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9000cfaf0 0x7fa900168e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9001545e0 0x7fa9000cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000ce0e0 0x7fa9001579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa90016c080 con 0x7fa9000ce0e0 2024-09-19T04:14:11.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa90016bf00 con 0x7fa9000cfaf0 2024-09-19T04:14:11.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa9048f9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa90016c200 con 0x7fa9001545e0 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000ce0e0 0x7fa9001579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000ce0e0 0x7fa9001579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53052/0 (socket says 172.21.15.76:53052) 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fed9d640 1 -- 172.21.15.76:0/1265371652 learned_addr learned my addr 172.21.15.76:0/1265371652 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8fe59c640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9000cfaf0 0x7fa900168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.205+0000 7fa8f7fff640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9001545e0 0x7fa9000cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fed9d640 1 -- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9001545e0 msgr2=0x7fa9000cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fed9d640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9001545e0 0x7fa9000cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fed9d640 1 -- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9000cfaf0 msgr2=0x7fa900168e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fed9d640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9000cfaf0 0x7fa900168e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fed9d640 1 -- 172.21.15.76:0/1265371652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa900154a60 con 0x7fa9000ce0e0 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fed9d640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000ce0e0 0x7fa9001579b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009e50 tx=0x7fa8f80197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8f6ffd640 1 -- 172.21.15.76:0/1265371652 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80bc030 con 0x7fa9000ce0e0 2024-09-19T04:14:11.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8f6ffd640 1 -- 172.21.15.76:0/1265371652 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8f8009210 con 0x7fa9000ce0e0 2024-09-19T04:14:11.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9000cb6a0 con 0x7fa9000ce0e0 2024-09-19T04:14:11.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8f6ffd640 1 -- 172.21.15.76:0/1265371652 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80b8880 con 0x7fa9000ce0e0 2024-09-19T04:14:11.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9000cbc60 con 0x7fa9000ce0e0 2024-09-19T04:14:11.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa90006d9c0 con 0x7fa9000ce0e0 2024-09-19T04:14:11.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8f6ffd640 1 -- 172.21.15.76:0/1265371652 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa8f80bf070 con 0x7fa9000ce0e0 2024-09-19T04:14:11.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8f6ffd640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8e40678c0 0x7fa8e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:11.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fe59c640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8e40678c0 0x7fa8e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:11.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8f6ffd640 1 -- 172.21.15.76:0/1265371652 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5335+0+0 (secure 0 0 0) 0x7fa8f8120500 con 0x7fa9000ce0e0 2024-09-19T04:14:11.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.209+0000 7fa8fe59c640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8e40678c0 0x7fa8e4069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa900159760 tx=0x7fa8f0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:11.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.213+0000 7fa8f6ffd640 1 -- 172.21.15.76:0/1265371652 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa8f8016620 con 0x7fa9000ce0e0 2024-09-19T04:14:11.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:11.389+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa900075780 con 0x7fa9000ce0e0 2024-09-19T04:14:12.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.013+0000 7fa8f6ffd640 1 -- 172.21.15.76:0/1265371652 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) ==== 97+0+0 (secure 0 0 0) 0x7fa8f80f0b60 con 0x7fa9000ce0e0 2024-09-19T04:14:12.017 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2024-09-19T04:14:12.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8e40678c0 msgr2=0x7fa8e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8e40678c0 0x7fa8e4069d80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa900159760 tx=0x7fa8f0009040 comp rx=0 tx=0).stop 2024-09-19T04:14:12.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000ce0e0 msgr2=0x7fa9001579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000ce0e0 0x7fa9001579b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009e50 tx=0x7fa8f80197d0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 shutdown_connections 2024-09-19T04:14:12.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8e40678c0 0x7fa8e4069d80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa9000ce0e0 0x7fa9001579b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa9001545e0 0x7fa9000cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 --2- 172.21.15.76:0/1265371652 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9000cfaf0 0x7fa900168e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 >> 172.21.15.76:0/1265371652 conn(0x7fa9000ce9d0 msgr2=0x7fa9000b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 shutdown_connections 2024-09-19T04:14:12.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.017+0000 7fa9048f9640 1 -- 172.21.15.76:0/1265371652 wait complete. 2024-09-19T04:14:12.047 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T04:14:12.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f494b247640 1 Processor -- start 2024-09-19T04:14:12.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f494b247640 1 -- start start 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f494b247640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49441587a0 0x7f49440b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f494b247640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494406ec60 con 0x7f49440b37d0 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f494b247640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f494406ef40 con 0x7f494406e680 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f494b247640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f49440b3c40 con 0x7f49441587a0 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4948fe3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49441587a0 0x7f49440b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4948fe3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f49441587a0 0x7f49440b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53056/0 (socket says 172.21.15.76:53056) 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4948fe3640 1 -- 172.21.15.76:0/3224970207 learned_addr learned my addr 172.21.15.76:0/3224970207 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4943fff640 1 --1- 172.21.15.76:0/3224970207 >> v1:172.21.15.76:6789/0 conn(0x7f49440b37d0 0x7f494406df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47658/0 (socket says 172.21.15.76:47658) 2024-09-19T04:14:12.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4948fe3640 1 -- 172.21.15.76:0/3224970207 >> v1:172.21.15.116:6789/0 conn(0x7f494406e680 legacy=0x7f4944072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:12.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4948fe3640 1 -- 172.21.15.76:0/3224970207 >> v1:172.21.15.76:6789/0 conn(0x7f49440b37d0 legacy=0x7f494406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4948fe3640 1 -- 172.21.15.76:0/3224970207 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f494406f220 con 0x7f49441587a0 2024-09-19T04:14:12.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f4948fe3640 1 --2- 172.21.15.76:0/3224970207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49441587a0 0x7f49440b3290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f493c009870 tx=0x7f493c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9440baabfce192 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f49437fe640 1 -- 172.21.15.76:0/3224970207 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0b0070 con 0x7f49441587a0 2024-09-19T04:14:12.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f49437fe640 1 -- 172.21.15.76:0/3224970207 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f493c0ab600 con 0x7f49441587a0 2024-09-19T04:14:12.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.165+0000 7f49437fe640 1 -- 172.21.15.76:0/3224970207 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493c0ab9d0 con 0x7f49441587a0 2024-09-19T04:14:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- 172.21.15.76:0/3224970207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49441587a0 msgr2=0x7f49440b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 --2- 172.21.15.76:0/3224970207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49441587a0 0x7f49440b3290 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f493c009870 tx=0x7f493c01c520 comp rx=0 tx=0).stop 2024-09-19T04:14:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- 172.21.15.76:0/3224970207 shutdown_connections 2024-09-19T04:14:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 --2- 172.21.15.76:0/3224970207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49441587a0 0x7f49440b3290 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- 172.21.15.76:0/3224970207 >> 172.21.15.76:0/3224970207 conn(0x7f49440b1f90 msgr2=0x7f49440b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- 172.21.15.76:0/3224970207 shutdown_connections 2024-09-19T04:14:12.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- 172.21.15.76:0/3224970207 wait complete. 2024-09-19T04:14:12.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 Processor -- start 2024-09-19T04:14:12.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- start start 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f494406e680 0x7f4944142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b37d0 0x7f4944157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f49441587a0 0x7f4944157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f494416c200 con 0x7f49440b37d0 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4944073310 con 0x7f494406e680 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f494416c380 con 0x7f49441587a0 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b37d0 0x7f4944157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b37d0 0x7f4944157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53062/0 (socket says 172.21.15.76:53062) 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 -- 172.21.15.76:0/134866352 learned_addr learned my addr 172.21.15.76:0/134866352 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:12.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 -- 172.21.15.76:0/134866352 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f49441587a0 msgr2=0x7f4944157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f49441587a0 0x7f4944157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 -- 172.21.15.76:0/134866352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f494406e680 msgr2=0x7f4944142100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f494406e680 0x7f4944142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 -- 172.21.15.76:0/134866352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4944142640 con 0x7f49440b37d0 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f4943fff640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b37d0 0x7f4944157a60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f493400c7c0 tx=0x7f493400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f49417fa640 1 -- 172.21.15.76:0/134866352 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4934017070 con 0x7f49440b37d0 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f49417fa640 1 -- 172.21.15.76:0/134866352 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f493400a640 con 0x7f49440b37d0 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f494413fc30 con 0x7f49440b37d0 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f49417fa640 1 -- 172.21.15.76:0/134866352 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f493400a9f0 con 0x7f49440b37d0 2024-09-19T04:14:12.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.169+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4944141be0 con 0x7f49440b37d0 2024-09-19T04:14:12.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.173+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49440b3290 con 0x7f49440b37d0 2024-09-19T04:14:12.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.173+0000 7f49417fa640 1 -- 172.21.15.76:0/134866352 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f493401a070 con 0x7f49440b37d0 2024-09-19T04:14:12.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.173+0000 7f49417fa640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f49240678c0 0x7f4924069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.173+0000 7f49417fa640 1 -- 172.21.15.76:0/134866352 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f4934084fc0 con 0x7f49440b37d0 2024-09-19T04:14:12.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.173+0000 7f4948fe3640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f49240678c0 0x7f4924069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.173+0000 7f4948fe3640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f49240678c0 0x7f4924069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f493c0ad000 tx=0x7f493c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.177+0000 7f49417fa640 1 -- 172.21.15.76:0/134866352 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4934055700 con 0x7f49440b37d0 2024-09-19T04:14:12.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.397+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f4944141db0 con 0x7f49440b37d0 2024-09-19T04:14:12.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.397+0000 7f49417fa640 1 -- 172.21.15.76:0/134866352 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7f4934059a00 con 0x7f49440b37d0 2024-09-19T04:14:12.398 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:12.398 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w==", 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA==", 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2024-09-19T04:14:12.400 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw==", 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA==", 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ==", 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w==", 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.403 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ==", 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw==", 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA==", 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw==", 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.406 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.407 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.407 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2024-09-19T04:14:12.407 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg==", 2024-09-19T04:14:12.407 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.407 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.407 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.407 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA==", 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:14:12.408 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg==", 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.409 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg==", 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.410 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ==", 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA==", 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.411 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCopOtmGhUPLBAA9AkjLm1xF81Mjf1IYlN6Fg==", 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:14:12.412 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-mds", 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDNpOtmi6Q2ARAAc4npShF3EGehMcPOOX8rww==", 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-mds" 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-mgr", 2024-09-19T04:14:12.413 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDNpOtmD7Y2ARAA+bEtdvi3kAIWSPr32D6j5Q==", 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-osd", 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDNpOtmGcQ2ARAAF0lRiYEJgGE5R4KBxfeoQg==", 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.414 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-osd" 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rbd", 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDNpOtmj9E2ARAA7oHH2dNX7ISEttvhgCie/w==", 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-19T04:14:12.415 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDNpOtmY982ARAAy9dSSKSn1o35szJRvTVEyA==", 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.bootstrap-rgw", 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDNpOtml+02ARAAMCKTFaNFAHslmTcCHgnzbA==", 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-19T04:14:12.416 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A==", 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:12.417 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg==", 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-19T04:14:12.418 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:12.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f49240678c0 msgr2=0x7f4924069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f49240678c0 0x7f4924069d80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f493c0ad000 tx=0x7f493c008040 comp rx=0 tx=0).stop 2024-09-19T04:14:12.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b37d0 msgr2=0x7f4944157a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b37d0 0x7f4944157a60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f493400c7c0 tx=0x7f493400cc80 comp rx=0 tx=0).stop 2024-09-19T04:14:12.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 shutdown_connections 2024-09-19T04:14:12.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f49240678c0 0x7f4924069d80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f49441587a0 0x7f4944157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f49440b37d0 0x7f4944157a60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 --2- 172.21.15.76:0/134866352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f494406e680 0x7f4944142100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 >> 172.21.15.76:0/134866352 conn(0x7f49440b1f90 msgr2=0x7f49440b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 shutdown_connections 2024-09-19T04:14:12.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.401+0000 7f494b247640 1 -- 172.21.15.76:0/134866352 wait complete. 2024-09-19T04:14:12.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-19T04:14:12.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.541+0000 7f31abbc1640 1 Processor -- start 2024-09-19T04:14:12.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.541+0000 7f31abbc1640 1 -- start start 2024-09-19T04:14:12.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.541+0000 7f31abbc1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31a41520a0 0x7f31a406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.541+0000 7f31abbc1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a4070250 con 0x7f31a415f3a0 2024-09-19T04:14:12.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.541+0000 7f31abbc1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a40aede0 con 0x7f31a406fe50 2024-09-19T04:14:12.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.541+0000 7f31abbc1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f31a4152480 con 0x7f31a41520a0 2024-09-19T04:14:12.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a995d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31a41520a0 0x7f31a406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a995d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31a41520a0 0x7f31a406f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53068/0 (socket says 172.21.15.76:53068) 2024-09-19T04:14:12.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a995d640 1 -- 172.21.15.76:0/742027643 learned_addr learned my addr 172.21.15.76:0/742027643 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:12.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a995d640 1 -- 172.21.15.76:0/742027643 >> v1:172.21.15.116:6789/0 conn(0x7f31a406fe50 legacy=0x7f31a415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a995d640 1 -- 172.21.15.76:0/742027643 >> v1:172.21.15.76:6789/0 conn(0x7f31a415f3a0 legacy=0x7f31a40adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a995d640 1 -- 172.21.15.76:0/742027643 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a415fa10 con 0x7f31a41520a0 2024-09-19T04:14:12.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a995d640 1 --2- 172.21.15.76:0/742027643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a406f910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f319400bb10 tx=0x7f319401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddce236a99efcf7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a895b640 1 -- 172.21.15.76:0/742027643 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31940b4070 con 0x7f31a41520a0 2024-09-19T04:14:12.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a895b640 1 -- 172.21.15.76:0/742027643 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31940a7c10 con 0x7f31a41520a0 2024-09-19T04:14:12.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.545+0000 7f31a895b640 1 -- 172.21.15.76:0/742027643 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a40aeba0 con 0x7f31a41520a0 2024-09-19T04:14:12.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- 172.21.15.76:0/742027643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 msgr2=0x7f31a406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 --2- 172.21.15.76:0/742027643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a406f910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f319400bb10 tx=0x7f319401ec50 comp rx=0 tx=0).stop 2024-09-19T04:14:12.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- 172.21.15.76:0/742027643 shutdown_connections 2024-09-19T04:14:12.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 --2- 172.21.15.76:0/742027643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a406f910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- 172.21.15.76:0/742027643 >> 172.21.15.76:0/742027643 conn(0x7f31a40b1f90 msgr2=0x7f31a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- 172.21.15.76:0/742027643 shutdown_connections 2024-09-19T04:14:12.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- 172.21.15.76:0/742027643 wait complete. 2024-09-19T04:14:12.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 Processor -- start 2024-09-19T04:14:12.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- start start 2024-09-19T04:14:12.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31a406fe50 0x7f31a415dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a414c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31a415f3a0 0x7f31a414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f31a4073d30 con 0x7f31a41520a0 2024-09-19T04:14:12.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f31a4073a50 con 0x7f31a406fe50 2024-09-19T04:14:12.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31abbc1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f31a406e800 con 0x7f31a415f3a0 2024-09-19T04:14:12.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31a915c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a414c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31a915c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a414c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:53082/0 (socket says 172.21.15.76:53082) 2024-09-19T04:14:12.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.553+0000 7f31a915c640 1 -- 172.21.15.76:0/1134960396 learned_addr learned my addr 172.21.15.76:0/1134960396 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31aa15e640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31a415f3a0 0x7f31a414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31a995d640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31a406fe50 0x7f31a415dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31aa15e640 1 -- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31a406fe50 msgr2=0x7f31a415dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31aa15e640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31a406fe50 0x7f31a415dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31aa15e640 1 -- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 msgr2=0x7f31a414c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31aa15e640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a414c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31aa15e640 1 -- 172.21.15.76:0/1134960396 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a4155010 con 0x7f31a415f3a0 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31a915c640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a414c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31aa15e640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31a415f3a0 0x7f31a414c6b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f319c00c3c0 tx=0x7f319c00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f319a7fc640 1 -- 172.21.15.76:0/1134960396 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c0052d0 con 0x7f31a415f3a0 2024-09-19T04:14:12.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f319a7fc640 1 -- 172.21.15.76:0/1134960396 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319c005490 con 0x7f31a415f3a0 2024-09-19T04:14:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a4154d30 con 0x7f31a415f3a0 2024-09-19T04:14:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f319a7fc640 1 -- 172.21.15.76:0/1134960396 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319c007240 con 0x7f31a415f3a0 2024-09-19T04:14:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a41f8690 con 0x7f31a415f3a0 2024-09-19T04:14:12.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a415ec90 con 0x7f31a415f3a0 2024-09-19T04:14:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.557+0000 7f319a7fc640 1 -- 172.21.15.76:0/1134960396 <== mon.2 v2:172.21.15.116:3301/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f319c0073e0 con 0x7f31a415f3a0 2024-09-19T04:14:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.561+0000 7f319a7fc640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31880678c0 0x7f3188069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.561+0000 7f319a7fc640 1 -- 172.21.15.76:0/1134960396 <== mon.2 v2:172.21.15.116:3301/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f319c085660 con 0x7f31a415f3a0 2024-09-19T04:14:12.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.561+0000 7f31a995d640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31880678c0 0x7f3188069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.561+0000 7f31a995d640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31880678c0 0x7f3188069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31940b1000 tx=0x7f3194008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.573+0000 7f319a7fc640 1 -- 172.21.15.76:0/1134960396 <== mon.2 v2:172.21.15.116:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f319c0563f0 con 0x7f31a415f3a0 2024-09-19T04:14:12.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.793+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f31a414daf0 con 0x7f31a415f3a0 2024-09-19T04:14:12.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.801+0000 7f319a7fc640 1 -- 172.21.15.76:0/1134960396 <== mon.2 v2:172.21.15.116:3301/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f319c055cf0 con 0x7f31a415f3a0 2024-09-19T04:14:12.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31880678c0 msgr2=0x7f3188069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31880678c0 0x7f3188069d80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f31940b1000 tx=0x7f3194008040 comp rx=0 tx=0).stop 2024-09-19T04:14:12.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31a415f3a0 msgr2=0x7f31a414c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31a415f3a0 0x7f31a414c6b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f319c00c3c0 tx=0x7f319c00c880 comp rx=0 tx=0).stop 2024-09-19T04:14:12.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 shutdown_connections 2024-09-19T04:14:12.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31880678c0 0x7f3188069d80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31a415f3a0 0x7f31a414c6b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31a41520a0 0x7f31a414c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 --2- 172.21.15.76:0/1134960396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31a406fe50 0x7f31a415dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 >> 172.21.15.76:0/1134960396 conn(0x7f31a40b1f90 msgr2=0x7f31a40b3ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 shutdown_connections 2024-09-19T04:14:12.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.805+0000 7f31abbc1640 1 -- 172.21.15.76:0/1134960396 wait complete. 2024-09-19T04:14:12.823 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-19T04:14:12.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f012e00f640 1 Processor -- start 2024-09-19T04:14:12.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f012e00f640 1 -- start start 2024-09-19T04:14:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f012e00f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01280cfaf0 0x7f0128168f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f012e00f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f012806d9c0 con 0x7f012815e5f0 2024-09-19T04:14:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f012e00f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f012806dca0 con 0x7f01280ce9d0 2024-09-19T04:14:12.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f012e00f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f012806d7a0 con 0x7f01280cfaf0 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f01277fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f012815e5f0 0x7f012806d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48284/0 (socket says 172.21.15.76:48284) 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f01277fe640 1 -- 172.21.15.76:0/2029687998 learned_addr learned my addr 172.21.15.76:0/2029687998 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f0127fff640 1 --2- 172.21.15.76:0/2029687998 >> v2:172.21.15.76:3300/0 conn(0x7f01280cfaf0 0x7f0128168f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f01267fc640 1 -- 172.21.15.76:0/2029687998 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4099929622 0 0) 0x7f012806d9c0 con 0x7f012815e5f0 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f01267fc640 1 -- 172.21.15.76:0/2029687998 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0114003530 con 0x7f012815e5f0 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f0127fff640 1 -- 172.21.15.76:0/2029687998 >> v1:172.21.15.116:6789/0 conn(0x7f01280ce9d0 legacy=0x7f01280cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f0127fff640 1 -- 172.21.15.76:0/2029687998 >> v1:172.21.15.76:6789/0 conn(0x7f012815e5f0 legacy=0x7f012806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.937+0000 7f0127fff640 1 -- 172.21.15.76:0/2029687998 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012806df80 con 0x7f01280cfaf0 2024-09-19T04:14:12.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f0127fff640 1 --2- 172.21.15.76:0/2029687998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280cfaf0 0x7f0128168f90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01200097c0 tx=0x7f012001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cb115a3495c5c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f01267fc640 1 -- 172.21.15.76:0/2029687998 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01200b0070 con 0x7f01280cfaf0 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f01267fc640 1 -- 172.21.15.76:0/2029687998 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01200ab600 con 0x7f01280cfaf0 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f01267fc640 1 -- 172.21.15.76:0/2029687998 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01200ab9d0 con 0x7f01280cfaf0 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- 172.21.15.76:0/2029687998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280cfaf0 msgr2=0x7f0128168f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 --2- 172.21.15.76:0/2029687998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280cfaf0 0x7f0128168f90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f01200097c0 tx=0x7f012001c460 comp rx=0 tx=0).stop 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- 172.21.15.76:0/2029687998 shutdown_connections 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 --2- 172.21.15.76:0/2029687998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280cfaf0 0x7f0128168f90 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- 172.21.15.76:0/2029687998 >> 172.21.15.76:0/2029687998 conn(0x7f0128074030 msgr2=0x7f012809d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- 172.21.15.76:0/2029687998 shutdown_connections 2024-09-19T04:14:12.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- 172.21.15.76:0/2029687998 wait complete. 2024-09-19T04:14:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 Processor -- start 2024-09-19T04:14:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- start start 2024-09-19T04:14:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01280cfaf0 0x7f01280cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280b9d30 0x7f0128151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0128154680 0x7f0128157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f012816c080 con 0x7f01280b9d30 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f012816bf00 con 0x7f0128154680 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.941+0000 7f012e00f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f012816c200 con 0x7f01280cfaf0 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280b9d30 0x7f0128151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280b9d30 0x7f0128151850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46604/0 (socket says 172.21.15.76:46604) 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 -- 172.21.15.76:0/275154367 learned_addr learned my addr 172.21.15.76:0/275154367 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f01277fe640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01280cfaf0 0x7f01280cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0127fff640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0128154680 0x7f0128157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 -- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01280cfaf0 msgr2=0x7f01280cdcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01280cfaf0 0x7f01280cdcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 -- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0128154680 msgr2=0x7f0128157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0128154680 0x7f0128157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 -- 172.21.15.76:0/275154367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01280cd530 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f0126ffd640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280b9d30 0x7f0128151850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f011000b410 tx=0x7f011000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f011ffff640 1 -- 172.21.15.76:0/275154367 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110015030 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f011ffff640 1 -- 172.21.15.76:0/275154367 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0110004020 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012815a820 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f011ffff640 1 -- 172.21.15.76:0/275154367 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0110004410 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f012815ab00 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01280b4210 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f011ffff640 1 -- 172.21.15.76:0/275154367 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0110018070 con 0x7f01280b9d30 2024-09-19T04:14:12.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f011ffff640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01080678c0 0x7f0108069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f011ffff640 1 -- 172.21.15.76:0/275154367 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f0110085590 con 0x7f01280b9d30 2024-09-19T04:14:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f01277fe640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01080678c0 0x7f0108069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:12.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.945+0000 7f01277fe640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01080678c0 0x7f0108069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0118005c10 tx=0x7f0118009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:12.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:12.949+0000 7f011ffff640 1 -- 172.21.15.76:0/275154367 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0110056320 con 0x7f01280b9d30 2024-09-19T04:14:13.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.165+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f012806bcc0 con 0x7f01280b9d30 2024-09-19T04:14:13.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.165+0000 7f011ffff640 1 -- 172.21.15.76:0/275154367 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f0110055c20 con 0x7f01280b9d30 2024-09-19T04:14:13.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01080678c0 msgr2=0x7f0108069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01080678c0 0x7f0108069d80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0118005c10 tx=0x7f0118009040 comp rx=0 tx=0).stop 2024-09-19T04:14:13.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280b9d30 msgr2=0x7f0128151850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280b9d30 0x7f0128151850 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f011000b410 tx=0x7f011000b8d0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 shutdown_connections 2024-09-19T04:14:13.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01080678c0 0x7f0108069d80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0128154680 0x7f0128157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01280b9d30 0x7f0128151850 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 --2- 172.21.15.76:0/275154367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01280cfaf0 0x7f01280cdcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 >> 172.21.15.76:0/275154367 conn(0x7f01280ce9d0 msgr2=0x7f01280d3d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:13.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 shutdown_connections 2024-09-19T04:14:13.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.169+0000 7f012e00f640 1 -- 172.21.15.76:0/275154367 wait complete. 2024-09-19T04:14:13.185 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-19T04:14:13.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e81d3640 1 Processor -- start 2024-09-19T04:14:13.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e81d3640 1 -- start start 2024-09-19T04:14:13.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e81d3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1e00adad0 0x7fd1e015f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e81d3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e00af300 con 0x7fd1e00aee70 2024-09-19T04:14:13.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e81d3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1e00af5e0 con 0x7fd1e0155450 2024-09-19T04:14:13.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e81d3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd1e00af8c0 con 0x7fd1e00adad0 2024-09-19T04:14:13.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e5f6f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd1e00adad0 0x7fd1e015f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e576e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd1e00aee70 0x7fd1e0154d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48292/0 (socket says 172.21.15.76:48292) 2024-09-19T04:14:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e576e640 1 -- 172.21.15.76:0/1930496560 learned_addr learned my addr 172.21.15.76:0/1930496560 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e5f6f640 1 -- 172.21.15.76:0/1930496560 >> v1:172.21.15.116:6789/0 conn(0x7fd1e0155450 legacy=0x7fd1e00cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e5f6f640 1 -- 172.21.15.76:0/1930496560 >> v1:172.21.15.76:6789/0 conn(0x7fd1e00aee70 legacy=0x7fd1e0154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e5f6f640 1 -- 172.21.15.76:0/1930496560 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e00afba0 con 0x7fd1e00adad0 2024-09-19T04:14:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.293+0000 7fd1e4f6d640 1 -- 172.21.15.76:0/1930496560 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1897450260 0 0) 0x7fd1e00af300 con 0x7fd1e00aee70 2024-09-19T04:14:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e5f6f640 1 --2- 172.21.15.76:0/1930496560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e015f320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0097c0 tx=0x7fd1dc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c872b2a6a2a9f4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e4f6d640 1 -- 172.21.15.76:0/1930496560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0b0070 con 0x7fd1e00adad0 2024-09-19T04:14:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e4f6d640 1 -- 172.21.15.76:0/1930496560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0ab600 con 0x7fd1e00adad0 2024-09-19T04:14:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e4f6d640 1 -- 172.21.15.76:0/1930496560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0ab9d0 con 0x7fd1e00adad0 2024-09-19T04:14:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1930496560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 msgr2=0x7fd1e015f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1930496560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e015f320 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0097c0 tx=0x7fd1dc01c460 comp rx=0 tx=0).stop 2024-09-19T04:14:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1930496560 shutdown_connections 2024-09-19T04:14:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1930496560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e015f320 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1930496560 >> 172.21.15.76:0/1930496560 conn(0x7fd1e00736d0 msgr2=0x7fd1e006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1930496560 shutdown_connections 2024-09-19T04:14:13.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.297+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1930496560 wait complete. 2024-09-19T04:14:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 Processor -- start 2024-09-19T04:14:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 -- start start 2024-09-19T04:14:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e014d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd1e0155450 0x7fd1e015dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e015e130 0x7fd1e00b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00b03a0 con 0x7fd1e00adad0 2024-09-19T04:14:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd1e00b0520 con 0x7fd1e015e130 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd1e016ca50 con 0x7fd1e0155450 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e014d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e014d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46610/0 (socket says 172.21.15.76:46610) 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 -- 172.21.15.76:0/1367364653 learned_addr learned my addr 172.21.15.76:0/1367364653 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 -- 172.21.15.76:0/1367364653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd1e0155450 msgr2=0x7fd1e015dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd1e0155450 0x7fd1e015dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 -- 172.21.15.76:0/1367364653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e015e130 msgr2=0x7fd1e00b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e015e130 0x7fd1e00b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 -- 172.21.15.76:0/1367364653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1e015c810 con 0x7fd1e00adad0 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e5f6f640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e014d470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0ae040 tx=0x7fd1dc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1d67fc640 1 -- 172.21.15.76:0/1367364653 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0b0050 con 0x7fd1e00adad0 2024-09-19T04:14:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1d67fc640 1 -- 172.21.15.76:0/1367364653 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1dc0bc070 con 0x7fd1e00adad0 2024-09-19T04:14:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1e015caf0 con 0x7fd1e00adad0 2024-09-19T04:14:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1d67fc640 1 -- 172.21.15.76:0/1367364653 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc0b5a30 con 0x7fd1e00adad0 2024-09-19T04:14:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1e015d0b0 con 0x7fd1e00adad0 2024-09-19T04:14:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1e015f320 con 0x7fd1e00adad0 2024-09-19T04:14:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1d67fc640 1 -- 172.21.15.76:0/1367364653 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd1dc0ab4a0 con 0x7fd1e00adad0 2024-09-19T04:14:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1d67fc640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd1c40678f0 0x7fd1c4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1d67fc640 1 -- 172.21.15.76:0/1367364653 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fd1dc120430 con 0x7fd1e00adad0 2024-09-19T04:14:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e576e640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd1c40678f0 0x7fd1c4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.301+0000 7fd1e576e640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd1c40678f0 0x7fd1c4069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00011f0 tx=0x7fd1d0007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.305+0000 7fd1d67fc640 1 -- 172.21.15.76:0/1367364653 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd1dc016050 con 0x7fd1e00adad0 2024-09-19T04:14:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.525+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fd1e006d1c0 con 0x7fd1e00adad0 2024-09-19T04:14:13.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1d67fc640 1 -- 172.21.15.76:0/1367364653 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7fd1dc0f1130 con 0x7fd1e00adad0 2024-09-19T04:14:13.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd1c40678f0 msgr2=0x7fd1c4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd1c40678f0 0x7fd1c4069db0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd1d00011f0 tx=0x7fd1d0007930 comp rx=0 tx=0).stop 2024-09-19T04:14:13.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 msgr2=0x7fd1e014d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e014d470 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc0ae040 tx=0x7fd1dc01aca0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 shutdown_connections 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd1c40678f0 0x7fd1c4069db0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1e015e130 0x7fd1e00b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd1e0155450 0x7fd1e015dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 --2- 172.21.15.76:0/1367364653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd1e00adad0 0x7fd1e014d470 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 >> 172.21.15.76:0/1367364653 conn(0x7fd1e00736d0 msgr2=0x7fd1e0160f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 shutdown_connections 2024-09-19T04:14:13.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.529+0000 7fd1e81d3640 1 -- 172.21.15.76:0/1367364653 wait complete. 2024-09-19T04:14:13.547 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-19T04:14:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d47f3d640 1 Processor -- start 2024-09-19T04:14:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d47f3d640 1 -- start start 2024-09-19T04:14:13.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d47f3d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d400ce9d0 0x7f9d400cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d47f3d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9d400cf920 0x7f9d400d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d47f3d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d4006d9c0 con 0x7f9d4015e5f0 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d47f3d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9d4006d7a0 con 0x7f9d400cf920 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d47f3d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9d400cfd90 con 0x7f9d400ce9d0 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d464da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9d400cf920 0x7f9d400d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d45cd9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9d4015e5f0 0x7f9d4006d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48296/0 (socket says 172.21.15.76:48296) 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d45cd9640 1 -- 172.21.15.76:0/675461613 learned_addr learned my addr 172.21.15.76:0/675461613 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d464da640 1 -- 172.21.15.76:0/675461613 >> v2:172.21.15.116:3300/0 conn(0x7f9d400ce9d0 msgr2=0x7f9d400cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d464da640 1 --2- 172.21.15.76:0/675461613 >> v2:172.21.15.116:3300/0 conn(0x7f9d400ce9d0 0x7f9d400cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d464da640 1 -- 172.21.15.76:0/675461613 >> v1:172.21.15.76:6789/0 conn(0x7f9d4015e5f0 legacy=0x7f9d4006d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d44cd7640 1 -- 172.21.15.76:0/675461613 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3555605798 0 0) 0x7f9d4006d9c0 con 0x7f9d4015e5f0 2024-09-19T04:14:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.661+0000 7f9d464da640 1 -- 172.21.15.76:0/675461613 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4006dca0 con 0x7f9d400cf920 2024-09-19T04:14:13.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d464da640 1 --2- 172.21.15.76:0/675461613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d400cf920 0x7f9d400d2b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d400ce980 tx=0x7f9d3801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34255fcab2d34a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d44cd7640 1 -- 172.21.15.76:0/675461613 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380b2070 con 0x7f9d400cf920 2024-09-19T04:14:13.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d44cd7640 1 -- 172.21.15.76:0/675461613 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d380a5900 con 0x7f9d400cf920 2024-09-19T04:14:13.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d44cd7640 1 -- 172.21.15.76:0/675461613 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380a5cd0 con 0x7f9d400cf920 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 -- 172.21.15.76:0/675461613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d400cf920 msgr2=0x7f9d400d2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/675461613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d400cf920 0x7f9d400d2b60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9d400ce980 tx=0x7f9d3801c930 comp rx=0 tx=0).stop 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 -- 172.21.15.76:0/675461613 shutdown_connections 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/675461613 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d400cf920 0x7f9d400d2b60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/675461613 >> v2:172.21.15.116:3300/0 conn(0x7f9d400ce9d0 0x7f9d400cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 -- 172.21.15.76:0/675461613 >> 172.21.15.76:0/675461613 conn(0x7f9d40074030 msgr2=0x7f9d4009d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 -- 172.21.15.76:0/675461613 shutdown_connections 2024-09-19T04:14:13.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 -- 172.21.15.76:0/675461613 wait complete. 2024-09-19T04:14:13.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 Processor -- start 2024-09-19T04:14:13.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.665+0000 7f9d47f3d640 1 -- start start 2024-09-19T04:14:13.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d400cf920 0x7f9d40156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d400cad90 0x7f9d400b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d40151b60 0x7f9d40151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d400d3c40 con 0x7f9d40151b60 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d400d3ac0 con 0x7f9d400cf920 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4016afa0 con 0x7f9d400cad90 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d40151b60 0x7f9d40151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d40151b60 0x7f9d40151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46620/0 (socket says 172.21.15.76:46620) 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 -- 172.21.15.76:0/2598522224 learned_addr learned my addr 172.21.15.76:0/2598522224 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 -- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d400cad90 msgr2=0x7f9d400b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d454d8640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d400cad90 0x7f9d400b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d45cd9640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d400cf920 0x7f9d40156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d400cad90 0x7f9d400b26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 -- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d400cf920 msgr2=0x7f9d40156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d400cf920 0x7f9d40156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 -- 172.21.15.76:0/2598522224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d400cb210 con 0x7f9d40151b60 2024-09-19T04:14:13.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d464da640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d40151b60 0x7f9d40151fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d3801ce10 tx=0x7f9d380196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d367fc640 1 -- 172.21.15.76:0/2598522224 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380bd020 con 0x7f9d40151b60 2024-09-19T04:14:13.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d367fc640 1 -- 172.21.15.76:0/2598522224 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d380b8670 con 0x7f9d40151b60 2024-09-19T04:14:13.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d400b90b0 con 0x7f9d40151b60 2024-09-19T04:14:13.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d367fc640 1 -- 172.21.15.76:0/2598522224 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d380b2040 con 0x7f9d40151b60 2024-09-19T04:14:13.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d400b9670 con 0x7f9d40151b60 2024-09-19T04:14:13.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d400b99a0 con 0x7f9d40151b60 2024-09-19T04:14:13.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d367fc640 1 -- 172.21.15.76:0/2598522224 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9d380c0020 con 0x7f9d40151b60 2024-09-19T04:14:13.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d367fc640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d240678c0 0x7f9d24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:13.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d367fc640 1 -- 172.21.15.76:0/2598522224 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f9d38120df0 con 0x7f9d40151b60 2024-09-19T04:14:13.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d45cd9640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d240678c0 0x7f9d24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:13.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.669+0000 7f9d45cd9640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d240678c0 0x7f9d24069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c000f70 tx=0x7f9d3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.673+0000 7f9d367fc640 1 -- 172.21.15.76:0/2598522224 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9d380c7020 con 0x7f9d40151b60 2024-09-19T04:14:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.893+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f9d400d19d0 con 0x7f9d40151b60 2024-09-19T04:14:13.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d367fc640 1 -- 172.21.15.76:0/2598522224 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f9d38016520 con 0x7f9d40151b60 2024-09-19T04:14:13.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d240678c0 msgr2=0x7f9d24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d240678c0 0x7f9d24069d80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c000f70 tx=0x7f9d3c009040 comp rx=0 tx=0).stop 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d40151b60 msgr2=0x7f9d40151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d40151b60 0x7f9d40151fc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9d3801ce10 tx=0x7f9d380196d0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 shutdown_connections 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d240678c0 0x7f9d24069d80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d40151b60 0x7f9d40151fc0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d400cad90 0x7f9d400b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 --2- 172.21.15.76:0/2598522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d400cf920 0x7f9d40156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 >> 172.21.15.76:0/2598522224 conn(0x7f9d400ce9d0 msgr2=0x7f9d400b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 shutdown_connections 2024-09-19T04:14:13.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:13.897+0000 7f9d47f3d640 1 -- 172.21.15.76:0/2598522224 wait complete. 2024-09-19T04:14:13.915 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-19T04:14:14.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.029+0000 7f6cf70cf640 1 Processor -- start 2024-09-19T04:14:14.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.029+0000 7f6cf70cf640 1 -- start start 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf70cf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf0073e60 0x7f6cf006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf70cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6cf00b3d50 0x7f6cf00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf70cf640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6cf006ee50 con 0x7f6cf006fef0 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf70cf640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6cf00b46c0 con 0x7f6cf00b3d50 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf70cf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf006f130 con 0x7f6cf0073e60 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6cf00b3d50 0x7f6cf00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6cf00b3d50 0x7f6cf00b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46626/0 (socket says 172.21.15.76:46626) 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 -- 172.21.15.76:0/3352350620 learned_addr learned my addr 172.21.15.76:0/3352350620 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf4e6b640 1 --2- 172.21.15.76:0/3352350620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf0073e60 0x7f6cf006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 -- 172.21.15.76:0/3352350620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf0073e60 msgr2=0x7f6cf006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 --2- 172.21.15.76:0/3352350620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf0073e60 0x7f6cf006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 -- 172.21.15.76:0/3352350620 >> v1:172.21.15.116:6789/0 conn(0x7f6cf006fef0 legacy=0x7f6cf015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:14.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 -- 172.21.15.76:0/3352350620 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf006f410 con 0x7f6cf00b3d50 2024-09-19T04:14:14.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cf566c640 1 --2- 172.21.15.76:0/3352350620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00b4180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800af60 tx=0x7f6ce801da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89cb9b390585989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cef7fe640 1 -- 172.21.15.76:0/3352350620 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce800a2d0 con 0x7f6cf00b3d50 2024-09-19T04:14:14.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cef7fe640 1 -- 172.21.15.76:0/3352350620 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce80a6710 con 0x7f6cf00b3d50 2024-09-19T04:14:14.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.033+0000 7f6cef7fe640 1 -- 172.21.15.76:0/3352350620 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80a6ae0 con 0x7f6cf00b3d50 2024-09-19T04:14:14.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- 172.21.15.76:0/3352350620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 msgr2=0x7f6cf00b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/3352350620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00b4180 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6ce800af60 tx=0x7f6ce801da60 comp rx=0 tx=0).stop 2024-09-19T04:14:14.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- 172.21.15.76:0/3352350620 shutdown_connections 2024-09-19T04:14:14.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/3352350620 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00b4180 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/3352350620 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf0073e60 0x7f6cf006e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- 172.21.15.76:0/3352350620 >> 172.21.15.76:0/3352350620 conn(0x7f6cf00b1f90 msgr2=0x7f6cf00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- 172.21.15.76:0/3352350620 shutdown_connections 2024-09-19T04:14:14.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- 172.21.15.76:0/3352350620 wait complete. 2024-09-19T04:14:14.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 Processor -- start 2024-09-19T04:14:14.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- start start 2024-09-19T04:14:14.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf006fef0 0x7f6cf00998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf0073e60 0x7f6cf00ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf00729e0 con 0x7f6cf00b3d50 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6cf0072860 con 0x7f6cf0073e60 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6cf0072b60 con 0x7f6cf006fef0 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46632/0 (socket says 172.21.15.76:46632) 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 -- 172.21.15.76:0/1986426723 learned_addr learned my addr 172.21.15.76:0/1986426723 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf4e6b640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf006fef0 0x7f6cf00998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6ceffff640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf0073e60 0x7f6cf00ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 -- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf006fef0 msgr2=0x7f6cf00998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf006fef0 0x7f6cf00998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 -- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf0073e60 msgr2=0x7f6cf00ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf0073e60 0x7f6cf00ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 -- 172.21.15.76:0/1986426723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cf009e300 con 0x7f6cf00b3d50 2024-09-19T04:14:14.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf566c640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00add40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6ce801c110 tx=0x7f6ce801a800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6ced7fa640 1 -- 172.21.15.76:0/1986426723 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce80a6d40 con 0x7f6cf00b3d50 2024-09-19T04:14:14.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6ced7fa640 1 -- 172.21.15.76:0/1986426723 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ce8008500 con 0x7f6cf00b3d50 2024-09-19T04:14:14.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cf009d260 con 0x7f6cf00b3d50 2024-09-19T04:14:14.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6ced7fa640 1 -- 172.21.15.76:0/1986426723 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8008930 con 0x7f6cf00b3d50 2024-09-19T04:14:14.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.037+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cf009d820 con 0x7f6cf00b3d50 2024-09-19T04:14:14.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.041+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cf006e800 con 0x7f6cf00b3d50 2024-09-19T04:14:14.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.041+0000 7f6ced7fa640 1 -- 172.21.15.76:0/1986426723 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6ce80bf020 con 0x7f6cf00b3d50 2024-09-19T04:14:14.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.041+0000 7f6ced7fa640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6cd40678c0 0x7f6cd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.041+0000 7f6ced7fa640 1 -- 172.21.15.76:0/1986426723 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f6ce8120590 con 0x7f6cf00b3d50 2024-09-19T04:14:14.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.041+0000 7f6cf4e6b640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6cd40678c0 0x7f6cd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.041+0000 7f6cf4e6b640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6cd40678c0 0x7f6cd4069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400b9d0 tx=0x7f6ce4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.045+0000 7f6ced7fa640 1 -- 172.21.15.76:0/1986426723 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6ce80176e0 con 0x7f6cf00b3d50 2024-09-19T04:14:14.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.265+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f6cf015dd90 con 0x7f6cf00b3d50 2024-09-19T04:14:14.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.269+0000 7f6ced7fa640 1 -- 172.21.15.76:0/1986426723 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7f6ce80f12c0 con 0x7f6cf00b3d50 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6cd40678c0 msgr2=0x7f6cd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6cd40678c0 0x7f6cd4069d80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ce400b9d0 tx=0x7f6ce4008040 comp rx=0 tx=0).stop 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 msgr2=0x7f6cf00add40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00add40 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6ce801c110 tx=0x7f6ce801a800 comp rx=0 tx=0).stop 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 shutdown_connections 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6cd40678c0 0x7f6cd4069d80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6cf00b3d50 0x7f6cf00add40 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf0073e60 0x7f6cf00ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 --2- 172.21.15.76:0/1986426723 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6cf006fef0 0x7f6cf00998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 >> 172.21.15.76:0/1986426723 conn(0x7f6cf00b1f90 msgr2=0x7f6cf0072f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 shutdown_connections 2024-09-19T04:14:14.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.273+0000 7f6cf70cf640 1 -- 172.21.15.76:0/1986426723 wait complete. 2024-09-19T04:14:14.291 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-19T04:14:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7788f35640 1 Processor -- start 2024-09-19T04:14:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7788f35640 1 -- start start 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7788f35640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f77840d3060 0x7f77840cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7788f35640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77840cf450 0x7f778406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7788f35640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77840d0300 con 0x7f7784161a60 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7788f35640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f77840b0f60 con 0x7f77840d3060 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7788f35640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f77840cfd40 con 0x7f77840cf450 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f778259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7784161a60 0x7f77840d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48298/0 (socket says 172.21.15.76:48298) 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f778259c640 1 -- 172.21.15.76:0/179174717 learned_addr learned my addr 172.21.15.76:0/179174717 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.397+0000 7f7779d9b640 1 --2- 172.21.15.76:0/179174717 >> v2:172.21.15.76:3300/0 conn(0x7f77840d3060 0x7f77840cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7782d9d640 1 --2- 172.21.15.76:0/179174717 >> v2:172.21.15.116:3300/0 conn(0x7f77840cf450 0x7f778406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7781d9b640 1 -- 172.21.15.76:0/179174717 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 855911 0 0) 0x7f7774001120 con 0x7f7784161a60 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7781d9b640 1 -- 172.21.15.76:0/179174717 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7770003530 con 0x7f7784161a60 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7779d9b640 1 -- 172.21.15.76:0/179174717 >> v2:172.21.15.116:3300/0 conn(0x7f77840cf450 msgr2=0x7f778406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7779d9b640 1 --2- 172.21.15.76:0/179174717 >> v2:172.21.15.116:3300/0 conn(0x7f77840cf450 0x7f778406d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7779d9b640 1 -- 172.21.15.76:0/179174717 >> v1:172.21.15.76:6789/0 conn(0x7f7784161a60 legacy=0x7f77840d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7779d9b640 1 -- 172.21.15.76:0/179174717 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77840d0020 con 0x7f77840d3060 2024-09-19T04:14:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7782d9d640 1 --2- 172.21.15.76:0/179174717 >> v2:172.21.15.116:3300/0 conn(0x7f77840cf450 0x7f778406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7779d9b640 1 --2- 172.21.15.76:0/179174717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840cef10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77840ce950 tx=0x7f776c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07c26d3abc4c991 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7781d9b640 1 -- 172.21.15.76:0/179174717 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0b1070 con 0x7f77840d3060 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7781d9b640 1 -- 172.21.15.76:0/179174717 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c0ac620 con 0x7f77840d3060 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7781d9b640 1 -- 172.21.15.76:0/179174717 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0ac9f0 con 0x7f77840d3060 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 -- 172.21.15.76:0/179174717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 msgr2=0x7f77840cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 --2- 172.21.15.76:0/179174717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840cef10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f77840ce950 tx=0x7f776c01c3c0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 -- 172.21.15.76:0/179174717 shutdown_connections 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 --2- 172.21.15.76:0/179174717 >> v2:172.21.15.116:3300/0 conn(0x7f77840cf450 0x7f778406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 --2- 172.21.15.76:0/179174717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840cef10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 -- 172.21.15.76:0/179174717 >> 172.21.15.76:0/179174717 conn(0x7f778408bf90 msgr2=0x7f778409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 -- 172.21.15.76:0/179174717 shutdown_connections 2024-09-19T04:14:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.401+0000 7f7788f35640 1 -- 172.21.15.76:0/179174717 wait complete. 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 Processor -- start 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 -- start start 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77840cf450 0x7f7784153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7784161a60 0x7f77840b2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f778416af20 con 0x7f77840d3060 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f778406e650 con 0x7f7784161a60 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f778416b0a0 con 0x7f77840cf450 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46650/0 (socket says 172.21.15.76:46650) 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 -- 172.21.15.76:0/4133181805 learned_addr learned my addr 172.21.15.76:0/4133181805 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 -- 172.21.15.76:0/4133181805 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77840cf450 msgr2=0x7f7784153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77840cf450 0x7f7784153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 -- 172.21.15.76:0/4133181805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7784161a60 msgr2=0x7f77840b2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7784161a60 0x7f77840b2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 -- 172.21.15.76:0/4133181805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77840b7680 con 0x7f77840d3060 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7779d9b640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840b2540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f776c019ff0 tx=0x7f776c019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f777b7fe640 1 -- 172.21.15.76:0/4133181805 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0b1040 con 0x7f77840d3060 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f777b7fe640 1 -- 172.21.15.76:0/4133181805 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f776c0ba070 con 0x7f77840d3060 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77840b7960 con 0x7f77840d3060 2024-09-19T04:14:14.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f777b7fe640 1 -- 172.21.15.76:0/4133181805 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0b5650 con 0x7f77840d3060 2024-09-19T04:14:14.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77840b7f20 con 0x7f77840d3060 2024-09-19T04:14:14.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f778409d8a0 con 0x7f77840d3060 2024-09-19T04:14:14.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f777b7fe640 1 -- 172.21.15.76:0/4133181805 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f776c0b57f0 con 0x7f77840d3060 2024-09-19T04:14:14.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f777b7fe640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77640678c0 0x7f7764069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f777b7fe640 1 -- 172.21.15.76:0/4133181805 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f776c11f870 con 0x7f77840d3060 2024-09-19T04:14:14.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.405+0000 7f778259c640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77640678c0 0x7f7764069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.409+0000 7f778259c640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77640678c0 0x7f7764069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7774000f70 tx=0x7f7774001120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.409+0000 7f777b7fe640 1 -- 172.21.15.76:0/4133181805 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f776c016070 con 0x7f77840d3060 2024-09-19T04:14:14.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.617+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f778415a8a0 con 0x7f77840d3060 2024-09-19T04:14:14.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f777b7fe640 1 -- 172.21.15.76:0/4133181805 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7f776c0f0570 con 0x7f77840d3060 2024-09-19T04:14:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77640678c0 msgr2=0x7f7764069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77640678c0 0x7f7764069d80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7774000f70 tx=0x7f7774001120 comp rx=0 tx=0).stop 2024-09-19T04:14:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 msgr2=0x7f77840b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840b2540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f776c019ff0 tx=0x7f776c019160 comp rx=0 tx=0).stop 2024-09-19T04:14:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 shutdown_connections 2024-09-19T04:14:14.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77640678c0 0x7f7764069d80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7784161a60 0x7f77840b2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77840d3060 0x7f77840b2540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 --2- 172.21.15.76:0/4133181805 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77840cf450 0x7f7784153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 >> 172.21.15.76:0/4133181805 conn(0x7f778408bf90 msgr2=0x7f77840d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 shutdown_connections 2024-09-19T04:14:14.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.621+0000 7f7788f35640 1 -- 172.21.15.76:0/4133181805 wait complete. 2024-09-19T04:14:14.639 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T04:14:14.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.753+0000 7f1a3a074640 1 Processor -- start 2024-09-19T04:14:14.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.753+0000 7f1a3a074640 1 -- start start 2024-09-19T04:14:14.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a3a074640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1a34070020 0x7f1a3415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a3a074640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a3415f1d0 0x7f1a340adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a3a074640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a340aede0 con 0x7f1a341520a0 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a3a074640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1a34152480 con 0x7f1a34070020 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a3a074640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3415fa10 con 0x7f1a3415f1d0 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1a34070020 0x7f1a3415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1a34070020 0x7f1a3415ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46664/0 (socket says 172.21.15.76:46664) 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 -- 172.21.15.76:0/3541501911 learned_addr learned my addr 172.21.15.76:0/3541501911 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a33fff640 1 --2- 172.21.15.76:0/3541501911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a3415f1d0 0x7f1a340adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 -- 172.21.15.76:0/3541501911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a3415f1d0 msgr2=0x7f1a340adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 --2- 172.21.15.76:0/3541501911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a3415f1d0 0x7f1a340adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 -- 172.21.15.76:0/3541501911 >> v1:172.21.15.116:6789/0 conn(0x7f1a341520a0 legacy=0x7f1a3406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:14.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 -- 172.21.15.76:0/3541501911 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a34074250 con 0x7f1a34070020 2024-09-19T04:14:14.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a2affd640 1 --2- 172.21.15.76:0/3541501911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a34070020 0x7f1a3415ec90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009870 tx=0x7f1a1c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e77119bbc97122 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a32ffd640 1 -- 172.21.15.76:0/3541501911 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0aa610 con 0x7f1a34070020 2024-09-19T04:14:14.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a32ffd640 1 -- 172.21.15.76:0/3541501911 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1c0af070 con 0x7f1a34070020 2024-09-19T04:14:14.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.757+0000 7f1a32ffd640 1 -- 172.21.15.76:0/3541501911 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1c0aa9d0 con 0x7f1a34070020 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- 172.21.15.76:0/3541501911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a34070020 msgr2=0x7f1a3415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 --2- 172.21.15.76:0/3541501911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a34070020 0x7f1a3415ec90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009870 tx=0x7f1a1c01c4f0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- 172.21.15.76:0/3541501911 shutdown_connections 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 --2- 172.21.15.76:0/3541501911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a3415f1d0 0x7f1a340adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 --2- 172.21.15.76:0/3541501911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a34070020 0x7f1a3415ec90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- 172.21.15.76:0/3541501911 >> 172.21.15.76:0/3541501911 conn(0x7f1a340b1f90 msgr2=0x7f1a340b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- 172.21.15.76:0/3541501911 shutdown_connections 2024-09-19T04:14:14.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- 172.21.15.76:0/3541501911 wait complete. 2024-09-19T04:14:14.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 Processor -- start 2024-09-19T04:14:14.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- start start 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a34070020 0x7f1a340998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a341520a0 0x7f1a340cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415f1d0 0x7f1a3415d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3415f600 con 0x7f1a3415f1d0 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1a34073d30 con 0x7f1a341520a0 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3415f780 con 0x7f1a34070020 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415f1d0 0x7f1a3415d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415f1d0 0x7f1a3415d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46680/0 (socket says 172.21.15.76:46680) 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 -- 172.21.15.76:0/2612086334 learned_addr learned my addr 172.21.15.76:0/2612086334 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a337fe640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a34070020 0x7f1a340998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a2affd640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a341520a0 0x7f1a340cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 -- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a34070020 msgr2=0x7f1a340998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a34070020 0x7f1a340998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 -- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a341520a0 msgr2=0x7f1a340cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a341520a0 0x7f1a340cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 -- 172.21.15.76:0/2612086334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a34164170 con 0x7f1a3415f1d0 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a2affd640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a341520a0 0x7f1a340cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a33fff640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415f1d0 0x7f1a3415d9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cda0 tx=0x7f1a2c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a30ff9640 1 -- 172.21.15.76:0/2612086334 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c003ba0 con 0x7f1a3415f1d0 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a30ff9640 1 -- 172.21.15.76:0/2612086334 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a2c003d40 con 0x7f1a3415f1d0 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a34164450 con 0x7f1a3415f1d0 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a30ff9640 1 -- 172.21.15.76:0/2612086334 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c004240 con 0x7f1a3415f1d0 2024-09-19T04:14:14.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.761+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a34099df0 con 0x7f1a3415f1d0 2024-09-19T04:14:14.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.765+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a3415ec90 con 0x7f1a3415f1d0 2024-09-19T04:14:14.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.765+0000 7f1a30ff9640 1 -- 172.21.15.76:0/2612086334 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1a2c0043e0 con 0x7f1a3415f1d0 2024-09-19T04:14:14.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.765+0000 7f1a30ff9640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1a140678f0 0x7f1a14069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:14.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.765+0000 7f1a337fe640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1a140678f0 0x7f1a14069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:14.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.765+0000 7f1a30ff9640 1 -- 172.21.15.76:0/2612086334 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7f1a2c0865c0 con 0x7f1a3415f1d0 2024-09-19T04:14:14.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.765+0000 7f1a337fe640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1a140678f0 0x7f1a14069db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b810 tx=0x7f1a24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:14.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.769+0000 7f1a30ff9640 1 -- 172.21.15.76:0/2612086334 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1a2c01d340 con 0x7f1a3415f1d0 2024-09-19T04:14:14.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.921+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f1a3406e540 con 0x7f1a3415f1d0 2024-09-19T04:14:14.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.921+0000 7f1a30ff9640 1 -- 172.21.15.76:0/2612086334 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) ==== 84+0+65 (secure 0 0 0) 0x7f1a2c057240 con 0x7f1a3415f1d0 2024-09-19T04:14:14.924 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:14:14.924 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:14:14.924 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-09-19T04:14:14.924 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-09-19T04:14:14.924 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:14.924 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:14:14.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1a140678f0 msgr2=0x7f1a14069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1a140678f0 0x7f1a14069db0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b810 tx=0x7f1a24008040 comp rx=0 tx=0).stop 2024-09-19T04:14:14.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415f1d0 msgr2=0x7f1a3415d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:14.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415f1d0 0x7f1a3415d9f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cda0 tx=0x7f1a2c00a660 comp rx=0 tx=0).stop 2024-09-19T04:14:14.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 shutdown_connections 2024-09-19T04:14:14.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1a140678f0 0x7f1a14069db0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1a3415f1d0 0x7f1a3415d9f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a341520a0 0x7f1a340cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 --2- 172.21.15.76:0/2612086334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1a34070020 0x7f1a340998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:14.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 >> 172.21.15.76:0/2612086334 conn(0x7f1a340b1f90 msgr2=0x7f1a34151590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:14.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 shutdown_connections 2024-09-19T04:14:14.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:14.925+0000 7f1a3a074640 1 -- 172.21.15.76:0/2612086334 wait complete. 2024-09-19T04:14:14.940 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:14:14.940 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:14:15.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd8268b640 1 Processor -- start 2024-09-19T04:14:15.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd8268b640 1 -- start start 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd8268b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd7c15bec0 0x7fcd7c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd8268b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcd7c0af470 0x7fcd7c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd8268b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd7c0b3290 con 0x7fcd7c15c2a0 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd8268b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7c0b3570 con 0x7fcd7c15bec0 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd8268b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcd7c0b3850 con 0x7fcd7c0af470 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcd7c15bec0 0x7fcd7c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcd7c15c2a0 0x7fcd7c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48302/0 (socket says 172.21.15.76:48302) 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7b7fe640 1 -- 172.21.15.76:0/744750438 learned_addr learned my addr 172.21.15.76:0/744750438 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:15.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd80c28640 1 --2- 172.21.15.76:0/744750438 >> v2:172.21.15.116:3300/0 conn(0x7fcd7c0af470 0x7fcd7c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:15.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7bfff640 1 -- 172.21.15.76:0/744750438 >> v2:172.21.15.116:3300/0 conn(0x7fcd7c0af470 msgr2=0x7fcd7c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:15.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7bfff640 1 --2- 172.21.15.76:0/744750438 >> v2:172.21.15.116:3300/0 conn(0x7fcd7c0af470 0x7fcd7c06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:15.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7bfff640 1 -- 172.21.15.76:0/744750438 >> v1:172.21.15.76:6789/0 conn(0x7fcd7c15c2a0 legacy=0x7fcd7c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:15.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7affd640 1 -- 172.21.15.76:0/744750438 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3502070834 0 0) 0x7fcd7c0b3290 con 0x7fcd7c15c2a0 2024-09-19T04:14:15.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.053+0000 7fcd7bfff640 1 -- 172.21.15.76:0/744750438 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c0b3b30 con 0x7fcd7c15bec0 2024-09-19T04:14:15.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd7bfff640 1 --2- 172.21.15.76:0/744750438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c15bec0 0x7fcd7c0ad200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0b2fb0 tx=0x7fcd7401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8036b22b0cfdd07d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:15.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd7affd640 1 -- 172.21.15.76:0/744750438 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd740b2070 con 0x7fcd7c15bec0 2024-09-19T04:14:15.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd7affd640 1 -- 172.21.15.76:0/744750438 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd740a58d0 con 0x7fcd7c15bec0 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd7affd640 1 -- 172.21.15.76:0/744750438 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd740a5ca0 con 0x7fcd7c15bec0 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- 172.21.15.76:0/744750438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c15bec0 msgr2=0x7fcd7c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 --2- 172.21.15.76:0/744750438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c15bec0 0x7fcd7c0ad200 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0b2fb0 tx=0x7fcd7401c900 comp rx=0 tx=0).stop 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- 172.21.15.76:0/744750438 shutdown_connections 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 --2- 172.21.15.76:0/744750438 >> v2:172.21.15.116:3300/0 conn(0x7fcd7c0af470 0x7fcd7c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 --2- 172.21.15.76:0/744750438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c15bec0 0x7fcd7c0ad200 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- 172.21.15.76:0/744750438 >> 172.21.15.76:0/744750438 conn(0x7fcd7c0b1f90 msgr2=0x7fcd7c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- 172.21.15.76:0/744750438 shutdown_connections 2024-09-19T04:14:15.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- 172.21.15.76:0/744750438 wait complete. 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 Processor -- start 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- start start 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c0af470 0x7fcd7c14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcd7c15bec0 0x7fcd7c150690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd7c15c2a0 0x7fcd7c150bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c06ed50 con 0x7fcd7c0af470 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c0ad740 con 0x7fcd7c15c2a0 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd8268b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7c06eed0 con 0x7fcd7c15bec0 2024-09-19T04:14:15.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.057+0000 7fcd7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c0af470 0x7fcd7c14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c0af470 0x7fcd7c14ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46698/0 (socket says 172.21.15.76:46698) 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 -- 172.21.15.76:0/319269558 learned_addr learned my addr 172.21.15.76:0/319269558 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7b7fe640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcd7c15bec0 0x7fcd7c150690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 -- 172.21.15.76:0/319269558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcd7c15bec0 msgr2=0x7fcd7c150690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcd7c15bec0 0x7fcd7c150690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 -- 172.21.15.76:0/319269558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd7c15c2a0 msgr2=0x7fcd7c150bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd7c15c2a0 0x7fcd7c150bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 -- 172.21.15.76:0/319269558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c1570e0 con 0x7fcd7c0af470 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7bfff640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c0af470 0x7fcd7c14ebc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd740b0040 tx=0x7fcd740196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd78ff9640 1 -- 172.21.15.76:0/319269558 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd740b2050 con 0x7fcd7c0af470 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd78ff9640 1 -- 172.21.15.76:0/319269558 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd740be070 con 0x7fcd7c0af470 2024-09-19T04:14:15.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c1573c0 con 0x7fcd7c0af470 2024-09-19T04:14:15.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd78ff9640 1 -- 172.21.15.76:0/319269558 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd740adb70 con 0x7fcd7c0af470 2024-09-19T04:14:15.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c157980 con 0x7fcd7c0af470 2024-09-19T04:14:15.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c06e000 con 0x7fcd7c0af470 2024-09-19T04:14:15.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd78ff9640 1 -- 172.21.15.76:0/319269558 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcd740add10 con 0x7fcd7c0af470 2024-09-19T04:14:15.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd78ff9640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcd680678c0 0x7fcd68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:15.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd78ff9640 1 -- 172.21.15.76:0/319269558 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5249+0+0 (secure 0 0 0) 0x7fcd74120ef0 con 0x7fcd7c0af470 2024-09-19T04:14:15.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7b7fe640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcd680678c0 0x7fcd68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:15.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.061+0000 7fcd7b7fe640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcd680678c0 0x7fcd68069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c09e4b0 tx=0x7fcd70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:15.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.065+0000 7fcd78ff9640 1 -- 172.21.15.76:0/319269558 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcd74016520 con 0x7fcd7c0af470 2024-09-19T04:14:15.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:15.217+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcd7c159440 con 0x7fcd7c0af470 2024-09-19T04:14:16.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.037+0000 7fcd78ff9640 1 -- 172.21.15.76:0/319269558 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) ==== 138+0+0 (secure 0 0 0) 0x7fcd740ab340 con 0x7fcd7c0af470 2024-09-19T04:14:16.039 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcd680678c0 msgr2=0x7fcd68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcd680678c0 0x7fcd68069d80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c09e4b0 tx=0x7fcd70009040 comp rx=0 tx=0).stop 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c0af470 msgr2=0x7fcd7c14ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c0af470 0x7fcd7c14ebc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fcd740b0040 tx=0x7fcd740196a0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 shutdown_connections 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcd680678c0 0x7fcd68069d80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcd7c15c2a0 0x7fcd7c150bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcd7c15bec0 0x7fcd7c150690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 --2- 172.21.15.76:0/319269558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcd7c0af470 0x7fcd7c14ebc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 >> 172.21.15.76:0/319269558 conn(0x7fcd7c0b1f90 msgr2=0x7fcd7c1598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 shutdown_connections 2024-09-19T04:14:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.041+0000 7fcd8268b640 1 -- 172.21.15.76:0/319269558 wait complete. 2024-09-19T04:14:16.069 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:14:16.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4ca7f640 1 Processor -- start 2024-09-19T04:14:16.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4ca7f640 1 -- start start 2024-09-19T04:14:16.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4ca7f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fee480cc870 0x7fee480cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4ca7f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee48073100 0x7fee480ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4ca7f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fee480cd190 con 0x7fee480cdfd0 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4ca7f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fee480ce8d0 con 0x7fee480cc870 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4ca7f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fee480cebb0 con 0x7fee48073100 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4659c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fee480cc870 0x7fee480cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee45d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee48073100 0x7fee480ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4659c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fee480cc870 0x7fee480cf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:52832/0 (socket says 172.21.15.76:52832) 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4659c640 1 -- 172.21.15.76:0/2426076320 learned_addr learned my addr 172.21.15.76:0/2426076320 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4659c640 1 -- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee48073100 msgr2=0x7fee480ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4659c640 1 --2- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee48073100 0x7fee480ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4659c640 1 -- 172.21.15.76:0/2426076320 >> v1:172.21.15.116:6789/0 conn(0x7fee480cdfd0 legacy=0x7fee4806de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee4659c640 1 -- 172.21.15.76:0/2426076320 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee480cee90 con 0x7fee480cc870 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.201+0000 7fee45d9b640 1 --2- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee48073100 0x7fee480ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4659c640 1 --2- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cc870 0x7fee480cf970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee40009e50 tx=0x7fee4806ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e73b34c72322d5d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4559a640 1 -- 172.21.15.76:0/2426076320 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400a58d0 con 0x7fee480cc870 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4559a640 1 -- 172.21.15.76:0/2426076320 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee400a5a70 con 0x7fee480cc870 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4559a640 1 -- 172.21.15.76:0/2426076320 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400a5e40 con 0x7fee480cc870 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 -- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cc870 msgr2=0x7fee480cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cc870 0x7fee480cf970 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fee40009e50 tx=0x7fee4806ed20 comp rx=0 tx=0).stop 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 -- 172.21.15.76:0/2426076320 shutdown_connections 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee48073100 0x7fee480ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/2426076320 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cc870 0x7fee480cf970 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 -- 172.21.15.76:0/2426076320 >> 172.21.15.76:0/2426076320 conn(0x7fee480b1c80 msgr2=0x7fee480b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 -- 172.21.15.76:0/2426076320 shutdown_connections 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.205+0000 7fee4ca7f640 1 -- 172.21.15.76:0/2426076320 wait complete. 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 Processor -- start 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 -- start start 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee48073100 0x7fee4815f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee480cc870 0x7fee4815c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cdfd0 0x7fee4815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fee48169310 con 0x7fee48073100 2024-09-19T04:14:16.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fee48169190 con 0x7fee480cdfd0 2024-09-19T04:14:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fee48169490 con 0x7fee480cc870 2024-09-19T04:14:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee48073100 0x7fee4815f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee48073100 0x7fee4815f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46706/0 (socket says 172.21.15.76:46706) 2024-09-19T04:14:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 -- 172.21.15.76:0/4052918655 learned_addr learned my addr 172.21.15.76:0/4052918655 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee45d9b640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee480cc870 0x7fee4815c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee46d9d640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cdfd0 0x7fee4815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 -- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee480cc870 msgr2=0x7fee4815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee480cc870 0x7fee4815c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 -- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cdfd0 msgr2=0x7fee4815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cdfd0 0x7fee4815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 -- 172.21.15.76:0/4052918655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4815fb30 con 0x7fee48073100 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee46d9d640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cdfd0 0x7fee4815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee45d9b640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee480cc870 0x7fee4815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4659c640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee48073100 0x7fee4815f430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fee4001a630 tx=0x7fee400197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee3effd640 1 -- 172.21.15.76:0/4052918655 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400bc030 con 0x7fee48073100 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee3effd640 1 -- 172.21.15.76:0/4052918655 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee40007c50 con 0x7fee48073100 2024-09-19T04:14:16.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4815fe10 con 0x7fee48073100 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee3effd640 1 -- 172.21.15.76:0/4052918655 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee400a58d0 con 0x7fee48073100 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4813dfe0 con 0x7fee48073100 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee480ccc50 con 0x7fee48073100 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.209+0000 7fee3effd640 1 -- 172.21.15.76:0/4052918655 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fee400bf070 con 0x7fee48073100 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.213+0000 7fee3effd640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fee280678c0 0x7fee28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.213+0000 7fee45d9b640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fee280678c0 0x7fee28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.213+0000 7fee3effd640 1 -- 172.21.15.76:0/4052918655 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5619+0+0 (secure 0 0 0) 0x7fee40120d20 con 0x7fee48073100 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.213+0000 7fee45d9b640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fee280678c0 0x7fee28069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee481f88d0 tx=0x7fee30008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:16.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.217+0000 7fee3effd640 1 -- 172.21.15.76:0/4052918655 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fee400c6070 con 0x7fee48073100 2024-09-19T04:14:16.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:16.377+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fee480af8d0 con 0x7fee48073100 2024-09-19T04:14:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.049+0000 7fee3effd640 1 -- 172.21.15.76:0/4052918655 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) ==== 170+0+0 (secure 0 0 0) 0x7fee40016620 con 0x7fee48073100 2024-09-19T04:14:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fee280678c0 msgr2=0x7fee28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fee280678c0 0x7fee28069d80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee481f88d0 tx=0x7fee30008040 comp rx=0 tx=0).stop 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee48073100 msgr2=0x7fee4815f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee48073100 0x7fee4815f430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fee4001a630 tx=0x7fee400197a0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 shutdown_connections 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fee280678c0 0x7fee28069d80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fee480cdfd0 0x7fee4815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fee480cc870 0x7fee4815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 --2- 172.21.15.76:0/4052918655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fee48073100 0x7fee4815f430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 >> 172.21.15.76:0/4052918655 conn(0x7fee480b1c80 msgr2=0x7fee4806f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:17.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 shutdown_connections 2024-09-19T04:14:17.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.053+0000 7fee4ca7f640 1 -- 172.21.15.76:0/4052918655 wait complete. 2024-09-19T04:14:17.090 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:14:17.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.209+0000 7f48a2be4640 1 Processor -- start 2024-09-19T04:14:17.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.209+0000 7f48a2be4640 1 -- start start 2024-09-19T04:14:17.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a2be4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f489c15e470 0x7f489c0d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:17.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a2be4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f489c0b38e0 0x7f489c0d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:17.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a2be4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0d4750 0x7f489c161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:17.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a2be4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f489c0d3bc0 con 0x7f489c0b38e0 2024-09-19T04:14:17.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a2be4640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f489c0d3730 con 0x7f489c15e470 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a2be4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f489c0d2b20 con 0x7f489c0d4750 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f489c0b38e0 0x7f489c0d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f489c0b38e0 0x7f489c0d4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46718/0 (socket says 172.21.15.76:46718) 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 -- 172.21.15.76:0/3132577419 learned_addr learned my addr 172.21.15.76:0/3132577419 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a0980640 1 --2- 172.21.15.76:0/3132577419 >> v2:172.21.15.116:3300/0 conn(0x7f489c15e470 0x7f489c0d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a1181640 1 --2- 172.21.15.76:0/3132577419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0d4750 0x7f489c161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 -- 172.21.15.76:0/3132577419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0d4750 msgr2=0x7f489c161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 --2- 172.21.15.76:0/3132577419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0d4750 0x7f489c161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 -- 172.21.15.76:0/3132577419 >> v2:172.21.15.116:3300/0 conn(0x7f489c15e470 msgr2=0x7f489c0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 --2- 172.21.15.76:0/3132577419 >> v2:172.21.15.116:3300/0 conn(0x7f489c15e470 0x7f489c0d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 -- 172.21.15.76:0/3132577419 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c0d2e00 con 0x7f489c0b38e0 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a1181640 1 --2- 172.21.15.76:0/3132577419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0d4750 0x7f489c161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f48a0980640 1 --2- 172.21.15.76:0/3132577419 >> v2:172.21.15.116:3300/0 conn(0x7f489c15e470 0x7f489c0d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:17.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489bfff640 1 --2- 172.21.15.76:0/3132577419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c0b38e0 0x7f489c0d4210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f489c0d2120 tx=0x7f489001c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d237e9af60d3f87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:17.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489b7fe640 1 -- 172.21.15.76:0/3132577419 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900b0070 con 0x7f489c0b38e0 2024-09-19T04:14:17.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489b7fe640 1 -- 172.21.15.76:0/3132577419 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48900ab600 con 0x7f489c0b38e0 2024-09-19T04:14:17.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.213+0000 7f489b7fe640 1 -- 172.21.15.76:0/3132577419 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48900ab9d0 con 0x7f489c0b38e0 2024-09-19T04:14:17.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- 172.21.15.76:0/3132577419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c0b38e0 msgr2=0x7f489c0d4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:17.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 --2- 172.21.15.76:0/3132577419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c0b38e0 0x7f489c0d4210 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f489c0d2120 tx=0x7f489001c1d0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- 172.21.15.76:0/3132577419 shutdown_connections 2024-09-19T04:14:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 --2- 172.21.15.76:0/3132577419 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0d4750 0x7f489c161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 --2- 172.21.15.76:0/3132577419 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c0b38e0 0x7f489c0d4210 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 --2- 172.21.15.76:0/3132577419 >> v2:172.21.15.116:3300/0 conn(0x7f489c15e470 0x7f489c0d15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- 172.21.15.76:0/3132577419 >> 172.21.15.76:0/3132577419 conn(0x7f489c09d730 msgr2=0x7f489c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- 172.21.15.76:0/3132577419 shutdown_connections 2024-09-19T04:14:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- 172.21.15.76:0/3132577419 wait complete. 2024-09-19T04:14:17.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 Processor -- start 2024-09-19T04:14:17.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- start start 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0b38e0 0x7f489c0ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f489c0d4750 0x7f489c0a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c15e470 0x7f489c0b5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f489c06d330 con 0x7f489c15e470 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f489c06d1b0 con 0x7f489c0d4750 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f489c06d4b0 con 0x7f489c0b38e0 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c15e470 0x7f489c0b5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c15e470 0x7f489c0b5e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46728/0 (socket says 172.21.15.76:46728) 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 -- 172.21.15.76:0/230138194 learned_addr learned my addr 172.21.15.76:0/230138194 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a0980640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0b38e0 0x7f489c0ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f489bfff640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f489c0d4750 0x7f489c0a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 -- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0b38e0 msgr2=0x7f489c0ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0b38e0 0x7f489c0ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 -- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f489c0d4750 msgr2=0x7f489c0a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f489c0d4750 0x7f489c0a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 -- 172.21.15.76:0/230138194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f489c0caa00 con 0x7f489c15e470 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a0980640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0b38e0 0x7f489c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f489bfff640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f489c0d4750 0x7f489c0a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a1181640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c15e470 0x7f489c0b5e60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f489400cf20 tx=0x7f489400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48997fa640 1 -- 172.21.15.76:0/230138194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489401a070 con 0x7f489c15e470 2024-09-19T04:14:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48997fa640 1 -- 172.21.15.76:0/230138194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f489400d840 con 0x7f489c15e470 2024-09-19T04:14:17.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f489c0a19d0 con 0x7f489c15e470 2024-09-19T04:14:17.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48997fa640 1 -- 172.21.15.76:0/230138194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489400dc30 con 0x7f489c15e470 2024-09-19T04:14:17.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.217+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f489c0a1f90 con 0x7f489c15e470 2024-09-19T04:14:17.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.221+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f489c161e80 con 0x7f489c15e470 2024-09-19T04:14:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.221+0000 7f48997fa640 1 -- 172.21.15.76:0/230138194 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f489401d070 con 0x7f489c15e470 2024-09-19T04:14:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.221+0000 7f48997fa640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48880678c0 0x7f4888069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.221+0000 7f48997fa640 1 -- 172.21.15.76:0/230138194 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 6001+0+0 (secure 0 0 0) 0x7f4894086710 con 0x7f489c15e470 2024-09-19T04:14:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.221+0000 7f48a0980640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48880678c0 0x7f4888069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.221+0000 7f48a0980640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48880678c0 0x7f4888069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f488c00b840 tx=0x7f488c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.225+0000 7f48997fa640 1 -- 172.21.15.76:0/230138194 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f489c0caa00 con 0x7f489c15e470 2024-09-19T04:14:17.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:17.421+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f489c0d4e70 con 0x7f489c15e470 2024-09-19T04:14:18.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.053+0000 7f48997fa640 1 -- 172.21.15.76:0/230138194 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) ==== 172+0+0 (secure 0 0 0) 0x7f4894057150 con 0x7f489c15e470 2024-09-19T04:14:18.053 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 5 and data pool 6 2024-09-19T04:14:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48880678c0 msgr2=0x7f4888069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48880678c0 0x7f4888069d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f488c00b840 tx=0x7f488c008040 comp rx=0 tx=0).stop 2024-09-19T04:14:18.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c15e470 msgr2=0x7f489c0b5e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c15e470 0x7f489c0b5e60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f489400cf20 tx=0x7f489400a7d0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 shutdown_connections 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48880678c0 0x7f4888069d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f489c15e470 0x7f489c0b5e60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f489c0d4750 0x7f489c0a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 --2- 172.21.15.76:0/230138194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f489c0b38e0 0x7f489c0ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 >> 172.21.15.76:0/230138194 conn(0x7f489c09d730 msgr2=0x7f489c15da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 shutdown_connections 2024-09-19T04:14:18.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.057+0000 7f48a2be4640 1 -- 172.21.15.76:0/230138194 wait complete. 2024-09-19T04:14:18.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:14:18.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.213+0000 7f305cc0f640 1 Processor -- start 2024-09-19T04:14:18.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.213+0000 7f305cc0f640 1 -- start start 2024-09-19T04:14:18.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f30581587a0 0x7f30580b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f305806e680 0x7f3058072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f305806ec60 con 0x7f30580b37d0 2024-09-19T04:14:18.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f305806f870 con 0x7f305806e680 2024-09-19T04:14:18.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f30580b3c10 con 0x7f30581587a0 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f305806e680 0x7f3058072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f305806e680 0x7f3058072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46730/0 (socket says 172.21.15.76:46730) 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 -- 172.21.15.76:0/2144295656 learned_addr learned my addr 172.21.15.76:0/2144295656 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 -- 172.21.15.76:0/2144295656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f30581587a0 msgr2=0x7f30580b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305659c640 1 --2- 172.21.15.76:0/2144295656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f30581587a0 0x7f30580b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 --2- 172.21.15.76:0/2144295656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f30581587a0 0x7f30580b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 -- 172.21.15.76:0/2144295656 >> v1:172.21.15.76:6789/0 conn(0x7f30580b37d0 legacy=0x7f305806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 -- 172.21.15.76:0/2144295656 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305806ef40 con 0x7f305806e680 2024-09-19T04:14:18.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f3056d9d640 1 --2- 172.21.15.76:0/2144295656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305806e680 0x7f3058072160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3050009e50 tx=0x7f305001ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07808624eff441e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305559a640 1 -- 172.21.15.76:0/2144295656 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500a58d0 con 0x7f305806e680 2024-09-19T04:14:18.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305559a640 1 -- 172.21.15.76:0/2144295656 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30500a5a70 con 0x7f305806e680 2024-09-19T04:14:18.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305559a640 1 -- 172.21.15.76:0/2144295656 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500a5e40 con 0x7f305806e680 2024-09-19T04:14:18.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- 172.21.15.76:0/2144295656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305806e680 msgr2=0x7f3058072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 --2- 172.21.15.76:0/2144295656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305806e680 0x7f3058072160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f3050009e50 tx=0x7f305001ca00 comp rx=0 tx=0).stop 2024-09-19T04:14:18.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- 172.21.15.76:0/2144295656 shutdown_connections 2024-09-19T04:14:18.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 --2- 172.21.15.76:0/2144295656 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305806e680 0x7f3058072160 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 --2- 172.21.15.76:0/2144295656 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f30581587a0 0x7f30580b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- 172.21.15.76:0/2144295656 >> 172.21.15.76:0/2144295656 conn(0x7f30580b1f90 msgr2=0x7f30580b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:18.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- 172.21.15.76:0/2144295656 shutdown_connections 2024-09-19T04:14:18.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.217+0000 7f305cc0f640 1 -- 172.21.15.76:0/2144295656 wait complete. 2024-09-19T04:14:18.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 Processor -- start 2024-09-19T04:14:18.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 -- start start 2024-09-19T04:14:18.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806e680 0x7f3058150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30580b37d0 0x7f305809a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581587a0 0x7f30581533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3058073030 con 0x7f30581587a0 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3058072eb0 con 0x7f30580b37d0 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f30580731b0 con 0x7f305806e680 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581587a0 0x7f30581533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581587a0 0x7f30581533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46736/0 (socket says 172.21.15.76:46736) 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 -- 172.21.15.76:0/1597841402 learned_addr learned my addr 172.21.15.76:0/1597841402 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305659c640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806e680 0x7f3058150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3055d9b640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30580b37d0 0x7f305809a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 -- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806e680 msgr2=0x7f3058150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806e680 0x7f3058150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 -- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30580b37d0 msgr2=0x7f305809a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30580b37d0 0x7f305809a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 -- 172.21.15.76:0/1597841402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3058151080 con 0x7f30581587a0 2024-09-19T04:14:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3056d9d640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581587a0 0x7f30581533d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f305001a630 tx=0x7f30500197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3046ffd640 1 -- 172.21.15.76:0/1597841402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3050009510 con 0x7f30581587a0 2024-09-19T04:14:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3046ffd640 1 -- 172.21.15.76:0/1597841402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3050007ca0 con 0x7f30581587a0 2024-09-19T04:14:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3058151380 con 0x7f30581587a0 2024-09-19T04:14:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f3046ffd640 1 -- 172.21.15.76:0/1597841402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500a58d0 con 0x7f30581587a0 2024-09-19T04:14:18.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305809a8c0 con 0x7f30581587a0 2024-09-19T04:14:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.221+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3058072160 con 0x7f30581587a0 2024-09-19T04:14:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.225+0000 7f3046ffd640 1 -- 172.21.15.76:0/1597841402 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f30500c0020 con 0x7f30581587a0 2024-09-19T04:14:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.225+0000 7f3046ffd640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30380678c0 0x7f3038069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.225+0000 7f3046ffd640 1 -- 172.21.15.76:0/1597841402 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f3050120d50 con 0x7f30581587a0 2024-09-19T04:14:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.225+0000 7f305659c640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30380678c0 0x7f3038069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.225+0000 7f305659c640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30380678c0 0x7f3038069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f304c00a8d0 tx=0x7f304c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.229+0000 7f3046ffd640 1 -- 172.21.15.76:0/1597841402 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3050124030 con 0x7f30581587a0 2024-09-19T04:14:18.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.393+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3058070080 con 0x7f30581587a0 2024-09-19T04:14:18.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.393+0000 7f3046ffd640 1 -- 172.21.15.76:0/1597841402 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17039 (secure 0 0 0) 0x7f30500c7020 con 0x7f30581587a0 2024-09-19T04:14:18.395 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:18.395 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":38,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:14:17.425964+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:14:15.256172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T04:14:16.419970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30380678c0 msgr2=0x7f3038069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30380678c0 0x7f3038069d80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f304c00a8d0 tx=0x7f304c008040 comp rx=0 tx=0).stop 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581587a0 msgr2=0x7f30581533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581587a0 0x7f30581533d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f305001a630 tx=0x7f30500197a0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 shutdown_connections 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30380678c0 0x7f3038069d80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581587a0 0x7f30581533d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30580b37d0 0x7f305809a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 --2- 172.21.15.76:0/1597841402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806e680 0x7f3058150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 >> 172.21.15.76:0/1597841402 conn(0x7f30580b1f90 msgr2=0x7f30580ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:18.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 shutdown_connections 2024-09-19T04:14:18.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.397+0000 7f305cc0f640 1 -- 172.21.15.76:0/1597841402 wait complete. 2024-09-19T04:14:18.414 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:14:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a91f8640 1 Processor -- start 2024-09-19T04:14:18.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a91f8640 1 -- start start 2024-09-19T04:14:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a91f8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01a414ba40 0x7f01a40b5510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a91f8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4162c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a91f8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a406d4b0 con 0x7f01a414b3f0 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a91f8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f01a406e0c0 con 0x7f01a414ba40 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a91f8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f01a406d790 con 0x7f01a40b5a50 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01a414ba40 0x7f01a40b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01a414ba40 0x7f01a40b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46738/0 (socket says 172.21.15.76:46738) 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a259c640 1 -- 172.21.15.76:0/2654789122 learned_addr learned my addr 172.21.15.76:0/2654789122 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a2d9d640 1 --1- 172.21.15.76:0/2654789122 >> v1:172.21.15.76:6789/0 conn(0x7f01a414b3f0 0x7f01a4166c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48324/0 (socket says 172.21.15.76:48324) 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a359e640 1 --2- 172.21.15.76:0/2654789122 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4162c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a259c640 1 -- 172.21.15.76:0/2654789122 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 msgr2=0x7f01a4162c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a259c640 1 --2- 172.21.15.76:0/2654789122 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4162c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a259c640 1 -- 172.21.15.76:0/2654789122 >> v1:172.21.15.76:6789/0 conn(0x7f01a414b3f0 legacy=0x7f01a4166c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a259c640 1 -- 172.21.15.76:0/2654789122 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a406da70 con 0x7f01a414ba40 2024-09-19T04:14:18.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a1d9b640 1 -- 172.21.15.76:0/2654789122 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3502374751 0 0) 0x7f01a406d4b0 con 0x7f01a414b3f0 2024-09-19T04:14:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.521+0000 7f01a359e640 1 --2- 172.21.15.76:0/2654789122 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a259c640 1 --2- 172.21.15.76:0/2654789122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a40b5510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0198009870 tx=0x7f019801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee4b671d77b7112 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a1d9b640 1 -- 172.21.15.76:0/2654789122 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01980b2070 con 0x7f01a414ba40 2024-09-19T04:14:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a1d9b640 1 -- 172.21.15.76:0/2654789122 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01980a9070 con 0x7f01a414ba40 2024-09-19T04:14:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a1d9b640 1 -- 172.21.15.76:0/2654789122 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01980ad880 con 0x7f01a414ba40 2024-09-19T04:14:18.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 -- 172.21.15.76:0/2654789122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 msgr2=0x7f01a40b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 --2- 172.21.15.76:0/2654789122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a40b5510 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0198009870 tx=0x7f019801c560 comp rx=0 tx=0).stop 2024-09-19T04:14:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 -- 172.21.15.76:0/2654789122 shutdown_connections 2024-09-19T04:14:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 --2- 172.21.15.76:0/2654789122 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4162c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 --2- 172.21.15.76:0/2654789122 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a40b5510 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 -- 172.21.15.76:0/2654789122 >> 172.21.15.76:0/2654789122 conn(0x7f01a40b3e70 msgr2=0x7f01a40b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 -- 172.21.15.76:0/2654789122 shutdown_connections 2024-09-19T04:14:18.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 -- 172.21.15.76:0/2654789122 wait complete. 2024-09-19T04:14:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.525+0000 7f01a91f8640 1 Processor -- start 2024-09-19T04:14:18.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 -- start start 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4160460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01a414b3f0 0x7f01a4145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a4145cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01a416f410 con 0x7f01a414ba40 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01a4163c70 con 0x7f01a414b3f0 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f01a416f590 con 0x7f01a40b5a50 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a4145cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a4145cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46740/0 (socket says 172.21.15.76:46740) 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 -- 172.21.15.76:0/1239142811 learned_addr learned my addr 172.21.15.76:0/1239142811 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a2d9d640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4160460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a259c640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01a414b3f0 0x7f01a4145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 -- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 msgr2=0x7f01a4160460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4160460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 -- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01a414b3f0 msgr2=0x7f01a4145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01a414b3f0 0x7f01a4145770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 -- 172.21.15.76:0/1239142811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a41622d0 con 0x7f01a414ba40 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a2d9d640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a359e640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a4145cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f019400c7f0 tx=0x7f019400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f018b7fe640 1 -- 172.21.15.76:0/1239142811 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194015070 con 0x7f01a414ba40 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f018b7fe640 1 -- 172.21.15.76:0/1239142811 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0194007860 con 0x7f01a414ba40 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a41615d0 con 0x7f01a414ba40 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f018b7fe640 1 -- 172.21.15.76:0/1239142811 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194007c50 con 0x7f01a414ba40 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a4161dd0 con 0x7f01a414ba40 2024-09-19T04:14:18.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a4068930 con 0x7f01a414ba40 2024-09-19T04:14:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f018b7fe640 1 -- 172.21.15.76:0/1239142811 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0194018070 con 0x7f01a414ba40 2024-09-19T04:14:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f018b7fe640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01900678f0 0x7f0190069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f01a2d9d640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01900678f0 0x7f0190069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.529+0000 7f018b7fe640 1 -- 172.21.15.76:0/1239142811 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f0194085d20 con 0x7f01a414ba40 2024-09-19T04:14:18.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.533+0000 7f01a2d9d640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01900678f0 0x7f0190069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f019c007fd0 tx=0x7f019c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.533+0000 7f018b7fe640 1 -- 172.21.15.76:0/1239142811 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0194056760 con 0x7f01a414ba40 2024-09-19T04:14:18.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.685+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01a4162c90 con 0x7f01a414ba40 2024-09-19T04:14:18.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.685+0000 7f018b7fe640 1 -- 172.21.15.76:0/1239142811 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+17039 (secure 0 0 0) 0x7f0194056060 con 0x7f01a414ba40 2024-09-19T04:14:18.688 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:18.688 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":38,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:14:17.425964+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:14:15.256172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T04:14:16.419970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:14:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01900678f0 msgr2=0x7f0190069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01900678f0 0x7f0190069db0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f019c007fd0 tx=0x7f019c009040 comp rx=0 tx=0).stop 2024-09-19T04:14:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 msgr2=0x7f01a4145cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a4145cb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f019400c7f0 tx=0x7f019400ccb0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 shutdown_connections 2024-09-19T04:14:18.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01900678f0 0x7f0190069db0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01a414ba40 0x7f01a4145cb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01a414b3f0 0x7f01a4145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 --2- 172.21.15.76:0/1239142811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01a40b5a50 0x7f01a4160460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 >> 172.21.15.76:0/1239142811 conn(0x7f01a40b3e70 msgr2=0x7f01a4140e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 shutdown_connections 2024-09-19T04:14:18.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.689+0000 7f01a91f8640 1 -- 172.21.15.76:0/1239142811 wait complete. 2024-09-19T04:14:18.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:14:18.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31b53e2640 1 Processor -- start 2024-09-19T04:14:18.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31b53e2640 1 -- start start 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31b53e2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31b0070020 0x7f31b015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31b53e2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f31b015f1d0 0x7f31b00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31b53e2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31b00aede0 con 0x7f31b01520a0 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31b53e2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f31b0152480 con 0x7f31b0070020 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31b53e2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f31b015fa10 con 0x7f31b015f1d0 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31ae7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31b0070020 0x7f31b015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31ae7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31b0070020 0x7f31b015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46742/0 (socket says 172.21.15.76:46742) 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31ae7fc640 1 -- 172.21.15.76:0/1953963981 learned_addr learned my addr 172.21.15.76:0/1953963981 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31af7fe640 1 --2- 172.21.15.76:0/1953963981 >> v2:172.21.15.116:3300/0 conn(0x7f31b015f1d0 0x7f31b00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31ae7fc640 1 -- 172.21.15.76:0/1953963981 >> v2:172.21.15.116:3300/0 conn(0x7f31b015f1d0 msgr2=0x7f31b00adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:18.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31ae7fc640 1 --2- 172.21.15.76:0/1953963981 >> v2:172.21.15.116:3300/0 conn(0x7f31b015f1d0 0x7f31b00adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31ae7fc640 1 -- 172.21.15.76:0/1953963981 >> v1:172.21.15.116:6789/0 conn(0x7f31b01520a0 legacy=0x7f31b006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:18.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.817+0000 7f31ae7fc640 1 -- 172.21.15.76:0/1953963981 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b0074250 con 0x7f31b0070020 2024-09-19T04:14:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31ae7fc640 1 --2- 172.21.15.76:0/1953963981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b0070020 0x7f31b015ec90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f31b00d0250 tx=0x7f319801eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffec28ab53175cb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31adffb640 1 -- 172.21.15.76:0/1953963981 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31980b4070 con 0x7f31b0070020 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31adffb640 1 -- 172.21.15.76:0/1953963981 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31980a7c50 con 0x7f31b0070020 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31adffb640 1 -- 172.21.15.76:0/1953963981 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31980af440 con 0x7f31b0070020 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- 172.21.15.76:0/1953963981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b0070020 msgr2=0x7f31b015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 --2- 172.21.15.76:0/1953963981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b0070020 0x7f31b015ec90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f31b00d0250 tx=0x7f319801eb80 comp rx=0 tx=0).stop 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- 172.21.15.76:0/1953963981 shutdown_connections 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 --2- 172.21.15.76:0/1953963981 >> v2:172.21.15.116:3300/0 conn(0x7f31b015f1d0 0x7f31b00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 --2- 172.21.15.76:0/1953963981 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b0070020 0x7f31b015ec90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- 172.21.15.76:0/1953963981 >> 172.21.15.76:0/1953963981 conn(0x7f31b00b1f90 msgr2=0x7f31b00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- 172.21.15.76:0/1953963981 shutdown_connections 2024-09-19T04:14:18.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- 172.21.15.76:0/1953963981 wait complete. 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 Processor -- start 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- start start 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31b0070020 0x7f31b0145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31b01520a0 0x7f31b0150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b015f1d0 0x7f31b0145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f31b015f5e0 con 0x7f31b015f1d0 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f31b0073d30 con 0x7f31b0070020 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f31b015f760 con 0x7f31b01520a0 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b015f1d0 0x7f31b0145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b015f1d0 0x7f31b0145e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46752/0 (socket says 172.21.15.76:46752) 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 -- 172.21.15.76:0/4031697011 learned_addr learned my addr 172.21.15.76:0/4031697011 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31ae7fc640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31b01520a0 0x7f31b0150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31aeffd640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31b0070020 0x7f31b0145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 -- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31b01520a0 msgr2=0x7f31b0150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31b01520a0 0x7f31b0150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 -- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31b0070020 msgr2=0x7f31b0145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31b0070020 0x7f31b0145610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 -- 172.21.15.76:0/4031697011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b014aa20 con 0x7f31b015f1d0 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31ae7fc640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31b01520a0 0x7f31b0150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31af7fe640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b015f1d0 0x7f31b0145e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f31a0007e40 tx=0x7f31a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31a77fe640 1 -- 172.21.15.76:0/4031697011 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0014620 con 0x7f31b015f1d0 2024-09-19T04:14:18.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31a77fe640 1 -- 172.21.15.76:0/4031697011 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a0019070 con 0x7f31b015f1d0 2024-09-19T04:14:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b014ff90 con 0x7f31b015f1d0 2024-09-19T04:14:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31a77fe640 1 -- 172.21.15.76:0/4031697011 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0014a20 con 0x7f31b015f1d0 2024-09-19T04:14:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.821+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b0150550 con 0x7f31b015f1d0 2024-09-19T04:14:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.825+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b015ec90 con 0x7f31b015f1d0 2024-09-19T04:14:18.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.825+0000 7f31a77fe640 1 -- 172.21.15.76:0/4031697011 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f31a0014bc0 con 0x7f31b015f1d0 2024-09-19T04:14:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.825+0000 7f31a77fe640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31900678c0 0x7f3190069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.825+0000 7f31a77fe640 1 -- 172.21.15.76:0/4031697011 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 6101+0+0 (secure 0 0 0) 0x7f31a0086650 con 0x7f31b015f1d0 2024-09-19T04:14:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.825+0000 7f31aeffd640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31900678c0 0x7f3190069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:18.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.825+0000 7f31aeffd640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31900678c0 0x7f3190069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31a8007fd0 tx=0x7f31a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:18.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:18.829+0000 7f31a77fe640 1 -- 172.21.15.76:0/4031697011 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f31a0057090 con 0x7f31b015f1d0 2024-09-19T04:14:19.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.009+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f31b006e540 con 0x7f31b015f1d0 2024-09-19T04:14:19.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.085+0000 7f31a77fe640 1 -- 172.21.15.76:0/4031697011 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) ==== 118+0+0 (secure 0 0 0) 0x7f31a0056990 con 0x7f31b015f1d0 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31900678c0 msgr2=0x7f3190069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31900678c0 0x7f3190069d80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31a8007fd0 tx=0x7f31a8009040 comp rx=0 tx=0).stop 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b015f1d0 msgr2=0x7f31b0145e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b015f1d0 0x7f31b0145e40 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f31a0007e40 tx=0x7f31a000e4b0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 shutdown_connections 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31900678c0 0x7f3190069d80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31b015f1d0 0x7f31b0145e40 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31b01520a0 0x7f31b0150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 --2- 172.21.15.76:0/4031697011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31b0070020 0x7f31b0145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 >> 172.21.15.76:0/4031697011 conn(0x7f31b00b1f90 msgr2=0x7f31b015b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:19.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 shutdown_connections 2024-09-19T04:14:19.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.089+0000 7f31b53e2640 1 -- 172.21.15.76:0/4031697011 wait complete. 2024-09-19T04:14:19.106 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T04:14:19.107 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T04:14:19.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.217+0000 7f2033133640 1 Processor -- start 2024-09-19T04:14:19.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.217+0000 7f2033133640 1 -- start start 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f2033133640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f202c160480 0x7f202c06d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f2033133640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f202c072220 0x7f202c06d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f2033133640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f202c06e060 con 0x7f202c071730 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f2033133640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f202c15efe0 con 0x7f202c160480 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f2033133640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f202c06e340 con 0x7f202c072220 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f202c160480 0x7f202c06d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f2030ecf640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f202c071730 0x7f202c071b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48332/0 (socket says 172.21.15.76:48332) 2024-09-19T04:14:19.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f2030ecf640 1 -- 172.21.15.76:0/4157083913 learned_addr learned my addr 172.21.15.76:0/4157083913 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:19.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202bfff640 1 -- 172.21.15.76:0/4157083913 >> v2:172.21.15.116:3300/0 conn(0x7f202c072220 msgr2=0x7f202c06d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:19.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202bfff640 1 --2- 172.21.15.76:0/4157083913 >> v2:172.21.15.116:3300/0 conn(0x7f202c072220 0x7f202c06d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202bfff640 1 -- 172.21.15.76:0/4157083913 >> v1:172.21.15.76:6789/0 conn(0x7f202c071730 legacy=0x7f202c071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202bfff640 1 -- 172.21.15.76:0/4157083913 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c15e990 con 0x7f202c160480 2024-09-19T04:14:19.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202b7fe640 1 -- 172.21.15.76:0/4157083913 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 598746690 0 0) 0x7f202c06e060 con 0x7f202c071730 2024-09-19T04:14:19.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202bfff640 1 --2- 172.21.15.76:0/4157083913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c160480 0x7f202c06d2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20180097c0 tx=0x7f201801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84446bd1bef5b246 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:19.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202b7fe640 1 -- 172.21.15.76:0/4157083913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180ab610 con 0x7f202c160480 2024-09-19T04:14:19.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202b7fe640 1 -- 172.21.15.76:0/4157083913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180b0070 con 0x7f202c160480 2024-09-19T04:14:19.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.221+0000 7f202b7fe640 1 -- 172.21.15.76:0/4157083913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20180ab9d0 con 0x7f202c160480 2024-09-19T04:14:19.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/4157083913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c160480 msgr2=0x7f202c06d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 --2- 172.21.15.76:0/4157083913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c160480 0x7f202c06d2f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20180097c0 tx=0x7f201801c430 comp rx=0 tx=0).stop 2024-09-19T04:14:19.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/4157083913 shutdown_connections 2024-09-19T04:14:19.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 --2- 172.21.15.76:0/4157083913 >> v2:172.21.15.116:3300/0 conn(0x7f202c072220 0x7f202c06d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 --2- 172.21.15.76:0/4157083913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c160480 0x7f202c06d2f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/4157083913 >> 172.21.15.76:0/4157083913 conn(0x7f202c0b3e70 msgr2=0x7f202c0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:19.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/4157083913 shutdown_connections 2024-09-19T04:14:19.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/4157083913 wait complete. 2024-09-19T04:14:19.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 Processor -- start 2024-09-19T04:14:19.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- start start 2024-09-19T04:14:19.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f202c071730 0x7f202c157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c072220 0x7f202c15f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f202c160480 0x7f202c15fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f202c16a640 con 0x7f202c072220 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f202c148110 con 0x7f202c071730 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f202c16a7c0 con 0x7f202c160480 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c072220 0x7f202c15f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c072220 0x7f202c15f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46764/0 (socket says 172.21.15.76:46764) 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 -- 172.21.15.76:0/2770704353 learned_addr learned my addr 172.21.15.76:0/2770704353 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 -- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f202c160480 msgr2=0x7f202c15fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2030ecf640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f202c071730 0x7f202c157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f20316d0640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f202c160480 0x7f202c15fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f202c160480 0x7f202c15fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 -- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f202c071730 msgr2=0x7f202c157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f202c071730 0x7f202c157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:19.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 -- 172.21.15.76:0/2770704353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202c13a380 con 0x7f202c072220 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f20316d0640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f202c160480 0x7f202c15fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f202bfff640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c072220 0x7f202c15f4e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f20180097c0 tx=0x7f201801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f20297fa640 1 -- 172.21.15.76:0/2770704353 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018004220 con 0x7f202c072220 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f20297fa640 1 -- 172.21.15.76:0/2770704353 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20180043c0 con 0x7f202c072220 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202c13a660 con 0x7f202c072220 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f20297fa640 1 -- 172.21.15.76:0/2770704353 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2018003820 con 0x7f202c072220 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f202c1383d0 con 0x7f202c072220 2024-09-19T04:14:19.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.225+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f202c071b90 con 0x7f202c072220 2024-09-19T04:14:19.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.229+0000 7f20297fa640 1 -- 172.21.15.76:0/2770704353 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f20180039c0 con 0x7f202c072220 2024-09-19T04:14:19.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.229+0000 7f20297fa640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20100678c0 0x7f2010069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:19.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.229+0000 7f20297fa640 1 -- 172.21.15.76:0/2770704353 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f20181206b0 con 0x7f202c072220 2024-09-19T04:14:19.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.229+0000 7f2030ecf640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20100678c0 0x7f2010069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:19.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.229+0000 7f2030ecf640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20100678c0 0x7f2010069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2020000f70 tx=0x7f2020009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:19.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.233+0000 7f20297fa640 1 -- 172.21.15.76:0/2770704353 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f20180a8050 con 0x7f202c072220 2024-09-19T04:14:19.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:19.417+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f202c0b3840 con 0x7f202c072220 2024-09-19T04:14:20.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.093+0000 7f20297fa640 1 -- 172.21.15.76:0/2770704353 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) ==== 122+0+0 (secure 0 0 0) 0x7f2018016050 con 0x7f202c072220 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20100678c0 msgr2=0x7f2010069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20100678c0 0x7f2010069d80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2020000f70 tx=0x7f2020009040 comp rx=0 tx=0).stop 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c072220 msgr2=0x7f202c15f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c072220 0x7f202c15f4e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f20180097c0 tx=0x7f201801ac70 comp rx=0 tx=0).stop 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 shutdown_connections 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20100678c0 0x7f2010069d80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f202c160480 0x7f202c15fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f202c072220 0x7f202c15f4e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 --2- 172.21.15.76:0/2770704353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f202c071730 0x7f202c157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 >> 172.21.15.76:0/2770704353 conn(0x7f202c0b3e70 msgr2=0x7f202c13dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:20.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 shutdown_connections 2024-09-19T04:14:20.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.097+0000 7f2033133640 1 -- 172.21.15.76:0/2770704353 wait complete. 2024-09-19T04:14:20.118 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T04:14:20.118 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:20.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaad310640 1 Processor -- start 2024-09-19T04:14:20.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaad310640 1 -- start start 2024-09-19T04:14:20.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaad310640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdaa815c2a0 0x7fdaa80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaad310640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa80af2a0 0x7fdaa806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaad310640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdaa80b3290 con 0x7fdaa815bec0 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaad310640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdaa80b3570 con 0x7fdaa815c2a0 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaad310640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa80b3850 con 0x7fdaa80af2a0 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdaa815bec0 0x7fdaa80ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48342/0 (socket says 172.21.15.76:48342) 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa6ffd640 1 -- 172.21.15.76:0/2235675802 learned_addr learned my addr 172.21.15.76:0/2235675802 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa67fc640 1 --2- 172.21.15.76:0/2235675802 >> v2:172.21.15.76:3300/0 conn(0x7fdaa815c2a0 0x7fdaa80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa77fe640 1 --2- 172.21.15.76:0/2235675802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa80af2a0 0x7fdaa806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa5ffb640 1 -- 172.21.15.76:0/2235675802 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2061569033 0 0) 0x7fdaa80b3290 con 0x7fdaa815bec0 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa5ffb640 1 -- 172.21.15.76:0/2235675802 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda90003530 con 0x7fdaa815bec0 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa67fc640 1 -- 172.21.15.76:0/2235675802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa80af2a0 msgr2=0x7fdaa806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa67fc640 1 --2- 172.21.15.76:0/2235675802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa80af2a0 0x7fdaa806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa67fc640 1 -- 172.21.15.76:0/2235675802 >> v1:172.21.15.76:6789/0 conn(0x7fdaa815bec0 legacy=0x7fdaa80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa67fc640 1 -- 172.21.15.76:0/2235675802 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa80b3b30 con 0x7fdaa815c2a0 2024-09-19T04:14:20.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.245+0000 7fdaa77fe640 1 --2- 172.21.15.76:0/2235675802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa80af2a0 0x7fdaa806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:20.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaa67fc640 1 --2- 172.21.15.76:0/2235675802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815c2a0 0x7fdaa80aed60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0097c0 tx=0x7fda9c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ab9b519c16ec39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaa5ffb640 1 -- 172.21.15.76:0/2235675802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0a7070 con 0x7fdaa815c2a0 2024-09-19T04:14:20.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaa5ffb640 1 -- 172.21.15.76:0/2235675802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c0b0070 con 0x7fdaa815c2a0 2024-09-19T04:14:20.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaa5ffb640 1 -- 172.21.15.76:0/2235675802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0ab890 con 0x7fdaa815c2a0 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- 172.21.15.76:0/2235675802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815c2a0 msgr2=0x7fdaa80aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 --2- 172.21.15.76:0/2235675802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815c2a0 0x7fdaa80aed60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0097c0 tx=0x7fda9c01c470 comp rx=0 tx=0).stop 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- 172.21.15.76:0/2235675802 shutdown_connections 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 --2- 172.21.15.76:0/2235675802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa80af2a0 0x7fdaa806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 --2- 172.21.15.76:0/2235675802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815c2a0 0x7fdaa80aed60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- 172.21.15.76:0/2235675802 >> 172.21.15.76:0/2235675802 conn(0x7fdaa80b1f90 msgr2=0x7fdaa80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- 172.21.15.76:0/2235675802 shutdown_connections 2024-09-19T04:14:20.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- 172.21.15.76:0/2235675802 wait complete. 2024-09-19T04:14:20.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 Processor -- start 2024-09-19T04:14:20.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- start start 2024-09-19T04:14:20.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaa80af2a0 0x7fdaa814e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815bec0 0x7fdaa814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa815c2a0 0x7fdaa814be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa806ed50 con 0x7fdaa815bec0 2024-09-19T04:14:20.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa80af6b0 con 0x7fdaa80af2a0 2024-09-19T04:14:20.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaad310640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdaa806eed0 con 0x7fdaa815c2a0 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.249+0000 7fdaa67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815bec0 0x7fdaa814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815bec0 0x7fdaa814b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46778/0 (socket says 172.21.15.76:46778) 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 -- 172.21.15.76:0/2443398505 learned_addr learned my addr 172.21.15.76:0/2443398505 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa77fe640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa815c2a0 0x7fdaa814be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 -- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa815c2a0 msgr2=0x7fdaa814be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa815c2a0 0x7fdaa814be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 -- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaa80af2a0 msgr2=0x7fdaa814e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaa80af2a0 0x7fdaa814e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 -- 172.21.15.76:0/2443398505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa8072ac0 con 0x7fdaa815bec0 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa77fe640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa815c2a0 0x7fdaa814be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:20.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa67fc640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815bec0 0x7fdaa814b940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fda9c01ab20 tx=0x7fda9c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fda8f7fe640 1 -- 172.21.15.76:0/2443398505 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0abd20 con 0x7fdaa815bec0 2024-09-19T04:14:20.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fda8f7fe640 1 -- 172.21.15.76:0/2443398505 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda9c0bf450 con 0x7fdaa815bec0 2024-09-19T04:14:20.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa8072d60 con 0x7fdaa815bec0 2024-09-19T04:14:20.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fda8f7fe640 1 -- 172.21.15.76:0/2443398505 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c0bf840 con 0x7fdaa815bec0 2024-09-19T04:14:20.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa8073320 con 0x7fdaa815bec0 2024-09-19T04:14:20.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa806e000 con 0x7fdaa815bec0 2024-09-19T04:14:20.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fda8f7fe640 1 -- 172.21.15.76:0/2443398505 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fda9c0bf9e0 con 0x7fdaa815bec0 2024-09-19T04:14:20.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fda8f7fe640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda940678c0 0x7fda94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa6ffd640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda940678c0 0x7fda94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fda8f7fe640 1 -- 172.21.15.76:0/2443398505 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fda9c120a90 con 0x7fdaa815bec0 2024-09-19T04:14:20.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.253+0000 7fdaa6ffd640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda940678c0 0x7fda94069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda98007fd0 tx=0x7fda98007900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.257+0000 7fda8f7fe640 1 -- 172.21.15.76:0/2443398505 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fda9c0c4060 con 0x7fdaa815bec0 2024-09-19T04:14:20.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.449+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaa806fcd0 con 0x7fdaa815bec0 2024-09-19T04:14:20.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.449+0000 7fda8f7fe640 1 -- 172.21.15.76:0/2443398505 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5229 (secure 0 0 0) 0x7fda9c016090 con 0x7fdaa815bec0 2024-09-19T04:14:20.451 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"btime":"2024-09-19T04:14:20:094237+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:19.421274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4863]},"id":2}]} 2024-09-19T04:14:20.451 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-09-19T04:14:20.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda940678c0 msgr2=0x7fda94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda940678c0 0x7fda94069d80 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fda98007fd0 tx=0x7fda98007900 comp rx=0 tx=0).stop 2024-09-19T04:14:20.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815bec0 msgr2=0x7fdaa814b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815bec0 0x7fdaa814b940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fda9c01ab20 tx=0x7fda9c019210 comp rx=0 tx=0).stop 2024-09-19T04:14:20.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 shutdown_connections 2024-09-19T04:14:20.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda940678c0 0x7fda94069d80 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdaa815c2a0 0x7fdaa814be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdaa815bec0 0x7fdaa814b940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 --2- 172.21.15.76:0/2443398505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdaa80af2a0 0x7fdaa814e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 >> 172.21.15.76:0/2443398505 conn(0x7fdaa80b1f90 msgr2=0x7fdaa81597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:20.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 shutdown_connections 2024-09-19T04:14:20.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.453+0000 7fdaad310640 1 -- 172.21.15.76:0/2443398505 wait complete. 2024-09-19T04:14:20.468 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:14:20.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f3251a03640 1 Processor -- start 2024-09-19T04:14:20.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f3251a03640 1 -- start start 2024-09-19T04:14:20.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f3251a03640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f3251a03640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f324c0cf450 0x7f324c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f3251a03640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f324c0d0300 con 0x7f324c161a60 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f3251a03640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f324c0b0f60 con 0x7f324c0cf450 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f3251a03640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f324c0cfd40 con 0x7f324c0d3060 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f324a7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f324a7fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:42634/0 (socket says 172.21.15.76:42634) 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f324a7fc640 1 -- 172.21.15.76:0/1499789330 learned_addr learned my addr 172.21.15.76:0/1499789330 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:20.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.581+0000 7f324b7fe640 1 --2- 172.21.15.76:0/1499789330 >> v2:172.21.15.116:3300/0 conn(0x7f324c0cf450 0x7f324c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f324a7fc640 1 -- 172.21.15.76:0/1499789330 >> v2:172.21.15.116:3300/0 conn(0x7f324c0cf450 msgr2=0x7f324c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:20.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f324a7fc640 1 --2- 172.21.15.76:0/1499789330 >> v2:172.21.15.116:3300/0 conn(0x7f324c0cf450 0x7f324c06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f324a7fc640 1 -- 172.21.15.76:0/1499789330 >> v1:172.21.15.116:6789/0 conn(0x7f324c161a60 legacy=0x7f324c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f324a7fc640 1 -- 172.21.15.76:0/1499789330 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c0d0020 con 0x7f324c0d3060 2024-09-19T04:14:20.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f324a7fc640 1 --2- 172.21.15.76:0/1499789330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c0cef10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3240009870 tx=0x7f324001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83b55f2862214c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f3249ffb640 1 -- 172.21.15.76:0/1499789330 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400aa600 con 0x7f324c0d3060 2024-09-19T04:14:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f3249ffb640 1 -- 172.21.15.76:0/1499789330 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32400af070 con 0x7f324c0d3060 2024-09-19T04:14:20.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f3249ffb640 1 -- 172.21.15.76:0/1499789330 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400aa9c0 con 0x7f324c0d3060 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f3251a03640 1 -- 172.21.15.76:0/1499789330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 msgr2=0x7f324c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.585+0000 7f3251a03640 1 --2- 172.21.15.76:0/1499789330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c0cef10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3240009870 tx=0x7f324001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- 172.21.15.76:0/1499789330 shutdown_connections 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 --2- 172.21.15.76:0/1499789330 >> v2:172.21.15.116:3300/0 conn(0x7f324c0cf450 0x7f324c06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 --2- 172.21.15.76:0/1499789330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c0cef10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- 172.21.15.76:0/1499789330 >> 172.21.15.76:0/1499789330 conn(0x7f324c08bf90 msgr2=0x7f324c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- 172.21.15.76:0/1499789330 shutdown_connections 2024-09-19T04:14:20.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- 172.21.15.76:0/1499789330 wait complete. 2024-09-19T04:14:20.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 Processor -- start 2024-09-19T04:14:20.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- start start 2024-09-19T04:14:20.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f324c0cf450 0x7f324c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f324c161a60 0x7f324c15cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f324c16af20 con 0x7f324c161a60 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f324c06e610 con 0x7f324c0cf450 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f324c16b0a0 con 0x7f324c0d3060 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f324c161a60 0x7f324c15cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f324c161a60 0x7f324c15cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46790/0 (socket says 172.21.15.76:46790) 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 -- 172.21.15.76:0/3452433164 learned_addr learned my addr 172.21.15.76:0/3452433164 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324a7fc640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324affd640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f324c0cf450 0x7f324c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 -- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 msgr2=0x7f324c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 -- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f324c0cf450 msgr2=0x7f324c15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f324c0cf450 0x7f324c15b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 -- 172.21.15.76:0/3452433164 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324c0caba0 con 0x7f324c161a60 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f324b7fe640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f324c161a60 0x7f324c15cf20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e820 tx=0x7f323c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3233fff640 1 -- 172.21.15.76:0/3452433164 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c017070 con 0x7f324c161a60 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3233fff640 1 -- 172.21.15.76:0/3452433164 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f323c00c940 con 0x7f324c161a60 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3233fff640 1 -- 172.21.15.76:0/3452433164 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f323c00cd10 con 0x7f324c161a60 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.589+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f324c0cae80 con 0x7f324c161a60 2024-09-19T04:14:20.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.593+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f324c152fb0 con 0x7f324c161a60 2024-09-19T04:14:20.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.593+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324c06d6b0 con 0x7f324c161a60 2024-09-19T04:14:20.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.593+0000 7f3233fff640 1 -- 172.21.15.76:0/3452433164 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f323c01a070 con 0x7f324c161a60 2024-09-19T04:14:20.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.593+0000 7f3233fff640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f32380678c0 0x7f3238069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.593+0000 7f3233fff640 1 -- 172.21.15.76:0/3452433164 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f323c085fc0 con 0x7f324c161a60 2024-09-19T04:14:20.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.593+0000 7f324affd640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f32380678c0 0x7f3238069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.593+0000 7f324affd640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f32380678c0 0x7f3238069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f324c15b300 tx=0x7f3244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.597+0000 7f3233fff640 1 -- 172.21.15.76:0/3452433164 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f323c056a00 con 0x7f324c161a60 2024-09-19T04:14:20.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.745+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f324c15b330 con 0x7f324c161a60 2024-09-19T04:14:20.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3233fff640 1 -- 172.21.15.76:0/3452433164 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) ==== 74+0+17026 (secure 0 0 0) 0x7f323c056300 con 0x7f324c161a60 2024-09-19T04:14:20.750 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:20.750 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:14:19.054094+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:14:15.256172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T04:14:16.419970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:14:20.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f32380678c0 msgr2=0x7f3238069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f32380678c0 0x7f3238069d80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f324c15b300 tx=0x7f3244009040 comp rx=0 tx=0).stop 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f324c161a60 msgr2=0x7f324c15cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f324c161a60 0x7f324c15cf20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f323c00e820 tx=0x7f323c00ece0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 shutdown_connections 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f32380678c0 0x7f3238069d80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f324c161a60 0x7f324c15cf20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f324c0d3060 0x7f324c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 --2- 172.21.15.76:0/3452433164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f324c0cf450 0x7f324c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 >> 172.21.15.76:0/3452433164 conn(0x7f324c08bf90 msgr2=0x7f324c0b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 shutdown_connections 2024-09-19T04:14:20.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.749+0000 7f3251a03640 1 -- 172.21.15.76:0/3452433164 wait complete. 2024-09-19T04:14:20.767 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:14:20.767 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:14:20.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f32f2640 1 Processor -- start 2024-09-19T04:14:20.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f32f2640 1 -- start start 2024-09-19T04:14:20.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f32f2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f32f2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f02ec0af470 0x7f02ec06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f32f2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02ec0b3290 con 0x7f02ec15c2a0 2024-09-19T04:14:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f32f2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f02ec0b3570 con 0x7f02ec0af470 2024-09-19T04:14:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f32f2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec0b3850 con 0x7f02ec15bec0 2024-09-19T04:14:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f108e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f188f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f02ec0af470 0x7f02ec06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f108e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:42654/0 (socket says 172.21.15.76:42654) 2024-09-19T04:14:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f108e640 1 -- 172.21.15.76:0/3433078857 learned_addr learned my addr 172.21.15.76:0/3433078857 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f088d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f02ec15c2a0 0x7f02ec0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:41856/0 (socket says 172.21.15.76:41856) 2024-09-19T04:14:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f108e640 1 -- 172.21.15.76:0/3433078857 >> v2:172.21.15.116:3300/0 conn(0x7f02ec0af470 msgr2=0x7f02ec06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f108e640 1 --2- 172.21.15.76:0/3433078857 >> v2:172.21.15.116:3300/0 conn(0x7f02ec0af470 0x7f02ec06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f108e640 1 -- 172.21.15.76:0/3433078857 >> v1:172.21.15.116:6789/0 conn(0x7f02ec15c2a0 legacy=0x7f02ec0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.872+0000 7f02f108e640 1 -- 172.21.15.76:0/3433078857 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec0b3b30 con 0x7f02ec15bec0 2024-09-19T04:14:20.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f108e640 1 --2- 172.21.15.76:0/3433078857 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec0ad200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009870 tx=0x7f02e401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e10088d2502ebba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02ebfff640 1 -- 172.21.15.76:0/3433078857 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40a95f0 con 0x7f02ec15bec0 2024-09-19T04:14:20.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02ebfff640 1 -- 172.21.15.76:0/3433078857 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e40a9790 con 0x7f02ec15bec0 2024-09-19T04:14:20.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02ebfff640 1 -- 172.21.15.76:0/3433078857 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40a9b60 con 0x7f02ec15bec0 2024-09-19T04:14:20.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 -- 172.21.15.76:0/3433078857 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 msgr2=0x7f02ec0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3433078857 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec0ad200 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f02e4009870 tx=0x7f02e401c530 comp rx=0 tx=0).stop 2024-09-19T04:14:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 -- 172.21.15.76:0/3433078857 shutdown_connections 2024-09-19T04:14:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3433078857 >> v2:172.21.15.116:3300/0 conn(0x7f02ec0af470 0x7f02ec06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3433078857 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec0ad200 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 -- 172.21.15.76:0/3433078857 >> 172.21.15.76:0/3433078857 conn(0x7f02ec0b1f90 msgr2=0x7f02ec0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 -- 172.21.15.76:0/3433078857 shutdown_connections 2024-09-19T04:14:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 -- 172.21.15.76:0/3433078857 wait complete. 2024-09-19T04:14:20.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 Processor -- start 2024-09-19T04:14:20.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.876+0000 7f02f32f2640 1 -- start start 2024-09-19T04:14:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02ec0af470 0x7f02ec0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec1550a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02ec15c2a0 0x7f02ec1555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec06ed50 con 0x7f02ec0af470 2024-09-19T04:14:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f02ec0ad740 con 0x7f02ec15c2a0 2024-09-19T04:14:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f02ec06eed0 con 0x7f02ec15bec0 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02ec0af470 0x7f02ec0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02ec0af470 0x7f02ec0b0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46800/0 (socket says 172.21.15.76:46800) 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 -- 172.21.15.76:0/3334575808 learned_addr learned my addr 172.21.15.76:0/3334575808 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f088d640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec1550a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f188f640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02ec15c2a0 0x7f02ec1555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 -- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 msgr2=0x7f02ec1550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec1550a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 -- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02ec15c2a0 msgr2=0x7f02ec1555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02ec15c2a0 0x7f02ec1555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 -- 172.21.15.76:0/3334575808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ec073490 con 0x7f02ec0af470 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f188f640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02ec15c2a0 0x7f02ec1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:20.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f108e640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02ec0af470 0x7f02ec0b0f00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f02e40ab040 tx=0x7f02e401ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02e9ffb640 1 -- 172.21.15.76:0/3334575808 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40b1800 con 0x7f02ec0af470 2024-09-19T04:14:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02e9ffb640 1 -- 172.21.15.76:0/3334575808 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02e40b19a0 con 0x7f02ec0af470 2024-09-19T04:14:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ec157750 con 0x7f02ec0af470 2024-09-19T04:14:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02e9ffb640 1 -- 172.21.15.76:0/3334575808 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e40b1d90 con 0x7f02ec0af470 2024-09-19T04:14:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ec157d10 con 0x7f02ec0af470 2024-09-19T04:14:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02ec06e000 con 0x7f02ec0af470 2024-09-19T04:14:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02e9ffb640 1 -- 172.21.15.76:0/3334575808 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f02e40a95b0 con 0x7f02ec0af470 2024-09-19T04:14:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02e9ffb640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02d80678c0 0x7f02d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02e9ffb640 1 -- 172.21.15.76:0/3334575808 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f02e4120c60 con 0x7f02ec0af470 2024-09-19T04:14:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f088d640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02d80678c0 0x7f02d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.880+0000 7f02f088d640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02d80678c0 0x7f02d8069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f02ec15b8c0 tx=0x7f02e0007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:20.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:20.884+0000 7f02e9ffb640 1 -- 172.21.15.76:0/3334575808 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f02e40cb080 con 0x7f02ec0af470 2024-09-19T04:14:21.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.032+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02ec14abf0 con 0x7f02d80678c0 2024-09-19T04:14:21.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.040+0000 7f02e9ffb640 1 -- 172.21.15.76:0/3334575808 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246639 (secure 0 0 0) 0x7f02ec14abf0 con 0x7f02d80678c0 2024-09-19T04:14:21.046 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:21.048 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:14:21.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02d80678c0 msgr2=0x7f02d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02d80678c0 0x7f02d8069d80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f02ec15b8c0 tx=0x7f02e0007cf0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02ec0af470 msgr2=0x7f02ec0b0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02ec0af470 0x7f02ec0b0f00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f02e40ab040 tx=0x7f02e401ad70 comp rx=0 tx=0).stop 2024-09-19T04:14:21.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 shutdown_connections 2024-09-19T04:14:21.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02d80678c0 0x7f02d8069d80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02ec15c2a0 0x7f02ec1555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f02ec15bec0 0x7f02ec1550a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 --2- 172.21.15.76:0/3334575808 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02ec0af470 0x7f02ec0b0f00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 >> 172.21.15.76:0/3334575808 conn(0x7f02ec0b1f90 msgr2=0x7f02ec0b4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 shutdown_connections 2024-09-19T04:14:21.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.044+0000 7f02f32f2640 1 -- 172.21.15.76:0/3334575808 wait complete. 2024-09-19T04:14:21.062 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-19T04:14:19.754180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229008,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745712,"statfs":{"total":773094113280,"available":772859609088,"internally_reserved":0,"allocated":4562944,"data_stored":2617280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001142"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065006+0000","last_change":"2024-09-19T04:14:17.065006+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065006+0000","last_undegraded":"2024-09-19T04:14:17.065006+0000","last_fullsized":"2024-09-19T04:14:17.065006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063769+0000","last_change":"2024-09-19T04:14:17.063769+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063769+0000","last_undegraded":"2024-09-19T04:14:17.063769+0000","last_fullsized":"2024-09-19T04:14:17.063769+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072719+0000","last_change":"2024-09-19T04:14:17.072719+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072719+0000","last_undegraded":"2024-09-19T04:14:17.072719+0000","last_fullsized":"2024-09-19T04:14:17.072719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066271+0000","last_change":"2024-09-19T04:14:17.066271+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066271+0000","last_undegraded":"2024-09-19T04:14:17.066271+0000","last_fullsized":"2024-09-19T04:14:17.066271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065750+0000","last_change":"2024-09-19T04:14:17.065750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065750+0000","last_undegraded":"2024-09-19T04:14:17.065750+0000","last_fullsized":"2024-09-19T04:14:17.065750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066325+0000","last_change":"2024-09-19T04:14:17.066325+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066325+0000","last_undegraded":"2024-09-19T04:14:17.066325+0000","last_fullsized":"2024-09-19T04:14:17.066325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068080+0000","last_change":"2024-09-19T04:14:17.068080+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068080+0000","last_undegraded":"2024-09-19T04:14:17.068080+0000","last_fullsized":"2024-09-19T04:14:17.068080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064742+0000","last_change":"2024-09-19T04:14:17.064742+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064742+0000","last_undegraded":"2024-09-19T04:14:17.064742+0000","last_fullsized":"2024-09-19T04:14:17.064742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073302+0000","last_change":"2024-09-19T04:14:17.073302+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073302+0000","last_undegraded":"2024-09-19T04:14:17.073302+0000","last_fullsized":"2024-09-19T04:14:17.073302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070827+0000","last_change":"2024-09-19T04:14:17.070827+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070827+0000","last_undegraded":"2024-09-19T04:14:17.070827+0000","last_fullsized":"2024-09-19T04:14:17.070827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070514+0000","last_change":"2024-09-19T04:14:17.070514+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070514+0000","last_undegraded":"2024-09-19T04:14:17.070514+0000","last_fullsized":"2024-09-19T04:14:17.070514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073632+0000","last_change":"2024-09-19T04:14:17.073632+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073632+0000","last_undegraded":"2024-09-19T04:14:17.073632+0000","last_fullsized":"2024-09-19T04:14:17.073632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065671+0000","last_change":"2024-09-19T04:14:17.065671+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065671+0000","last_undegraded":"2024-09-19T04:14:17.065671+0000","last_fullsized":"2024-09-19T04:14:17.065671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062750+0000","last_change":"2024-09-19T04:14:17.062750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062750+0000","last_undegraded":"2024-09-19T04:14:17.062750+0000","last_fullsized":"2024-09-19T04:14:17.062750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066992+0000","last_change":"2024-09-19T04:14:17.066992+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066992+0000","last_undegraded":"2024-09-19T04:14:17.066992+0000","last_fullsized":"2024-09-19T04:14:17.066992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.060421+0000","last_change":"2024-09-19T04:14:17.060421+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.060421+0000","last_undegraded":"2024-09-19T04:14:17.060421+0000","last_fullsized":"2024-09-19T04:14:17.060421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058740+0000","last_change":"2024-09-19T04:14:17.058740+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058740+0000","last_undegraded":"2024-09-19T04:14:17.058740+0000","last_fullsized":"2024-09-19T04:14:17.058740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066865+0000","last_change":"2024-09-19T04:14:17.066865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066865+0000","last_undegraded":"2024-09-19T04:14:17.066865+0000","last_fullsized":"2024-09-19T04:14:17.066865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063176+0000","last_change":"2024-09-19T04:14:17.063176+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063176+0000","last_undegraded":"2024-09-19T04:14:17.063176+0000","last_fullsized":"2024-09-19T04:14:17.063176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068706+0000","last_change":"2024-09-19T04:14:17.068706+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068706+0000","last_undegraded":"2024-09-19T04:14:17.068706+0000","last_fullsized":"2024-09-19T04:14:17.068706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064596+0000","last_change":"2024-09-19T04:14:17.064596+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064596+0000","last_undegraded":"2024-09-19T04:14:17.064596+0000","last_fullsized":"2024-09-19T04:14:17.064596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.056764+0000","last_change":"2024-09-19T04:14:17.056764+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.056764+0000","last_undegraded":"2024-09-19T04:14:17.056764+0000","last_fullsized":"2024-09-19T04:14:17.056764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059772+0000","last_change":"2024-09-19T04:14:17.059772+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059772+0000","last_undegraded":"2024-09-19T04:14:17.059772+0000","last_fullsized":"2024-09-19T04:14:17.059772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062552+0000","last_change":"2024-09-19T04:14:17.062552+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062552+0000","last_undegraded":"2024-09-19T04:14:17.062552+0000","last_fullsized":"2024-09-19T04:14:17.062552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072830+0000","last_change":"2024-09-19T04:14:17.072830+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072830+0000","last_undegraded":"2024-09-19T04:14:17.072830+0000","last_fullsized":"2024-09-19T04:14:17.072830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063731+0000","last_change":"2024-09-19T04:14:17.063731+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063731+0000","last_undegraded":"2024-09-19T04:14:17.063731+0000","last_fullsized":"2024-09-19T04:14:17.063731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065739+0000","last_change":"2024-09-19T04:14:17.065739+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065739+0000","last_undegraded":"2024-09-19T04:14:17.065739+0000","last_fullsized":"2024-09-19T04:14:17.065739+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063806+0000","last_change":"2024-09-19T04:14:17.063806+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063806+0000","last_undegraded":"2024-09-19T04:14:17.063806+0000","last_fullsized":"2024-09-19T04:14:17.063806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064604+0000","last_change":"2024-09-19T04:14:17.064604+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064604+0000","last_undegraded":"2024-09-19T04:14:17.064604+0000","last_fullsized":"2024-09-19T04:14:17.064604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064345+0000","last_change":"2024-09-19T04:14:17.064345+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064345+0000","last_undegraded":"2024-09-19T04:14:17.064345+0000","last_fullsized":"2024-09-19T04:14:17.064345+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058614+0000","last_change":"2024-09-19T04:14:17.058614+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058614+0000","last_undegraded":"2024-09-19T04:14:17.058614+0000","last_fullsized":"2024-09-19T04:14:17.058614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064314+0000","last_change":"2024-09-19T04:14:17.064314+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064314+0000","last_undegraded":"2024-09-19T04:14:17.064314+0000","last_fullsized":"2024-09-19T04:14:17.064314+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059239+0000","last_change":"2024-09-19T04:14:17.059239+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059239+0000","last_undegraded":"2024-09-19T04:14:17.059239+0000","last_fullsized":"2024-09-19T04:14:17.059239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091472+0000","last_change":"2024-09-19T04:14:17.088483+0000","last_active":"2024-09-19T04:14:17.091472+0000","last_peered":"2024-09-19T04:14:17.091472+0000","last_clean":"2024-09-19T04:14:17.091472+0000","last_became_active":"2024-09-19T04:14:17.087794+0000","last_became_peered":"2024-09-19T04:14:17.087794+0000","last_unstale":"2024-09-19T04:14:17.091472+0000","last_undegraded":"2024-09-19T04:14:17.091472+0000","last_fullsized":"2024-09-19T04:14:17.091472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065208+0000","last_change":"2024-09-19T04:14:17.065208+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065208+0000","last_undegraded":"2024-09-19T04:14:17.065208+0000","last_fullsized":"2024-09-19T04:14:17.065208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.093823+0000","last_change":"2024-09-19T04:14:17.093169+0000","last_active":"2024-09-19T04:14:17.093823+0000","last_peered":"2024-09-19T04:14:17.093823+0000","last_clean":"2024-09-19T04:14:17.093823+0000","last_became_active":"2024-09-19T04:14:17.092881+0000","last_became_peered":"2024-09-19T04:14:17.092881+0000","last_unstale":"2024-09-19T04:14:17.093823+0000","last_undegraded":"2024-09-19T04:14:17.093823+0000","last_fullsized":"2024-09-19T04:14:17.093823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058473+0000","last_change":"2024-09-19T04:14:17.058473+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058473+0000","last_undegraded":"2024-09-19T04:14:17.058473+0000","last_fullsized":"2024-09-19T04:14:17.058473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078344+0000","last_change":"2024-09-19T04:14:17.076870+0000","last_active":"2024-09-19T04:14:17.078344+0000","last_peered":"2024-09-19T04:14:17.078344+0000","last_clean":"2024-09-19T04:14:17.078344+0000","last_became_active":"2024-09-19T04:14:17.076604+0000","last_became_peered":"2024-09-19T04:14:17.076604+0000","last_unstale":"2024-09-19T04:14:17.078344+0000","last_undegraded":"2024-09-19T04:14:17.078344+0000","last_fullsized":"2024-09-19T04:14:17.078344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062853+0000","last_change":"2024-09-19T04:14:17.062853+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062853+0000","last_undegraded":"2024-09-19T04:14:17.062853+0000","last_fullsized":"2024-09-19T04:14:17.062853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.079157+0000","last_change":"2024-09-19T04:14:17.072407+0000","last_active":"2024-09-19T04:14:17.079157+0000","last_peered":"2024-09-19T04:14:17.079157+0000","last_clean":"2024-09-19T04:14:17.079157+0000","last_became_active":"2024-09-19T04:14:17.071689+0000","last_became_peered":"2024-09-19T04:14:17.071689+0000","last_unstale":"2024-09-19T04:14:17.079157+0000","last_undegraded":"2024-09-19T04:14:17.079157+0000","last_fullsized":"2024-09-19T04:14:17.079157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074684+0000","last_change":"2024-09-19T04:14:17.074684+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074684+0000","last_undegraded":"2024-09-19T04:14:17.074684+0000","last_fullsized":"2024-09-19T04:14:17.074684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098241+0000","last_change":"2024-09-19T04:14:17.083299+0000","last_active":"2024-09-19T04:14:17.098241+0000","last_peered":"2024-09-19T04:14:17.098241+0000","last_clean":"2024-09-19T04:14:17.098241+0000","last_became_active":"2024-09-19T04:14:17.081493+0000","last_became_peered":"2024-09-19T04:14:17.081493+0000","last_unstale":"2024-09-19T04:14:17.098241+0000","last_undegraded":"2024-09-19T04:14:17.098241+0000","last_fullsized":"2024-09-19T04:14:17.098241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065920+0000","last_change":"2024-09-19T04:14:17.065920+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065920+0000","last_undegraded":"2024-09-19T04:14:17.065920+0000","last_fullsized":"2024-09-19T04:14:17.065920+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100822+0000","last_change":"2024-09-19T04:14:17.098330+0000","last_active":"2024-09-19T04:14:17.100822+0000","last_peered":"2024-09-19T04:14:17.100822+0000","last_clean":"2024-09-19T04:14:17.100822+0000","last_became_active":"2024-09-19T04:14:17.097696+0000","last_became_peered":"2024-09-19T04:14:17.097696+0000","last_unstale":"2024-09-19T04:14:17.100822+0000","last_undegraded":"2024-09-19T04:14:17.100822+0000","last_fullsized":"2024-09-19T04:14:17.100822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:46.739371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058025+0000","last_change":"2024-09-19T04:14:17.058025+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058025+0000","last_undegraded":"2024-09-19T04:14:17.058025+0000","last_fullsized":"2024-09-19T04:14:17.058025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098750+0000","last_change":"2024-09-19T04:14:17.097839+0000","last_active":"2024-09-19T04:14:17.098750+0000","last_peered":"2024-09-19T04:14:17.098750+0000","last_clean":"2024-09-19T04:14:17.098750+0000","last_became_active":"2024-09-19T04:14:17.097522+0000","last_became_peered":"2024-09-19T04:14:17.097522+0000","last_unstale":"2024-09-19T04:14:17.098750+0000","last_undegraded":"2024-09-19T04:14:17.098750+0000","last_fullsized":"2024-09-19T04:14:17.098750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065710+0000","last_change":"2024-09-19T04:14:17.065710+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065710+0000","last_undegraded":"2024-09-19T04:14:17.065710+0000","last_fullsized":"2024-09-19T04:14:17.065710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094043+0000","last_change":"2024-09-19T04:14:17.088506+0000","last_active":"2024-09-19T04:14:17.094043+0000","last_peered":"2024-09-19T04:14:17.094043+0000","last_clean":"2024-09-19T04:14:17.094043+0000","last_became_active":"2024-09-19T04:14:17.087746+0000","last_became_peered":"2024-09-19T04:14:17.087746+0000","last_unstale":"2024-09-19T04:14:17.094043+0000","last_undegraded":"2024-09-19T04:14:17.094043+0000","last_fullsized":"2024-09-19T04:14:17.094043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062249+0000","last_change":"2024-09-19T04:14:17.062249+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062249+0000","last_undegraded":"2024-09-19T04:14:17.062249+0000","last_fullsized":"2024-09-19T04:14:17.062249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075544+0000","last_change":"2024-09-19T04:14:17.070017+0000","last_active":"2024-09-19T04:14:17.075544+0000","last_peered":"2024-09-19T04:14:17.075544+0000","last_clean":"2024-09-19T04:14:17.075544+0000","last_became_active":"2024-09-19T04:14:17.069539+0000","last_became_peered":"2024-09-19T04:14:17.069539+0000","last_unstale":"2024-09-19T04:14:17.075544+0000","last_undegraded":"2024-09-19T04:14:17.075544+0000","last_fullsized":"2024-09-19T04:14:17.075544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:31:50.043891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073417+0000","last_change":"2024-09-19T04:14:17.073417+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073417+0000","last_undegraded":"2024-09-19T04:14:17.073417+0000","last_fullsized":"2024-09-19T04:14:17.073417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101674+0000","last_change":"2024-09-19T04:14:17.094996+0000","last_active":"2024-09-19T04:14:17.101674+0000","last_peered":"2024-09-19T04:14:17.101674+0000","last_clean":"2024-09-19T04:14:17.101674+0000","last_became_active":"2024-09-19T04:14:17.083414+0000","last_became_peered":"2024-09-19T04:14:17.083414+0000","last_unstale":"2024-09-19T04:14:17.101674+0000","last_undegraded":"2024-09-19T04:14:17.101674+0000","last_fullsized":"2024-09-19T04:14:17.101674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:40.402841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074169+0000","last_change":"2024-09-19T04:14:17.074169+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074169+0000","last_undegraded":"2024-09-19T04:14:17.074169+0000","last_fullsized":"2024-09-19T04:14:17.074169+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098844+0000","last_change":"2024-09-19T04:14:17.094246+0000","last_active":"2024-09-19T04:14:17.098844+0000","last_peered":"2024-09-19T04:14:17.098844+0000","last_clean":"2024-09-19T04:14:17.098844+0000","last_became_active":"2024-09-19T04:14:17.083763+0000","last_became_peered":"2024-09-19T04:14:17.083763+0000","last_unstale":"2024-09-19T04:14:17.098844+0000","last_undegraded":"2024-09-19T04:14:17.098844+0000","last_fullsized":"2024-09-19T04:14:17.098844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:39.583489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062248+0000","last_change":"2024-09-19T04:14:17.062248+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062248+0000","last_undegraded":"2024-09-19T04:14:17.062248+0000","last_fullsized":"2024-09-19T04:14:17.062248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.095183+0000","last_change":"2024-09-19T04:14:17.094374+0000","last_active":"2024-09-19T04:14:17.095183+0000","last_peered":"2024-09-19T04:14:17.095183+0000","last_clean":"2024-09-19T04:14:17.095183+0000","last_became_active":"2024-09-19T04:14:17.093949+0000","last_became_peered":"2024-09-19T04:14:17.093949+0000","last_unstale":"2024-09-19T04:14:17.095183+0000","last_undegraded":"2024-09-19T04:14:17.095183+0000","last_fullsized":"2024-09-19T04:14:17.095183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:27.153452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.069357+0000","last_change":"2024-09-19T04:14:17.069357+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.069357+0000","last_undegraded":"2024-09-19T04:14:17.069357+0000","last_fullsized":"2024-09-19T04:14:17.069357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078270+0000","last_change":"2024-09-19T04:14:17.072319+0000","last_active":"2024-09-19T04:14:17.078270+0000","last_peered":"2024-09-19T04:14:17.078270+0000","last_clean":"2024-09-19T04:14:17.078270+0000","last_became_active":"2024-09-19T04:14:17.071960+0000","last_became_peered":"2024-09-19T04:14:17.071960+0000","last_unstale":"2024-09-19T04:14:17.078270+0000","last_undegraded":"2024-09-19T04:14:17.078270+0000","last_fullsized":"2024-09-19T04:14:17.078270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:56.776172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064480+0000","last_change":"2024-09-19T04:14:17.064480+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064480+0000","last_undegraded":"2024-09-19T04:14:17.064480+0000","last_fullsized":"2024-09-19T04:14:17.064480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.084958+0000","last_change":"2024-09-19T04:14:17.084245+0000","last_active":"2024-09-19T04:14:17.084958+0000","last_peered":"2024-09-19T04:14:17.084958+0000","last_clean":"2024-09-19T04:14:17.084958+0000","last_became_active":"2024-09-19T04:14:17.083828+0000","last_became_peered":"2024-09-19T04:14:17.083828+0000","last_unstale":"2024-09-19T04:14:17.084958+0000","last_undegraded":"2024-09-19T04:14:17.084958+0000","last_fullsized":"2024-09-19T04:14:17.084958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058557+0000","last_change":"2024-09-19T04:14:17.058557+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058557+0000","last_undegraded":"2024-09-19T04:14:17.058557+0000","last_fullsized":"2024-09-19T04:14:17.058557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091279+0000","last_change":"2024-09-19T04:14:17.085767+0000","last_active":"2024-09-19T04:14:17.091279+0000","last_peered":"2024-09-19T04:14:17.091279+0000","last_clean":"2024-09-19T04:14:17.091279+0000","last_became_active":"2024-09-19T04:14:17.085016+0000","last_became_peered":"2024-09-19T04:14:17.085016+0000","last_unstale":"2024-09-19T04:14:17.091279+0000","last_undegraded":"2024-09-19T04:14:17.091279+0000","last_fullsized":"2024-09-19T04:14:17.091279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065260+0000","last_change":"2024-09-19T04:14:17.065260+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065260+0000","last_undegraded":"2024-09-19T04:14:17.065260+0000","last_fullsized":"2024-09-19T04:14:17.065260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100905+0000","last_change":"2024-09-19T04:14:17.098594+0000","last_active":"2024-09-19T04:14:17.100905+0000","last_peered":"2024-09-19T04:14:17.100905+0000","last_clean":"2024-09-19T04:14:17.100905+0000","last_became_active":"2024-09-19T04:14:17.097858+0000","last_became_peered":"2024-09-19T04:14:17.097858+0000","last_unstale":"2024-09-19T04:14:17.100905+0000","last_undegraded":"2024-09-19T04:14:17.100905+0000","last_fullsized":"2024-09-19T04:14:17.100905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:26.792894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063556+0000","last_change":"2024-09-19T04:14:17.063556+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063556+0000","last_undegraded":"2024-09-19T04:14:17.063556+0000","last_fullsized":"2024-09-19T04:14:17.063556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101712+0000","last_change":"2024-09-19T04:14:17.095361+0000","last_active":"2024-09-19T04:14:17.101712+0000","last_peered":"2024-09-19T04:14:17.101712+0000","last_clean":"2024-09-19T04:14:17.101712+0000","last_became_active":"2024-09-19T04:14:17.094445+0000","last_became_peered":"2024-09-19T04:14:17.094445+0000","last_unstale":"2024-09-19T04:14:17.101712+0000","last_undegraded":"2024-09-19T04:14:17.101712+0000","last_fullsized":"2024-09-19T04:14:17.101712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:59.306800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065209+0000","last_change":"2024-09-19T04:14:17.065209+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065209+0000","last_undegraded":"2024-09-19T04:14:17.065209+0000","last_fullsized":"2024-09-19T04:14:17.065209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101020+0000","last_change":"2024-09-19T04:14:17.098487+0000","last_active":"2024-09-19T04:14:17.101020+0000","last_peered":"2024-09-19T04:14:17.101020+0000","last_clean":"2024-09-19T04:14:17.101020+0000","last_became_active":"2024-09-19T04:14:17.097805+0000","last_became_peered":"2024-09-19T04:14:17.097805+0000","last_unstale":"2024-09-19T04:14:17.101020+0000","last_undegraded":"2024-09-19T04:14:17.101020+0000","last_fullsized":"2024-09-19T04:14:17.101020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:55.367559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067888+0000","last_change":"2024-09-19T04:14:17.067888+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067888+0000","last_undegraded":"2024-09-19T04:14:17.067888+0000","last_fullsized":"2024-09-19T04:14:17.067888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094537+0000","last_change":"2024-09-19T04:14:17.093924+0000","last_active":"2024-09-19T04:14:17.094537+0000","last_peered":"2024-09-19T04:14:17.094537+0000","last_clean":"2024-09-19T04:14:17.094537+0000","last_became_active":"2024-09-19T04:14:17.093578+0000","last_became_peered":"2024-09-19T04:14:17.093578+0000","last_unstale":"2024-09-19T04:14:17.094537+0000","last_undegraded":"2024-09-19T04:14:17.094537+0000","last_fullsized":"2024-09-19T04:14:17.094537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:43:26.607609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061925+0000","last_change":"2024-09-19T04:14:17.061925+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061925+0000","last_undegraded":"2024-09-19T04:14:17.061925+0000","last_fullsized":"2024-09-19T04:14:17.061925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078381+0000","last_change":"2024-09-19T04:14:17.076993+0000","last_active":"2024-09-19T04:14:17.078381+0000","last_peered":"2024-09-19T04:14:17.078381+0000","last_clean":"2024-09-19T04:14:17.078381+0000","last_became_active":"2024-09-19T04:14:17.076271+0000","last_became_peered":"2024-09-19T04:14:17.076271+0000","last_unstale":"2024-09-19T04:14:17.078381+0000","last_undegraded":"2024-09-19T04:14:17.078381+0000","last_fullsized":"2024-09-19T04:14:17.078381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059387+0000","last_change":"2024-09-19T04:14:17.059387+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059387+0000","last_undegraded":"2024-09-19T04:14:17.059387+0000","last_fullsized":"2024-09-19T04:14:17.059387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075746+0000","last_change":"2024-09-19T04:14:17.071279+0000","last_active":"2024-09-19T04:14:17.075746+0000","last_peered":"2024-09-19T04:14:17.075746+0000","last_clean":"2024-09-19T04:14:17.075746+0000","last_became_active":"2024-09-19T04:14:17.070910+0000","last_became_peered":"2024-09-19T04:14:17.070910+0000","last_unstale":"2024-09-19T04:14:17.075746+0000","last_undegraded":"2024-09-19T04:14:17.075746+0000","last_fullsized":"2024-09-19T04:14:17.075746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:35.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067722+0000","last_change":"2024-09-19T04:14:17.067722+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067722+0000","last_undegraded":"2024-09-19T04:14:17.067722+0000","last_fullsized":"2024-09-19T04:14:17.067722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.081039+0000","last_change":"2024-09-19T04:14:17.080497+0000","last_active":"2024-09-19T04:14:17.081039+0000","last_peered":"2024-09-19T04:14:17.081039+0000","last_clean":"2024-09-19T04:14:17.081039+0000","last_became_active":"2024-09-19T04:14:17.080085+0000","last_became_peered":"2024-09-19T04:14:17.080085+0000","last_unstale":"2024-09-19T04:14:17.081039+0000","last_undegraded":"2024-09-19T04:14:17.081039+0000","last_fullsized":"2024-09-19T04:14:17.081039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:08.636704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065171+0000","last_change":"2024-09-19T04:14:17.065171+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065171+0000","last_undegraded":"2024-09-19T04:14:17.065171+0000","last_fullsized":"2024-09-19T04:14:17.065171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099918+0000","last_change":"2024-09-19T04:14:17.095657+0000","last_active":"2024-09-19T04:14:17.099918+0000","last_peered":"2024-09-19T04:14:17.099918+0000","last_clean":"2024-09-19T04:14:17.099918+0000","last_became_active":"2024-09-19T04:14:17.094938+0000","last_became_peered":"2024-09-19T04:14:17.094938+0000","last_unstale":"2024-09-19T04:14:17.099918+0000","last_undegraded":"2024-09-19T04:14:17.099918+0000","last_fullsized":"2024-09-19T04:14:17.099918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.454267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064362+0000","last_change":"2024-09-19T04:14:17.064362+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064362+0000","last_undegraded":"2024-09-19T04:14:17.064362+0000","last_fullsized":"2024-09-19T04:14:17.064362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.070502+0000","last_change":"2024-09-19T04:14:17.067723+0000","last_active":"2024-09-19T04:14:17.070502+0000","last_peered":"2024-09-19T04:14:17.070502+0000","last_clean":"2024-09-19T04:14:17.070502+0000","last_became_active":"2024-09-19T04:14:17.067393+0000","last_became_peered":"2024-09-19T04:14:17.067393+0000","last_unstale":"2024-09-19T04:14:17.070502+0000","last_undegraded":"2024-09-19T04:14:17.070502+0000","last_fullsized":"2024-09-19T04:14:17.070502+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:06.884322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064086+0000","last_change":"2024-09-19T04:14:17.064086+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064086+0000","last_undegraded":"2024-09-19T04:14:17.064086+0000","last_fullsized":"2024-09-19T04:14:17.064086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099437+0000","last_change":"2024-09-19T04:14:17.094649+0000","last_active":"2024-09-19T04:14:17.099437+0000","last_peered":"2024-09-19T04:14:17.099437+0000","last_clean":"2024-09-19T04:14:17.099437+0000","last_became_active":"2024-09-19T04:14:17.083575+0000","last_became_peered":"2024-09-19T04:14:17.083575+0000","last_unstale":"2024-09-19T04:14:17.099437+0000","last_undegraded":"2024-09-19T04:14:17.099437+0000","last_fullsized":"2024-09-19T04:14:17.099437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:13.045881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061389+0000","last_change":"2024-09-19T04:14:17.061389+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061389+0000","last_undegraded":"2024-09-19T04:14:17.061389+0000","last_fullsized":"2024-09-19T04:14:17.061389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098793+0000","last_change":"2024-09-19T04:14:17.097837+0000","last_active":"2024-09-19T04:14:17.098793+0000","last_peered":"2024-09-19T04:14:17.098793+0000","last_clean":"2024-09-19T04:14:17.098793+0000","last_became_active":"2024-09-19T04:14:17.097561+0000","last_became_peered":"2024-09-19T04:14:17.097561+0000","last_unstale":"2024-09-19T04:14:17.098793+0000","last_undegraded":"2024-09-19T04:14:17.098793+0000","last_fullsized":"2024-09-19T04:14:17.098793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:22.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062755+0000","last_change":"2024-09-19T04:14:17.062755+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062755+0000","last_undegraded":"2024-09-19T04:14:17.062755+0000","last_fullsized":"2024-09-19T04:14:17.062755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.083154+0000","last_change":"2024-09-19T04:14:17.074004+0000","last_active":"2024-09-19T04:14:17.083154+0000","last_peered":"2024-09-19T04:14:17.083154+0000","last_clean":"2024-09-19T04:14:17.083154+0000","last_became_active":"2024-09-19T04:14:17.073653+0000","last_became_peered":"2024-09-19T04:14:17.073653+0000","last_unstale":"2024-09-19T04:14:17.083154+0000","last_undegraded":"2024-09-19T04:14:17.083154+0000","last_fullsized":"2024-09-19T04:14:17.083154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068919+0000","last_change":"2024-09-19T04:14:17.068919+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068919+0000","last_undegraded":"2024-09-19T04:14:17.068919+0000","last_fullsized":"2024-09-19T04:14:17.068919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088034+0000","last_change":"2024-09-19T04:14:17.087371+0000","last_active":"2024-09-19T04:14:17.088034+0000","last_peered":"2024-09-19T04:14:17.088034+0000","last_clean":"2024-09-19T04:14:17.088034+0000","last_became_active":"2024-09-19T04:14:17.087047+0000","last_became_peered":"2024-09-19T04:14:17.087047+0000","last_unstale":"2024-09-19T04:14:17.088034+0000","last_undegraded":"2024-09-19T04:14:17.088034+0000","last_fullsized":"2024-09-19T04:14:17.088034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063449+0000","last_change":"2024-09-19T04:14:17.063449+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063449+0000","last_undegraded":"2024-09-19T04:14:17.063449+0000","last_fullsized":"2024-09-19T04:14:17.063449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088850+0000","last_change":"2024-09-19T04:14:17.086127+0000","last_active":"2024-09-19T04:14:17.088850+0000","last_peered":"2024-09-19T04:14:17.088850+0000","last_clean":"2024-09-19T04:14:17.088850+0000","last_became_active":"2024-09-19T04:14:17.084606+0000","last_became_peered":"2024-09-19T04:14:17.084606+0000","last_unstale":"2024-09-19T04:14:17.088850+0000","last_undegraded":"2024-09-19T04:14:17.088850+0000","last_fullsized":"2024-09-19T04:14:17.088850+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059105+0000","last_change":"2024-09-19T04:14:17.059105+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059105+0000","last_undegraded":"2024-09-19T04:14:17.059105+0000","last_fullsized":"2024-09-19T04:14:17.059105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078734+0000","last_change":"2024-09-19T04:14:17.070371+0000","last_active":"2024-09-19T04:14:17.078734+0000","last_peered":"2024-09-19T04:14:17.078734+0000","last_clean":"2024-09-19T04:14:17.078734+0000","last_became_active":"2024-09-19T04:14:17.069663+0000","last_became_peered":"2024-09-19T04:14:17.069663+0000","last_unstale":"2024-09-19T04:14:17.078734+0000","last_undegraded":"2024-09-19T04:14:17.078734+0000","last_fullsized":"2024-09-19T04:14:17.078734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059865+0000","last_change":"2024-09-19T04:14:17.059865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059865+0000","last_undegraded":"2024-09-19T04:14:17.059865+0000","last_fullsized":"2024-09-19T04:14:17.059865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075551+0000","last_change":"2024-09-19T04:14:17.070348+0000","last_active":"2024-09-19T04:14:17.075551+0000","last_peered":"2024-09-19T04:14:17.075551+0000","last_clean":"2024-09-19T04:14:17.075551+0000","last_became_active":"2024-09-19T04:14:17.069964+0000","last_became_peered":"2024-09-19T04:14:17.069964+0000","last_unstale":"2024-09-19T04:14:17.075551+0000","last_undegraded":"2024-09-19T04:14:17.075551+0000","last_fullsized":"2024-09-19T04:14:17.075551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:09.298918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068943+0000","last_change":"2024-09-19T04:14:17.068943+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068943+0000","last_undegraded":"2024-09-19T04:14:17.068943+0000","last_fullsized":"2024-09-19T04:14:17.068943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099762+0000","last_change":"2024-09-19T04:14:17.093374+0000","last_active":"2024-09-19T04:14:17.099762+0000","last_peered":"2024-09-19T04:14:17.099762+0000","last_clean":"2024-09-19T04:14:17.099762+0000","last_became_active":"2024-09-19T04:14:17.092649+0000","last_became_peered":"2024-09-19T04:14:17.092649+0000","last_unstale":"2024-09-19T04:14:17.099762+0000","last_undegraded":"2024-09-19T04:14:17.099762+0000","last_fullsized":"2024-09-19T04:14:17.099762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:34.491970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.087005+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:14:17.087005+0000","last_peered":"2024-09-19T04:14:17.087005+0000","last_clean":"2024-09-19T04:14:17.087005+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:14:17.087005+0000","last_undegraded":"2024-09-19T04:14:17.087005+0000","last_fullsized":"2024-09-19T04:14:17.087005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:14:21.066 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:14:21.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f75fdb640 1 Processor -- start 2024-09-19T04:14:21.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f75fdb640 1 -- start start 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f75fdb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1f700cf620 0x7f1f70168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f75fdb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f700d0300 con 0x7f1f700d3060 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f75fdb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f7006e6c0 con 0x7f1f70161a60 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f75fdb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1f700cfa90 con 0x7f1f700cf620 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6effd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1f700d3060 0x7f1f700cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48348/0 (socket says 172.21.15.76:48348) 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6effd640 1 -- 172.21.15.76:0/988071450 learned_addr learned my addr 172.21.15.76:0/988071450 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6ffff640 1 --2- 172.21.15.76:0/988071450 >> v2:172.21.15.76:3300/0 conn(0x7f1f700cf620 0x7f1f70168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6e7fc640 1 -- 172.21.15.76:0/988071450 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996597208 0 0) 0x7f1f64001120 con 0x7f1f700d3060 2024-09-19T04:14:21.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6e7fc640 1 -- 172.21.15.76:0/988071450 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f58003530 con 0x7f1f700d3060 2024-09-19T04:14:21.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6ffff640 1 -- 172.21.15.76:0/988071450 >> v1:172.21.15.116:6789/0 conn(0x7f1f70161a60 legacy=0x7f1f700d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:21.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6ffff640 1 -- 172.21.15.76:0/988071450 >> v1:172.21.15.76:6789/0 conn(0x7f1f700d3060 legacy=0x7f1f700cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.220+0000 7f1f6ffff640 1 -- 172.21.15.76:0/988071450 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f700b0f60 con 0x7f1f700cf620 2024-09-19T04:14:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 --2- 172.21.15.76:0/988071450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f700cf620 0x7f1f70168f10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009d90 tx=0x7f1f6801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb837adb1c09f02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6e7fc640 1 -- 172.21.15.76:0/988071450 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680b2070 con 0x7f1f700cf620 2024-09-19T04:14:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6e7fc640 1 -- 172.21.15.76:0/988071450 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f680a5930 con 0x7f1f700cf620 2024-09-19T04:14:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6e7fc640 1 -- 172.21.15.76:0/988071450 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680a5d00 con 0x7f1f700cf620 2024-09-19T04:14:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- 172.21.15.76:0/988071450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f700cf620 msgr2=0x7f1f70168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/988071450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f700cf620 0x7f1f70168f10 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009d90 tx=0x7f1f6801c870 comp rx=0 tx=0).stop 2024-09-19T04:14:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- 172.21.15.76:0/988071450 shutdown_connections 2024-09-19T04:14:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/988071450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f700cf620 0x7f1f70168f10 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- 172.21.15.76:0/988071450 >> 172.21.15.76:0/988071450 conn(0x7f1f7008bf90 msgr2=0x7f1f7009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- 172.21.15.76:0/988071450 shutdown_connections 2024-09-19T04:14:21.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- 172.21.15.76:0/988071450 wait complete. 2024-09-19T04:14:21.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 Processor -- start 2024-09-19T04:14:21.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- start start 2024-09-19T04:14:21.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f700cf620 0x7f1f700b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f700d3060 0x7f1f70158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f70161a60 0x7f1f70156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7016bc70 con 0x7f1f70161a60 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7006d190 con 0x7f1f700cf620 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1f7016bdf0 con 0x7f1f700d3060 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f70161a60 0x7f1f70156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f70161a60 0x7f1f70156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46822/0 (socket says 172.21.15.76:46822) 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 -- 172.21.15.76:0/3903594890 learned_addr learned my addr 172.21.15.76:0/3903594890 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 -- 172.21.15.76:0/3903594890 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f700d3060 msgr2=0x7f1f70158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f700d3060 0x7f1f70158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 -- 172.21.15.76:0/3903594890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f700cf620 msgr2=0x7f1f700b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f700cf620 0x7f1f700b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 -- 172.21.15.76:0/3903594890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f70156e60 con 0x7f1f70161a60 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f6ffff640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f70161a60 0x7f1f70156920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1f680b0040 tx=0x7f1f68019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f57fff640 1 -- 172.21.15.76:0/3903594890 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680bd020 con 0x7f1f70161a60 2024-09-19T04:14:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f57fff640 1 -- 172.21.15.76:0/3903594890 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f680b8670 con 0x7f1f70161a60 2024-09-19T04:14:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f700b11d0 con 0x7f1f70161a60 2024-09-19T04:14:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f57fff640 1 -- 172.21.15.76:0/3903594890 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680b2040 con 0x7f1f70161a60 2024-09-19T04:14:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.224+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f700b1790 con 0x7f1f70161a60 2024-09-19T04:14:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.228+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f700cef10 con 0x7f1f70161a60 2024-09-19T04:14:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.228+0000 7f1f57fff640 1 -- 172.21.15.76:0/3903594890 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1f680c0020 con 0x7f1f70161a60 2024-09-19T04:14:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.228+0000 7f1f57fff640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1f5c0678c0 0x7f1f5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.228+0000 7f1f57fff640 1 -- 172.21.15.76:0/3903594890 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f1f68120e30 con 0x7f1f70161a60 2024-09-19T04:14:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.228+0000 7f1f6f7fe640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1f5c0678c0 0x7f1f5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.228+0000 7f1f6f7fe640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1f5c0678c0 0x7f1f5c069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f701fa5a0 tx=0x7f1f60006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.232+0000 7f1f57fff640 1 -- 172.21.15.76:0/3903594890 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1f680c7030 con 0x7f1f70161a60 2024-09-19T04:14:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.388+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1f700d14f0 con 0x7f1f5c0678c0 2024-09-19T04:14:21.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.396+0000 7f1f57fff640 1 -- 172.21.15.76:0/3903594890 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246639 (secure 0 0 0) 0x7f1f700d14f0 con 0x7f1f5c0678c0 2024-09-19T04:14:21.401 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1f5c0678c0 msgr2=0x7f1f5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1f5c0678c0 0x7f1f5c069d80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1f701fa5a0 tx=0x7f1f60006bd0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f70161a60 msgr2=0x7f1f70156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f70161a60 0x7f1f70156920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1f680b0040 tx=0x7f1f68019610 comp rx=0 tx=0).stop 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 shutdown_connections 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1f5c0678c0 0x7f1f5c069d80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1f70161a60 0x7f1f70156920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1f700d3060 0x7f1f70158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 --2- 172.21.15.76:0/3903594890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1f700cf620 0x7f1f700b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 >> 172.21.15.76:0/3903594890 conn(0x7f1f7008bf90 msgr2=0x7f1f700b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 shutdown_connections 2024-09-19T04:14:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.400+0000 7f1f75fdb640 1 -- 172.21.15.76:0/3903594890 wait complete. 2024-09-19T04:14:21.418 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-19T04:14:19.754180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229008,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745712,"statfs":{"total":773094113280,"available":772859609088,"internally_reserved":0,"allocated":4562944,"data_stored":2617280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001142"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065006+0000","last_change":"2024-09-19T04:14:17.065006+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065006+0000","last_undegraded":"2024-09-19T04:14:17.065006+0000","last_fullsized":"2024-09-19T04:14:17.065006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063769+0000","last_change":"2024-09-19T04:14:17.063769+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063769+0000","last_undegraded":"2024-09-19T04:14:17.063769+0000","last_fullsized":"2024-09-19T04:14:17.063769+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072719+0000","last_change":"2024-09-19T04:14:17.072719+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072719+0000","last_undegraded":"2024-09-19T04:14:17.072719+0000","last_fullsized":"2024-09-19T04:14:17.072719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066271+0000","last_change":"2024-09-19T04:14:17.066271+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066271+0000","last_undegraded":"2024-09-19T04:14:17.066271+0000","last_fullsized":"2024-09-19T04:14:17.066271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065750+0000","last_change":"2024-09-19T04:14:17.065750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065750+0000","last_undegraded":"2024-09-19T04:14:17.065750+0000","last_fullsized":"2024-09-19T04:14:17.065750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066325+0000","last_change":"2024-09-19T04:14:17.066325+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066325+0000","last_undegraded":"2024-09-19T04:14:17.066325+0000","last_fullsized":"2024-09-19T04:14:17.066325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068080+0000","last_change":"2024-09-19T04:14:17.068080+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068080+0000","last_undegraded":"2024-09-19T04:14:17.068080+0000","last_fullsized":"2024-09-19T04:14:17.068080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064742+0000","last_change":"2024-09-19T04:14:17.064742+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064742+0000","last_undegraded":"2024-09-19T04:14:17.064742+0000","last_fullsized":"2024-09-19T04:14:17.064742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073302+0000","last_change":"2024-09-19T04:14:17.073302+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073302+0000","last_undegraded":"2024-09-19T04:14:17.073302+0000","last_fullsized":"2024-09-19T04:14:17.073302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070827+0000","last_change":"2024-09-19T04:14:17.070827+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070827+0000","last_undegraded":"2024-09-19T04:14:17.070827+0000","last_fullsized":"2024-09-19T04:14:17.070827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070514+0000","last_change":"2024-09-19T04:14:17.070514+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070514+0000","last_undegraded":"2024-09-19T04:14:17.070514+0000","last_fullsized":"2024-09-19T04:14:17.070514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073632+0000","last_change":"2024-09-19T04:14:17.073632+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073632+0000","last_undegraded":"2024-09-19T04:14:17.073632+0000","last_fullsized":"2024-09-19T04:14:17.073632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065671+0000","last_change":"2024-09-19T04:14:17.065671+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065671+0000","last_undegraded":"2024-09-19T04:14:17.065671+0000","last_fullsized":"2024-09-19T04:14:17.065671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062750+0000","last_change":"2024-09-19T04:14:17.062750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062750+0000","last_undegraded":"2024-09-19T04:14:17.062750+0000","last_fullsized":"2024-09-19T04:14:17.062750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066992+0000","last_change":"2024-09-19T04:14:17.066992+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066992+0000","last_undegraded":"2024-09-19T04:14:17.066992+0000","last_fullsized":"2024-09-19T04:14:17.066992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.060421+0000","last_change":"2024-09-19T04:14:17.060421+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.060421+0000","last_undegraded":"2024-09-19T04:14:17.060421+0000","last_fullsized":"2024-09-19T04:14:17.060421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058740+0000","last_change":"2024-09-19T04:14:17.058740+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058740+0000","last_undegraded":"2024-09-19T04:14:17.058740+0000","last_fullsized":"2024-09-19T04:14:17.058740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066865+0000","last_change":"2024-09-19T04:14:17.066865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066865+0000","last_undegraded":"2024-09-19T04:14:17.066865+0000","last_fullsized":"2024-09-19T04:14:17.066865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063176+0000","last_change":"2024-09-19T04:14:17.063176+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063176+0000","last_undegraded":"2024-09-19T04:14:17.063176+0000","last_fullsized":"2024-09-19T04:14:17.063176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068706+0000","last_change":"2024-09-19T04:14:17.068706+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068706+0000","last_undegraded":"2024-09-19T04:14:17.068706+0000","last_fullsized":"2024-09-19T04:14:17.068706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064596+0000","last_change":"2024-09-19T04:14:17.064596+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064596+0000","last_undegraded":"2024-09-19T04:14:17.064596+0000","last_fullsized":"2024-09-19T04:14:17.064596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.056764+0000","last_change":"2024-09-19T04:14:17.056764+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.056764+0000","last_undegraded":"2024-09-19T04:14:17.056764+0000","last_fullsized":"2024-09-19T04:14:17.056764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059772+0000","last_change":"2024-09-19T04:14:17.059772+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059772+0000","last_undegraded":"2024-09-19T04:14:17.059772+0000","last_fullsized":"2024-09-19T04:14:17.059772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062552+0000","last_change":"2024-09-19T04:14:17.062552+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062552+0000","last_undegraded":"2024-09-19T04:14:17.062552+0000","last_fullsized":"2024-09-19T04:14:17.062552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072830+0000","last_change":"2024-09-19T04:14:17.072830+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072830+0000","last_undegraded":"2024-09-19T04:14:17.072830+0000","last_fullsized":"2024-09-19T04:14:17.072830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063731+0000","last_change":"2024-09-19T04:14:17.063731+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063731+0000","last_undegraded":"2024-09-19T04:14:17.063731+0000","last_fullsized":"2024-09-19T04:14:17.063731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065739+0000","last_change":"2024-09-19T04:14:17.065739+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065739+0000","last_undegraded":"2024-09-19T04:14:17.065739+0000","last_fullsized":"2024-09-19T04:14:17.065739+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063806+0000","last_change":"2024-09-19T04:14:17.063806+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063806+0000","last_undegraded":"2024-09-19T04:14:17.063806+0000","last_fullsized":"2024-09-19T04:14:17.063806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064604+0000","last_change":"2024-09-19T04:14:17.064604+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064604+0000","last_undegraded":"2024-09-19T04:14:17.064604+0000","last_fullsized":"2024-09-19T04:14:17.064604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064345+0000","last_change":"2024-09-19T04:14:17.064345+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064345+0000","last_undegraded":"2024-09-19T04:14:17.064345+0000","last_fullsized":"2024-09-19T04:14:17.064345+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058614+0000","last_change":"2024-09-19T04:14:17.058614+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058614+0000","last_undegraded":"2024-09-19T04:14:17.058614+0000","last_fullsized":"2024-09-19T04:14:17.058614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064314+0000","last_change":"2024-09-19T04:14:17.064314+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064314+0000","last_undegraded":"2024-09-19T04:14:17.064314+0000","last_fullsized":"2024-09-19T04:14:17.064314+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059239+0000","last_change":"2024-09-19T04:14:17.059239+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059239+0000","last_undegraded":"2024-09-19T04:14:17.059239+0000","last_fullsized":"2024-09-19T04:14:17.059239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091472+0000","last_change":"2024-09-19T04:14:17.088483+0000","last_active":"2024-09-19T04:14:17.091472+0000","last_peered":"2024-09-19T04:14:17.091472+0000","last_clean":"2024-09-19T04:14:17.091472+0000","last_became_active":"2024-09-19T04:14:17.087794+0000","last_became_peered":"2024-09-19T04:14:17.087794+0000","last_unstale":"2024-09-19T04:14:17.091472+0000","last_undegraded":"2024-09-19T04:14:17.091472+0000","last_fullsized":"2024-09-19T04:14:17.091472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065208+0000","last_change":"2024-09-19T04:14:17.065208+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065208+0000","last_undegraded":"2024-09-19T04:14:17.065208+0000","last_fullsized":"2024-09-19T04:14:17.065208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.093823+0000","last_change":"2024-09-19T04:14:17.093169+0000","last_active":"2024-09-19T04:14:17.093823+0000","last_peered":"2024-09-19T04:14:17.093823+0000","last_clean":"2024-09-19T04:14:17.093823+0000","last_became_active":"2024-09-19T04:14:17.092881+0000","last_became_peered":"2024-09-19T04:14:17.092881+0000","last_unstale":"2024-09-19T04:14:17.093823+0000","last_undegraded":"2024-09-19T04:14:17.093823+0000","last_fullsized":"2024-09-19T04:14:17.093823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058473+0000","last_change":"2024-09-19T04:14:17.058473+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058473+0000","last_undegraded":"2024-09-19T04:14:17.058473+0000","last_fullsized":"2024-09-19T04:14:17.058473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078344+0000","last_change":"2024-09-19T04:14:17.076870+0000","last_active":"2024-09-19T04:14:17.078344+0000","last_peered":"2024-09-19T04:14:17.078344+0000","last_clean":"2024-09-19T04:14:17.078344+0000","last_became_active":"2024-09-19T04:14:17.076604+0000","last_became_peered":"2024-09-19T04:14:17.076604+0000","last_unstale":"2024-09-19T04:14:17.078344+0000","last_undegraded":"2024-09-19T04:14:17.078344+0000","last_fullsized":"2024-09-19T04:14:17.078344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062853+0000","last_change":"2024-09-19T04:14:17.062853+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062853+0000","last_undegraded":"2024-09-19T04:14:17.062853+0000","last_fullsized":"2024-09-19T04:14:17.062853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.079157+0000","last_change":"2024-09-19T04:14:17.072407+0000","last_active":"2024-09-19T04:14:17.079157+0000","last_peered":"2024-09-19T04:14:17.079157+0000","last_clean":"2024-09-19T04:14:17.079157+0000","last_became_active":"2024-09-19T04:14:17.071689+0000","last_became_peered":"2024-09-19T04:14:17.071689+0000","last_unstale":"2024-09-19T04:14:17.079157+0000","last_undegraded":"2024-09-19T04:14:17.079157+0000","last_fullsized":"2024-09-19T04:14:17.079157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074684+0000","last_change":"2024-09-19T04:14:17.074684+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074684+0000","last_undegraded":"2024-09-19T04:14:17.074684+0000","last_fullsized":"2024-09-19T04:14:17.074684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098241+0000","last_change":"2024-09-19T04:14:17.083299+0000","last_active":"2024-09-19T04:14:17.098241+0000","last_peered":"2024-09-19T04:14:17.098241+0000","last_clean":"2024-09-19T04:14:17.098241+0000","last_became_active":"2024-09-19T04:14:17.081493+0000","last_became_peered":"2024-09-19T04:14:17.081493+0000","last_unstale":"2024-09-19T04:14:17.098241+0000","last_undegraded":"2024-09-19T04:14:17.098241+0000","last_fullsized":"2024-09-19T04:14:17.098241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065920+0000","last_change":"2024-09-19T04:14:17.065920+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065920+0000","last_undegraded":"2024-09-19T04:14:17.065920+0000","last_fullsized":"2024-09-19T04:14:17.065920+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100822+0000","last_change":"2024-09-19T04:14:17.098330+0000","last_active":"2024-09-19T04:14:17.100822+0000","last_peered":"2024-09-19T04:14:17.100822+0000","last_clean":"2024-09-19T04:14:17.100822+0000","last_became_active":"2024-09-19T04:14:17.097696+0000","last_became_peered":"2024-09-19T04:14:17.097696+0000","last_unstale":"2024-09-19T04:14:17.100822+0000","last_undegraded":"2024-09-19T04:14:17.100822+0000","last_fullsized":"2024-09-19T04:14:17.100822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:46.739371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058025+0000","last_change":"2024-09-19T04:14:17.058025+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058025+0000","last_undegraded":"2024-09-19T04:14:17.058025+0000","last_fullsized":"2024-09-19T04:14:17.058025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098750+0000","last_change":"2024-09-19T04:14:17.097839+0000","last_active":"2024-09-19T04:14:17.098750+0000","last_peered":"2024-09-19T04:14:17.098750+0000","last_clean":"2024-09-19T04:14:17.098750+0000","last_became_active":"2024-09-19T04:14:17.097522+0000","last_became_peered":"2024-09-19T04:14:17.097522+0000","last_unstale":"2024-09-19T04:14:17.098750+0000","last_undegraded":"2024-09-19T04:14:17.098750+0000","last_fullsized":"2024-09-19T04:14:17.098750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065710+0000","last_change":"2024-09-19T04:14:17.065710+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065710+0000","last_undegraded":"2024-09-19T04:14:17.065710+0000","last_fullsized":"2024-09-19T04:14:17.065710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094043+0000","last_change":"2024-09-19T04:14:17.088506+0000","last_active":"2024-09-19T04:14:17.094043+0000","last_peered":"2024-09-19T04:14:17.094043+0000","last_clean":"2024-09-19T04:14:17.094043+0000","last_became_active":"2024-09-19T04:14:17.087746+0000","last_became_peered":"2024-09-19T04:14:17.087746+0000","last_unstale":"2024-09-19T04:14:17.094043+0000","last_undegraded":"2024-09-19T04:14:17.094043+0000","last_fullsized":"2024-09-19T04:14:17.094043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062249+0000","last_change":"2024-09-19T04:14:17.062249+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062249+0000","last_undegraded":"2024-09-19T04:14:17.062249+0000","last_fullsized":"2024-09-19T04:14:17.062249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075544+0000","last_change":"2024-09-19T04:14:17.070017+0000","last_active":"2024-09-19T04:14:17.075544+0000","last_peered":"2024-09-19T04:14:17.075544+0000","last_clean":"2024-09-19T04:14:17.075544+0000","last_became_active":"2024-09-19T04:14:17.069539+0000","last_became_peered":"2024-09-19T04:14:17.069539+0000","last_unstale":"2024-09-19T04:14:17.075544+0000","last_undegraded":"2024-09-19T04:14:17.075544+0000","last_fullsized":"2024-09-19T04:14:17.075544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:31:50.043891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073417+0000","last_change":"2024-09-19T04:14:17.073417+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073417+0000","last_undegraded":"2024-09-19T04:14:17.073417+0000","last_fullsized":"2024-09-19T04:14:17.073417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101674+0000","last_change":"2024-09-19T04:14:17.094996+0000","last_active":"2024-09-19T04:14:17.101674+0000","last_peered":"2024-09-19T04:14:17.101674+0000","last_clean":"2024-09-19T04:14:17.101674+0000","last_became_active":"2024-09-19T04:14:17.083414+0000","last_became_peered":"2024-09-19T04:14:17.083414+0000","last_unstale":"2024-09-19T04:14:17.101674+0000","last_undegraded":"2024-09-19T04:14:17.101674+0000","last_fullsized":"2024-09-19T04:14:17.101674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:40.402841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074169+0000","last_change":"2024-09-19T04:14:17.074169+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074169+0000","last_undegraded":"2024-09-19T04:14:17.074169+0000","last_fullsized":"2024-09-19T04:14:17.074169+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098844+0000","last_change":"2024-09-19T04:14:17.094246+0000","last_active":"2024-09-19T04:14:17.098844+0000","last_peered":"2024-09-19T04:14:17.098844+0000","last_clean":"2024-09-19T04:14:17.098844+0000","last_became_active":"2024-09-19T04:14:17.083763+0000","last_became_peered":"2024-09-19T04:14:17.083763+0000","last_unstale":"2024-09-19T04:14:17.098844+0000","last_undegraded":"2024-09-19T04:14:17.098844+0000","last_fullsized":"2024-09-19T04:14:17.098844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:39.583489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062248+0000","last_change":"2024-09-19T04:14:17.062248+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062248+0000","last_undegraded":"2024-09-19T04:14:17.062248+0000","last_fullsized":"2024-09-19T04:14:17.062248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.095183+0000","last_change":"2024-09-19T04:14:17.094374+0000","last_active":"2024-09-19T04:14:17.095183+0000","last_peered":"2024-09-19T04:14:17.095183+0000","last_clean":"2024-09-19T04:14:17.095183+0000","last_became_active":"2024-09-19T04:14:17.093949+0000","last_became_peered":"2024-09-19T04:14:17.093949+0000","last_unstale":"2024-09-19T04:14:17.095183+0000","last_undegraded":"2024-09-19T04:14:17.095183+0000","last_fullsized":"2024-09-19T04:14:17.095183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:27.153452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.069357+0000","last_change":"2024-09-19T04:14:17.069357+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.069357+0000","last_undegraded":"2024-09-19T04:14:17.069357+0000","last_fullsized":"2024-09-19T04:14:17.069357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078270+0000","last_change":"2024-09-19T04:14:17.072319+0000","last_active":"2024-09-19T04:14:17.078270+0000","last_peered":"2024-09-19T04:14:17.078270+0000","last_clean":"2024-09-19T04:14:17.078270+0000","last_became_active":"2024-09-19T04:14:17.071960+0000","last_became_peered":"2024-09-19T04:14:17.071960+0000","last_unstale":"2024-09-19T04:14:17.078270+0000","last_undegraded":"2024-09-19T04:14:17.078270+0000","last_fullsized":"2024-09-19T04:14:17.078270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:56.776172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064480+0000","last_change":"2024-09-19T04:14:17.064480+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064480+0000","last_undegraded":"2024-09-19T04:14:17.064480+0000","last_fullsized":"2024-09-19T04:14:17.064480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.084958+0000","last_change":"2024-09-19T04:14:17.084245+0000","last_active":"2024-09-19T04:14:17.084958+0000","last_peered":"2024-09-19T04:14:17.084958+0000","last_clean":"2024-09-19T04:14:17.084958+0000","last_became_active":"2024-09-19T04:14:17.083828+0000","last_became_peered":"2024-09-19T04:14:17.083828+0000","last_unstale":"2024-09-19T04:14:17.084958+0000","last_undegraded":"2024-09-19T04:14:17.084958+0000","last_fullsized":"2024-09-19T04:14:17.084958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058557+0000","last_change":"2024-09-19T04:14:17.058557+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058557+0000","last_undegraded":"2024-09-19T04:14:17.058557+0000","last_fullsized":"2024-09-19T04:14:17.058557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091279+0000","last_change":"2024-09-19T04:14:17.085767+0000","last_active":"2024-09-19T04:14:17.091279+0000","last_peered":"2024-09-19T04:14:17.091279+0000","last_clean":"2024-09-19T04:14:17.091279+0000","last_became_active":"2024-09-19T04:14:17.085016+0000","last_became_peered":"2024-09-19T04:14:17.085016+0000","last_unstale":"2024-09-19T04:14:17.091279+0000","last_undegraded":"2024-09-19T04:14:17.091279+0000","last_fullsized":"2024-09-19T04:14:17.091279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065260+0000","last_change":"2024-09-19T04:14:17.065260+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065260+0000","last_undegraded":"2024-09-19T04:14:17.065260+0000","last_fullsized":"2024-09-19T04:14:17.065260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100905+0000","last_change":"2024-09-19T04:14:17.098594+0000","last_active":"2024-09-19T04:14:17.100905+0000","last_peered":"2024-09-19T04:14:17.100905+0000","last_clean":"2024-09-19T04:14:17.100905+0000","last_became_active":"2024-09-19T04:14:17.097858+0000","last_became_peered":"2024-09-19T04:14:17.097858+0000","last_unstale":"2024-09-19T04:14:17.100905+0000","last_undegraded":"2024-09-19T04:14:17.100905+0000","last_fullsized":"2024-09-19T04:14:17.100905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:26.792894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063556+0000","last_change":"2024-09-19T04:14:17.063556+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063556+0000","last_undegraded":"2024-09-19T04:14:17.063556+0000","last_fullsized":"2024-09-19T04:14:17.063556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101712+0000","last_change":"2024-09-19T04:14:17.095361+0000","last_active":"2024-09-19T04:14:17.101712+0000","last_peered":"2024-09-19T04:14:17.101712+0000","last_clean":"2024-09-19T04:14:17.101712+0000","last_became_active":"2024-09-19T04:14:17.094445+0000","last_became_peered":"2024-09-19T04:14:17.094445+0000","last_unstale":"2024-09-19T04:14:17.101712+0000","last_undegraded":"2024-09-19T04:14:17.101712+0000","last_fullsized":"2024-09-19T04:14:17.101712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:59.306800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065209+0000","last_change":"2024-09-19T04:14:17.065209+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065209+0000","last_undegraded":"2024-09-19T04:14:17.065209+0000","last_fullsized":"2024-09-19T04:14:17.065209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101020+0000","last_change":"2024-09-19T04:14:17.098487+0000","last_active":"2024-09-19T04:14:17.101020+0000","last_peered":"2024-09-19T04:14:17.101020+0000","last_clean":"2024-09-19T04:14:17.101020+0000","last_became_active":"2024-09-19T04:14:17.097805+0000","last_became_peered":"2024-09-19T04:14:17.097805+0000","last_unstale":"2024-09-19T04:14:17.101020+0000","last_undegraded":"2024-09-19T04:14:17.101020+0000","last_fullsized":"2024-09-19T04:14:17.101020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:55.367559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067888+0000","last_change":"2024-09-19T04:14:17.067888+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067888+0000","last_undegraded":"2024-09-19T04:14:17.067888+0000","last_fullsized":"2024-09-19T04:14:17.067888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094537+0000","last_change":"2024-09-19T04:14:17.093924+0000","last_active":"2024-09-19T04:14:17.094537+0000","last_peered":"2024-09-19T04:14:17.094537+0000","last_clean":"2024-09-19T04:14:17.094537+0000","last_became_active":"2024-09-19T04:14:17.093578+0000","last_became_peered":"2024-09-19T04:14:17.093578+0000","last_unstale":"2024-09-19T04:14:17.094537+0000","last_undegraded":"2024-09-19T04:14:17.094537+0000","last_fullsized":"2024-09-19T04:14:17.094537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:43:26.607609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061925+0000","last_change":"2024-09-19T04:14:17.061925+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061925+0000","last_undegraded":"2024-09-19T04:14:17.061925+0000","last_fullsized":"2024-09-19T04:14:17.061925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078381+0000","last_change":"2024-09-19T04:14:17.076993+0000","last_active":"2024-09-19T04:14:17.078381+0000","last_peered":"2024-09-19T04:14:17.078381+0000","last_clean":"2024-09-19T04:14:17.078381+0000","last_became_active":"2024-09-19T04:14:17.076271+0000","last_became_peered":"2024-09-19T04:14:17.076271+0000","last_unstale":"2024-09-19T04:14:17.078381+0000","last_undegraded":"2024-09-19T04:14:17.078381+0000","last_fullsized":"2024-09-19T04:14:17.078381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059387+0000","last_change":"2024-09-19T04:14:17.059387+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059387+0000","last_undegraded":"2024-09-19T04:14:17.059387+0000","last_fullsized":"2024-09-19T04:14:17.059387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075746+0000","last_change":"2024-09-19T04:14:17.071279+0000","last_active":"2024-09-19T04:14:17.075746+0000","last_peered":"2024-09-19T04:14:17.075746+0000","last_clean":"2024-09-19T04:14:17.075746+0000","last_became_active":"2024-09-19T04:14:17.070910+0000","last_became_peered":"2024-09-19T04:14:17.070910+0000","last_unstale":"2024-09-19T04:14:17.075746+0000","last_undegraded":"2024-09-19T04:14:17.075746+0000","last_fullsized":"2024-09-19T04:14:17.075746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:35.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067722+0000","last_change":"2024-09-19T04:14:17.067722+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067722+0000","last_undegraded":"2024-09-19T04:14:17.067722+0000","last_fullsized":"2024-09-19T04:14:17.067722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.081039+0000","last_change":"2024-09-19T04:14:17.080497+0000","last_active":"2024-09-19T04:14:17.081039+0000","last_peered":"2024-09-19T04:14:17.081039+0000","last_clean":"2024-09-19T04:14:17.081039+0000","last_became_active":"2024-09-19T04:14:17.080085+0000","last_became_peered":"2024-09-19T04:14:17.080085+0000","last_unstale":"2024-09-19T04:14:17.081039+0000","last_undegraded":"2024-09-19T04:14:17.081039+0000","last_fullsized":"2024-09-19T04:14:17.081039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:08.636704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065171+0000","last_change":"2024-09-19T04:14:17.065171+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065171+0000","last_undegraded":"2024-09-19T04:14:17.065171+0000","last_fullsized":"2024-09-19T04:14:17.065171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099918+0000","last_change":"2024-09-19T04:14:17.095657+0000","last_active":"2024-09-19T04:14:17.099918+0000","last_peered":"2024-09-19T04:14:17.099918+0000","last_clean":"2024-09-19T04:14:17.099918+0000","last_became_active":"2024-09-19T04:14:17.094938+0000","last_became_peered":"2024-09-19T04:14:17.094938+0000","last_unstale":"2024-09-19T04:14:17.099918+0000","last_undegraded":"2024-09-19T04:14:17.099918+0000","last_fullsized":"2024-09-19T04:14:17.099918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.454267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064362+0000","last_change":"2024-09-19T04:14:17.064362+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064362+0000","last_undegraded":"2024-09-19T04:14:17.064362+0000","last_fullsized":"2024-09-19T04:14:17.064362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.070502+0000","last_change":"2024-09-19T04:14:17.067723+0000","last_active":"2024-09-19T04:14:17.070502+0000","last_peered":"2024-09-19T04:14:17.070502+0000","last_clean":"2024-09-19T04:14:17.070502+0000","last_became_active":"2024-09-19T04:14:17.067393+0000","last_became_peered":"2024-09-19T04:14:17.067393+0000","last_unstale":"2024-09-19T04:14:17.070502+0000","last_undegraded":"2024-09-19T04:14:17.070502+0000","last_fullsized":"2024-09-19T04:14:17.070502+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:06.884322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064086+0000","last_change":"2024-09-19T04:14:17.064086+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064086+0000","last_undegraded":"2024-09-19T04:14:17.064086+0000","last_fullsized":"2024-09-19T04:14:17.064086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099437+0000","last_change":"2024-09-19T04:14:17.094649+0000","last_active":"2024-09-19T04:14:17.099437+0000","last_peered":"2024-09-19T04:14:17.099437+0000","last_clean":"2024-09-19T04:14:17.099437+0000","last_became_active":"2024-09-19T04:14:17.083575+0000","last_became_peered":"2024-09-19T04:14:17.083575+0000","last_unstale":"2024-09-19T04:14:17.099437+0000","last_undegraded":"2024-09-19T04:14:17.099437+0000","last_fullsized":"2024-09-19T04:14:17.099437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:13.045881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061389+0000","last_change":"2024-09-19T04:14:17.061389+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061389+0000","last_undegraded":"2024-09-19T04:14:17.061389+0000","last_fullsized":"2024-09-19T04:14:17.061389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098793+0000","last_change":"2024-09-19T04:14:17.097837+0000","last_active":"2024-09-19T04:14:17.098793+0000","last_peered":"2024-09-19T04:14:17.098793+0000","last_clean":"2024-09-19T04:14:17.098793+0000","last_became_active":"2024-09-19T04:14:17.097561+0000","last_became_peered":"2024-09-19T04:14:17.097561+0000","last_unstale":"2024-09-19T04:14:17.098793+0000","last_undegraded":"2024-09-19T04:14:17.098793+0000","last_fullsized":"2024-09-19T04:14:17.098793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:22.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062755+0000","last_change":"2024-09-19T04:14:17.062755+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062755+0000","last_undegraded":"2024-09-19T04:14:17.062755+0000","last_fullsized":"2024-09-19T04:14:17.062755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.083154+0000","last_change":"2024-09-19T04:14:17.074004+0000","last_active":"2024-09-19T04:14:17.083154+0000","last_peered":"2024-09-19T04:14:17.083154+0000","last_clean":"2024-09-19T04:14:17.083154+0000","last_became_active":"2024-09-19T04:14:17.073653+0000","last_became_peered":"2024-09-19T04:14:17.073653+0000","last_unstale":"2024-09-19T04:14:17.083154+0000","last_undegraded":"2024-09-19T04:14:17.083154+0000","last_fullsized":"2024-09-19T04:14:17.083154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068919+0000","last_change":"2024-09-19T04:14:17.068919+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068919+0000","last_undegraded":"2024-09-19T04:14:17.068919+0000","last_fullsized":"2024-09-19T04:14:17.068919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088034+0000","last_change":"2024-09-19T04:14:17.087371+0000","last_active":"2024-09-19T04:14:17.088034+0000","last_peered":"2024-09-19T04:14:17.088034+0000","last_clean":"2024-09-19T04:14:17.088034+0000","last_became_active":"2024-09-19T04:14:17.087047+0000","last_became_peered":"2024-09-19T04:14:17.087047+0000","last_unstale":"2024-09-19T04:14:17.088034+0000","last_undegraded":"2024-09-19T04:14:17.088034+0000","last_fullsized":"2024-09-19T04:14:17.088034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063449+0000","last_change":"2024-09-19T04:14:17.063449+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063449+0000","last_undegraded":"2024-09-19T04:14:17.063449+0000","last_fullsized":"2024-09-19T04:14:17.063449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088850+0000","last_change":"2024-09-19T04:14:17.086127+0000","last_active":"2024-09-19T04:14:17.088850+0000","last_peered":"2024-09-19T04:14:17.088850+0000","last_clean":"2024-09-19T04:14:17.088850+0000","last_became_active":"2024-09-19T04:14:17.084606+0000","last_became_peered":"2024-09-19T04:14:17.084606+0000","last_unstale":"2024-09-19T04:14:17.088850+0000","last_undegraded":"2024-09-19T04:14:17.088850+0000","last_fullsized":"2024-09-19T04:14:17.088850+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059105+0000","last_change":"2024-09-19T04:14:17.059105+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059105+0000","last_undegraded":"2024-09-19T04:14:17.059105+0000","last_fullsized":"2024-09-19T04:14:17.059105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078734+0000","last_change":"2024-09-19T04:14:17.070371+0000","last_active":"2024-09-19T04:14:17.078734+0000","last_peered":"2024-09-19T04:14:17.078734+0000","last_clean":"2024-09-19T04:14:17.078734+0000","last_became_active":"2024-09-19T04:14:17.069663+0000","last_became_peered":"2024-09-19T04:14:17.069663+0000","last_unstale":"2024-09-19T04:14:17.078734+0000","last_undegraded":"2024-09-19T04:14:17.078734+0000","last_fullsized":"2024-09-19T04:14:17.078734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059865+0000","last_change":"2024-09-19T04:14:17.059865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059865+0000","last_undegraded":"2024-09-19T04:14:17.059865+0000","last_fullsized":"2024-09-19T04:14:17.059865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075551+0000","last_change":"2024-09-19T04:14:17.070348+0000","last_active":"2024-09-19T04:14:17.075551+0000","last_peered":"2024-09-19T04:14:17.075551+0000","last_clean":"2024-09-19T04:14:17.075551+0000","last_became_active":"2024-09-19T04:14:17.069964+0000","last_became_peered":"2024-09-19T04:14:17.069964+0000","last_unstale":"2024-09-19T04:14:17.075551+0000","last_undegraded":"2024-09-19T04:14:17.075551+0000","last_fullsized":"2024-09-19T04:14:17.075551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:09.298918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068943+0000","last_change":"2024-09-19T04:14:17.068943+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068943+0000","last_undegraded":"2024-09-19T04:14:17.068943+0000","last_fullsized":"2024-09-19T04:14:17.068943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099762+0000","last_change":"2024-09-19T04:14:17.093374+0000","last_active":"2024-09-19T04:14:17.099762+0000","last_peered":"2024-09-19T04:14:17.099762+0000","last_clean":"2024-09-19T04:14:17.099762+0000","last_became_active":"2024-09-19T04:14:17.092649+0000","last_became_peered":"2024-09-19T04:14:17.092649+0000","last_unstale":"2024-09-19T04:14:17.099762+0000","last_undegraded":"2024-09-19T04:14:17.099762+0000","last_fullsized":"2024-09-19T04:14:17.099762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:34.491970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.087005+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:14:17.087005+0000","last_peered":"2024-09-19T04:14:17.087005+0000","last_clean":"2024-09-19T04:14:17.087005+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:14:17.087005+0000","last_undegraded":"2024-09-19T04:14:17.087005+0000","last_fullsized":"2024-09-19T04:14:17.087005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:14:21.421 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:14:21.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.568+0000 7f6a7f543640 1 Processor -- start 2024-09-19T04:14:21.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.568+0000 7f6a7f543640 1 -- start start 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7f543640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a780cf620 0x7f6a78168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7f543640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a780d0300 con 0x7f6a78161a60 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7f543640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a7806e6c0 con 0x7f6a780d3060 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7f543640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6a780cfa90 con 0x7f6a780cf620 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7d2df640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6a78161a60 0x7f6a780d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48356/0 (socket says 172.21.15.76:48356) 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7d2df640 1 -- 172.21.15.76:0/4188826278 learned_addr learned my addr 172.21.15.76:0/4188826278 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7dae0640 1 --2- 172.21.15.76:0/4188826278 >> v2:172.21.15.76:3300/0 conn(0x7f6a780cf620 0x7f6a78168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7dae0640 1 -- 172.21.15.76:0/4188826278 >> v1:172.21.15.116:6789/0 conn(0x7f6a780d3060 legacy=0x7f6a780cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:21.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7dae0640 1 -- 172.21.15.76:0/4188826278 >> v1:172.21.15.76:6789/0 conn(0x7f6a78161a60 legacy=0x7f6a780d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a6ffff640 1 -- 172.21.15.76:0/4188826278 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1120563276 0 0) 0x7f6a68001120 con 0x7f6a78161a60 2024-09-19T04:14:21.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7dae0640 1 -- 172.21.15.76:0/4188826278 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a780b0f60 con 0x7f6a780cf620 2024-09-19T04:14:21.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a7dae0640 1 --2- 172.21.15.76:0/4188826278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780cf620 0x7f6a78168f10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009d90 tx=0x7f6a7001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe243d1e01c3db7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a6ffff640 1 -- 172.21.15.76:0/4188826278 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a700b2070 con 0x7f6a780cf620 2024-09-19T04:14:21.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a6ffff640 1 -- 172.21.15.76:0/4188826278 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a700a5930 con 0x7f6a780cf620 2024-09-19T04:14:21.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.572+0000 7f6a6ffff640 1 -- 172.21.15.76:0/4188826278 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a700a5d00 con 0x7f6a780cf620 2024-09-19T04:14:21.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/4188826278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780cf620 msgr2=0x7f6a78168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 --2- 172.21.15.76:0/4188826278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780cf620 0x7f6a78168f10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6a70009d90 tx=0x7f6a7001c870 comp rx=0 tx=0).stop 2024-09-19T04:14:21.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/4188826278 shutdown_connections 2024-09-19T04:14:21.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 --2- 172.21.15.76:0/4188826278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780cf620 0x7f6a78168f10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/4188826278 >> 172.21.15.76:0/4188826278 conn(0x7f6a7808bf90 msgr2=0x7f6a7809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/4188826278 shutdown_connections 2024-09-19T04:14:21.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/4188826278 wait complete. 2024-09-19T04:14:21.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 Processor -- start 2024-09-19T04:14:21.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- start start 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a780cf620 0x7f6a78156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780d3060 0x7f6a7815ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a78161a60 0x7f6a780b9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7816bc70 con 0x7f6a780d3060 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a7806d190 con 0x7f6a78161a60 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6a7816bdf0 con 0x7f6a780cf620 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780d3060 0x7f6a7815ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780d3060 0x7f6a7815ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46836/0 (socket says 172.21.15.76:46836) 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 -- 172.21.15.76:0/778182031 learned_addr learned my addr 172.21.15.76:0/778182031 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 -- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a780cf620 msgr2=0x7f6a78156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7d2df640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a780cf620 0x7f6a78156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a780cf620 0x7f6a78156d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7dae0640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a78161a60 0x7f6a780b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 -- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a78161a60 msgr2=0x7f6a780b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a78161a60 0x7f6a780b9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 -- 172.21.15.76:0/778182031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a781fad60 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7cade640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780d3060 0x7f6a7815ed00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400c490 tx=0x7f6a74001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a6dffb640 1 -- 172.21.15.76:0/778182031 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74017070 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a6dffb640 1 -- 172.21.15.76:0/778182031 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a74003600 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a780b84c0 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a6dffb640 1 -- 172.21.15.76:0/778182031 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a74003a30 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a780b8760 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.576+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a780cef10 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.580+0000 7f6a6dffb640 1 -- 172.21.15.76:0/778182031 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a7401a070 con 0x7f6a780d3060 2024-09-19T04:14:21.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.580+0000 7f6a6dffb640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a5c0678c0 0x7f6a5c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.580+0000 7f6a6dffb640 1 -- 172.21.15.76:0/778182031 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f6a74085230 con 0x7f6a780d3060 2024-09-19T04:14:21.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.580+0000 7f6a7d2df640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a5c0678c0 0x7f6a5c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.580+0000 7f6a7d2df640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a5c0678c0 0x7f6a5c069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000f70 tx=0x7f6a68009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.584+0000 7f6a6dffb640 1 -- 172.21.15.76:0/778182031 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6a74055c70 con 0x7f6a780d3060 2024-09-19T04:14:21.748 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.744+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a780d1090 con 0x7f6a5c0678c0 2024-09-19T04:14:21.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.752+0000 7f6a6dffb640 1 -- 172.21.15.76:0/778182031 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246639 (secure 0 0 0) 0x7f6a780d1090 con 0x7f6a5c0678c0 2024-09-19T04:14:21.758 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:21.760 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a5c0678c0 msgr2=0x7f6a5c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a5c0678c0 0x7f6a5c069d80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a68000f70 tx=0x7f6a68009040 comp rx=0 tx=0).stop 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780d3060 msgr2=0x7f6a7815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780d3060 0x7f6a7815ed00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6a7400c490 tx=0x7f6a74001e80 comp rx=0 tx=0).stop 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 shutdown_connections 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a5c0678c0 0x7f6a5c069d80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a78161a60 0x7f6a780b9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a780d3060 0x7f6a7815ed00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 --2- 172.21.15.76:0/778182031 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a780cf620 0x7f6a78156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 >> 172.21.15.76:0/778182031 conn(0x7f6a7808bf90 msgr2=0x7f6a780d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 shutdown_connections 2024-09-19T04:14:21.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.756+0000 7f6a7f543640 1 -- 172.21.15.76:0/778182031 wait complete. 2024-09-19T04:14:21.776 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-19T04:14:19.754180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229008,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745712,"statfs":{"total":773094113280,"available":772859609088,"internally_reserved":0,"allocated":4562944,"data_stored":2617280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001142"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065006+0000","last_change":"2024-09-19T04:14:17.065006+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065006+0000","last_undegraded":"2024-09-19T04:14:17.065006+0000","last_fullsized":"2024-09-19T04:14:17.065006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063769+0000","last_change":"2024-09-19T04:14:17.063769+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063769+0000","last_undegraded":"2024-09-19T04:14:17.063769+0000","last_fullsized":"2024-09-19T04:14:17.063769+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072719+0000","last_change":"2024-09-19T04:14:17.072719+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072719+0000","last_undegraded":"2024-09-19T04:14:17.072719+0000","last_fullsized":"2024-09-19T04:14:17.072719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066271+0000","last_change":"2024-09-19T04:14:17.066271+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066271+0000","last_undegraded":"2024-09-19T04:14:17.066271+0000","last_fullsized":"2024-09-19T04:14:17.066271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065750+0000","last_change":"2024-09-19T04:14:17.065750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065750+0000","last_undegraded":"2024-09-19T04:14:17.065750+0000","last_fullsized":"2024-09-19T04:14:17.065750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066325+0000","last_change":"2024-09-19T04:14:17.066325+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066325+0000","last_undegraded":"2024-09-19T04:14:17.066325+0000","last_fullsized":"2024-09-19T04:14:17.066325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068080+0000","last_change":"2024-09-19T04:14:17.068080+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068080+0000","last_undegraded":"2024-09-19T04:14:17.068080+0000","last_fullsized":"2024-09-19T04:14:17.068080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064742+0000","last_change":"2024-09-19T04:14:17.064742+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064742+0000","last_undegraded":"2024-09-19T04:14:17.064742+0000","last_fullsized":"2024-09-19T04:14:17.064742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073302+0000","last_change":"2024-09-19T04:14:17.073302+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073302+0000","last_undegraded":"2024-09-19T04:14:17.073302+0000","last_fullsized":"2024-09-19T04:14:17.073302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070827+0000","last_change":"2024-09-19T04:14:17.070827+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070827+0000","last_undegraded":"2024-09-19T04:14:17.070827+0000","last_fullsized":"2024-09-19T04:14:17.070827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070514+0000","last_change":"2024-09-19T04:14:17.070514+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070514+0000","last_undegraded":"2024-09-19T04:14:17.070514+0000","last_fullsized":"2024-09-19T04:14:17.070514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073632+0000","last_change":"2024-09-19T04:14:17.073632+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073632+0000","last_undegraded":"2024-09-19T04:14:17.073632+0000","last_fullsized":"2024-09-19T04:14:17.073632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065671+0000","last_change":"2024-09-19T04:14:17.065671+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065671+0000","last_undegraded":"2024-09-19T04:14:17.065671+0000","last_fullsized":"2024-09-19T04:14:17.065671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062750+0000","last_change":"2024-09-19T04:14:17.062750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062750+0000","last_undegraded":"2024-09-19T04:14:17.062750+0000","last_fullsized":"2024-09-19T04:14:17.062750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066992+0000","last_change":"2024-09-19T04:14:17.066992+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066992+0000","last_undegraded":"2024-09-19T04:14:17.066992+0000","last_fullsized":"2024-09-19T04:14:17.066992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.060421+0000","last_change":"2024-09-19T04:14:17.060421+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.060421+0000","last_undegraded":"2024-09-19T04:14:17.060421+0000","last_fullsized":"2024-09-19T04:14:17.060421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058740+0000","last_change":"2024-09-19T04:14:17.058740+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058740+0000","last_undegraded":"2024-09-19T04:14:17.058740+0000","last_fullsized":"2024-09-19T04:14:17.058740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066865+0000","last_change":"2024-09-19T04:14:17.066865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066865+0000","last_undegraded":"2024-09-19T04:14:17.066865+0000","last_fullsized":"2024-09-19T04:14:17.066865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063176+0000","last_change":"2024-09-19T04:14:17.063176+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063176+0000","last_undegraded":"2024-09-19T04:14:17.063176+0000","last_fullsized":"2024-09-19T04:14:17.063176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068706+0000","last_change":"2024-09-19T04:14:17.068706+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068706+0000","last_undegraded":"2024-09-19T04:14:17.068706+0000","last_fullsized":"2024-09-19T04:14:17.068706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064596+0000","last_change":"2024-09-19T04:14:17.064596+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064596+0000","last_undegraded":"2024-09-19T04:14:17.064596+0000","last_fullsized":"2024-09-19T04:14:17.064596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.056764+0000","last_change":"2024-09-19T04:14:17.056764+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.056764+0000","last_undegraded":"2024-09-19T04:14:17.056764+0000","last_fullsized":"2024-09-19T04:14:17.056764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059772+0000","last_change":"2024-09-19T04:14:17.059772+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059772+0000","last_undegraded":"2024-09-19T04:14:17.059772+0000","last_fullsized":"2024-09-19T04:14:17.059772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062552+0000","last_change":"2024-09-19T04:14:17.062552+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062552+0000","last_undegraded":"2024-09-19T04:14:17.062552+0000","last_fullsized":"2024-09-19T04:14:17.062552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072830+0000","last_change":"2024-09-19T04:14:17.072830+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072830+0000","last_undegraded":"2024-09-19T04:14:17.072830+0000","last_fullsized":"2024-09-19T04:14:17.072830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063731+0000","last_change":"2024-09-19T04:14:17.063731+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063731+0000","last_undegraded":"2024-09-19T04:14:17.063731+0000","last_fullsized":"2024-09-19T04:14:17.063731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065739+0000","last_change":"2024-09-19T04:14:17.065739+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065739+0000","last_undegraded":"2024-09-19T04:14:17.065739+0000","last_fullsized":"2024-09-19T04:14:17.065739+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063806+0000","last_change":"2024-09-19T04:14:17.063806+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063806+0000","last_undegraded":"2024-09-19T04:14:17.063806+0000","last_fullsized":"2024-09-19T04:14:17.063806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064604+0000","last_change":"2024-09-19T04:14:17.064604+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064604+0000","last_undegraded":"2024-09-19T04:14:17.064604+0000","last_fullsized":"2024-09-19T04:14:17.064604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064345+0000","last_change":"2024-09-19T04:14:17.064345+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064345+0000","last_undegraded":"2024-09-19T04:14:17.064345+0000","last_fullsized":"2024-09-19T04:14:17.064345+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058614+0000","last_change":"2024-09-19T04:14:17.058614+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058614+0000","last_undegraded":"2024-09-19T04:14:17.058614+0000","last_fullsized":"2024-09-19T04:14:17.058614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064314+0000","last_change":"2024-09-19T04:14:17.064314+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064314+0000","last_undegraded":"2024-09-19T04:14:17.064314+0000","last_fullsized":"2024-09-19T04:14:17.064314+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059239+0000","last_change":"2024-09-19T04:14:17.059239+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059239+0000","last_undegraded":"2024-09-19T04:14:17.059239+0000","last_fullsized":"2024-09-19T04:14:17.059239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091472+0000","last_change":"2024-09-19T04:14:17.088483+0000","last_active":"2024-09-19T04:14:17.091472+0000","last_peered":"2024-09-19T04:14:17.091472+0000","last_clean":"2024-09-19T04:14:17.091472+0000","last_became_active":"2024-09-19T04:14:17.087794+0000","last_became_peered":"2024-09-19T04:14:17.087794+0000","last_unstale":"2024-09-19T04:14:17.091472+0000","last_undegraded":"2024-09-19T04:14:17.091472+0000","last_fullsized":"2024-09-19T04:14:17.091472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065208+0000","last_change":"2024-09-19T04:14:17.065208+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065208+0000","last_undegraded":"2024-09-19T04:14:17.065208+0000","last_fullsized":"2024-09-19T04:14:17.065208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.093823+0000","last_change":"2024-09-19T04:14:17.093169+0000","last_active":"2024-09-19T04:14:17.093823+0000","last_peered":"2024-09-19T04:14:17.093823+0000","last_clean":"2024-09-19T04:14:17.093823+0000","last_became_active":"2024-09-19T04:14:17.092881+0000","last_became_peered":"2024-09-19T04:14:17.092881+0000","last_unstale":"2024-09-19T04:14:17.093823+0000","last_undegraded":"2024-09-19T04:14:17.093823+0000","last_fullsized":"2024-09-19T04:14:17.093823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058473+0000","last_change":"2024-09-19T04:14:17.058473+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058473+0000","last_undegraded":"2024-09-19T04:14:17.058473+0000","last_fullsized":"2024-09-19T04:14:17.058473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078344+0000","last_change":"2024-09-19T04:14:17.076870+0000","last_active":"2024-09-19T04:14:17.078344+0000","last_peered":"2024-09-19T04:14:17.078344+0000","last_clean":"2024-09-19T04:14:17.078344+0000","last_became_active":"2024-09-19T04:14:17.076604+0000","last_became_peered":"2024-09-19T04:14:17.076604+0000","last_unstale":"2024-09-19T04:14:17.078344+0000","last_undegraded":"2024-09-19T04:14:17.078344+0000","last_fullsized":"2024-09-19T04:14:17.078344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062853+0000","last_change":"2024-09-19T04:14:17.062853+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062853+0000","last_undegraded":"2024-09-19T04:14:17.062853+0000","last_fullsized":"2024-09-19T04:14:17.062853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.079157+0000","last_change":"2024-09-19T04:14:17.072407+0000","last_active":"2024-09-19T04:14:17.079157+0000","last_peered":"2024-09-19T04:14:17.079157+0000","last_clean":"2024-09-19T04:14:17.079157+0000","last_became_active":"2024-09-19T04:14:17.071689+0000","last_became_peered":"2024-09-19T04:14:17.071689+0000","last_unstale":"2024-09-19T04:14:17.079157+0000","last_undegraded":"2024-09-19T04:14:17.079157+0000","last_fullsized":"2024-09-19T04:14:17.079157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074684+0000","last_change":"2024-09-19T04:14:17.074684+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074684+0000","last_undegraded":"2024-09-19T04:14:17.074684+0000","last_fullsized":"2024-09-19T04:14:17.074684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098241+0000","last_change":"2024-09-19T04:14:17.083299+0000","last_active":"2024-09-19T04:14:17.098241+0000","last_peered":"2024-09-19T04:14:17.098241+0000","last_clean":"2024-09-19T04:14:17.098241+0000","last_became_active":"2024-09-19T04:14:17.081493+0000","last_became_peered":"2024-09-19T04:14:17.081493+0000","last_unstale":"2024-09-19T04:14:17.098241+0000","last_undegraded":"2024-09-19T04:14:17.098241+0000","last_fullsized":"2024-09-19T04:14:17.098241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065920+0000","last_change":"2024-09-19T04:14:17.065920+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065920+0000","last_undegraded":"2024-09-19T04:14:17.065920+0000","last_fullsized":"2024-09-19T04:14:17.065920+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100822+0000","last_change":"2024-09-19T04:14:17.098330+0000","last_active":"2024-09-19T04:14:17.100822+0000","last_peered":"2024-09-19T04:14:17.100822+0000","last_clean":"2024-09-19T04:14:17.100822+0000","last_became_active":"2024-09-19T04:14:17.097696+0000","last_became_peered":"2024-09-19T04:14:17.097696+0000","last_unstale":"2024-09-19T04:14:17.100822+0000","last_undegraded":"2024-09-19T04:14:17.100822+0000","last_fullsized":"2024-09-19T04:14:17.100822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:46.739371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058025+0000","last_change":"2024-09-19T04:14:17.058025+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058025+0000","last_undegraded":"2024-09-19T04:14:17.058025+0000","last_fullsized":"2024-09-19T04:14:17.058025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098750+0000","last_change":"2024-09-19T04:14:17.097839+0000","last_active":"2024-09-19T04:14:17.098750+0000","last_peered":"2024-09-19T04:14:17.098750+0000","last_clean":"2024-09-19T04:14:17.098750+0000","last_became_active":"2024-09-19T04:14:17.097522+0000","last_became_peered":"2024-09-19T04:14:17.097522+0000","last_unstale":"2024-09-19T04:14:17.098750+0000","last_undegraded":"2024-09-19T04:14:17.098750+0000","last_fullsized":"2024-09-19T04:14:17.098750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065710+0000","last_change":"2024-09-19T04:14:17.065710+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065710+0000","last_undegraded":"2024-09-19T04:14:17.065710+0000","last_fullsized":"2024-09-19T04:14:17.065710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094043+0000","last_change":"2024-09-19T04:14:17.088506+0000","last_active":"2024-09-19T04:14:17.094043+0000","last_peered":"2024-09-19T04:14:17.094043+0000","last_clean":"2024-09-19T04:14:17.094043+0000","last_became_active":"2024-09-19T04:14:17.087746+0000","last_became_peered":"2024-09-19T04:14:17.087746+0000","last_unstale":"2024-09-19T04:14:17.094043+0000","last_undegraded":"2024-09-19T04:14:17.094043+0000","last_fullsized":"2024-09-19T04:14:17.094043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062249+0000","last_change":"2024-09-19T04:14:17.062249+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062249+0000","last_undegraded":"2024-09-19T04:14:17.062249+0000","last_fullsized":"2024-09-19T04:14:17.062249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075544+0000","last_change":"2024-09-19T04:14:17.070017+0000","last_active":"2024-09-19T04:14:17.075544+0000","last_peered":"2024-09-19T04:14:17.075544+0000","last_clean":"2024-09-19T04:14:17.075544+0000","last_became_active":"2024-09-19T04:14:17.069539+0000","last_became_peered":"2024-09-19T04:14:17.069539+0000","last_unstale":"2024-09-19T04:14:17.075544+0000","last_undegraded":"2024-09-19T04:14:17.075544+0000","last_fullsized":"2024-09-19T04:14:17.075544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:31:50.043891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073417+0000","last_change":"2024-09-19T04:14:17.073417+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073417+0000","last_undegraded":"2024-09-19T04:14:17.073417+0000","last_fullsized":"2024-09-19T04:14:17.073417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101674+0000","last_change":"2024-09-19T04:14:17.094996+0000","last_active":"2024-09-19T04:14:17.101674+0000","last_peered":"2024-09-19T04:14:17.101674+0000","last_clean":"2024-09-19T04:14:17.101674+0000","last_became_active":"2024-09-19T04:14:17.083414+0000","last_became_peered":"2024-09-19T04:14:17.083414+0000","last_unstale":"2024-09-19T04:14:17.101674+0000","last_undegraded":"2024-09-19T04:14:17.101674+0000","last_fullsized":"2024-09-19T04:14:17.101674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:40.402841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074169+0000","last_change":"2024-09-19T04:14:17.074169+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074169+0000","last_undegraded":"2024-09-19T04:14:17.074169+0000","last_fullsized":"2024-09-19T04:14:17.074169+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098844+0000","last_change":"2024-09-19T04:14:17.094246+0000","last_active":"2024-09-19T04:14:17.098844+0000","last_peered":"2024-09-19T04:14:17.098844+0000","last_clean":"2024-09-19T04:14:17.098844+0000","last_became_active":"2024-09-19T04:14:17.083763+0000","last_became_peered":"2024-09-19T04:14:17.083763+0000","last_unstale":"2024-09-19T04:14:17.098844+0000","last_undegraded":"2024-09-19T04:14:17.098844+0000","last_fullsized":"2024-09-19T04:14:17.098844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:39.583489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062248+0000","last_change":"2024-09-19T04:14:17.062248+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062248+0000","last_undegraded":"2024-09-19T04:14:17.062248+0000","last_fullsized":"2024-09-19T04:14:17.062248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.095183+0000","last_change":"2024-09-19T04:14:17.094374+0000","last_active":"2024-09-19T04:14:17.095183+0000","last_peered":"2024-09-19T04:14:17.095183+0000","last_clean":"2024-09-19T04:14:17.095183+0000","last_became_active":"2024-09-19T04:14:17.093949+0000","last_became_peered":"2024-09-19T04:14:17.093949+0000","last_unstale":"2024-09-19T04:14:17.095183+0000","last_undegraded":"2024-09-19T04:14:17.095183+0000","last_fullsized":"2024-09-19T04:14:17.095183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:27.153452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.069357+0000","last_change":"2024-09-19T04:14:17.069357+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.069357+0000","last_undegraded":"2024-09-19T04:14:17.069357+0000","last_fullsized":"2024-09-19T04:14:17.069357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078270+0000","last_change":"2024-09-19T04:14:17.072319+0000","last_active":"2024-09-19T04:14:17.078270+0000","last_peered":"2024-09-19T04:14:17.078270+0000","last_clean":"2024-09-19T04:14:17.078270+0000","last_became_active":"2024-09-19T04:14:17.071960+0000","last_became_peered":"2024-09-19T04:14:17.071960+0000","last_unstale":"2024-09-19T04:14:17.078270+0000","last_undegraded":"2024-09-19T04:14:17.078270+0000","last_fullsized":"2024-09-19T04:14:17.078270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:56.776172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064480+0000","last_change":"2024-09-19T04:14:17.064480+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064480+0000","last_undegraded":"2024-09-19T04:14:17.064480+0000","last_fullsized":"2024-09-19T04:14:17.064480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.084958+0000","last_change":"2024-09-19T04:14:17.084245+0000","last_active":"2024-09-19T04:14:17.084958+0000","last_peered":"2024-09-19T04:14:17.084958+0000","last_clean":"2024-09-19T04:14:17.084958+0000","last_became_active":"2024-09-19T04:14:17.083828+0000","last_became_peered":"2024-09-19T04:14:17.083828+0000","last_unstale":"2024-09-19T04:14:17.084958+0000","last_undegraded":"2024-09-19T04:14:17.084958+0000","last_fullsized":"2024-09-19T04:14:17.084958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058557+0000","last_change":"2024-09-19T04:14:17.058557+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058557+0000","last_undegraded":"2024-09-19T04:14:17.058557+0000","last_fullsized":"2024-09-19T04:14:17.058557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091279+0000","last_change":"2024-09-19T04:14:17.085767+0000","last_active":"2024-09-19T04:14:17.091279+0000","last_peered":"2024-09-19T04:14:17.091279+0000","last_clean":"2024-09-19T04:14:17.091279+0000","last_became_active":"2024-09-19T04:14:17.085016+0000","last_became_peered":"2024-09-19T04:14:17.085016+0000","last_unstale":"2024-09-19T04:14:17.091279+0000","last_undegraded":"2024-09-19T04:14:17.091279+0000","last_fullsized":"2024-09-19T04:14:17.091279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065260+0000","last_change":"2024-09-19T04:14:17.065260+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065260+0000","last_undegraded":"2024-09-19T04:14:17.065260+0000","last_fullsized":"2024-09-19T04:14:17.065260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100905+0000","last_change":"2024-09-19T04:14:17.098594+0000","last_active":"2024-09-19T04:14:17.100905+0000","last_peered":"2024-09-19T04:14:17.100905+0000","last_clean":"2024-09-19T04:14:17.100905+0000","last_became_active":"2024-09-19T04:14:17.097858+0000","last_became_peered":"2024-09-19T04:14:17.097858+0000","last_unstale":"2024-09-19T04:14:17.100905+0000","last_undegraded":"2024-09-19T04:14:17.100905+0000","last_fullsized":"2024-09-19T04:14:17.100905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:26.792894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063556+0000","last_change":"2024-09-19T04:14:17.063556+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063556+0000","last_undegraded":"2024-09-19T04:14:17.063556+0000","last_fullsized":"2024-09-19T04:14:17.063556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101712+0000","last_change":"2024-09-19T04:14:17.095361+0000","last_active":"2024-09-19T04:14:17.101712+0000","last_peered":"2024-09-19T04:14:17.101712+0000","last_clean":"2024-09-19T04:14:17.101712+0000","last_became_active":"2024-09-19T04:14:17.094445+0000","last_became_peered":"2024-09-19T04:14:17.094445+0000","last_unstale":"2024-09-19T04:14:17.101712+0000","last_undegraded":"2024-09-19T04:14:17.101712+0000","last_fullsized":"2024-09-19T04:14:17.101712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:59.306800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065209+0000","last_change":"2024-09-19T04:14:17.065209+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065209+0000","last_undegraded":"2024-09-19T04:14:17.065209+0000","last_fullsized":"2024-09-19T04:14:17.065209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101020+0000","last_change":"2024-09-19T04:14:17.098487+0000","last_active":"2024-09-19T04:14:17.101020+0000","last_peered":"2024-09-19T04:14:17.101020+0000","last_clean":"2024-09-19T04:14:17.101020+0000","last_became_active":"2024-09-19T04:14:17.097805+0000","last_became_peered":"2024-09-19T04:14:17.097805+0000","last_unstale":"2024-09-19T04:14:17.101020+0000","last_undegraded":"2024-09-19T04:14:17.101020+0000","last_fullsized":"2024-09-19T04:14:17.101020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:55.367559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067888+0000","last_change":"2024-09-19T04:14:17.067888+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067888+0000","last_undegraded":"2024-09-19T04:14:17.067888+0000","last_fullsized":"2024-09-19T04:14:17.067888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094537+0000","last_change":"2024-09-19T04:14:17.093924+0000","last_active":"2024-09-19T04:14:17.094537+0000","last_peered":"2024-09-19T04:14:17.094537+0000","last_clean":"2024-09-19T04:14:17.094537+0000","last_became_active":"2024-09-19T04:14:17.093578+0000","last_became_peered":"2024-09-19T04:14:17.093578+0000","last_unstale":"2024-09-19T04:14:17.094537+0000","last_undegraded":"2024-09-19T04:14:17.094537+0000","last_fullsized":"2024-09-19T04:14:17.094537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:43:26.607609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061925+0000","last_change":"2024-09-19T04:14:17.061925+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061925+0000","last_undegraded":"2024-09-19T04:14:17.061925+0000","last_fullsized":"2024-09-19T04:14:17.061925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078381+0000","last_change":"2024-09-19T04:14:17.076993+0000","last_active":"2024-09-19T04:14:17.078381+0000","last_peered":"2024-09-19T04:14:17.078381+0000","last_clean":"2024-09-19T04:14:17.078381+0000","last_became_active":"2024-09-19T04:14:17.076271+0000","last_became_peered":"2024-09-19T04:14:17.076271+0000","last_unstale":"2024-09-19T04:14:17.078381+0000","last_undegraded":"2024-09-19T04:14:17.078381+0000","last_fullsized":"2024-09-19T04:14:17.078381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059387+0000","last_change":"2024-09-19T04:14:17.059387+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059387+0000","last_undegraded":"2024-09-19T04:14:17.059387+0000","last_fullsized":"2024-09-19T04:14:17.059387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075746+0000","last_change":"2024-09-19T04:14:17.071279+0000","last_active":"2024-09-19T04:14:17.075746+0000","last_peered":"2024-09-19T04:14:17.075746+0000","last_clean":"2024-09-19T04:14:17.075746+0000","last_became_active":"2024-09-19T04:14:17.070910+0000","last_became_peered":"2024-09-19T04:14:17.070910+0000","last_unstale":"2024-09-19T04:14:17.075746+0000","last_undegraded":"2024-09-19T04:14:17.075746+0000","last_fullsized":"2024-09-19T04:14:17.075746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:35.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067722+0000","last_change":"2024-09-19T04:14:17.067722+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067722+0000","last_undegraded":"2024-09-19T04:14:17.067722+0000","last_fullsized":"2024-09-19T04:14:17.067722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.081039+0000","last_change":"2024-09-19T04:14:17.080497+0000","last_active":"2024-09-19T04:14:17.081039+0000","last_peered":"2024-09-19T04:14:17.081039+0000","last_clean":"2024-09-19T04:14:17.081039+0000","last_became_active":"2024-09-19T04:14:17.080085+0000","last_became_peered":"2024-09-19T04:14:17.080085+0000","last_unstale":"2024-09-19T04:14:17.081039+0000","last_undegraded":"2024-09-19T04:14:17.081039+0000","last_fullsized":"2024-09-19T04:14:17.081039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:08.636704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065171+0000","last_change":"2024-09-19T04:14:17.065171+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065171+0000","last_undegraded":"2024-09-19T04:14:17.065171+0000","last_fullsized":"2024-09-19T04:14:17.065171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099918+0000","last_change":"2024-09-19T04:14:17.095657+0000","last_active":"2024-09-19T04:14:17.099918+0000","last_peered":"2024-09-19T04:14:17.099918+0000","last_clean":"2024-09-19T04:14:17.099918+0000","last_became_active":"2024-09-19T04:14:17.094938+0000","last_became_peered":"2024-09-19T04:14:17.094938+0000","last_unstale":"2024-09-19T04:14:17.099918+0000","last_undegraded":"2024-09-19T04:14:17.099918+0000","last_fullsized":"2024-09-19T04:14:17.099918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.454267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064362+0000","last_change":"2024-09-19T04:14:17.064362+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064362+0000","last_undegraded":"2024-09-19T04:14:17.064362+0000","last_fullsized":"2024-09-19T04:14:17.064362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.070502+0000","last_change":"2024-09-19T04:14:17.067723+0000","last_active":"2024-09-19T04:14:17.070502+0000","last_peered":"2024-09-19T04:14:17.070502+0000","last_clean":"2024-09-19T04:14:17.070502+0000","last_became_active":"2024-09-19T04:14:17.067393+0000","last_became_peered":"2024-09-19T04:14:17.067393+0000","last_unstale":"2024-09-19T04:14:17.070502+0000","last_undegraded":"2024-09-19T04:14:17.070502+0000","last_fullsized":"2024-09-19T04:14:17.070502+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:06.884322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064086+0000","last_change":"2024-09-19T04:14:17.064086+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064086+0000","last_undegraded":"2024-09-19T04:14:17.064086+0000","last_fullsized":"2024-09-19T04:14:17.064086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099437+0000","last_change":"2024-09-19T04:14:17.094649+0000","last_active":"2024-09-19T04:14:17.099437+0000","last_peered":"2024-09-19T04:14:17.099437+0000","last_clean":"2024-09-19T04:14:17.099437+0000","last_became_active":"2024-09-19T04:14:17.083575+0000","last_became_peered":"2024-09-19T04:14:17.083575+0000","last_unstale":"2024-09-19T04:14:17.099437+0000","last_undegraded":"2024-09-19T04:14:17.099437+0000","last_fullsized":"2024-09-19T04:14:17.099437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:13.045881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061389+0000","last_change":"2024-09-19T04:14:17.061389+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061389+0000","last_undegraded":"2024-09-19T04:14:17.061389+0000","last_fullsized":"2024-09-19T04:14:17.061389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098793+0000","last_change":"2024-09-19T04:14:17.097837+0000","last_active":"2024-09-19T04:14:17.098793+0000","last_peered":"2024-09-19T04:14:17.098793+0000","last_clean":"2024-09-19T04:14:17.098793+0000","last_became_active":"2024-09-19T04:14:17.097561+0000","last_became_peered":"2024-09-19T04:14:17.097561+0000","last_unstale":"2024-09-19T04:14:17.098793+0000","last_undegraded":"2024-09-19T04:14:17.098793+0000","last_fullsized":"2024-09-19T04:14:17.098793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:22.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062755+0000","last_change":"2024-09-19T04:14:17.062755+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062755+0000","last_undegraded":"2024-09-19T04:14:17.062755+0000","last_fullsized":"2024-09-19T04:14:17.062755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.083154+0000","last_change":"2024-09-19T04:14:17.074004+0000","last_active":"2024-09-19T04:14:17.083154+0000","last_peered":"2024-09-19T04:14:17.083154+0000","last_clean":"2024-09-19T04:14:17.083154+0000","last_became_active":"2024-09-19T04:14:17.073653+0000","last_became_peered":"2024-09-19T04:14:17.073653+0000","last_unstale":"2024-09-19T04:14:17.083154+0000","last_undegraded":"2024-09-19T04:14:17.083154+0000","last_fullsized":"2024-09-19T04:14:17.083154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068919+0000","last_change":"2024-09-19T04:14:17.068919+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068919+0000","last_undegraded":"2024-09-19T04:14:17.068919+0000","last_fullsized":"2024-09-19T04:14:17.068919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088034+0000","last_change":"2024-09-19T04:14:17.087371+0000","last_active":"2024-09-19T04:14:17.088034+0000","last_peered":"2024-09-19T04:14:17.088034+0000","last_clean":"2024-09-19T04:14:17.088034+0000","last_became_active":"2024-09-19T04:14:17.087047+0000","last_became_peered":"2024-09-19T04:14:17.087047+0000","last_unstale":"2024-09-19T04:14:17.088034+0000","last_undegraded":"2024-09-19T04:14:17.088034+0000","last_fullsized":"2024-09-19T04:14:17.088034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063449+0000","last_change":"2024-09-19T04:14:17.063449+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063449+0000","last_undegraded":"2024-09-19T04:14:17.063449+0000","last_fullsized":"2024-09-19T04:14:17.063449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088850+0000","last_change":"2024-09-19T04:14:17.086127+0000","last_active":"2024-09-19T04:14:17.088850+0000","last_peered":"2024-09-19T04:14:17.088850+0000","last_clean":"2024-09-19T04:14:17.088850+0000","last_became_active":"2024-09-19T04:14:17.084606+0000","last_became_peered":"2024-09-19T04:14:17.084606+0000","last_unstale":"2024-09-19T04:14:17.088850+0000","last_undegraded":"2024-09-19T04:14:17.088850+0000","last_fullsized":"2024-09-19T04:14:17.088850+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059105+0000","last_change":"2024-09-19T04:14:17.059105+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059105+0000","last_undegraded":"2024-09-19T04:14:17.059105+0000","last_fullsized":"2024-09-19T04:14:17.059105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078734+0000","last_change":"2024-09-19T04:14:17.070371+0000","last_active":"2024-09-19T04:14:17.078734+0000","last_peered":"2024-09-19T04:14:17.078734+0000","last_clean":"2024-09-19T04:14:17.078734+0000","last_became_active":"2024-09-19T04:14:17.069663+0000","last_became_peered":"2024-09-19T04:14:17.069663+0000","last_unstale":"2024-09-19T04:14:17.078734+0000","last_undegraded":"2024-09-19T04:14:17.078734+0000","last_fullsized":"2024-09-19T04:14:17.078734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059865+0000","last_change":"2024-09-19T04:14:17.059865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059865+0000","last_undegraded":"2024-09-19T04:14:17.059865+0000","last_fullsized":"2024-09-19T04:14:17.059865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075551+0000","last_change":"2024-09-19T04:14:17.070348+0000","last_active":"2024-09-19T04:14:17.075551+0000","last_peered":"2024-09-19T04:14:17.075551+0000","last_clean":"2024-09-19T04:14:17.075551+0000","last_became_active":"2024-09-19T04:14:17.069964+0000","last_became_peered":"2024-09-19T04:14:17.069964+0000","last_unstale":"2024-09-19T04:14:17.075551+0000","last_undegraded":"2024-09-19T04:14:17.075551+0000","last_fullsized":"2024-09-19T04:14:17.075551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:09.298918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068943+0000","last_change":"2024-09-19T04:14:17.068943+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068943+0000","last_undegraded":"2024-09-19T04:14:17.068943+0000","last_fullsized":"2024-09-19T04:14:17.068943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099762+0000","last_change":"2024-09-19T04:14:17.093374+0000","last_active":"2024-09-19T04:14:17.099762+0000","last_peered":"2024-09-19T04:14:17.099762+0000","last_clean":"2024-09-19T04:14:17.099762+0000","last_became_active":"2024-09-19T04:14:17.092649+0000","last_became_peered":"2024-09-19T04:14:17.092649+0000","last_unstale":"2024-09-19T04:14:17.099762+0000","last_undegraded":"2024-09-19T04:14:17.099762+0000","last_fullsized":"2024-09-19T04:14:17.099762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:34.491970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.087005+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:14:17.087005+0000","last_peered":"2024-09-19T04:14:17.087005+0000","last_clean":"2024-09-19T04:14:17.087005+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:14:17.087005+0000","last_undegraded":"2024-09-19T04:14:17.087005+0000","last_fullsized":"2024-09-19T04:14:17.087005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:14:21.779 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3f is not active+clean 2024-09-19T04:14:21.779 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065006+0000', 'last_change': '2024-09-19T04:14:17.065006+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065006+0000', 'last_undegraded': '2024-09-19T04:14:17.065006+0000', 'last_fullsized': '2024-09-19T04:14:17.065006+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3e is not active+clean 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.063769+0000', 'last_change': '2024-09-19T04:14:17.063769+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.063769+0000', 'last_undegraded': '2024-09-19T04:14:17.063769+0000', 'last_fullsized': '2024-09-19T04:14:17.063769+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3d is not active+clean 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.072719+0000', 'last_change': '2024-09-19T04:14:17.072719+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.072719+0000', 'last_undegraded': '2024-09-19T04:14:17.072719+0000', 'last_fullsized': '2024-09-19T04:14:17.072719+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3c is not active+clean 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.066271+0000', 'last_change': '2024-09-19T04:14:17.066271+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.066271+0000', 'last_undegraded': '2024-09-19T04:14:17.066271+0000', 'last_fullsized': '2024-09-19T04:14:17.066271+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3b is not active+clean 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065750+0000', 'last_change': '2024-09-19T04:14:17.065750+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065750+0000', 'last_undegraded': '2024-09-19T04:14:17.065750+0000', 'last_fullsized': '2024-09-19T04:14:17.065750+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3a is not active+clean 2024-09-19T04:14:21.780 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.066325+0000', 'last_change': '2024-09-19T04:14:17.066325+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.066325+0000', 'last_undegraded': '2024-09-19T04:14:17.066325+0000', 'last_fullsized': '2024-09-19T04:14:17.066325+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:PG 6.39 is not active+clean 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.068080+0000', 'last_change': '2024-09-19T04:14:17.068080+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.068080+0000', 'last_undegraded': '2024-09-19T04:14:17.068080+0000', 'last_fullsized': '2024-09-19T04:14:17.068080+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:PG 6.38 is not active+clean 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064742+0000', 'last_change': '2024-09-19T04:14:17.064742+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064742+0000', 'last_undegraded': '2024-09-19T04:14:17.064742+0000', 'last_fullsized': '2024-09-19T04:14:17.064742+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:PG 6.37 is not active+clean 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.073302+0000', 'last_change': '2024-09-19T04:14:17.073302+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.073302+0000', 'last_undegraded': '2024-09-19T04:14:17.073302+0000', 'last_fullsized': '2024-09-19T04:14:17.073302+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:PG 6.36 is not active+clean 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.070827+0000', 'last_change': '2024-09-19T04:14:17.070827+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.070827+0000', 'last_undegraded': '2024-09-19T04:14:17.070827+0000', 'last_fullsized': '2024-09-19T04:14:17.070827+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:PG 6.35 is not active+clean 2024-09-19T04:14:21.781 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.070514+0000', 'last_change': '2024-09-19T04:14:17.070514+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.070514+0000', 'last_undegraded': '2024-09-19T04:14:17.070514+0000', 'last_fullsized': '2024-09-19T04:14:17.070514+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:PG 6.34 is not active+clean 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.073632+0000', 'last_change': '2024-09-19T04:14:17.073632+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.073632+0000', 'last_undegraded': '2024-09-19T04:14:17.073632+0000', 'last_fullsized': '2024-09-19T04:14:17.073632+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:PG 6.33 is not active+clean 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065671+0000', 'last_change': '2024-09-19T04:14:17.065671+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065671+0000', 'last_undegraded': '2024-09-19T04:14:17.065671+0000', 'last_fullsized': '2024-09-19T04:14:17.065671+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:PG 6.32 is not active+clean 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.062750+0000', 'last_change': '2024-09-19T04:14:17.062750+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.062750+0000', 'last_undegraded': '2024-09-19T04:14:17.062750+0000', 'last_fullsized': '2024-09-19T04:14:17.062750+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:PG 6.31 is not active+clean 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.066992+0000', 'last_change': '2024-09-19T04:14:17.066992+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.066992+0000', 'last_undegraded': '2024-09-19T04:14:17.066992+0000', 'last_fullsized': '2024-09-19T04:14:17.066992+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:PG 6.30 is not active+clean 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.060421+0000', 'last_change': '2024-09-19T04:14:17.060421+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.060421+0000', 'last_undegraded': '2024-09-19T04:14:17.060421+0000', 'last_fullsized': '2024-09-19T04:14:17.060421+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.782 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2f is not active+clean 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.058740+0000', 'last_change': '2024-09-19T04:14:17.058740+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.058740+0000', 'last_undegraded': '2024-09-19T04:14:17.058740+0000', 'last_fullsized': '2024-09-19T04:14:17.058740+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2e is not active+clean 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.066865+0000', 'last_change': '2024-09-19T04:14:17.066865+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.066865+0000', 'last_undegraded': '2024-09-19T04:14:17.066865+0000', 'last_fullsized': '2024-09-19T04:14:17.066865+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2d is not active+clean 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.063176+0000', 'last_change': '2024-09-19T04:14:17.063176+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.063176+0000', 'last_undegraded': '2024-09-19T04:14:17.063176+0000', 'last_fullsized': '2024-09-19T04:14:17.063176+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2c is not active+clean 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.068706+0000', 'last_change': '2024-09-19T04:14:17.068706+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.068706+0000', 'last_undegraded': '2024-09-19T04:14:17.068706+0000', 'last_fullsized': '2024-09-19T04:14:17.068706+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2b is not active+clean 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064596+0000', 'last_change': '2024-09-19T04:14:17.064596+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064596+0000', 'last_undegraded': '2024-09-19T04:14:17.064596+0000', 'last_fullsized': '2024-09-19T04:14:17.064596+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2a is not active+clean 2024-09-19T04:14:21.783 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.056764+0000', 'last_change': '2024-09-19T04:14:17.056764+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.056764+0000', 'last_undegraded': '2024-09-19T04:14:17.056764+0000', 'last_fullsized': '2024-09-19T04:14:17.056764+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.29 is not active+clean 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.059772+0000', 'last_change': '2024-09-19T04:14:17.059772+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.059772+0000', 'last_undegraded': '2024-09-19T04:14:17.059772+0000', 'last_fullsized': '2024-09-19T04:14:17.059772+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.28 is not active+clean 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.062552+0000', 'last_change': '2024-09-19T04:14:17.062552+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.062552+0000', 'last_undegraded': '2024-09-19T04:14:17.062552+0000', 'last_fullsized': '2024-09-19T04:14:17.062552+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.27 is not active+clean 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.072830+0000', 'last_change': '2024-09-19T04:14:17.072830+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.072830+0000', 'last_undegraded': '2024-09-19T04:14:17.072830+0000', 'last_fullsized': '2024-09-19T04:14:17.072830+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.26 is not active+clean 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.063731+0000', 'last_change': '2024-09-19T04:14:17.063731+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.063731+0000', 'last_undegraded': '2024-09-19T04:14:17.063731+0000', 'last_fullsized': '2024-09-19T04:14:17.063731+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.25 is not active+clean 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065739+0000', 'last_change': '2024-09-19T04:14:17.065739+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065739+0000', 'last_undegraded': '2024-09-19T04:14:17.065739+0000', 'last_fullsized': '2024-09-19T04:14:17.065739+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:14:21.784 INFO:tasks.ceph.ceph_manager.ceph:PG 6.24 is not active+clean 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.063806+0000', 'last_change': '2024-09-19T04:14:17.063806+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.063806+0000', 'last_undegraded': '2024-09-19T04:14:17.063806+0000', 'last_fullsized': '2024-09-19T04:14:17.063806+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.23 is not active+clean 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064604+0000', 'last_change': '2024-09-19T04:14:17.064604+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064604+0000', 'last_undegraded': '2024-09-19T04:14:17.064604+0000', 'last_fullsized': '2024-09-19T04:14:17.064604+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.22 is not active+clean 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064345+0000', 'last_change': '2024-09-19T04:14:17.064345+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064345+0000', 'last_undegraded': '2024-09-19T04:14:17.064345+0000', 'last_fullsized': '2024-09-19T04:14:17.064345+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.21 is not active+clean 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.058614+0000', 'last_change': '2024-09-19T04:14:17.058614+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.058614+0000', 'last_undegraded': '2024-09-19T04:14:17.058614+0000', 'last_fullsized': '2024-09-19T04:14:17.058614+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.20 is not active+clean 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064314+0000', 'last_change': '2024-09-19T04:14:17.064314+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064314+0000', 'last_undegraded': '2024-09-19T04:14:17.064314+0000', 'last_fullsized': '2024-09-19T04:14:17.064314+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:14:21.785 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1c is not active+clean 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.059239+0000', 'last_change': '2024-09-19T04:14:17.059239+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.059239+0000', 'last_undegraded': '2024-09-19T04:14:17.059239+0000', 'last_fullsized': '2024-09-19T04:14:17.059239+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1d is not active+clean 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065208+0000', 'last_change': '2024-09-19T04:14:17.065208+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065208+0000', 'last_undegraded': '2024-09-19T04:14:17.065208+0000', 'last_fullsized': '2024-09-19T04:14:17.065208+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1e is not active+clean 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.058473+0000', 'last_change': '2024-09-19T04:14:17.058473+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.058473+0000', 'last_undegraded': '2024-09-19T04:14:17.058473+0000', 'last_fullsized': '2024-09-19T04:14:17.058473+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1f is not active+clean 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.062853+0000', 'last_change': '2024-09-19T04:14:17.062853+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.062853+0000', 'last_undegraded': '2024-09-19T04:14:17.062853+0000', 'last_fullsized': '2024-09-19T04:14:17.062853+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.18 is not active+clean 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.074684+0000', 'last_change': '2024-09-19T04:14:17.074684+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.074684+0000', 'last_undegraded': '2024-09-19T04:14:17.074684+0000', 'last_fullsized': '2024-09-19T04:14:17.074684+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:PG 6.19 is not active+clean 2024-09-19T04:14:21.786 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065920+0000', 'last_change': '2024-09-19T04:14:17.065920+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065920+0000', 'last_undegraded': '2024-09-19T04:14:17.065920+0000', 'last_fullsized': '2024-09-19T04:14:17.065920+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1a is not active+clean 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.058025+0000', 'last_change': '2024-09-19T04:14:17.058025+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.058025+0000', 'last_undegraded': '2024-09-19T04:14:17.058025+0000', 'last_fullsized': '2024-09-19T04:14:17.058025+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1b is not active+clean 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065710+0000', 'last_change': '2024-09-19T04:14:17.065710+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065710+0000', 'last_undegraded': '2024-09-19T04:14:17.065710+0000', 'last_fullsized': '2024-09-19T04:14:17.065710+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.14 is not active+clean 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.062249+0000', 'last_change': '2024-09-19T04:14:17.062249+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.062249+0000', 'last_undegraded': '2024-09-19T04:14:17.062249+0000', 'last_fullsized': '2024-09-19T04:14:17.062249+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.15 is not active+clean 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.073417+0000', 'last_change': '2024-09-19T04:14:17.073417+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.073417+0000', 'last_undegraded': '2024-09-19T04:14:17.073417+0000', 'last_fullsized': '2024-09-19T04:14:17.073417+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:PG 6.16 is not active+clean 2024-09-19T04:14:21.787 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.074169+0000', 'last_change': '2024-09-19T04:14:17.074169+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.074169+0000', 'last_undegraded': '2024-09-19T04:14:17.074169+0000', 'last_fullsized': '2024-09-19T04:14:17.074169+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.17 is not active+clean 2024-09-19T04:14:21.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.062248+0000', 'last_change': '2024-09-19T04:14:17.062248+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.062248+0000', 'last_undegraded': '2024-09-19T04:14:17.062248+0000', 'last_fullsized': '2024-09-19T04:14:17.062248+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:14:21.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.10 is not active+clean 2024-09-19T04:14:21.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.069357+0000', 'last_change': '2024-09-19T04:14:17.069357+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.069357+0000', 'last_undegraded': '2024-09-19T04:14:17.069357+0000', 'last_fullsized': '2024-09-19T04:14:17.069357+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.788 INFO:tasks.ceph.ceph_manager.ceph:PG 6.11 is not active+clean 2024-09-19T04:14:21.788 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064480+0000', 'last_change': '2024-09-19T04:14:17.064480+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064480+0000', 'last_undegraded': '2024-09-19T04:14:17.064480+0000', 'last_fullsized': '2024-09-19T04:14:17.064480+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.789 INFO:tasks.ceph.ceph_manager.ceph:PG 6.12 is not active+clean 2024-09-19T04:14:21.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.058557+0000', 'last_change': '2024-09-19T04:14:17.058557+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.058557+0000', 'last_undegraded': '2024-09-19T04:14:17.058557+0000', 'last_fullsized': '2024-09-19T04:14:17.058557+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.789 INFO:tasks.ceph.ceph_manager.ceph:PG 6.13 is not active+clean 2024-09-19T04:14:21.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065260+0000', 'last_change': '2024-09-19T04:14:17.065260+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065260+0000', 'last_undegraded': '2024-09-19T04:14:17.065260+0000', 'last_fullsized': '2024-09-19T04:14:17.065260+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.789 INFO:tasks.ceph.ceph_manager.ceph:PG 6.c is not active+clean 2024-09-19T04:14:21.789 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.063556+0000', 'last_change': '2024-09-19T04:14:17.063556+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.063556+0000', 'last_undegraded': '2024-09-19T04:14:17.063556+0000', 'last_fullsized': '2024-09-19T04:14:17.063556+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.789 INFO:tasks.ceph.ceph_manager.ceph:PG 6.d is not active+clean 2024-09-19T04:14:21.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065209+0000', 'last_change': '2024-09-19T04:14:17.065209+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065209+0000', 'last_undegraded': '2024-09-19T04:14:17.065209+0000', 'last_fullsized': '2024-09-19T04:14:17.065209+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.790 INFO:tasks.ceph.ceph_manager.ceph:PG 6.e is not active+clean 2024-09-19T04:14:21.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.067888+0000', 'last_change': '2024-09-19T04:14:17.067888+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.067888+0000', 'last_undegraded': '2024-09-19T04:14:17.067888+0000', 'last_fullsized': '2024-09-19T04:14:17.067888+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:14:21.790 INFO:tasks.ceph.ceph_manager.ceph:PG 6.f is not active+clean 2024-09-19T04:14:21.790 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.061925+0000', 'last_change': '2024-09-19T04:14:17.061925+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.061925+0000', 'last_undegraded': '2024-09-19T04:14:17.061925+0000', 'last_fullsized': '2024-09-19T04:14:17.061925+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:14:21.790 INFO:tasks.ceph.ceph_manager.ceph:PG 6.8 is not active+clean 2024-09-19T04:14:21.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.059387+0000', 'last_change': '2024-09-19T04:14:17.059387+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.059387+0000', 'last_undegraded': '2024-09-19T04:14:17.059387+0000', 'last_fullsized': '2024-09-19T04:14:17.059387+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.791 INFO:tasks.ceph.ceph_manager.ceph:PG 6.9 is not active+clean 2024-09-19T04:14:21.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.067722+0000', 'last_change': '2024-09-19T04:14:17.067722+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.067722+0000', 'last_undegraded': '2024-09-19T04:14:17.067722+0000', 'last_fullsized': '2024-09-19T04:14:17.067722+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.791 INFO:tasks.ceph.ceph_manager.ceph:PG 6.a is not active+clean 2024-09-19T04:14:21.791 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.065171+0000', 'last_change': '2024-09-19T04:14:17.065171+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.065171+0000', 'last_undegraded': '2024-09-19T04:14:17.065171+0000', 'last_fullsized': '2024-09-19T04:14:17.065171+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.792 INFO:tasks.ceph.ceph_manager.ceph:PG 6.b is not active+clean 2024-09-19T04:14:21.792 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064362+0000', 'last_change': '2024-09-19T04:14:17.064362+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064362+0000', 'last_undegraded': '2024-09-19T04:14:17.064362+0000', 'last_fullsized': '2024-09-19T04:14:17.064362+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.792 INFO:tasks.ceph.ceph_manager.ceph:PG 6.4 is not active+clean 2024-09-19T04:14:21.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.064086+0000', 'last_change': '2024-09-19T04:14:17.064086+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.064086+0000', 'last_undegraded': '2024-09-19T04:14:17.064086+0000', 'last_fullsized': '2024-09-19T04:14:17.064086+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:14:21.793 INFO:tasks.ceph.ceph_manager.ceph:PG 6.5 is not active+clean 2024-09-19T04:14:21.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.061389+0000', 'last_change': '2024-09-19T04:14:17.061389+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.061389+0000', 'last_undegraded': '2024-09-19T04:14:17.061389+0000', 'last_fullsized': '2024-09-19T04:14:17.061389+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.793 INFO:tasks.ceph.ceph_manager.ceph:PG 6.6 is not active+clean 2024-09-19T04:14:21.793 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.062755+0000', 'last_change': '2024-09-19T04:14:17.062755+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.062755+0000', 'last_undegraded': '2024-09-19T04:14:17.062755+0000', 'last_fullsized': '2024-09-19T04:14:17.062755+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:14:21.794 INFO:tasks.ceph.ceph_manager.ceph:PG 6.0 is not active+clean 2024-09-19T04:14:21.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.068919+0000', 'last_change': '2024-09-19T04:14:17.068919+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.068919+0000', 'last_undegraded': '2024-09-19T04:14:17.068919+0000', 'last_fullsized': '2024-09-19T04:14:17.068919+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:14:21.794 INFO:tasks.ceph.ceph_manager.ceph:PG 6.1 is not active+clean 2024-09-19T04:14:21.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.063449+0000', 'last_change': '2024-09-19T04:14:17.063449+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.063449+0000', 'last_undegraded': '2024-09-19T04:14:17.063449+0000', 'last_fullsized': '2024-09-19T04:14:17.063449+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:14:21.794 INFO:tasks.ceph.ceph_manager.ceph:PG 6.2 is not active+clean 2024-09-19T04:14:21.794 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.059105+0000', 'last_change': '2024-09-19T04:14:17.059105+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.059105+0000', 'last_undegraded': '2024-09-19T04:14:17.059105+0000', 'last_fullsized': '2024-09-19T04:14:17.059105+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:14:21.795 INFO:tasks.ceph.ceph_manager.ceph:PG 6.3 is not active+clean 2024-09-19T04:14:21.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.059865+0000', 'last_change': '2024-09-19T04:14:17.059865+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.059865+0000', 'last_undegraded': '2024-09-19T04:14:17.059865+0000', 'last_fullsized': '2024-09-19T04:14:17.059865+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:14:21.795 INFO:tasks.ceph.ceph_manager.ceph:PG 6.7 is not active+clean 2024-09-19T04:14:21.795 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '6.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 37, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:14:17.068943+0000', 'last_change': '2024-09-19T04:14:17.068943+0000', 'last_active': '2024-09-19T04:14:17.040576+0000', 'last_peered': '2024-09-19T04:14:17.040576+0000', 'last_clean': '2024-09-19T04:14:17.040576+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:14:17.068943+0000', 'last_undegraded': '2024-09-19T04:14:17.068943+0000', 'last_fullsized': '2024-09-19T04:14:17.068943+0000', 'mapping_epoch': 37, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 37, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'last_clean_scrub_stamp': '2024-09-19T04:14:17.040576+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:14:21.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:14:21.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5850fbd640 1 Processor -- start 2024-09-19T04:14:21.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5850fbd640 1 -- start start 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5850fbd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 0x7f584c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5850fbd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f584c0af470 0x7f584c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5850fbd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f584c0b3290 con 0x7f584c15c2a0 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5850fbd640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f584c0b3570 con 0x7f584c0af470 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5850fbd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f584c0b3850 con 0x7f584c15bec0 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5849d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f584c15c2a0 0x7f584c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48366/0 (socket says 172.21.15.76:48366) 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f5849d9b640 1 -- 172.21.15.76:0/518911443 learned_addr learned my addr 172.21.15.76:0/518911443 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:21.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f584a59c640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 0x7f584c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f584ad9d640 1 --2- 172.21.15.76:0/518911443 >> v2:172.21.15.116:3300/0 conn(0x7f584c0af470 0x7f584c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.928+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 403039355 0 0) 0x7f584c0b3290 con 0x7f584c15c2a0 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5834003530 con 0x7f584c15c2a0 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2652484230 0 0) 0x7f5834003530 con 0x7f584c15c2a0 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 msgr2=0x7f584c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 0x7f584c0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 >> v2:172.21.15.116:3300/0 conn(0x7f584c0af470 msgr2=0x7f584c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 --2- 172.21.15.76:0/518911443 >> v2:172.21.15.116:3300/0 conn(0x7f584c0af470 0x7f584c06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c0b3b30 con 0x7f584c15c2a0 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5840001680 con 0x7f584c15c2a0 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584ad9d640 1 --2- 172.21.15.76:0/518911443 >> v2:172.21.15.116:3300/0 conn(0x7f584c0af470 0x7f584c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:21.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 >> v1:172.21.15.76:6789/0 conn(0x7f584c15c2a0 legacy=0x7f584c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58340039c0 0x7f5834003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5834004810 0x7f5834004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 0x7f58340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0b3570 con 0x7f5834004810 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0b3850 con 0x7f58340039c0 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f583400b9d0 con 0x7f584c15bec0 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5849d9b640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5834004810 0x7f5834004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584a59c640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58340039c0 0x7f5834003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584ad9d640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 0x7f58340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5849d9b640 1 -- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 msgr2=0x7f58340051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5849d9b640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c15bec0 0x7f58340051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5849d9b640 1 -- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58340039c0 msgr2=0x7f5834003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:21.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5849d9b640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58340039c0 0x7f5834003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5849d9b640 1 -- 172.21.15.76:0/518911443 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5834003530 con 0x7f5834004810 2024-09-19T04:14:21.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5849d9b640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5834004810 0x7f5834004c90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f584c06da50 tx=0x7f584001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456bae672b606c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840007d40 con 0x7f5834004810 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58400ac020 con 0x7f5834004810 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584959a640 1 -- 172.21.15.76:0/518911443 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58400058e0 con 0x7f5834004810 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5850fbd640 1 -- 172.21.15.76:0/518911443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5834004810 msgr2=0x7f5834004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5850fbd640 1 --2- 172.21.15.76:0/518911443 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5834004810 0x7f5834004c90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f584c06da50 tx=0x7f584001f3b0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f584ad9d640 1 -- 172.21.15.76:0/518911443 reap_dead start 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5850fbd640 1 -- 172.21.15.76:0/518911443 shutdown_connections 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5850fbd640 1 -- 172.21.15.76:0/518911443 >> 172.21.15.76:0/518911443 conn(0x7f584c0b1f90 msgr2=0x7f584c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5850fbd640 1 -- 172.21.15.76:0/518911443 shutdown_connections 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.932+0000 7f5850fbd640 1 -- 172.21.15.76:0/518911443 wait complete. 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 Processor -- start 2024-09-19T04:14:21.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 -- start start 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5834004810 0x7f584c14fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584c09a280 0x7f584c1e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c1e3d60 0x7f584c151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0afc90 con 0x7f584c09a280 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f584c0b40f0 con 0x7f5834004810 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f584c0ad200 con 0x7f584c1e3d60 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584c09a280 0x7f584c1e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584c09a280 0x7f584c1e3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46860/0 (socket says 172.21.15.76:46860) 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 -- 172.21.15.76:0/2477696156 learned_addr learned my addr 172.21.15.76:0/2477696156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f584ad9d640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c1e3d60 0x7f584c151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f584a59c640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5834004810 0x7f584c14fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 -- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c1e3d60 msgr2=0x7f584c151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c1e3d60 0x7f584c151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 -- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5834004810 msgr2=0x7f584c14fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5834004810 0x7f584c14fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 -- 172.21.15.76:0/2477696156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f584c09e160 con 0x7f584c09a280 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5849d9b640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584c09a280 0x7f584c1e3820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5840025f20 tx=0x7f5834005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f583affd640 1 -- 172.21.15.76:0/2477696156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58400049f0 con 0x7f584c09a280 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f583affd640 1 -- 172.21.15.76:0/2477696156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58400b6070 con 0x7f584c09a280 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f584c152830 con 0x7f584c09a280 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f583affd640 1 -- 172.21.15.76:0/2477696156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5840004dd0 con 0x7f584c09a280 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f584c152df0 con 0x7f584c09a280 2024-09-19T04:14:21.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f584c0ad610 con 0x7f584c09a280 2024-09-19T04:14:21.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f583affd640 1 -- 172.21.15.76:0/2477696156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5840007070 con 0x7f584c09a280 2024-09-19T04:14:21.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.936+0000 7f583affd640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f582c0678c0 0x7f582c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:21.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.940+0000 7f583affd640 1 -- 172.21.15.76:0/2477696156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5840122f80 con 0x7f584c09a280 2024-09-19T04:14:21.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.940+0000 7f584a59c640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f582c0678c0 0x7f582c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:21.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.940+0000 7f584a59c640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f582c0678c0 0x7f582c069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f584c152510 tx=0x7f583c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:21.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:21.944+0000 7f583affd640 1 -- 172.21.15.76:0/2477696156 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f58400190c0 con 0x7f584c09a280 2024-09-19T04:14:22.016 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:14:22.012+0000 7f3b899a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:14:22.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f584c06fcd0 con 0x7f584c09a280 2024-09-19T04:14:22.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f583affd640 1 -- 172.21.15.76:0/2477696156 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1623 (secure 0 0 0) 0x7f58400f3930 con 0x7f584c09a280 2024-09-19T04:14:22.158 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:22.158 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":39,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":234504192,"bytes_avail":772859609088,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":15,"btime":"2024-09-19T04:14:20:094237+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4863}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-19T04:14:13.752562+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f582c0678c0 msgr2=0x7f582c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f582c0678c0 0x7f582c069d80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f584c152510 tx=0x7f583c008040 comp rx=0 tx=0).stop 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584c09a280 msgr2=0x7f584c1e3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584c09a280 0x7f584c1e3820 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5840025f20 tx=0x7f5834005840 comp rx=0 tx=0).stop 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 shutdown_connections 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f582c0678c0 0x7f582c069d80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f584c1e3d60 0x7f584c151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f584c09a280 0x7f584c1e3820 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 --2- 172.21.15.76:0/2477696156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5834004810 0x7f584c14fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 >> 172.21.15.76:0/2477696156 conn(0x7f584c0b1f90 msgr2=0x7f584c15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 shutdown_connections 2024-09-19T04:14:22.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.156+0000 7f5850fbd640 1 -- 172.21.15.76:0/2477696156 wait complete. 2024-09-19T04:14:22.175 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T04:14:22.176 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:14:22.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f334a4c3640 1 Processor -- start 2024-09-19T04:14:22.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f334a4c3640 1 -- start start 2024-09-19T04:14:22.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f334a4c3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 0x7f334406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f334a4c3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334406ee50 con 0x7f3344073e60 2024-09-19T04:14:22.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f334a4c3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f334406f130 con 0x7f33440b3d50 2024-09-19T04:14:22.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f334a4c3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f334406f410 con 0x7f334406fef0 2024-09-19T04:14:22.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3344073e60 0x7f334415b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:48374/0 (socket says 172.21.15.76:48374) 2024-09-19T04:14:22.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 -- 172.21.15.76:0/531681026 learned_addr learned my addr 172.21.15.76:0/531681026 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f333b7fe640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 0x7f334406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767427625 0 0) 0x7f334406ee50 con 0x7f3344073e60 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3330003530 con 0x7f3344073e60 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1378436874 0 0) 0x7f3330003530 con 0x7f3344073e60 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 msgr2=0x7f334406e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 0x7f334406e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 >> v1:172.21.15.116:6789/0 conn(0x7f33440b3d50 legacy=0x7f3344072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334406f6f0 con 0x7f3344073e60 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f3334001680 con 0x7f3344073e60 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 >> v1:172.21.15.76:6789/0 conn(0x7f3344073e60 legacy=0x7f334415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33300039f0 0x7f3330003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 0x7f3330004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3330004d50 0x7f33300051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f334406f410 con 0x7f33300039f0 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f333000dd70 con 0x7f3330004d50 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f333000df70 con 0x7f334406fef0 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33300039f0 0x7f3330003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f333b7fe640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 0x7f3330004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:22.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 -- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 msgr2=0x7f3330004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:22.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f334406fef0 0x7f3330004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 -- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3330004d50 msgr2=0x7f33300051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:22.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3330004d50 0x7f33300051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.292+0000 7f3343fff640 1 -- 172.21.15.76:0/531681026 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3330003530 con 0x7f33300039f0 2024-09-19T04:14:22.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f3343fff640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33300039f0 0x7f3330003e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3334007ce0 tx=0x7f33340210b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7efdf5b8adbdc6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:22.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334007d20 con 0x7f33300039f0 2024-09-19T04:14:22.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33340ad070 con 0x7f33300039f0 2024-09-19T04:14:22.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f33437fe640 1 -- 172.21.15.76:0/531681026 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33340058e0 con 0x7f33300039f0 2024-09-19T04:14:22.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- 172.21.15.76:0/531681026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33300039f0 msgr2=0x7f3330003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 --2- 172.21.15.76:0/531681026 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33300039f0 0x7f3330003e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3334007ce0 tx=0x7f33340210b0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f3348a60640 1 -- 172.21.15.76:0/531681026 reap_dead start 2024-09-19T04:14:22.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- 172.21.15.76:0/531681026 shutdown_connections 2024-09-19T04:14:22.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- 172.21.15.76:0/531681026 >> 172.21.15.76:0/531681026 conn(0x7f33440b1f90 msgr2=0x7f33440b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:22.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- 172.21.15.76:0/531681026 shutdown_connections 2024-09-19T04:14:22.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- 172.21.15.76:0/531681026 wait complete. 2024-09-19T04:14:22.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 Processor -- start 2024-09-19T04:14:22.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- start start 2024-09-19T04:14:22.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33300039f0 0x7f334409dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3344152410 0x7f3344157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33441531e0 0x7f3344155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f334406e300 con 0x7f3344152410 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f334406e180 con 0x7f33300039f0 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f334a4c3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f334406e480 con 0x7f33441531e0 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.296+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3344152410 0x7f3344157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3344152410 0x7f3344157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46874/0 (socket says 172.21.15.76:46874) 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 -- 172.21.15.76:0/2432769662 learned_addr learned my addr 172.21.15.76:0/2432769662 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f3343fff640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33300039f0 0x7f334409dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f3348a60640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33441531e0 0x7f3344155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:22.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 -- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33441531e0 msgr2=0x7f3344155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33441531e0 0x7f3344155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 -- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33300039f0 msgr2=0x7f334409dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33300039f0 0x7f334409dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 -- 172.21.15.76:0/2432769662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f334409e160 con 0x7f3344152410 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f3343fff640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33300039f0 0x7f334409dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f333b7fe640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3344152410 0x7f3344157f80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f332c010710 tx=0x7f332c010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f33417fa640 1 -- 172.21.15.76:0/2432769662 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c019070 con 0x7f3344152410 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f33417fa640 1 -- 172.21.15.76:0/2432769662 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f332c00ec60 con 0x7f3344152410 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3344151370 con 0x7f3344152410 2024-09-19T04:14:22.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f33417fa640 1 -- 172.21.15.76:0/2432769662 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c00b040 con 0x7f3344152410 2024-09-19T04:14:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3344151930 con 0x7f3344152410 2024-09-19T04:14:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f334406e860 con 0x7f3344152410 2024-09-19T04:14:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f33417fa640 1 -- 172.21.15.76:0/2432769662 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f332c01c070 con 0x7f3344152410 2024-09-19T04:14:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f33417fa640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33240678f0 0x7f3324069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f33417fa640 1 -- 172.21.15.76:0/2432769662 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f332c085f30 con 0x7f3344152410 2024-09-19T04:14:22.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f3343fff640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33240678f0 0x7f3324069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.300+0000 7f3343fff640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33240678f0 0x7f3324069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3334001980 tx=0x7f33340ab070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:22.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.304+0000 7f33417fa640 1 -- 172.21.15.76:0/2432769662 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f332c056970 con 0x7f3344152410 2024-09-19T04:14:22.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.464+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33440d0fa0 con 0x7f33240678f0 2024-09-19T04:14:22.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.480+0000 7f33417fa640 1 -- 172.21.15.76:0/2432769662 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246639 (secure 0 0 0) 0x7f33440d0fa0 con 0x7f33240678f0 2024-09-19T04:14:22.485 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:22.487 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:14:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33240678f0 msgr2=0x7f3324069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33240678f0 0x7f3324069db0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3334001980 tx=0x7f33340ab070 comp rx=0 tx=0).stop 2024-09-19T04:14:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3344152410 msgr2=0x7f3344157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3344152410 0x7f3344157f80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f332c010710 tx=0x7f332c010bd0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 shutdown_connections 2024-09-19T04:14:22.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33240678f0 0x7f3324069db0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33441531e0 0x7f3344155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3344152410 0x7f3344157f80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 --2- 172.21.15.76:0/2432769662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33300039f0 0x7f334409dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 >> 172.21.15.76:0/2432769662 conn(0x7f33440b1f90 msgr2=0x7f334415a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 shutdown_connections 2024-09-19T04:14:22.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:22.484+0000 7f334a4c3640 1 -- 172.21.15.76:0/2432769662 wait complete. 2024-09-19T04:14:22.504 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-19T04:14:21.755879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229008,"kb_used_data":4456,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745712,"statfs":{"total":773094113280,"available":772859609088,"internally_reserved":0,"allocated":4562944,"data_stored":2617280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.713119"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065006+0000","last_change":"2024-09-19T04:14:17.065006+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065006+0000","last_undegraded":"2024-09-19T04:14:17.065006+0000","last_fullsized":"2024-09-19T04:14:17.065006+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063769+0000","last_change":"2024-09-19T04:14:17.063769+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063769+0000","last_undegraded":"2024-09-19T04:14:17.063769+0000","last_fullsized":"2024-09-19T04:14:17.063769+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072719+0000","last_change":"2024-09-19T04:14:17.072719+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072719+0000","last_undegraded":"2024-09-19T04:14:17.072719+0000","last_fullsized":"2024-09-19T04:14:17.072719+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066271+0000","last_change":"2024-09-19T04:14:17.066271+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066271+0000","last_undegraded":"2024-09-19T04:14:17.066271+0000","last_fullsized":"2024-09-19T04:14:17.066271+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065750+0000","last_change":"2024-09-19T04:14:17.065750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065750+0000","last_undegraded":"2024-09-19T04:14:17.065750+0000","last_fullsized":"2024-09-19T04:14:17.065750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066325+0000","last_change":"2024-09-19T04:14:17.066325+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066325+0000","last_undegraded":"2024-09-19T04:14:17.066325+0000","last_fullsized":"2024-09-19T04:14:17.066325+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068080+0000","last_change":"2024-09-19T04:14:17.068080+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068080+0000","last_undegraded":"2024-09-19T04:14:17.068080+0000","last_fullsized":"2024-09-19T04:14:17.068080+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064742+0000","last_change":"2024-09-19T04:14:17.064742+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064742+0000","last_undegraded":"2024-09-19T04:14:17.064742+0000","last_fullsized":"2024-09-19T04:14:17.064742+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073302+0000","last_change":"2024-09-19T04:14:17.073302+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073302+0000","last_undegraded":"2024-09-19T04:14:17.073302+0000","last_fullsized":"2024-09-19T04:14:17.073302+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070827+0000","last_change":"2024-09-19T04:14:17.070827+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070827+0000","last_undegraded":"2024-09-19T04:14:17.070827+0000","last_fullsized":"2024-09-19T04:14:17.070827+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.070514+0000","last_change":"2024-09-19T04:14:17.070514+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.070514+0000","last_undegraded":"2024-09-19T04:14:17.070514+0000","last_fullsized":"2024-09-19T04:14:17.070514+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073632+0000","last_change":"2024-09-19T04:14:17.073632+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073632+0000","last_undegraded":"2024-09-19T04:14:17.073632+0000","last_fullsized":"2024-09-19T04:14:17.073632+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065671+0000","last_change":"2024-09-19T04:14:17.065671+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065671+0000","last_undegraded":"2024-09-19T04:14:17.065671+0000","last_fullsized":"2024-09-19T04:14:17.065671+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062750+0000","last_change":"2024-09-19T04:14:17.062750+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062750+0000","last_undegraded":"2024-09-19T04:14:17.062750+0000","last_fullsized":"2024-09-19T04:14:17.062750+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066992+0000","last_change":"2024-09-19T04:14:17.066992+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066992+0000","last_undegraded":"2024-09-19T04:14:17.066992+0000","last_fullsized":"2024-09-19T04:14:17.066992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.060421+0000","last_change":"2024-09-19T04:14:17.060421+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.060421+0000","last_undegraded":"2024-09-19T04:14:17.060421+0000","last_fullsized":"2024-09-19T04:14:17.060421+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058740+0000","last_change":"2024-09-19T04:14:17.058740+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058740+0000","last_undegraded":"2024-09-19T04:14:17.058740+0000","last_fullsized":"2024-09-19T04:14:17.058740+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.066865+0000","last_change":"2024-09-19T04:14:17.066865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.066865+0000","last_undegraded":"2024-09-19T04:14:17.066865+0000","last_fullsized":"2024-09-19T04:14:17.066865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063176+0000","last_change":"2024-09-19T04:14:17.063176+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063176+0000","last_undegraded":"2024-09-19T04:14:17.063176+0000","last_fullsized":"2024-09-19T04:14:17.063176+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068706+0000","last_change":"2024-09-19T04:14:17.068706+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068706+0000","last_undegraded":"2024-09-19T04:14:17.068706+0000","last_fullsized":"2024-09-19T04:14:17.068706+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064596+0000","last_change":"2024-09-19T04:14:17.064596+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064596+0000","last_undegraded":"2024-09-19T04:14:17.064596+0000","last_fullsized":"2024-09-19T04:14:17.064596+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.056764+0000","last_change":"2024-09-19T04:14:17.056764+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.056764+0000","last_undegraded":"2024-09-19T04:14:17.056764+0000","last_fullsized":"2024-09-19T04:14:17.056764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059772+0000","last_change":"2024-09-19T04:14:17.059772+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059772+0000","last_undegraded":"2024-09-19T04:14:17.059772+0000","last_fullsized":"2024-09-19T04:14:17.059772+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062552+0000","last_change":"2024-09-19T04:14:17.062552+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062552+0000","last_undegraded":"2024-09-19T04:14:17.062552+0000","last_fullsized":"2024-09-19T04:14:17.062552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.072830+0000","last_change":"2024-09-19T04:14:17.072830+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.072830+0000","last_undegraded":"2024-09-19T04:14:17.072830+0000","last_fullsized":"2024-09-19T04:14:17.072830+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063731+0000","last_change":"2024-09-19T04:14:17.063731+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063731+0000","last_undegraded":"2024-09-19T04:14:17.063731+0000","last_fullsized":"2024-09-19T04:14:17.063731+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065739+0000","last_change":"2024-09-19T04:14:17.065739+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065739+0000","last_undegraded":"2024-09-19T04:14:17.065739+0000","last_fullsized":"2024-09-19T04:14:17.065739+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063806+0000","last_change":"2024-09-19T04:14:17.063806+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063806+0000","last_undegraded":"2024-09-19T04:14:17.063806+0000","last_fullsized":"2024-09-19T04:14:17.063806+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064604+0000","last_change":"2024-09-19T04:14:17.064604+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064604+0000","last_undegraded":"2024-09-19T04:14:17.064604+0000","last_fullsized":"2024-09-19T04:14:17.064604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064345+0000","last_change":"2024-09-19T04:14:17.064345+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064345+0000","last_undegraded":"2024-09-19T04:14:17.064345+0000","last_fullsized":"2024-09-19T04:14:17.064345+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058614+0000","last_change":"2024-09-19T04:14:17.058614+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058614+0000","last_undegraded":"2024-09-19T04:14:17.058614+0000","last_fullsized":"2024-09-19T04:14:17.058614+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064314+0000","last_change":"2024-09-19T04:14:17.064314+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064314+0000","last_undegraded":"2024-09-19T04:14:17.064314+0000","last_fullsized":"2024-09-19T04:14:17.064314+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059239+0000","last_change":"2024-09-19T04:14:17.059239+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059239+0000","last_undegraded":"2024-09-19T04:14:17.059239+0000","last_fullsized":"2024-09-19T04:14:17.059239+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091472+0000","last_change":"2024-09-19T04:14:17.088483+0000","last_active":"2024-09-19T04:14:17.091472+0000","last_peered":"2024-09-19T04:14:17.091472+0000","last_clean":"2024-09-19T04:14:17.091472+0000","last_became_active":"2024-09-19T04:14:17.087794+0000","last_became_peered":"2024-09-19T04:14:17.087794+0000","last_unstale":"2024-09-19T04:14:17.091472+0000","last_undegraded":"2024-09-19T04:14:17.091472+0000","last_fullsized":"2024-09-19T04:14:17.091472+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065208+0000","last_change":"2024-09-19T04:14:17.065208+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065208+0000","last_undegraded":"2024-09-19T04:14:17.065208+0000","last_fullsized":"2024-09-19T04:14:17.065208+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.093823+0000","last_change":"2024-09-19T04:14:17.093169+0000","last_active":"2024-09-19T04:14:17.093823+0000","last_peered":"2024-09-19T04:14:17.093823+0000","last_clean":"2024-09-19T04:14:17.093823+0000","last_became_active":"2024-09-19T04:14:17.092881+0000","last_became_peered":"2024-09-19T04:14:17.092881+0000","last_unstale":"2024-09-19T04:14:17.093823+0000","last_undegraded":"2024-09-19T04:14:17.093823+0000","last_fullsized":"2024-09-19T04:14:17.093823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058473+0000","last_change":"2024-09-19T04:14:17.058473+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058473+0000","last_undegraded":"2024-09-19T04:14:17.058473+0000","last_fullsized":"2024-09-19T04:14:17.058473+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078344+0000","last_change":"2024-09-19T04:14:17.076870+0000","last_active":"2024-09-19T04:14:17.078344+0000","last_peered":"2024-09-19T04:14:17.078344+0000","last_clean":"2024-09-19T04:14:17.078344+0000","last_became_active":"2024-09-19T04:14:17.076604+0000","last_became_peered":"2024-09-19T04:14:17.076604+0000","last_unstale":"2024-09-19T04:14:17.078344+0000","last_undegraded":"2024-09-19T04:14:17.078344+0000","last_fullsized":"2024-09-19T04:14:17.078344+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062853+0000","last_change":"2024-09-19T04:14:17.062853+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062853+0000","last_undegraded":"2024-09-19T04:14:17.062853+0000","last_fullsized":"2024-09-19T04:14:17.062853+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.079157+0000","last_change":"2024-09-19T04:14:17.072407+0000","last_active":"2024-09-19T04:14:17.079157+0000","last_peered":"2024-09-19T04:14:17.079157+0000","last_clean":"2024-09-19T04:14:17.079157+0000","last_became_active":"2024-09-19T04:14:17.071689+0000","last_became_peered":"2024-09-19T04:14:17.071689+0000","last_unstale":"2024-09-19T04:14:17.079157+0000","last_undegraded":"2024-09-19T04:14:17.079157+0000","last_fullsized":"2024-09-19T04:14:17.079157+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074684+0000","last_change":"2024-09-19T04:14:17.074684+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074684+0000","last_undegraded":"2024-09-19T04:14:17.074684+0000","last_fullsized":"2024-09-19T04:14:17.074684+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098241+0000","last_change":"2024-09-19T04:14:17.083299+0000","last_active":"2024-09-19T04:14:17.098241+0000","last_peered":"2024-09-19T04:14:17.098241+0000","last_clean":"2024-09-19T04:14:17.098241+0000","last_became_active":"2024-09-19T04:14:17.081493+0000","last_became_peered":"2024-09-19T04:14:17.081493+0000","last_unstale":"2024-09-19T04:14:17.098241+0000","last_undegraded":"2024-09-19T04:14:17.098241+0000","last_fullsized":"2024-09-19T04:14:17.098241+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065920+0000","last_change":"2024-09-19T04:14:17.065920+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065920+0000","last_undegraded":"2024-09-19T04:14:17.065920+0000","last_fullsized":"2024-09-19T04:14:17.065920+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100822+0000","last_change":"2024-09-19T04:14:17.098330+0000","last_active":"2024-09-19T04:14:17.100822+0000","last_peered":"2024-09-19T04:14:17.100822+0000","last_clean":"2024-09-19T04:14:17.100822+0000","last_became_active":"2024-09-19T04:14:17.097696+0000","last_became_peered":"2024-09-19T04:14:17.097696+0000","last_unstale":"2024-09-19T04:14:17.100822+0000","last_undegraded":"2024-09-19T04:14:17.100822+0000","last_fullsized":"2024-09-19T04:14:17.100822+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:46.739371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058025+0000","last_change":"2024-09-19T04:14:17.058025+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058025+0000","last_undegraded":"2024-09-19T04:14:17.058025+0000","last_fullsized":"2024-09-19T04:14:17.058025+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098750+0000","last_change":"2024-09-19T04:14:17.097839+0000","last_active":"2024-09-19T04:14:17.098750+0000","last_peered":"2024-09-19T04:14:17.098750+0000","last_clean":"2024-09-19T04:14:17.098750+0000","last_became_active":"2024-09-19T04:14:17.097522+0000","last_became_peered":"2024-09-19T04:14:17.097522+0000","last_unstale":"2024-09-19T04:14:17.098750+0000","last_undegraded":"2024-09-19T04:14:17.098750+0000","last_fullsized":"2024-09-19T04:14:17.098750+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065710+0000","last_change":"2024-09-19T04:14:17.065710+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065710+0000","last_undegraded":"2024-09-19T04:14:17.065710+0000","last_fullsized":"2024-09-19T04:14:17.065710+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094043+0000","last_change":"2024-09-19T04:14:17.088506+0000","last_active":"2024-09-19T04:14:17.094043+0000","last_peered":"2024-09-19T04:14:17.094043+0000","last_clean":"2024-09-19T04:14:17.094043+0000","last_became_active":"2024-09-19T04:14:17.087746+0000","last_became_peered":"2024-09-19T04:14:17.087746+0000","last_unstale":"2024-09-19T04:14:17.094043+0000","last_undegraded":"2024-09-19T04:14:17.094043+0000","last_fullsized":"2024-09-19T04:14:17.094043+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:07:22.122410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062249+0000","last_change":"2024-09-19T04:14:17.062249+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062249+0000","last_undegraded":"2024-09-19T04:14:17.062249+0000","last_fullsized":"2024-09-19T04:14:17.062249+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075544+0000","last_change":"2024-09-19T04:14:17.070017+0000","last_active":"2024-09-19T04:14:17.075544+0000","last_peered":"2024-09-19T04:14:17.075544+0000","last_clean":"2024-09-19T04:14:17.075544+0000","last_became_active":"2024-09-19T04:14:17.069539+0000","last_became_peered":"2024-09-19T04:14:17.069539+0000","last_unstale":"2024-09-19T04:14:17.075544+0000","last_undegraded":"2024-09-19T04:14:17.075544+0000","last_fullsized":"2024-09-19T04:14:17.075544+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:31:50.043891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.073417+0000","last_change":"2024-09-19T04:14:17.073417+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.073417+0000","last_undegraded":"2024-09-19T04:14:17.073417+0000","last_fullsized":"2024-09-19T04:14:17.073417+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101674+0000","last_change":"2024-09-19T04:14:17.094996+0000","last_active":"2024-09-19T04:14:17.101674+0000","last_peered":"2024-09-19T04:14:17.101674+0000","last_clean":"2024-09-19T04:14:17.101674+0000","last_became_active":"2024-09-19T04:14:17.083414+0000","last_became_peered":"2024-09-19T04:14:17.083414+0000","last_unstale":"2024-09-19T04:14:17.101674+0000","last_undegraded":"2024-09-19T04:14:17.101674+0000","last_fullsized":"2024-09-19T04:14:17.101674+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:40.402841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.074169+0000","last_change":"2024-09-19T04:14:17.074169+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.074169+0000","last_undegraded":"2024-09-19T04:14:17.074169+0000","last_fullsized":"2024-09-19T04:14:17.074169+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098844+0000","last_change":"2024-09-19T04:14:17.094246+0000","last_active":"2024-09-19T04:14:17.098844+0000","last_peered":"2024-09-19T04:14:17.098844+0000","last_clean":"2024-09-19T04:14:17.098844+0000","last_became_active":"2024-09-19T04:14:17.083763+0000","last_became_peered":"2024-09-19T04:14:17.083763+0000","last_unstale":"2024-09-19T04:14:17.098844+0000","last_undegraded":"2024-09-19T04:14:17.098844+0000","last_fullsized":"2024-09-19T04:14:17.098844+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:39.583489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062248+0000","last_change":"2024-09-19T04:14:17.062248+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062248+0000","last_undegraded":"2024-09-19T04:14:17.062248+0000","last_fullsized":"2024-09-19T04:14:17.062248+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.095183+0000","last_change":"2024-09-19T04:14:17.094374+0000","last_active":"2024-09-19T04:14:17.095183+0000","last_peered":"2024-09-19T04:14:17.095183+0000","last_clean":"2024-09-19T04:14:17.095183+0000","last_became_active":"2024-09-19T04:14:17.093949+0000","last_became_peered":"2024-09-19T04:14:17.093949+0000","last_unstale":"2024-09-19T04:14:17.095183+0000","last_undegraded":"2024-09-19T04:14:17.095183+0000","last_fullsized":"2024-09-19T04:14:17.095183+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:27.153452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.069357+0000","last_change":"2024-09-19T04:14:17.069357+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.069357+0000","last_undegraded":"2024-09-19T04:14:17.069357+0000","last_fullsized":"2024-09-19T04:14:17.069357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078270+0000","last_change":"2024-09-19T04:14:17.072319+0000","last_active":"2024-09-19T04:14:17.078270+0000","last_peered":"2024-09-19T04:14:17.078270+0000","last_clean":"2024-09-19T04:14:17.078270+0000","last_became_active":"2024-09-19T04:14:17.071960+0000","last_became_peered":"2024-09-19T04:14:17.071960+0000","last_unstale":"2024-09-19T04:14:17.078270+0000","last_undegraded":"2024-09-19T04:14:17.078270+0000","last_fullsized":"2024-09-19T04:14:17.078270+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:56.776172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064480+0000","last_change":"2024-09-19T04:14:17.064480+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064480+0000","last_undegraded":"2024-09-19T04:14:17.064480+0000","last_fullsized":"2024-09-19T04:14:17.064480+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.084958+0000","last_change":"2024-09-19T04:14:17.084245+0000","last_active":"2024-09-19T04:14:17.084958+0000","last_peered":"2024-09-19T04:14:17.084958+0000","last_clean":"2024-09-19T04:14:17.084958+0000","last_became_active":"2024-09-19T04:14:17.083828+0000","last_became_peered":"2024-09-19T04:14:17.083828+0000","last_unstale":"2024-09-19T04:14:17.084958+0000","last_undegraded":"2024-09-19T04:14:17.084958+0000","last_fullsized":"2024-09-19T04:14:17.084958+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.058557+0000","last_change":"2024-09-19T04:14:17.058557+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.058557+0000","last_undegraded":"2024-09-19T04:14:17.058557+0000","last_fullsized":"2024-09-19T04:14:17.058557+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.091279+0000","last_change":"2024-09-19T04:14:17.085767+0000","last_active":"2024-09-19T04:14:17.091279+0000","last_peered":"2024-09-19T04:14:17.091279+0000","last_clean":"2024-09-19T04:14:17.091279+0000","last_became_active":"2024-09-19T04:14:17.085016+0000","last_became_peered":"2024-09-19T04:14:17.085016+0000","last_unstale":"2024-09-19T04:14:17.091279+0000","last_undegraded":"2024-09-19T04:14:17.091279+0000","last_fullsized":"2024-09-19T04:14:17.091279+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065260+0000","last_change":"2024-09-19T04:14:17.065260+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065260+0000","last_undegraded":"2024-09-19T04:14:17.065260+0000","last_fullsized":"2024-09-19T04:14:17.065260+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.100905+0000","last_change":"2024-09-19T04:14:17.098594+0000","last_active":"2024-09-19T04:14:17.100905+0000","last_peered":"2024-09-19T04:14:17.100905+0000","last_clean":"2024-09-19T04:14:17.100905+0000","last_became_active":"2024-09-19T04:14:17.097858+0000","last_became_peered":"2024-09-19T04:14:17.097858+0000","last_unstale":"2024-09-19T04:14:17.100905+0000","last_undegraded":"2024-09-19T04:14:17.100905+0000","last_fullsized":"2024-09-19T04:14:17.100905+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:26.792894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063556+0000","last_change":"2024-09-19T04:14:17.063556+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063556+0000","last_undegraded":"2024-09-19T04:14:17.063556+0000","last_fullsized":"2024-09-19T04:14:17.063556+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101712+0000","last_change":"2024-09-19T04:14:17.095361+0000","last_active":"2024-09-19T04:14:17.101712+0000","last_peered":"2024-09-19T04:14:17.101712+0000","last_clean":"2024-09-19T04:14:17.101712+0000","last_became_active":"2024-09-19T04:14:17.094445+0000","last_became_peered":"2024-09-19T04:14:17.094445+0000","last_unstale":"2024-09-19T04:14:17.101712+0000","last_undegraded":"2024-09-19T04:14:17.101712+0000","last_fullsized":"2024-09-19T04:14:17.101712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:59.306800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065209+0000","last_change":"2024-09-19T04:14:17.065209+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065209+0000","last_undegraded":"2024-09-19T04:14:17.065209+0000","last_fullsized":"2024-09-19T04:14:17.065209+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.101020+0000","last_change":"2024-09-19T04:14:17.098487+0000","last_active":"2024-09-19T04:14:17.101020+0000","last_peered":"2024-09-19T04:14:17.101020+0000","last_clean":"2024-09-19T04:14:17.101020+0000","last_became_active":"2024-09-19T04:14:17.097805+0000","last_became_peered":"2024-09-19T04:14:17.097805+0000","last_unstale":"2024-09-19T04:14:17.101020+0000","last_undegraded":"2024-09-19T04:14:17.101020+0000","last_fullsized":"2024-09-19T04:14:17.101020+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:55.367559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067888+0000","last_change":"2024-09-19T04:14:17.067888+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067888+0000","last_undegraded":"2024-09-19T04:14:17.067888+0000","last_fullsized":"2024-09-19T04:14:17.067888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.094537+0000","last_change":"2024-09-19T04:14:17.093924+0000","last_active":"2024-09-19T04:14:17.094537+0000","last_peered":"2024-09-19T04:14:17.094537+0000","last_clean":"2024-09-19T04:14:17.094537+0000","last_became_active":"2024-09-19T04:14:17.093578+0000","last_became_peered":"2024-09-19T04:14:17.093578+0000","last_unstale":"2024-09-19T04:14:17.094537+0000","last_undegraded":"2024-09-19T04:14:17.094537+0000","last_fullsized":"2024-09-19T04:14:17.094537+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:43:26.607609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061925+0000","last_change":"2024-09-19T04:14:17.061925+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061925+0000","last_undegraded":"2024-09-19T04:14:17.061925+0000","last_fullsized":"2024-09-19T04:14:17.061925+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078381+0000","last_change":"2024-09-19T04:14:17.076993+0000","last_active":"2024-09-19T04:14:17.078381+0000","last_peered":"2024-09-19T04:14:17.078381+0000","last_clean":"2024-09-19T04:14:17.078381+0000","last_became_active":"2024-09-19T04:14:17.076271+0000","last_became_peered":"2024-09-19T04:14:17.076271+0000","last_unstale":"2024-09-19T04:14:17.078381+0000","last_undegraded":"2024-09-19T04:14:17.078381+0000","last_fullsized":"2024-09-19T04:14:17.078381+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059387+0000","last_change":"2024-09-19T04:14:17.059387+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059387+0000","last_undegraded":"2024-09-19T04:14:17.059387+0000","last_fullsized":"2024-09-19T04:14:17.059387+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075746+0000","last_change":"2024-09-19T04:14:17.071279+0000","last_active":"2024-09-19T04:14:17.075746+0000","last_peered":"2024-09-19T04:14:17.075746+0000","last_clean":"2024-09-19T04:14:17.075746+0000","last_became_active":"2024-09-19T04:14:17.070910+0000","last_became_peered":"2024-09-19T04:14:17.070910+0000","last_unstale":"2024-09-19T04:14:17.075746+0000","last_undegraded":"2024-09-19T04:14:17.075746+0000","last_fullsized":"2024-09-19T04:14:17.075746+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:35.515291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.067722+0000","last_change":"2024-09-19T04:14:17.067722+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.067722+0000","last_undegraded":"2024-09-19T04:14:17.067722+0000","last_fullsized":"2024-09-19T04:14:17.067722+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.081039+0000","last_change":"2024-09-19T04:14:17.080497+0000","last_active":"2024-09-19T04:14:17.081039+0000","last_peered":"2024-09-19T04:14:17.081039+0000","last_clean":"2024-09-19T04:14:17.081039+0000","last_became_active":"2024-09-19T04:14:17.080085+0000","last_became_peered":"2024-09-19T04:14:17.080085+0000","last_unstale":"2024-09-19T04:14:17.081039+0000","last_undegraded":"2024-09-19T04:14:17.081039+0000","last_fullsized":"2024-09-19T04:14:17.081039+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:08.636704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.065171+0000","last_change":"2024-09-19T04:14:17.065171+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.065171+0000","last_undegraded":"2024-09-19T04:14:17.065171+0000","last_fullsized":"2024-09-19T04:14:17.065171+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099918+0000","last_change":"2024-09-19T04:14:17.095657+0000","last_active":"2024-09-19T04:14:17.099918+0000","last_peered":"2024-09-19T04:14:17.099918+0000","last_clean":"2024-09-19T04:14:17.099918+0000","last_became_active":"2024-09-19T04:14:17.094938+0000","last_became_peered":"2024-09-19T04:14:17.094938+0000","last_unstale":"2024-09-19T04:14:17.099918+0000","last_undegraded":"2024-09-19T04:14:17.099918+0000","last_fullsized":"2024-09-19T04:14:17.099918+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:27.454267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064362+0000","last_change":"2024-09-19T04:14:17.064362+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064362+0000","last_undegraded":"2024-09-19T04:14:17.064362+0000","last_fullsized":"2024-09-19T04:14:17.064362+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.070502+0000","last_change":"2024-09-19T04:14:17.067723+0000","last_active":"2024-09-19T04:14:17.070502+0000","last_peered":"2024-09-19T04:14:17.070502+0000","last_clean":"2024-09-19T04:14:17.070502+0000","last_became_active":"2024-09-19T04:14:17.067393+0000","last_became_peered":"2024-09-19T04:14:17.067393+0000","last_unstale":"2024-09-19T04:14:17.070502+0000","last_undegraded":"2024-09-19T04:14:17.070502+0000","last_fullsized":"2024-09-19T04:14:17.070502+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:09:06.884322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.064086+0000","last_change":"2024-09-19T04:14:17.064086+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.064086+0000","last_undegraded":"2024-09-19T04:14:17.064086+0000","last_fullsized":"2024-09-19T04:14:17.064086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099437+0000","last_change":"2024-09-19T04:14:17.094649+0000","last_active":"2024-09-19T04:14:17.099437+0000","last_peered":"2024-09-19T04:14:17.099437+0000","last_clean":"2024-09-19T04:14:17.099437+0000","last_became_active":"2024-09-19T04:14:17.083575+0000","last_became_peered":"2024-09-19T04:14:17.083575+0000","last_unstale":"2024-09-19T04:14:17.099437+0000","last_undegraded":"2024-09-19T04:14:17.099437+0000","last_fullsized":"2024-09-19T04:14:17.099437+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:13.045881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.061389+0000","last_change":"2024-09-19T04:14:17.061389+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.061389+0000","last_undegraded":"2024-09-19T04:14:17.061389+0000","last_fullsized":"2024-09-19T04:14:17.061389+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.098793+0000","last_change":"2024-09-19T04:14:17.097837+0000","last_active":"2024-09-19T04:14:17.098793+0000","last_peered":"2024-09-19T04:14:17.098793+0000","last_clean":"2024-09-19T04:14:17.098793+0000","last_became_active":"2024-09-19T04:14:17.097561+0000","last_became_peered":"2024-09-19T04:14:17.097561+0000","last_unstale":"2024-09-19T04:14:17.098793+0000","last_undegraded":"2024-09-19T04:14:17.098793+0000","last_fullsized":"2024-09-19T04:14:17.098793+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:33:22.796871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.062755+0000","last_change":"2024-09-19T04:14:17.062755+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.062755+0000","last_undegraded":"2024-09-19T04:14:17.062755+0000","last_fullsized":"2024-09-19T04:14:17.062755+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.083154+0000","last_change":"2024-09-19T04:14:17.074004+0000","last_active":"2024-09-19T04:14:17.083154+0000","last_peered":"2024-09-19T04:14:17.083154+0000","last_clean":"2024-09-19T04:14:17.083154+0000","last_became_active":"2024-09-19T04:14:17.073653+0000","last_became_peered":"2024-09-19T04:14:17.073653+0000","last_unstale":"2024-09-19T04:14:17.083154+0000","last_undegraded":"2024-09-19T04:14:17.083154+0000","last_fullsized":"2024-09-19T04:14:17.083154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068919+0000","last_change":"2024-09-19T04:14:17.068919+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068919+0000","last_undegraded":"2024-09-19T04:14:17.068919+0000","last_fullsized":"2024-09-19T04:14:17.068919+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088034+0000","last_change":"2024-09-19T04:14:17.087371+0000","last_active":"2024-09-19T04:14:17.088034+0000","last_peered":"2024-09-19T04:14:17.088034+0000","last_clean":"2024-09-19T04:14:17.088034+0000","last_became_active":"2024-09-19T04:14:17.087047+0000","last_became_peered":"2024-09-19T04:14:17.087047+0000","last_unstale":"2024-09-19T04:14:17.088034+0000","last_undegraded":"2024-09-19T04:14:17.088034+0000","last_fullsized":"2024-09-19T04:14:17.088034+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.063449+0000","last_change":"2024-09-19T04:14:17.063449+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.063449+0000","last_undegraded":"2024-09-19T04:14:17.063449+0000","last_fullsized":"2024-09-19T04:14:17.063449+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.088850+0000","last_change":"2024-09-19T04:14:17.086127+0000","last_active":"2024-09-19T04:14:17.088850+0000","last_peered":"2024-09-19T04:14:17.088850+0000","last_clean":"2024-09-19T04:14:17.088850+0000","last_became_active":"2024-09-19T04:14:17.084606+0000","last_became_peered":"2024-09-19T04:14:17.084606+0000","last_unstale":"2024-09-19T04:14:17.088850+0000","last_undegraded":"2024-09-19T04:14:17.088850+0000","last_fullsized":"2024-09-19T04:14:17.088850+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:45:00.860293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059105+0000","last_change":"2024-09-19T04:14:17.059105+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059105+0000","last_undegraded":"2024-09-19T04:14:17.059105+0000","last_fullsized":"2024-09-19T04:14:17.059105+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.078734+0000","last_change":"2024-09-19T04:14:17.070371+0000","last_active":"2024-09-19T04:14:17.078734+0000","last_peered":"2024-09-19T04:14:17.078734+0000","last_clean":"2024-09-19T04:14:17.078734+0000","last_became_active":"2024-09-19T04:14:17.069663+0000","last_became_peered":"2024-09-19T04:14:17.069663+0000","last_unstale":"2024-09-19T04:14:17.078734+0000","last_undegraded":"2024-09-19T04:14:17.078734+0000","last_fullsized":"2024-09-19T04:14:17.078734+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:32.566555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.059865+0000","last_change":"2024-09-19T04:14:17.059865+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.059865+0000","last_undegraded":"2024-09-19T04:14:17.059865+0000","last_fullsized":"2024-09-19T04:14:17.059865+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.075551+0000","last_change":"2024-09-19T04:14:17.070348+0000","last_active":"2024-09-19T04:14:17.075551+0000","last_peered":"2024-09-19T04:14:17.075551+0000","last_clean":"2024-09-19T04:14:17.075551+0000","last_became_active":"2024-09-19T04:14:17.069964+0000","last_became_peered":"2024-09-19T04:14:17.069964+0000","last_unstale":"2024-09-19T04:14:17.075551+0000","last_undegraded":"2024-09-19T04:14:17.075551+0000","last_fullsized":"2024-09-19T04:14:17.075551+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:09.298918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":37,"state":"creating+peering","last_fresh":"2024-09-19T04:14:17.068943+0000","last_change":"2024-09-19T04:14:17.068943+0000","last_active":"2024-09-19T04:14:17.040576+0000","last_peered":"2024-09-19T04:14:17.040576+0000","last_clean":"2024-09-19T04:14:17.040576+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:14:17.068943+0000","last_undegraded":"2024-09-19T04:14:17.068943+0000","last_fullsized":"2024-09-19T04:14:17.068943+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.099762+0000","last_change":"2024-09-19T04:14:17.093374+0000","last_active":"2024-09-19T04:14:17.099762+0000","last_peered":"2024-09-19T04:14:17.099762+0000","last_clean":"2024-09-19T04:14:17.099762+0000","last_became_active":"2024-09-19T04:14:17.092649+0000","last_became_peered":"2024-09-19T04:14:17.092649+0000","last_unstale":"2024-09-19T04:14:17.099762+0000","last_undegraded":"2024-09-19T04:14:17.099762+0000","last_fullsized":"2024-09-19T04:14:17.099762+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:58:34.491970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-19T04:14:17.087005+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:14:17.087005+0000","last_peered":"2024-09-19T04:14:17.087005+0000","last_clean":"2024-09-19T04:14:17.087005+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:14:17.087005+0000","last_undegraded":"2024-09-19T04:14:17.087005+0000","last_fullsized":"2024-09-19T04:14:17.087005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339656,"statfs":{"total":96636764160,"available":96603807744,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542161,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542160,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542160,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542161,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31604,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340236,"statfs":{"total":96636764160,"available":96604401664,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542161,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":179568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542161,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":769936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:14:23.115 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:14:23.118 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:14:25.509 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:14:25.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 Processor -- start 2024-09-19T04:14:25.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 -- start start 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015bec0 0x7feec00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feec015c2a0 0x7feec00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feec00af2a0 0x7feec015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7feec00b3290 con 0x7feec015c2a0 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7feec00b3570 con 0x7feec00af2a0 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec7c42640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7feec00b3850 con 0x7feec015bec0 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feec015c2a0 0x7feec00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feec015c2a0 0x7feec00aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59080/0 (socket says 172.21.15.76:59080) 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 -- 172.21.15.76:0/431686912 learned_addr learned my addr 172.21.15.76:0/431686912 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec61df640 1 --2- 172.21.15.76:0/431686912 >> v2:172.21.15.116:3300/0 conn(0x7feec00af2a0 0x7feec015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:25.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 -- 172.21.15.76:0/431686912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015bec0 msgr2=0x7feec00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 --2- 172.21.15.76:0/431686912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015bec0 0x7feec00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 -- 172.21.15.76:0/431686912 >> v2:172.21.15.116:3300/0 conn(0x7feec00af2a0 msgr2=0x7feec015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 --2- 172.21.15.76:0/431686912 >> v2:172.21.15.116:3300/0 conn(0x7feec00af2a0 0x7feec015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 -- 172.21.15.76:0/431686912 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec00b3b30 con 0x7feec015c2a0 2024-09-19T04:14:25.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec61df640 1 --2- 172.21.15.76:0/431686912 >> v2:172.21.15.116:3300/0 conn(0x7feec00af2a0 0x7feec015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec51dd640 1 --2- 172.21.15.76:0/431686912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec015c2a0 0x7feec00aed60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feec00ada40 tx=0x7feeb001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc01d4c8886d8d9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec49dc640 1 -- 172.21.15.76:0/431686912 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00b0070 con 0x7feec015c2a0 2024-09-19T04:14:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec49dc640 1 -- 172.21.15.76:0/431686912 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeb00ab600 con 0x7feec015c2a0 2024-09-19T04:14:25.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.640+0000 7feec49dc640 1 -- 172.21.15.76:0/431686912 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb00ab9d0 con 0x7feec015c2a0 2024-09-19T04:14:25.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- 172.21.15.76:0/431686912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec015c2a0 msgr2=0x7feec00aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:25.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 --2- 172.21.15.76:0/431686912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec015c2a0 0x7feec00aed60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7feec00ada40 tx=0x7feeb001c230 comp rx=0 tx=0).stop 2024-09-19T04:14:25.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- 172.21.15.76:0/431686912 shutdown_connections 2024-09-19T04:14:25.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 --2- 172.21.15.76:0/431686912 >> v2:172.21.15.116:3300/0 conn(0x7feec00af2a0 0x7feec015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 --2- 172.21.15.76:0/431686912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec015c2a0 0x7feec00aed60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 --2- 172.21.15.76:0/431686912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015bec0 0x7feec00ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- 172.21.15.76:0/431686912 >> 172.21.15.76:0/431686912 conn(0x7feec00b1f90 msgr2=0x7feec00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- 172.21.15.76:0/431686912 shutdown_connections 2024-09-19T04:14:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- 172.21.15.76:0/431686912 wait complete. 2024-09-19T04:14:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 Processor -- start 2024-09-19T04:14:25.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- start start 2024-09-19T04:14:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec00af2a0 0x7feec0151b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec015bec0 0x7feec01531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015c2a0 0x7feec0099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:25.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feec00af6b0 con 0x7feec00af2a0 2024-09-19T04:14:25.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feec00ad740 con 0x7feec015bec0 2024-09-19T04:14:25.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7feec006e000 con 0x7feec015c2a0 2024-09-19T04:14:25.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec00af2a0 0x7feec0151b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:25.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec00af2a0 0x7feec0151b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59086/0 (socket says 172.21.15.76:59086) 2024-09-19T04:14:25.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 -- 172.21.15.76:0/1397291258 learned_addr learned my addr 172.21.15.76:0/1397291258 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec61df640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015c2a0 0x7feec0099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec51dd640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec015bec0 0x7feec01531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 -- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015c2a0 msgr2=0x7feec0099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015c2a0 0x7feec0099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 -- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec015bec0 msgr2=0x7feec01531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:25.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec015bec0 0x7feec01531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 -- 172.21.15.76:0/1397291258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec0157210 con 0x7feec00af2a0 2024-09-19T04:14:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec61df640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015c2a0 0x7feec0099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:25.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec59de640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec00af2a0 0x7feec0151b10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feebc00ba50 tx=0x7feebc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feeb67fc640 1 -- 172.21.15.76:0/1397291258 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc019070 con 0x7feec00af2a0 2024-09-19T04:14:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feeb67fc640 1 -- 172.21.15.76:0/1397291258 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feebc00cc00 con 0x7feec00af2a0 2024-09-19T04:14:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec01574f0 con 0x7feec00af2a0 2024-09-19T04:14:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feeb67fc640 1 -- 172.21.15.76:0/1397291258 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feebc014440 con 0x7feec00af2a0 2024-09-19T04:14:25.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.644+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec0157fd0 con 0x7feec00af2a0 2024-09-19T04:14:25.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.648+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec00ad200 con 0x7feec00af2a0 2024-09-19T04:14:25.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.648+0000 7feeb67fc640 1 -- 172.21.15.76:0/1397291258 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feebc01c070 con 0x7feec00af2a0 2024-09-19T04:14:25.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.648+0000 7feeb67fc640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feea40678c0 0x7feea4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:25.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.648+0000 7feeb67fc640 1 -- 172.21.15.76:0/1397291258 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7feebc085b70 con 0x7feec00af2a0 2024-09-19T04:14:25.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.648+0000 7feec51dd640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feea40678c0 0x7feea4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:25.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.648+0000 7feec51dd640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feea40678c0 0x7feea4069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feeb00ad000 tx=0x7feeb00a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:25.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.652+0000 7feeb67fc640 1 -- 172.21.15.76:0/1397291258 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feebc0565b0 con 0x7feec00af2a0 2024-09-19T04:14:25.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.804+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feec01576c0 con 0x7feea40678c0 2024-09-19T04:14:25.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.816+0000 7feeb67fc640 1 -- 172.21.15.76:0/1397291258 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251975 (secure 0 0 0) 0x7feeb00ab580 con 0x7feea40678c0 2024-09-19T04:14:25.821 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:25.822 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:14:25.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feea40678c0 msgr2=0x7feea4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:25.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feea40678c0 0x7feea4069d80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7feeb00ad000 tx=0x7feeb00a6000 comp rx=0 tx=0).stop 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec00af2a0 msgr2=0x7feec0151b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec00af2a0 0x7feec0151b10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7feebc00ba50 tx=0x7feebc00bf10 comp rx=0 tx=0).stop 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 shutdown_connections 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feea40678c0 0x7feea4069d80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feec015c2a0 0x7feec0099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feec015bec0 0x7feec01531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 --2- 172.21.15.76:0/1397291258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feec00af2a0 0x7feec0151b10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:25.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 >> 172.21.15.76:0/1397291258 conn(0x7feec00b1f90 msgr2=0x7feec01546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:25.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 shutdown_connections 2024-09-19T04:14:25.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:25.820+0000 7feec7c42640 1 -- 172.21.15.76:0/1397291258 wait complete. 2024-09-19T04:14:25.837 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-19T04:14:25.757270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":229288,"kb_used_data":4712,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754745432,"statfs":{"total":773094113280,"available":772859322368,"internally_reserved":0,"allocated":4825088,"data_stored":2732972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":60,"apply_latency_ms":60,"commit_latency_ns":60000000,"apply_latency_ns":60000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003729"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951018+0000","last_change":"2024-09-19T04:14:18.081769+0000","last_active":"2024-09-19T04:14:19.951018+0000","last_peered":"2024-09-19T04:14:19.951018+0000","last_clean":"2024-09-19T04:14:19.951018+0000","last_became_active":"2024-09-19T04:14:18.081000+0000","last_became_peered":"2024-09-19T04:14:18.081000+0000","last_unstale":"2024-09-19T04:14:19.951018+0000","last_undegraded":"2024-09-19T04:14:19.951018+0000","last_fullsized":"2024-09-19T04:14:19.951018+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:55.071630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951554+0000","last_change":"2024-09-19T04:14:18.065719+0000","last_active":"2024-09-19T04:14:19.951554+0000","last_peered":"2024-09-19T04:14:19.951554+0000","last_clean":"2024-09-19T04:14:19.951554+0000","last_became_active":"2024-09-19T04:14:18.065464+0000","last_became_peered":"2024-09-19T04:14:18.065464+0000","last_unstale":"2024-09-19T04:14:19.951554+0000","last_undegraded":"2024-09-19T04:14:19.951554+0000","last_fullsized":"2024-09-19T04:14:19.951554+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:44:05.667870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.286215+0000","last_change":"2024-09-19T04:14:18.105347+0000","last_active":"2024-09-19T04:14:19.286215+0000","last_peered":"2024-09-19T04:14:19.286215+0000","last_clean":"2024-09-19T04:14:19.286215+0000","last_became_active":"2024-09-19T04:14:18.105016+0000","last_became_peered":"2024-09-19T04:14:18.105016+0000","last_unstale":"2024-09-19T04:14:19.286215+0000","last_undegraded":"2024-09-19T04:14:19.286215+0000","last_fullsized":"2024-09-19T04:14:19.286215+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:03.738572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060336+0000","last_change":"2024-09-19T04:14:18.121950+0000","last_active":"2024-09-19T04:14:19.060336+0000","last_peered":"2024-09-19T04:14:19.060336+0000","last_clean":"2024-09-19T04:14:19.060336+0000","last_became_active":"2024-09-19T04:14:18.121690+0000","last_became_peered":"2024-09-19T04:14:18.121690+0000","last_unstale":"2024-09-19T04:14:19.060336+0000","last_undegraded":"2024-09-19T04:14:19.060336+0000","last_fullsized":"2024-09-19T04:14:19.060336+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:35.400830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.059945+0000","last_change":"2024-09-19T04:14:18.117831+0000","last_active":"2024-09-19T04:14:19.059945+0000","last_peered":"2024-09-19T04:14:19.059945+0000","last_clean":"2024-09-19T04:14:19.059945+0000","last_became_active":"2024-09-19T04:14:18.117575+0000","last_became_peered":"2024-09-19T04:14:18.117575+0000","last_unstale":"2024-09-19T04:14:19.059945+0000","last_undegraded":"2024-09-19T04:14:19.059945+0000","last_fullsized":"2024-09-19T04:14:19.059945+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:10:47.754903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949766+0000","last_change":"2024-09-19T04:14:18.068697+0000","last_active":"2024-09-19T04:14:19.949766+0000","last_peered":"2024-09-19T04:14:19.949766+0000","last_clean":"2024-09-19T04:14:19.949766+0000","last_became_active":"2024-09-19T04:14:18.068439+0000","last_became_peered":"2024-09-19T04:14:18.068439+0000","last_unstale":"2024-09-19T04:14:19.949766+0000","last_undegraded":"2024-09-19T04:14:19.949766+0000","last_fullsized":"2024-09-19T04:14:19.949766+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:56.046807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948197+0000","last_change":"2024-09-19T04:14:18.076538+0000","last_active":"2024-09-19T04:14:19.948197+0000","last_peered":"2024-09-19T04:14:19.948197+0000","last_clean":"2024-09-19T04:14:19.948197+0000","last_became_active":"2024-09-19T04:14:18.075548+0000","last_became_peered":"2024-09-19T04:14:18.075548+0000","last_unstale":"2024-09-19T04:14:19.948197+0000","last_undegraded":"2024-09-19T04:14:19.948197+0000","last_fullsized":"2024-09-19T04:14:19.948197+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:19:53.104549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.950422+0000","last_change":"2024-09-19T04:14:18.077195+0000","last_active":"2024-09-19T04:14:19.950422+0000","last_peered":"2024-09-19T04:14:19.950422+0000","last_clean":"2024-09-19T04:14:19.950422+0000","last_became_active":"2024-09-19T04:14:18.076882+0000","last_became_peered":"2024-09-19T04:14:18.076882+0000","last_unstale":"2024-09-19T04:14:19.950422+0000","last_undegraded":"2024-09-19T04:14:19.950422+0000","last_fullsized":"2024-09-19T04:14:19.950422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.201913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.286455+0000","last_change":"2024-09-19T04:14:18.108477+0000","last_active":"2024-09-19T04:14:19.286455+0000","last_peered":"2024-09-19T04:14:19.286455+0000","last_clean":"2024-09-19T04:14:19.286455+0000","last_became_active":"2024-09-19T04:14:18.108253+0000","last_became_peered":"2024-09-19T04:14:18.108253+0000","last_unstale":"2024-09-19T04:14:19.286455+0000","last_undegraded":"2024-09-19T04:14:19.286455+0000","last_fullsized":"2024-09-19T04:14:19.286455+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:34.624509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948799+0000","last_change":"2024-09-19T04:14:18.078021+0000","last_active":"2024-09-19T04:14:19.948799+0000","last_peered":"2024-09-19T04:14:19.948799+0000","last_clean":"2024-09-19T04:14:19.948799+0000","last_became_active":"2024-09-19T04:14:18.076747+0000","last_became_peered":"2024-09-19T04:14:18.076747+0000","last_unstale":"2024-09-19T04:14:19.948799+0000","last_undegraded":"2024-09-19T04:14:19.948799+0000","last_fullsized":"2024-09-19T04:14:19.948799+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:57:55.071630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949037+0000","last_change":"2024-09-19T04:14:18.117370+0000","last_active":"2024-09-19T04:14:19.949037+0000","last_peered":"2024-09-19T04:14:19.949037+0000","last_clean":"2024-09-19T04:14:19.949037+0000","last_became_active":"2024-09-19T04:14:18.116770+0000","last_became_peered":"2024-09-19T04:14:18.116770+0000","last_unstale":"2024-09-19T04:14:19.949037+0000","last_undegraded":"2024-09-19T04:14:19.949037+0000","last_fullsized":"2024-09-19T04:14:19.949037+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:50.768804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.285852+0000","last_change":"2024-09-19T04:14:18.076870+0000","last_active":"2024-09-19T04:14:19.285852+0000","last_peered":"2024-09-19T04:14:19.285852+0000","last_clean":"2024-09-19T04:14:19.285852+0000","last_became_active":"2024-09-19T04:14:18.076597+0000","last_became_peered":"2024-09-19T04:14:18.076597+0000","last_unstale":"2024-09-19T04:14:19.285852+0000","last_undegraded":"2024-09-19T04:14:19.285852+0000","last_fullsized":"2024-09-19T04:14:19.285852+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:25.065945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948342+0000","last_change":"2024-09-19T04:14:18.099653+0000","last_active":"2024-09-19T04:14:19.948342+0000","last_peered":"2024-09-19T04:14:19.948342+0000","last_clean":"2024-09-19T04:14:19.948342+0000","last_became_active":"2024-09-19T04:14:18.098956+0000","last_became_peered":"2024-09-19T04:14:18.098956+0000","last_unstale":"2024-09-19T04:14:19.948342+0000","last_undegraded":"2024-09-19T04:14:19.948342+0000","last_fullsized":"2024-09-19T04:14:19.948342+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:35:47.217546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.152878+0000","last_change":"2024-09-19T04:14:18.123658+0000","last_active":"2024-09-19T04:14:19.152878+0000","last_peered":"2024-09-19T04:14:19.152878+0000","last_clean":"2024-09-19T04:14:19.152878+0000","last_became_active":"2024-09-19T04:14:18.119126+0000","last_became_peered":"2024-09-19T04:14:18.119126+0000","last_unstale":"2024-09-19T04:14:19.152878+0000","last_undegraded":"2024-09-19T04:14:19.152878+0000","last_fullsized":"2024-09-19T04:14:19.152878+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:28.168984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948338+0000","last_change":"2024-09-19T04:14:18.099057+0000","last_active":"2024-09-19T04:14:19.948338+0000","last_peered":"2024-09-19T04:14:19.948338+0000","last_clean":"2024-09-19T04:14:19.948338+0000","last_became_active":"2024-09-19T04:14:18.096369+0000","last_became_peered":"2024-09-19T04:14:18.096369+0000","last_unstale":"2024-09-19T04:14:19.948338+0000","last_undegraded":"2024-09-19T04:14:19.948338+0000","last_fullsized":"2024-09-19T04:14:19.948338+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:59.201913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.059757+0000","last_change":"2024-09-19T04:14:18.103274+0000","last_active":"2024-09-19T04:14:19.059757+0000","last_peered":"2024-09-19T04:14:19.059757+0000","last_clean":"2024-09-19T04:14:19.059757+0000","last_became_active":"2024-09-19T04:14:18.102615+0000","last_became_peered":"2024-09-19T04:14:18.102615+0000","last_unstale":"2024-09-19T04:14:19.059757+0000","last_undegraded":"2024-09-19T04:14:19.059757+0000","last_fullsized":"2024-09-19T04:14:19.059757+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:44.858244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.061716+0000","last_change":"2024-09-19T04:14:18.110202+0000","last_active":"2024-09-19T04:14:19.061716+0000","last_peered":"2024-09-19T04:14:19.061716+0000","last_clean":"2024-09-19T04:14:19.061716+0000","last_became_active":"2024-09-19T04:14:18.109689+0000","last_became_peered":"2024-09-19T04:14:18.109689+0000","last_unstale":"2024-09-19T04:14:19.061716+0000","last_undegraded":"2024-09-19T04:14:19.061716+0000","last_fullsized":"2024-09-19T04:14:19.061716+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:16:22.809055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.062756+0000","last_change":"2024-09-19T04:14:18.121921+0000","last_active":"2024-09-19T04:14:19.062756+0000","last_peered":"2024-09-19T04:14:19.062756+0000","last_clean":"2024-09-19T04:14:19.062756+0000","last_became_active":"2024-09-19T04:14:18.121058+0000","last_became_peered":"2024-09-19T04:14:18.121058+0000","last_unstale":"2024-09-19T04:14:19.062756+0000","last_undegraded":"2024-09-19T04:14:19.062756+0000","last_fullsized":"2024-09-19T04:14:19.062756+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:57.116494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060736+0000","last_change":"2024-09-19T04:14:18.072202+0000","last_active":"2024-09-19T04:14:19.060736+0000","last_peered":"2024-09-19T04:14:19.060736+0000","last_clean":"2024-09-19T04:14:19.060736+0000","last_became_active":"2024-09-19T04:14:18.071924+0000","last_became_peered":"2024-09-19T04:14:18.071924+0000","last_unstale":"2024-09-19T04:14:19.060736+0000","last_undegraded":"2024-09-19T04:14:19.060736+0000","last_fullsized":"2024-09-19T04:14:19.060736+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:10.005552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.062401+0000","last_change":"2024-09-19T04:14:18.121893+0000","last_active":"2024-09-19T04:14:19.062401+0000","last_peered":"2024-09-19T04:14:19.062401+0000","last_clean":"2024-09-19T04:14:19.062401+0000","last_became_active":"2024-09-19T04:14:18.121006+0000","last_became_peered":"2024-09-19T04:14:18.121006+0000","last_unstale":"2024-09-19T04:14:19.062401+0000","last_undegraded":"2024-09-19T04:14:19.062401+0000","last_fullsized":"2024-09-19T04:14:19.062401+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:28.168984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948482+0000","last_change":"2024-09-19T04:14:18.094745+0000","last_active":"2024-09-19T04:14:19.948482+0000","last_peered":"2024-09-19T04:14:19.948482+0000","last_clean":"2024-09-19T04:14:19.948482+0000","last_became_active":"2024-09-19T04:14:18.093906+0000","last_became_peered":"2024-09-19T04:14:18.093906+0000","last_unstale":"2024-09-19T04:14:19.948482+0000","last_undegraded":"2024-09-19T04:14:19.948482+0000","last_fullsized":"2024-09-19T04:14:19.948482+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:09.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.153765+0000","last_change":"2024-09-19T04:14:18.118549+0000","last_active":"2024-09-19T04:14:19.153765+0000","last_peered":"2024-09-19T04:14:19.153765+0000","last_clean":"2024-09-19T04:14:19.153765+0000","last_became_active":"2024-09-19T04:14:18.115358+0000","last_became_peered":"2024-09-19T04:14:18.115358+0000","last_unstale":"2024-09-19T04:14:19.153765+0000","last_undegraded":"2024-09-19T04:14:19.153765+0000","last_fullsized":"2024-09-19T04:14:19.153765+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:56.046807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060701+0000","last_change":"2024-09-19T04:14:18.081802+0000","last_active":"2024-09-19T04:14:19.060701+0000","last_peered":"2024-09-19T04:14:19.060701+0000","last_clean":"2024-09-19T04:14:19.060701+0000","last_became_active":"2024-09-19T04:14:18.080818+0000","last_became_peered":"2024-09-19T04:14:18.080818+0000","last_unstale":"2024-09-19T04:14:19.060701+0000","last_undegraded":"2024-09-19T04:14:19.060701+0000","last_fullsized":"2024-09-19T04:14:19.060701+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:08.781770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.059552+0000","last_change":"2024-09-19T04:14:18.085399+0000","last_active":"2024-09-19T04:14:19.059552+0000","last_peered":"2024-09-19T04:14:19.059552+0000","last_clean":"2024-09-19T04:14:19.059552+0000","last_became_active":"2024-09-19T04:14:18.084948+0000","last_became_peered":"2024-09-19T04:14:18.084948+0000","last_unstale":"2024-09-19T04:14:19.059552+0000","last_undegraded":"2024-09-19T04:14:19.059552+0000","last_fullsized":"2024-09-19T04:14:19.059552+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:58:28.469799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.063154+0000","last_change":"2024-09-19T04:14:18.081698+0000","last_active":"2024-09-19T04:14:19.063154+0000","last_peered":"2024-09-19T04:14:19.063154+0000","last_clean":"2024-09-19T04:14:19.063154+0000","last_became_active":"2024-09-19T04:14:18.080893+0000","last_became_peered":"2024-09-19T04:14:18.080893+0000","last_unstale":"2024-09-19T04:14:19.063154+0000","last_undegraded":"2024-09-19T04:14:19.063154+0000","last_fullsized":"2024-09-19T04:14:19.063154+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:29.232311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.950776+0000","last_change":"2024-09-19T04:14:18.112770+0000","last_active":"2024-09-19T04:14:19.950776+0000","last_peered":"2024-09-19T04:14:19.950776+0000","last_clean":"2024-09-19T04:14:19.950776+0000","last_became_active":"2024-09-19T04:14:18.112506+0000","last_became_peered":"2024-09-19T04:14:18.112506+0000","last_unstale":"2024-09-19T04:14:19.950776+0000","last_undegraded":"2024-09-19T04:14:19.950776+0000","last_fullsized":"2024-09-19T04:14:19.950776+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:33.111541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.062637+0000","last_change":"2024-09-19T04:14:18.093273+0000","last_active":"2024-09-19T04:14:19.062637+0000","last_peered":"2024-09-19T04:14:19.062637+0000","last_clean":"2024-09-19T04:14:19.062637+0000","last_became_active":"2024-09-19T04:14:18.092556+0000","last_became_peered":"2024-09-19T04:14:18.092556+0000","last_unstale":"2024-09-19T04:14:19.062637+0000","last_undegraded":"2024-09-19T04:14:19.062637+0000","last_fullsized":"2024-09-19T04:14:19.062637+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:56.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949528+0000","last_change":"2024-09-19T04:14:18.079816+0000","last_active":"2024-09-19T04:14:19.949528+0000","last_peered":"2024-09-19T04:14:19.949528+0000","last_clean":"2024-09-19T04:14:19.949528+0000","last_became_active":"2024-09-19T04:14:18.079016+0000","last_became_peered":"2024-09-19T04:14:18.079016+0000","last_unstale":"2024-09-19T04:14:19.949528+0000","last_undegraded":"2024-09-19T04:14:19.949528+0000","last_fullsized":"2024-09-19T04:14:19.949528+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:45.395620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.061966+0000","last_change":"2024-09-19T04:14:18.072880+0000","last_active":"2024-09-19T04:14:19.061966+0000","last_peered":"2024-09-19T04:14:19.061966+0000","last_clean":"2024-09-19T04:14:19.061966+0000","last_became_active":"2024-09-19T04:14:18.072142+0000","last_became_peered":"2024-09-19T04:14:18.072142+0000","last_unstale":"2024-09-19T04:14:19.061966+0000","last_undegraded":"2024-09-19T04:14:19.061966+0000","last_fullsized":"2024-09-19T04:14:19.061966+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:57.791704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060036+0000","last_change":"2024-09-19T04:14:18.079581+0000","last_active":"2024-09-19T04:14:19.060036+0000","last_peered":"2024-09-19T04:14:19.060036+0000","last_clean":"2024-09-19T04:14:19.060036+0000","last_became_active":"2024-09-19T04:14:18.078877+0000","last_became_peered":"2024-09-19T04:14:18.078877+0000","last_unstale":"2024-09-19T04:14:19.060036+0000","last_undegraded":"2024-09-19T04:14:19.060036+0000","last_fullsized":"2024-09-19T04:14:19.060036+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:10.314449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.061117+0000","last_change":"2024-09-19T04:14:18.082906+0000","last_active":"2024-09-19T04:14:19.061117+0000","last_peered":"2024-09-19T04:14:19.061117+0000","last_clean":"2024-09-19T04:14:19.061117+0000","last_became_active":"2024-09-19T04:14:18.081900+0000","last_became_peered":"2024-09-19T04:14:18.081900+0000","last_unstale":"2024-09-19T04:14:19.061117+0000","last_undegraded":"2024-09-19T04:14:19.061117+0000","last_fullsized":"2024-09-19T04:14:19.061117+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:23:04.858846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949451+0000","last_change":"2024-09-19T04:14:18.071493+0000","last_active":"2024-09-19T04:14:19.949451+0000","last_peered":"2024-09-19T04:14:19.949451+0000","last_clean":"2024-09-19T04:14:19.949451+0000","last_became_active":"2024-09-19T04:14:18.071223+0000","last_became_peered":"2024-09-19T04:14:18.071223+0000","last_unstale":"2024-09-19T04:14:19.949451+0000","last_undegraded":"2024-09-19T04:14:19.949451+0000","last_fullsized":"2024-09-19T04:14:19.949451+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:18.835309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060989+0000","last_change":"2024-09-19T04:14:18.109820+0000","last_active":"2024-09-19T04:14:19.060989+0000","last_peered":"2024-09-19T04:14:19.060989+0000","last_clean":"2024-09-19T04:14:19.060989+0000","last_became_active":"2024-09-19T04:14:18.109195+0000","last_became_peered":"2024-09-19T04:14:18.109195+0000","last_unstale":"2024-09-19T04:14:19.060989+0000","last_undegraded":"2024-09-19T04:14:19.060989+0000","last_fullsized":"2024-09-19T04:14:19.060989+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:53:57.625955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.063104+0000","last_change":"2024-09-19T04:14:17.088483+0000","last_active":"2024-09-19T04:14:19.063104+0000","last_peered":"2024-09-19T04:14:19.063104+0000","last_clean":"2024-09-19T04:14:19.063104+0000","last_became_active":"2024-09-19T04:14:17.087794+0000","last_became_peered":"2024-09-19T04:14:17.087794+0000","last_unstale":"2024-09-19T04:14:19.063104+0000","last_undegraded":"2024-09-19T04:14:19.063104+0000","last_fullsized":"2024-09-19T04:14:19.063104+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:10.662426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.950914+0000","last_change":"2024-09-19T04:14:18.095388+0000","last_active":"2024-09-19T04:14:19.950914+0000","last_peered":"2024-09-19T04:14:19.950914+0000","last_clean":"2024-09-19T04:14:19.950914+0000","last_became_active":"2024-09-19T04:14:18.095083+0000","last_became_peered":"2024-09-19T04:14:18.095083+0000","last_unstale":"2024-09-19T04:14:19.950914+0000","last_undegraded":"2024-09-19T04:14:19.950914+0000","last_fullsized":"2024-09-19T04:14:19.950914+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:10.000174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.285933+0000","last_change":"2024-09-19T04:14:17.093169+0000","last_active":"2024-09-19T04:14:19.285933+0000","last_peered":"2024-09-19T04:14:19.285933+0000","last_clean":"2024-09-19T04:14:19.285933+0000","last_became_active":"2024-09-19T04:14:17.092881+0000","last_became_peered":"2024-09-19T04:14:17.092881+0000","last_unstale":"2024-09-19T04:14:19.285933+0000","last_undegraded":"2024-09-19T04:14:19.285933+0000","last_fullsized":"2024-09-19T04:14:19.285933+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:13.045881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.153764+0000","last_change":"2024-09-19T04:14:18.120150+0000","last_active":"2024-09-19T04:14:19.153764+0000","last_peered":"2024-09-19T04:14:19.153764+0000","last_clean":"2024-09-19T04:14:19.153764+0000","last_became_active":"2024-09-19T04:14:18.115539+0000","last_became_peered":"2024-09-19T04:14:18.115539+0000","last_unstale":"2024-09-19T04:14:19.153764+0000","last_undegraded":"2024-09-19T04:14:19.153764+0000","last_fullsized":"2024-09-19T04:14:19.153764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:25.065945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951219+0000","last_change":"2024-09-19T04:14:17.076870+0000","last_active":"2024-09-19T04:14:19.951219+0000","last_peered":"2024-09-19T04:14:19.951219+0000","last_clean":"2024-09-19T04:14:19.951219+0000","last_became_active":"2024-09-19T04:14:17.076604+0000","last_became_peered":"2024-09-19T04:14:17.076604+0000","last_unstale":"2024-09-19T04:14:19.951219+0000","last_undegraded":"2024-09-19T04:14:19.951219+0000","last_fullsized":"2024-09-19T04:14:19.951219+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:59.306800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.950813+0000","last_change":"2024-09-19T04:14:18.082021+0000","last_active":"2024-09-19T04:14:19.950813+0000","last_peered":"2024-09-19T04:14:19.950813+0000","last_clean":"2024-09-19T04:14:19.950813+0000","last_became_active":"2024-09-19T04:14:18.081453+0000","last_became_peered":"2024-09-19T04:14:18.081453+0000","last_unstale":"2024-09-19T04:14:19.950813+0000","last_undegraded":"2024-09-19T04:14:19.950813+0000","last_fullsized":"2024-09-19T04:14:19.950813+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:09.811005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.153337+0000","last_change":"2024-09-19T04:14:17.072407+0000","last_active":"2024-09-19T04:14:19.153337+0000","last_peered":"2024-09-19T04:14:19.153337+0000","last_clean":"2024-09-19T04:14:19.153337+0000","last_became_active":"2024-09-19T04:14:17.071689+0000","last_became_peered":"2024-09-19T04:14:17.071689+0000","last_unstale":"2024-09-19T04:14:19.153337+0000","last_undegraded":"2024-09-19T04:14:19.153337+0000","last_fullsized":"2024-09-19T04:14:19.153337+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:50:28.051215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.286274+0000","last_change":"2024-09-19T04:14:18.078818+0000","last_active":"2024-09-19T04:14:19.286274+0000","last_peered":"2024-09-19T04:14:19.286274+0000","last_clean":"2024-09-19T04:14:19.286274+0000","last_became_active":"2024-09-19T04:14:18.078248+0000","last_became_peered":"2024-09-19T04:14:18.078248+0000","last_unstale":"2024-09-19T04:14:19.286274+0000","last_undegraded":"2024-09-19T04:14:19.286274+0000","last_fullsized":"2024-09-19T04:14:19.286274+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:04:10.000174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948697+0000","last_change":"2024-09-19T04:14:17.083299+0000","last_active":"2024-09-19T04:14:19.948697+0000","last_peered":"2024-09-19T04:14:19.948697+0000","last_clean":"2024-09-19T04:14:19.948697+0000","last_became_active":"2024-09-19T04:14:17.081493+0000","last_became_peered":"2024-09-19T04:14:17.081493+0000","last_unstale":"2024-09-19T04:14:19.948697+0000","last_undegraded":"2024-09-19T04:14:19.948697+0000","last_fullsized":"2024-09-19T04:14:19.948697+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:10:56.776172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948674+0000","last_change":"2024-09-19T04:14:18.096989+0000","last_active":"2024-09-19T04:14:19.948674+0000","last_peered":"2024-09-19T04:14:19.948674+0000","last_clean":"2024-09-19T04:14:19.948674+0000","last_became_active":"2024-09-19T04:14:18.095531+0000","last_became_peered":"2024-09-19T04:14:18.095531+0000","last_unstale":"2024-09-19T04:14:19.948674+0000","last_undegraded":"2024-09-19T04:14:19.948674+0000","last_fullsized":"2024-09-19T04:14:19.948674+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:56.131423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060609+0000","last_change":"2024-09-19T04:14:17.098330+0000","last_active":"2024-09-19T04:14:19.060609+0000","last_peered":"2024-09-19T04:14:19.060609+0000","last_clean":"2024-09-19T04:14:19.060609+0000","last_became_active":"2024-09-19T04:14:17.097696+0000","last_became_peered":"2024-09-19T04:14:17.097696+0000","last_unstale":"2024-09-19T04:14:19.060609+0000","last_undegraded":"2024-09-19T04:14:19.060609+0000","last_fullsized":"2024-09-19T04:14:19.060609+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:29:08.990020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.153410+0000","last_change":"2024-09-19T04:14:18.119201+0000","last_active":"2024-09-19T04:14:19.153410+0000","last_peered":"2024-09-19T04:14:19.153410+0000","last_clean":"2024-09-19T04:14:19.153410+0000","last_became_active":"2024-09-19T04:14:18.118113+0000","last_became_peered":"2024-09-19T04:14:18.118113+0000","last_unstale":"2024-09-19T04:14:19.153410+0000","last_undegraded":"2024-09-19T04:14:19.153410+0000","last_fullsized":"2024-09-19T04:14:19.153410+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:57.116494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949942+0000","last_change":"2024-09-19T04:14:17.097839+0000","last_active":"2024-09-19T04:14:19.949942+0000","last_peered":"2024-09-19T04:14:19.949942+0000","last_clean":"2024-09-19T04:14:19.949942+0000","last_became_active":"2024-09-19T04:14:17.097522+0000","last_became_peered":"2024-09-19T04:14:17.097522+0000","last_unstale":"2024-09-19T04:14:19.949942+0000","last_undegraded":"2024-09-19T04:14:19.949942+0000","last_fullsized":"2024-09-19T04:14:19.949942+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:25:13.045881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.950840+0000","last_change":"2024-09-19T04:14:18.086309+0000","last_active":"2024-09-19T04:14:19.950840+0000","last_peered":"2024-09-19T04:14:19.950840+0000","last_clean":"2024-09-19T04:14:19.950840+0000","last_became_active":"2024-09-19T04:14:18.086012+0000","last_became_peered":"2024-09-19T04:14:18.086012+0000","last_unstale":"2024-09-19T04:14:19.950840+0000","last_undegraded":"2024-09-19T04:14:19.950840+0000","last_fullsized":"2024-09-19T04:14:19.950840+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:22:56.131423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.152676+0000","last_change":"2024-09-19T04:14:17.088506+0000","last_active":"2024-09-19T04:14:19.152676+0000","last_peered":"2024-09-19T04:14:19.152676+0000","last_clean":"2024-09-19T04:14:19.152676+0000","last_became_active":"2024-09-19T04:14:17.087746+0000","last_became_peered":"2024-09-19T04:14:17.087746+0000","last_unstale":"2024-09-19T04:14:19.152676+0000","last_undegraded":"2024-09-19T04:14:19.152676+0000","last_fullsized":"2024-09-19T04:14:19.152676+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060107+0000","last_change":"2024-09-19T04:14:18.121813+0000","last_active":"2024-09-19T04:14:19.060107+0000","last_peered":"2024-09-19T04:14:19.060107+0000","last_clean":"2024-09-19T04:14:19.060107+0000","last_became_active":"2024-09-19T04:14:18.121603+0000","last_became_peered":"2024-09-19T04:14:18.121603+0000","last_unstale":"2024-09-19T04:14:19.060107+0000","last_undegraded":"2024-09-19T04:14:19.060107+0000","last_fullsized":"2024-09-19T04:14:19.060107+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:11:27.808426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951142+0000","last_change":"2024-09-19T04:14:17.070017+0000","last_active":"2024-09-19T04:14:19.951142+0000","last_peered":"2024-09-19T04:14:19.951142+0000","last_clean":"2024-09-19T04:14:19.951142+0000","last_became_active":"2024-09-19T04:14:17.069539+0000","last_became_peered":"2024-09-19T04:14:17.069539+0000","last_unstale":"2024-09-19T04:14:19.951142+0000","last_undegraded":"2024-09-19T04:14:19.951142+0000","last_fullsized":"2024-09-19T04:14:19.951142+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:17.819777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.061771+0000","last_change":"2024-09-19T04:14:18.110143+0000","last_active":"2024-09-19T04:14:19.061771+0000","last_peered":"2024-09-19T04:14:19.061771+0000","last_clean":"2024-09-19T04:14:19.061771+0000","last_became_active":"2024-09-19T04:14:18.109381+0000","last_became_peered":"2024-09-19T04:14:18.109381+0000","last_unstale":"2024-09-19T04:14:19.061771+0000","last_undegraded":"2024-09-19T04:14:19.061771+0000","last_fullsized":"2024-09-19T04:14:19.061771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:29:01.783897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949158+0000","last_change":"2024-09-19T04:14:17.094996+0000","last_active":"2024-09-19T04:14:19.949158+0000","last_peered":"2024-09-19T04:14:19.949158+0000","last_clean":"2024-09-19T04:14:19.949158+0000","last_became_active":"2024-09-19T04:14:17.083414+0000","last_became_peered":"2024-09-19T04:14:17.083414+0000","last_unstale":"2024-09-19T04:14:19.949158+0000","last_undegraded":"2024-09-19T04:14:19.949158+0000","last_fullsized":"2024-09-19T04:14:19.949158+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:31:50.043891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.285922+0000","last_change":"2024-09-19T04:14:18.083836+0000","last_active":"2024-09-19T04:14:19.285922+0000","last_peered":"2024-09-19T04:14:19.285922+0000","last_clean":"2024-09-19T04:14:19.285922+0000","last_became_active":"2024-09-19T04:14:18.083550+0000","last_became_peered":"2024-09-19T04:14:18.083550+0000","last_unstale":"2024-09-19T04:14:19.285922+0000","last_undegraded":"2024-09-19T04:14:19.285922+0000","last_fullsized":"2024-09-19T04:14:19.285922+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:38:56.046807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949373+0000","last_change":"2024-09-19T04:14:17.094246+0000","last_active":"2024-09-19T04:14:19.949373+0000","last_peered":"2024-09-19T04:14:19.949373+0000","last_clean":"2024-09-19T04:14:19.949373+0000","last_became_active":"2024-09-19T04:14:17.083763+0000","last_became_peered":"2024-09-19T04:14:17.083763+0000","last_unstale":"2024-09-19T04:14:19.949373+0000","last_undegraded":"2024-09-19T04:14:19.949373+0000","last_fullsized":"2024-09-19T04:14:19.949373+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:05:09.298918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.153774+0000","last_change":"2024-09-19T04:14:18.119690+0000","last_active":"2024-09-19T04:14:19.153774+0000","last_peered":"2024-09-19T04:14:19.153774+0000","last_clean":"2024-09-19T04:14:19.153774+0000","last_became_active":"2024-09-19T04:14:18.116171+0000","last_became_peered":"2024-09-19T04:14:18.116171+0000","last_unstale":"2024-09-19T04:14:19.153774+0000","last_undegraded":"2024-09-19T04:14:19.153774+0000","last_fullsized":"2024-09-19T04:14:19.153774+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:18.835309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951265+0000","last_change":"2024-09-19T04:14:17.094374+0000","last_active":"2024-09-19T04:14:19.951265+0000","last_peered":"2024-09-19T04:14:19.951265+0000","last_clean":"2024-09-19T04:14:19.951265+0000","last_became_active":"2024-09-19T04:14:17.093949+0000","last_became_peered":"2024-09-19T04:14:17.093949+0000","last_unstale":"2024-09-19T04:14:19.951265+0000","last_undegraded":"2024-09-19T04:14:19.951265+0000","last_fullsized":"2024-09-19T04:14:19.951265+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:28.216780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.285930+0000","last_change":"2024-09-19T04:14:18.108019+0000","last_active":"2024-09-19T04:14:19.285930+0000","last_peered":"2024-09-19T04:14:19.285930+0000","last_clean":"2024-09-19T04:14:19.285930+0000","last_became_active":"2024-09-19T04:14:18.107780+0000","last_became_peered":"2024-09-19T04:14:18.107780+0000","last_unstale":"2024-09-19T04:14:19.285930+0000","last_undegraded":"2024-09-19T04:14:19.285930+0000","last_fullsized":"2024-09-19T04:14:19.285930+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:50:33.111541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951473+0000","last_change":"2024-09-19T04:14:17.072319+0000","last_active":"2024-09-19T04:14:19.951473+0000","last_peered":"2024-09-19T04:14:19.951473+0000","last_clean":"2024-09-19T04:14:19.951473+0000","last_became_active":"2024-09-19T04:14:17.071960+0000","last_became_peered":"2024-09-19T04:14:17.071960+0000","last_unstale":"2024-09-19T04:14:19.951473+0000","last_undegraded":"2024-09-19T04:14:19.951473+0000","last_fullsized":"2024-09-19T04:14:19.951473+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:56.100962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949993+0000","last_change":"2024-09-19T04:14:18.080819+0000","last_active":"2024-09-19T04:14:19.949993+0000","last_peered":"2024-09-19T04:14:19.949993+0000","last_clean":"2024-09-19T04:14:19.949993+0000","last_became_active":"2024-09-19T04:14:18.078970+0000","last_became_peered":"2024-09-19T04:14:18.078970+0000","last_unstale":"2024-09-19T04:14:19.949993+0000","last_undegraded":"2024-09-19T04:14:19.949993+0000","last_fullsized":"2024-09-19T04:14:19.949993+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:30.468511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949903+0000","last_change":"2024-09-19T04:14:17.084245+0000","last_active":"2024-09-19T04:14:19.949903+0000","last_peered":"2024-09-19T04:14:19.949903+0000","last_clean":"2024-09-19T04:14:19.949903+0000","last_became_active":"2024-09-19T04:14:17.083828+0000","last_became_peered":"2024-09-19T04:14:17.083828+0000","last_unstale":"2024-09-19T04:14:19.949903+0000","last_undegraded":"2024-09-19T04:14:19.949903+0000","last_fullsized":"2024-09-19T04:14:19.949903+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:39.583489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060493+0000","last_change":"2024-09-19T04:14:18.081814+0000","last_active":"2024-09-19T04:14:19.060493+0000","last_peered":"2024-09-19T04:14:19.060493+0000","last_clean":"2024-09-19T04:14:19.060493+0000","last_became_active":"2024-09-19T04:14:18.080841+0000","last_became_peered":"2024-09-19T04:14:18.080841+0000","last_unstale":"2024-09-19T04:14:19.060493+0000","last_undegraded":"2024-09-19T04:14:19.060493+0000","last_fullsized":"2024-09-19T04:14:19.060493+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:33:52.782118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060147+0000","last_change":"2024-09-19T04:14:17.085767+0000","last_active":"2024-09-19T04:14:19.060147+0000","last_peered":"2024-09-19T04:14:19.060147+0000","last_clean":"2024-09-19T04:14:19.060147+0000","last_became_active":"2024-09-19T04:14:17.085016+0000","last_became_peered":"2024-09-19T04:14:17.085016+0000","last_unstale":"2024-09-19T04:14:19.060147+0000","last_undegraded":"2024-09-19T04:14:19.060147+0000","last_fullsized":"2024-09-19T04:14:19.060147+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:24.938921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951140+0000","last_change":"2024-09-19T04:14:18.076121+0000","last_active":"2024-09-19T04:14:19.951140+0000","last_peered":"2024-09-19T04:14:19.951140+0000","last_clean":"2024-09-19T04:14:19.951140+0000","last_became_active":"2024-09-19T04:14:18.075810+0000","last_became_peered":"2024-09-19T04:14:18.075810+0000","last_unstale":"2024-09-19T04:14:19.951140+0000","last_undegraded":"2024-09-19T04:14:19.951140+0000","last_fullsized":"2024-09-19T04:14:19.951140+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:21.837195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":22,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.059782+0000","last_change":"2024-09-19T04:14:17.098594+0000","last_active":"2024-09-19T04:14:19.059782+0000","last_peered":"2024-09-19T04:14:19.059782+0000","last_clean":"2024-09-19T04:14:19.059782+0000","last_became_active":"2024-09-19T04:14:17.097858+0000","last_became_peered":"2024-09-19T04:14:17.097858+0000","last_unstale":"2024-09-19T04:14:19.059782+0000","last_undegraded":"2024-09-19T04:14:19.059782+0000","last_fullsized":"2024-09-19T04:14:19.059782+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:14:34.385299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.950075+0000","last_change":"2024-09-19T04:14:18.072761+0000","last_active":"2024-09-19T04:14:19.950075+0000","last_peered":"2024-09-19T04:14:19.950075+0000","last_clean":"2024-09-19T04:14:19.950075+0000","last_became_active":"2024-09-19T04:14:18.072491+0000","last_became_peered":"2024-09-19T04:14:18.072491+0000","last_unstale":"2024-09-19T04:14:19.950075+0000","last_undegraded":"2024-09-19T04:14:19.950075+0000","last_fullsized":"2024-09-19T04:14:19.950075+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:35:47.217546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949349+0000","last_change":"2024-09-19T04:14:17.095361+0000","last_active":"2024-09-19T04:14:19.949349+0000","last_peered":"2024-09-19T04:14:19.949349+0000","last_clean":"2024-09-19T04:14:19.949349+0000","last_became_active":"2024-09-19T04:14:17.094445+0000","last_became_peered":"2024-09-19T04:14:17.094445+0000","last_unstale":"2024-09-19T04:14:19.949349+0000","last_undegraded":"2024-09-19T04:14:19.949349+0000","last_fullsized":"2024-09-19T04:14:19.949349+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:42:27.153452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949125+0000","last_change":"2024-09-19T04:14:18.096950+0000","last_active":"2024-09-19T04:14:19.949125+0000","last_peered":"2024-09-19T04:14:19.949125+0000","last_clean":"2024-09-19T04:14:19.949125+0000","last_became_active":"2024-09-19T04:14:18.095696+0000","last_became_peered":"2024-09-19T04:14:18.095696+0000","last_unstale":"2024-09-19T04:14:19.949125+0000","last_undegraded":"2024-09-19T04:14:19.949125+0000","last_fullsized":"2024-09-19T04:14:19.949125+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:44:05.667870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.153852+0000","last_change":"2024-09-19T04:14:17.098487+0000","last_active":"2024-09-19T04:14:19.153852+0000","last_peered":"2024-09-19T04:14:19.153852+0000","last_clean":"2024-09-19T04:14:19.153852+0000","last_became_active":"2024-09-19T04:14:17.097805+0000","last_became_peered":"2024-09-19T04:14:17.097805+0000","last_unstale":"2024-09-19T04:14:19.153852+0000","last_undegraded":"2024-09-19T04:14:19.153852+0000","last_fullsized":"2024-09-19T04:14:19.153852+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:39.583489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.153680+0000","last_change":"2024-09-19T04:14:18.123579+0000","last_active":"2024-09-19T04:14:19.153680+0000","last_peered":"2024-09-19T04:14:19.153680+0000","last_clean":"2024-09-19T04:14:19.153680+0000","last_became_active":"2024-09-19T04:14:18.116031+0000","last_became_peered":"2024-09-19T04:14:18.116031+0000","last_unstale":"2024-09-19T04:14:19.153680+0000","last_undegraded":"2024-09-19T04:14:19.153680+0000","last_fullsized":"2024-09-19T04:14:19.153680+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:04:29.232311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060638+0000","last_change":"2024-09-19T04:14:17.093924+0000","last_active":"2024-09-19T04:14:19.060638+0000","last_peered":"2024-09-19T04:14:19.060638+0000","last_clean":"2024-09-19T04:14:19.060638+0000","last_became_active":"2024-09-19T04:14:17.093578+0000","last_became_peered":"2024-09-19T04:14:17.093578+0000","last_unstale":"2024-09-19T04:14:19.060638+0000","last_undegraded":"2024-09-19T04:14:19.060638+0000","last_fullsized":"2024-09-19T04:14:19.060638+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:38:51.519957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060690+0000","last_change":"2024-09-19T04:14:18.066985+0000","last_active":"2024-09-19T04:14:19.060690+0000","last_peered":"2024-09-19T04:14:19.060690+0000","last_clean":"2024-09-19T04:14:19.060690+0000","last_became_active":"2024-09-19T04:14:18.066566+0000","last_became_peered":"2024-09-19T04:14:18.066566+0000","last_unstale":"2024-09-19T04:14:19.060690+0000","last_undegraded":"2024-09-19T04:14:19.060690+0000","last_fullsized":"2024-09-19T04:14:19.060690+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:52.901946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951712+0000","last_change":"2024-09-19T04:14:17.076993+0000","last_active":"2024-09-19T04:14:19.951712+0000","last_peered":"2024-09-19T04:14:19.951712+0000","last_clean":"2024-09-19T04:14:19.951712+0000","last_became_active":"2024-09-19T04:14:17.076271+0000","last_became_peered":"2024-09-19T04:14:17.076271+0000","last_unstale":"2024-09-19T04:14:19.951712+0000","last_undegraded":"2024-09-19T04:14:19.951712+0000","last_fullsized":"2024-09-19T04:14:19.951712+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:56:40.402841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060170+0000","last_change":"2024-09-19T04:14:18.083245+0000","last_active":"2024-09-19T04:14:19.060170+0000","last_peered":"2024-09-19T04:14:19.060170+0000","last_clean":"2024-09-19T04:14:19.060170+0000","last_became_active":"2024-09-19T04:14:18.082494+0000","last_became_peered":"2024-09-19T04:14:18.082494+0000","last_unstale":"2024-09-19T04:14:19.060170+0000","last_undegraded":"2024-09-19T04:14:19.060170+0000","last_fullsized":"2024-09-19T04:14:19.060170+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:00:52.798226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060212+0000","last_change":"2024-09-19T04:14:17.071279+0000","last_active":"2024-09-19T04:14:19.060212+0000","last_peered":"2024-09-19T04:14:19.060212+0000","last_clean":"2024-09-19T04:14:19.060212+0000","last_became_active":"2024-09-19T04:14:17.070910+0000","last_became_peered":"2024-09-19T04:14:17.070910+0000","last_unstale":"2024-09-19T04:14:19.060212+0000","last_undegraded":"2024-09-19T04:14:19.060212+0000","last_fullsized":"2024-09-19T04:14:19.060212+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:03:28.101753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.285764+0000","last_change":"2024-09-19T04:14:18.084102+0000","last_active":"2024-09-19T04:14:19.285764+0000","last_peered":"2024-09-19T04:14:19.285764+0000","last_clean":"2024-09-19T04:14:19.285764+0000","last_became_active":"2024-09-19T04:14:18.083826+0000","last_became_peered":"2024-09-19T04:14:18.083826+0000","last_unstale":"2024-09-19T04:14:19.285764+0000","last_undegraded":"2024-09-19T04:14:19.285764+0000","last_fullsized":"2024-09-19T04:14:19.285764+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:17:18.835309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060023+0000","last_change":"2024-09-19T04:14:17.080497+0000","last_active":"2024-09-19T04:14:19.060023+0000","last_peered":"2024-09-19T04:14:19.060023+0000","last_clean":"2024-09-19T04:14:19.060023+0000","last_became_active":"2024-09-19T04:14:17.080085+0000","last_became_peered":"2024-09-19T04:14:17.080085+0000","last_unstale":"2024-09-19T04:14:19.060023+0000","last_undegraded":"2024-09-19T04:14:19.060023+0000","last_fullsized":"2024-09-19T04:14:19.060023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:15:39.948570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948264+0000","last_change":"2024-09-19T04:14:18.107540+0000","last_active":"2024-09-19T04:14:19.948264+0000","last_peered":"2024-09-19T04:14:19.948264+0000","last_clean":"2024-09-19T04:14:19.948264+0000","last_became_active":"2024-09-19T04:14:18.106798+0000","last_became_peered":"2024-09-19T04:14:18.106798+0000","last_unstale":"2024-09-19T04:14:19.948264+0000","last_undegraded":"2024-09-19T04:14:19.948264+0000","last_fullsized":"2024-09-19T04:14:19.948264+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:30.468511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.061358+0000","last_change":"2024-09-19T04:14:17.095657+0000","last_active":"2024-09-19T04:14:19.061358+0000","last_peered":"2024-09-19T04:14:19.061358+0000","last_clean":"2024-09-19T04:14:19.061358+0000","last_became_active":"2024-09-19T04:14:17.094938+0000","last_became_peered":"2024-09-19T04:14:17.094938+0000","last_unstale":"2024-09-19T04:14:19.061358+0000","last_undegraded":"2024-09-19T04:14:19.061358+0000","last_fullsized":"2024-09-19T04:14:19.061358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:51.886415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951345+0000","last_change":"2024-09-19T04:14:18.086218+0000","last_active":"2024-09-19T04:14:19.951345+0000","last_peered":"2024-09-19T04:14:19.951345+0000","last_clean":"2024-09-19T04:14:19.951345+0000","last_became_active":"2024-09-19T04:14:18.084736+0000","last_became_peered":"2024-09-19T04:14:18.084736+0000","last_unstale":"2024-09-19T04:14:19.951345+0000","last_undegraded":"2024-09-19T04:14:19.951345+0000","last_fullsized":"2024-09-19T04:14:19.951345+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:29:50.768804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.059568+0000","last_change":"2024-09-19T04:14:17.067723+0000","last_active":"2024-09-19T04:14:19.059568+0000","last_peered":"2024-09-19T04:14:19.059568+0000","last_clean":"2024-09-19T04:14:19.059568+0000","last_became_active":"2024-09-19T04:14:17.067393+0000","last_became_peered":"2024-09-19T04:14:17.067393+0000","last_unstale":"2024-09-19T04:14:19.059568+0000","last_undegraded":"2024-09-19T04:14:19.059568+0000","last_fullsized":"2024-09-19T04:14:19.059568+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:36:41.208145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951571+0000","last_change":"2024-09-19T04:14:18.081930+0000","last_active":"2024-09-19T04:14:19.951571+0000","last_peered":"2024-09-19T04:14:19.951571+0000","last_clean":"2024-09-19T04:14:19.951571+0000","last_became_active":"2024-09-19T04:14:18.081650+0000","last_became_peered":"2024-09-19T04:14:18.081650+0000","last_unstale":"2024-09-19T04:14:19.951571+0000","last_undegraded":"2024-09-19T04:14:19.951571+0000","last_fullsized":"2024-09-19T04:14:19.951571+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:13:03.738572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949560+0000","last_change":"2024-09-19T04:14:17.094649+0000","last_active":"2024-09-19T04:14:19.949560+0000","last_peered":"2024-09-19T04:14:19.949560+0000","last_clean":"2024-09-19T04:14:19.949560+0000","last_became_active":"2024-09-19T04:14:17.083575+0000","last_became_peered":"2024-09-19T04:14:17.083575+0000","last_unstale":"2024-09-19T04:14:19.949560+0000","last_undegraded":"2024-09-19T04:14:19.949560+0000","last_fullsized":"2024-09-19T04:14:19.949560+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:55.313000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.062175+0000","last_change":"2024-09-19T04:14:18.109157+0000","last_active":"2024-09-19T04:14:19.062175+0000","last_peered":"2024-09-19T04:14:19.062175+0000","last_clean":"2024-09-19T04:14:19.062175+0000","last_became_active":"2024-09-19T04:14:18.108545+0000","last_became_peered":"2024-09-19T04:14:18.108545+0000","last_unstale":"2024-09-19T04:14:19.062175+0000","last_undegraded":"2024-09-19T04:14:19.062175+0000","last_fullsized":"2024-09-19T04:14:19.062175+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:47.566856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060483+0000","last_change":"2024-09-19T04:14:17.097837+0000","last_active":"2024-09-19T04:14:19.060483+0000","last_peered":"2024-09-19T04:14:19.060483+0000","last_clean":"2024-09-19T04:14:19.060483+0000","last_became_active":"2024-09-19T04:14:17.097561+0000","last_became_peered":"2024-09-19T04:14:17.097561+0000","last_unstale":"2024-09-19T04:14:19.060483+0000","last_undegraded":"2024-09-19T04:14:19.060483+0000","last_fullsized":"2024-09-19T04:14:19.060483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:07:20.852794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.951777+0000","last_change":"2024-09-19T04:14:18.123070+0000","last_active":"2024-09-19T04:14:19.951777+0000","last_peered":"2024-09-19T04:14:19.951777+0000","last_clean":"2024-09-19T04:14:19.951777+0000","last_became_active":"2024-09-19T04:14:18.122776+0000","last_became_peered":"2024-09-19T04:14:18.122776+0000","last_unstale":"2024-09-19T04:14:19.951777+0000","last_undegraded":"2024-09-19T04:14:19.951777+0000","last_fullsized":"2024-09-19T04:14:19.951777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:20:45.395620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.286128+0000","last_change":"2024-09-19T04:14:17.074004+0000","last_active":"2024-09-19T04:14:19.286128+0000","last_peered":"2024-09-19T04:14:19.286128+0000","last_clean":"2024-09-19T04:14:19.286128+0000","last_became_active":"2024-09-19T04:14:17.073653+0000","last_became_peered":"2024-09-19T04:14:17.073653+0000","last_unstale":"2024-09-19T04:14:19.286128+0000","last_undegraded":"2024-09-19T04:14:19.286128+0000","last_fullsized":"2024-09-19T04:14:19.286128+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:00:26.621305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.286343+0000","last_change":"2024-09-19T04:14:18.079025+0000","last_active":"2024-09-19T04:14:19.286343+0000","last_peered":"2024-09-19T04:14:19.286343+0000","last_clean":"2024-09-19T04:14:19.286343+0000","last_became_active":"2024-09-19T04:14:18.078597+0000","last_became_peered":"2024-09-19T04:14:18.078597+0000","last_unstale":"2024-09-19T04:14:19.286343+0000","last_undegraded":"2024-09-19T04:14:19.286343+0000","last_fullsized":"2024-09-19T04:14:19.286343+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:25:12.911714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.286070+0000","last_change":"2024-09-19T04:14:17.087371+0000","last_active":"2024-09-19T04:14:19.286070+0000","last_peered":"2024-09-19T04:14:19.286070+0000","last_clean":"2024-09-19T04:14:19.286070+0000","last_became_active":"2024-09-19T04:14:17.087047+0000","last_became_peered":"2024-09-19T04:14:17.087047+0000","last_unstale":"2024-09-19T04:14:19.286070+0000","last_undegraded":"2024-09-19T04:14:19.286070+0000","last_fullsized":"2024-09-19T04:14:19.286070+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:39.583489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949682+0000","last_change":"2024-09-19T04:14:18.080474+0000","last_active":"2024-09-19T04:14:19.949682+0000","last_peered":"2024-09-19T04:14:19.949682+0000","last_clean":"2024-09-19T04:14:19.949682+0000","last_became_active":"2024-09-19T04:14:18.080154+0000","last_became_peered":"2024-09-19T04:14:18.080154+0000","last_unstale":"2024-09-19T04:14:19.949682+0000","last_undegraded":"2024-09-19T04:14:19.949682+0000","last_fullsized":"2024-09-19T04:14:19.949682+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:25.065945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060025+0000","last_change":"2024-09-19T04:14:17.086127+0000","last_active":"2024-09-19T04:14:19.060025+0000","last_peered":"2024-09-19T04:14:19.060025+0000","last_clean":"2024-09-19T04:14:19.060025+0000","last_became_active":"2024-09-19T04:14:17.084606+0000","last_became_peered":"2024-09-19T04:14:17.084606+0000","last_unstale":"2024-09-19T04:14:19.060025+0000","last_undegraded":"2024-09-19T04:14:19.060025+0000","last_fullsized":"2024-09-19T04:14:19.060025+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:11.894761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.152937+0000","last_change":"2024-09-19T04:14:18.119664+0000","last_active":"2024-09-19T04:14:19.152937+0000","last_peered":"2024-09-19T04:14:19.152937+0000","last_clean":"2024-09-19T04:14:19.152937+0000","last_became_active":"2024-09-19T04:14:18.118187+0000","last_became_peered":"2024-09-19T04:14:18.118187+0000","last_unstale":"2024-09-19T04:14:19.152937+0000","last_undegraded":"2024-09-19T04:14:19.152937+0000","last_fullsized":"2024-09-19T04:14:19.152937+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:56.328531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.152700+0000","last_change":"2024-09-19T04:14:17.070371+0000","last_active":"2024-09-19T04:14:19.152700+0000","last_peered":"2024-09-19T04:14:19.152700+0000","last_clean":"2024-09-19T04:14:19.152700+0000","last_became_active":"2024-09-19T04:14:17.069663+0000","last_became_peered":"2024-09-19T04:14:17.069663+0000","last_unstale":"2024-09-19T04:14:19.152700+0000","last_undegraded":"2024-09-19T04:14:19.152700+0000","last_fullsized":"2024-09-19T04:14:19.152700+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:24.938921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.061152+0000","last_change":"2024-09-19T04:14:18.110491+0000","last_active":"2024-09-19T04:14:19.061152+0000","last_peered":"2024-09-19T04:14:19.061152+0000","last_clean":"2024-09-19T04:14:19.061152+0000","last_became_active":"2024-09-19T04:14:18.109486+0000","last_became_peered":"2024-09-19T04:14:18.109486+0000","last_unstale":"2024-09-19T04:14:19.061152+0000","last_undegraded":"2024-09-19T04:14:19.061152+0000","last_fullsized":"2024-09-19T04:14:19.061152+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:44.276683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.949471+0000","last_change":"2024-09-19T04:14:17.070348+0000","last_active":"2024-09-19T04:14:19.949471+0000","last_peered":"2024-09-19T04:14:19.949471+0000","last_clean":"2024-09-19T04:14:19.949471+0000","last_became_active":"2024-09-19T04:14:17.069964+0000","last_became_peered":"2024-09-19T04:14:17.069964+0000","last_unstale":"2024-09-19T04:14:19.949471+0000","last_undegraded":"2024-09-19T04:14:19.949471+0000","last_fullsized":"2024-09-19T04:14:19.949471+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:48:55.313000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.948804+0000","last_change":"2024-09-19T04:14:18.106178+0000","last_active":"2024-09-19T04:14:19.948804+0000","last_peered":"2024-09-19T04:14:19.948804+0000","last_clean":"2024-09-19T04:14:19.948804+0000","last_became_active":"2024-09-19T04:14:18.105436+0000","last_became_peered":"2024-09-19T04:14:18.105436+0000","last_unstale":"2024-09-19T04:14:19.948804+0000","last_undegraded":"2024-09-19T04:14:19.948804+0000","last_fullsized":"2024-09-19T04:14:19.948804+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:17.040576+0000","last_clean_scrub_stamp":"2024-09-19T04:14:17.040576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:00:21.837195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.061272+0000","last_change":"2024-09-19T04:14:17.093374+0000","last_active":"2024-09-19T04:14:19.061272+0000","last_peered":"2024-09-19T04:14:19.061272+0000","last_clean":"2024-09-19T04:14:19.061272+0000","last_became_active":"2024-09-19T04:14:17.092649+0000","last_became_peered":"2024-09-19T04:14:17.092649+0000","last_unstale":"2024-09-19T04:14:19.061272+0000","last_undegraded":"2024-09-19T04:14:19.061272+0000","last_fullsized":"2024-09-19T04:14:19.061272+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:14:16.025045+0000","last_clean_scrub_stamp":"2024-09-19T04:14:16.025045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:03:25.021986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-09-19T04:14:19.060283+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:14:19.060283+0000","last_peered":"2024-09-19T04:14:19.060283+0000","last_clean":"2024-09-19T04:14:19.060283+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:14:19.060283+0000","last_undegraded":"2024-09-19T04:14:19.060283+0000","last_fullsized":"2024-09-19T04:14:19.060283+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32224,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339616,"statfs":{"total":96636764160,"available":96603766784,"internally_reserved":0,"allocated":1056768,"data_stored":785477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27476,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344364,"statfs":{"total":96636764160,"available":96608628736,"internally_reserved":0,"allocated":450560,"data_stored":194009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":193539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542161,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":454656,"data_stored":194031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542161,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27480,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344360,"statfs":{"total":96636764160,"available":96608624640,"internally_reserved":0,"allocated":462848,"data_stored":195073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542162,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31632,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340208,"statfs":{"total":96636764160,"available":96604372992,"internally_reserved":0,"allocated":450560,"data_stored":193497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542162,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":440,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":450560,"data_stored":193517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542162,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1044480,"data_stored":783829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:14:25.844 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:14:25.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T04:14:26.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.008+0000 7f40474ba640 1 Processor -- start 2024-09-19T04:14:26.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.008+0000 7f40474ba640 1 -- start start 2024-09-19T04:14:26.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f40474ba640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400d18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f40474ba640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40400b0da0 0x7f40400ae690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f40474ba640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f40400aebd0 0x7f40400b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f40474ba640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f40400b1f20 con 0x7f40400b0da0 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f40474ba640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f404015b600 con 0x7f40400aebd0 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f40474ba640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f40400b1310 con 0x7f404015b220 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40400b0da0 0x7f40400ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f40400b0da0 0x7f40400ae690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59088/0 (socket says 172.21.15.76:59088) 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 -- 172.21.15.76:0/2441180469 learned_addr learned my addr 172.21.15.76:0/2441180469 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4045256640 1 --2- 172.21.15.76:0/2441180469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400d18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4045a57640 1 --2- 172.21.15.76:0/2441180469 >> v2:172.21.15.116:3300/0 conn(0x7f40400aebd0 0x7f40400b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 -- 172.21.15.76:0/2441180469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 msgr2=0x7f40400d18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 --2- 172.21.15.76:0/2441180469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400d18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 -- 172.21.15.76:0/2441180469 >> v2:172.21.15.116:3300/0 conn(0x7f40400aebd0 msgr2=0x7f40400b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:26.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 --2- 172.21.15.76:0/2441180469 >> v2:172.21.15.116:3300/0 conn(0x7f40400aebd0 0x7f40400b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 -- 172.21.15.76:0/2441180469 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40400b15f0 con 0x7f40400b0da0 2024-09-19T04:14:26.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4044a55640 1 --2- 172.21.15.76:0/2441180469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400ae690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f40400ae460 tx=0x7f403001eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ffa210a1d79b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4037fff640 1 -- 172.21.15.76:0/2441180469 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40300b4070 con 0x7f40400b0da0 2024-09-19T04:14:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4037fff640 1 -- 172.21.15.76:0/2441180469 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40300a7ad0 con 0x7f40400b0da0 2024-09-19T04:14:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.012+0000 7f4037fff640 1 -- 172.21.15.76:0/2441180469 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40300af440 con 0x7f40400b0da0 2024-09-19T04:14:26.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- 172.21.15.76:0/2441180469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 msgr2=0x7f40400ae690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 --2- 172.21.15.76:0/2441180469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400ae690 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f40400ae460 tx=0x7f403001eb30 comp rx=0 tx=0).stop 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- 172.21.15.76:0/2441180469 shutdown_connections 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 --2- 172.21.15.76:0/2441180469 >> v2:172.21.15.116:3300/0 conn(0x7f40400aebd0 0x7f40400b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 --2- 172.21.15.76:0/2441180469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400ae690 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 --2- 172.21.15.76:0/2441180469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400d18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- 172.21.15.76:0/2441180469 >> 172.21.15.76:0/2441180469 conn(0x7f404009aa00 msgr2=0x7f40400b6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- 172.21.15.76:0/2441180469 shutdown_connections 2024-09-19T04:14:26.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- 172.21.15.76:0/2441180469 wait complete. 2024-09-19T04:14:26.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 Processor -- start 2024-09-19T04:14:26.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- start start 2024-09-19T04:14:26.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40400aebd0 0x7f4040153100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400b5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400b5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f404015ee90 con 0x7f40400b0da0 2024-09-19T04:14:26.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f404015ed10 con 0x7f40400aebd0 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f404015f010 con 0x7f404015b220 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400b5930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59098/0 (socket says 172.21.15.76:59098) 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 -- 172.21.15.76:0/604163156 learned_addr learned my addr 172.21.15.76:0/604163156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4045a57640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400b5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4045256640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40400aebd0 0x7f4040153100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 -- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 msgr2=0x7f40400b5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400b5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 -- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40400aebd0 msgr2=0x7f4040153100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40400aebd0 0x7f4040153100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 -- 172.21.15.76:0/604163156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40401f4250 con 0x7f40400b0da0 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4045256640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40400aebd0 0x7f4040153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4044a55640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400b5930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f403001f3d0 tx=0x7f403001b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4035ffb640 1 -- 172.21.15.76:0/604163156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40300be070 con 0x7f40400b0da0 2024-09-19T04:14:26.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4035ffb640 1 -- 172.21.15.76:0/604163156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40300afca0 con 0x7f40400b0da0 2024-09-19T04:14:26.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404009b430 con 0x7f40400b0da0 2024-09-19T04:14:26.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f4035ffb640 1 -- 172.21.15.76:0/604163156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40300b4040 con 0x7f40400b0da0 2024-09-19T04:14:26.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.016+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4040153640 con 0x7f40400b0da0 2024-09-19T04:14:26.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.020+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40400ae690 con 0x7f40400b0da0 2024-09-19T04:14:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.020+0000 7f4035ffb640 1 -- 172.21.15.76:0/604163156 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40300c1070 con 0x7f40400b0da0 2024-09-19T04:14:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.020+0000 7f4035ffb640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40240678c0 0x7f4024069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.020+0000 7f4035ffb640 1 -- 172.21.15.76:0/604163156 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f40301231b0 con 0x7f40400b0da0 2024-09-19T04:14:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.020+0000 7f4045256640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40240678c0 0x7f4024069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.020+0000 7f4045256640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40240678c0 0x7f4024069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.024+0000 7f4035ffb640 1 -- 172.21.15.76:0/604163156 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f40300187e0 con 0x7f40400b0da0 2024-09-19T04:14:26.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.252+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f404015d790 con 0x7f40400b0da0 2024-09-19T04:14:26.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.256+0000 7f4035ffb640 1 -- 172.21.15.76:0/604163156 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f40300f3bf0 con 0x7f40400b0da0 2024-09-19T04:14:26.260 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-19T04:14:26.260 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:14:26.260 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2024-09-19T04:14:26.260 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2024-09-19T04:14:26.260 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T04:14:26.260 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40240678c0 msgr2=0x7f4024069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40240678c0 0x7f4024069d80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f403c0097a0 tx=0x7f403c008040 comp rx=0 tx=0).stop 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 msgr2=0x7f40400b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400b5930 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f403001f3d0 tx=0x7f403001b8d0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 shutdown_connections 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40240678c0 0x7f4024069d80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404015b220 0x7f40400b5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f40400b0da0 0x7f40400b5930 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 --2- 172.21.15.76:0/604163156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40400aebd0 0x7f4040153100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 >> 172.21.15.76:0/604163156 conn(0x7f404009aa00 msgr2=0x7f404006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 shutdown_connections 2024-09-19T04:14:26.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.260+0000 7f40474ba640 1 -- 172.21.15.76:0/604163156 wait complete. 2024-09-19T04:14:26.278 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:26.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.392+0000 7fe796116640 1 Processor -- start 2024-09-19T04:14:26.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.392+0000 7fe796116640 1 -- start start 2024-09-19T04:14:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe796116640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 0x7fe790166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe796116640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7900b3290 con 0x7fe79015bec0 2024-09-19T04:14:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe796116640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7900b3570 con 0x7fe79015c2a0 2024-09-19T04:14:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe796116640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe7900b3850 con 0x7fe79006f010 2024-09-19T04:14:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe79015bec0 0x7fe7900ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38126/0 (socket says 172.21.15.76:38126) 2024-09-19T04:14:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78f7fe640 1 -- 172.21.15.76:0/2313095547 learned_addr learned my addr 172.21.15.76:0/2313095547 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78ffff640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 0x7fe790166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612655543 0 0) 0x7fe7900b3290 con 0x7fe79015bec0 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe778003530 con 0x7fe79015bec0 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 608672694 0 0) 0x7fe778003530 con 0x7fe79015bec0 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 msgr2=0x7fe790166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 0x7fe790166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 >> v1:172.21.15.116:6789/0 conn(0x7fe79015c2a0 legacy=0x7fe7900aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7900b3b30 con 0x7fe79015bec0 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fe780001680 con 0x7fe79015bec0 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 >> v1:172.21.15.76:6789/0 conn(0x7fe79015bec0 legacy=0x7fe7900ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7780039f0 0x7fe778003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe778004810 0x7fe778004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 0x7fe7780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7900b3850 con 0x7fe778004810 2024-09-19T04:14:26.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe77800ba80 con 0x7fe7780039f0 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe77800bc80 con 0x7fe79006f010 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78effd640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe778004810 0x7fe778004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78f7fe640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7780039f0 0x7fe778003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78ffff640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 0x7fe7780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78effd640 1 -- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 msgr2=0x7fe7780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78effd640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe79006f010 0x7fe7780051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78effd640 1 -- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7780039f0 msgr2=0x7fe778003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78effd640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7780039f0 0x7fe778003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78effd640 1 -- 172.21.15.76:0/2313095547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe778003530 con 0x7fe778004810 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.396+0000 7fe78effd640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe778004810 0x7fe778004c90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe79015c6b0 tx=0x7fe78401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ea4fc3888f7276 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840013a0 con 0x7fe778004810 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7840052f0 con 0x7fe778004810 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78e7fc640 1 -- 172.21.15.76:0/2313095547 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7840a4e60 con 0x7fe778004810 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- 172.21.15.76:0/2313095547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe778004810 msgr2=0x7fe778004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 --2- 172.21.15.76:0/2313095547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe778004810 0x7fe778004c90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe79015c6b0 tx=0x7fe78401c3a0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 -- 172.21.15.76:0/2313095547 reap_dead start 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- 172.21.15.76:0/2313095547 shutdown_connections 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- 172.21.15.76:0/2313095547 >> 172.21.15.76:0/2313095547 conn(0x7fe7900b1f90 msgr2=0x7fe7900b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- 172.21.15.76:0/2313095547 shutdown_connections 2024-09-19T04:14:26.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- 172.21.15.76:0/2313095547 wait complete. 2024-09-19T04:14:26.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 Processor -- start 2024-09-19T04:14:26.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- start start 2024-09-19T04:14:26.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe778004810 0x7fe790156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe79009a280 0x7fe79015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79015a980 0x7fe7901e8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7900afc90 con 0x7fe79015a980 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7900b40f0 con 0x7fe79009a280 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe7900af470 con 0x7fe778004810 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79015a980 0x7fe7901e8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79015a980 0x7fe7901e8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59122/0 (socket says 172.21.15.76:59122) 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 -- 172.21.15.76:0/3752025623 learned_addr learned my addr 172.21.15.76:0/3752025623 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78f7fe640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe778004810 0x7fe790156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78effd640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe79009a280 0x7fe79015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 -- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe778004810 msgr2=0x7fe790156b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe778004810 0x7fe790156b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 -- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe79009a280 msgr2=0x7fe79015a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe79009a280 0x7fe79015a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 -- 172.21.15.76:0/3752025623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe79009a700 con 0x7fe79015a980 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe78ffff640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79015a980 0x7fe7901e8e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe79006f460 tx=0x7fe78800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe777fff640 1 -- 172.21.15.76:0/3752025623 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe78801c070 con 0x7fe79015a980 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe777fff640 1 -- 172.21.15.76:0/3752025623 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe788017400 con 0x7fe79015a980 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe79015c850 con 0x7fe79015a980 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe777fff640 1 -- 172.21.15.76:0/3752025623 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7880177f0 con 0x7fe79015a980 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.400+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe79015ce10 con 0x7fe79015a980 2024-09-19T04:14:26.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.404+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7900aed60 con 0x7fe79015a980 2024-09-19T04:14:26.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.404+0000 7fe777fff640 1 -- 172.21.15.76:0/3752025623 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe78801f070 con 0x7fe79015a980 2024-09-19T04:14:26.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.404+0000 7fe777fff640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe77c0678f0 0x7fe77c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:26.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.404+0000 7fe777fff640 1 -- 172.21.15.76:0/3752025623 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fe788085d80 con 0x7fe79015a980 2024-09-19T04:14:26.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.404+0000 7fe78f7fe640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe77c0678f0 0x7fe77c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:26.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.404+0000 7fe78f7fe640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe77c0678f0 0x7fe77c069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe780005280 tx=0x7fe780009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:26.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.408+0000 7fe777fff640 1 -- 172.21.15.76:0/3752025623 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe79009a700 con 0x7fe79015a980 2024-09-19T04:14:26.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.600+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe79006fcd0 con 0x7fe79015a980 2024-09-19T04:14:26.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.600+0000 7fe777fff640 1 -- 172.21.15.76:0/3752025623 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) ==== 94+0+5229 (secure 0 0 0) 0x7fe788012070 con 0x7fe79015a980 2024-09-19T04:14:26.603 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:26.603 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":15,"btime":"2024-09-19T04:14:20:094237+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4854,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:19.421274+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4863},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4863]},"id":2}]} 2024-09-19T04:14:26.604 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 15 2024-09-19T04:14:26.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe77c0678f0 msgr2=0x7fe77c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe77c0678f0 0x7fe77c069db0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe780005280 tx=0x7fe780009040 comp rx=0 tx=0).stop 2024-09-19T04:14:26.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79015a980 msgr2=0x7fe7901e8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:26.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79015a980 0x7fe7901e8e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe79006f460 tx=0x7fe78800f4a0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 shutdown_connections 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe77c0678f0 0x7fe77c069db0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe79015a980 0x7fe7901e8e30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe79009a280 0x7fe79015a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 --2- 172.21.15.76:0/3752025623 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe778004810 0x7fe790156b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 >> 172.21.15.76:0/3752025623 conn(0x7fe7900b1f90 msgr2=0x7fe7900cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 shutdown_connections 2024-09-19T04:14:26.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:26.604+0000 7fe796116640 1 -- 172.21.15.76:0/3752025623 wait complete. 2024-09-19T04:14:27.023 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:14:17.426311+0000', 'modified': '2024-09-19T04:14:19.421274+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4863}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4863': {'gid': 4863, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/1828076892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 1828076892}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 1828076892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4863, 'qdb_cluster': [4863]} 2024-09-19T04:14:27.024 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:14:27.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:14:27.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2f92d640 1 Processor -- start 2024-09-19T04:14:27.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2f92d640 1 -- start start 2024-09-19T04:14:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2f92d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3c28070020 0x7f3c2815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2f92d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c280aede0 con 0x7f3c281520a0 2024-09-19T04:14:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2f92d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c2815fa10 con 0x7f3c2815f1d0 2024-09-19T04:14:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2f92d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3c28152480 con 0x7f3c28070020 2024-09-19T04:14:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2cec8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3c28070020 0x7f3c2815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2cec8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3c28070020 0x7f3c2815ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59126/0 (socket says 172.21.15.76:59126) 2024-09-19T04:14:27.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2cec8640 1 -- 172.21.15.76:0/3578584786 learned_addr learned my addr 172.21.15.76:0/3578584786 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2cec8640 1 -- 172.21.15.76:0/3578584786 >> v1:172.21.15.116:6789/0 conn(0x7f3c2815f1d0 legacy=0x7f3c280adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c1ffff640 1 -- 172.21.15.76:0/3578584786 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 702103567 0 0) 0x7f3c280aede0 con 0x7f3c281520a0 2024-09-19T04:14:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2cec8640 1 -- 172.21.15.76:0/3578584786 >> v1:172.21.15.76:6789/0 conn(0x7f3c281520a0 legacy=0x7f3c2806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.156+0000 7f3c2cec8640 1 -- 172.21.15.76:0/3578584786 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c28074250 con 0x7f3c28070020 2024-09-19T04:14:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/3578584786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c28070020 0x7f3c2815ec90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009870 tx=0x7f3c1801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a7876b84bb8051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c1ffff640 1 -- 172.21.15.76:0/3578584786 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c280aeba0 con 0x7f3c28070020 2024-09-19T04:14:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c1ffff640 1 -- 172.21.15.76:0/3578584786 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c180af070 con 0x7f3c28070020 2024-09-19T04:14:27.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c1ffff640 1 -- 172.21.15.76:0/3578584786 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180aa870 con 0x7f3c28070020 2024-09-19T04:14:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 -- 172.21.15.76:0/3578584786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c28070020 msgr2=0x7f3c2815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/3578584786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c28070020 0x7f3c2815ec90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3c18009870 tx=0x7f3c1801c4f0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 -- 172.21.15.76:0/3578584786 shutdown_connections 2024-09-19T04:14:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/3578584786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c28070020 0x7f3c2815ec90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 -- 172.21.15.76:0/3578584786 >> 172.21.15.76:0/3578584786 conn(0x7f3c280b1f90 msgr2=0x7f3c280b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 -- 172.21.15.76:0/3578584786 shutdown_connections 2024-09-19T04:14:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 -- 172.21.15.76:0/3578584786 wait complete. 2024-09-19T04:14:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.160+0000 7f3c2f92d640 1 Processor -- start 2024-09-19T04:14:27.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 -- start start 2024-09-19T04:14:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c28070020 0x7f3c28146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c281520a0 0x7f3c281697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c2815f1d0 0x7f3c28159f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2815f680 con 0x7f3c2815f1d0 2024-09-19T04:14:27.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28073d30 con 0x7f3c28070020 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3c2806e800 con 0x7f3c281520a0 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c2815f1d0 0x7f3c28159f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c2815f1d0 0x7f3c28159f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59136/0 (socket says 172.21.15.76:59136) 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 -- 172.21.15.76:0/4066677335 learned_addr learned my addr 172.21.15.76:0/4066677335 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c281520a0 0x7f3c281697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c28070020 0x7f3c28146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 -- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c281520a0 msgr2=0x7f3c281697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c281520a0 0x7f3c281697c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 -- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c28070020 msgr2=0x7f3c28146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c28070020 0x7f3c28146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 -- 172.21.15.76:0/4066677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2816a8d0 con 0x7f3c2815f1d0 2024-09-19T04:14:27.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c281520a0 0x7f3c281697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:27.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2deca640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c2815f1d0 0x7f3c28159f80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c7b0 tx=0x7f3c2000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c1dffb640 1 -- 172.21.15.76:0/4066677335 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20004b70 con 0x7f3c2815f1d0 2024-09-19T04:14:27.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c1dffb640 1 -- 172.21.15.76:0/4066677335 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c20004d10 con 0x7f3c2815f1d0 2024-09-19T04:14:27.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2816abb0 con 0x7f3c2815f1d0 2024-09-19T04:14:27.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c1dffb640 1 -- 172.21.15.76:0/4066677335 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2000aa90 con 0x7f3c2815f1d0 2024-09-19T04:14:27.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c28168fc0 con 0x7f3c2815f1d0 2024-09-19T04:14:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c1dffb640 1 -- 172.21.15.76:0/4066677335 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3c2000ac30 con 0x7f3c2815f1d0 2024-09-19T04:14:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c1dffb640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c0c0678c0 0x7f3c0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c1dffb640 1 -- 172.21.15.76:0/4066677335 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f3c20086040 con 0x7f3c2815f1d0 2024-09-19T04:14:27.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.164+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c0c0678c0 0x7f3c0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c0c0678c0 0x7f3c0c069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3c24000f70 tx=0x7f3c24009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 Processor -- start 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- start start 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d3b20 0x56303a8d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 0x56303a889d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a88a290 0x56303a88a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56303a771b60 con 0x56303a88a290 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56303a6d8c70 con 0x56303a8d4440 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56303a67eed0 con 0x56303a8d3b20 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a88a290 0x56303a88a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a88a290 0x56303a88a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59144/0 (socket says 172.21.15.76:59144) 2024-09-19T04:14:27.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 -- 172.21.15.76:0/4091734298 learned_addr learned my addr 172.21.15.76:0/4091734298 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d3b20 0x56303a8d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 -- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d3b20 msgr2=0x56303a8d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d3b20 0x56303a8d3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 0x56303a889d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 -- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 msgr2=0x56303a889d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 0x56303a889d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 -- 172.21.15.76:0/4091734298 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303a690e50 con 0x56303a88a290 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d3b20 0x56303a8d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c2deca640 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a88a290 0x56303a88a670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c20084410 tx=0x7f3c20056a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89088a84fb98b4f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c12ffd640 1 -- 172.21.15.76:0/4091734298 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2005a730 con 0x56303a88a290 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c12ffd640 1 -- 172.21.15.76:0/4091734298 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2005ef60 con 0x56303a88a290 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c12ffd640 1 -- 172.21.15.76:0/4091734298 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20054680 con 0x56303a88a290 2024-09-19T04:14:27.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- 172.21.15.76:0/4091734298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a88a290 msgr2=0x56303a88a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a88a290 0x56303a88a670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3c20084410 tx=0x7f3c20056a80 comp rx=0 tx=0).stop 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- 172.21.15.76:0/4091734298 shutdown_connections 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a88a290 0x56303a88a670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 0x56303a889d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 --2- 172.21.15.76:0/4091734298 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d3b20 0x56303a8d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- 172.21.15.76:0/4091734298 >> 172.21.15.76:0/4091734298 conn(0x56303a890600 msgr2=0x56303a888630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- 172.21.15.76:0/4091734298 shutdown_connections 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- 172.21.15.76:0/4091734298 wait complete. 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 -- start start 2024-09-19T04:14:27.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.168+0000 7f3c329e9000 1 Processor -- start 2024-09-19T04:14:27.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a88ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d4440 0x56303a88f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a894cf0 0x56303a8950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56303a73ab00 con 0x56303a894cf0 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56303a6b3360 con 0x56303a8d3b20 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56303a6b02d0 con 0x56303a8d4440 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 10 client.0 _ll_register_callbacks cb 0x56303a875820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a894cf0 0x56303a8950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a894cf0 0x56303a8950d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59154/0 (socket says 172.21.15.76:59154) 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 -- 172.21.15.76:0/521147614 learned_addr learned my addr 172.21.15.76:0/521147614 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a88ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d4440 0x56303a88f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 -- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d4440 msgr2=0x56303a88f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d4440 0x56303a88f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 -- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 msgr2=0x56303a88ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a88ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 -- 172.21.15.76:0/521147614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303a67f120 con 0x56303a894cf0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2deca640 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a894cf0 0x56303a8950d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c200844c0 tx=0x7f3c20011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cfe17002774ab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 1 -- 172.21.15.76:0/521147614 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20054740 con 0x56303a894cf0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 1 -- 172.21.15.76:0/521147614 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f3c20082040 con 0x56303a894cf0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c2005ef60 con 0x56303a894cf0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 1 -- 172.21.15.76:0/521147614 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f3c2007b050 con 0x56303a894cf0 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 10 client.5073 fetch_fsmap learned FSMap version 15 2024-09-19T04:14:27.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x56303a678450 con 0x56303a894cf0 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 1 -- 172.21.15.76:0/521147614 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c2005ef60 con 0x56303a894cf0 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 1 -- 172.21.15.76:0/521147614 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) ==== 3502+0+0 (secure 0 0 0) 0x7f3c2005ead0 con 0x56303a894cf0 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 10 client.5073 fetch_fsmap finished waiting for FSMap version 15 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 10 client.5073 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 20 client.5073 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a8e2280 0x56303a8e4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 4 client.5073 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56303a672c00 con 0x56303a8e2280 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a8e2280 0x56303a8e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 10 client.5073 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a8e2280 0x56303a8e4670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4988b3f035992566 server_cookie=9c49f8e487c14b68 in_seq=0 out_seq=0 2024-09-19T04:14:27.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 1 -- 172.21.15.76:0/521147614 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56303a672c00 con 0x56303a8e2280 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 10 client.5073 handle_command_reply: tid=0 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 client.5073 shutdown 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a894cf0 msgr2=0x56303a8950d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a894cf0 0x56303a8950d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f3c200844c0 tx=0x7f3c20011000 comp rx=0 tx=0).stop 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 shutdown_connections 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a8e2280 0x56303a8e4670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a894cf0 0x56303a8950d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a8d4440 0x56303a88f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 --2- 172.21.15.76:0/521147614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a88ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c07fff640 0 client.5073 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 >> 172.21.15.76:0/521147614 conn(0x56303a88a290 msgr2=0x56303a888d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 shutdown_connections 2024-09-19T04:14:27.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 1 -- 172.21.15.76:0/521147614 wait complete. 2024-09-19T04:14:27.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.172+0000 7f3c329e9000 20 client.5073 trim_cache size 0 max 16384 2024-09-19T04:14:27.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 Processor -- start 2024-09-19T04:14:27.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- start start 2024-09-19T04:14:27.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8d2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d3b20 0x56303a8d30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 0x56303a8e4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3c28202c50 con 0x56303a8d3b20 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3c2814c770 con 0x56303a8d4440 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3c280cd560 con 0x56303a894cf0 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d3b20 0x56303a8d30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d3b20 0x56303a8d30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59162/0 (socket says 172.21.15.76:59162) 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 -- 172.21.15.76:0/2741117215 learned_addr learned my addr 172.21.15.76:0/2741117215 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8d2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 -- 172.21.15.76:0/2741117215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 msgr2=0x56303a8d2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8d2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 -- 172.21.15.76:0/2741117215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 msgr2=0x56303a8e4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 0x56303a8e4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 -- 172.21.15.76:0/2741117215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56303a65ffd0 con 0x56303a8d3b20 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d3b20 0x56303a8d30d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c1801a120 tx=0x7f3c180a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f45976246bf273 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c127fc640 1 -- 172.21.15.76:0/2741117215 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180aa9d0 con 0x56303a8d3b20 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c127fc640 1 -- 172.21.15.76:0/2741117215 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c180c2070 con 0x56303a8d3b20 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c127fc640 1 -- 172.21.15.76:0/2741117215 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c180aad70 con 0x56303a8d3b20 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- 172.21.15.76:0/2741117215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d3b20 msgr2=0x56303a8d30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d3b20 0x56303a8d30d0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3c1801a120 tx=0x7f3c180a7000 comp rx=0 tx=0).stop 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- 172.21.15.76:0/2741117215 shutdown_connections 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d4440 0x56303a8e4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d3b20 0x56303a8d30d0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2741117215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8d2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- 172.21.15.76:0/2741117215 >> 172.21.15.76:0/2741117215 conn(0x56303a88a290 msgr2=0x56303a88bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- 172.21.15.76:0/2741117215 shutdown_connections 2024-09-19T04:14:27.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- 172.21.15.76:0/2741117215 wait complete. 2024-09-19T04:14:27.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 -- start start 2024-09-19T04:14:27.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.192+0000 7f3c329e9000 1 Processor -- start 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8e7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a902960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d4440 0x56303a902ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56303a6d7f80 con 0x56303a8d4440 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56303a690ca0 con 0x56303a8d3b20 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56303a6dd2a0 con 0x56303a894cf0 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 10 client.0 _ll_register_callbacks cb 0x7f3c20055ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d4440 0x56303a902ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d4440 0x56303a902ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59178/0 (socket says 172.21.15.76:59178) 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 -- 172.21.15.76:0/2320028242 learned_addr learned my addr 172.21.15.76:0/2320028242 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8e7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2cec8640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a902960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 -- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 msgr2=0x56303a8e7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8e7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 -- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 msgr2=0x56303a902960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a902960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 -- 172.21.15.76:0/2320028242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56303a75b750 con 0x56303a8d4440 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:27.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2deca640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d4440 0x56303a902ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c2005ea30 tx=0x7f3c20084340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7969f16a4f2b0d12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 1 -- 172.21.15.76:0/2320028242 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c20006d60 con 0x56303a8d4440 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 1 -- 172.21.15.76:0/2320028242 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f3c200557d0 con 0x56303a8d4440 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c180c2070 con 0x56303a8d4440 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 1 -- 172.21.15.76:0/2320028242 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) ==== 24+0+0 (secure 0 0 0) 0x7f3c2006e3d0 con 0x56303a8d4440 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 10 client.5079 fetch_fsmap learned FSMap version 15 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) -- 0x56303a666a30 con 0x56303a8d4440 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 1 -- 172.21.15.76:0/2320028242 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c180c2070 con 0x56303a8d4440 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 1 -- 172.21.15.76:0/2320028242 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 15) ==== 3502+0+0 (secure 0 0 0) 0x7f3c200520d0 con 0x56303a8d4440 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 10 client.5079 fetch_fsmap finished waiting for FSMap version 15 2024-09-19T04:14:27.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 10 client.5079 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 20 client.5079 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a9033e0 0x56303a9467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 4 client.5079 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "status"}) -- 0x56303a6e63a0 con 0x56303a9033e0 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a9033e0 0x56303a9467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 10 client.5079 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2d6c9640 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a9033e0 0x56303a9467e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7552bd5340cf3be server_cookie=11df9c24978a2409 in_seq=0 out_seq=0 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 1 -- 172.21.15.76:0/2320028242 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56303a6e63a0 con 0x56303a9033e0 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 10 client.5079 handle_command_reply: tid=0 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 client.5079 shutdown 2024-09-19T04:14:27.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d4440 msgr2=0x56303a902ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d4440 0x56303a902ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3c2005ea30 tx=0x7f3c20084340 comp rx=0 tx=0).stop 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 shutdown_connections 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x56303a9033e0 0x56303a9467e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56303a8d4440 0x56303a902ea0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56303a8d3b20 0x56303a902960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 --2- 172.21.15.76:0/2320028242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56303a894cf0 0x56303a8e7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c07fff640 0 client.5079 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 >> 172.21.15.76:0/2320028242 conn(0x56303a88a290 msgr2=0x56303a8d1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 shutdown_connections 2024-09-19T04:14:27.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 1 -- 172.21.15.76:0/2320028242 wait complete. 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c329e9000 20 client.5079 trim_cache size 0 max 16384 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4863, 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:14:27.208 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 9.1446655020000005, 2024-09-19T04:14:27.209 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 15, 2024-09-19T04:14:27.209 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 39, 2024-09-19T04:14:27.209 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 39, 2024-09-19T04:14:27.209 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 16.042727263 2024-09-19T04:14:27.209 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c0c0678c0 msgr2=0x7f3c0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c0c0678c0 0x7f3c0c069d80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3c24000f70 tx=0x7f3c24009040 comp rx=0 tx=0).stop 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c2815f1d0 msgr2=0x7f3c28159f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c2815f1d0 0x7f3c28159f80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3c2000c7b0 tx=0x7f3c2000cc70 comp rx=0 tx=0).stop 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 shutdown_connections 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c0c0678c0 0x7f3c0c069d80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c2815f1d0 0x7f3c28159f80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c281520a0 0x7f3c281697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 --2- 172.21.15.76:0/4066677335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c28070020 0x7f3c28146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 >> 172.21.15.76:0/4066677335 conn(0x7f3c280b1f90 msgr2=0x7f3c28071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 shutdown_connections 2024-09-19T04:14:27.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.196+0000 7f3c2f92d640 1 -- 172.21.15.76:0/4066677335 wait complete. 2024-09-19T04:14:27.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:14:27.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.324+0000 7f08cb6d2640 1 Processor -- start 2024-09-19T04:14:27.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.324+0000 7f08cb6d2640 1 -- start start 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08cb6d2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40d3060 0x7f08c40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08cb6d2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f08c40cf450 0x7f08c406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08cb6d2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08c40d0300 con 0x7f08c4161a60 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08cb6d2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f08c40b0f60 con 0x7f08c40cf450 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08cb6d2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f08c40cfd40 con 0x7f08c40d3060 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c8c6d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40d3060 0x7f08c40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c9c6f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f08c40cf450 0x7f08c406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c8c6d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40d3060 0x7f08c40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:51832/0 (socket says 172.21.15.76:51832) 2024-09-19T04:14:27.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c8c6d640 1 -- 172.21.15.76:0/3431305897 learned_addr learned my addr 172.21.15.76:0/3431305897 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c8c6d640 1 -- 172.21.15.76:0/3431305897 >> v2:172.21.15.116:3300/0 conn(0x7f08c40cf450 msgr2=0x7f08c406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:27.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c8c6d640 1 --2- 172.21.15.76:0/3431305897 >> v2:172.21.15.116:3300/0 conn(0x7f08c40cf450 0x7f08c406d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c8c6d640 1 -- 172.21.15.76:0/3431305897 >> v1:172.21.15.116:6789/0 conn(0x7f08c4161a60 legacy=0x7f08c40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08b7fff640 1 -- 172.21.15.76:0/3431305897 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188844362 0 0) 0x7f08bc0014a0 con 0x7f08c4161a60 2024-09-19T04:14:27.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.328+0000 7f08c8c6d640 1 -- 172.21.15.76:0/3431305897 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c40d0020 con 0x7f08c40d3060 2024-09-19T04:14:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c8c6d640 1 --2- 172.21.15.76:0/3431305897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40d3060 0x7f08c40cef10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76bbf9b4831f4db6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08b7fff640 1 -- 172.21.15.76:0/3431305897 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80aa600 con 0x7f08c40d3060 2024-09-19T04:14:27.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08b7fff640 1 -- 172.21.15.76:0/3431305897 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08b80af070 con 0x7f08c40d3060 2024-09-19T04:14:27.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08b7fff640 1 -- 172.21.15.76:0/3431305897 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b80aa9c0 con 0x7f08c40d3060 2024-09-19T04:14:27.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3431305897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40d3060 msgr2=0x7f08c40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3431305897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40d3060 0x7f08c40cef10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b801c4a0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3431305897 shutdown_connections 2024-09-19T04:14:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3431305897 >> v2:172.21.15.116:3300/0 conn(0x7f08c40cf450 0x7f08c406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3431305897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40d3060 0x7f08c40cef10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3431305897 >> 172.21.15.76:0/3431305897 conn(0x7f08c408bf90 msgr2=0x7f08c409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3431305897 shutdown_connections 2024-09-19T04:14:27.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3431305897 wait complete. 2024-09-19T04:14:27.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 Processor -- start 2024-09-19T04:14:27.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- start start 2024-09-19T04:14:27.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40cf450 0x7f08c40ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c40d3060 0x7f08c41fac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c4161a60 0x7f08c41fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f08c416af20 con 0x7f08c4161a60 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f08c406e610 con 0x7f08c40d3060 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08cb6d2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f08c416b0a0 con 0x7f08c40cf450 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c4161a60 0x7f08c41fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c4161a60 0x7f08c41fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59192/0 (socket says 172.21.15.76:59192) 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 -- 172.21.15.76:0/3929854725 learned_addr learned my addr 172.21.15.76:0/3929854725 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c946e640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40cf450 0x7f08c40ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c8c6d640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c40d3060 0x7f08c41fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 -- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40cf450 msgr2=0x7f08c40ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40cf450 0x7f08c40ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 -- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c40d3060 msgr2=0x7f08c41fac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c40d3060 0x7f08c41fac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.332+0000 7f08c9c6f640 1 -- 172.21.15.76:0/3929854725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c408c4b0 con 0x7f08c4161a60 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08c9c6f640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c4161a60 0x7f08c41fb190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e7f0 tx=0x7f08c000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08b5ffb640 1 -- 172.21.15.76:0/3929854725 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c0017070 con 0x7f08c4161a60 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08b5ffb640 1 -- 172.21.15.76:0/3929854725 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08c000c940 con 0x7f08c4161a60 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c40b2ab0 con 0x7f08c4161a60 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08b5ffb640 1 -- 172.21.15.76:0/3929854725 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08c000eea0 con 0x7f08c4161a60 2024-09-19T04:14:27.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c40b73f0 con 0x7f08c4161a60 2024-09-19T04:14:27.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c406d6b0 con 0x7f08c4161a60 2024-09-19T04:14:27.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08b5ffb640 1 -- 172.21.15.76:0/3929854725 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f08c001a070 con 0x7f08c4161a60 2024-09-19T04:14:27.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08b5ffb640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f08a80678c0 0x7f08a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:27.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08b5ffb640 1 -- 172.21.15.76:0/3929854725 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f08c0085bd0 con 0x7f08c4161a60 2024-09-19T04:14:27.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08c946e640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f08a80678c0 0x7f08a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:27.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.336+0000 7f08c946e640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f08a80678c0 0x7f08a8069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0012f0 tx=0x7f08bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:27.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.340+0000 7f08b5ffb640 1 -- 172.21.15.76:0/3929854725 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f08c0055fc0 con 0x7f08c4161a60 2024-09-19T04:14:27.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.524+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f08c40b71a0 con 0x7f08c4161a60 2024-09-19T04:14:27.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.524+0000 7f08b5ffb640 1 -- 172.21.15.76:0/3929854725 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) ==== 53+0+73 (secure 0 0 0) 0x7f08c005a2c0 con 0x7f08c4161a60 2024-09-19T04:14:27.528 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f08a80678c0 msgr2=0x7f08a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f08a80678c0 0x7f08a8069d80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0012f0 tx=0x7f08bc009040 comp rx=0 tx=0).stop 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c4161a60 msgr2=0x7f08c41fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c4161a60 0x7f08c41fb190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f08c000e7f0 tx=0x7f08c000ecb0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 shutdown_connections 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f08a80678c0 0x7f08a8069d80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f08c4161a60 0x7f08c41fb190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08c40d3060 0x7f08c41fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 --2- 172.21.15.76:0/3929854725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f08c40cf450 0x7f08c40ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 >> 172.21.15.76:0/3929854725 conn(0x7f08c408bf90 msgr2=0x7f08c40b4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 shutdown_connections 2024-09-19T04:14:27.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27.528+0000 7f08cb6d2640 1 -- 172.21.15.76:0/3929854725 wait complete. 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:14:27.546 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:14:27.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:27.547 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:14:27.552 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:14:27.553 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42453sec preferred_lft 42453sec 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.554 INFO:teuthology.orchestra.run.smithi076.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T04:14:27.555 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:14:27.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:27.556 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:14:27.607 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:14:27.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:27.609 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:14:27.662 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:14:27.662 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:14:27.662 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:14:27.662 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:14:27.662 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:14:27.663 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:14:27.663 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:14:27.663 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:14:27.663 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:14:27.663 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:14:27.663 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:14:27.663 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:14:27.663 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:14:27.663 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:14:27.663 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:14:27.664 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:14:27.664 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:14:27.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:14:27.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:27.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:27.787 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:14:27.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T04:14:27.834 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:14:27.834 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:14:27.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:27.835 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:14:27.890 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T04:14:27.891 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:14:27.891 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:27.938 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:14:27.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:14:27.938 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:27.990 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:14:28.048 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:14:28.095 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:14:28.096 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:14:28.097 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:14:28.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:28.098 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:14:28.146 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:14:28.146 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T04:14:28.191 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:14:28.205 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:14:28.258 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:14:28.259 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:14:28.260 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:14:28.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:28.261 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:14:28.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-09-19T04:14:28.268+0000 7f2c801e4580 -1 init, newargv = 0x564b63065ef0 newargc=15 2024-09-19T04:14:28.270 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[38270]: starting ceph client 2024-09-19T04:14:28.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[38270]: starting fuse 2024-09-19T04:14:29.376 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:14:29.392 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:14:29.439 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:14:29.440 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:14:29.441 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:29.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:29.442 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:14:29.490 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:14:29.490 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:14:29.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:14:29.491 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-09-19T04:14:29.492 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:14:29.615 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.38270.asok 2024-09-19T04:14:29.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:29.627 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38270.asok 2024-09-19T04:14:29.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:29.628 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38270.asok status 2024-09-19T04:14:29.780 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:29.780 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:14:29.780 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:14:29.780 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:14:29.780 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-09-19T04:14:29.780 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:14:29.780 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "38270", 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5091, 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5091 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:29.781 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 4064285947 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 4064285947 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:14:29.782 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.5091 192.168.0.1:0/4064285947", 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/4064285947", 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 15, 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 39, 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:14:29.783 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:29.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:29.794 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:14:29.794 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:29.801 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:14:29.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:29.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:29.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:14:29.863 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-19T04:14:30.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.032+0000 7f0258b01640 1 Processor -- start 2024-09-19T04:14:30.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.032+0000 7f0258b01640 1 -- start start 2024-09-19T04:14:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0258b01640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02540b39a0 0x7f025406df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0258b01640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025406ec60 con 0x7f02541587a0 2024-09-19T04:14:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0258b01640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f025406ef40 con 0x7f025406e4b0 2024-09-19T04:14:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0258b01640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f025406f870 con 0x7f02540b39a0 2024-09-19T04:14:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0251d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f02540b39a0 0x7f025406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f025259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f02541587a0 0x7f02540b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38140/0 (socket says 172.21.15.76:38140) 2024-09-19T04:14:30.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f025259c640 1 -- 172.21.15.76:0/4148356188 learned_addr learned my addr 172.21.15.76:0/4148356188 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:30.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0251d9b640 1 -- 172.21.15.76:0/4148356188 >> v1:172.21.15.116:6789/0 conn(0x7f025406e4b0 legacy=0x7f0254072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:30.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0251d9b640 1 -- 172.21.15.76:0/4148356188 >> v1:172.21.15.76:6789/0 conn(0x7f02541587a0 legacy=0x7f02540b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0251d9b640 1 -- 172.21.15.76:0/4148356188 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025406f220 con 0x7f02540b39a0 2024-09-19T04:14:30.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.036+0000 7f0251d9b640 1 --2- 172.21.15.76:0/4148356188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025406df70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f025406f9f0 tx=0x7f024801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af33a30e91604e98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:30.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f025159a640 1 -- 172.21.15.76:0/4148356188 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480b1070 con 0x7f02540b39a0 2024-09-19T04:14:30.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f025159a640 1 -- 172.21.15.76:0/4148356188 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02480ac620 con 0x7f02540b39a0 2024-09-19T04:14:30.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f025159a640 1 -- 172.21.15.76:0/4148356188 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480ac9f0 con 0x7f02540b39a0 2024-09-19T04:14:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- 172.21.15.76:0/4148356188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 msgr2=0x7f025406df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 --2- 172.21.15.76:0/4148356188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025406df70 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f025406f9f0 tx=0x7f024801c430 comp rx=0 tx=0).stop 2024-09-19T04:14:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- 172.21.15.76:0/4148356188 shutdown_connections 2024-09-19T04:14:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 --2- 172.21.15.76:0/4148356188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025406df70 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- 172.21.15.76:0/4148356188 >> 172.21.15.76:0/4148356188 conn(0x7f02540b1f90 msgr2=0x7f02540b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- 172.21.15.76:0/4148356188 shutdown_connections 2024-09-19T04:14:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- 172.21.15.76:0/4148356188 wait complete. 2024-09-19T04:14:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 Processor -- start 2024-09-19T04:14:30.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- start start 2024-09-19T04:14:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f025406e4b0 0x7f02540992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025409bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02541587a0 0x7f02540adba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0254073200 con 0x7f02540b39a0 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0254073080 con 0x7f02541587a0 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0254073380 con 0x7f025406e4b0 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025409bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025409bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59208/0 (socket says 172.21.15.76:59208) 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 -- 172.21.15.76:0/4131360562 learned_addr learned my addr 172.21.15.76:0/4131360562 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f025259c640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f025406e4b0 0x7f02540992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 -- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f025406e4b0 msgr2=0x7f02540992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f025406e4b0 0x7f02540992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0252d9d640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02541587a0 0x7f02540adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 -- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02541587a0 msgr2=0x7f02540adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02541587a0 0x7f02540adba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 -- 172.21.15.76:0/4131360562 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025409a420 con 0x7f02540b39a0 2024-09-19T04:14:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f025259c640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f025406e4b0 0x7f02540992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0251d9b640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025409bff0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f02480af040 tx=0x7f02480191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0242ffd640 1 -- 172.21.15.76:0/4131360562 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480b9070 con 0x7f02540b39a0 2024-09-19T04:14:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0242ffd640 1 -- 172.21.15.76:0/4131360562 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02480b5610 con 0x7f02540b39a0 2024-09-19T04:14:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02540997e0 con 0x7f02540b39a0 2024-09-19T04:14:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0242ffd640 1 -- 172.21.15.76:0/4131360562 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02480b1040 con 0x7f02540b39a0 2024-09-19T04:14:30.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.040+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0254099c50 con 0x7f02540b39a0 2024-09-19T04:14:30.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.044+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0254147d20 con 0x7f02540b39a0 2024-09-19T04:14:30.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.044+0000 7f0242ffd640 1 -- 172.21.15.76:0/4131360562 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f02480bc070 con 0x7f02540b39a0 2024-09-19T04:14:30.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.044+0000 7f0242ffd640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02340678c0 0x7f0234069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.044+0000 7f0242ffd640 1 -- 172.21.15.76:0/4131360562 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f024811fd90 con 0x7f02540b39a0 2024-09-19T04:14:30.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.044+0000 7f025259c640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02340678c0 0x7f0234069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.044+0000 7f025259c640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02340678c0 0x7f0234069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0254074d80 tx=0x7f0244009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:30.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.048+0000 7f0242ffd640 1 -- 172.21.15.76:0/4131360562 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0248016050 con 0x7f02540b39a0 2024-09-19T04:14:30.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.240+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f02540aeb30 con 0x7f02540b39a0 2024-09-19T04:14:30.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.724+0000 7f0242ffd640 1 -- 172.21.15.76:0/4131360562 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v16) ==== 105+0+0 (secure 0 0 0) 0x7f02480f0750 con 0x7f02540b39a0 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02340678c0 msgr2=0x7f0234069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02340678c0 0x7f0234069d80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f0254074d80 tx=0x7f0244009040 comp rx=0 tx=0).stop 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 msgr2=0x7f025409bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025409bff0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f02480af040 tx=0x7f02480191d0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 shutdown_connections 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f02340678c0 0x7f0234069d80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f02541587a0 0x7f02540adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f02540b39a0 0x7f025409bff0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 --2- 172.21.15.76:0/4131360562 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f025406e4b0 0x7f02540992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 >> 172.21.15.76:0/4131360562 conn(0x7f02540b1f90 msgr2=0x7f02541609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:30.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 shutdown_connections 2024-09-19T04:14:30.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.732+0000 7f0258b01640 1 -- 172.21.15.76:0/4131360562 wait complete. 2024-09-19T04:14:30.759 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:30.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1dab07640 1 Processor -- start 2024-09-19T04:14:30.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1dab07640 1 -- start start 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1dab07640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1d415c2a0 0x7fa1d40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1dab07640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa1d40af2a0 0x7fa1d406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1dab07640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1d40b3290 con 0x7fa1d415bec0 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1dab07640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d40b3570 con 0x7fa1d415c2a0 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1dab07640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa1d40b3850 con 0x7fa1d40af2a0 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1d415c2a0 0x7fa1d40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1d415c2a0 0x7fa1d40aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59216/0 (socket says 172.21.15.76:59216) 2024-09-19T04:14:30.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 -- 172.21.15.76:0/1943098038 learned_addr learned my addr 172.21.15.76:0/1943098038 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:30.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d90a4640 1 --2- 172.21.15.76:0/1943098038 >> v2:172.21.15.116:3300/0 conn(0x7fa1d40af2a0 0x7fa1d406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 -- 172.21.15.76:0/1943098038 >> v2:172.21.15.116:3300/0 conn(0x7fa1d40af2a0 msgr2=0x7fa1d406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:30.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 --2- 172.21.15.76:0/1943098038 >> v2:172.21.15.116:3300/0 conn(0x7fa1d40af2a0 0x7fa1d406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 -- 172.21.15.76:0/1943098038 >> v1:172.21.15.116:6789/0 conn(0x7fa1d415bec0 legacy=0x7fa1d40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:30.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 -- 172.21.15.76:0/1943098038 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d40b3b30 con 0x7fa1d415c2a0 2024-09-19T04:14:30.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.876+0000 7fa1d3fff640 1 --2- 172.21.15.76:0/1943098038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d40aed60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40b2fb0 tx=0x7fa1c001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba5dbfb9d865884 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:30.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d37fe640 1 -- 172.21.15.76:0/1943098038 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00aa610 con 0x7fa1d415c2a0 2024-09-19T04:14:30.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d37fe640 1 -- 172.21.15.76:0/1943098038 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1c00af070 con 0x7fa1d415c2a0 2024-09-19T04:14:30.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d37fe640 1 -- 172.21.15.76:0/1943098038 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c00aa9d0 con 0x7fa1d415c2a0 2024-09-19T04:14:30.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- 172.21.15.76:0/1943098038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 msgr2=0x7fa1d40aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1943098038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d40aed60 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa1d40b2fb0 tx=0x7fa1c001c4b0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- 172.21.15.76:0/1943098038 shutdown_connections 2024-09-19T04:14:30.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1943098038 >> v2:172.21.15.116:3300/0 conn(0x7fa1d40af2a0 0x7fa1d406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1943098038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d40aed60 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- 172.21.15.76:0/1943098038 >> 172.21.15.76:0/1943098038 conn(0x7fa1d40b1f90 msgr2=0x7fa1d40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:30.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- 172.21.15.76:0/1943098038 shutdown_connections 2024-09-19T04:14:30.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- 172.21.15.76:0/1943098038 wait complete. 2024-09-19T04:14:30.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 Processor -- start 2024-09-19T04:14:30.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- start start 2024-09-19T04:14:30.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d40af2a0 0x7fa1d4149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1d415bec0 0x7fa1d4157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d406ed50 con 0x7fa1d415c2a0 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1d40af6b0 con 0x7fa1d40af2a0 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa1d406eed0 con 0x7fa1d415bec0 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d4157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59228/0 (socket says 172.21.15.76:59228) 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 -- 172.21.15.76:0/1177500242 learned_addr learned my addr 172.21.15.76:0/1177500242 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d3fff640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1d415bec0 0x7fa1d4157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d88a3640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d40af2a0 0x7fa1d4149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 -- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1d415bec0 msgr2=0x7fa1d4157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1d415bec0 0x7fa1d4157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 -- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d40af2a0 msgr2=0x7fa1d4149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d40af2a0 0x7fa1d4149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 -- 172.21.15.76:0/1177500242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1d415f520 con 0x7fa1d415c2a0 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d90a4640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d4157a80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc007e70 tx=0x7fa1cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d17fa640 1 -- 172.21.15.76:0/1177500242 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc014620 con 0x7fa1d415c2a0 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d17fa640 1 -- 172.21.15.76:0/1177500242 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1cc019070 con 0x7fa1d415c2a0 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1d415ea40 con 0x7fa1d415c2a0 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1d17fa640 1 -- 172.21.15.76:0/1177500242 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc014a20 con 0x7fa1d415c2a0 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.880+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1d414a0e0 con 0x7fa1d415c2a0 2024-09-19T04:14:30.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.884+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1d406f1d0 con 0x7fa1d415c2a0 2024-09-19T04:14:30.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.884+0000 7fa1d17fa640 1 -- 172.21.15.76:0/1177500242 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa1cc014bc0 con 0x7fa1d415c2a0 2024-09-19T04:14:30.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.884+0000 7fa1d17fa640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1b80678f0 0x7fa1b8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:30.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.884+0000 7fa1d17fa640 1 -- 172.21.15.76:0/1177500242 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa1cc0866a0 con 0x7fa1d415c2a0 2024-09-19T04:14:30.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.884+0000 7fa1d88a3640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1b80678f0 0x7fa1b8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:30.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.884+0000 7fa1d88a3640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1b80678f0 0x7fa1b8069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8004e30 tx=0x7fa1c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:30.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:30.888+0000 7fa1d17fa640 1 -- 172.21.15.76:0/1177500242 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa1cc057050 con 0x7fa1d415c2a0 2024-09-19T04:14:31.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.080+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1d4075040 con 0x7fa1d415c2a0 2024-09-19T04:14:31.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.080+0000 7fa1d17fa640 1 -- 172.21.15.76:0/1177500242 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5245 (secure 0 0 0) 0x7fa1cc056950 con 0x7fa1d415c2a0 2024-09-19T04:14:31.083 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:31.083 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":16,"btime":"2024-09-19T04:14:30:726998+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:30.726942+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":4863,"mds_1":4854},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4854":{"gid":4854,"name":"a","rank":1,"incarnation":16,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4863]},"id":2}]} 2024-09-19T04:14:31.084 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 16 2024-09-19T04:14:31.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1b80678f0 msgr2=0x7fa1b8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1b80678f0 0x7fa1b8069db0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa1c8004e30 tx=0x7fa1c8009040 comp rx=0 tx=0).stop 2024-09-19T04:14:31.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 msgr2=0x7fa1d4157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:31.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d4157a80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc007e70 tx=0x7fa1cc00e4b0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 shutdown_connections 2024-09-19T04:14:31.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1b80678f0 0x7fa1b8069db0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1d415c2a0 0x7fa1d4157a80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1d415bec0 0x7fa1d4157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 --2- 172.21.15.76:0/1177500242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1d40af2a0 0x7fa1d4149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:31.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 >> 172.21.15.76:0/1177500242 conn(0x7fa1d40b1f90 msgr2=0x7fa1d414f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:31.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 shutdown_connections 2024-09-19T04:14:31.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:31.084+0000 7fa1dab07640 1 -- 172.21.15.76:0/1177500242 wait complete. 2024-09-19T04:14:31.505 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:14:17.426311+0000', 'modified': '2024-09-19T04:14:30.726942+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 4863, 'mds_1': 4854}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4854': {'gid': 4854, 'name': 'a', 'rank': 1, 'incarnation': 16, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.76:6835/2829561253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 2829561253}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 2829561253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4863': {'gid': 4863, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/1828076892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 1828076892}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 1828076892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4863, 'qdb_cluster': [4863]} 2024-09-19T04:14:31.505 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4854:up:creating 2024-09-19T04:14:32.018 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:14:32.016+0000 7f3b899a5640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T04:14:32.506 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:32.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37ff51b640 1 Processor -- start 2024-09-19T04:14:32.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37ff51b640 1 -- start start 2024-09-19T04:14:32.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37ff51b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f37f80d4080 0x7f37f80d44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37ff51b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f80d2650 con 0x7f37f80cf900 2024-09-19T04:14:32.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37ff51b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37f80d1690 con 0x7f37f80d2ef0 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37ff51b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f37f80d1c50 con 0x7f37f80d4080 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37fdab8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f37f80d4080 0x7f37f80d44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37fcab6640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f37f80cf900 0x7f37f80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53520/0 (socket says 172.21.15.76:53520) 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.632+0000 7f37fcab6640 1 -- 172.21.15.76:0/723519126 learned_addr learned my addr 172.21.15.76:0/723519126 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37effff640 1 -- 172.21.15.76:0/723519126 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3069103332 0 0) 0x7f37f80d2650 con 0x7f37f80cf900 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37effff640 1 -- 172.21.15.76:0/723519126 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37e4003530 con 0x7f37f80cf900 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37fdab8640 1 -- 172.21.15.76:0/723519126 >> v1:172.21.15.116:6789/0 conn(0x7f37f80d2ef0 legacy=0x7f37f80d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37fdab8640 1 -- 172.21.15.76:0/723519126 >> v1:172.21.15.76:6789/0 conn(0x7f37f80cf900 legacy=0x7f37f80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37fdab8640 1 -- 172.21.15.76:0/723519126 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f80d1970 con 0x7f37f80d4080 2024-09-19T04:14:32.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37fdab8640 1 --2- 172.21.15.76:0/723519126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d4080 0x7f37f80d44d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009d90 tx=0x7f37f001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d1905e2977e82e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37effff640 1 -- 172.21.15.76:0/723519126 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f00b2070 con 0x7f37f80d4080 2024-09-19T04:14:32.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37effff640 1 -- 172.21.15.76:0/723519126 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f00a57f0 con 0x7f37f80d4080 2024-09-19T04:14:32.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37effff640 1 -- 172.21.15.76:0/723519126 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f00a5bc0 con 0x7f37f80d4080 2024-09-19T04:14:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37ff51b640 1 -- 172.21.15.76:0/723519126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d4080 msgr2=0x7f37f80d44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37ff51b640 1 --2- 172.21.15.76:0/723519126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d4080 0x7f37f80d44d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009d90 tx=0x7f37f001c930 comp rx=0 tx=0).stop 2024-09-19T04:14:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37ff51b640 1 -- 172.21.15.76:0/723519126 shutdown_connections 2024-09-19T04:14:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37ff51b640 1 --2- 172.21.15.76:0/723519126 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d4080 0x7f37f80d44d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37ff51b640 1 -- 172.21.15.76:0/723519126 >> 172.21.15.76:0/723519126 conn(0x7f37f809d710 msgr2=0x7f37f808bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37ff51b640 1 -- 172.21.15.76:0/723519126 shutdown_connections 2024-09-19T04:14:32.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.636+0000 7f37ff51b640 1 -- 172.21.15.76:0/723519126 wait complete. 2024-09-19T04:14:32.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 Processor -- start 2024-09-19T04:14:32.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 -- start start 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37f80cf900 0x7f37f8153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d2ef0 0x7f37f80b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37f80d4080 0x7f37f80b9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f37f80d1170 con 0x7f37f80d2ef0 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f37f80cf470 con 0x7f37f80d4080 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f37f80d12f0 con 0x7f37f80cf900 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d2ef0 0x7f37f80b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d2ef0 0x7f37f80b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56324/0 (socket says 172.21.15.76:56324) 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 -- 172.21.15.76:0/1106063486 learned_addr learned my addr 172.21.15.76:0/1106063486 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fd2b7640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37f80cf900 0x7f37f8153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fdab8640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37f80d4080 0x7f37f80b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 -- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37f80cf900 msgr2=0x7f37f8153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37f80cf900 0x7f37f8153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 -- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37f80d4080 msgr2=0x7f37f80b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37f80d4080 0x7f37f80b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 -- 172.21.15.76:0/1106063486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f80b4030 con 0x7f37f80d2ef0 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37fcab6640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d2ef0 0x7f37f80b4df0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c450 tx=0x7f37e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37edffb640 1 -- 172.21.15.76:0/1106063486 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8017070 con 0x7f37f80d2ef0 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37edffb640 1 -- 172.21.15.76:0/1106063486 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e8005cf0 con 0x7f37f80d2ef0 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f80b4310 con 0x7f37f80d2ef0 2024-09-19T04:14:32.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37edffb640 1 -- 172.21.15.76:0/1106063486 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e8007290 con 0x7f37f80d2ef0 2024-09-19T04:14:32.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f80b48d0 con 0x7f37f80d2ef0 2024-09-19T04:14:32.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f80b4ad0 con 0x7f37f80d2ef0 2024-09-19T04:14:32.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.640+0000 7f37edffb640 1 -- 172.21.15.76:0/1106063486 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f37e801a070 con 0x7f37f80d2ef0 2024-09-19T04:14:32.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.644+0000 7f37edffb640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f37dc0678c0 0x7f37dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:32.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.644+0000 7f37edffb640 1 -- 172.21.15.76:0/1106063486 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f37e8085240 con 0x7f37f80d2ef0 2024-09-19T04:14:32.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.644+0000 7f37fd2b7640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f37dc0678c0 0x7f37dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:32.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.644+0000 7f37fd2b7640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f37dc0678c0 0x7f37dc069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37f815b3e0 tx=0x7f37f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:32.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.648+0000 7f37edffb640 1 -- 172.21.15.76:0/1106063486 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f37e8055bf0 con 0x7f37f80d2ef0 2024-09-19T04:14:32.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.844+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f80d44d0 con 0x7f37f80d2ef0 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37edffb640 1 -- 172.21.15.76:0/1106063486 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5267 (secure 0 0 0) 0x7f37e80554f0 con 0x7f37f80d2ef0 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":19,"btime":"2024-09-19T04:14:32:739642+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:32.739633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4863,"mds_1":4854,"mds_2":4666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4666":{"gid":4666,"name":"d","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4854":{"gid":4854,"name":"a","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4666,4854,4863]},"id":2}]} 2024-09-19T04:14:32.850 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 19 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f37dc0678c0 msgr2=0x7f37dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f37dc0678c0 0x7f37dc069d80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f37f815b3e0 tx=0x7f37f4009040 comp rx=0 tx=0).stop 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d2ef0 msgr2=0x7f37f80b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d2ef0 0x7f37f80b4df0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c450 tx=0x7f37e800c910 comp rx=0 tx=0).stop 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 shutdown_connections 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f37dc0678c0 0x7f37dc069d80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37f80d4080 0x7f37f80b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f37f80d2ef0 0x7f37f80b4df0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 --2- 172.21.15.76:0/1106063486 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f37f80cf900 0x7f37f8153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:32.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 >> 172.21.15.76:0/1106063486 conn(0x7f37f809d710 msgr2=0x7f37f81698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 shutdown_connections 2024-09-19T04:14:32.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:32.848+0000 7f37ff51b640 1 -- 172.21.15.76:0/1106063486 wait complete. 2024-09-19T04:14:33.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:14:17.426311+0000', 'modified': '2024-09-19T04:14:32.739633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 4863, 'mds_1': 4854, 'mds_2': 4666}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4666': {'gid': 4666, 'name': 'd', 'rank': 2, 'incarnation': 18, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.116:6833/2447208752', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 2447208752}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 2447208752}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4854': {'gid': 4854, 'name': 'a', 'rank': 1, 'incarnation': 16, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.76:6835/2829561253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 2829561253}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 2829561253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_4863': {'gid': 4863, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/1828076892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 1828076892}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 1828076892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4863, 'qdb_cluster': [4666, 4854, 4863]} 2024-09-19T04:14:33.272 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:14:33.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T04:14:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95ab8ea640 1 Processor -- start 2024-09-19T04:14:33.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95ab8ea640 1 -- start start 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95ab8ea640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 0x7f95a40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95ab8ea640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a40b3290 con 0x7f95a40af2a0 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95ab8ea640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95a40b3570 con 0x7f95a415bec0 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95ab8ea640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f95a40b3850 con 0x7f95a415c2a0 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9e87640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f95a40af2a0 0x7f95a406e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:53524/0 (socket says 172.21.15.76:53524) 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9e87640 1 -- 172.21.15.76:0/2659456747 learned_addr learned my addr 172.21.15.76:0/2659456747 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a8e85640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 0x7f95a40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249444729 0 0) 0x7f95a40b3290 con 0x7f95a40af2a0 2024-09-19T04:14:33.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9590003530 con 0x7f95a40af2a0 2024-09-19T04:14:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1855533112 0 0) 0x7f9590003530 con 0x7f95a40af2a0 2024-09-19T04:14:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 msgr2=0x7f95a40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 0x7f95a40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 >> v1:172.21.15.116:6789/0 conn(0x7f95a415bec0 legacy=0x7f95a40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a40b3b30 con 0x7f95a40af2a0 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f959c001a00 con 0x7f95a40af2a0 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 >> v1:172.21.15.76:6789/0 conn(0x7f95a40af2a0 legacy=0x7f95a406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95900039f0 0x7f9590003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9590004810 0x7f9590004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 0x7f95900051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95a40b3850 con 0x7f95900039f0 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f959000ba80 con 0x7f9590004810 2024-09-19T04:14:33.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f959000bc80 con 0x7f95a415c2a0 2024-09-19T04:14:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9686640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95900039f0 0x7f9590003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a8e85640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9590004810 0x7f9590004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9e87640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 0x7f95900051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9686640 1 -- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 msgr2=0x7f95900051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9686640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 0x7f95900051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9686640 1 -- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9590004810 msgr2=0x7f9590004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9686640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9590004810 0x7f9590004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9686640 1 -- 172.21.15.76:0/2659456747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9590003530 con 0x7f95900039f0 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a8e85640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9590004810 0x7f9590004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.404+0000 7f95a9e87640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95a415c2a0 0x7f95900051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95a9686640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95900039f0 0x7f9590003e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9598001750 tx=0x7f959801c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ff0e63c01861a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980013a0 con 0x7f95900039f0 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95980a4e50 con 0x7f95900039f0 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f9597fff640 1 -- 172.21.15.76:0/2659456747 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598003600 con 0x7f95900039f0 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95ab8ea640 1 -- 172.21.15.76:0/2659456747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95900039f0 msgr2=0x7f9590003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95ab8ea640 1 --2- 172.21.15.76:0/2659456747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95900039f0 0x7f9590003e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9598001750 tx=0x7f959801c390 comp rx=0 tx=0).stop 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95a9e87640 1 -- 172.21.15.76:0/2659456747 reap_dead start 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95ab8ea640 1 -- 172.21.15.76:0/2659456747 shutdown_connections 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95ab8ea640 1 -- 172.21.15.76:0/2659456747 >> 172.21.15.76:0/2659456747 conn(0x7f95a40b1f90 msgr2=0x7f95a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95ab8ea640 1 -- 172.21.15.76:0/2659456747 shutdown_connections 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.408+0000 7f95ab8ea640 1 -- 172.21.15.76:0/2659456747 wait complete. 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 Processor -- start 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 -- start start 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95900039f0 0x7f95a409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95a409dcc0 0x7f95a409bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95a4154e10 0x7f95a41552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95a406ef20 con 0x7f95a4154e10 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f95a40af6d0 con 0x7f95a409dcc0 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f95a406f0a0 con 0x7f95900039f0 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95a4154e10 0x7f95a41552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95a4154e10 0x7f95a41552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56344/0 (socket says 172.21.15.76:56344) 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 -- 172.21.15.76:0/1437695909 learned_addr learned my addr 172.21.15.76:0/1437695909 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9686640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95900039f0 0x7f95a409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a8e85640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95a409dcc0 0x7f95a409bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 -- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95900039f0 msgr2=0x7f95a409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95900039f0 0x7f95a409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 -- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95a409dcc0 msgr2=0x7f95a409bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95a409dcc0 0x7f95a409bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 -- 172.21.15.76:0/1437695909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95a41f50f0 con 0x7f95a4154e10 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95a9e87640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95a4154e10 0x7f95a41552a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f95a40b5950 tx=0x7f959c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f9595ffb640 1 -- 172.21.15.76:0/1437695909 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c00de70 con 0x7f95a4154e10 2024-09-19T04:14:33.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f9595ffb640 1 -- 172.21.15.76:0/1437695909 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f959c00bb20 con 0x7f95a4154e10 2024-09-19T04:14:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95a409b270 con 0x7f95a4154e10 2024-09-19T04:14:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f9595ffb640 1 -- 172.21.15.76:0/1437695909 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c007070 con 0x7f95a4154e10 2024-09-19T04:14:33.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.412+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95a415c840 con 0x7f95a4154e10 2024-09-19T04:14:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f9595ffb640 1 -- 172.21.15.76:0/1437695909 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f959c01b020 con 0x7f95a4154e10 2024-09-19T04:14:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f9595ffb640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f95880678c0 0x7f9588069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f9595ffb640 1 -- 172.21.15.76:0/1437695909 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f959c085ff0 con 0x7f95a4154e10 2024-09-19T04:14:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f95880678c0 0x7f9588069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f95880678c0 0x7f9588069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95900057f0 tx=0x7f95980011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 Processor -- start 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- start start 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d12150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9d12690 0x5581f9cc8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9cc8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5581f9babb60 con 0x5581f9d11d70 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5581f9b12c70 con 0x5581f9cc86a0 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5581f9ab8ed0 con 0x5581f9d12690 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d12150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d12150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56358/0 (socket says 172.21.15.76:56358) 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 -- 172.21.15.76:0/4037213833 learned_addr learned my addr 172.21.15.76:0/4037213833 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 -- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9d12690 msgr2=0x5581f9cc8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a8e85640 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9d12690 0x5581f9cc8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9e87640 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9cc8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9d12690 0x5581f9cc8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 -- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 msgr2=0x5581f9cc8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9cc8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 -- 172.21.15.76:0/4037213833 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f9acae50 con 0x5581f9d11d70 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a8e85640 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9d12690 0x5581f9cc8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95a9686640 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d12150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9598000c10 tx=0x7f9598001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e3bb0b3e8cb1eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f958effd640 1 -- 172.21.15.76:0/4037213833 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980ba040 con 0x5581f9d11d70 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f958effd640 1 -- 172.21.15.76:0/4037213833 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9598007040 con 0x5581f9d11d70 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f958effd640 1 -- 172.21.15.76:0/4037213833 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980073f0 con 0x5581f9d11d70 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- 172.21.15.76:0/4037213833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 msgr2=0x5581f9d12150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d12150 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9598000c10 tx=0x7f9598001130 comp rx=0 tx=0).stop 2024-09-19T04:14:33.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- 172.21.15.76:0/4037213833 shutdown_connections 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9cc8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9d12690 0x5581f9cc8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/4037213833 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d12150 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- 172.21.15.76:0/4037213833 >> 172.21.15.76:0/4037213833 conn(0x5581f9cca600 msgr2=0x5581f9cc2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- 172.21.15.76:0/4037213833 shutdown_connections 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.416+0000 7f95ae9a6000 1 -- 172.21.15.76:0/4037213833 wait complete. 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 -- start start 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 Processor -- start 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d17eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d12690 0x5581f9d183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9cfe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5581f9b74b00 con 0x5581f9d11d70 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5581f9aed360 con 0x5581f9d12690 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5581f9aea2d0 con 0x5581f9cfe490 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d17eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 10 client.0 _ll_register_callbacks cb 0x5581f9caf820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d17eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56370/0 (socket says 172.21.15.76:56370) 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 -- 172.21.15.76:0/269141779 learned_addr learned my addr 172.21.15.76:0/269141779 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a8e85640 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d12690 0x5581f9d183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 -- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 msgr2=0x5581f9cfe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9cfe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 -- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d12690 msgr2=0x5581f9d183f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d12690 0x5581f9d183f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 -- 172.21.15.76:0/269141779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f9ab9120 con 0x5581f9d11d70 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a8e85640 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d12690 0x5581f9d183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a9686640 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d17eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9598000c10 tx=0x7f959801abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b006e1ba6a67cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f9583fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f9583fff640 1 -- 172.21.15.76:0/269141779 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598007a40 con 0x5581f9d11d70 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f9583fff640 1 -- 172.21.15.76:0/269141779 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f95980d0c50 con 0x5581f9d11d70 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9598007040 con 0x5581f9d11d70 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f9583fff640 1 -- 172.21.15.76:0/269141779 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f95980d0e50 con 0x5581f9d11d70 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 10 client.5127 fetch_fsmap learned FSMap version 19 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5581f9ab2450 con 0x5581f9d11d70 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f9583fff640 1 -- 172.21.15.76:0/269141779 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9598007040 con 0x5581f9d11d70 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f9583fff640 1 -- 172.21.15.76:0/269141779 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 19) ==== 3542+0+0 (secure 0 0 0) 0x7f95980067e0 con 0x5581f9d11d70 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 10 client.5127 fetch_fsmap finished waiting for FSMap version 19 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 10 client.5127 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 20 client.5127 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:33.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9cff280 0x5581f9cff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 4 client.5127 mds_command: new command op to 4666 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:33.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 --> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581f9aacc00 con 0x5581f9cff280 2024-09-19T04:14:33.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a8e85640 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9cff280 0x5581f9cff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f9583fff640 10 client.5127 ms_handle_connect on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:33.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.420+0000 7f95a8e85640 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9cff280 0x5581f9cff660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4a3b268bb9d3d990 server_cookie=c0c6f7780a2946eb in_seq=0 out_seq=0 2024-09-19T04:14:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f9583fff640 1 -- 172.21.15.76:0/269141779 <== mds.2 v2:172.21.15.116:6832/2447208752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581f9aacc00 con 0x5581f9cff280 2024-09-19T04:14:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f9583fff640 10 client.5127 handle_command_reply: tid=0 2024-09-19T04:14:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 client.5127 shutdown 2024-09-19T04:14:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 msgr2=0x5581f9d17eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d17eb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f9598000c10 tx=0x7f959801abd0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 shutdown_connections 2024-09-19T04:14:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9cff280 0x5581f9cff660 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9cfe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d12690 0x5581f9d183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/269141779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d17eb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f9583fff640 0 client.5127 ms_handle_reset on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 >> 172.21.15.76:0/269141779 conn(0x5581f9cc86a0 msgr2=0x5581f9cc99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 shutdown_connections 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 1 -- 172.21.15.76:0/269141779 wait complete. 2024-09-19T04:14:33.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.424+0000 7f95ae9a6000 20 client.5127 trim_cache size 0 max 16384 2024-09-19T04:14:33.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 Processor -- start 2024-09-19T04:14:33.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- start start 2024-09-19T04:14:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9cc86a0 0x5581f9d28260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9d287a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d11d70 0x5581f9d16ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41ff6d0 con 0x5581f9cc86a0 2024-09-19T04:14:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f95a41ffc10 con 0x5581f9d11d70 2024-09-19T04:14:33.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f95a4073120 con 0x5581f9cfe490 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9cc86a0 0x5581f9d28260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9cc86a0 0x5581f9d28260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56386/0 (socket says 172.21.15.76:56386) 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 -- 172.21.15.76:0/309350384 learned_addr learned my addr 172.21.15.76:0/309350384 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9e87640 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d11d70 0x5581f9d16ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 -- 172.21.15.76:0/309350384 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 msgr2=0x5581f9d287a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9d287a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 -- 172.21.15.76:0/309350384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d11d70 msgr2=0x5581f9d16ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d11d70 0x5581f9d16ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 -- 172.21.15.76:0/309350384 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581f9a99fd0 con 0x5581f9cc86a0 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9686640 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9cc86a0 0x5581f9d28260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f959801c1c0 tx=0x7f95980be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e952d88ed985eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f958e7fc640 1 -- 172.21.15.76:0/309350384 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95980dd070 con 0x5581f9cc86a0 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f958e7fc640 1 -- 172.21.15.76:0/309350384 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95980cb070 con 0x5581f9cc86a0 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f958e7fc640 1 -- 172.21.15.76:0/309350384 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598006f70 con 0x5581f9cc86a0 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- 172.21.15.76:0/309350384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9cc86a0 msgr2=0x5581f9d28260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9cc86a0 0x5581f9d28260 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f959801c1c0 tx=0x7f95980be000 comp rx=0 tx=0).stop 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- 172.21.15.76:0/309350384 shutdown_connections 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9d11d70 0x5581f9d16ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9d287a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/309350384 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9cc86a0 0x5581f9d28260 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- 172.21.15.76:0/309350384 >> 172.21.15.76:0/309350384 conn(0x5581f9cfdbf0 msgr2=0x5581f9cc2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- 172.21.15.76:0/309350384 shutdown_connections 2024-09-19T04:14:33.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- 172.21.15.76:0/309350384 wait complete. 2024-09-19T04:14:33.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- start start 2024-09-19T04:14:33.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 Processor -- start 2024-09-19T04:14:33.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9d85270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9d857b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d85cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5581f9b11f80 con 0x5581f9d11d70 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5581f9acaca0 con 0x5581f9cc86a0 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5581f9b172a0 con 0x5581f9cfe490 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95ae9a6000 10 client.0 _ll_register_callbacks cb 0x7f95900045d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9e87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d85cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.444+0000 7f95a9e87640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d85cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56390/0 (socket says 172.21.15.76:56390) 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9e87640 1 -- 172.21.15.76:0/1462798397 learned_addr learned my addr 172.21.15.76:0/1462798397 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a8e85640 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9d857b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9686640 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9d85270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9e87640 1 -- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 msgr2=0x5581f9d857b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9e87640 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9d857b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9e87640 1 -- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 msgr2=0x5581f9d85270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9e87640 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9d85270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9e87640 1 -- 172.21.15.76:0/1462798397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581f9b95750 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9e87640 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d85cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f959c084300 tx=0x7f959c056ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857e32bfaa74fe54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 1 -- 172.21.15.76:0/1462798397 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c06e120 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 1 -- 172.21.15.76:0/1462798397 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f959c05e9f0 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95980cb070 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 1 -- 172.21.15.76:0/1462798397 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f959c06e2c0 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 10 client.5133 fetch_fsmap learned FSMap version 19 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5581f9aa0a30 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 1 -- 172.21.15.76:0/1462798397 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95980cb070 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 1 -- 172.21.15.76:0/1462798397 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 19) ==== 3542+0+0 (secure 0 0 0) 0x7f959c0556f0 con 0x5581f9d11d70 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 10 client.5133 fetch_fsmap finished waiting for FSMap version 19 2024-09-19T04:14:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 10 client.5133 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 20 client.5133 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9d12690 0x5581f9d1d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 4 client.5133 mds_command: new command op to 4666 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 --> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] -- command(tid 0: {"prefix": "status"}) -- 0x5581f9b203a0 con 0x5581f9d12690 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9686640 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9d12690 0x5581f9d1d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 10 client.5133 ms_handle_connect on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95a9686640 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9d12690 0x5581f9d1d0e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f09c8f7faad6e5d server_cookie=b2f49f5668421742 in_seq=0 out_seq=0 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 1 -- 172.21.15.76:0/1462798397 <== mds.2 v2:172.21.15.116:6832/2447208752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5581f9b203a0 con 0x5581f9d12690 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 10 client.5133 handle_command_reply: tid=0 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 client.5133 shutdown 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 msgr2=0x5581f9d85cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d85cf0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f959c084300 tx=0x7f959c056ae0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 shutdown_connections 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5581f9d11d70 0x5581f9d85cf0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x5581f9d12690 0x5581f9d1d0e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5581f9cfe490 0x5581f9d857b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 --2- 172.21.15.76:0/1462798397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5581f9cc86a0 0x5581f9d85270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 >> 172.21.15.76:0/1462798397 conn(0x5581f9cfdbf0 msgr2=0x5581f9d27250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f9583fff640 0 client.5133 ms_handle_reset on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 shutdown_connections 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 1 -- 172.21.15.76:0/1462798397 wait complete. 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ae9a6000 20 client.5133 trim_cache size 0 max 16384 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:33.458 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4666, 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 1.7101672219999999, 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 19, 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 39, 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 39, 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 22.293020748 2024-09-19T04:14:33.459 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:33.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f95880678c0 msgr2=0x7f9588069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f95880678c0 0x7f9588069d80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95900057f0 tx=0x7f95980011a0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95a4154e10 msgr2=0x7f95a41552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95a4154e10 0x7f95a41552a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f95a40b5950 tx=0x7f959c00b4a0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 shutdown_connections 2024-09-19T04:14:33.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f95880678c0 0x7f9588069d80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95a4154e10 0x7f95a41552a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95a409dcc0 0x7f95a409bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 --2- 172.21.15.76:0/1437695909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f95900039f0 0x7f95a409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 >> 172.21.15.76:0/1437695909 conn(0x7f95a40b1f90 msgr2=0x7f95a40d0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 shutdown_connections 2024-09-19T04:14:33.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.448+0000 7f95ab8ea640 1 -- 172.21.15.76:0/1437695909 wait complete. 2024-09-19T04:14:33.469 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:14:33.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f9565128640 1 Processor -- start 2024-09-19T04:14:33.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f9565128640 1 -- start start 2024-09-19T04:14:33.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f9565128640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95600d3060 0x7f95600cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f9565128640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95600d0300 con 0x7f95600cf450 2024-09-19T04:14:33.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f9565128640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95600b0f60 con 0x7f9560161a60 2024-09-19T04:14:33.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f9565128640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f95600cfd40 con 0x7f95600d3060 2024-09-19T04:14:33.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f955e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95600d3060 0x7f95600cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f955e59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f95600d3060 0x7f95600cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56396/0 (socket says 172.21.15.76:56396) 2024-09-19T04:14:33.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.580+0000 7f955e59c640 1 -- 172.21.15.76:0/4009391048 learned_addr learned my addr 172.21.15.76:0/4009391048 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f955e59c640 1 -- 172.21.15.76:0/4009391048 >> v1:172.21.15.116:6789/0 conn(0x7f9560161a60 legacy=0x7f95600d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f955e59c640 1 -- 172.21.15.76:0/4009391048 >> v1:172.21.15.76:6789/0 conn(0x7f95600cf450 legacy=0x7f956006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f955e59c640 1 -- 172.21.15.76:0/4009391048 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95600d0020 con 0x7f95600d3060 2024-09-19T04:14:33.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f955e59c640 1 --2- 172.21.15.76:0/4009391048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600d3060 0x7f95600cef10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9554009870 tx=0x7f955401c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1eb8a44560fd47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f955dd9b640 1 -- 172.21.15.76:0/4009391048 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540b0070 con 0x7f95600d3060 2024-09-19T04:14:33.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f955dd9b640 1 -- 172.21.15.76:0/4009391048 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95540ab600 con 0x7f95600d3060 2024-09-19T04:14:33.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f955dd9b640 1 -- 172.21.15.76:0/4009391048 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540ab9d0 con 0x7f95600d3060 2024-09-19T04:14:33.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f9565128640 1 -- 172.21.15.76:0/4009391048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600d3060 msgr2=0x7f95600cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f9565128640 1 --2- 172.21.15.76:0/4009391048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600d3060 0x7f95600cef10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9554009870 tx=0x7f955401c360 comp rx=0 tx=0).stop 2024-09-19T04:14:33.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f9565128640 1 -- 172.21.15.76:0/4009391048 shutdown_connections 2024-09-19T04:14:33.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f9565128640 1 --2- 172.21.15.76:0/4009391048 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600d3060 0x7f95600cef10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f9565128640 1 -- 172.21.15.76:0/4009391048 >> 172.21.15.76:0/4009391048 conn(0x7f956008bf90 msgr2=0x7f956009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f9565128640 1 -- 172.21.15.76:0/4009391048 shutdown_connections 2024-09-19T04:14:33.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.584+0000 7f9565128640 1 -- 172.21.15.76:0/4009391048 wait complete. 2024-09-19T04:14:33.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 Processor -- start 2024-09-19T04:14:33.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 -- start start 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600cf450 0x7f956014dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600d3060 0x7f95600b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9560161a60 0x7f956014e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95600cf8c0 con 0x7f95600cf450 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f956006d190 con 0x7f95600d3060 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f95600cfa40 con 0x7f9560161a60 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600cf450 0x7f956014dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600cf450 0x7f956014dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56400/0 (socket says 172.21.15.76:56400) 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 -- 172.21.15.76:0/1304597698 learned_addr learned my addr 172.21.15.76:0/1304597698 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955f59e640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9560161a60 0x7f956014e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955e59c640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600d3060 0x7f95600b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 -- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9560161a60 msgr2=0x7f956014e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9560161a60 0x7f956014e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 -- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600d3060 msgr2=0x7f95600b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600d3060 0x7f95600b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 -- 172.21.15.76:0/1304597698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956014e1c0 con 0x7f95600cf450 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955f59e640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9560161a60 0x7f956014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f955ed9d640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600cf450 0x7f956014dc80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f955000c3a0 tx=0x7f955000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f95477fe640 1 -- 172.21.15.76:0/1304597698 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500052d0 con 0x7f95600cf450 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f95477fe640 1 -- 172.21.15.76:0/1304597698 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9550005470 con 0x7f95600cf450 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f95477fe640 1 -- 172.21.15.76:0/1304597698 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550007260 con 0x7f95600cf450 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95600ccbb0 con 0x7f95600cf450 2024-09-19T04:14:33.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95600cd450 con 0x7f95600cf450 2024-09-19T04:14:33.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f95477fe640 1 -- 172.21.15.76:0/1304597698 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9550007400 con 0x7f95600cf450 2024-09-19T04:14:33.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.588+0000 7f95477fe640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95477fe640 1 -- 172.21.15.76:0/1304597698 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f9550085320 con 0x7f95600cf450 2024-09-19T04:14:33.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955e59c640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955e59c640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f95540ad000 tx=0x7f9554008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 Processor -- start 2024-09-19T04:14:33.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- start start 2024-09-19T04:14:33.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea25b20 0x55e8fea25f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea26440 0x55e8fe9dbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9dc290 0x55e8fe9dc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e8fe8c3b60 con 0x55e8fea25b20 2024-09-19T04:14:33.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e8fe82ac70 con 0x55e8fe9dc290 2024-09-19T04:14:33.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55e8fe7d0ed0 con 0x55e8fea26440 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea25b20 0x55e8fea25f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea25b20 0x55e8fea25f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56410/0 (socket says 172.21.15.76:56410) 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 -- 172.21.15.76:0/1024524136 learned_addr learned my addr 172.21.15.76:0/1024524136 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 -- 172.21.15.76:0/1024524136 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea26440 msgr2=0x55e8fe9dbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 --2- 172.21.15.76:0/1024524136 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea26440 0x55e8fe9dbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 -- 172.21.15.76:0/1024524136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9dc290 msgr2=0x55e8fe9dc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 --2- 172.21.15.76:0/1024524136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9dc290 0x55e8fe9dc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 -- 172.21.15.76:0/1024524136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8fe7e2e50 con 0x55e8fea25b20 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f955ed9d640 1 --2- 172.21.15.76:0/1024524136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea25b20 0x55e8fea25f00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f955005dc80 tx=0x7f9550000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2a2a5101d91493 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f9544ff9640 1 -- 172.21.15.76:0/1024524136 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955006d020 con 0x55e8fea25b20 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f9544ff9640 1 -- 172.21.15.76:0/1024524136 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95500850d0 con 0x55e8fea25b20 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f9544ff9640 1 -- 172.21.15.76:0/1024524136 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550051bb0 con 0x55e8fea25b20 2024-09-19T04:14:33.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- 172.21.15.76:0/1024524136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea25b20 msgr2=0x55e8fea25f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 --2- 172.21.15.76:0/1024524136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea25b20 0x55e8fea25f00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f955005dc80 tx=0x7f9550000ee0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- 172.21.15.76:0/1024524136 shutdown_connections 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 --2- 172.21.15.76:0/1024524136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9dc290 0x55e8fe9dc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 --2- 172.21.15.76:0/1024524136 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea26440 0x55e8fe9dbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 --2- 172.21.15.76:0/1024524136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea25b20 0x55e8fea25f00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- 172.21.15.76:0/1024524136 >> 172.21.15.76:0/1024524136 conn(0x55e8fe9e2600 msgr2=0x55e8fe9da630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- 172.21.15.76:0/1024524136 shutdown_connections 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- 172.21.15.76:0/1024524136 wait complete. 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 -- start start 2024-09-19T04:14:33.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.592+0000 7f95681e4000 1 Processor -- start 2024-09-19T04:14:33.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fea25b20 0x55e8fe9e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fe9e6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fe9e1390 0x55e8fe9e7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e8fe88cb00 con 0x55e8fea26440 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e8fe805360 con 0x55e8fea25b20 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55e8fe8022d0 con 0x55e8fe9e1390 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 10 client.0 _ll_register_callbacks cb 0x55e8fe9c7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fe9e6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fe9e6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56424/0 (socket says 172.21.15.76:56424) 2024-09-19T04:14:33.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 -- 172.21.15.76:0/693239558 learned_addr learned my addr 172.21.15.76:0/693239558 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955ed9d640 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fea25b20 0x55e8fe9e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955f59e640 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fe9e1390 0x55e8fe9e7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 -- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fe9e1390 msgr2=0x55e8fe9e7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fe9e1390 0x55e8fe9e7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 -- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fea25b20 msgr2=0x55e8fe9e0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fea25b20 0x55e8fe9e0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 -- 172.21.15.76:0/693239558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8fe7d1120 con 0x55e8fea26440 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955e59c640 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fe9e6cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95540ae040 tx=0x7f9554019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc021269e87248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 1 -- 172.21.15.76:0/693239558 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540c6060 con 0x55e8fea26440 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95500850d0 con 0x55e8fea26440 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 1 -- 172.21.15.76:0/693239558 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f95540c1420 con 0x55e8fea26440 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 1 -- 172.21.15.76:0/693239558 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f95540cc050 con 0x55e8fea26440 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 10 client.5148 fetch_fsmap learned FSMap version 19 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55e8fe7ca450 con 0x55e8fea26440 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 1 -- 172.21.15.76:0/693239558 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95500850d0 con 0x55e8fea26440 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 1 -- 172.21.15.76:0/693239558 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 19) ==== 3542+0+0 (secure 0 0 0) 0x7f95540cf020 con 0x55e8fea26440 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 10 client.5148 fetch_fsmap finished waiting for FSMap version 19 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 10 client.5148 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 20 client.5148 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea34280 0x55e8fea36670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 4 client.5148 mds_command: new command op to 4854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 --> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e8fe7c4c00 con 0x55e8fea34280 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955ed9d640 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea34280 0x55e8fea36670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 10 client.5148 ms_handle_connect on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f955ed9d640 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea34280 0x55e8fea36670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4e7f42b7d019dd4 server_cookie=37b304f035473b74 in_seq=0 out_seq=0 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 1 -- 172.21.15.76:0/693239558 <== mds.1 v2:172.21.15.76:6834/2829561253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e8fe7c4c00 con 0x55e8fea34280 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 10 client.5148 handle_command_reply: tid=0 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 client.5148 shutdown 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 msgr2=0x55e8fe9e6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fe9e6cf0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f95540ae040 tx=0x7f9554019100 comp rx=0 tx=0).stop 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 shutdown_connections 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea34280 0x55e8fea36670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fe9e1390 0x55e8fe9e7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fe9e6cf0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 --2- 172.21.15.76:0/693239558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fea25b20 0x55e8fe9e0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f953dffb640 0 client.5148 ms_handle_reset on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 >> 172.21.15.76:0/693239558 conn(0x55e8fe9dc290 msgr2=0x55e8fe9dad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 shutdown_connections 2024-09-19T04:14:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 1 -- 172.21.15.76:0/693239558 wait complete. 2024-09-19T04:14:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.596+0000 7f95681e4000 20 client.5148 trim_cache size 0 max 16384 2024-09-19T04:14:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 Processor -- start 2024-09-19T04:14:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 -- start start 2024-09-19T04:14:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea24cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8fea25230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fea36670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95602004e0 con 0x55e8fea26440 2024-09-19T04:14:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9560200bb0 con 0x55e8fe9e1390 2024-09-19T04:14:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9560151170 con 0x55e8fea25b20 2024-09-19T04:14:33.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fea36670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fea36670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56432/0 (socket says 172.21.15.76:56432) 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 -- 172.21.15.76:0/5278331 learned_addr learned my addr 172.21.15.76:0/5278331 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955e59c640 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8fea25230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955ed9d640 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea24cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 -- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 msgr2=0x55e8fea25230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8fea25230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 -- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 msgr2=0x55e8fea24cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea24cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 -- 172.21.15.76:0/5278331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e8fe7b1fd0 con 0x55e8fea26440 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.616+0000 7f955f59e640 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fea36670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f95580122a0 tx=0x7f9558012760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ba8ccf5c003db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953ffff640 1 -- 172.21.15.76:0/5278331 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558006bf0 con 0x55e8fea26440 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953ffff640 1 -- 172.21.15.76:0/5278331 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955801b070 con 0x55e8fea26440 2024-09-19T04:14:33.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953ffff640 1 -- 172.21.15.76:0/5278331 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95580080d0 con 0x55e8fea26440 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/5278331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 msgr2=0x55e8fea36670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fea36670 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f95580122a0 tx=0x7f9558012760 comp rx=0 tx=0).stop 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/5278331 shutdown_connections 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8fea36670 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8fea25230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- 172.21.15.76:0/5278331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea24cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/5278331 >> 172.21.15.76:0/5278331 conn(0x55e8fe9dc290 msgr2=0x55e8fe9e0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/5278331 shutdown_connections 2024-09-19T04:14:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/5278331 wait complete. 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- start start 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 Processor -- start 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea39590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8feaa0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8feaa0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e8fe829f80 con 0x55e8fea26440 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e8fe7e2ca0 con 0x55e8fe9e1390 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55e8fe82f2a0 con 0x55e8fea25b20 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 10 client.0 _ll_register_callbacks cb 0x7f95540ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8feaa0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8feaa0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56448/0 (socket says 172.21.15.76:56448) 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 -- 172.21.15.76:0/3188361224 learned_addr learned my addr 172.21.15.76:0/3188361224 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955ed9d640 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea39590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955e59c640 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8feaa0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 -- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 msgr2=0x55e8feaa0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8feaa0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 -- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 msgr2=0x55e8fea39590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea39590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 -- 172.21.15.76:0/3188361224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8fe8ad750 con 0x55e8fea26440 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955f59e640 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8feaa0cb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f95580012f0 tx=0x7f9558001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1452be33b73fed88 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953dffb640 1 -- 172.21.15.76:0/3188361224 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558008740 con 0x55e8fea26440 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953dffb640 1 -- 172.21.15.76:0/3188361224 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f955802ac70 con 0x55e8fea26440 2024-09-19T04:14:33.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f955801b070 con 0x55e8fea26440 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953dffb640 1 -- 172.21.15.76:0/3188361224 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9558031050 con 0x55e8fea26440 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 10 client.5154 fetch_fsmap learned FSMap version 19 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x55e8fe7b8a30 con 0x55e8fea26440 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953dffb640 1 -- 172.21.15.76:0/3188361224 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f955801b070 con 0x55e8fea26440 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953dffb640 1 -- 172.21.15.76:0/3188361224 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 19) ==== 3542+0+0 (secure 0 0 0) 0x7f9558034020 con 0x55e8fea26440 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 10 client.5154 fetch_fsmap finished waiting for FSMap version 19 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 10 client.5154 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 20 client.5154 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea5d170 0x55e8fea5d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 4 client.5154 mds_command: new command op to 4854 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 --> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] -- command(tid 0: {"prefix": "status"}) -- 0x55e8fe8383a0 con 0x55e8fea5d170 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955ed9d640 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea5d170 0x55e8fea5d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f955ed9d640 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea5d170 0x55e8fea5d550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1753f550b4cdb5ff server_cookie=77576222367bc822 in_seq=0 out_seq=0 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.620+0000 7f953dffb640 10 client.5154 ms_handle_connect on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f953dffb640 1 -- 172.21.15.76:0/3188361224 <== mds.1 v2:172.21.15.76:6834/2829561253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55e8fe8383a0 con 0x55e8fea5d170 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f953dffb640 10 client.5154 handle_command_reply: tid=0 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 client.5154 shutdown 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 msgr2=0x55e8feaa0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8feaa0cb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f95580012f0 tx=0x7f9558001260 comp rx=0 tx=0).stop 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 shutdown_connections 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e8fea26440 0x55e8feaa0cb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x55e8fea5d170 0x55e8fea5d550 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e8fea25b20 0x55e8feaa0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f953dffb640 0 client.5154 ms_handle_reset on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 --2- 172.21.15.76:0/3188361224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e8fe9e1390 0x55e8fea39590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 >> 172.21.15.76:0/3188361224 conn(0x55e8fe9dc290 msgr2=0x55e8fea23ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 shutdown_connections 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 1 -- 172.21.15.76:0/3188361224 wait complete. 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f95681e4000 20 client.5154 trim_cache size 0 max 16384 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4854, 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:14:33.632 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:14:33.633 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 2.8950026040000001, 2024-09-19T04:14:33.633 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 19, 2024-09-19T04:14:33.633 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 39, 2024-09-19T04:14:33.633 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 39, 2024-09-19T04:14:33.633 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 22.476972192000002 2024-09-19T04:14:33.633 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 msgr2=0x7f954c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f95540ad000 tx=0x7f9554008040 comp rx=0 tx=0).stop 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600cf450 msgr2=0x7f956014dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600cf450 0x7f956014dc80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f955000c3a0 tx=0x7f955000c860 comp rx=0 tx=0).stop 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 shutdown_connections 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9560161a60 0x7f956014e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600d3060 0x7f95600b1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 --2- 172.21.15.76:0/1304597698 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f95600cf450 0x7f956014dc80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 >> 172.21.15.76:0/1304597698 conn(0x7f956008bf90 msgr2=0x7f95600d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 shutdown_connections 2024-09-19T04:14:33.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.624+0000 7f9565128640 1 -- 172.21.15.76:0/1304597698 wait complete. 2024-09-19T04:14:33.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:14:33.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f554214e640 1 Processor -- start 2024-09-19T04:14:33.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f554214e640 1 -- start start 2024-09-19T04:14:33.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f554214e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f553c06c3d0 0x7f553c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f554214e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f553c0b40c0 0x7f553c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f554214e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f553c0b5c90 con 0x7f553c15e450 2024-09-19T04:14:33.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f554214e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f553c0b4860 con 0x7f553c06c3d0 2024-09-19T04:14:33.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f554214e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f553c0b4580 con 0x7f553c0b40c0 2024-09-19T04:14:33.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f553c06c3d0 0x7f553c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f553c06c3d0 0x7f553c09aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56454/0 (socket says 172.21.15.76:56454) 2024-09-19T04:14:33.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553affd640 1 -- 172.21.15.76:0/3049175129 learned_addr learned my addr 172.21.15.76:0/3049175129 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553bfff640 1 --2- 172.21.15.76:0/3049175129 >> v2:172.21.15.116:3300/0 conn(0x7f553c0b40c0 0x7f553c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553affd640 1 -- 172.21.15.76:0/3049175129 >> v2:172.21.15.116:3300/0 conn(0x7f553c0b40c0 msgr2=0x7f553c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553affd640 1 --2- 172.21.15.76:0/3049175129 >> v2:172.21.15.116:3300/0 conn(0x7f553c0b40c0 0x7f553c15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553affd640 1 -- 172.21.15.76:0/3049175129 >> v1:172.21.15.76:6789/0 conn(0x7f553c15e450 legacy=0x7f553c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.756+0000 7f553affd640 1 -- 172.21.15.76:0/3049175129 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c0b5f10 con 0x7f553c06c3d0 2024-09-19T04:14:33.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f553affd640 1 --2- 172.21.15.76:0/3049175129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c06c3d0 0x7f553c09aa00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f553c0723d0 tx=0x7f553001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0ef10a21ad2cb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f553a7fc640 1 -- 172.21.15.76:0/3049175129 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300b1070 con 0x7f553c06c3d0 2024-09-19T04:14:33.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f553a7fc640 1 -- 172.21.15.76:0/3049175129 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55300ac600 con 0x7f553c06c3d0 2024-09-19T04:14:33.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f553a7fc640 1 -- 172.21.15.76:0/3049175129 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300ac9d0 con 0x7f553c06c3d0 2024-09-19T04:14:33.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- 172.21.15.76:0/3049175129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c06c3d0 msgr2=0x7f553c09aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 --2- 172.21.15.76:0/3049175129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c06c3d0 0x7f553c09aa00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f553c0723d0 tx=0x7f553001c490 comp rx=0 tx=0).stop 2024-09-19T04:14:33.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- 172.21.15.76:0/3049175129 shutdown_connections 2024-09-19T04:14:33.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 --2- 172.21.15.76:0/3049175129 >> v2:172.21.15.116:3300/0 conn(0x7f553c0b40c0 0x7f553c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 --2- 172.21.15.76:0/3049175129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c06c3d0 0x7f553c09aa00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- 172.21.15.76:0/3049175129 >> 172.21.15.76:0/3049175129 conn(0x7f553c0b6800 msgr2=0x7f553c0b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- 172.21.15.76:0/3049175129 shutdown_connections 2024-09-19T04:14:33.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- 172.21.15.76:0/3049175129 wait complete. 2024-09-19T04:14:33.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 Processor -- start 2024-09-19T04:14:33.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- start start 2024-09-19T04:14:33.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f553c06c3d0 0x7f553c14bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f553c0b40c0 0x7f553c14eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c15e450 0x7f553c14a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f553c162d00 con 0x7f553c15e450 2024-09-19T04:14:33.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f553c162b80 con 0x7f553c06c3d0 2024-09-19T04:14:33.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f554214e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f553c162e80 con 0x7f553c0b40c0 2024-09-19T04:14:33.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f553bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c15e450 0x7f553c14a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f553bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c15e450 0x7f553c14a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56458/0 (socket says 172.21.15.76:56458) 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.760+0000 7f553bfff640 1 -- 172.21.15.76:0/522901684 learned_addr learned my addr 172.21.15.76:0/522901684 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553affd640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f553c0b40c0 0x7f553c14eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553b7fe640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f553c06c3d0 0x7f553c14bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553bfff640 1 -- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f553c0b40c0 msgr2=0x7f553c14eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553bfff640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f553c0b40c0 0x7f553c14eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553bfff640 1 -- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f553c06c3d0 msgr2=0x7f553c14bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553bfff640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f553c06c3d0 0x7f553c14bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553bfff640 1 -- 172.21.15.76:0/522901684 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c158720 con 0x7f553c15e450 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553bfff640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c15e450 0x7f553c14a160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5534007e60 tx=0x7f553400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f5523fff640 1 -- 172.21.15.76:0/522901684 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534014620 con 0x7f553c15e450 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f5523fff640 1 -- 172.21.15.76:0/522901684 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5534019070 con 0x7f553c15e450 2024-09-19T04:14:33.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c158a00 con 0x7f553c15e450 2024-09-19T04:14:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f5523fff640 1 -- 172.21.15.76:0/522901684 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534014a00 con 0x7f553c15e450 2024-09-19T04:14:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c158fc0 con 0x7f553c15e450 2024-09-19T04:14:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f5523fff640 1 -- 172.21.15.76:0/522901684 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5534014ba0 con 0x7f553c15e450 2024-09-19T04:14:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f5523fff640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55280678c0 0x7f5528069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f5523fff640 1 -- 172.21.15.76:0/522901684 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f5534086730 con 0x7f553c15e450 2024-09-19T04:14:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553b7fe640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55280678c0 0x7f5528069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f553b7fe640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55280678c0 0x7f5528069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f553c0ae150 tx=0x7f552c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f554520a000 1 Processor -- start 2024-09-19T04:14:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.764+0000 7f554520a000 1 -- start start 2024-09-19T04:14:33.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb1b20 0x5570f3bb1f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb2440 0x5570f3b67d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3b68670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5570f3a4fb60 con 0x5570f3bb2440 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5570f39b6c70 con 0x5570f3b68290 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5570f395ced0 con 0x5570f3bb1b20 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb2440 0x5570f3b67d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb2440 0x5570f3b67d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56464/0 (socket says 172.21.15.76:56464) 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 -- 172.21.15.76:0/5220236 learned_addr learned my addr 172.21.15.76:0/5220236 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553b7fe640 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb1b20 0x5570f3bb1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553bfff640 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3b68670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 -- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb1b20 msgr2=0x5570f3bb1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb1b20 0x5570f3bb1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 -- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 msgr2=0x5570f3b68670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3b68670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 -- 172.21.15.76:0/5220236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f396ee50 con 0x5570f3bb2440 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553b7fe640 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb1b20 0x5570f3bb1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553bfff640 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3b68670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb2440 0x5570f3b67d50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55300190a0 tx=0x7f55300a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bfb4bea2e7f5cd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f55217fa640 1 -- 172.21.15.76:0/5220236 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300b1040 con 0x5570f3bb2440 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f55217fa640 1 -- 172.21.15.76:0/5220236 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55300ac8b0 con 0x5570f3bb2440 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f55217fa640 1 -- 172.21.15.76:0/5220236 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300acca0 con 0x5570f3bb2440 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- 172.21.15.76:0/5220236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb2440 msgr2=0x5570f3b67d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb2440 0x5570f3b67d50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f55300190a0 tx=0x7f55300a7000 comp rx=0 tx=0).stop 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- 172.21.15.76:0/5220236 shutdown_connections 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3b68670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb2440 0x5570f3b67d50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- 172.21.15.76:0/5220236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb1b20 0x5570f3bb1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- 172.21.15.76:0/5220236 >> 172.21.15.76:0/5220236 conn(0x5570f3b6e600 msgr2=0x5570f3b66630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- 172.21.15.76:0/5220236 shutdown_connections 2024-09-19T04:14:33.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- 172.21.15.76:0/5220236 wait complete. 2024-09-19T04:14:33.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- start start 2024-09-19T04:14:33.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 Processor -- start 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3b6cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb2440 0x5570f3b72cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3b73230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5570f3a18b00 con 0x5570f3b6d330 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5570f3991360 con 0x5570f3bb1b20 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5570f398e2d0 con 0x5570f3bb2440 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f554520a000 10 client.0 _ll_register_callbacks cb 0x5570f3b53820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3b73230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3b73230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56480/0 (socket says 172.21.15.76:56480) 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553bfff640 1 -- 172.21.15.76:0/605257373 learned_addr learned my addr 172.21.15.76:0/605257373 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.768+0000 7f553affd640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb2440 0x5570f3b72cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553b7fe640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3b6cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553bfff640 1 -- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb2440 msgr2=0x5570f3b72cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553bfff640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb2440 0x5570f3b72cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553bfff640 1 -- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 msgr2=0x5570f3b6cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553bfff640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3b6cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553bfff640 1 -- 172.21.15.76:0/605257373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f395d120 con 0x5570f3b6d330 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553affd640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb2440 0x5570f3b72cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553b7fe640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3b6cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553bfff640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3b73230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5534056970 tx=0x7f5534056aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c80d6aaeb184fca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 1 -- 172.21.15.76:0/605257373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553405f130 con 0x5570f3b6d330 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 1 -- 172.21.15.76:0/605257373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f553405f2d0 con 0x5570f3b6d330 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55300ac8b0 con 0x5570f3b6d330 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 1 -- 172.21.15.76:0/605257373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5534055e30 con 0x5570f3b6d330 2024-09-19T04:14:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 10 client.5169 fetch_fsmap learned FSMap version 19 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5570f3956450 con 0x5570f3b6d330 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 1 -- 172.21.15.76:0/605257373 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55300ac8b0 con 0x5570f3b6d330 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 1 -- 172.21.15.76:0/605257373 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 19) ==== 3542+0+0 (secure 0 0 0) 0x7f5534055fd0 con 0x5570f3b6d330 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 10 client.5169 fetch_fsmap finished waiting for FSMap version 19 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 10 client.5169 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 20 client.5169 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bc0280 0x5570f3bc2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 4 client.5169 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570f3950c00 con 0x5570f3bc0280 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553b7fe640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bc0280 0x5570f3bc2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 10 client.5169 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f553b7fe640 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bc0280 0x5570f3bc2670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45c716310c1c986 server_cookie=f73e08193617339e in_seq=0 out_seq=0 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 1 -- 172.21.15.76:0/605257373 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570f3950c00 con 0x5570f3bc0280 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 10 client.5169 handle_command_reply: tid=0 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 client.5169 shutdown 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 msgr2=0x5570f3b73230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3b73230 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5534056970 tx=0x7f5534056aa0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 shutdown_connections 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bc0280 0x5570f3bc2670 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3b73230 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3bb2440 0x5570f3b72cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 --2- 172.21.15.76:0/605257373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3b6cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 >> 172.21.15.76:0/605257373 conn(0x5570f3b68290 msgr2=0x5570f3b66d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f551a7fc640 0 client.5169 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 shutdown_connections 2024-09-19T04:14:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 1 -- 172.21.15.76:0/605257373 wait complete. 2024-09-19T04:14:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.772+0000 7f554520a000 20 client.5169 trim_cache size 0 max 16384 2024-09-19T04:14:33.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 Processor -- start 2024-09-19T04:14:33.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- start start 2024-09-19T04:14:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3bb0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b6d330 0x5570f3bb1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb1b20 0x5570f3bc2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f553c203080 con 0x5570f3bb1b20 2024-09-19T04:14:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f553c203750 con 0x5570f3b68290 2024-09-19T04:14:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f553c14e200 con 0x5570f3b6d330 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb1b20 0x5570f3bc2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb1b20 0x5570f3bc2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56488/0 (socket says 172.21.15.76:56488) 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 -- 172.21.15.76:0/1676177953 learned_addr learned my addr 172.21.15.76:0/1676177953 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553b7fe640 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3bb0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553affd640 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b6d330 0x5570f3bb1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 -- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b6d330 msgr2=0x5570f3bb1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b6d330 0x5570f3bb1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 -- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 msgr2=0x5570f3bb0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3bb0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 -- 172.21.15.76:0/1676177953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f393dfd0 con 0x5570f3bb1b20 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553affd640 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b6d330 0x5570f3bb1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553b7fe640 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3bb0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f553bfff640 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb1b20 0x5570f3bc2290 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5534016010 tx=0x7f553401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd4bf1ba94d5ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f5520ff9640 1 -- 172.21.15.76:0/1676177953 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55340570e0 con 0x5570f3bb1b20 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f5520ff9640 1 -- 172.21.15.76:0/1676177953 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5534053050 con 0x5570f3bb1b20 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f5520ff9640 1 -- 172.21.15.76:0/1676177953 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553405f330 con 0x5570f3bb1b20 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- 172.21.15.76:0/1676177953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb1b20 msgr2=0x5570f3bc2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb1b20 0x5570f3bc2290 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5534016010 tx=0x7f553401f000 comp rx=0 tx=0).stop 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- 172.21.15.76:0/1676177953 shutdown_connections 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3bb1b20 0x5570f3bc2290 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b6d330 0x5570f3bb1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 --2- 172.21.15.76:0/1676177953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3b68290 0x5570f3bb0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- 172.21.15.76:0/1676177953 >> 172.21.15.76:0/1676177953 conn(0x5570f3bbd040 msgr2=0x5570f3b6c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- 172.21.15.76:0/1676177953 shutdown_connections 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.792+0000 7f554520a000 1 -- 172.21.15.76:0/1676177953 wait complete. 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- start start 2024-09-19T04:14:33.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 Processor -- start 2024-09-19T04:14:33.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b68290 0x5570f3bc5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3be0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3be0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5570f39b5f80 con 0x5570f3b6d330 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5570f396eca0 con 0x5570f3bb1b20 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5570f39bb2a0 con 0x5570f3b68290 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 10 client.0 _ll_register_callbacks cb 0x7f5534056850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3be0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3be0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56498/0 (socket says 172.21.15.76:56498) 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 -- 172.21.15.76:0/1549550370 learned_addr learned my addr 172.21.15.76:0/1549550370 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553b7fe640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b68290 0x5570f3bc5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553bfff640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3be0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 -- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b68290 msgr2=0x5570f3bc5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b68290 0x5570f3bc5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 -- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 msgr2=0x5570f3be0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3be0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 -- 172.21.15.76:0/1549550370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f3a39750 con 0x5570f3b6d330 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553b7fe640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b68290 0x5570f3bc5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553affd640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3be0960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f553001c970 tx=0x7f553001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62caa6e8c1fee649 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 1 -- 172.21.15.76:0/1549550370 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55300c3020 con 0x5570f3b6d330 2024-09-19T04:14:33.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 1 -- 172.21.15.76:0/1549550370 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f55300ac970 con 0x5570f3b6d330 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5534053050 con 0x5570f3b6d330 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 1 -- 172.21.15.76:0/1549550370 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f55300cd050 con 0x5570f3b6d330 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 10 client.5175 fetch_fsmap learned FSMap version 19 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x5570f3944a30 con 0x5570f3b6d330 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 1 -- 172.21.15.76:0/1549550370 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5534053050 con 0x5570f3b6d330 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 1 -- 172.21.15.76:0/1549550370 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 19) ==== 3542+0+0 (secure 0 0 0) 0x7f55300d0020 con 0x5570f3b6d330 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 10 client.5175 fetch_fsmap finished waiting for FSMap version 19 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 10 client.5175 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 20 client.5175 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bb2440 0x5570f3be13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 4 client.5175 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "status"}) -- 0x5570f39c43a0 con 0x5570f3bb2440 2024-09-19T04:14:33.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553b7fe640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bb2440 0x5570f3be13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 10 client.5175 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f553b7fe640 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bb2440 0x5570f3be13e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89884ff282528c37 server_cookie=ea0404e6b3af204e in_seq=0 out_seq=0 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 1 -- 172.21.15.76:0/1549550370 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5570f39c43a0 con 0x5570f3bb2440 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 10 client.5175 handle_command_reply: tid=0 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 client.5175 shutdown 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 msgr2=0x5570f3be0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3be0960 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f553001c970 tx=0x7f553001acd0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 shutdown_connections 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5570f3bb2440 0x5570f3be13e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570f3bb1b20 0x5570f3be0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570f3b6d330 0x5570f3be0960 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 --2- 172.21.15.76:0/1549550370 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570f3b68290 0x5570f3bc5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 >> 172.21.15.76:0/1549550370 conn(0x5570f3bbd040 msgr2=0x5570f3bafce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f551a7fc640 0 client.5175 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 shutdown_connections 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 1 -- 172.21.15.76:0/1549550370 wait complete. 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554520a000 20 client.5175 trim_cache size 0 max 16384 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:14:33.807 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4863, 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 15.745627263999999, 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 19, 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 39, 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 39, 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 22.643689890000001 2024-09-19T04:14:33.808 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:33.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55280678c0 msgr2=0x7f5528069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554214e640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55280678c0 0x7f5528069d80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f553c0ae150 tx=0x7f552c009040 comp rx=0 tx=0).stop 2024-09-19T04:14:33.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c15e450 msgr2=0x7f553c14a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.796+0000 7f554214e640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c15e450 0x7f553c14a160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5534007e60 tx=0x7f553400e4b0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 shutdown_connections 2024-09-19T04:14:33.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55280678c0 0x7f5528069d80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f553c15e450 0x7f553c14a160 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f553c0b40c0 0x7f553c14eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 --2- 172.21.15.76:0/522901684 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f553c06c3d0 0x7f553c14bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 >> 172.21.15.76:0/522901684 conn(0x7f553c0b6800 msgr2=0x7f553c0b21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 shutdown_connections 2024-09-19T04:14:33.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.800+0000 7f554214e640 1 -- 172.21.15.76:0/522901684 wait complete. 2024-09-19T04:14:33.817 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:14:33.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc825e54640 1 Processor -- start 2024-09-19T04:14:33.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc825e54640 1 -- start start 2024-09-19T04:14:33.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc825e54640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc82015c2a0 0x7fc8200aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc825e54640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8200af2a0 0x7fc82006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc825e54640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8200b3290 con 0x7fc82015bec0 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc825e54640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc8200b3570 con 0x7fc8200af2a0 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc825e54640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc8200b3850 con 0x7fc82015c2a0 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc81ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8200af2a0 0x7fc82006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc81ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc8200af2a0 0x7fc82006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56510/0 (socket says 172.21.15.76:56510) 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.924+0000 7fc81ffff640 1 -- 172.21.15.76:0/3914082269 learned_addr learned my addr 172.21.15.76:0/3914082269 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81effd640 1 --2- 172.21.15.76:0/3914082269 >> v2:172.21.15.116:3300/0 conn(0x7fc82015c2a0 0x7fc8200aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81ffff640 1 -- 172.21.15.76:0/3914082269 >> v2:172.21.15.116:3300/0 conn(0x7fc82015c2a0 msgr2=0x7fc8200aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81ffff640 1 --2- 172.21.15.76:0/3914082269 >> v2:172.21.15.116:3300/0 conn(0x7fc82015c2a0 0x7fc8200aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81ffff640 1 -- 172.21.15.76:0/3914082269 >> v1:172.21.15.116:6789/0 conn(0x7fc82015bec0 legacy=0x7fc8200ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:33.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81ffff640 1 -- 172.21.15.76:0/3914082269 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8200b3b30 con 0x7fc8200af2a0 2024-09-19T04:14:33.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81ffff640 1 --2- 172.21.15.76:0/3914082269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8200af2a0 0x7fc82006e000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc8200ae510 tx=0x7fc81801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a88465dc7678293 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81e7fc640 1 -- 172.21.15.76:0/3914082269 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180a5840 con 0x7fc8200af2a0 2024-09-19T04:14:33.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81e7fc640 1 -- 172.21.15.76:0/3914082269 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8180a59e0 con 0x7fc8200af2a0 2024-09-19T04:14:33.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc81e7fc640 1 -- 172.21.15.76:0/3914082269 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180a5db0 con 0x7fc8200af2a0 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 -- 172.21.15.76:0/3914082269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8200af2a0 msgr2=0x7fc82006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 --2- 172.21.15.76:0/3914082269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8200af2a0 0x7fc82006e000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fc8200ae510 tx=0x7fc81801c980 comp rx=0 tx=0).stop 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 -- 172.21.15.76:0/3914082269 shutdown_connections 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 --2- 172.21.15.76:0/3914082269 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8200af2a0 0x7fc82006e000 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 --2- 172.21.15.76:0/3914082269 >> v2:172.21.15.116:3300/0 conn(0x7fc82015c2a0 0x7fc8200aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 -- 172.21.15.76:0/3914082269 >> 172.21.15.76:0/3914082269 conn(0x7fc8200b1f90 msgr2=0x7fc8200b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 -- 172.21.15.76:0/3914082269 shutdown_connections 2024-09-19T04:14:33.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.928+0000 7fc825e54640 1 -- 172.21.15.76:0/3914082269 wait complete. 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 Processor -- start 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 -- start start 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8200af2a0 0x7fc8201640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc82015bec0 0x7fc820156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc82015c2a0 0x7fc820154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc82006ed50 con 0x7fc82015c2a0 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc8200af710 con 0x7fc82015bec0 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc82006eed0 con 0x7fc8200af2a0 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc82015c2a0 0x7fc820154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc82015c2a0 0x7fc820154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56520/0 (socket says 172.21.15.76:56520) 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 -- 172.21.15.76:0/3599986740 learned_addr learned my addr 172.21.15.76:0/3599986740 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:33.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81f7fe640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8200af2a0 0x7fc8201640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81effd640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc82015bec0 0x7fc820156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 -- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8200af2a0 msgr2=0x7fc8201640f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8200af2a0 0x7fc8201640f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 -- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc82015bec0 msgr2=0x7fc820156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc82015bec0 0x7fc820156950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 -- 172.21.15.76:0/3599986740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82014a4f0 con 0x7fc82015c2a0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81f7fe640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8200af2a0 0x7fc8201640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc81ffff640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc82015c2a0 0x7fc820154f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc81801a5b0 tx=0x7fc818019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc807fff640 1 -- 172.21.15.76:0/3599986740 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180b8880 con 0x7fc82015c2a0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc807fff640 1 -- 172.21.15.76:0/3599986740 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8180be070 con 0x7fc82015c2a0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc820153bb0 con 0x7fc82015c2a0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc807fff640 1 -- 172.21.15.76:0/3599986740 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8180b8c80 con 0x7fc82015c2a0 2024-09-19T04:14:33.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc820154170 con 0x7fc82015c2a0 2024-09-19T04:14:33.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82006f260 con 0x7fc82015c2a0 2024-09-19T04:14:33.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc807fff640 1 -- 172.21.15.76:0/3599986740 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc8180c3460 con 0x7fc82015c2a0 2024-09-19T04:14:33.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.932+0000 7fc807fff640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc80c0678c0 0x7fc80c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:33.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.936+0000 7fc807fff640 1 -- 172.21.15.76:0/3599986740 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc8181211a0 con 0x7fc82015c2a0 2024-09-19T04:14:33.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.936+0000 7fc81f7fe640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc80c0678c0 0x7fc80c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:33.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.936+0000 7fc81f7fe640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc80c0678c0 0x7fc80c069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8200740f0 tx=0x7fc810009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:33.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:33.940+0000 7fc807fff640 1 -- 172.21.15.76:0/3599986740 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc82014a4f0 con 0x7fc82015c2a0 2024-09-19T04:14:34.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.127+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc82006fcd0 con 0x7fc82015c2a0 2024-09-19T04:14:34.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.127+0000 7fc807fff640 1 -- 172.21.15.76:0/3599986740 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5267 (secure 0 0 0) 0x7fc8180165a0 con 0x7fc82015c2a0 2024-09-19T04:14:34.132 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:14:34.132 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":19,"btime":"2024-09-19T04:14:32:739642+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:32.739633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4863,"mds_1":4854,"mds_2":4666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4666":{"gid":4666,"name":"d","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4854":{"gid":4854,"name":"a","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4666,4854,4863]},"id":2}]} 2024-09-19T04:14:34.132 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 19 2024-09-19T04:14:34.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc80c0678c0 msgr2=0x7fc80c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc80c0678c0 0x7fc80c069d80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc8200740f0 tx=0x7fc810009040 comp rx=0 tx=0).stop 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc82015c2a0 msgr2=0x7fc820154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc82015c2a0 0x7fc820154f30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc81801a5b0 tx=0x7fc818019720 comp rx=0 tx=0).stop 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 shutdown_connections 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc80c0678c0 0x7fc80c069d80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc82015c2a0 0x7fc820154f30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc82015bec0 0x7fc820156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 --2- 172.21.15.76:0/3599986740 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8200af2a0 0x7fc8201640f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 >> 172.21.15.76:0/3599986740 conn(0x7fc8200b1f90 msgr2=0x7fc8200b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 shutdown_connections 2024-09-19T04:14:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:34.131+0000 7fc825e54640 1 -- 172.21.15.76:0/3599986740 wait complete. 2024-09-19T04:14:34.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:34.152 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:14:34.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:34.174 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:14:34.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:34.243 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-19T04:14:34.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:34.291 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-19T04:14:34.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:34.342 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-19T04:14:34.394 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:14:34.573 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:14:34.573 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:14:34.573 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:14:36.574 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:14:36.721 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:14:36.721 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:14:36.721 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:14:38.722 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:14:38.869 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:14:38.869 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:14:38.869 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:14:40.870 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:14:41.019 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:14:41.019 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:14:41.020 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:14:43.020 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:14:43.167 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-19T04:14:43.167 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:14:43.168 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:14:43.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:14:43.306 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:14:43.307 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:14:43.307 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:14:45.308 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:14:45.454 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:14:45.455 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:14:45.455 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:14:47.456 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:14:47.602 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-19T04:14:47.603 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:14:47.603 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:14:47.603 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-19T04:14:47.752 INFO:teuthology.orchestra.run.smithi116.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:14:47.752 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:14:47.753 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:14:47.753 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 flush journal 2024-09-19T04:14:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.879+0000 7fcc6002d640 1 Processor -- start 2024-09-19T04:14:47.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.879+0000 7fcc6002d640 1 -- start start 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc6002d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc5815b5a0 0x7fcc5815ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc6002d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc580b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc6002d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcc5815c040 con 0x7fcc580ce0a0 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc6002d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcc580cfe00 con 0x7fcc5815b5a0 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc6002d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcc5815c320 con 0x7fcc580b4e50 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc5815b5a0 0x7fcc5815ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcc5815b5a0 0x7fcc5815ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42928/0 (socket says 172.21.15.76:42928) 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 -- 172.21.15.76:0/4008316626 learned_addr learned my addr 172.21.15.76:0/4008316626 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:47.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/4008316626 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc580b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 -- 172.21.15.76:0/4008316626 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 msgr2=0x7fcc580b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/4008316626 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc580b52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 -- 172.21.15.76:0/4008316626 >> v1:172.21.15.116:6789/0 conn(0x7fcc580ce0a0 legacy=0x7fcc580d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 -- 172.21.15.76:0/4008316626 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc580cfa90 con 0x7fcc5815b5a0 2024-09-19T04:14:47.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/4008316626 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc580b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:47.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.883+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/4008316626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc5815ba10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bb40 tx=0x7fcc4801ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a129cb00f91c475 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc5cdc7640 1 -- 172.21.15.76:0/4008316626 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc480b4070 con 0x7fcc5815b5a0 2024-09-19T04:14:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc5cdc7640 1 -- 172.21.15.76:0/4008316626 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc480a7a60 con 0x7fcc5815b5a0 2024-09-19T04:14:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc5cdc7640 1 -- 172.21.15.76:0/4008316626 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc480a7e30 con 0x7fcc5815b5a0 2024-09-19T04:14:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 -- 172.21.15.76:0/4008316626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 msgr2=0x7fcc5815ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 --2- 172.21.15.76:0/4008316626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc5815ba10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcc4800bb40 tx=0x7fcc4801ebd0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 -- 172.21.15.76:0/4008316626 shutdown_connections 2024-09-19T04:14:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 --2- 172.21.15.76:0/4008316626 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc580b52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 --2- 172.21.15.76:0/4008316626 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc5815ba10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 -- 172.21.15.76:0/4008316626 >> 172.21.15.76:0/4008316626 conn(0x7fcc58073580 msgr2=0x7fcc5806bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 -- 172.21.15.76:0/4008316626 shutdown_connections 2024-09-19T04:14:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 -- 172.21.15.76:0/4008316626 wait complete. 2024-09-19T04:14:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 Processor -- start 2024-09-19T04:14:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.887+0000 7fcc6002d640 1 -- start start 2024-09-19T04:14:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc58141c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc580ce0a0 0x7fcc58142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc581426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5815d1e0 con 0x7fcc5815b5a0 2024-09-19T04:14:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5815d060 con 0x7fcc580ce0a0 2024-09-19T04:14:47.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcc580b3d40 con 0x7fcc580b4e50 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc581426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc581426b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42936/0 (socket says 172.21.15.76:42936) 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/771378811 learned_addr learned my addr 172.21.15.76:0/771378811 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc58141c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc580ce0a0 0x7fcc58142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 msgr2=0x7fcc58141c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc58141c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc580ce0a0 msgr2=0x7fcc58142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc580ce0a0 0x7fcc58142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/771378811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc58143d80 con 0x7fcc5815b5a0 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc580ce0a0 0x7fcc58142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc58141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:14:47.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc581426b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000cef0 tx=0x7fcc5000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:47.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc4e7fc640 1 -- 172.21.15.76:0/771378811 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5001a070 con 0x7fcc5815b5a0 2024-09-19T04:14:47.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc4e7fc640 1 -- 172.21.15.76:0/771378811 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5000d860 con 0x7fcc5815b5a0 2024-09-19T04:14:47.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc58144060 con 0x7fcc5815b5a0 2024-09-19T04:14:47.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc4e7fc640 1 -- 172.21.15.76:0/771378811 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000dc50 con 0x7fcc5815b5a0 2024-09-19T04:14:47.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc58155e90 con 0x7fcc5815b5a0 2024-09-19T04:14:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc4e7fc640 1 -- 172.21.15.76:0/771378811 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcc5001d070 con 0x7fcc5815b5a0 2024-09-19T04:14:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc4e7fc640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcc3c0678c0 0x7fcc3c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc4e7fc640 1 -- 172.21.15.76:0/771378811 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fcc500866b0 con 0x7fcc5815b5a0 2024-09-19T04:14:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcc3c0678c0 0x7fcc3c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.891+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcc3c0678c0 0x7fcc3c069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc54003ec0 tx=0x7fcc54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 Processor -- start 2024-09-19T04:14:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- start start 2024-09-19T04:14:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c10f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c10a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5639c0f91960 con 0x5639c10f3d70 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5639c0ed5900 con 0x5639c10a7db0 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5639c0ef8d40 con 0x5639c10f4690 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c10f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c10f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42940/0 (socket says 172.21.15.76:42940) 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/2471782514 learned_addr learned my addr 172.21.15.76:0/2471782514 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 msgr2=0x5639c10a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c10a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 msgr2=0x5639c10a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c10a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/2471782514 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c0e9f580 con 0x5639c10f3d70 2024-09-19T04:14:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c10f4150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcc54003ca0 tx=0x7fcc54011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5645f9793ee4a8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc3b7fe640 1 -- 172.21.15.76:0/2471782514 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54005030 con 0x5639c10f3d70 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc3b7fe640 1 -- 172.21.15.76:0/2471782514 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc540051d0 con 0x5639c10f3d70 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc3b7fe640 1 -- 172.21.15.76:0/2471782514 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc54007240 con 0x5639c10f3d70 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- 172.21.15.76:0/2471782514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 msgr2=0x5639c10f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c10f4150 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fcc54003ca0 tx=0x7fcc54011000 comp rx=0 tx=0).stop 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- 172.21.15.76:0/2471782514 shutdown_connections 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c10a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 --2- 172.21.15.76:0/2471782514 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c10f4150 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- 172.21.15.76:0/2471782514 >> 172.21.15.76:0/2471782514 conn(0x5639c10d5040 msgr2=0x5639c10d7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- 172.21.15.76:0/2471782514 shutdown_connections 2024-09-19T04:14:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- 172.21.15.76:0/2471782514 wait complete. 2024-09-19T04:14:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 -- start start 2024-09-19T04:14:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.895+0000 7fcc630e9000 1 Processor -- start 2024-09-19T04:14:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10a7db0 0x5639c10face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10f3d70 0x5639c10fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5639c0ef4d00 con 0x5639c10a7db0 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5639c0eb3aa0 con 0x5639c10f3d70 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5639c0e92340 con 0x5639c10f4690 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 10 client.0 _ll_register_callbacks cb 0x5639c1095760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10a7db0 0x5639c10face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10a7db0 0x5639c10face0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42952/0 (socket says 172.21.15.76:42952) 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/1331876377 learned_addr learned my addr 172.21.15.76:0/1331876377 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10f3d70 0x5639c10fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 msgr2=0x5639c10fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10f3d70 msgr2=0x5639c10fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10f3d70 0x5639c10fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 -- 172.21.15.76:0/1331876377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c0e9bdc0 con 0x5639c10a7db0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10a7db0 0x5639c10face0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcc540074f0 tx=0x7fcc54003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae700c99c2f0ca5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 1 -- 172.21.15.76:0/1331876377 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc540076d0 con 0x5639c10a7db0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 1 -- 172.21.15.76:0/1331876377 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fcc5401e040 con 0x5639c10a7db0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc540051d0 con 0x5639c10a7db0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 1 -- 172.21.15.76:0/1331876377 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcc5400e9f0 con 0x5639c10a7db0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 10 client.5193 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5639c0e987f0 con 0x5639c10a7db0 2024-09-19T04:14:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 1 -- 172.21.15.76:0/1331876377 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc540051d0 con 0x5639c10a7db0 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 1 -- 172.21.15.76:0/1331876377 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fcc54014070 con 0x5639c10a7db0 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 10 client.5193 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 10 client.5193 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 20 client.5193 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c11030b0 0x5639c11054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 4 client.5193 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639c0e92e10 con 0x5639c11030b0 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c11030b0 0x5639c11054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 10 client.5193 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c11030b0 0x5639c11054a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63cafa5636c80d77 server_cookie=c63d8d794d332bcf in_seq=0 out_seq=0 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 1 -- 172.21.15.76:0/1331876377 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639c0e92e10 con 0x5639c11030b0 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 10 client.5193 handle_command_reply: tid=0 2024-09-19T04:14:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 client.5193 shutdown 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10a7db0 msgr2=0x5639c10face0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10a7db0 0x5639c10face0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fcc540074f0 tx=0x7fcc54003700 comp rx=0 tx=0).stop 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 shutdown_connections 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c11030b0 0x5639c11054a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10f4690 0x5639c10fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10f3d70 0x5639c10fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 --2- 172.21.15.76:0/1331876377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10a7db0 0x5639c10face0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc38ff9640 0 client.5193 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 >> 172.21.15.76:0/1331876377 conn(0x5639c10d5040 msgr2=0x5639c10a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 shutdown_connections 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 1 -- 172.21.15.76:0/1331876377 wait complete. 2024-09-19T04:14:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.899+0000 7fcc630e9000 20 client.5193 trim_cache size 0 max 16384 2024-09-19T04:14:47.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 Processor -- start 2024-09-19T04:14:47.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 -- start start 2024-09-19T04:14:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 0x5639c1108d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c11013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c1101930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581ffc30 con 0x5639c10f3d70 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581fee00 con 0x5639c10a7db0 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcc581ff500 con 0x5639c10d5040 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c1101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c1101930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42968/0 (socket says 172.21.15.76:42968) 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/834434906 learned_addr learned my addr 172.21.15.76:0/834434906 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/834434906 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 msgr2=0x5639c1108d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/834434906 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 0x5639c1108d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/834434906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 msgr2=0x5639c11013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/834434906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c11013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.915+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/834434906 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c0eb6470 con 0x5639c10f3d70 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/834434906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c1101930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcc5005bfd0 tx=0x7fcc5005d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42324ac8decf81f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc3affd640 1 -- 172.21.15.76:0/834434906 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5005add0 con 0x5639c10f3d70 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc3affd640 1 -- 172.21.15.76:0/834434906 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5005f600 con 0x5639c10f3d70 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc3affd640 1 -- 172.21.15.76:0/834434906 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50053090 con 0x5639c10f3d70 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/834434906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 msgr2=0x5639c1101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- 172.21.15.76:0/834434906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c1101930 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcc5005bfd0 tx=0x7fcc5005d040 comp rx=0 tx=0).stop 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/834434906 shutdown_connections 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- 172.21.15.76:0/834434906 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c1101930 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- 172.21.15.76:0/834434906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c11013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- 172.21.15.76:0/834434906 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 0x5639c1108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/834434906 >> 172.21.15.76:0/834434906 conn(0x5639c10ac890 msgr2=0x5639c1100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/834434906 shutdown_connections 2024-09-19T04:14:47.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/834434906 wait complete. 2024-09-19T04:14:47.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- start start 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 Processor -- start 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c1107e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 0x5639c112bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c112c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5639c0ef8050 con 0x5639c10f3d70 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcc581fde90 con 0x5639c10a7db0 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5639c0f06fd0 con 0x5639c10d5040 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 10 client.0 _ll_register_callbacks cb 0x7fcc54006870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c112c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c112c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42976/0 (socket says 172.21.15.76:42976) 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/3719648479 learned_addr learned my addr 172.21.15.76:0/3719648479 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:47.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c1107e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 msgr2=0x5639c112bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5d5c8640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 0x5639c112bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 0x5639c112bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 msgr2=0x5639c1107e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c1107e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 -- 172.21.15.76:0/3719648479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c0e8c640 con 0x5639c10f3d70 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c1107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5e5ca640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c112c4e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcc50084aa0 tx=0x7fcc50017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40386b0eef18a629 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:47.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc38ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc38ff9640 1 -- 172.21.15.76:0/3719648479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50056e80 con 0x5639c10f3d70 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc38ff9640 1 -- 172.21.15.76:0/3719648479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fcc5005f170 con 0x5639c10f3d70 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc5005f600 con 0x5639c10f3d70 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc38ff9640 1 -- 172.21.15.76:0/3719648479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fcc5007d050 con 0x5639c10f3d70 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 10 client.5199 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x5639c0e903b0 con 0x5639c10f3d70 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc38ff9640 1 -- 172.21.15.76:0/3719648479 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc5005f600 con 0x5639c10f3d70 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc38ff9640 1 -- 172.21.15.76:0/3719648479 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fcc5006ebc0 con 0x5639c10f3d70 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 10 client.5199 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 10 client.5199 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 20 client.5199 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:47.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c10f4690 0x5639c112ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:47.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 4 client.5199 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:14:47.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639c0f06470 con 0x5639c10f4690 2024-09-19T04:14:47.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c10f4690 0x5639c112ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:47.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.919+0000 7fcc38ff9640 10 client.5199 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:47.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.923+0000 7fcc5ddc9640 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c10f4690 0x5639c112ca20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f3641cda9994c7 server_cookie=601a399db311ead9 in_seq=0 out_seq=0 2024-09-19T04:14:47.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc38ff9640 1 -- 172.21.15.76:0/3719648479 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639c0f06470 con 0x5639c10f4690 2024-09-19T04:14:47.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc38ff9640 10 client.5199 handle_command_reply: tid=0 2024-09-19T04:14:47.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 client.5199 shutdown 2024-09-19T04:14:47.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 msgr2=0x5639c112c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c112c4e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fcc50084aa0 tx=0x7fcc50017000 comp rx=0 tx=0).stop 2024-09-19T04:14:47.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 shutdown_connections 2024-09-19T04:14:47.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5639c10f4690 0x5639c112ca20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639c10f3d70 0x5639c112c4e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639c10d5040 0x5639c112bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc38ff9640 0 client.5199 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 --2- 172.21.15.76:0/3719648479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639c10a7db0 0x5639c1107e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 >> 172.21.15.76:0/3719648479 conn(0x5639c10ac890 msgr2=0x5639c11012f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 shutdown_connections 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 1 -- 172.21.15.76:0/3719648479 wait complete. 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc630e9000 20 client.5199 trim_cache size 0 max 16384 2024-09-19T04:14:47.946 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:47.947 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:14:47.947 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:14:47.947 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:47.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcc3c0678c0 msgr2=0x7fcc3c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcc3c0678c0 0x7fcc3c069d80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcc54003ec0 tx=0x7fcc54009040 comp rx=0 tx=0).stop 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 msgr2=0x7fcc581426b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc581426b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000cef0 tx=0x7fcc5000a710 comp rx=0 tx=0).stop 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 shutdown_connections 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcc3c0678c0 0x7fcc3c069d80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcc5815b5a0 0x7fcc581426b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc580ce0a0 0x7fcc58142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 --2- 172.21.15.76:0/771378811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcc580b4e50 0x7fcc58141c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 >> 172.21.15.76:0/771378811 conn(0x7fcc58073580 msgr2=0x7fcc5815a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 shutdown_connections 2024-09-19T04:14:47.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:47.939+0000 7fcc6002d640 1 -- 172.21.15.76:0/771378811 wait complete. 2024-09-19T04:14:47.961 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:1 flush journal 2024-09-19T04:14:48.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.071+0000 7fdf8738a640 1 Processor -- start 2024-09-19T04:14:48.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.071+0000 7fdf8738a640 1 -- start start 2024-09-19T04:14:48.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf8738a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 0x7fdf8015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf8738a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf800b4620 con 0x7fdf800b40d0 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf8738a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdf800b4900 con 0x7fdf801567e0 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf8738a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdf800b4be0 con 0x7fdf800b3c40 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdf800b40d0 0x7fdf8015d920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46062/0 (socket says 172.21.15.76:46062) 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 -- 172.21.15.76:0/1845179648 learned_addr learned my addr 172.21.15.76:0/1845179648 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf84925640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 0x7fdf8015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 321219948 0 0) 0x7fdf800b4620 con 0x7fdf800b40d0 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdf6c003530 con 0x7fdf800b40d0 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1134398351 0 0) 0x7fdf6c003530 con 0x7fdf800b40d0 2024-09-19T04:14:48.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 msgr2=0x7fdf8015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 0x7fdf8015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 >> v1:172.21.15.116:6789/0 conn(0x7fdf801567e0 legacy=0x7fdf800b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf800b4ec0 con 0x7fdf800b40d0 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fdf78001a00 con 0x7fdf800b40d0 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 >> v1:172.21.15.76:6789/0 conn(0x7fdf800b40d0 legacy=0x7fdf8015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 0x7fdf6c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf6c003f70 0x7fdf6c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf6c004d50 0x7fdf6c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf800b4be0 con 0x7fdf6c004d50 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c00bc10 con 0x7fdf6c003f70 2024-09-19T04:14:48.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c00be10 con 0x7fdf800b3c40 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf6c004d50 0x7fdf6c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85126640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 0x7fdf6c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf84925640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf6c003f70 0x7fdf6c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 -- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 msgr2=0x7fdf6c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf800b3c40 0x7fdf6c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 -- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf6c003f70 msgr2=0x7fdf6c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf6c003f70 0x7fdf6c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf85927640 1 -- 172.21.15.76:0/1845179648 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c003530 con 0x7fdf6c004d50 2024-09-19T04:14:48.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.075+0000 7fdf84925640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf6c003f70 0x7fdf6c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf85927640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf6c004d50 0x7fdf6c0051d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800a860 tx=0x7fdf7801d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf56d4861e1218b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7800a480 con 0x7fdf6c004d50 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf780ad630 con 0x7fdf6c004d50 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf77fff640 1 -- 172.21.15.76:0/1845179648 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780adaa0 con 0x7fdf6c004d50 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- 172.21.15.76:0/1845179648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf6c004d50 msgr2=0x7fdf6c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 --2- 172.21.15.76:0/1845179648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf6c004d50 0x7fdf6c0051d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800a860 tx=0x7fdf7801d350 comp rx=0 tx=0).stop 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf85927640 1 -- 172.21.15.76:0/1845179648 reap_dead start 2024-09-19T04:14:48.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- 172.21.15.76:0/1845179648 shutdown_connections 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- 172.21.15.76:0/1845179648 >> 172.21.15.76:0/1845179648 conn(0x7fdf80073580 msgr2=0x7fdf8006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- 172.21.15.76:0/1845179648 shutdown_connections 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- 172.21.15.76:0/1845179648 wait complete. 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 Processor -- start 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- start start 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf801506d0 0x7fdf80150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf80150ab0 0x7fdf800a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf801514e0 0x7fdf800ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80160b50 con 0x7fdf80150ab0 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf801609d0 con 0x7fdf801506d0 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf8738a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdf801680f0 con 0x7fdf801514e0 2024-09-19T04:14:48.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf80150ab0 0x7fdf800a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf80150ab0 0x7fdf800a9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42998/0 (socket says 172.21.15.76:42998) 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 -- 172.21.15.76:0/2411837720 learned_addr learned my addr 172.21.15.76:0/2411837720 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf85927640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf801514e0 0x7fdf800ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf85126640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf801506d0 0x7fdf80150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 -- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf801514e0 msgr2=0x7fdf800ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf801514e0 0x7fdf800ae660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 -- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf801506d0 msgr2=0x7fdf80150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf801506d0 0x7fdf80150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf84925640 1 -- 172.21.15.76:0/2411837720 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf80152e20 con 0x7fdf80150ab0 2024-09-19T04:14:48.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.079+0000 7fdf85126640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf801506d0 0x7fdf80150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf84925640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf80150ab0 0x7fdf800a9be0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf700108c0 tx=0x7fdf70010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf75ffb640 1 -- 172.21.15.76:0/2411837720 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000f040 con 0x7fdf80150ab0 2024-09-19T04:14:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf75ffb640 1 -- 172.21.15.76:0/2411837720 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf70004220 con 0x7fdf80150ab0 2024-09-19T04:14:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf80153100 con 0x7fdf80150ab0 2024-09-19T04:14:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf75ffb640 1 -- 172.21.15.76:0/2411837720 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000e960 con 0x7fdf80150ab0 2024-09-19T04:14:48.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf801f94c0 con 0x7fdf80150ab0 2024-09-19T04:14:48.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf75ffb640 1 -- 172.21.15.76:0/2411837720 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdf7000eb00 con 0x7fdf80150ab0 2024-09-19T04:14:48.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf75ffb640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf640678c0 0x7fdf64069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf75ffb640 1 -- 172.21.15.76:0/2411837720 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdf70086100 con 0x7fdf80150ab0 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf85126640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf640678c0 0x7fdf64069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf85126640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf640678c0 0x7fdf64069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c007fd0 tx=0x7fdf7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf8a446000 1 Processor -- start 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.083+0000 7fdf8a446000 1 -- start start 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a4d70 0x556f7c1a5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a5690 0x556f7c158870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c159190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556f7c042960 con 0x556f7c158db0 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556f7bf86900 con 0x556f7c1a4d70 2024-09-19T04:14:48.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556f7bfa9d40 con 0x556f7c1a5690 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c159190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c159190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43012/0 (socket says 172.21.15.76:43012) 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3742562903 learned_addr learned my addr 172.21.15.76:0/3742562903 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf84925640 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a5690 0x556f7c158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85126640 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a4d70 0x556f7c1a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a5690 msgr2=0x556f7c158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a5690 0x556f7c158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a4d70 msgr2=0x556f7c1a5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a4d70 0x556f7c1a5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3742562903 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f7bf50580 con 0x556f7c158db0 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c159190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800a860 tx=0x7fdf80072400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4893cd341cbc3e13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf6affd640 1 -- 172.21.15.76:0/3742562903 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78001fe0 con 0x556f7c158db0 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf6affd640 1 -- 172.21.15.76:0/3742562903 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7800a480 con 0x556f7c158db0 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf6affd640 1 -- 172.21.15.76:0/3742562903 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78001800 con 0x556f7c158db0 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- 172.21.15.76:0/3742562903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 msgr2=0x556f7c159190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c159190 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fdf7800a860 tx=0x7fdf80072400 comp rx=0 tx=0).stop 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- 172.21.15.76:0/3742562903 shutdown_connections 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c159190 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a5690 0x556f7c158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3742562903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a4d70 0x556f7c1a5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- 172.21.15.76:0/3742562903 >> 172.21.15.76:0/3742562903 conn(0x556f7c186040 msgr2=0x556f7c188430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- 172.21.15.76:0/3742562903 shutdown_connections 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- 172.21.15.76:0/3742562903 wait complete. 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- start start 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 Processor -- start 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c158db0 0x556f7c160530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c160a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c1a5690 0x556f7c186420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556f7bfa5d00 con 0x556f7c1a5690 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556f7bf64aa0 con 0x556f7c158db0 2024-09-19T04:14:48.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556f7bf43340 con 0x556f7c1a4d70 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 10 client.0 _ll_register_callbacks cb 0x556f7c146760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c1a5690 0x556f7c186420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c1a5690 0x556f7c186420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43024/0 (socket says 172.21.15.76:43024) 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3610069665 learned_addr learned my addr 172.21.15.76:0/3610069665 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf84925640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85126640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c158db0 0x556f7c160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 msgr2=0x556f7c160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c158db0 msgr2=0x556f7c160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c158db0 0x556f7c160530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 -- 172.21.15.76:0/3610069665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7bf4cdc0 con 0x556f7c1a5690 2024-09-19T04:14:48.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf84925640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf85927640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c1a5690 0x556f7c186420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdf78098d70 tx=0x7fdf7801a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1760860da68043ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf5ffff640 1 -- 172.21.15.76:0/3610069665 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf780011d0 con 0x556f7c1a5690 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf5ffff640 1 -- 172.21.15.76:0/3610069665 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdf780b2040 con 0x556f7c1a5690 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.087+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf7800a480 con 0x556f7c1a5690 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf5ffff640 1 -- 172.21.15.76:0/3610069665 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdf780c9050 con 0x556f7c1a5690 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 10 client.5211 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x556f7bf497f0 con 0x556f7c1a5690 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf5ffff640 1 -- 172.21.15.76:0/3610069665 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7800a480 con 0x556f7c1a5690 2024-09-19T04:14:48.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf5ffff640 1 -- 172.21.15.76:0/3610069665 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fdf780cc020 con 0x556f7c1a5690 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 10 client.5211 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 10 client.5211 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.a 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 20 client.5211 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c186e30 0x556f7c1b6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 4 client.5211 mds_command: new command op to 4854 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 --> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f7bf43e10 con 0x556f7c186e30 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf85126640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c186e30 0x556f7c1b6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf85126640 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c186e30 0x556f7c1b6020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4f36294f9cccf754 server_cookie=d8d7b06d8da47711 in_seq=0 out_seq=0 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf5ffff640 10 client.5211 ms_handle_connect on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:48.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf5ffff640 1 -- 172.21.15.76:0/3610069665 <== mds.1 v2:172.21.15.76:6834/2829561253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556f7bf43e10 con 0x556f7c186e30 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf5ffff640 10 client.5211 handle_command_reply: tid=0 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 client.5211 shutdown 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c1a5690 msgr2=0x556f7c186420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c1a5690 0x556f7c186420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdf78098d70 tx=0x7fdf7801a0f0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 shutdown_connections 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c1a5690 0x556f7c186420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c186e30 0x556f7c1b6020 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 --2- 172.21.15.76:0/3610069665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c158db0 0x556f7c160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf5ffff640 0 client.5211 ms_handle_reset on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:48.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 >> 172.21.15.76:0/3610069665 conn(0x556f7c186040 msgr2=0x556f7c157f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 shutdown_connections 2024-09-19T04:14:48.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 1 -- 172.21.15.76:0/3610069665 wait complete. 2024-09-19T04:14:48.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.091+0000 7fdf8a446000 20 client.5211 trim_cache size 0 max 16384 2024-09-19T04:14:48.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 Processor -- start 2024-09-19T04:14:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- start start 2024-09-19T04:14:48.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c1a4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c1a3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 0x556f7c1a3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80202c80 con 0x556f7c158db0 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdf80201e50 con 0x556f7c1a5690 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdf80202550 con 0x556f7c1a4d70 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c1a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c1a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43038/0 (socket says 172.21.15.76:43038) 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 -- 172.21.15.76:0/695780600 learned_addr learned my addr 172.21.15.76:0/695780600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf84925640 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c1a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 -- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 msgr2=0x556f7c1a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c1a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 -- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 msgr2=0x556f7c1a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 0x556f7c1a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 -- 172.21.15.76:0/695780600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f7bf67470 con 0x556f7c158db0 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf84925640 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf85126640 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c1a4480 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0054b0 tx=0x7fdf7c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c0c59c64dc01ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf6a7fc640 1 -- 172.21.15.76:0/695780600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c01a040 con 0x556f7c158db0 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf6a7fc640 1 -- 172.21.15.76:0/695780600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c001d40 con 0x556f7c158db0 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf6a7fc640 1 -- 172.21.15.76:0/695780600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c024d00 con 0x556f7c158db0 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- 172.21.15.76:0/695780600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 msgr2=0x556f7c1a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c1a4480 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0054b0 tx=0x7fdf7c013000 comp rx=0 tx=0).stop 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- 172.21.15.76:0/695780600 shutdown_connections 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 0x556f7c1a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 --2- 172.21.15.76:0/695780600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c1a4480 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- 172.21.15.76:0/695780600 >> 172.21.15.76:0/695780600 conn(0x556f7c186040 msgr2=0x556f7c184f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- 172.21.15.76:0/695780600 shutdown_connections 2024-09-19T04:14:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- 172.21.15.76:0/695780600 wait complete. 2024-09-19T04:14:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 -- start start 2024-09-19T04:14:48.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.107+0000 7fdf8a446000 1 Processor -- start 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c160570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c21c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 0x556f7c21cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556f7bfa9d40 con 0x556f7c158db0 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556f7bf86900 con 0x556f7c1a5690 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556f7bfa5d00 con 0x556f7c1a4d70 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 10 client.0 _ll_register_callbacks cb 0x7fdf6c004630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:48.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43046/0 (socket says 172.21.15.76:43046) 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 -- 172.21.15.76:0/1469841355 learned_addr learned my addr 172.21.15.76:0/1469841355 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf84925640 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c21c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 -- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 msgr2=0x556f7c21c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c21c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 -- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 msgr2=0x556f7c21cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85927640 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 0x556f7c21cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 0x556f7c21cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 -- 172.21.15.76:0/1469841355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f7bf3d640 con 0x556f7c158db0 2024-09-19T04:14:48.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf85126640 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c160570 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009010 tx=0x7fdf7c001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba48417d5268f30c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf5ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf5ffff640 1 -- 172.21.15.76:0/1469841355 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c01a050 con 0x556f7c158db0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf5ffff640 1 -- 172.21.15.76:0/1469841355 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fdf7c024da0 con 0x556f7c158db0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf7c001d40 con 0x556f7c158db0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf5ffff640 1 -- 172.21.15.76:0/1469841355 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c007020 con 0x556f7c158db0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 10 client.5217 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x556f7bf413b0 con 0x556f7c158db0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf5ffff640 1 -- 172.21.15.76:0/1469841355 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf7c001d40 con 0x556f7c158db0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf5ffff640 1 -- 172.21.15.76:0/1469841355 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fdf7c020040 con 0x556f7c158db0 2024-09-19T04:14:48.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 10 client.5217 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 10 client.5217 resolve_mds: resolved 2:1 to role '2:1' aka daemon mds.a 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 20 client.5217 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c160ab0 0x556f7c1b7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 4 client.5217 mds_command: new command op to 4854 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 --> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f7bfb7470 con 0x556f7c160ab0 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf84925640 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c160ab0 0x556f7c1b7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf5ffff640 10 client.5217 ms_handle_connect on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:48.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.111+0000 7fdf84925640 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c160ab0 0x556f7c1b7070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87251b788dcaf080 server_cookie=2f13ec93cdd2d25f in_seq=0 out_seq=0 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf5ffff640 1 -- 172.21.15.76:0/1469841355 <== mds.1 v2:172.21.15.76:6834/2829561253 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556f7bfb7470 con 0x556f7c160ab0 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf5ffff640 10 client.5217 handle_command_reply: tid=0 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 client.5217 shutdown 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 msgr2=0x556f7c160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c160570 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c009010 tx=0x7fdf7c001130 comp rx=0 tx=0).stop 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 shutdown_connections 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556f7c1a5690 0x556f7c21cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:6834/2829561253,v1:172.21.15.76:6835/2829561253] conn(0x556f7c160ab0 0x556f7c1b7070 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556f7c1a4d70 0x556f7c21c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 --2- 172.21.15.76:0/1469841355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556f7c158db0 0x556f7c160570 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf5ffff640 0 client.5217 ms_handle_reset on v2:172.21.15.76:6834/2829561253 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 >> 172.21.15.76:0/1469841355 conn(0x556f7c186040 msgr2=0x556f7c1a3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 shutdown_connections 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 1 -- 172.21.15.76:0/1469841355 wait complete. 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8a446000 20 client.5217 trim_cache size 0 max 16384 2024-09-19T04:14:48.130 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:14:48.131 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:14:48.131 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf640678c0 msgr2=0x7fdf64069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf640678c0 0x7fdf64069d80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c007fd0 tx=0x7fdf7c009040 comp rx=0 tx=0).stop 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf80150ab0 msgr2=0x7fdf800a9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf80150ab0 0x7fdf800a9be0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fdf700108c0 tx=0x7fdf70010d80 comp rx=0 tx=0).stop 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 shutdown_connections 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdf640678c0 0x7fdf64069d80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdf801514e0 0x7fdf800ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdf80150ab0 0x7fdf800a9be0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 --2- 172.21.15.76:0/2411837720 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf801506d0 0x7fdf80150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 >> 172.21.15.76:0/2411837720 conn(0x7fdf80073580 msgr2=0x7fdf800b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 shutdown_connections 2024-09-19T04:14:48.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.123+0000 7fdf8738a640 1 -- 172.21.15.76:0/2411837720 wait complete. 2024-09-19T04:14:48.146 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:2 flush journal 2024-09-19T04:14:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.259+0000 7fa1ab4da640 1 Processor -- start 2024-09-19T04:14:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.259+0000 7fa1ab4da640 1 -- start start 2024-09-19T04:14:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1a406c3d0 0x7fa1a409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a40b5c90 con 0x7fa1a40b40c0 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1a40b4580 con 0x7fa1a415e450 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa1a40b4860 con 0x7fa1a406c3d0 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1a8a75640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa1a406c3d0 0x7fa1a409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1a9a77640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa1a40b40c0 0x7fa1a4162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46078/0 (socket says 172.21.15.76:46078) 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1a9a77640 1 -- 172.21.15.76:0/1737441918 learned_addr learned my addr 172.21.15.76:0/1737441918 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1a8a75640 1 -- 172.21.15.76:0/1737441918 >> v1:172.21.15.116:6789/0 conn(0x7fa1a415e450 legacy=0x7fa1a406bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1a8a75640 1 -- 172.21.15.76:0/1737441918 >> v1:172.21.15.76:6789/0 conn(0x7fa1a40b40c0 legacy=0x7fa1a4162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1a8a75640 1 -- 172.21.15.76:0/1737441918 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a40b5f10 con 0x7fa1a406c3d0 2024-09-19T04:14:48.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa19bfff640 1 -- 172.21.15.76:0/1737441918 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1597236704 0 0) 0x7fa1a00014a0 con 0x7fa1a40b40c0 2024-09-19T04:14:48.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/1737441918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a406c3d0 0x7fa1a409aa00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097c0 tx=0x7fa19401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82eee40497be88a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa19bfff640 1 -- 172.21.15.76:0/1737441918 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940b0070 con 0x7fa1a406c3d0 2024-09-19T04:14:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa19bfff640 1 -- 172.21.15.76:0/1737441918 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1940ab600 con 0x7fa1a406c3d0 2024-09-19T04:14:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa19bfff640 1 -- 172.21.15.76:0/1737441918 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940ab9d0 con 0x7fa1a406c3d0 2024-09-19T04:14:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- 172.21.15.76:0/1737441918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a406c3d0 msgr2=0x7fa1a409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/1737441918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a406c3d0 0x7fa1a409aa00 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097c0 tx=0x7fa19401c3d0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- 172.21.15.76:0/1737441918 shutdown_connections 2024-09-19T04:14:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/1737441918 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a406c3d0 0x7fa1a409aa00 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- 172.21.15.76:0/1737441918 >> 172.21.15.76:0/1737441918 conn(0x7fa1a40b6800 msgr2=0x7fa1a40b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- 172.21.15.76:0/1737441918 shutdown_connections 2024-09-19T04:14:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.263+0000 7fa1ab4da640 1 -- 172.21.15.76:0/1737441918 wait complete. 2024-09-19T04:14:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 Processor -- start 2024-09-19T04:14:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 -- start start 2024-09-19T04:14:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1a406c3d0 0x7fa1a41549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1a40b40c0 0x7fa1a4152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e450 0x7fa1a4157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a415f9c0 con 0x7fa1a415e450 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a415f840 con 0x7fa1a406c3d0 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a4162e90 con 0x7fa1a40b40c0 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e450 0x7fa1a4157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e450 0x7fa1a4157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43076/0 (socket says 172.21.15.76:43076) 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 -- 172.21.15.76:0/4201596941 learned_addr learned my addr 172.21.15.76:0/4201596941 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1a40b40c0 0x7fa1a4152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9276640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1a406c3d0 0x7fa1a41549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 -- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1a40b40c0 msgr2=0x7fa1a4152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1a40b40c0 0x7fa1a4152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 -- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1a406c3d0 msgr2=0x7fa1a41549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1a406c3d0 0x7fa1a41549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 -- 172.21.15.76:0/4201596941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a4157090 con 0x7fa1a415e450 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e450 0x7fa1a4157b30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000c800 tx=0x7fa1a000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa199ffb640 1 -- 172.21.15.76:0/4201596941 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0018070 con 0x7fa1a415e450 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa199ffb640 1 -- 172.21.15.76:0/4201596941 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1a000a5e0 con 0x7fa1a415e450 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a4153160 con 0x7fa1a415e450 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa199ffb640 1 -- 172.21.15.76:0/4201596941 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a000a9d0 con 0x7fa1a415e450 2024-09-19T04:14:48.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.267+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a4152e50 con 0x7fa1a415e450 2024-09-19T04:14:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa199ffb640 1 -- 172.21.15.76:0/4201596941 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa1a001b070 con 0x7fa1a415e450 2024-09-19T04:14:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa199ffb640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1880678c0 0x7fa188069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa199ffb640 1 -- 172.21.15.76:0/4201596941 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa1a0085450 con 0x7fa1a415e450 2024-09-19T04:14:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a9276640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1880678c0 0x7fa188069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a9276640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1880678c0 0x7fa188069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00b810 tx=0x7fa19c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 Processor -- start 2024-09-19T04:14:48.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 -- start start 2024-09-19T04:14:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d200150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d200690 0x55801d1b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d1b3db0 0x55801d1b4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55801d09d960 con 0x55801d200690 2024-09-19T04:14:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55801cfe1900 con 0x55801d1b3db0 2024-09-19T04:14:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55801d004d40 con 0x55801d1ffd70 2024-09-19T04:14:48.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d200690 0x55801d1b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d200690 0x55801d1b3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43086/0 (socket says 172.21.15.76:43086) 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 -- 172.21.15.76:0/1723142094 learned_addr learned my addr 172.21.15.76:0/1723142094 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a9276640 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d200150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d1b3db0 0x55801d1b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 -- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 msgr2=0x55801d200150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d200150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 -- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d1b3db0 msgr2=0x55801d1b4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d1b3db0 0x55801d1b4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 -- 172.21.15.76:0/1723142094 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801cfab580 con 0x55801d200690 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d1b3db0 0x55801d1b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d200690 0x55801d1b3870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097c0 tx=0x7fa1940a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dac3b1d960a854c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa18effd640 1 -- 172.21.15.76:0/1723142094 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940bc070 con 0x55801d200690 2024-09-19T04:14:48.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa18effd640 1 -- 172.21.15.76:0/1723142094 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1940ab950 con 0x55801d200690 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa18effd640 1 -- 172.21.15.76:0/1723142094 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940b0040 con 0x55801d200690 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 -- 172.21.15.76:0/1723142094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d200690 msgr2=0x55801d1b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d200690 0x55801d1b3870 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097c0 tx=0x7fa1940a6000 comp rx=0 tx=0).stop 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.271+0000 7fa1ae596000 1 -- 172.21.15.76:0/1723142094 shutdown_connections 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d1b3db0 0x55801d1b4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d200690 0x55801d1b3870 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 --2- 172.21.15.76:0/1723142094 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d200150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- 172.21.15.76:0/1723142094 >> 172.21.15.76:0/1723142094 conn(0x55801d1e1040 msgr2=0x55801d1e3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- 172.21.15.76:0/1723142094 shutdown_connections 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- 172.21.15.76:0/1723142094 wait complete. 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- start start 2024-09-19T04:14:48.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 Processor -- start 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1b3db0 0x55801d206ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d207220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d207760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55801d000d00 con 0x55801d1b3db0 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55801cfbfaa0 con 0x55801d200690 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55801cf9e340 con 0x55801d1ffd70 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 10 client.0 _ll_register_callbacks cb 0x55801d1a1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1b3db0 0x55801d206ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1b3db0 0x55801d206ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43100/0 (socket says 172.21.15.76:43100) 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 -- 172.21.15.76:0/2940096055 learned_addr learned my addr 172.21.15.76:0/2940096055 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d207760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d207220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 -- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 msgr2=0x55801d207220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d207220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 -- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 msgr2=0x55801d207760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d207760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 -- 172.21.15.76:0/2940096055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801cfa7dc0 con 0x55801d1b3db0 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d207760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d207220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a9276640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1b3db0 0x55801d206ce0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00dea0 tx=0x7fa19c004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdbce25f80742483 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa183fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa183fff640 1 -- 172.21.15.76:0/2940096055 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c014e40 con 0x55801d1b3db0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa183fff640 1 -- 172.21.15.76:0/2940096055 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa19c00f040 con 0x55801d1b3db0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1940ab950 con 0x55801d1b3db0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa183fff640 1 -- 172.21.15.76:0/2940096055 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa19c005920 con 0x55801d1b3db0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 10 client.5232 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55801cfa47f0 con 0x55801d1b3db0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa183fff640 1 -- 172.21.15.76:0/2940096055 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1940ab950 con 0x55801d1b3db0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa183fff640 1 -- 172.21.15.76:0/2940096055 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fa19c030080 con 0x55801d1b3db0 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 10 client.5232 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 10 client.5232 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.d 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 20 client.5232 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d20f0b0 0x55801d2114a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 4 client.5232 mds_command: new command op to 4666 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 --> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55801cf9ee10 con 0x55801d20f0b0 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d20f0b0 0x55801d2114a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa183fff640 10 client.5232 ms_handle_connect on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.275+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d20f0b0 0x55801d2114a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5cec69285627851f server_cookie=dc663e80078e8d8 in_seq=0 out_seq=0 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa183fff640 1 -- 172.21.15.76:0/2940096055 <== mds.2 v2:172.21.15.116:6832/2447208752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55801cf9ee10 con 0x55801d20f0b0 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa183fff640 10 client.5232 handle_command_reply: tid=0 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 client.5232 shutdown 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1b3db0 msgr2=0x55801d206ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1b3db0 0x55801d206ce0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00dea0 tx=0x7fa19c004690 comp rx=0 tx=0).stop 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 shutdown_connections 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d207760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d20f0b0 0x55801d2114a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1ffd70 0x55801d207220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 --2- 172.21.15.76:0/2940096055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1b3db0 0x55801d206ce0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa183fff640 0 client.5232 ms_handle_reset on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:48.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 >> 172.21.15.76:0/2940096055 conn(0x55801d1e1040 msgr2=0x55801d1b2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 shutdown_connections 2024-09-19T04:14:48.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 1 -- 172.21.15.76:0/2940096055 wait complete. 2024-09-19T04:14:48.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.279+0000 7fa1ae596000 20 client.5232 trim_cache size 0 max 16384 2024-09-19T04:14:48.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 Processor -- start 2024-09-19T04:14:48.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- start start 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d1bbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d1bc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a42046e0 con 0x55801d1ffd70 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a42038b0 con 0x55801d200690 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a4203fb0 con 0x55801d1b3db0 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d1bbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d1bbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43114/0 (socket says 172.21.15.76:43114) 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 -- 172.21.15.76:0/189681920 learned_addr learned my addr 172.21.15.76:0/189681920 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a9276640 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 -- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 msgr2=0x55801d2148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 -- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 msgr2=0x55801d1bc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d1bc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 -- 172.21.15.76:0/189681920 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55801cfc2470 con 0x55801d1ffd70 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a9276640 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d1bbc00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097c0 tx=0x7fa194004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b980b1d47a3acfc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa18e7fc640 1 -- 172.21.15.76:0/189681920 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940bc070 con 0x55801d1ffd70 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa18e7fc640 1 -- 172.21.15.76:0/189681920 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1940c7040 con 0x55801d1ffd70 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa18e7fc640 1 -- 172.21.15.76:0/189681920 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940b0040 con 0x55801d1ffd70 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- 172.21.15.76:0/189681920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 msgr2=0x55801d1bbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d1bbc00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa1940097c0 tx=0x7fa194004740 comp rx=0 tx=0).stop 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- 172.21.15.76:0/189681920 shutdown_connections 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d1bc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d1bbc00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 --2- 172.21.15.76:0/189681920 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- 172.21.15.76:0/189681920 >> 172.21.15.76:0/189681920 conn(0x55801d1e1040 msgr2=0x55801d20cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- 172.21.15.76:0/189681920 shutdown_connections 2024-09-19T04:14:48.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- 172.21.15.76:0/189681920 wait complete. 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 -- start start 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.295+0000 7fa1ae596000 1 Processor -- start 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d2137e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d213d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55801d1999f0 con 0x55801d1ffd70 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55801d044780 con 0x55801d200690 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55801cfcc400 con 0x55801d1b3db0 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 10 client.0 _ll_register_callbacks cb 0x7fa19c00b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d2137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d2137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43122/0 (socket says 172.21.15.76:43122) 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 -- 172.21.15.76:0/3673337251 learned_addr learned my addr 172.21.15.76:0/3673337251 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:48.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a9276640 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 -- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 msgr2=0x55801d2729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a9a77640 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d213d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 -- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 msgr2=0x55801d213d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d213d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 -- 172.21.15.76:0/3673337251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55801cf98640 con 0x55801d1ffd70 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a8a75640 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d2137e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa1940c4040 tx=0x7fa19401ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fef82432b0d088 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa183fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa183fff640 1 -- 172.21.15.76:0/3673337251 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1940b0030 con 0x55801d1ffd70 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1940c7040 con 0x55801d1ffd70 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa183fff640 1 -- 172.21.15.76:0/3673337251 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fa1940c2480 con 0x55801d1ffd70 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa183fff640 1 -- 172.21.15.76:0/3673337251 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa1940d6050 con 0x55801d1ffd70 2024-09-19T04:14:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 10 client.5238 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55801cf9c3b0 con 0x55801d1ffd70 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa183fff640 1 -- 172.21.15.76:0/3673337251 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1940c7040 con 0x55801d1ffd70 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa183fff640 1 -- 172.21.15.76:0/3673337251 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fa1940d9020 con 0x55801d1ffd70 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 10 client.5238 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 10 client.5238 resolve_mds: resolved 2:2 to role '2:2' aka daemon mds.d 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 20 client.5238 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d214260 0x55801d214670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 4 client.5238 mds_command: new command op to 4666 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 --> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55801d012470 con 0x55801d214260 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a9276640 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d214260 0x55801d214670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa183fff640 10 client.5238 ms_handle_connect on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.299+0000 7fa1a9276640 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d214260 0x55801d214670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b32488147a34985f server_cookie=b9012e51150da15f in_seq=0 out_seq=0 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa183fff640 1 -- 172.21.15.76:0/3673337251 <== mds.2 v2:172.21.15.116:6832/2447208752 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55801d012470 con 0x55801d214260 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa183fff640 10 client.5238 handle_command_reply: tid=0 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 client.5238 shutdown 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 msgr2=0x55801d2137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d2137e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa1940c4040 tx=0x7fa19401ac10 comp rx=0 tx=0).stop 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 shutdown_connections 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55801d200690 0x55801d213d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55801d1ffd70 0x55801d2137e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:6832/2447208752,v1:172.21.15.116:6833/2447208752] conn(0x55801d214260 0x55801d214670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 --2- 172.21.15.76:0/3673337251 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55801d1b3db0 0x55801d2729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa183fff640 0 client.5238 ms_handle_reset on v2:172.21.15.116:6832/2447208752 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 >> 172.21.15.76:0/3673337251 conn(0x55801d1e1040 msgr2=0x55801d1ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 shutdown_connections 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 1 -- 172.21.15.76:0/3673337251 wait complete. 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ae596000 20 client.5238 trim_cache size 0 max 16384 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:48.320 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1880678c0 msgr2=0x7fa188069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1880678c0 0x7fa188069d80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00b810 tx=0x7fa19c008040 comp rx=0 tx=0).stop 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e450 msgr2=0x7fa1a4157b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e450 0x7fa1a4157b30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000c800 tx=0x7fa1a000ccc0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 shutdown_connections 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa1880678c0 0x7fa188069d80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1a415e450 0x7fa1a4157b30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1a40b40c0 0x7fa1a4152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 --2- 172.21.15.76:0/4201596941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1a406c3d0 0x7fa1a41549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 >> 172.21.15.76:0/4201596941 conn(0x7fa1a40b6800 msgr2=0x7fa1a415bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 shutdown_connections 2024-09-19T04:14:48.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48.315+0000 7fa1ab4da640 1 -- 172.21.15.76:0/4201596941 wait complete. 2024-09-19T04:14:48.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:48.336 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-19T04:14:48.366 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/ 2024-09-19T04:14:48.366 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4 2024-09-19T04:14:48.366 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5 2024-09-19T04:14:48.366 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6 2024-09-19T04:14:48.366 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-19T04:14:48.366 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-19T04:14:48.367 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:14:48.367 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:14:48.367 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:14:48.367 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:48.367 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-19T04:14:48.367 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:14:48.468 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627778 2024-09-19T04:14:48.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:48.477 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:14:48.478 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:14:48.478 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:14:48.478 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:48.478 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-19T04:14:48.478 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:14:48.578 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627779 2024-09-19T04:14:48.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:48.587 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:14:48.588 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:14:48.588 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:14:48.588 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:48.588 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-19T04:14:48.588 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:14:48.686 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627780 2024-09-19T04:14:48.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:48.694 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:14:48.695 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:14:48.695 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:14:48.695 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:48.695 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-19T04:14:48.695 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:14:48.801 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627781 2024-09-19T04:14:48.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:48.810 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:14:48.810 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:14:48.810 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:14:48.810 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:48.810 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-19T04:14:48.810 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:14:48.911 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627782 2024-09-19T04:14:48.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:48.921 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:14:48.921 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:14:48.921 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:14:48.921 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:14:48.921 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-19T04:14:48.921 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:14:49.028 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627783 2024-09-19T04:14:49.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:14:49.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 tag path /d1/d2/d3 tag123 2024-09-19T04:14:49.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.199+0000 7f2fc65cf640 1 Processor -- start 2024-09-19T04:14:49.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.199+0000 7f2fc65cf640 1 -- start start 2024-09-19T04:14:49.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.199+0000 7f2fc65cf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc00b3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.199+0000 7f2fc65cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2fc00b37c0 0x7f2fc015fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.199+0000 7f2fc65cf640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2fc01533f0 con 0x7f2fc0153010 2024-09-19T04:14:49.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.199+0000 7f2fc65cf640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2fc00b3c30 con 0x7f2fc00b37c0 2024-09-19T04:14:49.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.199+0000 7f2fc65cf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc015d4e0 con 0x7f2fc015d090 2024-09-19T04:14:49.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2fc00b37c0 0x7f2fc015fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2fc00b37c0 0x7f2fc015fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43130/0 (socket says 172.21.15.76:43130) 2024-09-19T04:14:49.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/2355358802 learned_addr learned my addr 172.21.15.76:0/2355358802 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/2355358802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc00b3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/2355358802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 msgr2=0x7f2fc00b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 --2- 172.21.15.76:0/2355358802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc00b3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/2355358802 >> v1:172.21.15.116:6789/0 conn(0x7f2fc0153010 legacy=0x7f2fc0072dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/2355358802 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc0073850 con 0x7f2fc00b37c0 2024-09-19T04:14:49.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc4b6c640 1 --2- 172.21.15.76:0/2355358802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc015fb30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009e50 tx=0x7f2fb801c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95abef06bf061218 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fbeffd640 1 -- 172.21.15.76:0/2355358802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80a5580 con 0x7f2fc00b37c0 2024-09-19T04:14:49.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fbeffd640 1 -- 172.21.15.76:0/2355358802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb80a5720 con 0x7f2fc00b37c0 2024-09-19T04:14:49.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fbeffd640 1 -- 172.21.15.76:0/2355358802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80a5af0 con 0x7f2fc00b37c0 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 -- 172.21.15.76:0/2355358802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 msgr2=0x7f2fc015fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/2355358802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc015fb30 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2fb8009e50 tx=0x7f2fb801c810 comp rx=0 tx=0).stop 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 -- 172.21.15.76:0/2355358802 shutdown_connections 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/2355358802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc015fb30 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/2355358802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc00b3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 -- 172.21.15.76:0/2355358802 >> 172.21.15.76:0/2355358802 conn(0x7f2fc00b6870 msgr2=0x7f2fc00b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 -- 172.21.15.76:0/2355358802 shutdown_connections 2024-09-19T04:14:49.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.203+0000 7f2fc65cf640 1 -- 172.21.15.76:0/2355358802 wait complete. 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 Processor -- start 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 -- start start 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc01f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc0153010 0x7f2fc0149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc0155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc006fe00 con 0x7f2fc00b37c0 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc015c540 con 0x7f2fc0153010 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc00734e0 con 0x7f2fc015d090 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc01f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc01f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43132/0 (socket says 172.21.15.76:43132) 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 -- 172.21.15.76:0/292138612 learned_addr learned my addr 172.21.15.76:0/292138612 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc0153010 0x7f2fc0149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc4b6c640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc0155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 -- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 msgr2=0x7f2fc0155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc0155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 -- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc0153010 msgr2=0x7f2fc0149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc0153010 0x7f2fc0149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 -- 172.21.15.76:0/292138612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc00b44d0 con 0x7f2fc00b37c0 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbffff640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc01f6e60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000c450 tx=0x7f2fb000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbcff9640 1 -- 172.21.15.76:0/292138612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0012830 con 0x7f2fc00b37c0 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbcff9640 1 -- 172.21.15.76:0/292138612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb0017070 con 0x7f2fc00b37c0 2024-09-19T04:14:49.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc00b47b0 con 0x7f2fc00b37c0 2024-09-19T04:14:49.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fbcff9640 1 -- 172.21.15.76:0/292138612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0012c30 con 0x7f2fc00b37c0 2024-09-19T04:14:49.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.207+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc00b4d70 con 0x7f2fc00b37c0 2024-09-19T04:14:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbcff9640 1 -- 172.21.15.76:0/292138612 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2fb001c460 con 0x7f2fc00b37c0 2024-09-19T04:14:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbcff9640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2fac0678c0 0x7f2fac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbcff9640 1 -- 172.21.15.76:0/292138612 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2fb0085bd0 con 0x7f2fc00b37c0 2024-09-19T04:14:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2fac0678c0 0x7f2fac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2fac0678c0 0x7f2fac069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2fc014dcc0 tx=0x7f2fb4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 Processor -- start 2024-09-19T04:14:49.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- start start 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b393c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 0x55bd1b349ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b34a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b197b20 con 0x55bd1b393840 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b152870 con 0x55bd1b394160 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bd1b193ae0 con 0x55bd1b34a000 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b393c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b393c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43138/0 (socket says 172.21.15.76:43138) 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 -- 172.21.15.76:0/162556194 learned_addr learned my addr 172.21.15.76:0/162556194 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc4b6c640 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b34a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 0x55bd1b349ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 -- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 msgr2=0x55bd1b34a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b34a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 -- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 msgr2=0x55bd1b349ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 0x55bd1b349ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 -- 172.21.15.76:0/162556194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1b16a940 con 0x55bd1b393840 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fbffff640 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b393c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fb005e530 tx=0x7f2fb0055f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f830b58aab241e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fa5ffb640 1 -- 172.21.15.76:0/162556194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0054180 con 0x55bd1b393840 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fa5ffb640 1 -- 172.21.15.76:0/162556194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb00524f0 con 0x55bd1b393840 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fa5ffb640 1 -- 172.21.15.76:0/162556194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb005e7f0 con 0x55bd1b393840 2024-09-19T04:14:49.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- 172.21.15.76:0/162556194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 msgr2=0x55bd1b393c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b393c20 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2fb005e530 tx=0x7f2fb0055f40 comp rx=0 tx=0).stop 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- 172.21.15.76:0/162556194 shutdown_connections 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b34a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 0x55bd1b349ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 --2- 172.21.15.76:0/162556194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b393c20 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- 172.21.15.76:0/162556194 >> 172.21.15.76:0/162556194 conn(0x55bd1b350160 msgr2=0x55bd1b3483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- 172.21.15.76:0/162556194 shutdown_connections 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.211+0000 7f2fc968b000 1 -- 172.21.15.76:0/162556194 wait complete. 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- start start 2024-09-19T04:14:49.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 Processor -- start 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b380710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b34e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 0x55bd1b34ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b1648e0 con 0x55bd1b393840 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b13b9b0 con 0x55bd1b394160 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bd1b15f260 con 0x55bd1b34a000 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 10 client.0 _ll_register_callbacks cb 0x55bd1b335540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:49.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b34e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b34e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43144/0 (socket says 172.21.15.76:43144) 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 -- 172.21.15.76:0/3787503640 learned_addr learned my addr 172.21.15.76:0/3787503640 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbffff640 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b380710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 -- 172.21.15.76:0/3787503640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 msgr2=0x55bd1b380710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b380710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 -- 172.21.15.76:0/3787503640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 msgr2=0x55bd1b34ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 0x55bd1b34ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 -- 172.21.15.76:0/3787503640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1b152ac0 con 0x55bd1b393840 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b34e950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400fff0 tx=0x7f2fb4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b00c1de91a4139 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 1 -- 172.21.15.76:0/3787503640 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4015820 con 0x55bd1b393840 2024-09-19T04:14:49.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 1 -- 172.21.15.76:0/3787503640 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2fb40159c0 con 0x55bd1b393840 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb00524f0 con 0x55bd1b393840 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 1 -- 172.21.15.76:0/3787503640 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2fb402e050 con 0x55bd1b393840 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 10 client.5250 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55bd1b137810 con 0x55bd1b393840 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 1 -- 172.21.15.76:0/3787503640 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb00524f0 con 0x55bd1b393840 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 1 -- 172.21.15.76:0/3787503640 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7f2fb4031020 con 0x55bd1b393840 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 10 client.5250 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 10 client.5250 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 20 client.5250 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b381710 0x55bd1b3a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 4 client.5250 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:49.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bd1b131e30 con 0x55bd1b381710 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbffff640 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b381710 0x55bd1b3a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fbffff640 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b381710 0x55bd1b3a48e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=415af75a70dfd7a6 server_cookie=8eefeb1896a9dcbe in_seq=0 out_seq=0 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 10 client.5250 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 1 -- 172.21.15.76:0/3787503640 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bd1b131e30 con 0x55bd1b381710 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 10 client.5250 handle_command_reply: tid=0 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 client.5250 shutdown 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 msgr2=0x55bd1b34e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b34e950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400fff0 tx=0x7f2fb4004740 comp rx=0 tx=0).stop 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 shutdown_connections 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b381710 0x55bd1b3a48e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b394160 0x55bd1b34ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2f9effd640 0 client.5250 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b34e950 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3787503640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b380710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 >> 172.21.15.76:0/3787503640 conn(0x55bd1b350160 msgr2=0x55bd1b3483a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 shutdown_connections 2024-09-19T04:14:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 1 -- 172.21.15.76:0/3787503640 wait complete. 2024-09-19T04:14:49.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.215+0000 7f2fc968b000 20 client.5250 trim_cache size 0 max 16384 2024-09-19T04:14:49.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 Processor -- start 2024-09-19T04:14:49.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 -- start start 2024-09-19T04:14:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b350160 0x55bd1b402a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b402f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b393840 0x55bd1b4034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0203ed0 con 0x55bd1b350160 2024-09-19T04:14:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc02030a0 con 0x55bd1b393840 2024-09-19T04:14:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2fc02037a0 con 0x55bd1b34a000 2024-09-19T04:14:49.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b350160 0x55bd1b402a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b350160 0x55bd1b402a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43152/0 (socket says 172.21.15.76:43152) 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 -- 172.21.15.76:0/2385629909 learned_addr learned my addr 172.21.15.76:0/2385629909 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 -- 172.21.15.76:0/2385629909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 msgr2=0x55bd1b402f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 --2- 172.21.15.76:0/2385629909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b402f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 -- 172.21.15.76:0/2385629909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b393840 msgr2=0x55bd1b4034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 --2- 172.21.15.76:0/2385629909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b393840 0x55bd1b4034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 -- 172.21.15.76:0/2385629909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd1b155240 con 0x55bd1b350160 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fbffff640 1 --2- 172.21.15.76:0/2385629909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b350160 0x55bd1b402a20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00522e0 tx=0x7f2fb0055af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02cfdef72f5b7a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fa57fa640 1 -- 172.21.15.76:0/2385629909 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0078030 con 0x55bd1b350160 2024-09-19T04:14:49.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.235+0000 7f2fa57fa640 1 -- 172.21.15.76:0/2385629909 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb00524f0 con 0x55bd1b350160 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fa57fa640 1 -- 172.21.15.76:0/2385629909 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0052120 con 0x55bd1b350160 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/2385629909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b350160 msgr2=0x55bd1b402a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- 172.21.15.76:0/2385629909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b350160 0x55bd1b402a20 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00522e0 tx=0x7f2fb0055af0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/2385629909 shutdown_connections 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- 172.21.15.76:0/2385629909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b393840 0x55bd1b4034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- 172.21.15.76:0/2385629909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b34a000 0x55bd1b402f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- 172.21.15.76:0/2385629909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b350160 0x55bd1b402a20 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/2385629909 >> 172.21.15.76:0/2385629909 conn(0x55bd1b34bc30 msgr2=0x55bd1b392a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/2385629909 shutdown_connections 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/2385629909 wait complete. 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- start start 2024-09-19T04:14:49.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 Processor -- start 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b34a000 0x55bd1b406c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b350160 0x55bd1b4071b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b3c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bd1b164520 con 0x55bd1b393840 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0151530 con 0x55bd1b34a000 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bd1b196e30 con 0x55bd1b350160 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 10 client.0 _ll_register_callbacks cb 0x7f2fb400b5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b3c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b3c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43154/0 (socket says 172.21.15.76:43154) 2024-09-19T04:14:49.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/3734077947 learned_addr learned my addr 172.21.15.76:0/3734077947 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b350160 msgr2=0x55bd1b4071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fbf7fe640 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b350160 0x55bd1b4071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fbffff640 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b34a000 0x55bd1b406c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b350160 0x55bd1b4071b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b34a000 msgr2=0x55bd1b406c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b34a000 0x55bd1b406c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 -- 172.21.15.76:0/3734077947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd1b21a330 con 0x55bd1b393840 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc4b6c640 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b3c1c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2fb801d0b0 tx=0x7f2fb8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e6515ece37e020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2f9effd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2f9effd640 1 -- 172.21.15.76:0/3734077947 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80a5840 con 0x55bd1b393840 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2f9effd640 1 -- 172.21.15.76:0/3734077947 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7f2fb80c1c30 con 0x55bd1b393840 2024-09-19T04:14:49.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb00524f0 con 0x55bd1b393840 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2f9effd640 1 -- 172.21.15.76:0/3734077947 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2fb80c1e50 con 0x55bd1b393840 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 10 client.5256 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55bd1ae52400 con 0x55bd1b393840 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2f9effd640 1 -- 172.21.15.76:0/3734077947 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fb00524f0 con 0x55bd1b393840 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2f9effd640 1 -- 172.21.15.76:0/3734077947 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7f2fb80c8050 con 0x55bd1b393840 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 10 client.5256 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 10 client.5256 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 20 client.5256 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b394160 0x55bd1b3c2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 4 client.5256 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}] 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "tag path", "path": "/d1/d2/d3", "tag": "tag123"}) -- 0x55bd1b1a5250 con 0x55bd1b394160 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fbffff640 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b394160 0x55bd1b3c2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2f9effd640 10 client.5256 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.239+0000 7f2fbffff640 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b394160 0x55bd1b3c2150 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef1ea13dcb8a66c server_cookie=185a2717ac10ef92 in_seq=0 out_seq=0 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2f9effd640 1 -- 172.21.15.76:0/3734077947 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55bd1b1a5250 con 0x55bd1b394160 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2f9effd640 10 client.5256 handle_command_reply: tid=0 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 client.5256 shutdown 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 msgr2=0x55bd1b3c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b3c1c10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2fb801d0b0 tx=0x7f2fb8009510 comp rx=0 tx=0).stop 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 shutdown_connections 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55bd1b394160 0x55bd1b3c2150 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd1b393840 0x55bd1b3c1c10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2f9effd640 0 client.5256 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd1b350160 0x55bd1b4071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 --2- 172.21.15.76:0/3734077947 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd1b34a000 0x55bd1b406c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 >> 172.21.15.76:0/3734077947 conn(0x55bd1b34bc30 msgr2=0x55bd1b393110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 shutdown_connections 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 1 -- 172.21.15.76:0/3734077947 wait complete. 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc968b000 20 client.5256 trim_cache size 0 max 16384 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "tag123", 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-09-19T04:14:49.254 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2fac0678c0 msgr2=0x7f2fac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2fac0678c0 0x7f2fac069d80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2fc014dcc0 tx=0x7f2fb4008040 comp rx=0 tx=0).stop 2024-09-19T04:14:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 msgr2=0x7f2fc01f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc01f6e60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000c450 tx=0x7f2fb000c910 comp rx=0 tx=0).stop 2024-09-19T04:14:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 shutdown_connections 2024-09-19T04:14:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2fac0678c0 0x7f2fac069d80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2fc015d090 0x7f2fc0155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2fc0153010 0x7f2fc0149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 --2- 172.21.15.76:0/292138612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2fc00b37c0 0x7f2fc01f6e60 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2fb000c450 tx=0x7f2fb000c910 comp rx=0 tx=0).stop 2024-09-19T04:14:49.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 >> 172.21.15.76:0/292138612 conn(0x7f2fc00b6870 msgr2=0x7f2fc006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 shutdown_connections 2024-09-19T04:14:49.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.243+0000 7f2fc65cf640 1 -- 172.21.15.76:0/292138612 wait complete. 2024-09-19T04:14:49.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:14:49.265 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-19T04:14:49.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.375+0000 7fc46abdb640 1 Processor -- start 2024-09-19T04:14:49.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.375+0000 7fc46abdb640 1 -- start start 2024-09-19T04:14:49.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc464161a60 0x7fc4640d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4640d2e90 0x7fc4640cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4640d32b0 con 0x7fc4640cf450 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc4640d0300 con 0x7fc4640d2e90 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc46406e6c0 con 0x7fc464161a60 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4640d2e90 0x7fc4640cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4640d2e90 0x7fc4640cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43158/0 (socket says 172.21.15.76:43158) 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 -- 172.21.15.76:0/2105857393 learned_addr learned my addr 172.21.15.76:0/2105857393 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc468977640 1 --2- 172.21.15.76:0/2105857393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc464161a60 0x7fc4640d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 -- 172.21.15.76:0/2105857393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc464161a60 msgr2=0x7fc4640d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 --2- 172.21.15.76:0/2105857393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc464161a60 0x7fc4640d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 -- 172.21.15.76:0/2105857393 >> v1:172.21.15.116:6789/0 conn(0x7fc4640cf450 legacy=0x7fc4640d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 -- 172.21.15.76:0/2105857393 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4640b0f60 con 0x7fc4640d2e90 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45bfff640 1 --2- 172.21.15.76:0/2105857393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640d2e90 0x7fc4640cef10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc454009870 tx=0x7fc45401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41168cddda9b59e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45b7fe640 1 -- 172.21.15.76:0/2105857393 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4540aa600 con 0x7fc4640d2e90 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45b7fe640 1 -- 172.21.15.76:0/2105857393 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4540af070 con 0x7fc4640d2e90 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc45b7fe640 1 -- 172.21.15.76:0/2105857393 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4540aa9c0 con 0x7fc4640d2e90 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- 172.21.15.76:0/2105857393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640d2e90 msgr2=0x7fc4640cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2105857393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640d2e90 0x7fc4640cef10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc454009870 tx=0x7fc45401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- 172.21.15.76:0/2105857393 shutdown_connections 2024-09-19T04:14:49.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2105857393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640d2e90 0x7fc4640cef10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2105857393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc464161a60 0x7fc4640d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- 172.21.15.76:0/2105857393 >> 172.21.15.76:0/2105857393 conn(0x7fc46408bf90 msgr2=0x7fc46409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- 172.21.15.76:0/2105857393 shutdown_connections 2024-09-19T04:14:49.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.379+0000 7fc46abdb640 1 -- 172.21.15.76:0/2105857393 wait complete. 2024-09-19T04:14:49.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 Processor -- start 2024-09-19T04:14:49.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 -- start start 2024-09-19T04:14:49.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640cf450 0x7fc46415fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4640d2e90 0x7fc4640ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc464161a60 0x7fc4640b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc46406d190 con 0x7fc4640cf450 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc4640d1930 con 0x7fc464161a60 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc4640cf880 con 0x7fc4640d2e90 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640cf450 0x7fc46415fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640cf450 0x7fc46415fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43172/0 (socket says 172.21.15.76:43172) 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 -- 172.21.15.76:0/2960315847 learned_addr learned my addr 172.21.15.76:0/2960315847 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 -- 172.21.15.76:0/2960315847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4640d2e90 msgr2=0x7fc4640ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4640d2e90 0x7fc4640ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 -- 172.21.15.76:0/2960315847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc464161a60 msgr2=0x7fc4640b4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc469178640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc464161a60 0x7fc4640b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc464161a60 0x7fc4640b4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 -- 172.21.15.76:0/2960315847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4640cd3e0 con 0x7fc4640cf450 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc468977640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640cf450 0x7fc46415fa50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc46000e8a0 tx=0x7fc46000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc4597fa640 1 -- 172.21.15.76:0/2960315847 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460017070 con 0x7fc4640cf450 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc4597fa640 1 -- 172.21.15.76:0/2960315847 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46000c940 con 0x7fc4640cf450 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4640cd6c0 con 0x7fc4640cf450 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc4597fa640 1 -- 172.21.15.76:0/2960315847 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc46000ccf0 con 0x7fc4640cf450 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.383+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4640cdc80 con 0x7fc4640cf450 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc4597fa640 1 -- 172.21.15.76:0/2960315847 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc46001a070 con 0x7fc4640cf450 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc4597fa640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4500678c0 0x7fc450069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc4597fa640 1 -- 172.21.15.76:0/2960315847 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc4600860b0 con 0x7fc4640cf450 2024-09-19T04:14:49.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4500678c0 0x7fc450069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4500678c0 0x7fc450069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4540ae000 tx=0x7fc454004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 Processor -- start 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- start start 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9f8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9f8690 0x55b0ab9ab870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9abdb0 0x55b0ab9ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b0ab895960 con 0x55b0ab9f8690 2024-09-19T04:14:49.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b0ab7d9900 con 0x55b0ab9f7d70 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b0ab7fcd40 con 0x55b0ab9abdb0 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9f8690 0x55b0ab9ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9f8690 0x55b0ab9ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43188/0 (socket says 172.21.15.76:43188) 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 -- 172.21.15.76:0/172270265 learned_addr learned my addr 172.21.15.76:0/172270265 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc468977640 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 -- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9abdb0 msgr2=0x55b0ab9ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc469178640 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9abdb0 0x55b0ab9ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9abdb0 0x55b0ab9ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 -- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 msgr2=0x55b0ab9f8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9f8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 -- 172.21.15.76:0/172270265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ab7a3580 con 0x55b0ab9f8690 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc468977640 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc45bfff640 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9f8690 0x55b0ab9ab870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc4540ac040 tx=0x7fc4540a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b11ccb6eee93c81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc44a7fc640 1 -- 172.21.15.76:0/172270265 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4540c4070 con 0x55b0ab9f8690 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc44a7fc640 1 -- 172.21.15.76:0/172270265 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4540ba050 con 0x55b0ab9f8690 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc44a7fc640 1 -- 172.21.15.76:0/172270265 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc454004430 con 0x55b0ab9f8690 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- 172.21.15.76:0/172270265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9f8690 msgr2=0x55b0ab9ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9f8690 0x55b0ab9ab870 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc4540ac040 tx=0x7fc4540a9000 comp rx=0 tx=0).stop 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- 172.21.15.76:0/172270265 shutdown_connections 2024-09-19T04:14:49.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9abdb0 0x55b0ab9ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9f8690 0x55b0ab9ab870 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 --2- 172.21.15.76:0/172270265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9f8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- 172.21.15.76:0/172270265 >> 172.21.15.76:0/172270265 conn(0x55b0ab9d9040 msgr2=0x55b0ab9db430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- 172.21.15.76:0/172270265 shutdown_connections 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.387+0000 7fc46dc97000 1 -- 172.21.15.76:0/172270265 wait complete. 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- start start 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 Processor -- start 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9fece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9ff220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0ab9ff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b0ab7f8d00 con 0x55b0ab9abdb0 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b0ab7b7aa0 con 0x55b0ab9f7d70 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b0ab796340 con 0x55b0ab9f8690 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 10 client.0 _ll_register_callbacks cb 0x55b0ab999760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9fece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9fece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43194/0 (socket says 172.21.15.76:43194) 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 -- 172.21.15.76:0/3679865678 learned_addr learned my addr 172.21.15.76:0/3679865678 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc45bfff640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9ff220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc469178640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0ab9ff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 -- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 msgr2=0x55b0ab9ff760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0ab9ff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 -- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 msgr2=0x55b0ab9ff220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9ff220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 -- 172.21.15.76:0/3679865678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ab79fdc0 con 0x55b0ab9abdb0 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc45bfff640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc468977640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9fece0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc460000c10 tx=0x7fc460056420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29df964184b9b06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 1 -- 172.21.15.76:0/3679865678 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460021030 con 0x55b0ab9abdb0 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 1 -- 172.21.15.76:0/3679865678 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc4600557b0 con 0x55b0ab9abdb0 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4540ba050 con 0x55b0ab9abdb0 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 1 -- 172.21.15.76:0/3679865678 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc460084050 con 0x55b0ab9abdb0 2024-09-19T04:14:49.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 10 client.5268 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55b0ab79c7f0 con 0x55b0ab9abdb0 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 1 -- 172.21.15.76:0/3679865678 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4540ba050 con 0x55b0ab9abdb0 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 1 -- 172.21.15.76:0/3679865678 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fc46005a1b0 con 0x55b0ab9abdb0 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 10 client.5268 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 10 client.5268 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 20 client.5268 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba070b0 0x55b0aba094a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 4 client.5268 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0ab796e10 con 0x55b0aba070b0 2024-09-19T04:14:49.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc45bfff640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba070b0 0x55b0aba094a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc45bfff640 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba070b0 0x55b0aba094a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6213677847b9ca4 server_cookie=6b99a8da3bd413a1 in_seq=0 out_seq=0 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 10 client.5268 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 1 -- 172.21.15.76:0/3679865678 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0ab796e10 con 0x55b0aba070b0 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 10 client.5268 handle_command_reply: tid=0 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 client.5268 shutdown 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 msgr2=0x55b0ab9fece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9fece0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fc460000c10 tx=0x7fc460056420 comp rx=0 tx=0).stop 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 shutdown_connections 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba070b0 0x55b0aba094a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0ab9ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc4437fe640 0 client.5268 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0ab9ff220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 --2- 172.21.15.76:0/3679865678 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9fece0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 >> 172.21.15.76:0/3679865678 conn(0x55b0ab9d9040 msgr2=0x55b0ab9aaf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 shutdown_connections 2024-09-19T04:14:49.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.391+0000 7fc46dc97000 1 -- 172.21.15.76:0/3679865678 wait complete. 2024-09-19T04:14:49.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.395+0000 7fc46dc97000 20 client.5268 trim_cache size 0 max 16384 2024-09-19T04:14:49.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 Processor -- start 2024-09-19T04:14:49.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- start start 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f7d70 0x55b0ab9b4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f8690 0x55b0aba01d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc464203d80 con 0x55b0ab9abdb0 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc464202f50 con 0x55b0ab9f8690 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc464203650 con 0x55b0ab9f7d70 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43208/0 (socket says 172.21.15.76:43208) 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 -- 172.21.15.76:0/364963226 learned_addr learned my addr 172.21.15.76:0/364963226 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 -- 172.21.15.76:0/364963226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f7d70 msgr2=0x55b0ab9b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 --2- 172.21.15.76:0/364963226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f7d70 0x55b0ab9b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 -- 172.21.15.76:0/364963226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f8690 msgr2=0x55b0aba01d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 --2- 172.21.15.76:0/364963226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f8690 0x55b0aba01d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 -- 172.21.15.76:0/364963226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0ab7ba470 con 0x55b0ab9abdb0 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc468977640 1 --2- 172.21.15.76:0/364963226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9b3c30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc4600527c0 tx=0x7fc460058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e13ea3a42a18ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc449ffb640 1 -- 172.21.15.76:0/364963226 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460072070 con 0x55b0ab9abdb0 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc449ffb640 1 -- 172.21.15.76:0/364963226 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46005a710 con 0x55b0ab9abdb0 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc449ffb640 1 -- 172.21.15.76:0/364963226 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460021070 con 0x55b0ab9abdb0 2024-09-19T04:14:49.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- 172.21.15.76:0/364963226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 msgr2=0x55b0ab9b3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 --2- 172.21.15.76:0/364963226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9b3c30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fc4600527c0 tx=0x7fc460058000 comp rx=0 tx=0).stop 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- 172.21.15.76:0/364963226 shutdown_connections 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 --2- 172.21.15.76:0/364963226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f8690 0x55b0aba01d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 --2- 172.21.15.76:0/364963226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f7d70 0x55b0ab9b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 --2- 172.21.15.76:0/364963226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0ab9b3c30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- 172.21.15.76:0/364963226 >> 172.21.15.76:0/364963226 conn(0x55b0ab9d9040 msgr2=0x55b0ab9feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- 172.21.15.76:0/364963226 shutdown_connections 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.411+0000 7fc46dc97000 1 -- 172.21.15.76:0/364963226 wait complete. 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- start start 2024-09-19T04:14:49.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 Processor -- start 2024-09-19T04:14:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0aba04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0aba27790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0aba27cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b0ab801370 con 0x55b0ab9abdb0 2024-09-19T04:14:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc46414e820 con 0x55b0ab9f7d70 2024-09-19T04:14:49.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b0ab7a2f20 con 0x55b0ab9f8690 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 10 client.0 _ll_register_callbacks cb 0x7fc460052600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0aba04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0aba04440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43224/0 (socket says 172.21.15.76:43224) 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 -- 172.21.15.76:0/318256092 learned_addr learned my addr 172.21.15.76:0/318256092 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc45bfff640 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0aba27790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc469178640 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0aba27cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 -- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 msgr2=0x55b0aba27cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0aba27cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 -- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 msgr2=0x55b0aba27790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0aba27790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 -- 172.21.15.76:0/318256092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0ab790640 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc468977640 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0aba04440 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4600527c0 tx=0x7fc460084320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eadac535562b2501 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 1 -- 172.21.15.76:0/318256092 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc460072060 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 1 -- 172.21.15.76:0/318256092 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(39..39 src has 1..39) ==== 6101+0+0 (secure 0 0 0) 0x7fc46001dc90 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc46005a710 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 1 -- 172.21.15.76:0/318256092 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fc460081030 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 10 client.5274 fetch_fsmap learned FSMap version 21 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) -- 0x55b0ab7943b0 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 1 -- 172.21.15.76:0/318256092 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc46005a710 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 1 -- 172.21.15.76:0/318256092 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 21) ==== 3554+0+0 (secure 0 0 0) 0x7fc4600520b0 con 0x55b0ab9abdb0 2024-09-19T04:14:49.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 10 client.5274 fetch_fsmap finished waiting for FSMap version 21 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 10 client.5274 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 20 client.5274 populate_metadata read hostname 'smithi076' 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba28210 0x55b0aba6bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 4 client.5274 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b0ab80a470 con 0x55b0aba28210 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc45bfff640 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba28210 0x55b0aba6bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc45bfff640 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba28210 0x55b0aba6bb30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2291f55e2ec0adee server_cookie=a9a16e85a43afe47 in_seq=0 out_seq=0 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 10 client.5274 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 1 -- 172.21.15.76:0/318256092 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+207 (crc 0 0 0) 0x55b0ab80a470 con 0x55b0aba28210 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 10 client.5274 handle_command_reply: tid=0 2024-09-19T04:14:49.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 client.5274 shutdown 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 msgr2=0x55b0aba04440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0aba04440 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4600527c0 tx=0x7fc460084320 comp rx=0 tx=0).stop 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 shutdown_connections 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x55b0aba28210 0x55b0aba6bb30 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0ab9f8690 0x55b0aba27cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0ab9f7d70 0x55b0aba27790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc4437fe640 0 client.5274 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 --2- 172.21.15.76:0/318256092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0ab9abdb0 0x55b0aba04440 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 >> 172.21.15.76:0/318256092 conn(0x55b0ab9d9040 msgr2=0x55b0ab9b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 shutdown_connections 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 1 -- 172.21.15.76:0/318256092 wait complete. 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46dc97000 20 client.5274 trim_cache size 0 max 16384 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stdout: "tag123": { 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:14:49.429 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "tag123", 2024-09-19T04:14:49.430 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:14:49.430 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:49.430 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:14:49.430 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4500678c0 msgr2=0x7fc450069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4500678c0 0x7fc450069d80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4540ae000 tx=0x7fc454004740 comp rx=0 tx=0).stop 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640cf450 msgr2=0x7fc46415fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640cf450 0x7fc46415fa50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc46000e8a0 tx=0x7fc46000eca0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 shutdown_connections 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4500678c0 0x7fc450069d80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc464161a60 0x7fc4640b4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4640d2e90 0x7fc4640ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 --2- 172.21.15.76:0/2960315847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4640cf450 0x7fc46415fa50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:14:49.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 >> 172.21.15.76:0/2960315847 conn(0x7fc46408bf90 msgr2=0x7fc464150090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:14:49.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 shutdown_connections 2024-09-19T04:14:49.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:14:49.415+0000 7fc46abdb640 1 -- 172.21.15.76:0/2960315847 wait complete. 2024-09-19T04:14:49.437 INFO:tasks.cephfs.filesystem:scrub status for tag:tag123 - {'path': '/d1/d2/d3', 'tag': 'tag123', 'options': 'recursive,force'} 2024-09-19T04:14:53.165 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:14:53.168 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:15:19.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:19.439 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-09-19T04:15:19.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.570+0000 7fdb09077640 1 Processor -- start 2024-09-19T04:15:19.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.570+0000 7fdb09077640 1 -- start start 2024-09-19T04:15:19.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb0415bec0 0x7fdb040ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdb0415c2a0 0x7fdb040aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb040b3290 con 0x7fdb040af2a0 2024-09-19T04:15:19.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdb040b3570 con 0x7fdb0415bec0 2024-09-19T04:15:19.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fdb040b3850 con 0x7fdb0415c2a0 2024-09-19T04:15:19.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb02d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb0415bec0 0x7fdb040ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb0359e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdb040af2a0 0x7fdb0415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57718/0 (socket says 172.21.15.76:57718) 2024-09-19T04:15:19.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb0359e640 1 -- 172.21.15.76:0/3073955986 learned_addr learned my addr 172.21.15.76:0/3073955986 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:19.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb0259c640 1 --2- 172.21.15.76:0/3073955986 >> v2:172.21.15.116:3300/0 conn(0x7fdb0415c2a0 0x7fdb040aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb01d9b640 1 -- 172.21.15.76:0/3073955986 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1501876858 0 0) 0x7fdb040b3290 con 0x7fdb040af2a0 2024-09-19T04:15:19.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb01d9b640 1 -- 172.21.15.76:0/3073955986 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdaf0003530 con 0x7fdb040af2a0 2024-09-19T04:15:19.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb02d9d640 1 -- 172.21.15.76:0/3073955986 >> v2:172.21.15.116:3300/0 conn(0x7fdb0415c2a0 msgr2=0x7fdb040aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/3073955986 >> v2:172.21.15.116:3300/0 conn(0x7fdb0415c2a0 0x7fdb040aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb02d9d640 1 -- 172.21.15.76:0/3073955986 >> v1:172.21.15.76:6789/0 conn(0x7fdb040af2a0 legacy=0x7fdb0415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb02d9d640 1 -- 172.21.15.76:0/3073955986 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb040b3b30 con 0x7fdb0415bec0 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/3073955986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0415bec0 0x7fdb040ad200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdb040ae510 tx=0x7fdafc01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fdf4001b44a4b65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb01d9b640 1 -- 172.21.15.76:0/3073955986 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0b2070 con 0x7fdb0415bec0 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb01d9b640 1 -- 172.21.15.76:0/3073955986 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc0a59c0 con 0x7fdb0415bec0 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb01d9b640 1 -- 172.21.15.76:0/3073955986 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0a5d70 con 0x7fdb0415bec0 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- 172.21.15.76:0/3073955986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0415bec0 msgr2=0x7fdb040ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 --2- 172.21.15.76:0/3073955986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0415bec0 0x7fdb040ad200 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fdb040ae510 tx=0x7fdafc01c900 comp rx=0 tx=0).stop 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- 172.21.15.76:0/3073955986 shutdown_connections 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 --2- 172.21.15.76:0/3073955986 >> v2:172.21.15.116:3300/0 conn(0x7fdb0415c2a0 0x7fdb040aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 --2- 172.21.15.76:0/3073955986 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb0415bec0 0x7fdb040ad200 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- 172.21.15.76:0/3073955986 >> 172.21.15.76:0/3073955986 conn(0x7fdb040b1f90 msgr2=0x7fdb040b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- 172.21.15.76:0/3073955986 shutdown_connections 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.574+0000 7fdb09077640 1 -- 172.21.15.76:0/3073955986 wait complete. 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 Processor -- start 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 -- start start 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb040af2a0 0x7fdb0414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb0415bec0 0x7fdb041580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb0415c2a0 0x7fdb04151e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb040af6b0 con 0x7fdb040af2a0 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb040ad740 con 0x7fdb0415c2a0 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdb04169000 con 0x7fdb0415bec0 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb040af2a0 0x7fdb0414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb040af2a0 0x7fdb0414e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56508/0 (socket says 172.21.15.76:56508) 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 -- 172.21.15.76:0/3514156295 learned_addr learned my addr 172.21.15.76:0/3514156295 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb0359e640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb0415c2a0 0x7fdb04151e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb0259c640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb0415bec0 0x7fdb041580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 -- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb0415bec0 msgr2=0x7fdb041580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb0415bec0 0x7fdb041580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 -- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb0415c2a0 msgr2=0x7fdb04151e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb0415c2a0 0x7fdb04151e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 -- 172.21.15.76:0/3514156295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0415d860 con 0x7fdb040af2a0 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb0259c640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb0415bec0 0x7fdb041580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb040af2a0 0x7fdb0414e9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f90 tx=0x7fdafc0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdafb7fe640 1 -- 172.21.15.76:0/3514156295 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0b2040 con 0x7fdb040af2a0 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdafb7fe640 1 -- 172.21.15.76:0/3514156295 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc0bf070 con 0x7fdb040af2a0 2024-09-19T04:15:19.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb04158a60 con 0x7fdb040af2a0 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdafb7fe640 1 -- 172.21.15.76:0/3514156295 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc003da0 con 0x7fdb040af2a0 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.578+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0415dad0 con 0x7fdb040af2a0 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdafb7fe640 1 -- 172.21.15.76:0/3514156295 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdafc004040 con 0x7fdb040af2a0 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdafb7fe640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdae40678c0 0x7fdae4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdafb7fe640 1 -- 172.21.15.76:0/3514156295 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fdafc121010 con 0x7fdb040af2a0 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0259c640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdae40678c0 0x7fdae4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0259c640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdae40678c0 0x7fdae4069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb0415fb80 tx=0x7fdaec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 Processor -- start 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 -- start start 2024-09-19T04:15:19.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997ed70 0x5606b997f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997f690 0x5606b9932870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9933190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5606b981c960 con 0x5606b9932db0 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5606b9760900 con 0x5606b997f690 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5606b9783d40 con 0x5606b997ed70 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9933190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9933190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56512/0 (socket says 172.21.15.76:56512) 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 -- 172.21.15.76:0/3513051179 learned_addr learned my addr 172.21.15.76:0/3513051179 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997ed70 0x5606b997f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0259c640 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997f690 0x5606b9932870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 -- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997ed70 msgr2=0x5606b997f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997ed70 0x5606b997f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 -- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997f690 msgr2=0x5606b9932870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997f690 0x5606b9932870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 -- 172.21.15.76:0/3513051179 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b972a580 con 0x5606b9932db0 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdb0359e640 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9933190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40015d0 tx=0x7fdaf4001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5af855c55fda9cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdaf8ff9640 1 -- 172.21.15.76:0/3513051179 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4019070 con 0x5606b9932db0 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdaf8ff9640 1 -- 172.21.15.76:0/3513051179 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf4005860 con 0x5606b9932db0 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.582+0000 7fdaf8ff9640 1 -- 172.21.15.76:0/3513051179 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4005c10 con 0x5606b9932db0 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/3513051179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 msgr2=0x5606b9933190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9933190 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdaf40015d0 tx=0x7fdaf4001a90 comp rx=0 tx=0).stop 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/3513051179 shutdown_connections 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9933190 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997f690 0x5606b9932870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- 172.21.15.76:0/3513051179 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997ed70 0x5606b997f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/3513051179 >> 172.21.15.76:0/3513051179 conn(0x5606b9960040 msgr2=0x5606b9962430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/3513051179 shutdown_connections 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/3513051179 wait complete. 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- start start 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 Processor -- start 2024-09-19T04:15:19.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9985ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b9986220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997f690 0x5606b9986760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5606b977fd00 con 0x5606b9932db0 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5606b973eaa0 con 0x5606b997ed70 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5606b971d340 con 0x5606b997f690 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 10 client.0 _ll_register_callbacks cb 0x5606b9920760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9985ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9985ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56526/0 (socket says 172.21.15.76:56526) 2024-09-19T04:15:19.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2042458322 learned_addr learned my addr 172.21.15.76:0/2042458322 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2042458322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997f690 msgr2=0x5606b9986760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997f690 0x5606b9986760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2042458322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 msgr2=0x5606b9986220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b9986220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2042458322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b9726dc0 con 0x5606b9932db0 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9985ce0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0f0fa0 tx=0x7fdafc0f1ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665086ca48420595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2042458322 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0f1500 con 0x5606b9932db0 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2042458322 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fdafc0f16a0 con 0x5606b9932db0 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf4005860 con 0x5606b9932db0 2024-09-19T04:15:19.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2042458322 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdafc0f99f0 con 0x5606b9932db0 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 10 client.5289 fetch_fsmap learned FSMap version 22 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5606b97237f0 con 0x5606b9932db0 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2042458322 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf4005860 con 0x5606b9932db0 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2042458322 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 3558+0+0 (secure 0 0 0) 0x7fdafc109030 con 0x5606b9932db0 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 10 client.5289 fetch_fsmap finished waiting for FSMap version 22 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 10 client.5289 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 20 client.5289 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b998e0b0 0x5606b99904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 4 client.5289 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:19.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606b971de10 con 0x5606b998e0b0 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0259c640 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b998e0b0 0x5606b99904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 10 client.5289 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0259c640 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b998e0b0 0x5606b99904a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4f87271f2657a1 server_cookie=5492114f31160cd4 in_seq=0 out_seq=0 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2042458322 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5606b971de10 con 0x5606b998e0b0 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdae1ffb640 10 client.5289 handle_command_reply: tid=0 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.586+0000 7fdb0c133000 1 client.5289 shutdown 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 msgr2=0x5606b9985ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9985ce0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0f0fa0 tx=0x7fdafc0f1ae0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 shutdown_connections 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b998e0b0 0x5606b99904a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b997f690 0x5606b9986760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b9986220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdae1ffb640 0 client.5289 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:15:19.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2042458322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b9985ce0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 >> 172.21.15.76:0/2042458322 conn(0x5606b9960040 msgr2=0x5606b9931f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 shutdown_connections 2024-09-19T04:15:19.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 1 -- 172.21.15.76:0/2042458322 wait complete. 2024-09-19T04:15:19.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.590+0000 7fdb0c133000 20 client.5289 trim_cache size 0 max 16384 2024-09-19T04:15:19.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 Processor -- start 2024-09-19T04:15:19.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 -- start start 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9960040 0x5606b993aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9932db0 0x5606b993b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b9988cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04201f60 con 0x5606b9960040 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb04201100 con 0x5606b997ed70 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdb04201830 con 0x5606b9932db0 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9960040 0x5606b993aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9960040 0x5606b993aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56530/0 (socket says 172.21.15.76:56530) 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 -- 172.21.15.76:0/1999532744 learned_addr learned my addr 172.21.15.76:0/1999532744 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 -- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9932db0 msgr2=0x5606b993b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0259c640 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9932db0 0x5606b993b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9932db0 0x5606b993b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 -- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 msgr2=0x5606b9988cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb0359e640 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b9988cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b9988cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 -- 172.21.15.76:0/1999532744 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606b9741470 con 0x5606b9960040 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9960040 0x5606b993aba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdafc01cde0 tx=0x7fdafc0ed670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee58ce3657592cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdae3fff640 1 -- 172.21.15.76:0/1999532744 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0f9b90 con 0x5606b9960040 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdae3fff640 1 -- 172.21.15.76:0/1999532744 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc0f1500 con 0x5606b9960040 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.606+0000 7fdae3fff640 1 -- 172.21.15.76:0/1999532744 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0f5150 con 0x5606b9960040 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/1999532744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9960040 msgr2=0x5606b993aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9960040 0x5606b993aba0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdafc01cde0 tx=0x7fdafc0ed670 comp rx=0 tx=0).stop 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/1999532744 shutdown_connections 2024-09-19T04:15:19.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b9988cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9932db0 0x5606b993b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/1999532744 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9960040 0x5606b993aba0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/1999532744 >> 172.21.15.76:0/1999532744 conn(0x5606b9937770 msgr2=0x5606b99862c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/1999532744 shutdown_connections 2024-09-19T04:15:19.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/1999532744 wait complete. 2024-09-19T04:15:19.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- start start 2024-09-19T04:15:19.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 Processor -- start 2024-09-19T04:15:19.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b998bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9960040 0x5606b998c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b99ef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5606b9788370 con 0x5606b9932db0 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0415edd0 con 0x5606b997ed70 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5606b98400c0 con 0x5606b9960040 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 10 client.0 _ll_register_callbacks cb 0x7fdafc009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b998bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b998bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56544/0 (socket says 172.21.15.76:56544) 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2105396493 learned_addr learned my addr 172.21.15.76:0/2105396493 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0359e640 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b99ef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9960040 msgr2=0x5606b998c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9960040 0x5606b998c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 msgr2=0x5606b99ef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b99ef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 -- 172.21.15.76:0/2105396493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606b9717640 con 0x5606b9932db0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0359e640 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b99ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb02d9d640 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b998bf00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdafc01c940 tx=0x7fdafc11f260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d187839de9894d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2105396493 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0ed540 con 0x5606b9932db0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2105396493 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fdafc109020 con 0x5606b9932db0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdafc0f1500 con 0x5606b9932db0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2105396493 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) ==== 24+0+0 (secure 0 0 0) 0x7fdafc0f0c00 con 0x5606b9932db0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 10 client.5295 fetch_fsmap learned FSMap version 22 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) -- 0x5606b971b3b0 con 0x5606b9932db0 2024-09-19T04:15:19.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2105396493 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdafc0f1500 con 0x5606b9932db0 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2105396493 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 22) ==== 3558+0+0 (secure 0 0 0) 0x7fdafc10fbb0 con 0x5606b9932db0 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 10 client.5295 fetch_fsmap finished waiting for FSMap version 22 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 10 client.5295 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 20 client.5295 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b997f690 0x5606b99efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 4 client.5295 mds_command: new command op to 4863 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 --> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5606b9791470 con 0x5606b997f690 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0259c640 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b997f690 0x5606b99efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 10 client.5295 ms_handle_connect on v2:172.21.15.76:6836/1828076892 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0259c640 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b997f690 0x5606b99efcb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a206383404918e6b server_cookie=449cd81917ec8f34 in_seq=0 out_seq=0 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 1 -- 172.21.15.76:0/2105396493 <== mds.0 v2:172.21.15.76:6836/1828076892 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5606b9791470 con 0x5606b997f690 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 10 client.5295 handle_command_reply: tid=0 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 client.5295 shutdown 2024-09-19T04:15:19.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 msgr2=0x5606b998bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b998bf00 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdafc01c940 tx=0x7fdafc11f260 comp rx=0 tx=0).stop 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 shutdown_connections 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:6836/1828076892,v1:172.21.15.76:6837/1828076892] conn(0x5606b997f690 0x5606b99efcb0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5606b997ed70 0x5606b99ef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdae1ffb640 0 client.5295 ms_handle_reset on v2:172.21.15.76:6836/1828076892 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5606b9960040 0x5606b998c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 --2- 172.21.15.76:0/2105396493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5606b9932db0 0x5606b998bf00 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.610+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 >> 172.21.15.76:0/2105396493 conn(0x5606b9937770 msgr2=0x5606b9939b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 shutdown_connections 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb0c133000 1 -- 172.21.15.76:0/2105396493 wait complete. 2024-09-19T04:15:19.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb0c133000 20 client.5295 trim_cache size 0 max 16384 2024-09-19T04:15:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:15:19.621 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-09-19T04:15:19.621 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-09-19T04:15:19.621 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdae40678c0 msgr2=0x7fdae4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdae40678c0 0x7fdae4069d80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdb0415fb80 tx=0x7fdaec008040 comp rx=0 tx=0).stop 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb040af2a0 msgr2=0x7fdb0414e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb040af2a0 0x7fdb0414e9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdafc000f90 tx=0x7fdafc0196a0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 shutdown_connections 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdae40678c0 0x7fdae4069d80 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb0415c2a0 0x7fdb04151e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb0415bec0 0x7fdb041580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 --2- 172.21.15.76:0/3514156295 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb040af2a0 0x7fdb0414e9c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 >> 172.21.15.76:0/3514156295 conn(0x7fdb040b1f90 msgr2=0x7fdb040aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 shutdown_connections 2024-09-19T04:15:19.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.614+0000 7fdb09077640 1 -- 172.21.15.76:0/3514156295 wait complete. 2024-09-19T04:15:19.631 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:15:19.632 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000002.00000000 scrub_tag 2024-09-19T04:15:19.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 Processor -- start 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- start start 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f1322a2700 0x55f1323750f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 0x55f13236ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f132373220 0x55f1322ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f1321ef1b0 con 0x55f132373220 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f1322487b0 con 0x55f1322e9390 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f1322a16c0 con 0x55f1322a2700 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f132373220 0x55f1322ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f132373220 0x55f1322ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56560/0 (socket says 172.21.15.76:56560) 2024-09-19T04:15:19.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 -- 172.21.15.76:0/3267287262 learned_addr learned my addr 172.21.15.76:0/3267287262 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 -- 172.21.15.76:0/3267287262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f1322a2700 msgr2=0x55f1323750f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 --2- 172.21.15.76:0/3267287262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f1322a2700 0x55f1323750f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 -- 172.21.15.76:0/3267287262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 msgr2=0x55f13236ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 --2- 172.21.15.76:0/3267287262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 0x55f13236ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 -- 172.21.15.76:0/3267287262 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f132373020 con 0x55f132373220 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2932b640 1 --2- 172.21.15.76:0/3267287262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f132373220 0x55f1322ce2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4b240ae040 tx=0x7f4b240033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8165b7187454382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b22ffd640 1 -- 172.21.15.76:0/3267287262 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240b9070 con 0x55f132373220 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b22ffd640 1 -- 172.21.15.76:0/3267287262 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b240ab640 con 0x55f132373220 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b22ffd640 1 -- 172.21.15.76:0/3267287262 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b240b0040 con 0x55f132373220 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- 172.21.15.76:0/3267287262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f132373220 msgr2=0x55f1322ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3267287262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f132373220 0x55f1322ce2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4b240ae040 tx=0x7f4b240033b0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- 172.21.15.76:0/3267287262 shutdown_connections 2024-09-19T04:15:19.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3267287262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f132373220 0x55f1322ce2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3267287262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 0x55f13236ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3267287262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f1322a2700 0x55f1323750f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- 172.21.15.76:0/3267287262 >> 172.21.15.76:0/3267287262 conn(0x55f1322caef0 msgr2=0x55f1322a1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- 172.21.15.76:0/3267287262 shutdown_connections 2024-09-19T04:15:19.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 -- 172.21.15.76:0/3267287262 wait complete. 2024-09-19T04:15:19.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.662+0000 7f4b2b017080 1 Processor -- start 2024-09-19T04:15:19.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 -- start start 2024-09-19T04:15:19.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f1322a2700 0x55f13236cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 0x55f1322c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f132373220 0x55f13236ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f132289900 con 0x55f1322a2700 2024-09-19T04:15:19.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f13236f370 con 0x55f1322e9390 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f13236f5d0 con 0x55f132373220 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f1322a2700 0x55f13236cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f1322a2700 0x55f13236cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56564/0 (socket says 172.21.15.76:56564) 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 -- 172.21.15.76:0/3760640566 learned_addr learned my addr 172.21.15.76:0/3760640566 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2932b640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f132373220 0x55f13236ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b23fff640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 0x55f1322c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 -- 172.21.15.76:0/3760640566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f132373220 msgr2=0x55f13236ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f132373220 0x55f13236ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 -- 172.21.15.76:0/3760640566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 msgr2=0x55f1322c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 0x55f1322c17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 -- 172.21.15.76:0/3760640566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f13229fdb0 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b23fff640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f1322e9390 0x55f1322c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b28b2a640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f1322a2700 0x55f13236cba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00be40 tx=0x7f4b1c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b21ffb640 1 -- 172.21.15.76:0/3760640566 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00e480 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b21ffb640 1 -- 172.21.15.76:0/3760640566 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b1c00e620 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f13229f330 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b21ffb640 1 -- 172.21.15.76:0/3760640566 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00ea10 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f13229f870 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b21ffb640 1 -- 172.21.15.76:0/3760640566 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4b1c00ebb0 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b21ffb640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b0c0662a0 0x7f4b0c068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b21ffb640 1 -- 172.21.15.76:0/3760640566 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f4b1c085050 con 0x55f1322a2700 2024-09-19T04:15:19.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b23fff640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b0c0662a0 0x7f4b0c068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x55f1322c1d20 0x55f1322c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.666+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55f1322c40d0 con 0x55f1322c1d20 2024-09-19T04:15:19.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2932b640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x55f1322c1d20 0x55f1322c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b23fff640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b0c0662a0 0x7f4b0c068760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400b410 tx=0x7f4b14005540 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2932b640 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x55f1322c1d20 0x55f1322c3b90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2932b640 1 -- 172.21.15.76:0/3760640566 <== osd.1 v2:172.21.15.76:6808/1010487110 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (10) out=10b] v0'0 uv8 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f4b240ab8f0 con 0x55f1322c1d20 2024-09-19T04:15:19.674 INFO:teuthology.orchestra.run.smithi076.stdout:tag123 2024-09-19T04:15:19.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x55f1322c1d20 msgr2=0x55f1322c3b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x55f1322c1d20 0x55f1322c3b90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b0c0662a0 msgr2=0x7f4b0c068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b0c0662a0 0x7f4b0c068760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400b410 tx=0x7f4b14005540 comp rx=0 tx=0).stop 2024-09-19T04:15:19.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f1322a2700 msgr2=0x55f13236cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 --2- 172.21.15.76:0/3760640566 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f1322a2700 0x55f13236cba0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00be40 tx=0x7f4b1c009650 comp rx=0 tx=0).stop 2024-09-19T04:15:19.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2932b640 1 -- 172.21.15.76:0/3760640566 reap_dead start 2024-09-19T04:15:19.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 shutdown_connections 2024-09-19T04:15:19.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 >> 172.21.15.76:0/3760640566 conn(0x55f1322caef0 msgr2=0x55f1322cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 shutdown_connections 2024-09-19T04:15:19.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.670+0000 7f4b2b017080 1 -- 172.21.15.76:0/3760640566 wait complete. 2024-09-19T04:15:19.679 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000003.00000000 scrub_tag 2024-09-19T04:15:19.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 Processor -- start 2024-09-19T04:15:19.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- start start 2024-09-19T04:15:19.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 0x561d09362850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d094815c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 0x561d0935ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561d0934c3f0 con 0x561d093ed4c0 2024-09-19T04:15:19.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561d0926b1b0 con 0x561d09481b00 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561d09362fc0 con 0x561d094811e0 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d094815c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d094815c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56586/0 (socket says 172.21.15.76:56586) 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 -- 172.21.15.76:0/2721753390 learned_addr learned my addr 172.21.15.76:0/2721753390 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcf7b2640 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 0x561d09362850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcffb3640 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 0x561d0935ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 -- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 msgr2=0x561d09362850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 0x561d09362850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 -- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 msgr2=0x561d0935ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 0x561d0935ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 -- 172.21.15.76:0/2721753390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d0931d420 con 0x561d093ed4c0 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcefb1640 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d094815c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009870 tx=0x7f1fb8009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc59445abfd6fdf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcdfaf640 1 -- 172.21.15.76:0/2721753390 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8008e00 con 0x561d093ed4c0 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcdfaf640 1 -- 172.21.15.76:0/2721753390 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb8019070 con 0x561d093ed4c0 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fcdfaf640 1 -- 172.21.15.76:0/2721753390 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80145f0 con 0x561d093ed4c0 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/2721753390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 msgr2=0x561d094815c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d094815c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009870 tx=0x7f1fb8009d30 comp rx=0 tx=0).stop 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/2721753390 shutdown_connections 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 0x561d0935ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d094815c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/2721753390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 0x561d09362850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/2721753390 >> 172.21.15.76:0/2721753390 conn(0x561d09319680 msgr2=0x561d0931cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/2721753390 shutdown_connections 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.750+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/2721753390 wait complete. 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 Processor -- start 2024-09-19T04:15:19.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 -- start start 2024-09-19T04:15:19.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d0935dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 0x561d093442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 0x561d093447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561d0931d960 con 0x561d093ed4c0 2024-09-19T04:15:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561d09362d90 con 0x561d09481b00 2024-09-19T04:15:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561d0931d6c0 con 0x561d094811e0 2024-09-19T04:15:19.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d0935dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d0935dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56600/0 (socket says 172.21.15.76:56600) 2024-09-19T04:15:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 -- 172.21.15.76:0/3818517438 learned_addr learned my addr 172.21.15.76:0/3818517438 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcefb1640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 0x561d093442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcffb3640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 0x561d093447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 -- 172.21.15.76:0/3818517438 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 msgr2=0x561d093442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d094811e0 0x561d093442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 -- 172.21.15.76:0/3818517438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 msgr2=0x561d093447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d09481b00 0x561d093447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 -- 172.21.15.76:0/3818517438 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d09344e20 con 0x561d093ed4c0 2024-09-19T04:15:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fcf7b2640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d0935dca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ecf0 tx=0x7f1fc800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fccfad640 1 -- 172.21.15.76:0/3818517438 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8006550 con 0x561d093ed4c0 2024-09-19T04:15:19.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fccfad640 1 -- 172.21.15.76:0/3818517438 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc80066f0 con 0x561d093ed4c0 2024-09-19T04:15:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fccfad640 1 -- 172.21.15.76:0/3818517438 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc8006aa0 con 0x561d093ed4c0 2024-09-19T04:15:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561d093450c0 con 0x561d093ed4c0 2024-09-19T04:15:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d093d72f0 con 0x561d093ed4c0 2024-09-19T04:15:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.754+0000 7f1fccfad640 1 -- 172.21.15.76:0/3818517438 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1fc8006c40 con 0x561d093ed4c0 2024-09-19T04:15:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fccfad640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fb00662d0 0x7f1fb0068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fccfad640 1 -- 172.21.15.76:0/3818517438 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f1fc8087ab0 con 0x561d093ed4c0 2024-09-19T04:15:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fcefb1640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fb00662d0 0x7f1fb0068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x561d0935e1e0 0x561d09343160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- osd_op(unknown.0.0:1 5.12 5:4def72d4:::10000000003.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x561d093d7a30 con 0x561d0935e1e0 2024-09-19T04:15:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fcffb3640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x561d0935e1e0 0x561d09343160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fcefb1640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fb00662d0 0x7f1fb0068790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006010 tx=0x7f1fb800c000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fcffb3640 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x561d0935e1e0 0x561d09343160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fcffb3640 1 -- 172.21.15.76:0/3818517438 <== osd.1 v2:172.21.15.76:6808/1010487110 1 ==== osd_op_reply(1 10000000003.00000000 [getxattr (10) out=10b] v0'0 uv9 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f1fc0005040 con 0x561d0935e1e0 2024-09-19T04:15:19.767 INFO:teuthology.orchestra.run.smithi076.stdout:tag123 2024-09-19T04:15:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x561d0935e1e0 msgr2=0x561d09343160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x561d0935e1e0 0x561d09343160 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fb00662d0 msgr2=0x7f1fb0068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fb00662d0 0x7f1fb0068790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8006010 tx=0x7f1fb800c000 comp rx=0 tx=0).stop 2024-09-19T04:15:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 msgr2=0x561d0935dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 --2- 172.21.15.76:0/3818517438 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d093ed4c0 0x561d0935dca0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800ecf0 tx=0x7f1fc800b730 comp rx=0 tx=0).stop 2024-09-19T04:15:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fcffb3640 1 -- 172.21.15.76:0/3818517438 reap_dead start 2024-09-19T04:15:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 shutdown_connections 2024-09-19T04:15:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 >> 172.21.15.76:0/3818517438 conn(0x561d09319680 msgr2=0x561d0935b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 shutdown_connections 2024-09-19T04:15:19.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.758+0000 7f1fd1c9f080 1 -- 172.21.15.76:0/3818517438 wait complete. 2024-09-19T04:15:19.772 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000004.00000000 scrub_tag 2024-09-19T04:15:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 Processor -- start 2024-09-19T04:15:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 -- start start 2024-09-19T04:15:19.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152353a0 0x55b9152f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b915267dd0 0x55b9152f1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b9152f2bb0 0x55b91530b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b9152f43a0 con 0x55b9152353a0 2024-09-19T04:15:19.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b9151861b0 con 0x55b915267dd0 2024-09-19T04:15:19.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128752c080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b9152f4520 con 0x55b9152f2bb0 2024-09-19T04:15:19.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152353a0 0x55b9152f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152353a0 0x55b9152f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56622/0 (socket says 172.21.15.76:56622) 2024-09-19T04:15:19.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 -- 172.21.15.76:0/2983331103 learned_addr learned my addr 172.21.15.76:0/2983331103 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f1285840640 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b9152f2bb0 0x55b91530b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128483e640 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b915267dd0 0x55b9152f1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 -- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b9152f2bb0 msgr2=0x55b91530b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b9152f2bb0 0x55b91530b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 -- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b915267dd0 msgr2=0x55b9152f1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b915267dd0 0x55b9152f1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 -- 172.21.15.76:0/2983331103 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9152f2020 con 0x55b9152353a0 2024-09-19T04:15:19.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f1285840640 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b9152f2bb0 0x55b91530b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:19.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f128503f640 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152353a0 0x55b9152f0b00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1280001eb0 tx=0x7f128000b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31106c85964864b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.846+0000 7f127f7fe640 1 -- 172.21.15.76:0/2983331103 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128000a040 con 0x55b9152353a0 2024-09-19T04:15:19.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f127f7fe640 1 -- 172.21.15.76:0/2983331103 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128000f620 con 0x55b9152353a0 2024-09-19T04:15:19.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f127f7fe640 1 -- 172.21.15.76:0/2983331103 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128000fa10 con 0x55b9152353a0 2024-09-19T04:15:19.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- 172.21.15.76:0/2983331103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152353a0 msgr2=0x55b9152f0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152353a0 0x55b9152f0b00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1280001eb0 tx=0x7f128000b680 comp rx=0 tx=0).stop 2024-09-19T04:15:19.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- 172.21.15.76:0/2983331103 shutdown_connections 2024-09-19T04:15:19.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b9152f2bb0 0x55b91530b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b915267dd0 0x55b9152f1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 --2- 172.21.15.76:0/2983331103 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152353a0 0x55b9152f0b00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- 172.21.15.76:0/2983331103 >> 172.21.15.76:0/2983331103 conn(0x55b915235dc0 msgr2=0x55b915264660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- 172.21.15.76:0/2983331103 shutdown_connections 2024-09-19T04:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- 172.21.15.76:0/2983331103 wait complete. 2024-09-19T04:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 Processor -- start 2024-09-19T04:15:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- start start 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b9152353a0 0x55b9153082b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b915267dd0 0x55b9152642c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152f2bb0 0x55b915264800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b9152f5bd0 con 0x55b9152f2bb0 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b9152f5930 con 0x55b9152353a0 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b9152f5e70 con 0x55b915267dd0 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152f2bb0 0x55b915264800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152f2bb0 0x55b915264800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56634/0 (socket says 172.21.15.76:56634) 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 -- 172.21.15.76:0/647979583 learned_addr learned my addr 172.21.15.76:0/647979583 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128483e640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b915267dd0 0x55b9152642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128503f640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b9152353a0 0x55b9153082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 -- 172.21.15.76:0/647979583 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b915267dd0 msgr2=0x55b9152642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b915267dd0 0x55b9152642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 -- 172.21.15.76:0/647979583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b9152353a0 msgr2=0x55b9153082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b9152353a0 0x55b9153082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 -- 172.21.15.76:0/647979583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b91525f030 con 0x55b9152f2bb0 2024-09-19T04:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f1285840640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152f2bb0 0x55b915264800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f12700076b0 tx=0x7f1270007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f127e7fc640 1 -- 172.21.15.76:0/647979583 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127000b0d0 con 0x55b9152f2bb0 2024-09-19T04:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f127e7fc640 1 -- 172.21.15.76:0/647979583 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f127000b270 con 0x55b9152f2bb0 2024-09-19T04:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b91525f2d0 con 0x55b9152f2bb0 2024-09-19T04:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f127e7fc640 1 -- 172.21.15.76:0/647979583 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1270006b10 con 0x55b9152f2bb0 2024-09-19T04:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.850+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b91525f810 con 0x55b9152f2bb0 2024-09-19T04:15:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f127e7fc640 1 -- 172.21.15.76:0/647979583 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1270006cb0 con 0x55b9152f2bb0 2024-09-19T04:15:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f127e7fc640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f126c0662a0 0x7f126c068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f127e7fc640 1 -- 172.21.15.76:0/647979583 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f1270084a20 con 0x55b9152f2bb0 2024-09-19T04:15:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128503f640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f126c0662a0 0x7f126c068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b91525ff50 0x55b915260330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 5.5 5:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55b915260870 con 0x55b91525ff50 2024-09-19T04:15:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128483e640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b91525ff50 0x55b915260330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128503f640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f126c0662a0 0x7f126c068760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1280003040 tx=0x7f1280009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128483e640 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b91525ff50 0x55b915260330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128483e640 1 -- 172.21.15.76:0/647979583 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f12780b3040 con 0x55b91525ff50 2024-09-19T04:15:19.866 INFO:teuthology.orchestra.run.smithi076.stdout:tag123 2024-09-19T04:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b91525ff50 msgr2=0x55b915260330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b91525ff50 0x55b915260330 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f126c0662a0 msgr2=0x7f126c068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f126c0662a0 0x7f126c068760 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1280003040 tx=0x7f1280009000 comp rx=0 tx=0).stop 2024-09-19T04:15:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152f2bb0 msgr2=0x55b915264800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 --2- 172.21.15.76:0/647979583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b9152f2bb0 0x55b915264800 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f12700076b0 tx=0x7f1270007b70 comp rx=0 tx=0).stop 2024-09-19T04:15:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f1285840640 1 -- 172.21.15.76:0/647979583 reap_dead start 2024-09-19T04:15:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 shutdown_connections 2024-09-19T04:15:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 >> 172.21.15.76:0/647979583 conn(0x55b915235dc0 msgr2=0x55b9152f0110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 shutdown_connections 2024-09-19T04:15:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.854+0000 7f128752c080 1 -- 172.21.15.76:0/647979583 wait complete. 2024-09-19T04:15:19.871 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000005.00000000 scrub_tag 2024-09-19T04:15:19.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 Processor -- start 2024-09-19T04:15:19.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- start start 2024-09-19T04:15:19.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16ad40 0x55685e1f5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16c9e0 0x55685e1ea2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1ea830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55685e152e50 con 0x55685e16ad40 2024-09-19T04:15:19.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55685e0731b0 con 0x55685e1f60c0 2024-09-19T04:15:19.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55685e152fd0 con 0x55685e16c9e0 2024-09-19T04:15:19.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16ad40 0x55685e1f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16ad40 0x55685e1f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56652/0 (socket says 172.21.15.76:56652) 2024-09-19T04:15:19.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 -- 172.21.15.76:0/1937467844 learned_addr learned my addr 172.21.15.76:0/1937467844 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b80ebb640 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1ea830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b7feb9640 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16c9e0 0x55685e1ea2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 -- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16c9e0 msgr2=0x55685e1ea2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16c9e0 0x55685e1ea2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 -- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 msgr2=0x55685e1ea830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:19.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1ea830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 -- 172.21.15.76:0/1937467844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685e1eb7b0 con 0x55685e16ad40 2024-09-19T04:15:19.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b80ebb640 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:19.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b806ba640 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16ad40 0x55685e1f5240 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2b680097d0 tx=0x7f2b68005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e560d740dc27f790 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b7eeb7640 1 -- 172.21.15.76:0/1937467844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68019070 con 0x55685e16ad40 2024-09-19T04:15:19.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b7eeb7640 1 -- 172.21.15.76:0/1937467844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b68008d40 con 0x55685e16ad40 2024-09-19T04:15:19.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b7eeb7640 1 -- 172.21.15.76:0/1937467844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b680146a0 con 0x55685e16ad40 2024-09-19T04:15:19.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- 172.21.15.76:0/1937467844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16ad40 msgr2=0x55685e1f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16ad40 0x55685e1f5240 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2b680097d0 tx=0x7f2b68005a10 comp rx=0 tx=0).stop 2024-09-19T04:15:19.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- 172.21.15.76:0/1937467844 shutdown_connections 2024-09-19T04:15:19.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1ea830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16c9e0 0x55685e1ea2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/1937467844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16ad40 0x55685e1f5240 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- 172.21.15.76:0/1937467844 >> 172.21.15.76:0/1937467844 conn(0x55685e14d220 msgr2=0x55685e1f66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- 172.21.15.76:0/1937467844 shutdown_connections 2024-09-19T04:15:19.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.946+0000 7f2b82ba7080 1 -- 172.21.15.76:0/1937467844 wait complete. 2024-09-19T04:15:19.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 Processor -- start 2024-09-19T04:15:19.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 -- start start 2024-09-19T04:15:19.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16ad40 0x55685e1d3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16c9e0 0x55685e1ea4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55685e1eba50 con 0x55685e16c9e0 2024-09-19T04:15:19.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55685e1ebcf0 con 0x55685e1f60c0 2024-09-19T04:15:19.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55685e1f5780 con 0x55685e16ad40 2024-09-19T04:15:19.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16c9e0 0x55685e1ea4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16c9e0 0x55685e1ea4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56660/0 (socket says 172.21.15.76:56660) 2024-09-19T04:15:19.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 -- 172.21.15.76:0/2665715292 learned_addr learned my addr 172.21.15.76:0/2665715292 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:19.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b806ba640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16ad40 0x55685e1d3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b80ebb640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 -- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16ad40 msgr2=0x55685e1d3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55685e16ad40 0x55685e1d3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 -- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 msgr2=0x55685e1d0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55685e1f60c0 0x55685e1d0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 -- 172.21.15.76:0/2665715292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55685e1d15f0 con 0x55685e16c9e0 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7feb9640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16c9e0 0x55685e1ea4d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c460 tx=0x7f2b7800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7deb5640 1 -- 172.21.15.76:0/2665715292 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78011610 con 0x55685e16c9e0 2024-09-19T04:15:19.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7deb5640 1 -- 172.21.15.76:0/2665715292 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b78016070 con 0x55685e16c9e0 2024-09-19T04:15:19.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b7deb5640 1 -- 172.21.15.76:0/2665715292 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b780119d0 con 0x55685e16c9e0 2024-09-19T04:15:19.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55685e1d1890 con 0x55685e16c9e0 2024-09-19T04:15:19.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.950+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55685e1d1dd0 con 0x55685e16c9e0 2024-09-19T04:15:19.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b7deb5640 1 -- 172.21.15.76:0/2665715292 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2b78011b70 con 0x55685e16c9e0 2024-09-19T04:15:19.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b7deb5640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2b600662d0 0x7f2b60068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b7deb5640 1 -- 172.21.15.76:0/2665715292 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f2b78084770 con 0x55685e16c9e0 2024-09-19T04:15:19.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b806ba640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2b600662d0 0x7f2b60068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55685e1eaa10 0x55685e1f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:19.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- osd_op(unknown.0.0:1 5.9 5:9258b67d:::10000000005.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55685e1d2510 con 0x55685e1eaa10 2024-09-19T04:15:19.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b806ba640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2b600662d0 0x7f2b60068790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005a50 tx=0x7f2b68011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b80ebb640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55685e1eaa10 0x55685e1f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:19.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b80ebb640 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55685e1eaa10 0x55685e1f0630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:19.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b80ebb640 1 -- 172.21.15.76:0/2665715292 <== osd.7 v2:172.21.15.116:6824/2361616500 1 ==== osd_op_reply(1 10000000005.00000000 [getxattr (10) out=10b] v0'0 uv6 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f2b700a8070 con 0x55685e1eaa10 2024-09-19T04:15:19.967 INFO:teuthology.orchestra.run.smithi076.stdout:tag123 2024-09-19T04:15:19.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55685e1eaa10 msgr2=0x55685e1f0630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55685e1eaa10 0x55685e1f0630 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:19.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2b600662d0 msgr2=0x7f2b60068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2b600662d0 0x7f2b60068790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2b68005a50 tx=0x7f2b68011000 comp rx=0 tx=0).stop 2024-09-19T04:15:19.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16c9e0 msgr2=0x55685e1ea4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:19.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 --2- 172.21.15.76:0/2665715292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55685e16c9e0 0x55685e1ea4d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2b7800c460 tx=0x7f2b7800c920 comp rx=0 tx=0).stop 2024-09-19T04:15:19.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b80ebb640 1 -- 172.21.15.76:0/2665715292 reap_dead start 2024-09-19T04:15:19.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 shutdown_connections 2024-09-19T04:15:19.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 >> 172.21.15.76:0/2665715292 conn(0x55685e14d220 msgr2=0x55685e1e98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:19.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 shutdown_connections 2024-09-19T04:15:19.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:19.954+0000 7f2b82ba7080 1 -- 172.21.15.76:0/2665715292 wait complete. 2024-09-19T04:15:19.971 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000006.00000000 scrub_tag 2024-09-19T04:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.050+0000 7f8eb3c40080 1 Processor -- start 2024-09-19T04:15:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.050+0000 7f8eb3c40080 1 -- start start 2024-09-19T04:15:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea0002820 0x55b13c84a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c8db8a0 0x55b13c98b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b13c98b860 0x55b13c98bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b13c8641b0 con 0x55b13c98b860 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b13c76b1b0 con 0x55b13c8db8a0 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b13c864330 con 0x7f8ea0002820 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b13c98b860 0x55b13c98bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b13c98b860 0x55b13c98bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56676/0 (socket says 172.21.15.76:56676) 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 -- 172.21.15.76:0/3592484427 learned_addr learned my addr 172.21.15.76:0/3592484427 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb0f52640 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c8db8a0 0x55b13c98b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1753640 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea0002820 0x55b13c84a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 -- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea0002820 msgr2=0x55b13c84a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea0002820 0x55b13c84a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 -- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c8db8a0 msgr2=0x55b13c98b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:20.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c8db8a0 0x55b13c98b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 -- 172.21.15.76:0/3592484427 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13c8f6610 con 0x55b13c98b860 2024-09-19T04:15:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb1f54640 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b13c98b860 0x55b13c98bc70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55b13c818650 tx=0x7f8ea4005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e05b02416cfeba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eab7fe640 1 -- 172.21.15.76:0/3592484427 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4008b70 con 0x55b13c98b860 2024-09-19T04:15:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eab7fe640 1 -- 172.21.15.76:0/3592484427 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea4019070 con 0x55b13c98b860 2024-09-19T04:15:20.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eab7fe640 1 -- 172.21.15.76:0/3592484427 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4014400 con 0x55b13c98b860 2024-09-19T04:15:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- 172.21.15.76:0/3592484427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b13c98b860 msgr2=0x55b13c98bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b13c98b860 0x55b13c98bc70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x55b13c818650 tx=0x7f8ea4005a10 comp rx=0 tx=0).stop 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- 172.21.15.76:0/3592484427 shutdown_connections 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b13c98b860 0x55b13c98bc70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c8db8a0 0x55b13c98b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/3592484427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea0002820 0x55b13c84a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- 172.21.15.76:0/3592484427 >> 172.21.15.76:0/3592484427 conn(0x55b13c845220 msgr2=0x55b13c81d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- 172.21.15.76:0/3592484427 shutdown_connections 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- 172.21.15.76:0/3592484427 wait complete. 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 Processor -- start 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.054+0000 7f8eb3c40080 1 -- start start 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b13c8db8a0 0x55b13c8f2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c98b860 0x55b13c85db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea0002820 0x55b13c85e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b13c8dbef0 con 0x7f8ea0002820 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b13c98c1b0 con 0x55b13c98b860 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b13c8f6370 con 0x55b13c8db8a0 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea0002820 0x55b13c85e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea0002820 0x55b13c85e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56682/0 (socket says 172.21.15.76:56682) 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 -- 172.21.15.76:0/1357887802 learned_addr learned my addr 172.21.15.76:0/1357887802 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1753640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b13c8db8a0 0x55b13c8f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb0f52640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c98b860 0x55b13c85db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 -- 172.21.15.76:0/1357887802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b13c8db8a0 msgr2=0x55b13c8f2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b13c8db8a0 0x55b13c8f2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 -- 172.21.15.76:0/1357887802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c98b860 msgr2=0x55b13c85db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b13c98b860 0x55b13c85db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 -- 172.21.15.76:0/1357887802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b13c860dd0 con 0x7f8ea0002820 2024-09-19T04:15:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1f54640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea0002820 0x55b13c85e0a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4000c10 tx=0x7f8ea4002fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eaa7fc640 1 -- 172.21.15.76:0/1357887802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4014d30 con 0x7f8ea0002820 2024-09-19T04:15:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eaa7fc640 1 -- 172.21.15.76:0/1357887802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea4027430 con 0x7f8ea0002820 2024-09-19T04:15:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b13c85b2d0 con 0x7f8ea0002820 2024-09-19T04:15:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eaa7fc640 1 -- 172.21.15.76:0/1357887802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4027820 con 0x7f8ea0002820 2024-09-19T04:15:20.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b13c85b810 con 0x7f8ea0002820 2024-09-19T04:15:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eaa7fc640 1 -- 172.21.15.76:0/1357887802 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ea4013070 con 0x7f8ea0002820 2024-09-19T04:15:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eaa7fc640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940662d0 0x7f8e94068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1753640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940662d0 0x7f8e94068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eaa7fc640 1 -- 172.21.15.76:0/1357887802 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f8ea4086d70 con 0x7f8ea0002820 2024-09-19T04:15:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b13c85bf50 0x55b13c85c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 5.5 5:a5feb6f3:::10000000006.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x55b13c85c870 con 0x55b13c85bf50 2024-09-19T04:15:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb0f52640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b13c85bf50 0x55b13c85c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.058+0000 7f8eb1753640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940662d0 0x7f8e94068790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001cb0 tx=0x7f8eac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb0f52640 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b13c85bf50 0x55b13c85c330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb0f52640 1 -- 172.21.15.76:0/1357887802 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000006.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7f8e9c0af030 con 0x55b13c85bf50 2024-09-19T04:15:20.064 INFO:teuthology.orchestra.run.smithi076.stdout:tag123 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b13c85bf50 msgr2=0x55b13c85c330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b13c85bf50 0x55b13c85c330 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940662d0 msgr2=0x7f8e94068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940662d0 0x7f8e94068790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8eac001cb0 tx=0x7f8eac008040 comp rx=0 tx=0).stop 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea0002820 msgr2=0x55b13c85e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 --2- 172.21.15.76:0/1357887802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea0002820 0x55b13c85e0a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4000c10 tx=0x7f8ea4002fc0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb1f54640 1 -- 172.21.15.76:0/1357887802 reap_dead start 2024-09-19T04:15:20.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 shutdown_connections 2024-09-19T04:15:20.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 >> 172.21.15.76:0/1357887802 conn(0x55b13c845220 msgr2=0x55b13c81d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:20.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 shutdown_connections 2024-09-19T04:15:20.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.062+0000 7f8eb3c40080 1 -- 172.21.15.76:0/1357887802 wait complete. 2024-09-19T04:15:20.067 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000007.00000000 scrub_tag 2024-09-19T04:15:20.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 Processor -- start 2024-09-19T04:15:20.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 -- start start 2024-09-19T04:15:20.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886552db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x56188653a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x56188653b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5618863cc1b0 con 0x7fcfac002860 2024-09-19T04:15:20.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5618864257b0 con 0x56188653ad10 2024-09-19T04:15:20.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.146+0000 7fcfc2434080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5618864aef60 con 0x561886534650 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886552db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886552db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56704/0 (socket says 172.21.15.76:56704) 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 -- 172.21.15.76:0/3923956382 learned_addr learned my addr 172.21.15.76:0/3923956382 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbf746640 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x56188653a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x56188653b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 -- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 msgr2=0x56188653a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x56188653a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 -- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 msgr2=0x56188653b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x56188653b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 -- 172.21.15.76:0/3923956382 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561886534de0 con 0x7fcfac002860 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbf746640 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x56188653a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:20.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x56188653b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886552db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000d720 tx=0x7fcfb000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd50ef31a3f0a52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbe744640 1 -- 172.21.15.76:0/3923956382 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000bae0 con 0x7fcfac002860 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbe744640 1 -- 172.21.15.76:0/3923956382 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb0012030 con 0x7fcfac002860 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbe744640 1 -- 172.21.15.76:0/3923956382 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb0017420 con 0x7fcfac002860 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- 172.21.15.76:0/3923956382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 msgr2=0x561886552db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886552db0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000d720 tx=0x7fcfb000dbe0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- 172.21.15.76:0/3923956382 shutdown_connections 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x56188653b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x56188653a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 --2- 172.21.15.76:0/3923956382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886552db0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- 172.21.15.76:0/3923956382 >> 172.21.15.76:0/3923956382 conn(0x5618864ab460 msgr2=0x561886537710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- 172.21.15.76:0/3923956382 shutdown_connections 2024-09-19T04:15:20.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- 172.21.15.76:0/3923956382 wait complete. 2024-09-19T04:15:20.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 Processor -- start 2024-09-19T04:15:20.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- start start 2024-09-19T04:15:20.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x561886551cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x5618865521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886544740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5618864a8f60 con 0x7fcfac002860 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561886535080 con 0x56188653ad10 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5618864aa880 con 0x561886534650 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886544740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886544740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56714/0 (socket says 172.21.15.76:56714) 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 -- 172.21.15.76:0/1694700119 learned_addr learned my addr 172.21.15.76:0/1694700119 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbff47640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x561886551cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbf746640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x5618865521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 -- 172.21.15.76:0/1694700119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 msgr2=0x561886551cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561886534650 0x561886551cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 -- 172.21.15.76:0/1694700119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 msgr2=0x5618865521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56188653ad10 0x5618865521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 -- 172.21.15.76:0/1694700119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5618865ec7a0 con 0x7fcfac002860 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc0748640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886544740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5618864ad470 tx=0x7fcfb800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbd742640 1 -- 172.21.15.76:0/1694700119 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8017070 con 0x7fcfac002860 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbd742640 1 -- 172.21.15.76:0/1694700119 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb8013800 con 0x7fcfac002860 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5618865eca40 con 0x7fcfac002860 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfbd742640 1 -- 172.21.15.76:0/1694700119 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800e040 con 0x7fcfac002860 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.150+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5618865ecf80 con 0x7fcfac002860 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbd742640 1 -- 172.21.15.76:0/1694700119 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcfb801a070 con 0x7fcfac002860 2024-09-19T04:15:20.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbd742640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcfa00662d0 0x7fcfa0068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbd742640 1 -- 172.21.15.76:0/1694700119 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fcfb8084840 con 0x7fcfac002860 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbff47640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcfa00662d0 0x7fcfa0068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x5618865ec300 0x5618865427f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- osd_op(unknown.0.0:1 5.1d 5:bbf5850a:::10000000007.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41) -- 0x5618865ee6d0 con 0x5618865ec300 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbf746640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x5618865ec300 0x5618865427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbff47640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcfa00662d0 0x7fcfa0068790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000dc20 tx=0x7fcfb001a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbf746640 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x5618865ec300 0x5618865427f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfbf746640 1 -- 172.21.15.76:0/1694700119 <== osd.1 v2:172.21.15.76:6808/1010487110 1 ==== osd_op_reply(1 10000000007.00000000 [getxattr (10) out=10b] v0'0 uv7 ondisk = 0) ==== 164+0+10 (crc 0 0 0) 0x7fcfa80b3040 con 0x5618865ec300 2024-09-19T04:15:20.159 INFO:teuthology.orchestra.run.smithi076.stdout:tag123 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x5618865ec300 msgr2=0x5618865427f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x5618865ec300 0x5618865427f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcfa00662d0 msgr2=0x7fcfa0068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcfa00662d0 0x7fcfa0068790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000dc20 tx=0x7fcfb001a040 comp rx=0 tx=0).stop 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 msgr2=0x561886544740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 --2- 172.21.15.76:0/1694700119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcfac002860 0x561886544740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x5618864ad470 tx=0x7fcfb800f8b0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc0748640 1 -- 172.21.15.76:0/1694700119 reap_dead start 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 shutdown_connections 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 >> 172.21.15.76:0/1694700119 conn(0x5618864ab460 msgr2=0x56188653b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:20.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 shutdown_connections 2024-09-19T04:15:20.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.154+0000 7fcfc2434080 1 -- 172.21.15.76:0/1694700119 wait complete. 2024-09-19T04:15:20.162 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:15:20.162 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:15:20.259 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:15:20.344 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:15:20.345 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:15:20.345 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:15:20.345 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:20.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:15:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 Processor -- start 2024-09-19T04:15:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 -- start start 2024-09-19T04:15:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba681520a0 0x7fba6806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fba6806fe50 0x7fba6815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fba6815f1d0 0x7fba68071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fba68152480 con 0x7fba6806fe50 2024-09-19T04:15:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fba68070250 con 0x7fba6815f1d0 2024-09-19T04:15:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba70115640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fba680aede0 con 0x7fba681520a0 2024-09-19T04:15:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fba6806fe50 0x7fba6815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fba6806fe50 0x7fba6815ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56726/0 (socket says 172.21.15.76:56726) 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 -- 172.21.15.76:0/798682075 learned_addr learned my addr 172.21.15.76:0/798682075 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6e6b2640 1 --2- 172.21.15.76:0/798682075 >> v2:172.21.15.116:3300/0 conn(0x7fba6815f1d0 0x7fba68071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6deb1640 1 --2- 172.21.15.76:0/798682075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba681520a0 0x7fba6806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 -- 172.21.15.76:0/798682075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba681520a0 msgr2=0x7fba6806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 --2- 172.21.15.76:0/798682075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba681520a0 0x7fba6806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 -- 172.21.15.76:0/798682075 >> v2:172.21.15.116:3300/0 conn(0x7fba6815f1d0 msgr2=0x7fba68071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 --2- 172.21.15.76:0/798682075 >> v2:172.21.15.116:3300/0 conn(0x7fba6815f1d0 0x7fba68071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.522+0000 7fba6d6b0640 1 -- 172.21.15.76:0/798682075 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6815fa10 con 0x7fba6806fe50 2024-09-19T04:15:20.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba6d6b0640 1 --2- 172.21.15.76:0/798682075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba6815ec90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fba6806e250 tx=0x7fba5801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d5be996854ba8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba6ceaf640 1 -- 172.21.15.76:0/798682075 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580b0070 con 0x7fba6806fe50 2024-09-19T04:15:20.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba6ceaf640 1 -- 172.21.15.76:0/798682075 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba580ab600 con 0x7fba6806fe50 2024-09-19T04:15:20.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba6ceaf640 1 -- 172.21.15.76:0/798682075 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba580ab9d0 con 0x7fba6806fe50 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 -- 172.21.15.76:0/798682075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 msgr2=0x7fba6815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 --2- 172.21.15.76:0/798682075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba6815ec90 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fba6806e250 tx=0x7fba5801c240 comp rx=0 tx=0).stop 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 -- 172.21.15.76:0/798682075 shutdown_connections 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 --2- 172.21.15.76:0/798682075 >> v2:172.21.15.116:3300/0 conn(0x7fba6815f1d0 0x7fba68071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 --2- 172.21.15.76:0/798682075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba6815ec90 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 --2- 172.21.15.76:0/798682075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba681520a0 0x7fba6806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 -- 172.21.15.76:0/798682075 >> 172.21.15.76:0/798682075 conn(0x7fba680b1f90 msgr2=0x7fba680b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 -- 172.21.15.76:0/798682075 shutdown_connections 2024-09-19T04:15:20.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 -- 172.21.15.76:0/798682075 wait complete. 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 Processor -- start 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.526+0000 7fba70115640 1 -- start start 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba68099610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba681520a0 0x7fba68145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba6815f1d0 0x7fba68149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fba68073d30 con 0x7fba6806fe50 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fba68073a50 con 0x7fba681520a0 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fba6815f5e0 con 0x7fba6815f1d0 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba68099610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba68099610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56736/0 (socket says 172.21.15.76:56736) 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 -- 172.21.15.76:0/2218990955 learned_addr learned my addr 172.21.15.76:0/2218990955 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:20.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6d6b0640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba681520a0 0x7fba68145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6e6b2640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba6815f1d0 0x7fba68149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 -- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba6815f1d0 msgr2=0x7fba68149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba6815f1d0 0x7fba68149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 -- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba681520a0 msgr2=0x7fba68145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba681520a0 0x7fba68145050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 -- 172.21.15.76:0/2218990955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba68149080 con 0x7fba6806fe50 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6d6b0640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba681520a0 0x7fba68145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6e6b2640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba6815f1d0 0x7fba68149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6deb1640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba68099610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fba64007e60 tx=0x7fba6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba5e7fc640 1 -- 172.21.15.76:0/2218990955 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64014620 con 0x7fba6806fe50 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba5e7fc640 1 -- 172.21.15.76:0/2218990955 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba64019070 con 0x7fba6806fe50 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba68149360 con 0x7fba6806fe50 2024-09-19T04:15:20.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba5e7fc640 1 -- 172.21.15.76:0/2218990955 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba64014a00 con 0x7fba6806fe50 2024-09-19T04:15:20.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba68099b50 con 0x7fba6806fe50 2024-09-19T04:15:20.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba6815ec90 con 0x7fba6806fe50 2024-09-19T04:15:20.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba5e7fc640 1 -- 172.21.15.76:0/2218990955 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba64014ba0 con 0x7fba6806fe50 2024-09-19T04:15:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba5e7fc640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba4c0678c0 0x7fba4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba5e7fc640 1 -- 172.21.15.76:0/2218990955 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fba64086620 con 0x7fba6806fe50 2024-09-19T04:15:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.530+0000 7fba6d6b0640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba4c0678c0 0x7fba4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:20.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.534+0000 7fba6d6b0640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba4c0678c0 0x7fba4c069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba580ad000 tx=0x7fba58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:20.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.538+0000 7fba5e7fc640 1 -- 172.21.15.76:0/2218990955 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fba64057040 con 0x7fba6806fe50 2024-09-19T04:15:20.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.706+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba6806e540 con 0x7fba6806fe50 2024-09-19T04:15:20.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba5e7fc640 1 -- 172.21.15.76:0/2218990955 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v41) ==== 80+0+0 (secure 0 0 0) 0x7fba64056940 con 0x7fba6806fe50 2024-09-19T04:15:20.710 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:15:20.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba4c0678c0 msgr2=0x7fba4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba4c0678c0 0x7fba4c069d80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fba580ad000 tx=0x7fba58008040 comp rx=0 tx=0).stop 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 msgr2=0x7fba68099610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba68099610 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fba64007e60 tx=0x7fba6400e4b0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 shutdown_connections 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba4c0678c0 0x7fba4c069d80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba6815f1d0 0x7fba68149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba681520a0 0x7fba68145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 --2- 172.21.15.76:0/2218990955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba6806fe50 0x7fba68099610 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 >> 172.21.15.76:0/2218990955 conn(0x7fba680b1f90 msgr2=0x7fba6815b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 shutdown_connections 2024-09-19T04:15:20.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:20.710+0000 7fba70115640 1 -- 172.21.15.76:0/2218990955 wait complete. 2024-09-19T04:15:20.729 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:15:20.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:20.730 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:20.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[38270]: fuse finished with error 0 and tester_r 0 2024-09-19T04:15:20.742 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:15:20.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:20.742 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:15:20.910 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:15:20.910 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:15:20.910 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:20.916 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:15:20.917 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:15:20.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:20.917 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T04:15:20.963 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T04:15:20.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:15:20.964 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:15:20.965 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:15:20.965 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:20.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:15:20.971 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:15:20.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:20.971 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T04:15:21.018 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T04:15:21.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:15:21.019 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:15:21.019 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:15:21.019 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:21.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:15:21.025 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:15:21.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:21.026 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T04:15:21.071 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T04:15:21.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:15:21.072 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:21.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac965f640 1 Processor -- start 2024-09-19T04:15:21.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac965f640 1 -- start start 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac965f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415c2a0 0x7f8ac40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac965f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ac40af2a0 0x7f8ac406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac965f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ac40b3290 con 0x7f8ac415bec0 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac965f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8ac40b3570 con 0x7f8ac40af2a0 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac965f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac40b3850 con 0x7f8ac415c2a0 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8ac40af2a0 0x7f8ac406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac2ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8ac415bec0 0x7f8ac40ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57756/0 (socket says 172.21.15.76:57756) 2024-09-19T04:15:21.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac2ffd640 1 -- 172.21.15.76:0/4188769454 learned_addr learned my addr 172.21.15.76:0/4188769454 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:21.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac27fc640 1 --2- 172.21.15.76:0/4188769454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415c2a0 0x7f8ac40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac37fe640 1 -- 172.21.15.76:0/4188769454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415c2a0 msgr2=0x7f8ac40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:21.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac37fe640 1 --2- 172.21.15.76:0/4188769454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415c2a0 0x7f8ac40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac37fe640 1 -- 172.21.15.76:0/4188769454 >> v1:172.21.15.76:6789/0 conn(0x7f8ac415bec0 legacy=0x7f8ac40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac37fe640 1 -- 172.21.15.76:0/4188769454 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac40b3b30 con 0x7f8ac40af2a0 2024-09-19T04:15:21.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.202+0000 7f8ac1ffb640 1 -- 172.21.15.76:0/4188769454 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2706083730 0 0) 0x7f8ac40b3290 con 0x7f8ac415bec0 2024-09-19T04:15:21.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac37fe640 1 --2- 172.21.15.76:0/4188769454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac406e000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d90 tx=0x7f8abc01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad8efe28576fea2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac1ffb640 1 -- 172.21.15.76:0/4188769454 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0b2070 con 0x7f8ac40af2a0 2024-09-19T04:15:21.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac1ffb640 1 -- 172.21.15.76:0/4188769454 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8abc0a5a00 con 0x7f8ac40af2a0 2024-09-19T04:15:21.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac1ffb640 1 -- 172.21.15.76:0/4188769454 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8abc0a5dd0 con 0x7f8ac40af2a0 2024-09-19T04:15:21.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 -- 172.21.15.76:0/4188769454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 msgr2=0x7f8ac406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 --2- 172.21.15.76:0/4188769454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac406e000 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009d90 tx=0x7f8abc01c940 comp rx=0 tx=0).stop 2024-09-19T04:15:21.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 -- 172.21.15.76:0/4188769454 shutdown_connections 2024-09-19T04:15:21.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 --2- 172.21.15.76:0/4188769454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac406e000 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 --2- 172.21.15.76:0/4188769454 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415c2a0 0x7f8ac40aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 -- 172.21.15.76:0/4188769454 >> 172.21.15.76:0/4188769454 conn(0x7f8ac40b1f90 msgr2=0x7f8ac40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:21.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 -- 172.21.15.76:0/4188769454 shutdown_connections 2024-09-19T04:15:21.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 -- 172.21.15.76:0/4188769454 wait complete. 2024-09-19T04:15:21.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 Processor -- start 2024-09-19T04:15:21.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.206+0000 7f8ac965f640 1 -- start start 2024-09-19T04:15:21.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415bec0 0x7f8ac414ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ac415c2a0 0x7f8ac41556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac406ed50 con 0x7f8ac40af2a0 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ac40afc90 con 0x7f8ac415c2a0 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ac406eed0 con 0x7f8ac415bec0 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac414b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56752/0 (socket says 172.21.15.76:56752) 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 -- 172.21.15.76:0/70848685 learned_addr learned my addr 172.21.15.76:0/70848685 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac27fc640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415bec0 0x7f8ac414ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac37fe640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ac415c2a0 0x7f8ac41556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 -- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415bec0 msgr2=0x7f8ac414ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415bec0 0x7f8ac414ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 -- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ac415c2a0 msgr2=0x7f8ac41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ac415c2a0 0x7f8ac41556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 -- 172.21.15.76:0/70848685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ac4154640 con 0x7f8ac40af2a0 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac27fc640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415bec0 0x7f8ac414ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac37fe640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ac415c2a0 0x7f8ac41556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac2ffd640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac414b040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80011f0 tx=0x7f8ab800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8aab7fe640 1 -- 172.21.15.76:0/70848685 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8018070 con 0x7f8ac40af2a0 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8aab7fe640 1 -- 172.21.15.76:0/70848685 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ab80074e0 con 0x7f8ac40af2a0 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ac4154920 con 0x7f8ac40af2a0 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8aab7fe640 1 -- 172.21.15.76:0/70848685 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8004a10 con 0x7f8ac40af2a0 2024-09-19T04:15:21.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ac4154ee0 con 0x7f8ac40af2a0 2024-09-19T04:15:21.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ac406f230 con 0x7f8ac40af2a0 2024-09-19T04:15:21.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8aab7fe640 1 -- 172.21.15.76:0/70848685 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ab801b070 con 0x7f8ac40af2a0 2024-09-19T04:15:21.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8aab7fe640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ab00678f0 0x7f8ab0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8ac27fc640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ab00678f0 0x7f8ab0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.210+0000 7f8aab7fe640 1 -- 172.21.15.76:0/70848685 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f8ab80850f0 con 0x7f8ac40af2a0 2024-09-19T04:15:21.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.214+0000 7f8ac27fc640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ab00678f0 0x7f8ab0069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097a0 tx=0x7f8ab4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.218+0000 7f8aab7fe640 1 -- 172.21.15.76:0/70848685 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8ab8055b10 con 0x7f8ac40af2a0 2024-09-19T04:15:21.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.410+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8ac406fcd0 con 0x7f8ac40af2a0 2024-09-19T04:15:21.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.410+0000 7f8aab7fe640 1 -- 172.21.15.76:0/70848685 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5273 (secure 0 0 0) 0x7f8ab8055410 con 0x7f8ac40af2a0 2024-09-19T04:15:21.413 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:21.413 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"btime":"2024-09-19T04:14:51:896699+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:51.152737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4863,"mds_1":4854,"mds_2":4666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4666":{"gid":4666,"name":"d","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4854":{"gid":4854,"name":"a","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4863,4854,4666]},"id":2}]} 2024-09-19T04:15:21.413 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 22 2024-09-19T04:15:21.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ab00678f0 msgr2=0x7f8ab0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ab00678f0 0x7f8ab0069db0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8ab40097a0 tx=0x7f8ab4008040 comp rx=0 tx=0).stop 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 msgr2=0x7f8ac414b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac414b040 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8ab80011f0 tx=0x7f8ab800c9b0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 shutdown_connections 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ab00678f0 0x7f8ab0069db0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ac415c2a0 0x7f8ac41556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ac415bec0 0x7f8ac414ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 --2- 172.21.15.76:0/70848685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ac40af2a0 0x7f8ac414b040 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 >> 172.21.15.76:0/70848685 conn(0x7f8ac40b1f90 msgr2=0x7f8ac415af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 shutdown_connections 2024-09-19T04:15:21.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.414+0000 7f8ac965f640 1 -- 172.21.15.76:0/70848685 wait complete. 2024-09-19T04:15:21.433 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:21.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd813214640 1 Processor -- start 2024-09-19T04:15:21.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd813214640 1 -- start start 2024-09-19T04:15:21.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd813214640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 0x7fd80c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd813214640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd80c15c2a0 0x7fd80c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd813214640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd80c0b3290 con 0x7fd80c0af2a0 2024-09-19T04:15:21.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd813214640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd80c0b3570 con 0x7fd80c15c2a0 2024-09-19T04:15:21.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd813214640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0b3850 con 0x7fd80c15bec0 2024-09-19T04:15:21.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd8117b1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd80c0af2a0 0x7fd80c15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57770/0 (socket says 172.21.15.76:57770) 2024-09-19T04:15:21.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd8117b1640 1 -- 172.21.15.76:0/451834396 learned_addr learned my addr 172.21.15.76:0/451834396 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd810fb0640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 0x7fd80c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80bfff640 1 --2- 172.21.15.76:0/451834396 >> v2:172.21.15.116:3300/0 conn(0x7fd80c15c2a0 0x7fd80c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 346967077 0 0) 0x7fd80c0b3290 con 0x7fd80c0af2a0 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7f8003530 con 0x7fd80c0af2a0 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3562403317 0 0) 0x7fd7f8003530 con 0x7fd80c0af2a0 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 msgr2=0x7fd80c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 0x7fd80c0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 >> v2:172.21.15.116:3300/0 conn(0x7fd80c15c2a0 msgr2=0x7fd80c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:21.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 --2- 172.21.15.76:0/451834396 >> v2:172.21.15.116:3300/0 conn(0x7fd80c15c2a0 0x7fd80c0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0b3b30 con 0x7fd80c0af2a0 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fd804001a00 con 0x7fd80c0af2a0 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 >> v1:172.21.15.76:6789/0 conn(0x7fd80c0af2a0 legacy=0x7fd80c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7f80039c0 0x7fd7f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f8004810 0x7fd7f8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 0x7fd7f80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0b3570 con 0x7fd7f80039c0 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0b3850 con 0x7fd7f8004810 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7f800b9d0 con 0x7fd80c15bec0 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.554+0000 7fd810fb0640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7f80039c0 0x7fd7f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd80bfff640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f8004810 0x7fd7f8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd810fb0640 1 -- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 msgr2=0x7fd7f80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd8117b1640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 0x7fd7f80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd810fb0640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 0x7fd7f80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd810fb0640 1 -- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f8004810 msgr2=0x7fd7f8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd810fb0640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7f8004810 0x7fd7f8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd810fb0640 1 -- 172.21.15.76:0/451834396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f8003530 con 0x7fd7f80039c0 2024-09-19T04:15:21.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd8117b1640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd80c15bec0 0x7fd7f80051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:21.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd810fb0640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7f80039c0 0x7fd7f8003de0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009ac0 tx=0x7fd7f401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978bc52fd450f6de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f40b2070 con 0x7fd7f80039c0 2024-09-19T04:15:21.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7f40a5510 con 0x7fd7f80039c0 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd80b7fe640 1 -- 172.21.15.76:0/451834396 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4009690 con 0x7fd7f80039c0 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd813214640 1 -- 172.21.15.76:0/451834396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7f80039c0 msgr2=0x7fd7f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd813214640 1 --2- 172.21.15.76:0/451834396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7f80039c0 0x7fd7f8003de0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4009ac0 tx=0x7fd7f401ca20 comp rx=0 tx=0).stop 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd8117b1640 1 -- 172.21.15.76:0/451834396 reap_dead start 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd813214640 1 -- 172.21.15.76:0/451834396 shutdown_connections 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd813214640 1 -- 172.21.15.76:0/451834396 >> 172.21.15.76:0/451834396 conn(0x7fd80c0b1f90 msgr2=0x7fd80c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd813214640 1 -- 172.21.15.76:0/451834396 shutdown_connections 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.558+0000 7fd813214640 1 -- 172.21.15.76:0/451834396 wait complete. 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 Processor -- start 2024-09-19T04:15:21.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 -- start start 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7f80039c0 0x7fd80c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80c15ef00 0x7fd80c1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80c0d0410 0x7fd80c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0ad610 con 0x7fd80c15ef00 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd80c0af6d0 con 0x7fd80c0d0410 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd80c0ad790 con 0x7fd7f80039c0 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80c15ef00 0x7fd80c1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80c15ef00 0x7fd80c1f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56774/0 (socket says 172.21.15.76:56774) 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 -- 172.21.15.76:0/2365586958 learned_addr learned my addr 172.21.15.76:0/2365586958 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:21.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd810fb0640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7f80039c0 0x7fd80c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd8117b1640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80c0d0410 0x7fd80c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 -- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7f80039c0 msgr2=0x7fd80c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7f80039c0 0x7fd80c155cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 -- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80c0d0410 msgr2=0x7fd80c0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80c0d0410 0x7fd80c0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 -- 172.21.15.76:0/2365586958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd80c0cf1a0 con 0x7fd80c15ef00 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd8117b1640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80c0d0410 0x7fd80c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd80bfff640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80c15ef00 0x7fd80c1f52e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc011600 tx=0x7fd7fc011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd8097fa640 1 -- 172.21.15.76:0/2365586958 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00f8c0 con 0x7fd80c15ef00 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd8097fa640 1 -- 172.21.15.76:0/2365586958 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7fc00fa60 con 0x7fd80c15ef00 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd8097fa640 1 -- 172.21.15.76:0/2365586958 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7fc00fe10 con 0x7fd80c15ef00 2024-09-19T04:15:21.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd80c156380 con 0x7fd80c15ef00 2024-09-19T04:15:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd80c156940 con 0x7fd80c15ef00 2024-09-19T04:15:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.562+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd80c0d0dd0 con 0x7fd80c15ef00 2024-09-19T04:15:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.566+0000 7fd8097fa640 1 -- 172.21.15.76:0/2365586958 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd7fc015460 con 0x7fd80c15ef00 2024-09-19T04:15:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.566+0000 7fd8097fa640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7ec0678f0 0x7fd7ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.566+0000 7fd810fb0640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7ec0678f0 0x7fd7ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.566+0000 7fd8097fa640 1 -- 172.21.15.76:0/2365586958 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fd7fc086710 con 0x7fd80c15ef00 2024-09-19T04:15:21.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.566+0000 7fd810fb0640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7ec0678f0 0x7fd7ec069db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40af000 tx=0x7fd7f40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.570+0000 7fd8097fa640 1 -- 172.21.15.76:0/2365586958 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd7fc019030 con 0x7fd80c15ef00 2024-09-19T04:15:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd80c0aed60 con 0x7fd80c15ef00 2024-09-19T04:15:21.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd8097fa640 1 -- 172.21.15.76:0/2365586958 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5273 (secure 0 0 0) 0x7fd7fc057080 con 0x7fd80c15ef00 2024-09-19T04:15:21.771 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:21.771 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"btime":"2024-09-19T04:14:51:896699+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:51.152737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4863,"mds_1":4854,"mds_2":4666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4666":{"gid":4666,"name":"d","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4854":{"gid":4854,"name":"a","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4863,4854,4666]},"id":2}]} 2024-09-19T04:15:21.771 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 22 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7ec0678f0 msgr2=0x7fd7ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7ec0678f0 0x7fd7ec069db0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd7f40af000 tx=0x7fd7f40046d0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80c15ef00 msgr2=0x7fd80c1f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80c15ef00 0x7fd80c1f52e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc011600 tx=0x7fd7fc011ac0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 shutdown_connections 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7ec0678f0 0x7fd7ec069db0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd80c0d0410 0x7fd80c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd80c15ef00 0x7fd80c1f52e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 --2- 172.21.15.76:0/2365586958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7f80039c0 0x7fd80c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 >> 172.21.15.76:0/2365586958 conn(0x7fd80c0b1f90 msgr2=0x7fd80c0b1120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:21.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 shutdown_connections 2024-09-19T04:15:21.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.770+0000 7fd813214640 1 -- 172.21.15.76:0/2365586958 wait complete. 2024-09-19T04:15:21.789 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:15:21.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.894+0000 7f404c253640 1 Processor -- start 2024-09-19T04:15:21.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.894+0000 7f404c253640 1 -- start start 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f404c253640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f404415bec0 0x7f40440ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f404c253640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f404415c2a0 0x7f40440aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f404c253640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40440b3290 con 0x7f40440af2a0 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f404c253640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f40440b3570 con 0x7f404415bec0 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f404c253640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f40440b3850 con 0x7f404415c2a0 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f404415bec0 0x7f40440ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f404415bec0 0x7f40440ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56778/0 (socket says 172.21.15.76:56778) 2024-09-19T04:15:21.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 -- 172.21.15.76:0/1165978599 learned_addr learned my addr 172.21.15.76:0/1165978599 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:21.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f40497ee640 1 --2- 172.21.15.76:0/1165978599 >> v2:172.21.15.116:3300/0 conn(0x7f404415c2a0 0x7f40440aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 -- 172.21.15.76:0/1165978599 >> v2:172.21.15.116:3300/0 conn(0x7f404415c2a0 msgr2=0x7f40440aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:21.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 --2- 172.21.15.76:0/1165978599 >> v2:172.21.15.116:3300/0 conn(0x7f404415c2a0 0x7f40440aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 -- 172.21.15.76:0/1165978599 >> v1:172.21.15.116:6789/0 conn(0x7f40440af2a0 legacy=0x7f404415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 -- 172.21.15.76:0/1165978599 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440b3b30 con 0x7f404415bec0 2024-09-19T04:15:21.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4049fef640 1 --2- 172.21.15.76:0/1165978599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415bec0 0x7f40440ad200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40440ae510 tx=0x7f404001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece62a78c247b577 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4048fed640 1 -- 172.21.15.76:0/1165978599 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40400aa650 con 0x7f404415bec0 2024-09-19T04:15:21.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4048fed640 1 -- 172.21.15.76:0/1165978599 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40400af070 con 0x7f404415bec0 2024-09-19T04:15:21.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.898+0000 7f4048fed640 1 -- 172.21.15.76:0/1165978599 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40400aaa10 con 0x7f404415bec0 2024-09-19T04:15:21.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/1165978599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415bec0 msgr2=0x7f40440ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 --2- 172.21.15.76:0/1165978599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415bec0 0x7f40440ad200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40440ae510 tx=0x7f404001c4f0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/1165978599 shutdown_connections 2024-09-19T04:15:21.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 --2- 172.21.15.76:0/1165978599 >> v2:172.21.15.116:3300/0 conn(0x7f404415c2a0 0x7f40440aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 --2- 172.21.15.76:0/1165978599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415bec0 0x7f40440ad200 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/1165978599 >> 172.21.15.76:0/1165978599 conn(0x7f40440b1f90 msgr2=0x7f40440b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:21.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/1165978599 shutdown_connections 2024-09-19T04:15:21.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/1165978599 wait complete. 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 Processor -- start 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- start start 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40440af2a0 0x7f4044156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404415bec0 0x7f404414f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415c2a0 0x7f4044159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f40440af6b0 con 0x7f404415c2a0 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f40440ad740 con 0x7f40440af2a0 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4044169000 con 0x7f404415bec0 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415c2a0 0x7f4044159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415c2a0 0x7f4044159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56786/0 (socket says 172.21.15.76:56786) 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 -- 172.21.15.76:0/3132529691 learned_addr learned my addr 172.21.15.76:0/3132529691 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f4049fef640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40440af2a0 0x7f4044156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 -- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404415bec0 msgr2=0x7f404414f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f40497ee640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404415bec0 0x7f404414f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404415bec0 0x7f404414f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 -- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40440af2a0 msgr2=0x7f4044156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:21.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40440af2a0 0x7f4044156d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:21.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 -- 172.21.15.76:0/3132529691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4044157650 con 0x7f404415c2a0 2024-09-19T04:15:21.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404a7f0640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415c2a0 0x7f4044159da0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f403c00c770 tx=0x7f403c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f403a7fc640 1 -- 172.21.15.76:0/3132529691 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c003b80 con 0x7f404415c2a0 2024-09-19T04:15:21.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f403a7fc640 1 -- 172.21.15.76:0/3132529691 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f403c003d20 con 0x7f404415c2a0 2024-09-19T04:15:21.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4044157930 con 0x7f404415c2a0 2024-09-19T04:15:21.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f403a7fc640 1 -- 172.21.15.76:0/3132529691 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c00aa90 con 0x7f404415c2a0 2024-09-19T04:15:21.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4044157ef0 con 0x7f404415c2a0 2024-09-19T04:15:21.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.902+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40440ad200 con 0x7f404415c2a0 2024-09-19T04:15:21.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.906+0000 7f403a7fc640 1 -- 172.21.15.76:0/3132529691 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f403c00ac30 con 0x7f404415c2a0 2024-09-19T04:15:21.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.906+0000 7f403a7fc640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40280678f0 0x7f4028069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:21.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.906+0000 7f403a7fc640 1 -- 172.21.15.76:0/3132529691 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f403c085e10 con 0x7f404415c2a0 2024-09-19T04:15:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.906+0000 7f4049fef640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40280678f0 0x7f4028069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.906+0000 7f4049fef640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40280678f0 0x7f4028069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f40400ae000 tx=0x7f4040008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:21.910+0000 7f403a7fc640 1 -- 172.21.15.76:0/3132529691 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f403c089030 con 0x7f404415c2a0 2024-09-19T04:15:22.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.062+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f404406fcd0 con 0x7f404415c2a0 2024-09-19T04:15:22.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f403a7fc640 1 -- 172.21.15.76:0/3132529691 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17075 (secure 0 0 0) 0x7f403c056700 con 0x7f404415c2a0 2024-09-19T04:15:22.066 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:22.067 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":41,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:15:06.946979+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:14:15.256172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T04:14:16.419970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40280678f0 msgr2=0x7f4028069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40280678f0 0x7f4028069db0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f40400ae000 tx=0x7f4040008040 comp rx=0 tx=0).stop 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415c2a0 msgr2=0x7f4044159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415c2a0 0x7f4044159da0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f403c00c770 tx=0x7f403c00cc30 comp rx=0 tx=0).stop 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 shutdown_connections 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40280678f0 0x7f4028069db0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404415c2a0 0x7f4044159da0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404415bec0 0x7f404414f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 --2- 172.21.15.76:0/3132529691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40440af2a0 0x7f4044156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 >> 172.21.15.76:0/3132529691 conn(0x7f40440b1f90 msgr2=0x7f40440b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 shutdown_connections 2024-09-19T04:15:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.066+0000 7f404c253640 1 -- 172.21.15.76:0/3132529691 wait complete. 2024-09-19T04:15:22.085 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:15:22.085 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:22.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fcc143640 1 Processor -- start 2024-09-19T04:15:22.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fcc143640 1 -- start start 2024-09-19T04:15:22.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fcc143640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fc415c2a0 0x7f1fc40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fcc143640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc40af2a0 0x7f1fc406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fcc143640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc40b3290 con 0x7f1fc415bec0 2024-09-19T04:15:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fcc143640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc40b3570 con 0x7f1fc415c2a0 2024-09-19T04:15:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fcc143640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc40b3850 con 0x7f1fc40af2a0 2024-09-19T04:15:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fc9edf640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1fc415bec0 0x7f1fc40ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:50956/0 (socket says 172.21.15.76:50956) 2024-09-19T04:15:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fc9edf640 1 -- 172.21.15.76:0/2928184598 learned_addr learned my addr 172.21.15.76:0/2928184598 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fca6e0640 1 --2- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc40af2a0 0x7f1fc406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fc96de640 1 --2- 172.21.15.76:0/2928184598 >> v2:172.21.15.116:3300/0 conn(0x7f1fc415c2a0 0x7f1fc40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fc96de640 1 -- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc40af2a0 msgr2=0x7f1fc406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fc96de640 1 --2- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc40af2a0 0x7f1fc406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fc96de640 1 -- 172.21.15.76:0/2928184598 >> v1:172.21.15.116:6789/0 conn(0x7f1fc415bec0 legacy=0x7f1fc40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.194+0000 7f1fc96de640 1 -- 172.21.15.76:0/2928184598 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc40b3b30 con 0x7f1fc415c2a0 2024-09-19T04:15:22.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fc96de640 1 --2- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 0x7f1fc40aed60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00097c0 tx=0x7f1fc001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b3148f214938b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fc8edd640 1 -- 172.21.15.76:0/2928184598 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00a7070 con 0x7f1fc415c2a0 2024-09-19T04:15:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fc8edd640 1 -- 172.21.15.76:0/2928184598 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fc00b0070 con 0x7f1fc415c2a0 2024-09-19T04:15:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fc8edd640 1 -- 172.21.15.76:0/2928184598 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc00ab890 con 0x7f1fc415c2a0 2024-09-19T04:15:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 -- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 msgr2=0x7f1fc40aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 --2- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 0x7f1fc40aed60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00097c0 tx=0x7f1fc001c470 comp rx=0 tx=0).stop 2024-09-19T04:15:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 -- 172.21.15.76:0/2928184598 shutdown_connections 2024-09-19T04:15:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 --2- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc40af2a0 0x7f1fc406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 --2- 172.21.15.76:0/2928184598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 0x7f1fc40aed60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 -- 172.21.15.76:0/2928184598 >> 172.21.15.76:0/2928184598 conn(0x7f1fc40b1f90 msgr2=0x7f1fc40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 -- 172.21.15.76:0/2928184598 shutdown_connections 2024-09-19T04:15:22.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 -- 172.21.15.76:0/2928184598 wait complete. 2024-09-19T04:15:22.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 Processor -- start 2024-09-19T04:15:22.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.198+0000 7f1fcc143640 1 -- start start 2024-09-19T04:15:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc40af2a0 0x7f1fc414ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc415bec0 0x7f1fc4157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 0x7f1fc414e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc406ed50 con 0x7f1fc40af2a0 2024-09-19T04:15:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc40af6b0 con 0x7f1fc415c2a0 2024-09-19T04:15:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc406eed0 con 0x7f1fc415bec0 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc40af2a0 0x7f1fc414ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc40af2a0 0x7f1fc414ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56802/0 (socket says 172.21.15.76:56802) 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 -- 172.21.15.76:0/754857437 learned_addr learned my addr 172.21.15.76:0/754857437 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc96de640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc415bec0 0x7f1fc4157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fca6e0640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 0x7f1fc414e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 -- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc415bec0 msgr2=0x7f1fc4157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc415bec0 0x7f1fc4157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 -- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 msgr2=0x7f1fc414e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 0x7f1fc414e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 -- 172.21.15.76:0/754857437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc415c6b0 con 0x7f1fc40af2a0 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc9edf640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc40af2a0 0x7f1fc414ac20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c450 tx=0x7f1fb400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fba7fc640 1 -- 172.21.15.76:0/754857437 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4017070 con 0x7f1fc40af2a0 2024-09-19T04:15:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fba7fc640 1 -- 172.21.15.76:0/754857437 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb4005cf0 con 0x7f1fc40af2a0 2024-09-19T04:15:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc414bef0 con 0x7f1fc40af2a0 2024-09-19T04:15:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fba7fc640 1 -- 172.21.15.76:0/754857437 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4007290 con 0x7f1fc40af2a0 2024-09-19T04:15:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc414c4b0 con 0x7f1fc40af2a0 2024-09-19T04:15:22.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc406e000 con 0x7f1fc40af2a0 2024-09-19T04:15:22.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fba7fc640 1 -- 172.21.15.76:0/754857437 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1fb401a070 con 0x7f1fc40af2a0 2024-09-19T04:15:22.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fba7fc640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fa80678c0 0x7f1fa8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fba7fc640 1 -- 172.21.15.76:0/754857437 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f1fb4085150 con 0x7f1fc40af2a0 2024-09-19T04:15:22.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc96de640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fa80678c0 0x7f1fa8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.202+0000 7f1fc96de640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fa80678c0 0x7f1fa8069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00af000 tx=0x7f1fc00b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.206+0000 7f1fba7fc640 1 -- 172.21.15.76:0/754857437 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1fb4055ae0 con 0x7f1fc40af2a0 2024-09-19T04:15:22.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.390+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fc406fcd0 con 0x7f1fc40af2a0 2024-09-19T04:15:22.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fba7fc640 1 -- 172.21.15.76:0/754857437 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5273 (secure 0 0 0) 0x7f1fb40553e0 con 0x7f1fc40af2a0 2024-09-19T04:15:22.394 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:22.395 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"btime":"2024-09-19T04:14:51:896699+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:51.152737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4863,"mds_1":4854,"mds_2":4666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4666":{"gid":4666,"name":"d","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4854":{"gid":4854,"name":"a","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4863,4854,4666]},"id":2}]} 2024-09-19T04:15:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 22 2024-09-19T04:15:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fa80678c0 msgr2=0x7f1fa8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fa80678c0 0x7f1fa8069d80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00af000 tx=0x7f1fc00b6040 comp rx=0 tx=0).stop 2024-09-19T04:15:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc40af2a0 msgr2=0x7f1fc414ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc40af2a0 0x7f1fc414ac20 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400c450 tx=0x7f1fb400c910 comp rx=0 tx=0).stop 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 shutdown_connections 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1fa80678c0 0x7f1fa8069d80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fc415c2a0 0x7f1fc414e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1fc415bec0 0x7f1fc4157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 --2- 172.21.15.76:0/754857437 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1fc40af2a0 0x7f1fc414ac20 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 >> 172.21.15.76:0/754857437 conn(0x7f1fc40b1f90 msgr2=0x7f1fc41597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 shutdown_connections 2024-09-19T04:15:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.394+0000 7f1fcc143640 1 -- 172.21.15.76:0/754857437 wait complete. 2024-09-19T04:15:22.412 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:22.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 Processor -- start 2024-09-19T04:15:22.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 -- start start 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06801520a0 0x7f068006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068006fe50 0x7f068015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f068015f1d0 0x7f0680071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0680152480 con 0x7f06801520a0 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0680070250 con 0x7f068015f1d0 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f0685c32640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f06800aede0 con 0x7f068006fe50 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f067f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06801520a0 0x7f068006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f067f7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f06801520a0 0x7f068006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56810/0 (socket says 172.21.15.76:56810) 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.530+0000 7f067f7fe640 1 -- 172.21.15.76:0/460419825 learned_addr learned my addr 172.21.15.76:0/460419825 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067effd640 1 --2- 172.21.15.76:0/460419825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068006fe50 0x7f068015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067ffff640 1 --2- 172.21.15.76:0/460419825 >> v2:172.21.15.116:3300/0 conn(0x7f068015f1d0 0x7f0680071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067f7fe640 1 -- 172.21.15.76:0/460419825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068006fe50 msgr2=0x7f068015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067f7fe640 1 --2- 172.21.15.76:0/460419825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068006fe50 0x7f068015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067f7fe640 1 -- 172.21.15.76:0/460419825 >> v2:172.21.15.116:3300/0 conn(0x7f068015f1d0 msgr2=0x7f0680071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067f7fe640 1 --2- 172.21.15.76:0/460419825 >> v2:172.21.15.116:3300/0 conn(0x7f068015f1d0 0x7f0680071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067f7fe640 1 -- 172.21.15.76:0/460419825 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f068015fa10 con 0x7f06801520a0 2024-09-19T04:15:22.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067f7fe640 1 --2- 172.21.15.76:0/460419825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06801520a0 0x7f068006f910 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f068006e250 tx=0x7f067001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d305fc98100034e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067e7fc640 1 -- 172.21.15.76:0/460419825 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700a5590 con 0x7f06801520a0 2024-09-19T04:15:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067e7fc640 1 -- 172.21.15.76:0/460419825 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06700a5730 con 0x7f06801520a0 2024-09-19T04:15:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f067e7fc640 1 -- 172.21.15.76:0/460419825 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700a5b00 con 0x7f06801520a0 2024-09-19T04:15:22.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 -- 172.21.15.76:0/460419825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06801520a0 msgr2=0x7f068006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 --2- 172.21.15.76:0/460419825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06801520a0 0x7f068006f910 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f068006e250 tx=0x7f067001c710 comp rx=0 tx=0).stop 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 -- 172.21.15.76:0/460419825 shutdown_connections 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 --2- 172.21.15.76:0/460419825 >> v2:172.21.15.116:3300/0 conn(0x7f068015f1d0 0x7f0680071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 --2- 172.21.15.76:0/460419825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f068006fe50 0x7f068015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 --2- 172.21.15.76:0/460419825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06801520a0 0x7f068006f910 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 -- 172.21.15.76:0/460419825 >> 172.21.15.76:0/460419825 conn(0x7f06800b1f90 msgr2=0x7f06800b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 -- 172.21.15.76:0/460419825 shutdown_connections 2024-09-19T04:15:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.534+0000 7f0685c32640 1 -- 172.21.15.76:0/460419825 wait complete. 2024-09-19T04:15:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 Processor -- start 2024-09-19T04:15:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 -- start start 2024-09-19T04:15:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f068006fe50 0x7f068014bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06801520a0 0x7f06800cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068015f1d0 0x7f06800b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0680073d30 con 0x7f068006fe50 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0680073a50 con 0x7f068015f1d0 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f068015f5e0 con 0x7f06801520a0 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f068006fe50 0x7f068014bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f068006fe50 0x7f068014bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56814/0 (socket says 172.21.15.76:56814) 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 -- 172.21.15.76:0/761507998 learned_addr learned my addr 172.21.15.76:0/761507998 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067effd640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06801520a0 0x7f06800cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067ffff640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068015f1d0 0x7f06800b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 -- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06801520a0 msgr2=0x7f06800cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06801520a0 0x7f06800cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 -- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068015f1d0 msgr2=0x7f06800b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068015f1d0 0x7f06800b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 -- 172.21.15.76:0/761507998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06800ce550 con 0x7f068006fe50 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067ffff640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068015f1d0 0x7f06800b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f067f7fe640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f068006fe50 0x7f068014bba0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f067001cfb0 tx=0x7f06700194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0667fff640 1 -- 172.21.15.76:0/761507998 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700bc020 con 0x7f068006fe50 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0667fff640 1 -- 172.21.15.76:0/761507998 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06700b8670 con 0x7f068006fe50 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0667fff640 1 -- 172.21.15.76:0/761507998 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0670009510 con 0x7f068006fe50 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06800ce830 con 0x7f068006fe50 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06800cedf0 con 0x7f068006fe50 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f068015ec90 con 0x7f068006fe50 2024-09-19T04:15:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.538+0000 7f0667fff640 1 -- 172.21.15.76:0/761507998 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06700bf020 con 0x7f068006fe50 2024-09-19T04:15:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.542+0000 7f0667fff640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f066c0678c0 0x7f066c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.542+0000 7f067effd640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f066c0678c0 0x7f066c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.542+0000 7f0667fff640 1 -- 172.21.15.76:0/761507998 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f0670120d70 con 0x7f068006fe50 2024-09-19T04:15:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.542+0000 7f067effd640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f066c0678c0 0x7f066c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06800b3e30 tx=0x7f0674008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.546+0000 7f0667fff640 1 -- 172.21.15.76:0/761507998 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f06700c6030 con 0x7f068006fe50 2024-09-19T04:15:22.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.726+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f068015e4b0 con 0x7f068006fe50 2024-09-19T04:15:22.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.726+0000 7f0667fff640 1 -- 172.21.15.76:0/761507998 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) ==== 94+0+5273 (secure 0 0 0) 0x7f06700165c0 con 0x7f068006fe50 2024-09-19T04:15:22.729 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:22.729 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":22,"btime":"2024-09-19T04:14:51:896699+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:14:17.426311+0000","modified":"2024-09-19T04:14:51.152737+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":4863,"mds_1":4854,"mds_2":4666},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4666":{"gid":4666,"name":"d","rank":2,"incarnation":18,"state":"up:active","state_seq":7,"addr":"172.21.15.116:6833/2447208752","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2447208752},{"type":"v1","addr":"172.21.15.116:6833","nonce":2447208752}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4854":{"gid":4854,"name":"a","rank":1,"incarnation":16,"state":"up:active","state_seq":6,"addr":"172.21.15.76:6835/2829561253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2829561253},{"type":"v1","addr":"172.21.15.76:6835","nonce":2829561253}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_4863":{"gid":4863,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/1828076892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1828076892},{"type":"v1","addr":"172.21.15.76:6837","nonce":1828076892}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4863,"qdb_cluster":[4863,4854,4666]},"id":2}]} 2024-09-19T04:15:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 22 2024-09-19T04:15:22.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f066c0678c0 msgr2=0x7f066c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f066c0678c0 0x7f066c069d80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f06800b3e30 tx=0x7f0674008040 comp rx=0 tx=0).stop 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f068006fe50 msgr2=0x7f068014bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f068006fe50 0x7f068014bba0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f067001cfb0 tx=0x7f06700194b0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 shutdown_connections 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f066c0678c0 0x7f066c069d80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f068015f1d0 0x7f06800b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06801520a0 0x7f06800cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 --2- 172.21.15.76:0/761507998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f068006fe50 0x7f068014bba0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 >> 172.21.15.76:0/761507998 conn(0x7f06800b1f90 msgr2=0x7f068006f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 shutdown_connections 2024-09-19T04:15:22.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.730+0000 7f0685c32640 1 -- 172.21.15.76:0/761507998 wait complete. 2024-09-19T04:15:22.747 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:15:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.858+0000 7f33ab9da640 1 Processor -- start 2024-09-19T04:15:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.858+0000 7f33ab9da640 1 -- start start 2024-09-19T04:15:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33ab9da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33a415b170 0x7f33a409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33ab9da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a40b0f00 0x7f33a415ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33ab9da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a40ae8b0 con 0x7f33a40af950 2024-09-19T04:15:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33ab9da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f33a40aeb90 con 0x7f33a415b170 2024-09-19T04:15:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33ab9da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f33a40aee70 con 0x7f33a40b0f00 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9776640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33a415b170 0x7f33a409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a8f75640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f33a40af950 0x7f33a40b07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52602/0 (socket says 172.21.15.76:52602) 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a8f75640 1 -- 172.21.15.76:0/3102275742 learned_addr learned my addr 172.21.15.76:0/3102275742 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9776640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33a415b170 0x7f33a409aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55658/0 (socket says 172.21.15.76:55658) 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9f77640 1 --2- 172.21.15.76:0/3102275742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a40b0f00 0x7f33a415ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9776640 1 -- 172.21.15.76:0/3102275742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a40b0f00 msgr2=0x7f33a415ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9776640 1 --2- 172.21.15.76:0/3102275742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a40b0f00 0x7f33a415ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9776640 1 -- 172.21.15.76:0/3102275742 >> v1:172.21.15.76:6789/0 conn(0x7f33a40af950 legacy=0x7f33a40b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9776640 1 -- 172.21.15.76:0/3102275742 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a40af150 con 0x7f33a415b170 2024-09-19T04:15:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f33a9776640 1 --2- 172.21.15.76:0/3102275742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a415b170 0x7f33a409aa00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f339c0097c0 tx=0x7f339c01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaca104b44d8228 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f339bfff640 1 -- 172.21.15.76:0/3102275742 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0b0070 con 0x7f33a415b170 2024-09-19T04:15:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f339bfff640 1 -- 172.21.15.76:0/3102275742 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c0ab600 con 0x7f33a415b170 2024-09-19T04:15:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.862+0000 7f339bfff640 1 -- 172.21.15.76:0/3102275742 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0ab9d0 con 0x7f33a415b170 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- 172.21.15.76:0/3102275742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a415b170 msgr2=0x7f33a409aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 --2- 172.21.15.76:0/3102275742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a415b170 0x7f33a409aa00 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f339c0097c0 tx=0x7f339c01c4a0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- 172.21.15.76:0/3102275742 shutdown_connections 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 --2- 172.21.15.76:0/3102275742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a40b0f00 0x7f33a415ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 --2- 172.21.15.76:0/3102275742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a415b170 0x7f33a409aa00 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- 172.21.15.76:0/3102275742 >> 172.21.15.76:0/3102275742 conn(0x7f33a40b6800 msgr2=0x7f33a40b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- 172.21.15.76:0/3102275742 shutdown_connections 2024-09-19T04:15:22.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- 172.21.15.76:0/3102275742 wait complete. 2024-09-19T04:15:22.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 Processor -- start 2024-09-19T04:15:22.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- start start 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a40af950 0x7f33a414e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a40b0f00 0x7f33a414ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a415b170 0x7f33a41f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33a4168370 con 0x7f33a40af950 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33a415b760 con 0x7f33a40b0f00 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f33a41684f0 con 0x7f33a415b170 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a40af950 0x7f33a414e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a40af950 0x7f33a414e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55662/0 (socket says 172.21.15.76:55662) 2024-09-19T04:15:22.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 -- 172.21.15.76:0/1083366649 learned_addr learned my addr 172.21.15.76:0/1083366649 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 -- 172.21.15.76:0/1083366649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a415b170 msgr2=0x7f33a41f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a415b170 0x7f33a41f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 -- 172.21.15.76:0/1083366649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a40b0f00 msgr2=0x7f33a414ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a40b0f00 0x7f33a414ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 -- 172.21.15.76:0/1083366649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a414be60 con 0x7f33a40af950 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33a9776640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a40af950 0x7f33a414e670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f339c0ae040 tx=0x7f339c01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f3399ffb640 1 -- 172.21.15.76:0/1083366649 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0b0050 con 0x7f33a40af950 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f3399ffb640 1 -- 172.21.15.76:0/1083366649 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c0bc070 con 0x7f33a40af950 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a414ebb0 con 0x7f33a40af950 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f3399ffb640 1 -- 172.21.15.76:0/1083366649 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339c0b5a70 con 0x7f33a40af950 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.866+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a4151b00 con 0x7f33a40af950 2024-09-19T04:15:22.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.870+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a415ee20 con 0x7f33a40af950 2024-09-19T04:15:22.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.870+0000 7f3399ffb640 1 -- 172.21.15.76:0/1083366649 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f339c0ab4c0 con 0x7f33a40af950 2024-09-19T04:15:22.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.870+0000 7f3399ffb640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33880678c0 0x7f3388069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:22.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.870+0000 7f3399ffb640 1 -- 172.21.15.76:0/1083366649 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7f339c120800 con 0x7f33a40af950 2024-09-19T04:15:22.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.870+0000 7f33a8f75640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33880678c0 0x7f3388069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:22.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.870+0000 7f33a8f75640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33880678c0 0x7f3388069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3394000f70 tx=0x7f3394007a90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:22.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:22.874+0000 7f3399ffb640 1 -- 172.21.15.76:0/1083366649 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f33a414ebb0 con 0x7f33a40af950 2024-09-19T04:15:23.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.038+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33a414b380 con 0x7f33a40af950 2024-09-19T04:15:23.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f3399ffb640 1 -- 172.21.15.76:0/1083366649 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+17075 (secure 0 0 0) 0x7f33a414be60 con 0x7f33a40af950 2024-09-19T04:15:23.042 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:23.042 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":41,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:15:06.946979+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:14:15.256172+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-09-19T04:14:16.419970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.3d","mappings":[{"from":5,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:23.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33880678c0 msgr2=0x7f3388069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33880678c0 0x7f3388069d80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3394000f70 tx=0x7f3394007a90 comp rx=0 tx=0).stop 2024-09-19T04:15:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a40af950 msgr2=0x7f33a414e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a40af950 0x7f33a414e670 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f339c0ae040 tx=0x7f339c01ace0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 shutdown_connections 2024-09-19T04:15:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33880678c0 0x7f3388069d80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a415b170 0x7f33a41f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a40b0f00 0x7f33a414ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 --2- 172.21.15.76:0/1083366649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a40af950 0x7f33a414e670 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 >> 172.21.15.76:0/1083366649 conn(0x7f33a40b6800 msgr2=0x7f33a406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:23.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 shutdown_connections 2024-09-19T04:15:23.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.042+0000 7f33ab9da640 1 -- 172.21.15.76:0/1083366649 wait complete. 2024-09-19T04:15:23.062 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T04:15:23.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa577a72640 1 Processor -- start 2024-09-19T04:15:23.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa577a72640 1 -- start start 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa577a72640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa57015bec0 0x7fa5700ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa577a72640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5700af470 0x7fa57006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa577a72640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5700b3290 con 0x7fa57015c2a0 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa577a72640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa5700b3570 con 0x7fa57015bec0 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa577a72640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa5700b3850 con 0x7fa5700af470 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa57015bec0 0x7fa5700ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa57015bec0 0x7fa5700ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55666/0 (socket says 172.21.15.76:55666) 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 -- 172.21.15.76:0/2444247340 learned_addr learned my addr 172.21.15.76:0/2444247340 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 -- 172.21.15.76:0/2444247340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5700af470 msgr2=0x7fa57006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57600f640 1 --2- 172.21.15.76:0/2444247340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5700af470 0x7fa57006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 --2- 172.21.15.76:0/2444247340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5700af470 0x7fa57006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 -- 172.21.15.76:0/2444247340 >> v1:172.21.15.116:6789/0 conn(0x7fa57015c2a0 legacy=0x7fa5700aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:23.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 -- 172.21.15.76:0/2444247340 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5700b3b30 con 0x7fa57015bec0 2024-09-19T04:15:23.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57600f640 1 --2- 172.21.15.76:0/2444247340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5700af470 0x7fa57006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:23.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57580e640 1 --2- 172.21.15.76:0/2444247340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa5700ad200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009e50 tx=0x7fa56c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e3c4223737d4bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:23.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57480c640 1 -- 172.21.15.76:0/2444247340 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c0a57b0 con 0x7fa57015bec0 2024-09-19T04:15:23.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57480c640 1 -- 172.21.15.76:0/2444247340 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c0a5950 con 0x7fa57015bec0 2024-09-19T04:15:23.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.178+0000 7fa57480c640 1 -- 172.21.15.76:0/2444247340 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c0a5d20 con 0x7fa57015bec0 2024-09-19T04:15:23.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- 172.21.15.76:0/2444247340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 msgr2=0x7fa5700ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 --2- 172.21.15.76:0/2444247340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa5700ad200 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009e50 tx=0x7fa56c01c930 comp rx=0 tx=0).stop 2024-09-19T04:15:23.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- 172.21.15.76:0/2444247340 shutdown_connections 2024-09-19T04:15:23.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 --2- 172.21.15.76:0/2444247340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5700af470 0x7fa57006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 --2- 172.21.15.76:0/2444247340 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa5700ad200 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- 172.21.15.76:0/2444247340 >> 172.21.15.76:0/2444247340 conn(0x7fa5700b1f90 msgr2=0x7fa5700b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:23.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- 172.21.15.76:0/2444247340 shutdown_connections 2024-09-19T04:15:23.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- 172.21.15.76:0/2444247340 wait complete. 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 Processor -- start 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- start start 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5700af470 0x7fa570148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa57014a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa57015c2a0 0x7fa5700b08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa57006ed50 con 0x7fa57015bec0 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa5700ad740 con 0x7fa5700af470 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa57006eed0 con 0x7fa57015c2a0 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa57014a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa57014a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55680/0 (socket says 172.21.15.76:55680) 2024-09-19T04:15:23.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 -- 172.21.15.76:0/2811055463 learned_addr learned my addr 172.21.15.76:0/2811055463 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57580e640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5700af470 0x7fa570148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57600f640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa57015c2a0 0x7fa5700b08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 -- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa57015c2a0 msgr2=0x7fa5700b08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa57015c2a0 0x7fa5700b08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 -- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5700af470 msgr2=0x7fa570148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5700af470 0x7fa570148ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 -- 172.21.15.76:0/2811055463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa570149380 con 0x7fa57015bec0 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa57500d640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa57014a420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa56000b3e0 tx=0x7fa56000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa5667fc640 1 -- 172.21.15.76:0/2811055463 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560015020 con 0x7fa57015bec0 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa5667fc640 1 -- 172.21.15.76:0/2811055463 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa560011830 con 0x7fa57015bec0 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa570149660 con 0x7fa57015bec0 2024-09-19T04:15:23.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa5667fc640 1 -- 172.21.15.76:0/2811055463 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560011c60 con 0x7fa57015bec0 2024-09-19T04:15:23.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.182+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa570149c20 con 0x7fa57015bec0 2024-09-19T04:15:23.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.186+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57006f280 con 0x7fa57015bec0 2024-09-19T04:15:23.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.186+0000 7fa5667fc640 1 -- 172.21.15.76:0/2811055463 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa560018020 con 0x7fa57015bec0 2024-09-19T04:15:23.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.186+0000 7fa5667fc640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5540678c0 0x7fa554069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.186+0000 7fa5667fc640 1 -- 172.21.15.76:0/2811055463 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6130+0+0 (secure 0 0 0) 0x7fa560085ac0 con 0x7fa57015bec0 2024-09-19T04:15:23.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.186+0000 7fa57580e640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5540678c0 0x7fa554069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.186+0000 7fa57580e640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5540678c0 0x7fa554069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0ae000 tx=0x7fa56c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:23.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.190+0000 7fa5667fc640 1 -- 172.21.15.76:0/2811055463 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa56001f030 con 0x7fa57015bec0 2024-09-19T04:15:23.279 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:15:23.281 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:15:23.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.374+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa57006fcd0 con 0x7fa57015bec0 2024-09-19T04:15:23.376 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:15:23.374+0000 7fa674384640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:15:23.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.378+0000 7fa5667fc640 1 -- 172.21.15.76:0/2811055463 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v23) ==== 161+0+0 (secure 0 0 0) 0x7fa5600172c0 con 0x7fa57015bec0 2024-09-19T04:15:23.379 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5540678c0 msgr2=0x7fa554069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5540678c0 0x7fa554069d80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa56c0ae000 tx=0x7fa56c0a7070 comp rx=0 tx=0).stop 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 msgr2=0x7fa57014a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa57014a420 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fa56000b3e0 tx=0x7fa56000b8a0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 shutdown_connections 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5540678c0 0x7fa554069d80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa57015c2a0 0x7fa5700b08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa57015bec0 0x7fa57014a420 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 --2- 172.21.15.76:0/2811055463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5700af470 0x7fa570148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 >> 172.21.15.76:0/2811055463 conn(0x7fa5700b1f90 msgr2=0x7fa570074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:23.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 shutdown_connections 2024-09-19T04:15:23.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.386+0000 7fa577a72640 1 -- 172.21.15.76:0/2811055463 wait complete. 2024-09-19T04:15:23.404 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:15:23.419 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:15:23.442 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:15:23.458 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:15:23.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.518+0000 7f405105e640 1 Processor -- start 2024-09-19T04:15:23.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.518+0000 7f405105e640 1 -- start start 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f405105e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 0x7f404c06e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f405105e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404c06ee50 con 0x7f404c073e60 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f405105e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404c06f130 con 0x7f404c0b3d50 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f405105e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f404c06f410 con 0x7f404c06fef0 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f404c073e60 0x7f404c15b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52610/0 (socket says 172.21.15.76:52610) 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404ad9d640 1 -- 172.21.15.76:0/521169024 learned_addr learned my addr 172.21.15.76:0/521169024 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404a59c640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 0x7f404c06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4154852659 0 0) 0x7f404c06ee50 con 0x7f404c073e60 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4034003530 con 0x7f404c073e60 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3807600097 0 0) 0x7f4034003530 con 0x7f404c073e60 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 msgr2=0x7f404c06e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 0x7f404c06e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 >> v1:172.21.15.116:6789/0 conn(0x7f404c0b3d50 legacy=0x7f404c072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c06f6f0 con 0x7f404c073e60 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f403c001680 con 0x7f404c073e60 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 >> v1:172.21.15.76:6789/0 conn(0x7f404c073e60 legacy=0x7f404c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40340039f0 0x7f4034003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 0x7f4034004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4034004d50 0x7f40340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f404c06f410 con 0x7f4034004d50 2024-09-19T04:15:23.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f403400bb60 con 0x7f40340039f0 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f403400df70 con 0x7f404c06fef0 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404b59e640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4034004d50 0x7f40340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404ad9d640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40340039f0 0x7f4034003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404a59c640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 0x7f4034004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404b59e640 1 -- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 msgr2=0x7f4034004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404b59e640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c06fef0 0x7f4034004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404b59e640 1 -- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40340039f0 msgr2=0x7f4034003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404b59e640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f40340039f0 0x7f4034003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404b59e640 1 -- 172.21.15.76:0/521169024 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4034003530 con 0x7f4034004d50 2024-09-19T04:15:23.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f404b59e640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4034004d50 0x7f40340051d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4044001e70 tx=0x7f404401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313973382091abfa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044001a90 con 0x7f4034004d50 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4044003b80 con 0x7f4034004d50 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.522+0000 7f4049d9b640 1 -- 172.21.15.76:0/521169024 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044005080 con 0x7f4034004d50 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- 172.21.15.76:0/521169024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4034004d50 msgr2=0x7f40340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 --2- 172.21.15.76:0/521169024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4034004d50 0x7f40340051d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4044001e70 tx=0x7f404401c780 comp rx=0 tx=0).stop 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 -- 172.21.15.76:0/521169024 reap_dead start 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- 172.21.15.76:0/521169024 shutdown_connections 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- 172.21.15.76:0/521169024 >> 172.21.15.76:0/521169024 conn(0x7f404c0b1f90 msgr2=0x7f404c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- 172.21.15.76:0/521169024 shutdown_connections 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- 172.21.15.76:0/521169024 wait complete. 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 Processor -- start 2024-09-19T04:15:23.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- start start 2024-09-19T04:15:23.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c1f3ca0 0x7f404c150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c1f4080 0x7f404c0b0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404c1f4500 0x7f404c0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f404c0b4300 con 0x7f404c1f4500 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f404c0b4180 con 0x7f404c1f4080 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f404c0b4480 con 0x7f404c1f3ca0 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404c1f4500 0x7f404c0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404c1f4500 0x7f404c0b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55716/0 (socket says 172.21.15.76:55716) 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 -- 172.21.15.76:0/162654801 learned_addr learned my addr 172.21.15.76:0/162654801 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404ad9d640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c1f3ca0 0x7f404c150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404a59c640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c1f4080 0x7f404c0b0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 -- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c1f3ca0 msgr2=0x7f404c150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c1f3ca0 0x7f404c150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 -- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c1f4080 msgr2=0x7f404c0b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:23.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c1f4080 0x7f404c0b0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 -- 172.21.15.76:0/162654801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404c1f49a0 con 0x7f404c1f4500 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f404b59e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404c1f4500 0x7f404c0b1440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f404401cc60 tx=0x7f4034005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f40337fe640 1 -- 172.21.15.76:0/162654801 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40440b8070 con 0x7f404c1f4500 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f40337fe640 1 -- 172.21.15.76:0/162654801 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4044005580 con 0x7f404c1f4500 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404c1f4c80 con 0x7f404c1f4500 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f40337fe640 1 -- 172.21.15.76:0/162654801 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4044001a90 con 0x7f404c1f4500 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.526+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f404c156810 con 0x7f404c1f4500 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.530+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404c1f51a0 con 0x7f404c1f4500 2024-09-19T04:15:23.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.530+0000 7f40337fe640 1 -- 172.21.15.76:0/162654801 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f40440bb070 con 0x7f404c1f4500 2024-09-19T04:15:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.530+0000 7f40337fe640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40380678c0 0x7f4038069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.530+0000 7f40337fe640 1 -- 172.21.15.76:0/162654801 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f404411eca0 con 0x7f404c1f4500 2024-09-19T04:15:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.530+0000 7f404ad9d640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40380678c0 0x7f4038069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:23.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.530+0000 7f404ad9d640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40380678c0 0x7f4038069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f403c001b90 tx=0x7f403c001b20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:23.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.534+0000 7f40337fe640 1 -- 172.21.15.76:0/162654801 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f404c1f51a0 con 0x7f404c1f4500 2024-09-19T04:15:23.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:23.722+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f404c156a10 con 0x7f404c1f4500 2024-09-19T04:15:24.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.386+0000 7f40337fe640 1 -- 172.21.15.76:0/162654801 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v24) ==== 104+0+0 (secure 0 0 0) 0x7f404c156a10 con 0x7f404c1f4500 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40380678c0 msgr2=0x7f4038069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40380678c0 0x7f4038069d80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f403c001b90 tx=0x7f403c001b20 comp rx=0 tx=0).stop 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404c1f4500 msgr2=0x7f404c0b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404c1f4500 0x7f404c0b1440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f404401cc60 tx=0x7f4034005840 comp rx=0 tx=0).stop 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 shutdown_connections 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f40380678c0 0x7f4038069d80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f404c1f4500 0x7f404c0b1440 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c1f4080 0x7f404c0b0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 --2- 172.21.15.76:0/162654801 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f404c1f3ca0 0x7f404c150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 >> 172.21.15.76:0/162654801 conn(0x7f404c0b1f90 msgr2=0x7f404c15d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:24.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 shutdown_connections 2024-09-19T04:15:24.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.390+0000 7f405105e640 1 -- 172.21.15.76:0/162654801 wait complete. 2024-09-19T04:15:24.411 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:15:24.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6472578640 1 Processor -- start 2024-09-19T04:15:24.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6472578640 1 -- start start 2024-09-19T04:15:24.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6472578640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f646c15f830 0x7f646c06bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6472578640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646c15d290 0x7f646c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6472578640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f646c15fc10 con 0x7f646c09aa00 2024-09-19T04:15:24.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6472578640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f646c15da40 con 0x7f646c15f830 2024-09-19T04:15:24.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6472578640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f646c15d800 con 0x7f646c15d290 2024-09-19T04:15:24.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6470b15640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f646c09aa00 0x7f646c15fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52614/0 (socket says 172.21.15.76:52614) 2024-09-19T04:15:24.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6470b15640 1 -- 172.21.15.76:0/1416532147 learned_addr learned my addr 172.21.15.76:0/1416532147 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:24.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646bfff640 1 --2- 172.21.15.76:0/1416532147 >> v2:172.21.15.116:3300/0 conn(0x7f646c15f830 0x7f646c06bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 195872969 0 0) 0x7f646c15fc10 con 0x7f646c09aa00 2024-09-19T04:15:24.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646c15d290 0x7f646c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6454003530 con 0x7f646c09aa00 2024-09-19T04:15:24.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3252184419 0 0) 0x7f6454003530 con 0x7f646c09aa00 2024-09-19T04:15:24.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646c15d290 msgr2=0x7f646c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646c15d290 0x7f646c06c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 >> v2:172.21.15.116:3300/0 conn(0x7f646c15f830 msgr2=0x7f646c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 --2- 172.21.15.76:0/1416532147 >> v2:172.21.15.116:3300/0 conn(0x7f646c15f830 0x7f646c06bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c0b3570 con 0x7f646c09aa00 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f645c001680 con 0x7f646c09aa00 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646bfff640 1 --2- 172.21.15.76:0/1416532147 >> v2:172.21.15.116:3300/0 conn(0x7f646c15f830 0x7f646c06bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 >> v1:172.21.15.76:6789/0 conn(0x7f646c09aa00 legacy=0x7f646c15fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:24.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64540039c0 0x7f6454003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454004810 0x7f6454004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64540051d0 0x7f645400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f646c15da40 con 0x7f6454004810 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f646c15d800 con 0x7f64540039c0 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f645400c010 con 0x7f64540051d0 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f646c15d290 0x7f646c06c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454004810 0x7f6454004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646bfff640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64540039c0 0x7f6454003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f6470b15640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64540051d0 0x7f645400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 -- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64540051d0 msgr2=0x7f645400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:24.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64540051d0 0x7f645400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 -- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64540039c0 msgr2=0x7f6454003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:24.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64540039c0 0x7f6454003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646b7fe640 1 -- 172.21.15.76:0/1416532147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6454003530 con 0x7f6454004810 2024-09-19T04:15:24.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.582+0000 7f646bfff640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64540039c0 0x7f6454003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:24.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f646b7fe640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454004810 0x7f6454004c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f64600096e0 tx=0x7f646001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaf9e0e04a505d51 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:24.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64600b1070 con 0x7f6454004810 2024-09-19T04:15:24.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64600ac640 con 0x7f6454004810 2024-09-19T04:15:24.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f646affd640 1 -- 172.21.15.76:0/1416532147 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646000f040 con 0x7f6454004810 2024-09-19T04:15:24.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 -- 172.21.15.76:0/1416532147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454004810 msgr2=0x7f6454004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:24.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 --2- 172.21.15.76:0/1416532147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6454004810 0x7f6454004c90 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f64600096e0 tx=0x7f646001c3d0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6470b15640 1 -- 172.21.15.76:0/1416532147 reap_dead start 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 -- 172.21.15.76:0/1416532147 shutdown_connections 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 -- 172.21.15.76:0/1416532147 >> 172.21.15.76:0/1416532147 conn(0x7f646c0b6800 msgr2=0x7f646c0b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 -- 172.21.15.76:0/1416532147 shutdown_connections 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 -- 172.21.15.76:0/1416532147 wait complete. 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 Processor -- start 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.586+0000 7f6472578640 1 -- start start 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6454004810 0x7f646c09ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f646c0b4e40 0x7f646c149870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646c149db0 0x7f646c14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f646c073460 con 0x7f646c149db0 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f646c0b3b30 con 0x7f646c0b4e40 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f646c0735e0 con 0x7f6454004810 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646c149db0 0x7f646c14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646c149db0 0x7f646c14d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55734/0 (socket says 172.21.15.76:55734) 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 -- 172.21.15.76:0/95176301 learned_addr learned my addr 172.21.15.76:0/95176301 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f646b7fe640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f646c0b4e40 0x7f646c149870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f646bfff640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6454004810 0x7f646c09ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 -- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6454004810 msgr2=0x7f646c09ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6454004810 0x7f646c09ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 -- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f646c0b4e40 msgr2=0x7f646c149870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f646c0b4e40 0x7f646c149870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 -- 172.21.15.76:0/95176301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646c14a340 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6470b15640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646c149db0 0x7f646c14d100 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f646c075b90 tx=0x7f645c011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6468ff9640 1 -- 172.21.15.76:0/95176301 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c010040 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6468ff9640 1 -- 172.21.15.76:0/95176301 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f645c01a070 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646c0b4360 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6468ff9640 1 -- 172.21.15.76:0/95176301 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645c00fac0 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646c0b60d0 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f646c06bcc0 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6468ff9640 1 -- 172.21.15.76:0/95176301 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f645c00fc60 con 0x7f646c149db0 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6468ff9640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64580678c0 0x7f6458069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f6468ff9640 1 -- 172.21.15.76:0/95176301 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6388+0+0 (secure 0 0 0) 0x7f645c088d90 con 0x7f646c149db0 2024-09-19T04:15:24.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f646bfff640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64580678c0 0x7f6458069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:24.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.590+0000 7f646bfff640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64580678c0 0x7f6458069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f646c1f7880 tx=0x7f646400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:24.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.594+0000 7f6468ff9640 1 -- 172.21.15.76:0/95176301 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f645c059600 con 0x7f646c149db0 2024-09-19T04:15:24.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:24.758+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f646c06e7c0 con 0x7f646c149db0 2024-09-19T04:15:25.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.386+0000 7f6468ff9640 1 -- 172.21.15.76:0/95176301 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) ==== 181+0+0 (secure 0 0 0) 0x7f645c058f00 con 0x7f646c149db0 2024-09-19T04:15:25.390 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:15:25.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64580678c0 msgr2=0x7f6458069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:25.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64580678c0 0x7f6458069d80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f646c1f7880 tx=0x7f646400b040 comp rx=0 tx=0).stop 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646c149db0 msgr2=0x7f646c14d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646c149db0 0x7f646c14d100 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f646c075b90 tx=0x7f645c011c20 comp rx=0 tx=0).stop 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 shutdown_connections 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64580678c0 0x7f6458069d80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f646c149db0 0x7f646c14d100 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f646c0b4e40 0x7f646c149870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 --2- 172.21.15.76:0/95176301 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6454004810 0x7f646c09ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 >> 172.21.15.76:0/95176301 conn(0x7f646c0b6800 msgr2=0x7f646c15bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 shutdown_connections 2024-09-19T04:15:25.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.394+0000 7f6472578640 1 -- 172.21.15.76:0/95176301 wait complete. 2024-09-19T04:15:25.419 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:15:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 Processor -- start 2024-09-19T04:15:25.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 -- start start 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fddb0161a60 0x7fddb00d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fddb00d2e90 0x7fddb00cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00cf450 0x7fddb00d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fddb00d32b0 con 0x7fddb0161a60 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fddb00d0300 con 0x7fddb00d2e90 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb7d66640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fddb00b0f60 con 0x7fddb00cf450 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb5b02640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fddb0161a60 0x7fddb00d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb5b02640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fddb0161a60 0x7fddb00d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55748/0 (socket says 172.21.15.76:55748) 2024-09-19T04:15:25.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb5b02640 1 -- 172.21.15.76:0/120438450 learned_addr learned my addr 172.21.15.76:0/120438450 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb6303640 1 --2- 172.21.15.76:0/120438450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00cf450 0x7fddb00d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.542+0000 7fddb5301640 1 --2- 172.21.15.76:0/120438450 >> v2:172.21.15.116:3300/0 conn(0x7fddb00d2e90 0x7fddb00cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb5b02640 1 -- 172.21.15.76:0/120438450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00cf450 msgr2=0x7fddb00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb5b02640 1 --2- 172.21.15.76:0/120438450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00cf450 0x7fddb00d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb5b02640 1 -- 172.21.15.76:0/120438450 >> v2:172.21.15.116:3300/0 conn(0x7fddb00d2e90 msgr2=0x7fddb00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb5b02640 1 --2- 172.21.15.76:0/120438450 >> v2:172.21.15.116:3300/0 conn(0x7fddb00d2e90 0x7fddb00cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb5b02640 1 -- 172.21.15.76:0/120438450 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb00cfd40 con 0x7fddb0161a60 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb5301640 1 --2- 172.21.15.76:0/120438450 >> v2:172.21.15.116:3300/0 conn(0x7fddb00d2e90 0x7fddb00cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:25.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb6303640 1 --2- 172.21.15.76:0/120438450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00cf450 0x7fddb00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb5b02640 1 --2- 172.21.15.76:0/120438450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00d2950 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fddb008d850 tx=0x7fdda001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93fe673ff661fbd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb4b00640 1 -- 172.21.15.76:0/120438450 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00ab620 con 0x7fddb0161a60 2024-09-19T04:15:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb4b00640 1 -- 172.21.15.76:0/120438450 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda00b0070 con 0x7fddb0161a60 2024-09-19T04:15:25.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb4b00640 1 -- 172.21.15.76:0/120438450 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00ab9e0 con 0x7fddb0161a60 2024-09-19T04:15:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 -- 172.21.15.76:0/120438450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 msgr2=0x7fddb00d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 --2- 172.21.15.76:0/120438450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00d2950 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fddb008d850 tx=0x7fdda001c460 comp rx=0 tx=0).stop 2024-09-19T04:15:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 -- 172.21.15.76:0/120438450 shutdown_connections 2024-09-19T04:15:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 --2- 172.21.15.76:0/120438450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00cf450 0x7fddb00d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 --2- 172.21.15.76:0/120438450 >> v2:172.21.15.116:3300/0 conn(0x7fddb00d2e90 0x7fddb00cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 --2- 172.21.15.76:0/120438450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00d2950 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 -- 172.21.15.76:0/120438450 >> 172.21.15.76:0/120438450 conn(0x7fddb008bf90 msgr2=0x7fddb009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:25.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 -- 172.21.15.76:0/120438450 shutdown_connections 2024-09-19T04:15:25.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.546+0000 7fddb7d66640 1 -- 172.21.15.76:0/120438450 wait complete. 2024-09-19T04:15:25.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 Processor -- start 2024-09-19T04:15:25.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 -- start start 2024-09-19T04:15:25.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddb00cf450 0x7fddb01599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:25.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00d2e90 0x7fddb0151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:25.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:25.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fddb006d830 con 0x7fddb0161a60 2024-09-19T04:15:25.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fddb006d6b0 con 0x7fddb00cf450 2024-09-19T04:15:25.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fddb006d9b0 con 0x7fddb00d2e90 2024-09-19T04:15:25.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55764/0 (socket says 172.21.15.76:55764) 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 -- 172.21.15.76:0/2659542300 learned_addr learned my addr 172.21.15.76:0/2659542300 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb5b02640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddb00cf450 0x7fddb01599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb5301640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00d2e90 0x7fddb0151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 -- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00d2e90 msgr2=0x7fddb0151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00d2e90 0x7fddb0151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 -- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddb00cf450 msgr2=0x7fddb01599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddb00cf450 0x7fddb01599f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 -- 172.21.15.76:0/2659542300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb01fa960 con 0x7fddb0161a60 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb5b02640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddb00cf450 0x7fddb01599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb6303640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00b4d20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdda800cef0 tx=0x7fdda800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fdda67fc640 1 -- 172.21.15.76:0/2659542300 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda801a070 con 0x7fddb0161a60 2024-09-19T04:15:25.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fdda67fc640 1 -- 172.21.15.76:0/2659542300 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda800d840 con 0x7fddb0161a60 2024-09-19T04:15:25.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb01fac40 con 0x7fddb0161a60 2024-09-19T04:15:25.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fdda67fc640 1 -- 172.21.15.76:0/2659542300 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda800dc30 con 0x7fddb0161a60 2024-09-19T04:15:25.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb01fb200 con 0x7fddb0161a60 2024-09-19T04:15:25.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb00cef10 con 0x7fddb0161a60 2024-09-19T04:15:25.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.550+0000 7fdda67fc640 1 -- 172.21.15.76:0/2659542300 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdda801d070 con 0x7fddb0161a60 2024-09-19T04:15:25.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.554+0000 7fdda67fc640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd940678c0 0x7fdd94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:25.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.554+0000 7fddb5b02640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd940678c0 0x7fdd94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:25.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.554+0000 7fdda67fc640 1 -- 172.21.15.76:0/2659542300 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5950+0+0 (secure 0 0 0) 0x7fdda80865b0 con 0x7fddb0161a60 2024-09-19T04:15:25.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.554+0000 7fddb5b02640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd940678c0 0x7fdd94069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdda00af000 tx=0x7fdda00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:25.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.558+0000 7fdda67fc640 1 -- 172.21.15.76:0/2659542300 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdda8057000 con 0x7fddb0161a60 2024-09-19T04:15:25.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:25.717+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fddb00cb200 con 0x7fddb0161a60 2024-09-19T04:15:26.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.389+0000 7fdda67fc640 1 -- 172.21.15.76:0/2659542300 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v44) ==== 169+0+0 (secure 0 0 0) 0x7fdda8056900 con 0x7fddb0161a60 2024-09-19T04:15:26.392 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-09-19T04:15:26.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.393+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd940678c0 msgr2=0x7fdd94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.393+0000 7fddb7d66640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd940678c0 0x7fdd94069d80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdda00af000 tx=0x7fdda00a7000 comp rx=0 tx=0).stop 2024-09-19T04:15:26.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.393+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 msgr2=0x7fddb00b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.393+0000 7fddb7d66640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00b4d20 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fdda800cef0 tx=0x7fdda800a710 comp rx=0 tx=0).stop 2024-09-19T04:15:26.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 shutdown_connections 2024-09-19T04:15:26.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd940678c0 0x7fdd94069d80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddb0161a60 0x7fddb00b4d20 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddb00d2e90 0x7fddb0151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 --2- 172.21.15.76:0/2659542300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddb00cf450 0x7fddb01599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 >> 172.21.15.76:0/2659542300 conn(0x7fddb008bf90 msgr2=0x7fddb00d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 shutdown_connections 2024-09-19T04:15:26.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.397+0000 7fddb7d66640 1 -- 172.21.15.76:0/2659542300 wait complete. 2024-09-19T04:15:26.421 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T04:15:26.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.541+0000 7f65865cb640 1 Processor -- start 2024-09-19T04:15:26.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.541+0000 7f65865cb640 1 -- start start 2024-09-19T04:15:26.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f65865cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 0x7f65800d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f65865cb640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f65800cf620 0x7f658006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f65865cb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65800d32b0 con 0x7f65800d2e90 2024-09-19T04:15:26.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f65865cb640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f65800d0300 con 0x7f65800cf620 2024-09-19T04:15:26.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f65865cb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f65800b0f60 con 0x7f6580161a60 2024-09-19T04:15:26.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f65800d2e90 0x7f65800cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52616/0 (socket says 172.21.15.76:52616) 2024-09-19T04:15:26.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 -- 172.21.15.76:0/715012691 learned_addr learned my addr 172.21.15.76:0/715012691 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:26.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f6584b68640 1 --2- 172.21.15.76:0/715012691 >> v2:172.21.15.116:3300/0 conn(0x7f65800cf620 0x7f658006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657ffff640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 0x7f65800d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3194456481 0 0) 0x7f65800d32b0 con 0x7f65800d2e90 2024-09-19T04:15:26.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6568003530 con 0x7f65800d2e90 2024-09-19T04:15:26.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 985285974 0 0) 0x7f6568003530 con 0x7f65800d2e90 2024-09-19T04:15:26.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 msgr2=0x7f65800d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:26.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 0x7f65800d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 >> v2:172.21.15.116:3300/0 conn(0x7f65800cf620 msgr2=0x7f658006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 --2- 172.21.15.76:0/715012691 >> v2:172.21.15.116:3300/0 conn(0x7f65800cf620 0x7f658006d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65800cfad0 con 0x7f65800d2e90 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6574001680 con 0x7f65800d2e90 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 >> v1:172.21.15.76:6789/0 conn(0x7f65800d2e90 legacy=0x7f65800cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65680039c0 0x7f6568003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6568004810 0x7f6568004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 0x7f65680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f65800d0300 con 0x7f6568004810 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f65800b0f60 con 0x7f65680039c0 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f656800ba20 con 0x7f6580161a60 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6568004810 0x7f6568004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657ffff640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65680039c0 0x7f6568003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f6584b68640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 0x7f65680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 -- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 msgr2=0x7f65680051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6580161a60 0x7f65680051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 -- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65680039c0 msgr2=0x7f6568003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:26.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65680039c0 0x7f6568003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 -- 172.21.15.76:0/715012691 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6568003530 con 0x7f6568004810 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.545+0000 7f657f7fe640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6568004810 0x7f6568004c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f657400c160 tx=0x7f657401cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63c40ad6286f74e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574007c90 con 0x7f6568004810 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65740a9020 con 0x7f6568004810 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657effd640 1 -- 172.21.15.76:0/715012691 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6574005b00 con 0x7f6568004810 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- 172.21.15.76:0/715012691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6568004810 msgr2=0x7f6568004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 --2- 172.21.15.76:0/715012691 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6568004810 0x7f6568004c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f657400c160 tx=0x7f657401cb50 comp rx=0 tx=0).stop 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 -- 172.21.15.76:0/715012691 reap_dead start 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- 172.21.15.76:0/715012691 shutdown_connections 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- 172.21.15.76:0/715012691 >> 172.21.15.76:0/715012691 conn(0x7f658008bf90 msgr2=0x7f658009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- 172.21.15.76:0/715012691 shutdown_connections 2024-09-19T04:15:26.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- 172.21.15.76:0/715012691 wait complete. 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 Processor -- start 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- start start 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6568004810 0x7f65801606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65800b9980 0x7f65801ecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65801ed120 0x7f658015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f658006e400 con 0x7f65801ed120 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f658006d190 con 0x7f65800b9980 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f658006e580 con 0x7f6568004810 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65801ed120 0x7f658015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65801ed120 0x7f658015e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55776/0 (socket says 172.21.15.76:55776) 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 -- 172.21.15.76:0/2529989630 learned_addr learned my addr 172.21.15.76:0/2529989630 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657f7fe640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65800b9980 0x7f65801ecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657ffff640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6568004810 0x7f65801606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 -- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6568004810 msgr2=0x7f65801606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6568004810 0x7f65801606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 -- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65800b9980 msgr2=0x7f65801ecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65800b9980 0x7f65801ecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 -- 172.21.15.76:0/2529989630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6580160bf0 con 0x7f65801ed120 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f6584b68640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65801ed120 0x7f658015e9f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f658008e010 tx=0x7f65780115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657cff9640 1 -- 172.21.15.76:0/2529989630 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6578011e70 con 0x7f65801ed120 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657cff9640 1 -- 172.21.15.76:0/2529989630 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657801b070 con 0x7f65801ed120 2024-09-19T04:15:26.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658015aaf0 con 0x7f65801ed120 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f657cff9640 1 -- 172.21.15.76:0/2529989630 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65780039a0 con 0x7f65801ed120 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.549+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65800ca190 con 0x7f65801ed120 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.553+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658006d6b0 con 0x7f65801ed120 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.553+0000 7f657cff9640 1 -- 172.21.15.76:0/2529989630 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6578003b40 con 0x7f65801ed120 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.553+0000 7f657cff9640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f656c0678c0 0x7f656c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.553+0000 7f657cff9640 1 -- 172.21.15.76:0/2529989630 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7f6578086330 con 0x7f65801ed120 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.553+0000 7f657ffff640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f656c0678c0 0x7f656c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.553+0000 7f657ffff640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f656c0678c0 0x7f656c069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65700098d0 tx=0x7f6570008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:26.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.557+0000 7f657cff9640 1 -- 172.21.15.76:0/2529989630 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6580160bf0 con 0x7f65801ed120 2024-09-19T04:15:26.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.721+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f65800d1990 con 0x7f65801ed120 2024-09-19T04:15:26.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.725+0000 7f657cff9640 1 -- 172.21.15.76:0/2529989630 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f6578056f30 con 0x7f65801ed120 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f656c0678c0 msgr2=0x7f656c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f656c0678c0 0x7f656c069d80 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f65700098d0 tx=0x7f6570008040 comp rx=0 tx=0).stop 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65801ed120 msgr2=0x7f658015e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65801ed120 0x7f658015e9f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f658008e010 tx=0x7f65780115a0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 shutdown_connections 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f656c0678c0 0x7f656c069d80 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f65801ed120 0x7f658015e9f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f65800b9980 0x7f65801ecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 --2- 172.21.15.76:0/2529989630 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6568004810 0x7f65801606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 >> 172.21.15.76:0/2529989630 conn(0x7f658008bf90 msgr2=0x7f65800d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 shutdown_connections 2024-09-19T04:15:26.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.729+0000 7f65865cb640 1 -- 172.21.15.76:0/2529989630 wait complete. 2024-09-19T04:15:26.748 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag' 2024-09-19T04:15:26.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc356332640 1 Processor -- start 2024-09-19T04:15:26.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc356332640 1 -- start start 2024-09-19T04:15:26.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc356332640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc35015f3a0 0x7fc350165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc356332640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3500aede0 con 0x7fc350070020 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc356332640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc35015fa10 con 0x7fc3501520a0 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc356332640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc350152480 con 0x7fc35015f3a0 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc3548cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc35015f3a0 0x7fc350165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc3548cf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc35015f3a0 0x7fc350165f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55790/0 (socket says 172.21.15.76:55790) 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc3548cf640 1 -- 172.21.15.76:0/272366605 learned_addr learned my addr 172.21.15.76:0/272366605 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc34f7fe640 1 --1- 172.21.15.76:0/272366605 >> v1:172.21.15.76:6789/0 conn(0x7fc350070020 0x7fc35015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52618/0 (socket says 172.21.15.76:52618) 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc3548cf640 1 -- 172.21.15.76:0/272366605 >> v1:172.21.15.116:6789/0 conn(0x7fc3501520a0 legacy=0x7fc35006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc3548cf640 1 -- 172.21.15.76:0/272366605 >> v1:172.21.15.76:6789/0 conn(0x7fc350070020 legacy=0x7fc35015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc3548cf640 1 -- 172.21.15.76:0/272366605 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc350074250 con 0x7fc35015f3a0 2024-09-19T04:15:26.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc3548cf640 1 --2- 172.21.15.76:0/272366605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc35015f3a0 0x7fc350165f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc348009e50 tx=0x7fc34801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d540bbae44c3f72d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:26.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.869+0000 7fc34effd640 1 -- 172.21.15.76:0/272366605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3480a57b0 con 0x7fc35015f3a0 2024-09-19T04:15:26.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34effd640 1 -- 172.21.15.76:0/272366605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3480a5950 con 0x7fc35015f3a0 2024-09-19T04:15:26.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34effd640 1 -- 172.21.15.76:0/272366605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3480a5d20 con 0x7fc35015f3a0 2024-09-19T04:15:26.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- 172.21.15.76:0/272366605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc35015f3a0 msgr2=0x7fc350165f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 --2- 172.21.15.76:0/272366605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc35015f3a0 0x7fc350165f00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc348009e50 tx=0x7fc34801c9f0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- 172.21.15.76:0/272366605 shutdown_connections 2024-09-19T04:15:26.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 --2- 172.21.15.76:0/272366605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc35015f3a0 0x7fc350165f00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- 172.21.15.76:0/272366605 >> 172.21.15.76:0/272366605 conn(0x7fc3500b1f90 msgr2=0x7fc3500b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:26.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- 172.21.15.76:0/272366605 shutdown_connections 2024-09-19T04:15:26.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- 172.21.15.76:0/272366605 wait complete. 2024-09-19T04:15:26.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 Processor -- start 2024-09-19T04:15:26.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- start start 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc350070020 0x7fc350146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc3501520a0 0x7fc3501473d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc35015f3a0 0x7fc35014c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc35006f010 con 0x7fc350070020 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc350073d30 con 0x7fc35015f3a0 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc35006f190 con 0x7fc3501520a0 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc350070020 0x7fc350146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc350070020 0x7fc350146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55806/0 (socket says 172.21.15.76:55806) 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 -- 172.21.15.76:0/1225326308 learned_addr learned my addr 172.21.15.76:0/1225326308 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:26.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc3548cf640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc35015f3a0 0x7fc35014c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 -- 172.21.15.76:0/1225326308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc3501520a0 msgr2=0x7fc3501473d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc3501520a0 0x7fc3501473d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 -- 172.21.15.76:0/1225326308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc35015f3a0 msgr2=0x7fc35014c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc35015f3a0 0x7fc35014c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 -- 172.21.15.76:0/1225326308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3500cf970 con 0x7fc350070020 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34ffff640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc350070020 0x7fc350146e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c450 tx=0x7fc34000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34cff9640 1 -- 172.21.15.76:0/1225326308 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340012830 con 0x7fc350070020 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34cff9640 1 -- 172.21.15.76:0/1225326308 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc340017070 con 0x7fc350070020 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3500ccf00 con 0x7fc350070020 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc34cff9640 1 -- 172.21.15.76:0/1225326308 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340012c10 con 0x7fc350070020 2024-09-19T04:15:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.873+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc35014d9f0 con 0x7fc350070020 2024-09-19T04:15:26.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.877+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc35006f550 con 0x7fc350070020 2024-09-19T04:15:26.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.877+0000 7fc34cff9640 1 -- 172.21.15.76:0/1225326308 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc34001c460 con 0x7fc350070020 2024-09-19T04:15:26.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.877+0000 7fc34cff9640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc33c0678c0 0x7fc33c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:26.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.877+0000 7fc34cff9640 1 -- 172.21.15.76:0/1225326308 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7fc340085950 con 0x7fc350070020 2024-09-19T04:15:26.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.877+0000 7fc34f7fe640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc33c0678c0 0x7fc33c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:26.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.877+0000 7fc34f7fe640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc33c0678c0 0x7fc33c069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc344000f70 tx=0x7fc344009c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:26.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:26.881+0000 7fc34cff9640 1 -- 172.21.15.76:0/1225326308 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc35014d9f0 con 0x7fc350070020 2024-09-19T04:15:27.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.101+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]} v 0) -- 0x7fc35006e540 con 0x7fc350070020 2024-09-19T04:15:27.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.405+0000 7fc34cff9640 1 -- 172.21.15.76:0/1225326308 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_apply_tag"]}]=0 v119) ==== 136+0+0 (secure 0 0 0) 0x7fc3400565e0 con 0x7fc350070020 2024-09-19T04:15:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc33c0678c0 msgr2=0x7fc33c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc33c0678c0 0x7fc33c069d80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc344000f70 tx=0x7fc344009c20 comp rx=0 tx=0).stop 2024-09-19T04:15:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc350070020 msgr2=0x7fc350146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc350070020 0x7fc350146e90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c450 tx=0x7fc34000c910 comp rx=0 tx=0).stop 2024-09-19T04:15:27.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 shutdown_connections 2024-09-19T04:15:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc33c0678c0 0x7fc33c069d80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc35015f3a0 0x7fc35014c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc3501520a0 0x7fc3501473d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 --2- 172.21.15.76:0/1225326308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc350070020 0x7fc350146e90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 >> 172.21.15.76:0/1225326308 conn(0x7fc3500b1f90 msgr2=0x7fc3500b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 shutdown_connections 2024-09-19T04:15:27.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.409+0000 7fc356332640 1 -- 172.21.15.76:0/1225326308 wait complete. 2024-09-19T04:15:27.431 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-19T04:15:27.432 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-19T04:15:27.432 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:15:27.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 Processor -- start 2024-09-19T04:15:27.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 -- start start 2024-09-19T04:15:27.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c153030 0x7fa43c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa43c06ccd0 0x7fa43c06d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa43c0af4c0 0x7fa43c0af900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa43c06d650 con 0x7fa43c06ccd0 2024-09-19T04:15:27.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa43c15d5e0 con 0x7fa43c0af4c0 2024-09-19T04:15:27.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa440a21640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c0b3e20 con 0x7fa43c153030 2024-09-19T04:15:27.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa43c06ccd0 0x7fa43c06d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa43c06ccd0 0x7fa43c06d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55808/0 (socket says 172.21.15.76:55808) 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 -- 172.21.15.76:0/2161686176 learned_addr learned my addr 172.21.15.76:0/2161686176 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa43ad9d640 1 --2- 172.21.15.76:0/2161686176 >> v2:172.21.15.116:3300/0 conn(0x7fa43c0af4c0 0x7fa43c0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa43a59c640 1 --2- 172.21.15.76:0/2161686176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c153030 0x7fa43c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 -- 172.21.15.76:0/2161686176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c153030 msgr2=0x7fa43c15d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 --2- 172.21.15.76:0/2161686176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c153030 0x7fa43c15d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 -- 172.21.15.76:0/2161686176 >> v2:172.21.15.116:3300/0 conn(0x7fa43c0af4c0 msgr2=0x7fa43c0af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 --2- 172.21.15.76:0/2161686176 >> v2:172.21.15.116:3300/0 conn(0x7fa43c0af4c0 0x7fa43c0af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa439d9b640 1 -- 172.21.15.76:0/2161686176 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c0b4100 con 0x7fa43c06ccd0 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa43ad9d640 1 --2- 172.21.15.76:0/2161686176 >> v2:172.21.15.116:3300/0 conn(0x7fa43c0af4c0 0x7fa43c0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:27.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.557+0000 7fa43a59c640 1 --2- 172.21.15.76:0/2161686176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c153030 0x7fa43c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:27.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa439d9b640 1 --2- 172.21.15.76:0/2161686176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c06ccd0 0x7fa43c06d110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa43c068000 tx=0x7fa42401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1bdef2592756b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:27.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa43959a640 1 -- 172.21.15.76:0/2161686176 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b1070 con 0x7fa43c06ccd0 2024-09-19T04:15:27.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa43959a640 1 -- 172.21.15.76:0/2161686176 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4240ac630 con 0x7fa43c06ccd0 2024-09-19T04:15:27.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa43959a640 1 -- 172.21.15.76:0/2161686176 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240a8050 con 0x7fa43c06ccd0 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- 172.21.15.76:0/2161686176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c06ccd0 msgr2=0x7fa43c06d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 --2- 172.21.15.76:0/2161686176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c06ccd0 0x7fa43c06d110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa43c068000 tx=0x7fa42401c520 comp rx=0 tx=0).stop 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- 172.21.15.76:0/2161686176 shutdown_connections 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 --2- 172.21.15.76:0/2161686176 >> v2:172.21.15.116:3300/0 conn(0x7fa43c0af4c0 0x7fa43c0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 --2- 172.21.15.76:0/2161686176 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c06ccd0 0x7fa43c06d110 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 --2- 172.21.15.76:0/2161686176 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c153030 0x7fa43c15d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- 172.21.15.76:0/2161686176 >> 172.21.15.76:0/2161686176 conn(0x7fa43c072690 msgr2=0x7fa43c072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:27.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- 172.21.15.76:0/2161686176 shutdown_connections 2024-09-19T04:15:27.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- 172.21.15.76:0/2161686176 wait complete. 2024-09-19T04:15:27.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 Processor -- start 2024-09-19T04:15:27.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- start start 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c06ccd0 0x7fa43c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c0af4c0 0x7fa43c1576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa43c153030 0x7fa43c13e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0b5a30 con 0x7fa43c0af4c0 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0b3b30 con 0x7fa43c153030 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.561+0000 7fa440a21640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c0b5bb0 con 0x7fa43c06ccd0 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c0af4c0 0x7fa43c1576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c0af4c0 0x7fa43c1576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55822/0 (socket says 172.21.15.76:55822) 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 -- 172.21.15.76:0/2602303614 learned_addr learned my addr 172.21.15.76:0/2602303614 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa43a59c640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c06ccd0 0x7fa43c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa43ad9d640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa43c153030 0x7fa43c13e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 -- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c06ccd0 msgr2=0x7fa43c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c06ccd0 0x7fa43c15c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 -- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa43c153030 msgr2=0x7fa43c13e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa43c153030 0x7fa43c13e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 -- 172.21.15.76:0/2602303614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c158730 con 0x7fa43c0af4c0 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa43ad9d640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa43c153030 0x7fa43c13e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa439d9b640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c0af4c0 0x7fa43c1576f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa42401abd0 tx=0x7fa4240192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa432ffd640 1 -- 172.21.15.76:0/2602303614 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240bc070 con 0x7fa43c0af4c0 2024-09-19T04:15:27.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa432ffd640 1 -- 172.21.15.76:0/2602303614 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4240ab5e0 con 0x7fa43c0af4c0 2024-09-19T04:15:27.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa432ffd640 1 -- 172.21.15.76:0/2602303614 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4240b1070 con 0x7fa43c0af4c0 2024-09-19T04:15:27.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c13ea10 con 0x7fa43c0af4c0 2024-09-19T04:15:27.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c158170 con 0x7fa43c0af4c0 2024-09-19T04:15:27.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43c153410 con 0x7fa43c0af4c0 2024-09-19T04:15:27.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa432ffd640 1 -- 172.21.15.76:0/2602303614 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4240bf070 con 0x7fa43c0af4c0 2024-09-19T04:15:27.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa432ffd640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa41c0678c0 0x7fa41c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa432ffd640 1 -- 172.21.15.76:0/2602303614 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7fa42411f4d0 con 0x7fa43c0af4c0 2024-09-19T04:15:27.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa43a59c640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa41c0678c0 0x7fa41c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.565+0000 7fa43a59c640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa41c0678c0 0x7fa41c069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa43400ba10 tx=0x7fa434008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:27.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.569+0000 7fa432ffd640 1 -- 172.21.15.76:0/2602303614 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa424016110 con 0x7fa43c0af4c0 2024-09-19T04:15:27.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.733+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa43c0ce770 con 0x7fa43c0af4c0 2024-09-19T04:15:27.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.733+0000 7fa432ffd640 1 -- 172.21.15.76:0/2602303614 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+13591 (secure 0 0 0) 0x7fa4240f00d0 con 0x7fa43c0af4c0 2024-09-19T04:15:27.737 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:27.737 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":44,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:15:26.389666+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6834/2829561253":"2024-09-20T04:15:23.376691+0000","172.21.15.76:6835/2829561253":"2024-09-20T04:15:23.376691+0000","172.21.15.76:6837/1828076892":"2024-09-20T04:15:23.376704+0000","172.21.15.76:6836/1828076892":"2024-09-20T04:15:23.376704+0000","172.21.15.116:6833/2447208752":"2024-09-20T04:15:23.376664+0000","172.21.15.116:6832/2447208752":"2024-09-20T04:15:23.376664+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa41c0678c0 msgr2=0x7fa41c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa41c0678c0 0x7fa41c069d80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa43400ba10 tx=0x7fa434008040 comp rx=0 tx=0).stop 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c0af4c0 msgr2=0x7fa43c1576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c0af4c0 0x7fa43c1576f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa42401abd0 tx=0x7fa4240192c0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 shutdown_connections 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa41c0678c0 0x7fa41c069d80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa43c153030 0x7fa43c13e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa43c0af4c0 0x7fa43c1576f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 --2- 172.21.15.76:0/2602303614 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa43c06ccd0 0x7fa43c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 >> 172.21.15.76:0/2602303614 conn(0x7fa43c072690 msgr2=0x7fa43c15df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:27.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 shutdown_connections 2024-09-19T04:15:27.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.737+0000 7fa440a21640 1 -- 172.21.15.76:0/2602303614 wait complete. 2024-09-19T04:15:27.757 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:13.812655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:15:27.757 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:15:27.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f6505dc6640 1 Processor -- start 2024-09-19T04:15:27.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f6505dc6640 1 -- start start 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f6505dc6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6500073170 0x7f650006e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f6505dc6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650006ec60 con 0x7f65000b39a0 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f6505dc6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650006ef40 con 0x7f65001587a0 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f6505dc6640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f650006f870 con 0x7f6500073170 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64feffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f65000b39a0 0x7f650006df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52620/0 (socket says 172.21.15.76:52620) 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64feffd640 1 -- 172.21.15.76:0/872324279 learned_addr learned my addr 172.21.15.76:0/872324279 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fffff640 1 --2- 172.21.15.76:0/872324279 >> v2:172.21.15.116:3300/0 conn(0x7f6500073170 0x7f650006e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3565442692 0 0) 0x7f650006ec60 con 0x7f65000b39a0 2024-09-19T04:15:27.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64e8003530 con 0x7f65000b39a0 2024-09-19T04:15:27.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 629604252 0 0) 0x7f64e8003530 con 0x7f65000b39a0 2024-09-19T04:15:27.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 >> v2:172.21.15.116:3300/0 conn(0x7f6500073170 msgr2=0x7f650006e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:27.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 --2- 172.21.15.76:0/872324279 >> v2:172.21.15.116:3300/0 conn(0x7f6500073170 0x7f650006e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 >> v1:172.21.15.116:6789/0 conn(0x7f65001587a0 legacy=0x7f65000b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650006f220 con 0x7f65000b39a0 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f64f4001680 con 0x7f65000b39a0 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 >> v1:172.21.15.76:6789/0 conn(0x7f65000b39a0 legacy=0x7f650006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f64e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e8004810 0x7f64e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e80051d0 0x7f64e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f650006f870 con 0x7f64e80039c0 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64e800c0a0 con 0x7f64e8004810 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f64e800c2a0 con 0x7f64e80051d0 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64ff7fe640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f64e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64feffd640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e8004810 0x7f64e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64ff7fe640 1 -- 172.21.15.76:0/872324279 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e80051d0 msgr2=0x7f64e800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:27.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64ff7fe640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64e80051d0 0x7f64e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64ff7fe640 1 -- 172.21.15.76:0/872324279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e8004810 msgr2=0x7f64e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:27.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64ff7fe640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64e8004810 0x7f64e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64ff7fe640 1 -- 172.21.15.76:0/872324279 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64e8003530 con 0x7f64e80039c0 2024-09-19T04:15:27.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.877+0000 7f64fffff640 1 -- 172.21.15.76:0/872324279 reap_dead start 2024-09-19T04:15:27.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f64ff7fe640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f64e8003e00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001780 tx=0x7f64f001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f808d66639c0b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:27.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f00013a0 con 0x7f64e80039c0 2024-09-19T04:15:27.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f00052f0 con 0x7f64e80039c0 2024-09-19T04:15:27.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f64fe7fc640 1 -- 172.21.15.76:0/872324279 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0003840 con 0x7f64e80039c0 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 -- 172.21.15.76:0/872324279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 msgr2=0x7f64e8003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f64e8003e00 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f64f0001780 tx=0x7f64f001c3f0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 -- 172.21.15.76:0/872324279 shutdown_connections 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 --2- 172.21.15.76:0/872324279 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f64e8003e00 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 -- 172.21.15.76:0/872324279 >> 172.21.15.76:0/872324279 conn(0x7f65000b1f90 msgr2=0x7f65000b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 -- 172.21.15.76:0/872324279 shutdown_connections 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 -- 172.21.15.76:0/872324279 wait complete. 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 Processor -- start 2024-09-19T04:15:27.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.881+0000 7f6505dc6640 1 -- start start 2024-09-19T04:15:27.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f650015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650015d3c0 0x7f65001fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f650009d0a0 0x7f650009d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f650006e900 con 0x7f64e80039c0 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f650006e780 con 0x7f650015d3c0 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6500072160 con 0x7f650009d0a0 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f650015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f650015ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55838/0 (socket says 172.21.15.76:55838) 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 -- 172.21.15.76:0/1874943714 learned_addr learned my addr 172.21.15.76:0/1874943714 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64fffff640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f650009d0a0 0x7f650009d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 -- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f650009d0a0 msgr2=0x7f650009d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64feffd640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650015d3c0 0x7f65001fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f650009d0a0 0x7f650009d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 -- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650015d3c0 msgr2=0x7f65001fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650015d3c0 0x7f65001fa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 -- 172.21.15.76:0/1874943714 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65001fa9c0 con 0x7f64e80039c0 2024-09-19T04:15:27.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64ff7fe640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f650015ad80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f64f00ad040 tx=0x7f64e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:27.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64e7fff640 1 -- 172.21.15.76:0/1874943714 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f0007a40 con 0x7f64e80039c0 2024-09-19T04:15:27.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64e7fff640 1 -- 172.21.15.76:0/1874943714 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64f00bd070 con 0x7f64e80039c0 2024-09-19T04:15:27.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650015b4f0 con 0x7f64e80039c0 2024-09-19T04:15:27.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64e7fff640 1 -- 172.21.15.76:0/1874943714 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f00013a0 con 0x7f64e80039c0 2024-09-19T04:15:27.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650015bab0 con 0x7f64e80039c0 2024-09-19T04:15:27.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65000724f0 con 0x7f64e80039c0 2024-09-19T04:15:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64e7fff640 1 -- 172.21.15.76:0/1874943714 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f64f00b4040 con 0x7f64e80039c0 2024-09-19T04:15:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64e7fff640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64ec0678f0 0x7f64ec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64e7fff640 1 -- 172.21.15.76:0/1874943714 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7f64f011fc40 con 0x7f64e80039c0 2024-09-19T04:15:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64feffd640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64ec0678f0 0x7f64ec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:27.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.885+0000 7f64feffd640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64ec0678f0 0x7f64ec069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f64f4001ec0 tx=0x7f64f40011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:27.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:27.889+0000 7f64e7fff640 1 -- 172.21.15.76:0/1874943714 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f64f0016130 con 0x7f64e80039c0 2024-09-19T04:15:28.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6500162f90 con 0x7f64e80039c0 2024-09-19T04:15:28.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f64e7fff640 1 -- 172.21.15.76:0/1874943714 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) ==== 93+0+10 (secure 0 0 0) 0x7f64f00f0870 con 0x7f64e80039c0 2024-09-19T04:15:28.064 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-09-19T04:15:28.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64ec0678f0 msgr2=0x7f64ec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:28.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f6505dc6640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64ec0678f0 0x7f64ec069db0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f64f4001ec0 tx=0x7f64f40011a0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 msgr2=0x7f650015ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:28.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f6505dc6640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f650015ad80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f64f00ad040 tx=0x7f64e8005840 comp rx=0 tx=0).stop 2024-09-19T04:15:28.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 shutdown_connections 2024-09-19T04:15:28.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.061+0000 7f6505dc6640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64ec0678f0 0x7f64ec069db0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.065+0000 7f6505dc6640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f650009d0a0 0x7f650009d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.065+0000 7f6505dc6640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f650015d3c0 0x7f65001fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.065+0000 7f6505dc6640 1 --2- 172.21.15.76:0/1874943714 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64e80039c0 0x7f650015ad80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.065+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 >> 172.21.15.76:0/1874943714 conn(0x7f65000b1f90 msgr2=0x7f6500161de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:28.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.065+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 shutdown_connections 2024-09-19T04:15:28.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.065+0000 7f6505dc6640 1 -- 172.21.15.76:0/1874943714 wait complete. 2024-09-19T04:15:28.081 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-09-19T04:15:28.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec3886640 1 Processor -- start 2024-09-19T04:15:28.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec3886640 1 -- start start 2024-09-19T04:15:28.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec3886640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:28.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec3886640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ebc06e680 0x7f5ebc072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:28.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec3886640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc06ec60 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec3886640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc06f870 con 0x7f5ebc06e680 2024-09-19T04:15:28.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec3886640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0b3c10 con 0x7f5ebc1587a0 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec1e23640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ebc06e680 0x7f5ebc072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec0e21640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5ebc0b37d0 0x7f5ebc06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52622/0 (socket says 172.21.15.76:52622) 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec0e21640 1 -- 172.21.15.76:0/2912746927 learned_addr learned my addr 172.21.15.76:0/2912746927 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec1622640 1 --2- 172.21.15.76:0/2912746927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5eb3fff640 1 -- 172.21.15.76:0/2912746927 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 248023737 0 0) 0x7f5ebc06ec60 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5eb3fff640 1 -- 172.21.15.76:0/2912746927 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ea8003530 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec1e23640 1 -- 172.21.15.76:0/2912746927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 msgr2=0x7f5ebc0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec1e23640 1 --2- 172.21.15.76:0/2912746927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc0b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec1e23640 1 -- 172.21.15.76:0/2912746927 >> v1:172.21.15.76:6789/0 conn(0x7f5ebc0b37d0 legacy=0x7f5ebc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec1e23640 1 -- 172.21.15.76:0/2912746927 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc06ef40 con 0x7f5ebc06e680 2024-09-19T04:15:28.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.201+0000 7f5ec1622640 1 --2- 172.21.15.76:0/2912746927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:28.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec1e23640 1 --2- 172.21.15.76:0/2912746927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc06e680 0x7f5ebc072160 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009d90 tx=0x7f5eb401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ea5f80e4a11405 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:28.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5eb3fff640 1 -- 172.21.15.76:0/2912746927 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40b2070 con 0x7f5ebc06e680 2024-09-19T04:15:28.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5eb3fff640 1 -- 172.21.15.76:0/2912746927 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb40a5a00 con 0x7f5ebc06e680 2024-09-19T04:15:28.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5eb3fff640 1 -- 172.21.15.76:0/2912746927 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40a5db0 con 0x7f5ebc06e680 2024-09-19T04:15:28.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- 172.21.15.76:0/2912746927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc06e680 msgr2=0x7f5ebc072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:28.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 --2- 172.21.15.76:0/2912746927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc06e680 0x7f5ebc072160 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009d90 tx=0x7f5eb401c940 comp rx=0 tx=0).stop 2024-09-19T04:15:28.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- 172.21.15.76:0/2912746927 shutdown_connections 2024-09-19T04:15:28.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 --2- 172.21.15.76:0/2912746927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc06e680 0x7f5ebc072160 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 --2- 172.21.15.76:0/2912746927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- 172.21.15.76:0/2912746927 >> 172.21.15.76:0/2912746927 conn(0x7f5ebc0b1f90 msgr2=0x7f5ebc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:28.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- 172.21.15.76:0/2912746927 shutdown_connections 2024-09-19T04:15:28.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- 172.21.15.76:0/2912746927 wait complete. 2024-09-19T04:15:28.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 Processor -- start 2024-09-19T04:15:28.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- start start 2024-09-19T04:15:28.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ebc06e680 0x7f5ebc09dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:28.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc0b37d0 0x7f5ebc09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc09afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc073030 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc072eb0 con 0x7f5ebc06e680 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0731b0 con 0x7f5ebc1587a0 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc0b37d0 0x7f5ebc09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc0b37d0 0x7f5ebc09aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55856/0 (socket says 172.21.15.76:55856) 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 -- 172.21.15.76:0/1486863363 learned_addr learned my addr 172.21.15.76:0/1486863363 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 -- 172.21.15.76:0/1486863363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 msgr2=0x7f5ebc09afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc09afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 -- 172.21.15.76:0/1486863363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ebc06e680 msgr2=0x7f5ebc09dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ebc06e680 0x7f5ebc09dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 -- 172.21.15.76:0/1486863363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc1f66e0 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec0e21640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc0b37d0 0x7f5ebc09aa70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00c4d0 tx=0x7f5eac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5eb1ffb640 1 -- 172.21.15.76:0/1486863363 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac017070 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5eb1ffb640 1 -- 172.21.15.76:0/1486863363 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac005cf0 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc09b8d0 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5eb1ffb640 1 -- 172.21.15.76:0/1486863363 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac007270 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.205+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc09d0d0 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.209+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc072160 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.209+0000 7f5eb1ffb640 1 -- 172.21.15.76:0/1486863363 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5eac01a070 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.209+0000 7f5eb1ffb640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ea00678c0 0x7f5ea0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:28.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.209+0000 7f5ec1622640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ea00678c0 0x7f5ea0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:28.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.209+0000 7f5eb1ffb640 1 -- 172.21.15.76:0/1486863363 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7f5eac084de0 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.209+0000 7f5ec1622640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ea00678c0 0x7f5ea0069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8006f10 tx=0x7f5eb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:28.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.213+0000 7f5eb1ffb640 1 -- 172.21.15.76:0/1486863363 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5eac055420 con 0x7f5ebc0b37d0 2024-09-19T04:15:28.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:28.421+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7f5ebc0b4250 con 0x7f5ebc0b37d0 2024-09-19T04:15:29.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.417+0000 7f5eb1ffb640 1 -- 172.21.15.76:0/1486863363 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v121) ==== 146+0+0 (secure 0 0 0) 0x7f5eac059720 con 0x7f5ebc0b37d0 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ea00678c0 msgr2=0x7f5ea0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ea00678c0 0x7f5ea0069d80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8006f10 tx=0x7f5eb8008040 comp rx=0 tx=0).stop 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc0b37d0 msgr2=0x7f5ebc09aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc0b37d0 0x7f5ebc09aa70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5eac00c4d0 tx=0x7f5eac00c990 comp rx=0 tx=0).stop 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 shutdown_connections 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ea00678c0 0x7f5ea0069d80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ebc1587a0 0x7f5ebc09afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ebc0b37d0 0x7f5ebc09aa70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 --2- 172.21.15.76:0/1486863363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ebc06e680 0x7f5ebc09dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 >> 172.21.15.76:0/1486863363 conn(0x7f5ebc0b1f90 msgr2=0x7f5ebc0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:29.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 shutdown_connections 2024-09-19T04:15:29.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.421+0000 7f5ec3886640 1 -- 172.21.15.76:0/1486863363 wait complete. 2024-09-19T04:15:29.447 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T04:15:29.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f9130b640 1 Processor -- start 2024-09-19T04:15:29.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f9130b640 1 -- start start 2024-09-19T04:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f9130b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f8c1587a0 0x7f5f8c0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f9130b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5f8c0b37d0 0x7f5f8c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f9130b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f8c06ec60 con 0x7f5f8c06e4b0 2024-09-19T04:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f9130b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5f8c0b3c40 con 0x7f5f8c1587a0 2024-09-19T04:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f9130b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5f8c06f870 con 0x7f5f8c0b37d0 2024-09-19T04:15:29.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f8c1587a0 0x7f5f8c0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f8c1587a0 0x7f5f8c0b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55862/0 (socket says 172.21.15.76:55862) 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8affd640 1 -- 172.21.15.76:0/2115316791 learned_addr learned my addr 172.21.15.76:0/2115316791 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5f8c06e4b0 0x7f5f8c161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52636/0 (socket says 172.21.15.76:52636) 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8a7fc640 1 --2- 172.21.15.76:0/2115316791 >> v2:172.21.15.116:3300/0 conn(0x7f5f8c0b37d0 0x7f5f8c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8affd640 1 -- 172.21.15.76:0/2115316791 >> v2:172.21.15.116:3300/0 conn(0x7f5f8c0b37d0 msgr2=0x7f5f8c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8affd640 1 --2- 172.21.15.76:0/2115316791 >> v2:172.21.15.116:3300/0 conn(0x7f5f8c0b37d0 0x7f5f8c06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8affd640 1 -- 172.21.15.76:0/2115316791 >> v1:172.21.15.76:6789/0 conn(0x7f5f8c06e4b0 legacy=0x7f5f8c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f89ffb640 1 -- 172.21.15.76:0/2115316791 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2966224073 0 0) 0x7f5f8c06ec60 con 0x7f5f8c06e4b0 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8affd640 1 -- 172.21.15.76:0/2115316791 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c06ef40 con 0x7f5f8c1587a0 2024-09-19T04:15:29.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.573+0000 7f5f8a7fc640 1 --2- 172.21.15.76:0/2115316791 >> v2:172.21.15.116:3300/0 conn(0x7f5f8c0b37d0 0x7f5f8c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:29.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f8affd640 1 --2- 172.21.15.76:0/2115316791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c0b3290 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c161d90 tx=0x7f5f7c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6757b72bc217e752 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:29.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f89ffb640 1 -- 172.21.15.76:0/2115316791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0a57a0 con 0x7f5f8c1587a0 2024-09-19T04:15:29.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f89ffb640 1 -- 172.21.15.76:0/2115316791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7c0a5940 con 0x7f5f8c1587a0 2024-09-19T04:15:29.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f89ffb640 1 -- 172.21.15.76:0/2115316791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0a5d10 con 0x7f5f8c1587a0 2024-09-19T04:15:29.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 -- 172.21.15.76:0/2115316791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 msgr2=0x7f5f8c0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 --2- 172.21.15.76:0/2115316791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c0b3290 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c161d90 tx=0x7f5f7c01c9c0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 -- 172.21.15.76:0/2115316791 shutdown_connections 2024-09-19T04:15:29.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 --2- 172.21.15.76:0/2115316791 >> v2:172.21.15.116:3300/0 conn(0x7f5f8c0b37d0 0x7f5f8c06df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 --2- 172.21.15.76:0/2115316791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c0b3290 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 -- 172.21.15.76:0/2115316791 >> 172.21.15.76:0/2115316791 conn(0x7f5f8c0b1f90 msgr2=0x7f5f8c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:29.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 -- 172.21.15.76:0/2115316791 shutdown_connections 2024-09-19T04:15:29.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 -- 172.21.15.76:0/2115316791 wait complete. 2024-09-19T04:15:29.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 Processor -- start 2024-09-19T04:15:29.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 -- start start 2024-09-19T04:15:29.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.577+0000 7f5f9130b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c06e4b0 0x7f5f8c150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:29.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0b37d0 0x7f5f8c1459a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:29.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:29.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c158b80 con 0x7f5f8c1587a0 2024-09-19T04:15:29.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c073170 con 0x7f5f8c06e4b0 2024-09-19T04:15:29.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8c14c660 con 0x7f5f8c0b37d0 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c146040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55876/0 (socket says 172.21.15.76:55876) 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 -- 172.21.15.76:0/1877946167 learned_addr learned my addr 172.21.15.76:0/1877946167 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 -- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0b37d0 msgr2=0x7f5f8c1459a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8a7fc640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0b37d0 0x7f5f8c1459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8affd640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c06e4b0 0x7f5f8c150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0b37d0 0x7f5f8c1459a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 -- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c06e4b0 msgr2=0x7f5f8c150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c06e4b0 0x7f5f8c150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 -- 172.21.15.76:0/1877946167 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c14eb20 con 0x7f5f8c1587a0 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8b7fe640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c146040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c470 tx=0x7f5f8400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:29.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f737fe640 1 -- 172.21.15.76:0/1877946167 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84018070 con 0x7f5f8c1587a0 2024-09-19T04:15:29.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f737fe640 1 -- 172.21.15.76:0/1877946167 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f840074e0 con 0x7f5f8c1587a0 2024-09-19T04:15:29.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c150790 con 0x7f5f8c1587a0 2024-09-19T04:15:29.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f737fe640 1 -- 172.21.15.76:0/1877946167 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84004a30 con 0x7f5f8c1587a0 2024-09-19T04:15:29.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c14f0a0 con 0x7f5f8c1587a0 2024-09-19T04:15:29.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c150960 con 0x7f5f8c1587a0 2024-09-19T04:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f737fe640 1 -- 172.21.15.76:0/1877946167 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5f8401b070 con 0x7f5f8c1587a0 2024-09-19T04:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f737fe640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f780678f0 0x7f5f78069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f737fe640 1 -- 172.21.15.76:0/1877946167 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7f5f84084e50 con 0x7f5f8c1587a0 2024-09-19T04:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8affd640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f780678f0 0x7f5f78069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:29.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.581+0000 7f5f8affd640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f780678f0 0x7f5f78069db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0ae000 tx=0x7f5f7c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:29.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.585+0000 7f5f737fe640 1 -- 172.21.15.76:0/1877946167 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5f84055a50 con 0x7f5f8c1587a0 2024-09-19T04:15:29.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.745+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5f8c1583b0 con 0x7f5f8c1587a0 2024-09-19T04:15:29.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.753+0000 7f5f737fe640 1 -- 172.21.15.76:0/1877946167 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f5f84055350 con 0x7f5f8c1587a0 2024-09-19T04:15:29.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f780678f0 msgr2=0x7f5f78069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f780678f0 0x7f5f78069db0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0ae000 tx=0x7f5f7c0a7070 comp rx=0 tx=0).stop 2024-09-19T04:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 msgr2=0x7f5f8c146040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c146040 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400c470 tx=0x7f5f8400c930 comp rx=0 tx=0).stop 2024-09-19T04:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 shutdown_connections 2024-09-19T04:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f780678f0 0x7f5f78069db0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c1587a0 0x7f5f8c146040 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0b37d0 0x7f5f8c1459a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 --2- 172.21.15.76:0/1877946167 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c06e4b0 0x7f5f8c150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:29.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 >> 172.21.15.76:0/1877946167 conn(0x7f5f8c0b1f90 msgr2=0x7f5f8c151dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:29.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 shutdown_connections 2024-09-19T04:15:29.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:29.757+0000 7f5f9130b640 1 -- 172.21.15.76:0/1877946167 wait complete. 2024-09-19T04:15:29.775 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:15:29.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:15:29.835 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:15:29.920 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:15:29.920 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:15:29.920 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:29.926 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:15:29.927 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:15:29.927 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:29.933 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:15:29.933 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:15:29.933 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:29.975 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:15:29.975 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:15:29.975 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:15:29.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:15:29.981 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T04:15:29.981 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:30.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb3859640 1 Processor -- start 2024-09-19T04:15:30.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb3859640 1 -- start start 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb3859640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac1520a0 0x7f9fac06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb3859640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9fac15f3a0 0x7f9fac0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb3859640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fac070250 con 0x7f9fac06fe50 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb3859640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9fac152480 con 0x7f9fac15f3a0 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb3859640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9fac0aede0 con 0x7f9fac1520a0 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb1df6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9fac15f3a0 0x7f9fac0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb1df6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9fac15f3a0 0x7f9fac0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55878/0 (socket says 172.21.15.76:55878) 2024-09-19T04:15:30.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb1df6640 1 -- 172.21.15.76:0/1817677373 learned_addr learned my addr 172.21.15.76:0/1817677373 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:30.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb15f5640 1 --2- 172.21.15.76:0/1817677373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac1520a0 0x7f9fac06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb1df6640 1 -- 172.21.15.76:0/1817677373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac1520a0 msgr2=0x7f9fac06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:30.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb1df6640 1 --2- 172.21.15.76:0/1817677373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac1520a0 0x7f9fac06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb1df6640 1 -- 172.21.15.76:0/1817677373 >> v1:172.21.15.116:6789/0 conn(0x7f9fac06fe50 legacy=0x7f9fac15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:30.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.141+0000 7f9fb1df6640 1 -- 172.21.15.76:0/1817677373 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac15fa10 con 0x7f9fac15f3a0 2024-09-19T04:15:30.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb1df6640 1 --2- 172.21.15.76:0/1817677373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac15f3a0 0x7f9fac0adc80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009e50 tx=0x7f9fa401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae58ab31453e4f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:30.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fa3fff640 1 -- 172.21.15.76:0/1817677373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40a5870 con 0x7f9fac15f3a0 2024-09-19T04:15:30.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fa3fff640 1 -- 172.21.15.76:0/1817677373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa40a5a10 con 0x7f9fac15f3a0 2024-09-19T04:15:30.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fa3fff640 1 -- 172.21.15.76:0/1817677373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40a5de0 con 0x7f9fac15f3a0 2024-09-19T04:15:30.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- 172.21.15.76:0/1817677373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac15f3a0 msgr2=0x7f9fac0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 --2- 172.21.15.76:0/1817677373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac15f3a0 0x7f9fac0adc80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009e50 tx=0x7f9fa401c9f0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- 172.21.15.76:0/1817677373 shutdown_connections 2024-09-19T04:15:30.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 --2- 172.21.15.76:0/1817677373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac15f3a0 0x7f9fac0adc80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 --2- 172.21.15.76:0/1817677373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac1520a0 0x7f9fac06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- 172.21.15.76:0/1817677373 >> 172.21.15.76:0/1817677373 conn(0x7f9fac0b1f90 msgr2=0x7f9fac0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- 172.21.15.76:0/1817677373 shutdown_connections 2024-09-19T04:15:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- 172.21.15.76:0/1817677373 wait complete. 2024-09-19T04:15:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 Processor -- start 2024-09-19T04:15:30.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- start start 2024-09-19T04:15:30.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac06fe50 0x7f9fac147910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac1520a0 0x7f9fac06e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fac15f3a0 0x7f9fac06eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac073d30 con 0x7f9fac1520a0 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac073a50 con 0x7f9fac15f3a0 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb3859640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9fac167f10 con 0x7f9fac06fe50 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac1520a0 0x7f9fac06e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac1520a0 0x7f9fac06e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55884/0 (socket says 172.21.15.76:55884) 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 -- 172.21.15.76:0/4174910472 learned_addr learned my addr 172.21.15.76:0/4174910472 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb1df6640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fac15f3a0 0x7f9fac06eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 -- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac06fe50 msgr2=0x7f9fac147910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb15f5640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac06fe50 0x7f9fac147910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac06fe50 0x7f9fac147910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 -- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fac15f3a0 msgr2=0x7f9fac06eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fac15f3a0 0x7f9fac06eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb0df4640 1 -- 172.21.15.76:0/4174910472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac0b4300 con 0x7f9fac1520a0 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.145+0000 7f9fb1df6640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fac15f3a0 0x7f9fac06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fb0df4640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac1520a0 0x7f9fac06e9b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00b3e0 tx=0x7f9f9c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fa1ffb640 1 -- 172.21.15.76:0/4174910472 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c015020 con 0x7f9fac1520a0 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fa1ffb640 1 -- 172.21.15.76:0/4174910472 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9c011830 con 0x7f9fac1520a0 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac15e170 con 0x7f9fac1520a0 2024-09-19T04:15:30.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fa1ffb640 1 -- 172.21.15.76:0/4174910472 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c011c60 con 0x7f9fac1520a0 2024-09-19T04:15:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac14ac30 con 0x7f9fac1520a0 2024-09-19T04:15:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac06f910 con 0x7f9fac1520a0 2024-09-19T04:15:30.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fa1ffb640 1 -- 172.21.15.76:0/4174910472 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f9c018020 con 0x7f9fac1520a0 2024-09-19T04:15:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fa1ffb640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f900678f0 0x7f9f90069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fa1ffb640 1 -- 172.21.15.76:0/4174910472 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7f9f9c085720 con 0x7f9fac1520a0 2024-09-19T04:15:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fb15f5640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f900678f0 0x7f9f90069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.149+0000 7f9fb15f5640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f900678f0 0x7f9f90069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009790 tx=0x7f9fa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:30.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.153+0000 7f9fa1ffb640 1 -- 172.21.15.76:0/4174910472 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9f9c01f030 con 0x7f9fac1520a0 2024-09-19T04:15:30.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fac146ca0 con 0x7f9fac1520a0 2024-09-19T04:15:30.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fa1ffb640 1 -- 172.21.15.76:0/4174910472 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+3914 (secure 0 0 0) 0x7f9f9c0172c0 con 0x7f9fac1520a0 2024-09-19T04:15:30.355 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:30.355 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":24,"btime":"2024-09-19T04:15:24:380010+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4490,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1832856240","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1832856240},{"type":"v1","addr":"172.21.15.116:6835","nonce":1832856240}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1503570167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1503570167},{"type":"v1","addr":"172.21.15.116:6833","nonce":1503570167}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":5409,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/418235710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":418235710},{"type":"v1","addr":"172.21.15.76:6835","nonce":418235710}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":5412,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2519903200","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2519903200},{"type":"v1","addr":"172.21.15.76:6837","nonce":2519903200}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[]} 2024-09-19T04:15:30.356 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 24 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f900678f0 msgr2=0x7f9f90069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f900678f0 0x7f9f90069db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9fa8009790 tx=0x7f9fa8008040 comp rx=0 tx=0).stop 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac1520a0 msgr2=0x7f9fac06e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac1520a0 0x7f9fac06e9b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00b3e0 tx=0x7f9f9c00b8a0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 shutdown_connections 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f900678f0 0x7f9f90069db0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fac15f3a0 0x7f9fac06eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9fac1520a0 0x7f9fac06e9b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 --2- 172.21.15.76:0/4174910472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9fac06fe50 0x7f9fac147910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 >> 172.21.15.76:0/4174910472 conn(0x7f9fac0b1f90 msgr2=0x7f9fac0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 shutdown_connections 2024-09-19T04:15:30.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.353+0000 7f9fb3859640 1 -- 172.21.15.76:0/4174910472 wait complete. 2024-09-19T04:15:30.375 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T04:15:30.375 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T04:15:30.375 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T04:15:30.375 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:15:30.375 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T04:15:30.376 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T04:15:30.376 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T04:15:30.376 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:15:30.376 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T04:15:30.376 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T04:15:30.376 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T04:15:30.376 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:15:30.377 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T04:15:30.377 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T04:15:30.377 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T04:15:30.377 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:15:30.385 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T04:15:30.385 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T04:15:30.386 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T04:15:30.386 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T04:15:30.389 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T04:15:30.390 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T04:15:30.390 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T04:15:30.390 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T04:15:30.427 INFO:tasks.ceph.mds.a:Started 2024-09-19T04:15:30.428 DEBUG:teuthology.parallel:result is None 2024-09-19T04:15:30.428 INFO:tasks.ceph.mds.c:Started 2024-09-19T04:15:30.428 DEBUG:teuthology.parallel:result is None 2024-09-19T04:15:30.431 INFO:tasks.ceph.mds.b:Started 2024-09-19T04:15:30.432 DEBUG:teuthology.parallel:result is None 2024-09-19T04:15:30.432 INFO:tasks.ceph.mds.d:Started 2024-09-19T04:15:30.432 DEBUG:teuthology.parallel:result is None 2024-09-19T04:15:30.432 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T04:15:30.487 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:15:30.496 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:15:30.501 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:15:30.503 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:15:30.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc543a99640 1 Processor -- start 2024-09-19T04:15:30.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc543a99640 1 -- start start 2024-09-19T04:15:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc543a99640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc53c0cf080 0x7fc53c0cf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc543a99640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc53c0b5a20 con 0x7fc53c06e7a0 2024-09-19T04:15:30.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc543a99640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc53c0b5ca0 con 0x7fc53c162540 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc543a99640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc53c0b5f80 con 0x7fc53c0cf080 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc541034640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc53c0cf080 0x7fc53c0cf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc541034640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc53c0cf080 0x7fc53c0cf4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55934/0 (socket says 172.21.15.76:55934) 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc542036640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc53c06e7a0 0x7fc53c06ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52642/0 (socket says 172.21.15.76:52642) 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc541034640 1 -- 172.21.15.76:0/564977033 learned_addr learned my addr 172.21.15.76:0/564977033 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc541034640 1 -- 172.21.15.76:0/564977033 >> v1:172.21.15.116:6789/0 conn(0x7fc53c162540 legacy=0x7fc53c0b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc541034640 1 -- 172.21.15.76:0/564977033 >> v1:172.21.15.76:6789/0 conn(0x7fc53c06e7a0 legacy=0x7fc53c06ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc541034640 1 -- 172.21.15.76:0/564977033 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c0ce5a0 con 0x7fc53c0cf080 2024-09-19T04:15:30.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc541034640 1 --2- 172.21.15.76:0/564977033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c0cf080 0x7fc53c0cf4f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc538009870 tx=0x7fc53801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be11ad21fcb741fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc540833640 1 -- 172.21.15.76:0/564977033 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380ab660 con 0x7fc53c0cf080 2024-09-19T04:15:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc540833640 1 -- 172.21.15.76:0/564977033 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5380b0070 con 0x7fc53c0cf080 2024-09-19T04:15:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.549+0000 7fc540833640 1 -- 172.21.15.76:0/564977033 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5380aba20 con 0x7fc53c0cf080 2024-09-19T04:15:30.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- 172.21.15.76:0/564977033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c0cf080 msgr2=0x7fc53c0cf4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 --2- 172.21.15.76:0/564977033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c0cf080 0x7fc53c0cf4f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc538009870 tx=0x7fc53801c460 comp rx=0 tx=0).stop 2024-09-19T04:15:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- 172.21.15.76:0/564977033 shutdown_connections 2024-09-19T04:15:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 --2- 172.21.15.76:0/564977033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c0cf080 0x7fc53c0cf4f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- 172.21.15.76:0/564977033 >> 172.21.15.76:0/564977033 conn(0x7fc53c072670 msgr2=0x7fc53c072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- 172.21.15.76:0/564977033 shutdown_connections 2024-09-19T04:15:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- 172.21.15.76:0/564977033 wait complete. 2024-09-19T04:15:30.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 Processor -- start 2024-09-19T04:15:30.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- start start 2024-09-19T04:15:30.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06e7a0 0x7fc53c15cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc53c0cf080 0x7fc53c152750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c162540 0x7fc53c152c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c0d06b0 con 0x7fc53c06e7a0 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c0d0530 con 0x7fc53c162540 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c06d3e0 con 0x7fc53c0cf080 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06e7a0 0x7fc53c15cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06e7a0 0x7fc53c15cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55946/0 (socket says 172.21.15.76:55946) 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 -- 172.21.15.76:0/638132506 learned_addr learned my addr 172.21.15.76:0/638132506 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc542036640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c162540 0x7fc53c152c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541034640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc53c0cf080 0x7fc53c152750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 -- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc53c0cf080 msgr2=0x7fc53c152750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc53c0cf080 0x7fc53c152750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 -- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c162540 msgr2=0x7fc53c152c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c162540 0x7fc53c152c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 -- 172.21.15.76:0/638132506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c151ef0 con 0x7fc53c06e7a0 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc541835640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06e7a0 0x7fc53c15cfb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b440 tx=0x7fc52c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc5327fc640 1 -- 172.21.15.76:0/638132506 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c015020 con 0x7fc53c06e7a0 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc5327fc640 1 -- 172.21.15.76:0/638132506 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc52c011890 con 0x7fc53c06e7a0 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c1521d0 con 0x7fc53c06e7a0 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc5327fc640 1 -- 172.21.15.76:0/638132506 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c011c80 con 0x7fc53c06e7a0 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.553+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c15d4f0 con 0x7fc53c06e7a0 2024-09-19T04:15:30.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.557+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c15cd30 con 0x7fc53c06e7a0 2024-09-19T04:15:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.557+0000 7fc5327fc640 1 -- 172.21.15.76:0/638132506 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc52c018020 con 0x7fc53c06e7a0 2024-09-19T04:15:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.557+0000 7fc5327fc640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5200678c0 0x7fc520069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.557+0000 7fc5327fc640 1 -- 172.21.15.76:0/638132506 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5507+0+0 (secure 0 0 0) 0x7fc52c085cd0 con 0x7fc53c06e7a0 2024-09-19T04:15:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.557+0000 7fc541034640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5200678c0 0x7fc520069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:30.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.557+0000 7fc541034640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5200678c0 0x7fc520069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5380af000 tx=0x7fc5380a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:30.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.561+0000 7fc5327fc640 1 -- 172.21.15.76:0/638132506 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc52c01f020 con 0x7fc53c06e7a0 2024-09-19T04:15:30.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:30.733+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc53c0cf4f0 con 0x7fc53c06e7a0 2024-09-19T04:15:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.785+0000 7fc5327fc640 1 -- 172.21.15.76:0/638132506 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v49) ==== 97+0+0 (secure 0 0 0) 0x7fc52c00d040 con 0x7fc53c06e7a0 2024-09-19T04:15:31.788 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5200678c0 msgr2=0x7fc520069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5200678c0 0x7fc520069d80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5380af000 tx=0x7fc5380a7000 comp rx=0 tx=0).stop 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06e7a0 msgr2=0x7fc53c15cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06e7a0 0x7fc53c15cfb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc52c00b440 tx=0x7fc52c00b900 comp rx=0 tx=0).stop 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 shutdown_connections 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5200678c0 0x7fc520069d80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc53c162540 0x7fc53c152c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc53c0cf080 0x7fc53c152750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 --2- 172.21.15.76:0/638132506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc53c06e7a0 0x7fc53c15cfb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 >> 172.21.15.76:0/638132506 conn(0x7fc53c072670 msgr2=0x7fc53c157780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:31.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 shutdown_connections 2024-09-19T04:15:31.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.789+0000 7fc543a99640 1 -- 172.21.15.76:0/638132506 wait complete. 2024-09-19T04:15:31.813 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T04:15:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b4080640 1 Processor -- start 2024-09-19T04:15:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b4080640 1 -- start start 2024-09-19T04:15:31.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b4080640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6ac160460 0x7fc6ac15d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b4080640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac0b4620 con 0x7fc6ac0b3c40 2024-09-19T04:15:31.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b4080640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6ac0b4900 con 0x7fc6ac1567e0 2024-09-19T04:15:31.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b4080640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc6ac0b4be0 con 0x7fc6ac160460 2024-09-19T04:15:31.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b161b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc6ac0b3c40 0x7fc6ac15fd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52648/0 (socket says 172.21.15.76:52648) 2024-09-19T04:15:31.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b161b640 1 -- 172.21.15.76:0/2841542148 learned_addr learned my addr 172.21.15.76:0/2841542148 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:31.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b261d640 1 --2- 172.21.15.76:0/2841542148 >> v2:172.21.15.116:3300/0 conn(0x7fc6ac160460 0x7fc6ac15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 181473298 0 0) 0x7fc6ac0b4620 con 0x7fc6ac0b3c40 2024-09-19T04:15:31.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc698003530 con 0x7fc6ac0b3c40 2024-09-19T04:15:31.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3718456219 0 0) 0x7fc698003530 con 0x7fc6ac0b3c40 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 >> v2:172.21.15.116:3300/0 conn(0x7fc6ac160460 msgr2=0x7fc6ac15d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 --2- 172.21.15.76:0/2841542148 >> v2:172.21.15.116:3300/0 conn(0x7fc6ac160460 0x7fc6ac15d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 >> v1:172.21.15.116:6789/0 conn(0x7fc6ac1567e0 legacy=0x7fc6ac0b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac0b4ec0 con 0x7fc6ac0b3c40 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fc69c001680 con 0x7fc6ac0b3c40 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 >> v1:172.21.15.76:6789/0 conn(0x7fc6ac0b3c40 legacy=0x7fc6ac15fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6980039c0 0x7fc698003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc698004810 0x7fc698004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc69800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac0b4be0 con 0x7fc698005320 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc69800c1f0 con 0x7fc698004810 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc69800c3f0 con 0x7fc6980039c0 2024-09-19T04:15:31.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b261d640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc69800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b1e1c640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6980039c0 0x7fc698003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b161b640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc698004810 0x7fc698004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b261d640 1 -- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6980039c0 msgr2=0x7fc698003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b261d640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6980039c0 0x7fc698003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b261d640 1 -- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc698004810 msgr2=0x7fc698004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b261d640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc698004810 0x7fc698004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b261d640 1 -- 172.21.15.76:0/2841542148 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc698003530 con 0x7fc698005320 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b161b640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc698004810 0x7fc698004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.933+0000 7fc6b1e1c640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6980039c0 0x7fc698003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b261d640 1 -- 172.21.15.76:0/2841542148 reap_dead start 2024-09-19T04:15:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b261d640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc69800b9d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009fc0 tx=0x7fc6a401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9671aa93a72770a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4009c10 con 0x7fc698005320 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a40a5ca0 con 0x7fc698005320 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b0e1a640 1 -- 172.21.15.76:0/2841542148 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4003be0 con 0x7fc698005320 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 -- 172.21.15.76:0/2841542148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 msgr2=0x7fc69800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc69800b9d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009fc0 tx=0x7fc6a401cc90 comp rx=0 tx=0).stop 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 -- 172.21.15.76:0/2841542148 shutdown_connections 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2841542148 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc69800b9d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 -- 172.21.15.76:0/2841542148 >> 172.21.15.76:0/2841542148 conn(0x7fc6ac073580 msgr2=0x7fc6ac06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 -- 172.21.15.76:0/2841542148 shutdown_connections 2024-09-19T04:15:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 -- 172.21.15.76:0/2841542148 wait complete. 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 Processor -- start 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.937+0000 7fc6b4080640 1 -- start start 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc6ac1495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ac1f95f0 0x7fc6ac146b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ac14c9d0 0x7fc6ac14ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac15e270 con 0x7fc698005320 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc6ac160be0 con 0x7fc6ac1f95f0 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc6ac15e3f0 con 0x7fc6ac14c9d0 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc6ac1495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc6ac1495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55962/0 (socket says 172.21.15.76:55962) 2024-09-19T04:15:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 -- 172.21.15.76:0/2010532092 learned_addr learned my addr 172.21.15.76:0/2010532092 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b161b640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ac1f95f0 0x7fc6ac146b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 -- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ac14c9d0 msgr2=0x7fc6ac14ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b261d640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ac14c9d0 0x7fc6ac14ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ac14c9d0 0x7fc6ac14ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 -- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ac1f95f0 msgr2=0x7fc6ac146b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ac1f95f0 0x7fc6ac146b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 -- 172.21.15.76:0/2010532092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6ac1f68c0 con 0x7fc698005320 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b161b640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ac1f95f0 0x7fc6ac146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b1e1c640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc6ac1495d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800e4b0 tx=0x7fc6a800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6a27fc640 1 -- 172.21.15.76:0/2010532092 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a80052e0 con 0x7fc698005320 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6a27fc640 1 -- 172.21.15.76:0/2010532092 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6a8018070 con 0x7fc698005320 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6ac1f6ba0 con 0x7fc698005320 2024-09-19T04:15:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6a27fc640 1 -- 172.21.15.76:0/2010532092 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6a8005480 con 0x7fc698005320 2024-09-19T04:15:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6ac14bee0 con 0x7fc698005320 2024-09-19T04:15:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6ac0b4620 con 0x7fc698005320 2024-09-19T04:15:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6a27fc640 1 -- 172.21.15.76:0/2010532092 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc6a8003820 con 0x7fc698005320 2024-09-19T04:15:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6a27fc640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc6900678f0 0x7fc690069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6a27fc640 1 -- 172.21.15.76:0/2010532092 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7fc6a8085e30 con 0x7fc698005320 2024-09-19T04:15:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b161b640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc6900678f0 0x7fc690069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:31.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.941+0000 7fc6b161b640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc6900678f0 0x7fc690069db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0010f0 tx=0x7fc69c001db0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:31.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:31.945+0000 7fc6a27fc640 1 -- 172.21.15.76:0/2010532092 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc6a8056c10 con 0x7fc698005320 2024-09-19T04:15:32.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.149+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc6ac153ba0 con 0x7fc698005320 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.149+0000 7fc6a27fc640 1 -- 172.21.15.76:0/2010532092 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7fc6a8056510 con 0x7fc698005320 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w==", 2024-09-19T04:15:32.152 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA==", 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.153 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw==", 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:15:32.154 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA==", 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ==", 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w==", 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.156 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ==", 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.157 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw==", 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA==", 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.158 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw==", 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.159 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg==", 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.160 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA==", 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg==", 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.161 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg==", 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:15:32.162 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ==", 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:15:32.163 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA==", 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.164 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCopOtmGhUPLBAA9AkjLm1xF81Mjf1IYlN6Fg==", 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2024-09-19T04:15:32.165 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A==", 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg==", 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:15:32.166 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:15:32.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.167 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.167 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-19T04:15:32.167 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.149+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc6900678f0 msgr2=0x7fc690069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.149+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc6900678f0 0x7fc690069db0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc69c0010f0 tx=0x7fc69c001db0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.149+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 msgr2=0x7fc6ac1495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.149+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc6ac1495d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800e4b0 tx=0x7fc6a800e970 comp rx=0 tx=0).stop 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 shutdown_connections 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc6900678f0 0x7fc690069db0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc6ac14c9d0 0x7fc6ac14ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6ac1f95f0 0x7fc6ac146b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 --2- 172.21.15.76:0/2010532092 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc698005320 0x7fc6ac1495d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 >> 172.21.15.76:0/2010532092 conn(0x7fc6ac073580 msgr2=0x7fc6ac0b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:32.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 shutdown_connections 2024-09-19T04:15:32.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.153+0000 7fc6b4080640 1 -- 172.21.15.76:0/2010532092 wait complete. 2024-09-19T04:15:32.170 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T04:15:32.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.273+0000 7f61a8852640 1 Processor -- start 2024-09-19T04:15:32.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.273+0000 7f61a8852640 1 -- start start 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a406fef0 0x7f61a406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61a40b3d50 0x7f61a40b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61a406ee50 con 0x7f61a4073e60 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f61a40b46c0 con 0x7f61a40b3d50 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f61a406f130 con 0x7f61a406fef0 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61a40b3d50 0x7f61a40b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f61a40b3d50 0x7f61a40b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55974/0 (socket says 172.21.15.76:55974) 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 -- 172.21.15.76:0/1372936837 learned_addr learned my addr 172.21.15.76:0/1372936837 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a1d9b640 1 --2- 172.21.15.76:0/1372936837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a406fef0 0x7f61a406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 -- 172.21.15.76:0/1372936837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a406fef0 msgr2=0x7f61a406e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 --2- 172.21.15.76:0/1372936837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a406fef0 0x7f61a406e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 -- 172.21.15.76:0/1372936837 >> v1:172.21.15.116:6789/0 conn(0x7f61a4073e60 legacy=0x7f61a415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:32.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 -- 172.21.15.76:0/1372936837 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a406f410 con 0x7f61a40b3d50 2024-09-19T04:15:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a2d9d640 1 --2- 172.21.15.76:0/1372936837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a40b3d50 0x7f61a40b4180 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f619c00d150 tx=0x7f619c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e0ca5cba881beb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a159a640 1 -- 172.21.15.76:0/1372936837 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c00b2d0 con 0x7f61a40b3d50 2024-09-19T04:15:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a159a640 1 -- 172.21.15.76:0/1372936837 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f619c00b470 con 0x7f61a40b3d50 2024-09-19T04:15:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a159a640 1 -- 172.21.15.76:0/1372936837 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c0a7aa0 con 0x7f61a40b3d50 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- 172.21.15.76:0/1372936837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a40b3d50 msgr2=0x7f61a40b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 --2- 172.21.15.76:0/1372936837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a40b3d50 0x7f61a40b4180 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f619c00d150 tx=0x7f619c01eba0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- 172.21.15.76:0/1372936837 shutdown_connections 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 --2- 172.21.15.76:0/1372936837 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a40b3d50 0x7f61a40b4180 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 --2- 172.21.15.76:0/1372936837 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a406fef0 0x7f61a406e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- 172.21.15.76:0/1372936837 >> 172.21.15.76:0/1372936837 conn(0x7f61a40b1f90 msgr2=0x7f61a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- 172.21.15.76:0/1372936837 shutdown_connections 2024-09-19T04:15:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.277+0000 7f61a8852640 1 -- 172.21.15.76:0/1372936837 wait complete. 2024-09-19T04:15:32.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 Processor -- start 2024-09-19T04:15:32.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 -- start start 2024-09-19T04:15:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a406fef0 0x7f61a41f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a4073e60 0x7f61a40ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a40b3d50 0x7f61a4163940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f61a40729a0 con 0x7f61a406fef0 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f61a4072820 con 0x7f61a4073e60 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f61a4072b20 con 0x7f61a40b3d50 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a406fef0 0x7f61a41f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a406fef0 0x7f61a41f7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55988/0 (socket says 172.21.15.76:55988) 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 -- 172.21.15.76:0/1114032055 learned_addr learned my addr 172.21.15.76:0/1114032055 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a2d9d640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a40b3d50 0x7f61a4163940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a1d9b640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a4073e60 0x7f61a40ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 -- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a40b3d50 msgr2=0x7f61a4163940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a40b3d50 0x7f61a4163940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 -- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a4073e60 msgr2=0x7f61a40ad550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a4073e60 0x7f61a40ad550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 -- 172.21.15.76:0/1114032055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a4164040 con 0x7f61a406fef0 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a1d9b640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a4073e60 0x7f61a40ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a259c640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a406fef0 0x7f61a41f7e40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f619400c450 tx=0x7f619400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f6192ffd640 1 -- 172.21.15.76:0/1114032055 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194016020 con 0x7f61a406fef0 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f6192ffd640 1 -- 172.21.15.76:0/1114032055 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61940052d0 con 0x7f61a406fef0 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a4164320 con 0x7f61a406fef0 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f6192ffd640 1 -- 172.21.15.76:0/1114032055 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6194007260 con 0x7f61a406fef0 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a41648e0 con 0x7f61a406fef0 2024-09-19T04:15:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.281+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a406e800 con 0x7f61a406fef0 2024-09-19T04:15:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.285+0000 7f6192ffd640 1 -- 172.21.15.76:0/1114032055 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6194007400 con 0x7f61a406fef0 2024-09-19T04:15:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.285+0000 7f6192ffd640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61840678c0 0x7f6184069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.285+0000 7f61a1d9b640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61840678c0 0x7f6184069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.285+0000 7f6192ffd640 1 -- 172.21.15.76:0/1114032055 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f6194085410 con 0x7f61a406fef0 2024-09-19T04:15:32.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.285+0000 7f61a1d9b640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61840678c0 0x7f6184069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61a41fa7e0 tx=0x7f6198008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.289+0000 7f6192ffd640 1 -- 172.21.15.76:0/1114032055 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6194020020 con 0x7f61a406fef0 2024-09-19T04:15:32.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f61a40723f0 con 0x7f61a406fef0 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f6192ffd640 1 -- 172.21.15.76:0/1114032055 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v49) ==== 84+0+65 (secure 0 0 0) 0x7f6194005480 con 0x7f61a406fef0 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:32.455 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:15:32.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61840678c0 msgr2=0x7f6184069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61840678c0 0x7f6184069d80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f61a41fa7e0 tx=0x7f6198008040 comp rx=0 tx=0).stop 2024-09-19T04:15:32.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a406fef0 msgr2=0x7f61a41f7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a406fef0 0x7f61a41f7e40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f619400c450 tx=0x7f619400c910 comp rx=0 tx=0).stop 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 shutdown_connections 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61840678c0 0x7f6184069d80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f61a40b3d50 0x7f61a4163940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61a4073e60 0x7f61a40ad550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 --2- 172.21.15.76:0/1114032055 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f61a406fef0 0x7f61a41f7e40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 >> 172.21.15.76:0/1114032055 conn(0x7f61a40b1f90 msgr2=0x7f61a415a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 shutdown_connections 2024-09-19T04:15:32.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.453+0000 7f61a8852640 1 -- 172.21.15.76:0/1114032055 wait complete. 2024-09-19T04:15:32.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:15:32.473 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:15:32.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa71c8640 1 Processor -- start 2024-09-19T04:15:32.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa71c8640 1 -- start start 2024-09-19T04:15:32.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa71c8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5aa01520a0 0x7f5aa006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa71c8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5aa006fe50 0x7f5aa015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa71c8640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aa0070250 con 0x7f5aa015f1d0 2024-09-19T04:15:32.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa71c8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5aa0152480 con 0x7f5aa01520a0 2024-09-19T04:15:32.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa71c8640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5aa00aede0 con 0x7f5aa006fe50 2024-09-19T04:15:32.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa4f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5aa01520a0 0x7f5aa006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa4f64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5aa01520a0 0x7f5aa006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:55994/0 (socket says 172.21.15.76:55994) 2024-09-19T04:15:32.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa4f64640 1 -- 172.21.15.76:0/4150813522 learned_addr learned my addr 172.21.15.76:0/4150813522 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:32.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5a9ffff640 1 --2- 172.21.15.76:0/4150813522 >> v2:172.21.15.116:3300/0 conn(0x7f5aa006fe50 0x7f5aa015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa4f64640 1 -- 172.21.15.76:0/4150813522 >> v2:172.21.15.116:3300/0 conn(0x7f5aa006fe50 msgr2=0x7f5aa015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:32.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa4f64640 1 --2- 172.21.15.76:0/4150813522 >> v2:172.21.15.116:3300/0 conn(0x7f5aa006fe50 0x7f5aa015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa4f64640 1 -- 172.21.15.76:0/4150813522 >> v1:172.21.15.116:6789/0 conn(0x7f5aa015f1d0 legacy=0x7f5aa0071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:32.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.593+0000 7f5aa4f64640 1 -- 172.21.15.76:0/4150813522 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa015fa10 con 0x7f5aa01520a0 2024-09-19T04:15:32.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa4f64640 1 --2- 172.21.15.76:0/4150813522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa01520a0 0x7f5aa006f910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5aa006e510 tx=0x7f5a8801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fa63db077b9d1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5a9f7fe640 1 -- 172.21.15.76:0/4150813522 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880b1070 con 0x7f5aa01520a0 2024-09-19T04:15:32.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5a9f7fe640 1 -- 172.21.15.76:0/4150813522 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a880a8050 con 0x7f5aa01520a0 2024-09-19T04:15:32.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5a9f7fe640 1 -- 172.21.15.76:0/4150813522 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a880ac860 con 0x7f5aa01520a0 2024-09-19T04:15:32.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 -- 172.21.15.76:0/4150813522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa01520a0 msgr2=0x7f5aa006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/4150813522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa01520a0 0x7f5aa006f910 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f5aa006e510 tx=0x7f5a8801c4c0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 -- 172.21.15.76:0/4150813522 shutdown_connections 2024-09-19T04:15:32.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/4150813522 >> v2:172.21.15.116:3300/0 conn(0x7f5aa006fe50 0x7f5aa015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/4150813522 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa01520a0 0x7f5aa006f910 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 -- 172.21.15.76:0/4150813522 >> 172.21.15.76:0/4150813522 conn(0x7f5aa00b1f90 msgr2=0x7f5aa00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:32.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 -- 172.21.15.76:0/4150813522 shutdown_connections 2024-09-19T04:15:32.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.597+0000 7f5aa71c8640 1 -- 172.21.15.76:0/4150813522 wait complete. 2024-09-19T04:15:32.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 Processor -- start 2024-09-19T04:15:32.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 -- start start 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aa006fe50 0x7f5aa014bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aa01520a0 0x7f5aa014d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa015f1d0 0x7f5aa00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0073d30 con 0x7f5aa015f1d0 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa0073a50 con 0x7f5aa006fe50 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa015f660 con 0x7f5aa01520a0 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa015f1d0 0x7f5aa00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa015f1d0 0x7f5aa00cfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56004/0 (socket says 172.21.15.76:56004) 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 -- 172.21.15.76:0/3377187519 learned_addr learned my addr 172.21.15.76:0/3377187519 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:32.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa4f64640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aa006fe50 0x7f5aa014bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5a9ffff640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aa01520a0 0x7f5aa014d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 -- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aa01520a0 msgr2=0x7f5aa014d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aa01520a0 0x7f5aa014d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 -- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aa006fe50 msgr2=0x7f5aa014bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aa006fe50 0x7f5aa014bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 -- 172.21.15.76:0/3377187519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa014c510 con 0x7f5aa015f1d0 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa4f64640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aa006fe50 0x7f5aa014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa5765640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa015f1d0 0x7f5aa00cfa20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800c780 tx=0x7f5a9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5a9d7fa640 1 -- 172.21.15.76:0/3377187519 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a98004b70 con 0x7f5aa015f1d0 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5a9d7fa640 1 -- 172.21.15.76:0/3377187519 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a98004d10 con 0x7f5aa015f1d0 2024-09-19T04:15:32.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5a9d7fa640 1 -- 172.21.15.76:0/3377187519 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800aa90 con 0x7f5aa015f1d0 2024-09-19T04:15:32.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa014c7f0 con 0x7f5aa015f1d0 2024-09-19T04:15:32.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa014cdb0 con 0x7f5aa015f1d0 2024-09-19T04:15:32.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aa015ec90 con 0x7f5aa015f1d0 2024-09-19T04:15:32.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.601+0000 7f5a9d7fa640 1 -- 172.21.15.76:0/3377187519 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a9800ac30 con 0x7f5aa015f1d0 2024-09-19T04:15:32.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.605+0000 7f5a9d7fa640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a800678c0 0x7f5a80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:32.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.605+0000 7f5a9d7fa640 1 -- 172.21.15.76:0/3377187519 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5249+0+0 (secure 0 0 0) 0x7f5a98085d40 con 0x7f5aa015f1d0 2024-09-19T04:15:32.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.605+0000 7f5aa4f64640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a800678c0 0x7f5a80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:32.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.605+0000 7f5aa4f64640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a800678c0 0x7f5a80069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5a880ae000 tx=0x7f5a88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:32.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.609+0000 7f5a9d7fa640 1 -- 172.21.15.76:0/3377187519 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5a98020070 con 0x7f5aa015f1d0 2024-09-19T04:15:32.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:32.769+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f5aa006e540 con 0x7f5aa015f1d0 2024-09-19T04:15:33.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.449+0000 7f5a9d7fa640 1 -- 172.21.15.76:0/3377187519 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v50) ==== 138+0+0 (secure 0 0 0) 0x7f5a98056ad0 con 0x7f5aa015f1d0 2024-09-19T04:15:33.451 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-09-19T04:15:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a800678c0 msgr2=0x7f5a80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a800678c0 0x7f5a80069d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5a880ae000 tx=0x7f5a88008040 comp rx=0 tx=0).stop 2024-09-19T04:15:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa015f1d0 msgr2=0x7f5aa00cfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:33.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa015f1d0 0x7f5aa00cfa20 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800c780 tx=0x7f5a9800cc40 comp rx=0 tx=0).stop 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 shutdown_connections 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a800678c0 0x7f5a80069d80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aa015f1d0 0x7f5aa00cfa20 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aa01520a0 0x7f5aa014d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 --2- 172.21.15.76:0/3377187519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aa006fe50 0x7f5aa014bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 >> 172.21.15.76:0/3377187519 conn(0x7f5aa00b1f90 msgr2=0x7f5aa006ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 shutdown_connections 2024-09-19T04:15:33.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.453+0000 7f5aa71c8640 1 -- 172.21.15.76:0/3377187519 wait complete. 2024-09-19T04:15:33.480 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:15:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee4098640 1 Processor -- start 2024-09-19T04:15:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee4098640 1 -- start start 2024-09-19T04:15:33.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee4098640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faedc0cc870 0x7faedc0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:33.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee4098640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc06de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:33.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee4098640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faedc0ce8d0 con 0x7faedc073100 2024-09-19T04:15:33.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee4098640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7faedc0cebb0 con 0x7faedc0cc870 2024-09-19T04:15:33.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee4098640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7faedc0cee90 con 0x7faedc0cdfd0 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee2635640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee1e34640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faedc0cc870 0x7faedc0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee2635640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc06de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:49944/0 (socket says 172.21.15.76:49944) 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee2635640 1 -- 172.21.15.76:0/580358573 learned_addr learned my addr 172.21.15.76:0/580358573 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee2635640 1 -- 172.21.15.76:0/580358573 >> v2:172.21.15.116:3300/0 conn(0x7faedc0cc870 msgr2=0x7faedc0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee2635640 1 --2- 172.21.15.76:0/580358573 >> v2:172.21.15.116:3300/0 conn(0x7faedc0cc870 0x7faedc0cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee2635640 1 -- 172.21.15.76:0/580358573 >> v1:172.21.15.116:6789/0 conn(0x7faedc073100 legacy=0x7faedc0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:33.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.605+0000 7faee2635640 1 -- 172.21.15.76:0/580358573 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc0cf170 con 0x7faedc0cdfd0 2024-09-19T04:15:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee2635640 1 --2- 172.21.15.76:0/580358573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc06de00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faed4009e50 tx=0x7faed401ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9f1c6bf18a3cb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee0e32640 1 -- 172.21.15.76:0/580358573 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed40a58f0 con 0x7faedc0cdfd0 2024-09-19T04:15:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee0e32640 1 -- 172.21.15.76:0/580358573 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faed40a5a90 con 0x7faedc0cdfd0 2024-09-19T04:15:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee0e32640 1 -- 172.21.15.76:0/580358573 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed40a5e60 con 0x7faedc0cdfd0 2024-09-19T04:15:33.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 -- 172.21.15.76:0/580358573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 msgr2=0x7faedc06de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:33.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 --2- 172.21.15.76:0/580358573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc06de00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faed4009e50 tx=0x7faed401ca00 comp rx=0 tx=0).stop 2024-09-19T04:15:33.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 -- 172.21.15.76:0/580358573 shutdown_connections 2024-09-19T04:15:33.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 --2- 172.21.15.76:0/580358573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc06de00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 --2- 172.21.15.76:0/580358573 >> v2:172.21.15.116:3300/0 conn(0x7faedc0cc870 0x7faedc0cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 -- 172.21.15.76:0/580358573 >> 172.21.15.76:0/580358573 conn(0x7faedc0b1c80 msgr2=0x7faedc0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:33.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 -- 172.21.15.76:0/580358573 shutdown_connections 2024-09-19T04:15:33.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.609+0000 7faee4098640 1 -- 172.21.15.76:0/580358573 wait complete. 2024-09-19T04:15:33.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 Processor -- start 2024-09-19T04:15:33.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 -- start start 2024-09-19T04:15:33.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faedc073100 0x7faedc159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:33.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faedc0cc870 0x7faedc142210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc13ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faedc169310 con 0x7faedc0cc870 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faedc169190 con 0x7faedc073100 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7faedc169490 con 0x7faedc0cdfd0 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faedc0cc870 0x7faedc142210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faedc0cc870 0x7faedc142210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34994/0 (socket says 172.21.15.76:34994) 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 -- 172.21.15.76:0/2875229535 learned_addr learned my addr 172.21.15.76:0/2875229535 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1e34640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faedc073100 0x7faedc159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee2635640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc13ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:33.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 -- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 msgr2=0x7faedc13ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc13ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 -- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faedc073100 msgr2=0x7faedc159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faedc073100 0x7faedc159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 -- 172.21.15.76:0/2875229535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faedc13cb80 con 0x7faedc0cc870 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1e34640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faedc073100 0x7faedc159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee1633640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faedc0cc870 0x7faedc142210 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faecc00c3c0 tx=0x7faecc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faed27fc640 1 -- 172.21.15.76:0/2875229535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc017070 con 0x7faedc0cc870 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faed27fc640 1 -- 172.21.15.76:0/2875229535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faecc0052d0 con 0x7faedc0cc870 2024-09-19T04:15:33.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faedc13ce60 con 0x7faedc0cc870 2024-09-19T04:15:33.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faed27fc640 1 -- 172.21.15.76:0/2875229535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faecc003820 con 0x7faedc0cc870 2024-09-19T04:15:33.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faedc13d420 con 0x7faedc0cc870 2024-09-19T04:15:33.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.613+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faedc0cf970 con 0x7faedc0cc870 2024-09-19T04:15:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.617+0000 7faed27fc640 1 -- 172.21.15.76:0/2875229535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faecc01a070 con 0x7faedc0cc870 2024-09-19T04:15:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.617+0000 7faed27fc640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faec00678c0 0x7faec0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.617+0000 7faed27fc640 1 -- 172.21.15.76:0/2875229535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5619+0+0 (secure 0 0 0) 0x7faecc085010 con 0x7faedc0cc870 2024-09-19T04:15:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.617+0000 7faee1e34640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faec00678c0 0x7faec0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:33.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.617+0000 7faee1e34640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faec00678c0 0x7faec0069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faed8009bb0 tx=0x7faed8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:33.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.621+0000 7faed27fc640 1 -- 172.21.15.76:0/2875229535 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faecc005480 con 0x7faedc0cc870 2024-09-19T04:15:33.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:33.773+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7faedc0af8d0 con 0x7faedc0cc870 2024-09-19T04:15:34.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.461+0000 7faed27fc640 1 -- 172.21.15.76:0/2875229535 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v51) ==== 170+0+0 (secure 0 0 0) 0x7faecc055b10 con 0x7faedc0cc870 2024-09-19T04:15:34.464 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-09-19T04:15:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faec00678c0 msgr2=0x7faec0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faec00678c0 0x7faec0069d80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7faed8009bb0 tx=0x7faed8008040 comp rx=0 tx=0).stop 2024-09-19T04:15:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faedc0cc870 msgr2=0x7faedc142210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faedc0cc870 0x7faedc142210 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7faecc00c3c0 tx=0x7faecc00c880 comp rx=0 tx=0).stop 2024-09-19T04:15:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 shutdown_connections 2024-09-19T04:15:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faec00678c0 0x7faec0069d80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faedc0cdfd0 0x7faedc13ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faedc0cc870 0x7faedc142210 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 --2- 172.21.15.76:0/2875229535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faedc073100 0x7faedc159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 >> 172.21.15.76:0/2875229535 conn(0x7faedc0b1c80 msgr2=0x7faedc0afbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 shutdown_connections 2024-09-19T04:15:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.465+0000 7faee4098640 1 -- 172.21.15.76:0/2875229535 wait complete. 2024-09-19T04:15:34.500 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:15:34.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f02c4d640 1 Processor -- start 2024-09-19T04:15:34.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f02c4d640 1 -- start start 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f02c4d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2efc161a60 0x7f2efc0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f02c4d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2efc0cf620 0x7f2efc06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f02c4d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2efc0d32b0 con 0x7f2efc0d2e90 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f02c4d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2efc0d0300 con 0x7f2efc0cf620 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f02c4d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2efc0cfa90 con 0x7f2efc161a60 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2efbfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2efc0d2e90 0x7f2efc0cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47250/0 (socket says 172.21.15.76:47250) 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f011ea640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2efc0cf620 0x7f2efc06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:34.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2efbfff640 1 -- 172.21.15.76:0/369453931 learned_addr learned my addr 172.21.15.76:0/369453931 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:34.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f009e9640 1 --2- 172.21.15.76:0/369453931 >> v2:172.21.15.116:3300/0 conn(0x7f2efc161a60 0x7f2efc0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:34.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f011ea640 1 -- 172.21.15.76:0/369453931 >> v2:172.21.15.116:3300/0 conn(0x7f2efc161a60 msgr2=0x7f2efc0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:34.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f011ea640 1 --2- 172.21.15.76:0/369453931 >> v2:172.21.15.116:3300/0 conn(0x7f2efc161a60 0x7f2efc0d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f011ea640 1 -- 172.21.15.76:0/369453931 >> v1:172.21.15.76:6789/0 conn(0x7f2efc0d2e90 legacy=0x7f2efc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:34.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2efb7fe640 1 -- 172.21.15.76:0/369453931 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1943308343 0 0) 0x7f2efc0d32b0 con 0x7f2efc0d2e90 2024-09-19T04:15:34.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f011ea640 1 -- 172.21.15.76:0/369453931 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc0b0f60 con 0x7f2efc0cf620 2024-09-19T04:15:34.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.617+0000 7f2f009e9640 1 --2- 172.21.15.76:0/369453931 >> v2:172.21.15.116:3300/0 conn(0x7f2efc161a60 0x7f2efc0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:34.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f011ea640 1 --2- 172.21.15.76:0/369453931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0cf620 0x7f2efc06d6b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2efc08e010 tx=0x7f2ef401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a95630880dc25a33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:34.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2efb7fe640 1 -- 172.21.15.76:0/369453931 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40b2070 con 0x7f2efc0cf620 2024-09-19T04:15:34.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2efb7fe640 1 -- 172.21.15.76:0/369453931 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef40a5930 con 0x7f2efc0cf620 2024-09-19T04:15:34.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2efb7fe640 1 -- 172.21.15.76:0/369453931 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40a5d00 con 0x7f2efc0cf620 2024-09-19T04:15:34.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 -- 172.21.15.76:0/369453931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0cf620 msgr2=0x7f2efc06d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:34.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/369453931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0cf620 0x7f2efc06d6b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2efc08e010 tx=0x7f2ef401c870 comp rx=0 tx=0).stop 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 -- 172.21.15.76:0/369453931 shutdown_connections 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/369453931 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0cf620 0x7f2efc06d6b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/369453931 >> v2:172.21.15.116:3300/0 conn(0x7f2efc161a60 0x7f2efc0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 -- 172.21.15.76:0/369453931 >> 172.21.15.76:0/369453931 conn(0x7f2efc08bf90 msgr2=0x7f2efc09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 -- 172.21.15.76:0/369453931 shutdown_connections 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 -- 172.21.15.76:0/369453931 wait complete. 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 Processor -- start 2024-09-19T04:15:34.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.621+0000 7f2f02c4d640 1 -- start start 2024-09-19T04:15:34.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2efc0cf620 0x7f2efc154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0d2e90 0x7f2efc0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2efc161a60 0x7f2efc0b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc06e400 con 0x7f2efc0d2e90 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc06d190 con 0x7f2efc161a60 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2efc06e580 con 0x7f2efc0cf620 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0d2e90 0x7f2efc0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0d2e90 0x7f2efc0c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35010/0 (socket says 172.21.15.76:35010) 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 -- 172.21.15.76:0/28826439 learned_addr learned my addr 172.21.15.76:0/28826439 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f011ea640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2efc161a60 0x7f2efc0b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f009e9640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2efc0cf620 0x7f2efc154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 -- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2efc0cf620 msgr2=0x7f2efc154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:34.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2efc0cf620 0x7f2efc154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 -- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2efc161a60 msgr2=0x7f2efc0b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2efc161a60 0x7f2efc0b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 -- 172.21.15.76:0/28826439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc0c9c20 con 0x7f2efc0d2e90 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f011ea640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2efc161a60 0x7f2efc0b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2efbfff640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0d2e90 0x7f2efc0c7e80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00011f0 tx=0x7f2ef000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2ef97fa640 1 -- 172.21.15.76:0/28826439 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0018070 con 0x7f2efc0d2e90 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2ef97fa640 1 -- 172.21.15.76:0/28826439 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef00074e0 con 0x7f2efc0d2e90 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc0c9f00 con 0x7f2efc0d2e90 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2ef97fa640 1 -- 172.21.15.76:0/28826439 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0004a10 con 0x7f2efc0d2e90 2024-09-19T04:15:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc0ca4c0 con 0x7f2efc0d2e90 2024-09-19T04:15:34.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2efc06d6b0 con 0x7f2efc0d2e90 2024-09-19T04:15:34.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2ef97fa640 1 -- 172.21.15.76:0/28826439 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ef001b070 con 0x7f2efc0d2e90 2024-09-19T04:15:34.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2ef97fa640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee80678f0 0x7f2ee8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:34.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2ef97fa640 1 -- 172.21.15.76:0/28826439 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 6001+0+0 (secure 0 0 0) 0x7f2ef0085400 con 0x7f2efc0d2e90 2024-09-19T04:15:34.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f009e9640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee80678f0 0x7f2ee8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:34.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.625+0000 7f2f009e9640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee80678f0 0x7f2ee8069db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2eec006fd0 tx=0x7f2eec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:34.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.629+0000 7f2ef97fa640 1 -- 172.21.15.76:0/28826439 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2ef0055ef0 con 0x7f2efc0d2e90 2024-09-19T04:15:34.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:34.825+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f2efc0d1440 con 0x7f2efc0d2e90 2024-09-19T04:15:35.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.489+0000 7f2ef97fa640 1 -- 172.21.15.76:0/28826439 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v26) ==== 172+0+0 (secure 0 0 0) 0x7f2ef00557f0 con 0x7f2efc0d2e90 2024-09-19T04:15:35.495 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 7 and data pool 8 2024-09-19T04:15:35.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee80678f0 msgr2=0x7f2ee8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee80678f0 0x7f2ee8069db0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2eec006fd0 tx=0x7f2eec008040 comp rx=0 tx=0).stop 2024-09-19T04:15:35.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0d2e90 msgr2=0x7f2efc0c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0d2e90 0x7f2efc0c7e80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00011f0 tx=0x7f2ef000c9b0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 shutdown_connections 2024-09-19T04:15:35.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee80678f0 0x7f2ee8069db0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2efc161a60 0x7f2efc0b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2efc0d2e90 0x7f2efc0c7e80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 --2- 172.21.15.76:0/28826439 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2efc0cf620 0x7f2efc154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.497+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 >> 172.21.15.76:0/28826439 conn(0x7f2efc08bf90 msgr2=0x7f2efc09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:35.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.501+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 shutdown_connections 2024-09-19T04:15:35.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.501+0000 7f2f02c4d640 1 -- 172.21.15.76:0/28826439 wait complete. 2024-09-19T04:15:35.534 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:15:35.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.645+0000 7efd81f37640 1 Processor -- start 2024-09-19T04:15:35.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.645+0000 7efd81f37640 1 -- start start 2024-09-19T04:15:35.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd81f37640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efd7c161a60 0x7efd7c0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd81f37640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 0x7efd7c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd81f37640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd7c0d32b0 con 0x7efd7c0d2e90 2024-09-19T04:15:35.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd81f37640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efd7c0d0300 con 0x7efd7c161a60 2024-09-19T04:15:35.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd81f37640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c0cfab0 con 0x7efd7c0cf620 2024-09-19T04:15:35.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efd7c0d2e90 0x7efd7c0cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47252/0 (socket says 172.21.15.76:47252) 2024-09-19T04:15:35.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7affd640 1 -- 172.21.15.76:0/2360347424 learned_addr learned my addr 172.21.15.76:0/2360347424 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:35.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7b7fe640 1 --2- 172.21.15.76:0/2360347424 >> v2:172.21.15.116:3300/0 conn(0x7efd7c161a60 0x7efd7c0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7bfff640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 0x7efd7c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 170277159 0 0) 0x7efd7c0d32b0 con 0x7efd7c0d2e90 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd68003530 con 0x7efd7c0d2e90 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2547495358 0 0) 0x7efd68003530 con 0x7efd7c0d2e90 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 msgr2=0x7efd7c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 0x7efd7c06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 >> v2:172.21.15.116:3300/0 conn(0x7efd7c161a60 msgr2=0x7efd7c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 --2- 172.21.15.76:0/2360347424 >> v2:172.21.15.116:3300/0 conn(0x7efd7c161a60 0x7efd7c0d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c0b0f60 con 0x7efd7c0d2e90 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7efd64001680 con 0x7efd7c0d2e90 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 >> v1:172.21.15.76:6789/0 conn(0x7efd7c0d2e90 legacy=0x7efd7c0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd680039c0 0x7efd68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 0x7efd68004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd68004d50 0x7efd680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c0d0300 con 0x7efd68004d50 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c0cfab0 con 0x7efd680039c0 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efd6800b9d0 con 0x7efd7c0cf620 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7bfff640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd68004d50 0x7efd680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7b7fe640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd680039c0 0x7efd68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7affd640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 0x7efd68004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7bfff640 1 -- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 msgr2=0x7efd68004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:35.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7bfff640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c0cf620 0x7efd68004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7bfff640 1 -- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd680039c0 msgr2=0x7efd68003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:35.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7bfff640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd680039c0 0x7efd68003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.649+0000 7efd7bfff640 1 -- 172.21.15.76:0/2360347424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd68003530 con 0x7efd68004d50 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7bfff640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd68004d50 0x7efd680051d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efd74009fa0 tx=0x7efd7401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815114325ec7bca4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd740b2070 con 0x7efd68004d50 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd740a5d10 con 0x7efd68004d50 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7a7fc640 1 -- 172.21.15.76:0/2360347424 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74009bf0 con 0x7efd68004d50 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- 172.21.15.76:0/2360347424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd68004d50 msgr2=0x7efd680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 --2- 172.21.15.76:0/2360347424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd68004d50 0x7efd680051d0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7efd74009fa0 tx=0x7efd7401cc40 comp rx=0 tx=0).stop 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7bfff640 1 -- 172.21.15.76:0/2360347424 reap_dead start 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- 172.21.15.76:0/2360347424 shutdown_connections 2024-09-19T04:15:35.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- 172.21.15.76:0/2360347424 >> 172.21.15.76:0/2360347424 conn(0x7efd7c08bf90 msgr2=0x7efd7c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- 172.21.15.76:0/2360347424 shutdown_connections 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- 172.21.15.76:0/2360347424 wait complete. 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 Processor -- start 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- start start 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c150600 0x7efd7c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c151b00 0x7efd7c1509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c150f20 0x7efd7c151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c06ceb0 con 0x7efd7c150600 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c0d0020 con 0x7efd7c150f20 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd81f37640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c06d190 con 0x7efd7c151b00 2024-09-19T04:15:35.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c150600 0x7efd7c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c150600 0x7efd7c0caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35016/0 (socket says 172.21.15.76:35016) 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 -- 172.21.15.76:0/3437781436 learned_addr learned my addr 172.21.15.76:0/3437781436 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 -- 172.21.15.76:0/3437781436 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c151b00 msgr2=0x7efd7c1509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c151b00 0x7efd7c1509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7bfff640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c150f20 0x7efd7c151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 -- 172.21.15.76:0/3437781436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c150f20 msgr2=0x7efd7c151390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c150f20 0x7efd7c151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 -- 172.21.15.76:0/3437781436 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c0cd0f0 con 0x7efd7c150600 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.653+0000 7efd7b7fe640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c150600 0x7efd7c0caa40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0115e0 tx=0x7efd6c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd73fff640 1 -- 172.21.15.76:0/3437781436 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00f8f0 con 0x7efd7c150600 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd73fff640 1 -- 172.21.15.76:0/3437781436 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd6c00fa90 con 0x7efd7c150600 2024-09-19T04:15:35.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c0cd3d0 con 0x7efd7c150600 2024-09-19T04:15:35.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd73fff640 1 -- 172.21.15.76:0/3437781436 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00fe40 con 0x7efd7c150600 2024-09-19T04:15:35.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c0cd990 con 0x7efd7c150600 2024-09-19T04:15:35.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c0d1700 con 0x7efd7c150600 2024-09-19T04:15:35.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd73fff640 1 -- 172.21.15.76:0/3437781436 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efd6c015460 con 0x7efd7c150600 2024-09-19T04:15:35.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd73fff640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd5c0678f0 0x7efd5c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd7affd640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd5c0678f0 0x7efd5c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd73fff640 1 -- 172.21.15.76:0/3437781436 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7efd6c086800 con 0x7efd7c150600 2024-09-19T04:15:35.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.657+0000 7efd7affd640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd5c0678f0 0x7efd5c069db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd64004850 tx=0x7efd64001440 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:35.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.661+0000 7efd73fff640 1 -- 172.21.15.76:0/3437781436 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efd6c019030 con 0x7efd7c150600 2024-09-19T04:15:35.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.829+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efd7c0d1420 con 0x7efd7c150600 2024-09-19T04:15:35.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.829+0000 7efd73fff640 1 -- 172.21.15.76:0/3437781436 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+17039 (secure 0 0 0) 0x7efd6c0571b0 con 0x7efd7c150600 2024-09-19T04:15:35.833 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:35.833 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":52,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:15:34.830913+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:15:32.810605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T04:15:33.811920+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:35.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd5c0678f0 msgr2=0x7efd5c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd5c0678f0 0x7efd5c069db0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7efd64004850 tx=0x7efd64001440 comp rx=0 tx=0).stop 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c150600 msgr2=0x7efd7c0caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c150600 0x7efd7c0caa40 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7efd6c0115e0 tx=0x7efd6c011aa0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 shutdown_connections 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd5c0678f0 0x7efd5c069db0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c150f20 0x7efd7c151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd7c151b00 0x7efd7c1509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 --2- 172.21.15.76:0/3437781436 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c150600 0x7efd7c0caa40 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 >> 172.21.15.76:0/3437781436 conn(0x7efd7c08bf90 msgr2=0x7efd7c0d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:35.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 shutdown_connections 2024-09-19T04:15:35.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.833+0000 7efd81f37640 1 -- 172.21.15.76:0/3437781436 wait complete. 2024-09-19T04:15:35.852 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:15:35.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f13433c1640 1 Processor -- start 2024-09-19T04:15:35.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f13433c1640 1 -- start start 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f13433c1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 0x7f133c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f13433c1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c070250 con 0x7f133c15f3a0 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f13433c1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133c0aede0 con 0x7f133c06fe50 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f13433c1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f133c152480 con 0x7f133c1520a0 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f134195e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f133c15f3a0 0x7f133c0adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47258/0 (socket says 172.21.15.76:47258) 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f134195e640 1 -- 172.21.15.76:0/617513674 learned_addr learned my addr 172.21.15.76:0/617513674 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f134115d640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 0x7f133c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 368616684 0 0) 0x7f133c070250 con 0x7f133c15f3a0 2024-09-19T04:15:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.965+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1328003530 con 0x7f133c15f3a0 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3788096585 0 0) 0x7f1328003530 con 0x7f133c15f3a0 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 msgr2=0x7f133c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 0x7f133c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 >> v1:172.21.15.116:6789/0 conn(0x7f133c06fe50 legacy=0x7f133c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c15fa10 con 0x7f133c15f3a0 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f1334001a00 con 0x7f133c15f3a0 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 >> v1:172.21.15.76:6789/0 conn(0x7f133c15f3a0 legacy=0x7f133c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 0x7f13280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328003f70 0x7f1328004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1328004d50 0x7f13280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f132800dc90 con 0x7f1328004d50 2024-09-19T04:15:35.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f132800de40 con 0x7f1328003f70 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f132800e040 con 0x7f133c1520a0 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1328004d50 0x7f13280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134095c640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328003f70 0x7f1328004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134115d640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 0x7f13280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 -- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 msgr2=0x7f13280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 0x7f13280039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 -- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328003f70 msgr2=0x7f1328004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328003f70 0x7f1328004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 -- 172.21.15.76:0/617513674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1328003530 con 0x7f1328004d50 2024-09-19T04:15:35.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134115d640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1520a0 0x7f13280039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1328004d50 0x7f13280051d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f133400a840 tx=0x7f133401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be91bc612989295a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133400a460 con 0x7f1328004d50 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13340ad650 con 0x7f1328004d50 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f1333fff640 1 -- 172.21.15.76:0/617513674 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340ada40 con 0x7f1328004d50 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f13433c1640 1 -- 172.21.15.76:0/617513674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1328004d50 msgr2=0x7f13280051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f13433c1640 1 --2- 172.21.15.76:0/617513674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1328004d50 0x7f13280051d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f133400a840 tx=0x7f133401d360 comp rx=0 tx=0).stop 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f134195e640 1 -- 172.21.15.76:0/617513674 reap_dead start 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f13433c1640 1 -- 172.21.15.76:0/617513674 shutdown_connections 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f13433c1640 1 -- 172.21.15.76:0/617513674 >> 172.21.15.76:0/617513674 conn(0x7f133c0b1f90 msgr2=0x7f133c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:35.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f13433c1640 1 -- 172.21.15.76:0/617513674 shutdown_connections 2024-09-19T04:15:35.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.969+0000 7f13433c1640 1 -- 172.21.15.76:0/617513674 wait complete. 2024-09-19T04:15:35.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 Processor -- start 2024-09-19T04:15:35.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 -- start start 2024-09-19T04:15:35.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328004d50 0x7f133c14a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c145aa0 0x7f133c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1494e0 0x7f133c15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f133c073770 con 0x7f133c145aa0 2024-09-19T04:15:35.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f133c073490 con 0x7f1328004d50 2024-09-19T04:15:35.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f133c073a50 con 0x7f133c1494e0 2024-09-19T04:15:35.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c145aa0 0x7f133c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c145aa0 0x7f133c0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35040/0 (socket says 172.21.15.76:35040) 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 -- 172.21.15.76:0/148126515 learned_addr learned my addr 172.21.15.76:0/148126515 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 -- 172.21.15.76:0/148126515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1494e0 msgr2=0x7f133c15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134195e640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1494e0 0x7f133c15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1494e0 0x7f133c15d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 -- 172.21.15.76:0/148126515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328004d50 msgr2=0x7f133c14a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328004d50 0x7f133c14a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 -- 172.21.15.76:0/148126515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c14ac20 con 0x7f133c145aa0 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f134095c640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c145aa0 0x7f133c0cccb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f133800e4b0 tx=0x7f133800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f1331ffb640 1 -- 172.21.15.76:0/148126515 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13380052e0 con 0x7f133c145aa0 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f1331ffb640 1 -- 172.21.15.76:0/148126515 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1338018070 con 0x7f133c145aa0 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c14af00 con 0x7f133c145aa0 2024-09-19T04:15:35.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f1331ffb640 1 -- 172.21.15.76:0/148126515 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1338003820 con 0x7f133c145aa0 2024-09-19T04:15:35.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c14b4c0 con 0x7f133c145aa0 2024-09-19T04:15:35.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.973+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133c073d30 con 0x7f133c145aa0 2024-09-19T04:15:35.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.977+0000 7f1331ffb640 1 -- 172.21.15.76:0/148126515 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f13380039c0 con 0x7f133c145aa0 2024-09-19T04:15:35.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.977+0000 7f1331ffb640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13200678f0 0x7f1320069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:35.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.977+0000 7f1331ffb640 1 -- 172.21.15.76:0/148126515 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f1338085ad0 con 0x7f133c145aa0 2024-09-19T04:15:35.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.977+0000 7f134115d640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13200678f0 0x7f1320069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:35.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.977+0000 7f134115d640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13200678f0 0x7f1320069db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f132c009a00 tx=0x7f132c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:35.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:35.981+0000 7f1331ffb640 1 -- 172.21.15.76:0/148126515 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1338056480 con 0x7f133c145aa0 2024-09-19T04:15:36.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.145+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f133c06e540 con 0x7f133c145aa0 2024-09-19T04:15:36.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.145+0000 7f1331ffb640 1 -- 172.21.15.76:0/148126515 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+17039 (secure 0 0 0) 0x7f1338005480 con 0x7f133c145aa0 2024-09-19T04:15:36.149 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:36.149 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":52,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:15:34.830913+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:15:32.810605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T04:15:33.811920+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13200678f0 msgr2=0x7f1320069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13200678f0 0x7f1320069db0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f132c009a00 tx=0x7f132c008040 comp rx=0 tx=0).stop 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c145aa0 msgr2=0x7f133c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c145aa0 0x7f133c0cccb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f133800e4b0 tx=0x7f133800e970 comp rx=0 tx=0).stop 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 shutdown_connections 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13200678f0 0x7f1320069db0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f133c1494e0 0x7f133c15d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f133c145aa0 0x7f133c0cccb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 --2- 172.21.15.76:0/148126515 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1328004d50 0x7f133c14a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 >> 172.21.15.76:0/148126515 conn(0x7f133c0b1f90 msgr2=0x7f133c0b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:36.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 shutdown_connections 2024-09-19T04:15:36.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.149+0000 7f13433c1640 1 -- 172.21.15.76:0/148126515 wait complete. 2024-09-19T04:15:36.168 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:15:36.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a9d608640 1 Processor -- start 2024-09-19T04:15:36.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a9d608640 1 -- start start 2024-09-19T04:15:36.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a9d608640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:36.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a9d608640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a980d2e90 0x7f6a980cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a9d608640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a980d32b0 con 0x7f6a980cf450 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a9d608640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6a980d0300 con 0x7f6a980d2e90 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a9d608640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9806e6c0 con 0x7f6a98161a60 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a977fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6a980cf450 0x7f6a980d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47270/0 (socket says 172.21.15.76:47270) 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a977fe640 1 -- 172.21.15.76:0/3375026560 learned_addr learned my addr 172.21.15.76:0/3375026560 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a967fc640 1 --2- 172.21.15.76:0/3375026560 >> v2:172.21.15.76:3300/0 conn(0x7f6a980d2e90 0x7f6a980cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a96ffd640 1 --2- 172.21.15.76:0/3375026560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a967fc640 1 -- 172.21.15.76:0/3375026560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 msgr2=0x7f6a980d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a967fc640 1 --2- 172.21.15.76:0/3375026560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a967fc640 1 -- 172.21.15.76:0/3375026560 >> v1:172.21.15.76:6789/0 conn(0x7f6a980cf450 legacy=0x7f6a980d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:36.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.289+0000 7f6a967fc640 1 -- 172.21.15.76:0/3375026560 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a980b0f60 con 0x7f6a980d2e90 2024-09-19T04:15:36.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a967fc640 1 --2- 172.21.15.76:0/3375026560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980cef10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097c0 tx=0x7f6a8801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781bda9978eb8051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:36.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a95ffb640 1 -- 172.21.15.76:0/3375026560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880b1070 con 0x7f6a980d2e90 2024-09-19T04:15:36.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a95ffb640 1 -- 172.21.15.76:0/3375026560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a880a8050 con 0x7f6a980d2e90 2024-09-19T04:15:36.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a95ffb640 1 -- 172.21.15.76:0/3375026560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880ac860 con 0x7f6a980d2e90 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- 172.21.15.76:0/3375026560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 msgr2=0x7f6a980cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 --2- 172.21.15.76:0/3375026560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980cef10 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6a880097c0 tx=0x7f6a8801c3e0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- 172.21.15.76:0/3375026560 shutdown_connections 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 --2- 172.21.15.76:0/3375026560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980cef10 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 --2- 172.21.15.76:0/3375026560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- 172.21.15.76:0/3375026560 >> 172.21.15.76:0/3375026560 conn(0x7f6a9808bf90 msgr2=0x7f6a9809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- 172.21.15.76:0/3375026560 shutdown_connections 2024-09-19T04:15:36.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- 172.21.15.76:0/3375026560 wait complete. 2024-09-19T04:15:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 Processor -- start 2024-09-19T04:15:36.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- start start 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a980cf450 0x7f6a980cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980cb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9806d190 con 0x7f6a980d2e90 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a980d1930 con 0x7f6a980cf450 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a9d608640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6a980cf880 con 0x7f6a98161a60 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.293+0000 7f6a967fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35054/0 (socket says 172.21.15.76:35054) 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 -- 172.21.15.76:0/2712644121 learned_addr learned my addr 172.21.15.76:0/2712644121 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 -- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 msgr2=0x7f6a980cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a977fe640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:36.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a96ffd640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a980cf450 0x7f6a980cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 -- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a980cf450 msgr2=0x7f6a980cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a980cf450 0x7f6a980cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 -- 172.21.15.76:0/2712644121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a980b1c60 con 0x7f6a980d2e90 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a967fc640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980b12f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6a8801a010 tx=0x7f6a88019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a7f7fe640 1 -- 172.21.15.76:0/2712644121 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880b1040 con 0x7f6a980d2e90 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a7f7fe640 1 -- 172.21.15.76:0/2712644121 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a880ba070 con 0x7f6a980d2e90 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a7f7fe640 1 -- 172.21.15.76:0/2712644121 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a880b5870 con 0x7f6a980d2e90 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a980b1f40 con 0x7f6a980d2e90 2024-09-19T04:15:36.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a980b2500 con 0x7f6a980d2e90 2024-09-19T04:15:36.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a980cef10 con 0x7f6a980d2e90 2024-09-19T04:15:36.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a7f7fe640 1 -- 172.21.15.76:0/2712644121 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a880b5a10 con 0x7f6a980d2e90 2024-09-19T04:15:36.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a7f7fe640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a840678c0 0x7f6a84069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:36.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a7f7fe640 1 -- 172.21.15.76:0/2712644121 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6101+0+0 (secure 0 0 0) 0x7f6a88120b80 con 0x7f6a980d2e90 2024-09-19T04:15:36.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a96ffd640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a840678c0 0x7f6a84069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:36.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.297+0000 7f6a96ffd640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a840678c0 0x7f6a84069d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a980cd450 tx=0x7f6a8c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:36.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.301+0000 7f6a7f7fe640 1 -- 172.21.15.76:0/2712644121 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6a88016090 con 0x7f6a980d2e90 2024-09-19T04:15:36.443 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:15:36.441+0000 7fccdf431640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:15:36.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:36.501+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6a98075630 con 0x7f6a980d2e90 2024-09-19T04:15:37.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.469+0000 7f6a7f7fe640 1 -- 172.21.15.76:0/2712644121 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) ==== 118+0+0 (secure 0 0 0) 0x7f6a880f16a0 con 0x7f6a980d2e90 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a840678c0 msgr2=0x7f6a84069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a840678c0 0x7f6a84069d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a980cd450 tx=0x7f6a8c008040 comp rx=0 tx=0).stop 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 msgr2=0x7f6a980b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980b12f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6a8801a010 tx=0x7f6a88019180 comp rx=0 tx=0).stop 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 shutdown_connections 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a840678c0 0x7f6a84069d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a98161a60 0x7f6a980cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a980d2e90 0x7f6a980b12f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 --2- 172.21.15.76:0/2712644121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a980cf450 0x7f6a980cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 >> 172.21.15.76:0/2712644121 conn(0x7f6a9808bf90 msgr2=0x7f6a980d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 shutdown_connections 2024-09-19T04:15:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.473+0000 7f6a9d608640 1 -- 172.21.15.76:0/2712644121 wait complete. 2024-09-19T04:15:37.499 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T04:15:37.499 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T04:15:37.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 Processor -- start 2024-09-19T04:15:37.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 -- start start 2024-09-19T04:15:37.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe484161a60 0x7fe4840d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840d2e90 0x7fe4840cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe4840cf450 0x7fe4840d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe4840d32b0 con 0x7fe484161a60 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe4840d0300 con 0x7fe4840cf450 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48c0ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe4840b0f60 con 0x7fe4840d2e90 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe484161a60 0x7fe4840d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe484161a60 0x7fe4840d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35060/0 (socket says 172.21.15.76:35060) 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 -- 172.21.15.76:0/404891220 learned_addr learned my addr 172.21.15.76:0/404891220 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe48a68b640 1 --2- 172.21.15.76:0/404891220 >> v2:172.21.15.116:3300/0 conn(0x7fe4840cf450 0x7fe4840d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:37.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 -- 172.21.15.76:0/404891220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840d2e90 msgr2=0x7fe4840cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:37.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489689640 1 --2- 172.21.15.76:0/404891220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840d2e90 0x7fe4840cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:37.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 --2- 172.21.15.76:0/404891220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840d2e90 0x7fe4840cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 -- 172.21.15.76:0/404891220 >> v2:172.21.15.116:3300/0 conn(0x7fe4840cf450 msgr2=0x7fe4840d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:37.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 --2- 172.21.15.76:0/404891220 >> v2:172.21.15.116:3300/0 conn(0x7fe4840cf450 0x7fe4840d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489e8a640 1 -- 172.21.15.76:0/404891220 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4840cfd40 con 0x7fe484161a60 2024-09-19T04:15:37.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.625+0000 7fe489689640 1 --2- 172.21.15.76:0/404891220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840d2e90 0x7fe4840cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:37.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489e8a640 1 --2- 172.21.15.76:0/404891220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe484161a60 0x7fe4840d2950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4840d23a0 tx=0x7fe48001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e77ec8fece3a85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:37.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe488e88640 1 -- 172.21.15.76:0/404891220 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4800a57b0 con 0x7fe484161a60 2024-09-19T04:15:37.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe488e88640 1 -- 172.21.15.76:0/404891220 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4800a5950 con 0x7fe484161a60 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe488e88640 1 -- 172.21.15.76:0/404891220 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4800a5d20 con 0x7fe484161a60 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- 172.21.15.76:0/404891220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe484161a60 msgr2=0x7fe4840d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/404891220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe484161a60 0x7fe4840d2950 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fe4840d23a0 tx=0x7fe48001c930 comp rx=0 tx=0).stop 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- 172.21.15.76:0/404891220 shutdown_connections 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/404891220 >> v2:172.21.15.116:3300/0 conn(0x7fe4840cf450 0x7fe4840d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/404891220 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840d2e90 0x7fe4840cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/404891220 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe484161a60 0x7fe4840d2950 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- 172.21.15.76:0/404891220 >> 172.21.15.76:0/404891220 conn(0x7fe48408bf90 msgr2=0x7fe48409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- 172.21.15.76:0/404891220 shutdown_connections 2024-09-19T04:15:37.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- 172.21.15.76:0/404891220 wait complete. 2024-09-19T04:15:37.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 Processor -- start 2024-09-19T04:15:37.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- start start 2024-09-19T04:15:37.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840cf450 0x7fe4840b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:37.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4840d2e90 0x7fe48415dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:37.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe484161a60 0x7fe4840cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:37.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe48406d830 con 0x7fe4840d2e90 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe48406d6b0 con 0x7fe484161a60 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe48406d9b0 con 0x7fe4840cf450 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4840d2e90 0x7fe48415dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4840d2e90 0x7fe48415dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35076/0 (socket says 172.21.15.76:35076) 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 -- 172.21.15.76:0/4217686822 learned_addr learned my addr 172.21.15.76:0/4217686822 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489e8a640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840cf450 0x7fe4840b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48a68b640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe484161a60 0x7fe4840cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 -- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840cf450 msgr2=0x7fe4840b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840cf450 0x7fe4840b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 -- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe484161a60 msgr2=0x7fe4840cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe484161a60 0x7fe4840cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 -- 172.21.15.76:0/4217686822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4840ba210 con 0x7fe4840d2e90 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48a68b640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe484161a60 0x7fe4840cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe489689640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4840d2e90 0x7fe48415dc10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe47400c7c0 tx=0x7fe47400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe47a7fc640 1 -- 172.21.15.76:0/4217686822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe474007a30 con 0x7fe4840d2e90 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe47a7fc640 1 -- 172.21.15.76:0/4217686822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe474007bd0 con 0x7fe4840d2e90 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4840b9550 con 0x7fe4840d2e90 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.629+0000 7fe47a7fc640 1 -- 172.21.15.76:0/4217686822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4740113f0 con 0x7fe4840d2e90 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.633+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4840c8200 con 0x7fe4840d2e90 2024-09-19T04:15:37.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.633+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4840cef10 con 0x7fe4840d2e90 2024-09-19T04:15:37.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.633+0000 7fe47a7fc640 1 -- 172.21.15.76:0/4217686822 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe474011590 con 0x7fe4840d2e90 2024-09-19T04:15:37.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.633+0000 7fe47a7fc640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe4680678c0 0x7fe468069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:37.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.633+0000 7fe47a7fc640 1 -- 172.21.15.76:0/4217686822 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fe474086400 con 0x7fe4840d2e90 2024-09-19T04:15:37.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.633+0000 7fe489e8a640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe4680678c0 0x7fe468069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:37.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.633+0000 7fe489e8a640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe4680678c0 0x7fe468069d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe4800ae000 tx=0x7fe4800a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:37.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.637+0000 7fe47a7fc640 1 -- 172.21.15.76:0/4217686822 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe474015020 con 0x7fe4840d2e90 2024-09-19T04:15:37.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:37.837+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe4840b34c0 con 0x7fe4840d2e90 2024-09-19T04:15:38.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.493+0000 7fe47a7fc640 1 -- 172.21.15.76:0/4217686822 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v29) ==== 122+0+0 (secure 0 0 0) 0x7fe474056e90 con 0x7fe4840d2e90 2024-09-19T04:15:38.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe4680678c0 msgr2=0x7fe468069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe4680678c0 0x7fe468069d80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe4800ae000 tx=0x7fe4800a7070 comp rx=0 tx=0).stop 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4840d2e90 msgr2=0x7fe48415dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4840d2e90 0x7fe48415dc10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fe47400c7c0 tx=0x7fe47400cc80 comp rx=0 tx=0).stop 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 shutdown_connections 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe4680678c0 0x7fe468069d80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe484161a60 0x7fe4840cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe4840d2e90 0x7fe48415dc10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 --2- 172.21.15.76:0/4217686822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe4840cf450 0x7fe4840b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 >> 172.21.15.76:0/4217686822 conn(0x7fe48408bf90 msgr2=0x7fe4840b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 shutdown_connections 2024-09-19T04:15:38.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.497+0000 7fe48c0ee640 1 -- 172.21.15.76:0/4217686822 wait complete. 2024-09-19T04:15:38.521 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T04:15:38.521 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:38.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.641+0000 7fc195f65640 1 Processor -- start 2024-09-19T04:15:38.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.641+0000 7fc195f65640 1 -- start start 2024-09-19T04:15:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.641+0000 7fc195f65640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1901587a0 0x7fc1900b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.641+0000 7fc195f65640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc19006e680 0x7fc190072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.641+0000 7fc195f65640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc19006ec60 con 0x7fc1900b37d0 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.641+0000 7fc195f65640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc19006f870 con 0x7fc19006e680 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.641+0000 7fc195f65640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900b3c10 con 0x7fc1901587a0 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc19006e680 0x7fc190072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc19006e680 0x7fc190072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35090/0 (socket says 172.21.15.76:35090) 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 -- 172.21.15.76:0/3864059972 learned_addr learned my addr 172.21.15.76:0/3864059972 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18f7fe640 1 --2- 172.21.15.76:0/3864059972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1901587a0 0x7fc1900b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 -- 172.21.15.76:0/3864059972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1901587a0 msgr2=0x7fc1900b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 --2- 172.21.15.76:0/3864059972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1901587a0 0x7fc1900b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 -- 172.21.15.76:0/3864059972 >> v1:172.21.15.116:6789/0 conn(0x7fc1900b37d0 legacy=0x7fc19006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 -- 172.21.15.76:0/3864059972 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19006ef40 con 0x7fc19006e680 2024-09-19T04:15:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18ffff640 1 --2- 172.21.15.76:0/3864059972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc19006e680 0x7fc190072160 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc180009870 tx=0x7fc18001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5694454de65451f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18e7fc640 1 -- 172.21.15.76:0/3864059972 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800a95f0 con 0x7fc19006e680 2024-09-19T04:15:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18e7fc640 1 -- 172.21.15.76:0/3864059972 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1800a9790 con 0x7fc19006e680 2024-09-19T04:15:38.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc18e7fc640 1 -- 172.21.15.76:0/3864059972 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800a9b60 con 0x7fc19006e680 2024-09-19T04:15:38.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 -- 172.21.15.76:0/3864059972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc19006e680 msgr2=0x7fc190072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 --2- 172.21.15.76:0/3864059972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc19006e680 0x7fc190072160 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc180009870 tx=0x7fc18001c530 comp rx=0 tx=0).stop 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 -- 172.21.15.76:0/3864059972 shutdown_connections 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 --2- 172.21.15.76:0/3864059972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc19006e680 0x7fc190072160 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 --2- 172.21.15.76:0/3864059972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1901587a0 0x7fc1900b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 -- 172.21.15.76:0/3864059972 >> 172.21.15.76:0/3864059972 conn(0x7fc1900b1f90 msgr2=0x7fc1900b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 -- 172.21.15.76:0/3864059972 shutdown_connections 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.645+0000 7fc195f65640 1 -- 172.21.15.76:0/3864059972 wait complete. 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 Processor -- start 2024-09-19T04:15:38.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 -- start start 2024-09-19T04:15:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19006e680 0x7fc190150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1900b37d0 0x7fc19009a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1901587a0 0x7fc1901533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc190073030 con 0x7fc1901587a0 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc190072eb0 con 0x7fc19006e680 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc1900731b0 con 0x7fc1900b37d0 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1901587a0 0x7fc1901533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1901587a0 0x7fc1901533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35098/0 (socket says 172.21.15.76:35098) 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 -- 172.21.15.76:0/121820748 learned_addr learned my addr 172.21.15.76:0/121820748 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18f7fe640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19006e680 0x7fc190150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18effd640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1900b37d0 0x7fc19009a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 -- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1900b37d0 msgr2=0x7fc19009a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1900b37d0 0x7fc19009a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 -- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19006e680 msgr2=0x7fc190150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19006e680 0x7fc190150030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 -- 172.21.15.76:0/121820748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc190151080 con 0x7fc1901587a0 2024-09-19T04:15:38.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18f7fe640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19006e680 0x7fc190150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc18ffff640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1901587a0 0x7fc1901533d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc18001a160 tx=0x7fc18001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc187fff640 1 -- 172.21.15.76:0/121820748 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800b1800 con 0x7fc1901587a0 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc187fff640 1 -- 172.21.15.76:0/121820748 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1800b19a0 con 0x7fc1901587a0 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc190151380 con 0x7fc1901587a0 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc187fff640 1 -- 172.21.15.76:0/121820748 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1800b1d50 con 0x7fc1901587a0 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc19009a8c0 con 0x7fc1901587a0 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.649+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc190072160 con 0x7fc1901587a0 2024-09-19T04:15:38.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.653+0000 7fc187fff640 1 -- 172.21.15.76:0/121820748 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc1800c53f0 con 0x7fc1901587a0 2024-09-19T04:15:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.653+0000 7fc187fff640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc1700678c0 0x7fc170069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.653+0000 7fc187fff640 1 -- 172.21.15.76:0/121820748 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc180120890 con 0x7fc1901587a0 2024-09-19T04:15:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.653+0000 7fc18f7fe640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc1700678c0 0x7fc170069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.653+0000 7fc18f7fe640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc1700678c0 0x7fc170069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc188009fd0 tx=0x7fc18800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:38.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.657+0000 7fc187fff640 1 -- 172.21.15.76:0/121820748 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc190151380 con 0x7fc1901587a0 2024-09-19T04:15:38.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc190070080 con 0x7fc1901587a0 2024-09-19T04:15:38.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc187fff640 1 -- 172.21.15.76:0/121820748 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7fc1800ad020 con 0x7fc1901587a0 2024-09-19T04:15:38.851 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:38.851 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"btime":"2024-09-19T04:15:38:482457+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:15:37.839123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502]},"id":3}]} 2024-09-19T04:15:38.851 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2024-09-19T04:15:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc1700678c0 msgr2=0x7fc170069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc1700678c0 0x7fc170069d80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc188009fd0 tx=0x7fc18800b040 comp rx=0 tx=0).stop 2024-09-19T04:15:38.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1901587a0 msgr2=0x7fc1901533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1901587a0 0x7fc1901533d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc18001a160 tx=0x7fc18001ad70 comp rx=0 tx=0).stop 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 shutdown_connections 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc1700678c0 0x7fc170069d80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc1901587a0 0x7fc1901533d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc1900b37d0 0x7fc19009a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 --2- 172.21.15.76:0/121820748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19006e680 0x7fc190150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 >> 172.21.15.76:0/121820748 conn(0x7fc1900b1f90 msgr2=0x7fc1900ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 shutdown_connections 2024-09-19T04:15:38.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.849+0000 7fc195f65640 1 -- 172.21.15.76:0/121820748 wait complete. 2024-09-19T04:15:38.869 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:15:38.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.977+0000 7f939be49640 1 Processor -- start 2024-09-19T04:15:38.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.977+0000 7f939be49640 1 -- start start 2024-09-19T04:15:38.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f939be49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f93940ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f939be49640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f939415c2a0 0x7f93940aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f939be49640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93940b3290 con 0x7f93940af2a0 2024-09-19T04:15:38.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f939be49640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f93940b3570 con 0x7f939415c2a0 2024-09-19T04:15:38.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f939be49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f93940b3850 con 0x7f939415bec0 2024-09-19T04:15:38.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f93940ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f93993e4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f939415c2a0 0x7f93940aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f93940ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:50098/0 (socket says 172.21.15.76:50098) 2024-09-19T04:15:38.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 -- 172.21.15.76:0/2668483338 learned_addr learned my addr 172.21.15.76:0/2668483338 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:38.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 -- 172.21.15.76:0/2668483338 >> v2:172.21.15.116:3300/0 conn(0x7f939415c2a0 msgr2=0x7f93940aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:38.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 --2- 172.21.15.76:0/2668483338 >> v2:172.21.15.116:3300/0 conn(0x7f939415c2a0 0x7f93940aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 -- 172.21.15.76:0/2668483338 >> v1:172.21.15.116:6789/0 conn(0x7f93940af2a0 legacy=0x7f939415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 -- 172.21.15.76:0/2668483338 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93940b3b30 con 0x7f939415bec0 2024-09-19T04:15:38.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9399be5640 1 --2- 172.21.15.76:0/2668483338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f93940ad200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9390009870 tx=0x7f939001c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aee40ce240861fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:38.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9398be3640 1 -- 172.21.15.76:0/2668483338 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93900a93f0 con 0x7f939415bec0 2024-09-19T04:15:38.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9398be3640 1 -- 172.21.15.76:0/2668483338 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93900a9590 con 0x7f939415bec0 2024-09-19T04:15:38.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.981+0000 7f9398be3640 1 -- 172.21.15.76:0/2668483338 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93900a9960 con 0x7f939415bec0 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- 172.21.15.76:0/2668483338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 msgr2=0x7f93940ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 --2- 172.21.15.76:0/2668483338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f93940ad200 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9390009870 tx=0x7f939001c530 comp rx=0 tx=0).stop 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- 172.21.15.76:0/2668483338 shutdown_connections 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 --2- 172.21.15.76:0/2668483338 >> v2:172.21.15.116:3300/0 conn(0x7f939415c2a0 0x7f93940aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 --2- 172.21.15.76:0/2668483338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f93940ad200 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- 172.21.15.76:0/2668483338 >> 172.21.15.76:0/2668483338 conn(0x7f93940b1f90 msgr2=0x7f93940b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- 172.21.15.76:0/2668483338 shutdown_connections 2024-09-19T04:15:38.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- 172.21.15.76:0/2668483338 wait complete. 2024-09-19T04:15:38.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 Processor -- start 2024-09-19T04:15:38.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- start start 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93940af2a0 0x7f93941498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f9394147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f939415c2a0 0x7f9394151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f93940af6b0 con 0x7f93940af2a0 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f93940ad740 con 0x7f939415c2a0 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9394166ff0 con 0x7f939415bec0 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93940af2a0 0x7f93941498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93940af2a0 0x7f93941498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35106/0 (socket says 172.21.15.76:35106) 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 -- 172.21.15.76:0/2249463189 learned_addr learned my addr 172.21.15.76:0/2249463189 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f93993e4640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f9394147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939a3e6640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f939415c2a0 0x7f9394151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 -- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 msgr2=0x7f9394147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f9394147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 -- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f939415c2a0 msgr2=0x7f9394151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f939415c2a0 0x7f9394151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 -- 172.21.15.76:0/2249463189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93941496e0 con 0x7f93940af2a0 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f9399be5640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93940af2a0 0x7f93941498f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f93900ab040 tx=0x7f939001ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f938a7fc640 1 -- 172.21.15.76:0/2249463189 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93900a93f0 con 0x7f93940af2a0 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f938a7fc640 1 -- 172.21.15.76:0/2249463189 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93900b1800 con 0x7f93940af2a0 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941504d0 con 0x7f93940af2a0 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f938a7fc640 1 -- 172.21.15.76:0/2249463189 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93900b1bf0 con 0x7f93940af2a0 2024-09-19T04:15:38.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.985+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939414d590 con 0x7f93940af2a0 2024-09-19T04:15:38.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.989+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93940ad200 con 0x7f93940af2a0 2024-09-19T04:15:38.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.989+0000 7f938a7fc640 1 -- 172.21.15.76:0/2249463189 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f93900bc450 con 0x7f93940af2a0 2024-09-19T04:15:38.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.989+0000 7f938a7fc640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f93780678c0 0x7f9378069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:38.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.989+0000 7f93993e4640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f93780678c0 0x7f9378069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:38.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.989+0000 7f938a7fc640 1 -- 172.21.15.76:0/2249463189 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9390120870 con 0x7f93940af2a0 2024-09-19T04:15:38.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.989+0000 7f93993e4640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f93780678c0 0x7f9378069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f939415c8b0 tx=0x7f938400f040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:38.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:38.993+0000 7f938a7fc640 1 -- 172.21.15.76:0/2249463189 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f93900ad020 con 0x7f93940af2a0 2024-09-19T04:15:39.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.153+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93940b2c60 con 0x7f93940af2a0 2024-09-19T04:15:39.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f938a7fc640 1 -- 172.21.15.76:0/2249463189 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+17026 (secure 0 0 0) 0x7f9390016150 con 0x7f93940af2a0 2024-09-19T04:15:39.159 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:39.159 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":53,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:15:36.458357+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:15:32.810605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T04:15:33.811920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:15:39.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f93780678c0 msgr2=0x7f9378069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f93780678c0 0x7f9378069d80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f939415c8b0 tx=0x7f938400f040 comp rx=0 tx=0).stop 2024-09-19T04:15:39.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93940af2a0 msgr2=0x7f93941498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93940af2a0 0x7f93941498f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f93900ab040 tx=0x7f939001ad70 comp rx=0 tx=0).stop 2024-09-19T04:15:39.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 shutdown_connections 2024-09-19T04:15:39.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f93780678c0 0x7f9378069d80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f939415c2a0 0x7f9394151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f939415bec0 0x7f9394147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 --2- 172.21.15.76:0/2249463189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f93940af2a0 0x7f93941498f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 >> 172.21.15.76:0/2249463189 conn(0x7f93940b1f90 msgr2=0x7f9394159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:39.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 shutdown_connections 2024-09-19T04:15:39.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.157+0000 7f939be49640 1 -- 172.21.15.76:0/2249463189 wait complete. 2024-09-19T04:15:39.177 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:15:39.178 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:15:39.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0f80b640 1 Processor -- start 2024-09-19T04:15:39.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0f80b640 1 -- start start 2024-09-19T04:15:39.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0f80b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fce08161a60 0x7fce080d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0f80b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080cf620 0x7fce0806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0f80b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce080d32b0 con 0x7fce080d2e90 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0f80b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fce080d0300 con 0x7fce08161a60 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0f80b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fce080cfab0 con 0x7fce080cf620 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0dda8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080cf620 0x7fce0806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0d5a7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fce08161a60 0x7fce080d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0d5a7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fce08161a60 0x7fce080d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:54108/0 (socket says 172.21.15.76:54108) 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1178813955 learned_addr learned my addr 172.21.15.76:0/1178813955 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:39.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.281+0000 7fce0cda6640 1 --1- 172.21.15.76:0/1178813955 >> v1:172.21.15.116:6789/0 conn(0x7fce080d2e90 0x7fce080cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:59996/0 (socket says 172.21.15.76:59996) 2024-09-19T04:15:39.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1178813955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080cf620 msgr2=0x7fce0806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:39.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fce0d5a7640 1 --2- 172.21.15.76:0/1178813955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080cf620 0x7fce0806d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1178813955 >> v1:172.21.15.116:6789/0 conn(0x7fce080d2e90 legacy=0x7fce080cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1178813955 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce080b0f60 con 0x7fce08161a60 2024-09-19T04:15:39.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fce0d5a7640 1 --2- 172.21.15.76:0/1178813955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 0x7fce080d2950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fce04009870 tx=0x7fce0401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e734fc19f255ecd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:39.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fcdfffff640 1 -- 172.21.15.76:0/1178813955 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040aa600 con 0x7fce08161a60 2024-09-19T04:15:39.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fcdfffff640 1 -- 172.21.15.76:0/1178813955 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce040af070 con 0x7fce08161a60 2024-09-19T04:15:39.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.285+0000 7fcdfffff640 1 -- 172.21.15.76:0/1178813955 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040aa9c0 con 0x7fce08161a60 2024-09-19T04:15:39.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1178813955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 msgr2=0x7fce080d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1178813955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 0x7fce080d2950 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fce04009870 tx=0x7fce0401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1178813955 shutdown_connections 2024-09-19T04:15:39.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1178813955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080cf620 0x7fce0806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1178813955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 0x7fce080d2950 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1178813955 >> 172.21.15.76:0/1178813955 conn(0x7fce0808bf90 msgr2=0x7fce0809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:39.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1178813955 shutdown_connections 2024-09-19T04:15:39.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1178813955 wait complete. 2024-09-19T04:15:39.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 Processor -- start 2024-09-19T04:15:39.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- start start 2024-09-19T04:15:39.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce080cf620 0x7fce080cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080d2e90 0x7fce08154a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 0x7fce080cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fce0806e400 con 0x7fce080cf620 2024-09-19T04:15:39.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce0806d190 con 0x7fce08161a60 2024-09-19T04:15:39.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fce0806e580 con 0x7fce080d2e90 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce080cf620 0x7fce080cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce080cf620 0x7fce080cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35110/0 (socket says 172.21.15.76:35110) 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1593743121 learned_addr learned my addr 172.21.15.76:0/1593743121 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0cda6640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080d2e90 0x7fce08154a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0dda8640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 0x7fce080cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080d2e90 msgr2=0x7fce08154a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080d2e90 0x7fce08154a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 msgr2=0x7fce080cbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 0x7fce080cbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 -- 172.21.15.76:0/1593743121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce080cda50 con 0x7fce080cf620 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0d5a7640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce080cf620 0x7fce080cb650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fce040ac040 tx=0x7fce0401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fcdfdffb640 1 -- 172.21.15.76:0/1593743121 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040bd5f0 con 0x7fce080cf620 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fcdfdffb640 1 -- 172.21.15.76:0/1593743121 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce040bd790 con 0x7fce080cf620 2024-09-19T04:15:39.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce080cdcf0 con 0x7fce080cf620 2024-09-19T04:15:39.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fcdfdffb640 1 -- 172.21.15.76:0/1593743121 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce040bdb80 con 0x7fce080cf620 2024-09-19T04:15:39.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce08157fb0 con 0x7fce080cf620 2024-09-19T04:15:39.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.289+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce0806d6b0 con 0x7fce080cf620 2024-09-19T04:15:39.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.293+0000 7fcdfdffb640 1 -- 172.21.15.76:0/1593743121 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce040a9070 con 0x7fce080cf620 2024-09-19T04:15:39.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.293+0000 7fcdfdffb640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdec0678f0 0x7fcdec069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.293+0000 7fcdfdffb640 1 -- 172.21.15.76:0/1593743121 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fce04120970 con 0x7fce080cf620 2024-09-19T04:15:39.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.293+0000 7fce0cda6640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdec0678f0 0x7fcdec069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.293+0000 7fce0cda6640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdec0678f0 0x7fcdec069db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce0815faa0 tx=0x7fcdf8007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:39.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.297+0000 7fcdfdffb640 1 -- 172.21.15.76:0/1593743121 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fce040c3060 con 0x7fce080cf620 2024-09-19T04:15:39.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.453+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce080ceed0 con 0x7fcdec0678f0 2024-09-19T04:15:39.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.461+0000 7fcdfdffb640 1 -- 172.21.15.76:0/1593743121 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239061 (secure 0 0 0) 0x7fce080ceed0 con 0x7fcdec0678f0 2024-09-19T04:15:39.469 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:15:39.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdec0678f0 msgr2=0x7fcdec069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdec0678f0 0x7fcdec069db0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fce0815faa0 tx=0x7fcdf8007cb0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce080cf620 msgr2=0x7fce080cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce080cf620 0x7fce080cb650 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fce040ac040 tx=0x7fce0401ace0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 shutdown_connections 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdec0678f0 0x7fcdec069db0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce08161a60 0x7fce080cbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce080d2e90 0x7fce08154a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 --2- 172.21.15.76:0/1593743121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce080cf620 0x7fce080cb650 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 >> 172.21.15.76:0/1593743121 conn(0x7fce0808bf90 msgr2=0x7fce080d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:39.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 shutdown_connections 2024-09-19T04:15:39.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.469+0000 7fce0f80b640 1 -- 172.21.15.76:0/1593743121 wait complete. 2024-09-19T04:15:39.488 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-19T04:15:37.781368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591020,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":167,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":3,"kb":754974720,"kb_used":230236,"kb_used_data":5668,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744484,"statfs":{"total":773094113280,"available":772858351616,"internally_reserved":0,"allocated":5804032,"data_stored":3213324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":652,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.329254"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668643+0000","last_change":"2024-09-19T04:15:35.523138+0000","last_active":"2024-09-19T04:15:36.668643+0000","last_peered":"2024-09-19T04:15:36.668643+0000","last_clean":"2024-09-19T04:15:36.668643+0000","last_became_active":"2024-09-19T04:15:35.522403+0000","last_became_peered":"2024-09-19T04:15:35.522403+0000","last_unstale":"2024-09-19T04:15:36.668643+0000","last_undegraded":"2024-09-19T04:15:36.668643+0000","last_fullsized":"2024-09-19T04:15:36.668643+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:57:21.740917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668482+0000","last_change":"2024-09-19T04:15:35.515485+0000","last_active":"2024-09-19T04:15:36.668482+0000","last_peered":"2024-09-19T04:15:36.668482+0000","last_clean":"2024-09-19T04:15:36.668482+0000","last_became_active":"2024-09-19T04:15:35.509562+0000","last_became_peered":"2024-09-19T04:15:35.509562+0000","last_unstale":"2024-09-19T04:15:36.668482+0000","last_undegraded":"2024-09-19T04:15:36.668482+0000","last_fullsized":"2024-09-19T04:15:36.668482+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:31.328898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668366+0000","last_change":"2024-09-19T04:15:35.502323+0000","last_active":"2024-09-19T04:15:36.668366+0000","last_peered":"2024-09-19T04:15:36.668366+0000","last_clean":"2024-09-19T04:15:36.668366+0000","last_became_active":"2024-09-19T04:15:35.501535+0000","last_became_peered":"2024-09-19T04:15:35.501535+0000","last_unstale":"2024-09-19T04:15:36.668366+0000","last_undegraded":"2024-09-19T04:15:36.668366+0000","last_fullsized":"2024-09-19T04:15:36.668366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:43.709884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668459+0000","last_change":"2024-09-19T04:15:34.495140+0000","last_active":"2024-09-19T04:15:36.668459+0000","last_peered":"2024-09-19T04:15:36.668459+0000","last_clean":"2024-09-19T04:15:36.668459+0000","last_became_active":"2024-09-19T04:15:34.494396+0000","last_became_peered":"2024-09-19T04:15:34.494396+0000","last_unstale":"2024-09-19T04:15:36.668459+0000","last_undegraded":"2024-09-19T04:15:36.668459+0000","last_fullsized":"2024-09-19T04:15:36.668459+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:40.535985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668098+0000","last_change":"2024-09-19T04:15:35.509583+0000","last_active":"2024-09-19T04:15:36.668098+0000","last_peered":"2024-09-19T04:15:36.668098+0000","last_clean":"2024-09-19T04:15:36.668098+0000","last_became_active":"2024-09-19T04:15:35.509008+0000","last_became_peered":"2024-09-19T04:15:35.509008+0000","last_unstale":"2024-09-19T04:15:36.668098+0000","last_undegraded":"2024-09-19T04:15:36.668098+0000","last_fullsized":"2024-09-19T04:15:36.668098+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:38.558469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668469+0000","last_change":"2024-09-19T04:15:34.500494+0000","last_active":"2024-09-19T04:15:36.668469+0000","last_peered":"2024-09-19T04:15:36.668469+0000","last_clean":"2024-09-19T04:15:36.668469+0000","last_became_active":"2024-09-19T04:15:34.499595+0000","last_became_peered":"2024-09-19T04:15:34.499595+0000","last_unstale":"2024-09-19T04:15:36.668469+0000","last_undegraded":"2024-09-19T04:15:36.668469+0000","last_fullsized":"2024-09-19T04:15:36.668469+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.803393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668265+0000","last_change":"2024-09-19T04:15:34.500317+0000","last_active":"2024-09-19T04:15:36.668265+0000","last_peered":"2024-09-19T04:15:36.668265+0000","last_clean":"2024-09-19T04:15:36.668265+0000","last_became_active":"2024-09-19T04:15:34.499498+0000","last_became_peered":"2024-09-19T04:15:34.499498+0000","last_unstale":"2024-09-19T04:15:36.668265+0000","last_undegraded":"2024-09-19T04:15:36.668265+0000","last_fullsized":"2024-09-19T04:15:36.668265+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:13.930894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668489+0000","last_change":"2024-09-19T04:15:34.500954+0000","last_active":"2024-09-19T04:15:36.668489+0000","last_peered":"2024-09-19T04:15:36.668489+0000","last_clean":"2024-09-19T04:15:36.668489+0000","last_became_active":"2024-09-19T04:15:34.500170+0000","last_became_peered":"2024-09-19T04:15:34.500170+0000","last_unstale":"2024-09-19T04:15:36.668489+0000","last_undegraded":"2024-09-19T04:15:36.668489+0000","last_fullsized":"2024-09-19T04:15:36.668489+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:36.874338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668638+0000","last_change":"2024-09-19T04:15:34.503068+0000","last_active":"2024-09-19T04:15:36.668638+0000","last_peered":"2024-09-19T04:15:36.668638+0000","last_clean":"2024-09-19T04:15:36.668638+0000","last_became_active":"2024-09-19T04:15:34.502332+0000","last_became_peered":"2024-09-19T04:15:34.502332+0000","last_unstale":"2024-09-19T04:15:36.668638+0000","last_undegraded":"2024-09-19T04:15:36.668638+0000","last_fullsized":"2024-09-19T04:15:36.668638+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:01.529754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668600+0000","last_change":"2024-09-19T04:15:35.523389+0000","last_active":"2024-09-19T04:15:36.668600+0000","last_peered":"2024-09-19T04:15:36.668600+0000","last_clean":"2024-09-19T04:15:36.668600+0000","last_became_active":"2024-09-19T04:15:35.522199+0000","last_became_peered":"2024-09-19T04:15:35.522199+0000","last_unstale":"2024-09-19T04:15:36.668600+0000","last_undegraded":"2024-09-19T04:15:36.668600+0000","last_fullsized":"2024-09-19T04:15:36.668600+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:28.365203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668618+0000","last_change":"2024-09-19T04:15:34.502107+0000","last_active":"2024-09-19T04:15:36.668618+0000","last_peered":"2024-09-19T04:15:36.668618+0000","last_clean":"2024-09-19T04:15:36.668618+0000","last_became_active":"2024-09-19T04:15:34.501387+0000","last_became_peered":"2024-09-19T04:15:34.501387+0000","last_unstale":"2024-09-19T04:15:36.668618+0000","last_undegraded":"2024-09-19T04:15:36.668618+0000","last_fullsized":"2024-09-19T04:15:36.668618+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:06.890512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:34.451892+0000","last_change":"2024-09-19T04:15:34.451892+0000","last_active":"2024-09-19T04:15:34.451892+0000","last_peered":"2024-09-19T04:15:34.451892+0000","last_clean":"2024-09-19T04:15:34.451892+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:34.451892+0000","last_undegraded":"2024-09-19T04:15:34.451892+0000","last_fullsized":"2024-09-19T04:15:34.451892+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668156+0000","last_change":"2024-09-19T04:15:35.514179+0000","last_active":"2024-09-19T04:15:36.668156+0000","last_peered":"2024-09-19T04:15:36.668156+0000","last_clean":"2024-09-19T04:15:36.668156+0000","last_became_active":"2024-09-19T04:15:35.513473+0000","last_became_peered":"2024-09-19T04:15:35.513473+0000","last_unstale":"2024-09-19T04:15:36.668156+0000","last_undegraded":"2024-09-19T04:15:36.668156+0000","last_fullsized":"2024-09-19T04:15:36.668156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:49.274377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-19T04:15:33.441722+0000","last_change":"2024-09-19T04:15:33.441722+0000","last_active":"2024-09-19T04:15:33.441722+0000","last_peered":"2024-09-19T04:15:33.441722+0000","last_clean":"2024-09-19T04:15:33.441722+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:15:33.441722+0000","last_undegraded":"2024-09-19T04:15:33.441722+0000","last_fullsized":"2024-09-19T04:15:33.441722+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":221,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-19T04:15:32.463741+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:15:32.463741+0000","last_peered":"2024-09-19T04:15:32.463741+0000","last_clean":"2024-09-19T04:15:32.463741+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:15:32.463741+0000","last_undegraded":"2024-09-19T04:15:32.463741+0000","last_fullsized":"2024-09-19T04:15:32.463741+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":652,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":12,"acting":12,"num_store_stats":1},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32324,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339516,"statfs":{"total":96636764160,"available":96603664384,"internally_reserved":0,"allocated":1159168,"data_stored":838908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":634880,"data_stored":281586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":577536,"data_stored":254199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":577536,"data_stored":254199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27592,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344248,"statfs":{"total":96636764160,"available":96608509952,"internally_reserved":0,"allocated":577536,"data_stored":254199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31720,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340120,"statfs":{"total":96636764160,"available":96604282880,"internally_reserved":0,"allocated":540672,"data_stored":237126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27596,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344244,"statfs":{"total":96636764160,"available":96608505856,"internally_reserved":0,"allocated":577536,"data_stored":254199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1159168,"data_stored":838908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:15:39.492 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:15:39.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba9f09640 1 Processor -- start 2024-09-19T04:15:39.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba9f09640 1 -- start start 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba9f09640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffba4158c90 0x7ffba40cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba9f09640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba4159070 con 0x7ffba40b3e00 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba9f09640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffba406d2c0 con 0x7ffba415f840 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba9f09640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ffba406d070 con 0x7ffba4158c90 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffba40b3e00 0x7ffba40cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47284/0 (socket says 172.21.15.76:47284) 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba3fff640 1 -- 172.21.15.76:0/1634824927 learned_addr learned my addr 172.21.15.76:0/1634824927 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba37fe640 1 --2- 172.21.15.76:0/1634824927 >> v2:172.21.15.116:3300/0 conn(0x7ffba4158c90 0x7ffba40cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 789238151 0 0) 0x7ffba4159070 con 0x7ffba40b3e00 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb8c003530 con 0x7ffba40b3e00 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1205442273 0 0) 0x7ffb8c003530 con 0x7ffba40b3e00 2024-09-19T04:15:39.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 >> v2:172.21.15.116:3300/0 conn(0x7ffba4158c90 msgr2=0x7ffba40cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 --2- 172.21.15.76:0/1634824927 >> v2:172.21.15.116:3300/0 conn(0x7ffba4158c90 0x7ffba40cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 >> v1:172.21.15.116:6789/0 conn(0x7ffba415f840 legacy=0x7ffba406e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba406d5a0 con 0x7ffba40b3e00 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7ffb9c001680 con 0x7ffba40b3e00 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 >> v1:172.21.15.76:6789/0 conn(0x7ffba40b3e00 legacy=0x7ffba40cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb8c0039c0 0x7ffb8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb8c0051d0 0x7ffb8c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406d070 con 0x7ffb8c004810 2024-09-19T04:15:39.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c00c070 con 0x7ffb8c0051d0 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8c00c270 con 0x7ffb8c0039c0 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba37fe640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb8c0039c0 0x7ffb8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba3fff640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb8c0051d0 0x7ffb8c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba2ffd640 1 -- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb8c0039c0 msgr2=0x7ffb8c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb8c0039c0 0x7ffb8c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba2ffd640 1 -- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb8c0051d0 msgr2=0x7ffb8c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb8c0051d0 0x7ffb8c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba3fff640 1 -- 172.21.15.76:0/1634824927 reap_dead start 2024-09-19T04:15:39.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.645+0000 7ffba2ffd640 1 -- 172.21.15.76:0/1634824927 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c003530 con 0x7ffb8c004810 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb98001e70 tx=0x7ffb9801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cd0f2ae80973a37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98001a90 con 0x7ffb8c004810 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb98003b80 con 0x7ffb8c004810 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba27fc640 1 -- 172.21.15.76:0/1634824927 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb98005080 con 0x7ffb8c004810 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 -- 172.21.15.76:0/1634824927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 msgr2=0x7ffb8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffb98001e70 tx=0x7ffb9801c760 comp rx=0 tx=0).stop 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 -- 172.21.15.76:0/1634824927 shutdown_connections 2024-09-19T04:15:39.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 --2- 172.21.15.76:0/1634824927 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffb8c004c90 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 -- 172.21.15.76:0/1634824927 >> 172.21.15.76:0/1634824927 conn(0x7ffba406bcc0 msgr2=0x7ffba406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 -- 172.21.15.76:0/1634824927 shutdown_connections 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 -- 172.21.15.76:0/1634824927 wait complete. 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 Processor -- start 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.649+0000 7ffba9f09640 1 -- start start 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffba415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffba4150e90 0x7ffba4150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffba40cd820 0x7ffba40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406de40 con 0x7ffb8c004810 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffba406db60 con 0x7ffba40cd820 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffba40cb9c0 con 0x7ffba4150e90 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffba415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffba415a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35124/0 (socket says 172.21.15.76:35124) 2024-09-19T04:15:39.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 -- 172.21.15.76:0/2404821273 learned_addr learned my addr 172.21.15.76:0/2404821273 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:39.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba3fff640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffba40cd820 0x7ffba40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffba4150e90 0x7ffba4150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 -- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffba4150e90 msgr2=0x7ffba4150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffba4150e90 0x7ffba4150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 -- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffba40cd820 msgr2=0x7ffba40cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffba40cd820 0x7ffba40cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 -- 172.21.15.76:0/2404821273 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffba415aa10 con 0x7ffb8c004810 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba3fff640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffba40cd820 0x7ffba40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffba4150e90 0x7ffba4150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba37fe640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffba415a4d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb94011690 tx=0x7ffb94011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffb8bfff640 1 -- 172.21.15.76:0/2404821273 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb9400f9c0 con 0x7ffb8c004810 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffb8bfff640 1 -- 172.21.15.76:0/2404821273 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb9400fb60 con 0x7ffb8c004810 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffba4150c70 con 0x7ffb8c004810 2024-09-19T04:15:39.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffb8bfff640 1 -- 172.21.15.76:0/2404821273 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb94015460 con 0x7ffb8c004810 2024-09-19T04:15:39.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffba415af90 con 0x7ffb8c004810 2024-09-19T04:15:39.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffba40ce1e0 con 0x7ffb8c004810 2024-09-19T04:15:39.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffb8bfff640 1 -- 172.21.15.76:0/2404821273 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb94015600 con 0x7ffb8c004810 2024-09-19T04:15:39.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffb8bfff640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb900678c0 0x7ffb90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:39.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffb8bfff640 1 -- 172.21.15.76:0/2404821273 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ffb94086730 con 0x7ffb8c004810 2024-09-19T04:15:39.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb900678c0 0x7ffb90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:39.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.653+0000 7ffba2ffd640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb900678c0 0x7ffb90069d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005000 tx=0x7ffb98001520 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:39.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.657+0000 7ffb8bfff640 1 -- 172.21.15.76:0/2404821273 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffb94019030 con 0x7ffb8c004810 2024-09-19T04:15:39.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.805+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffba40b3b80 con 0x7ffb900678c0 2024-09-19T04:15:39.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.813+0000 7ffb8bfff640 1 -- 172.21.15.76:0/2404821273 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7ffb98004c10 con 0x7ffb900678c0 2024-09-19T04:15:39.818 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:39.819 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb900678c0 msgr2=0x7ffb90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb900678c0 0x7ffb90069d80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb98005000 tx=0x7ffb98001520 comp rx=0 tx=0).stop 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 msgr2=0x7ffba415a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffba415a4d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb94011690 tx=0x7ffb94011b50 comp rx=0 tx=0).stop 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 shutdown_connections 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb900678c0 0x7ffb90069d80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffba40cd820 0x7ffba40cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffba4150e90 0x7ffba4150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 --2- 172.21.15.76:0/2404821273 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb8c004810 0x7ffba415a4d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:39.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 >> 172.21.15.76:0/2404821273 conn(0x7ffba406bcc0 msgr2=0x7ffba415fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:39.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 shutdown_connections 2024-09-19T04:15:39.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:39.817+0000 7ffba9f09640 1 -- 172.21.15.76:0/2404821273 wait complete. 2024-09-19T04:15:39.836 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-19T04:15:39.781984+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230696,"kb_used_data":6120,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744024,"statfs":{"total":773094113280,"available":772857880576,"internally_reserved":0,"allocated":6266880,"data_stored":3433028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":49,"apply_latency_ms":49,"commit_latency_ns":49000000,"apply_latency_ns":49000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001469"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475634+0000","last_change":"2024-09-19T04:15:35.503282+0000","last_active":"2024-09-19T04:15:36.475634+0000","last_peered":"2024-09-19T04:15:36.475634+0000","last_clean":"2024-09-19T04:15:36.475634+0000","last_became_active":"2024-09-19T04:15:35.502941+0000","last_became_peered":"2024-09-19T04:15:35.502941+0000","last_unstale":"2024-09-19T04:15:36.475634+0000","last_undegraded":"2024-09-19T04:15:36.475634+0000","last_fullsized":"2024-09-19T04:15:36.475634+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:10.368231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506655+0000","last_change":"2024-09-19T04:15:35.498013+0000","last_active":"2024-09-19T04:15:36.506655+0000","last_peered":"2024-09-19T04:15:36.506655+0000","last_clean":"2024-09-19T04:15:36.506655+0000","last_became_active":"2024-09-19T04:15:35.497297+0000","last_became_peered":"2024-09-19T04:15:35.497297+0000","last_unstale":"2024-09-19T04:15:36.506655+0000","last_undegraded":"2024-09-19T04:15:36.506655+0000","last_fullsized":"2024-09-19T04:15:36.506655+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:39.501083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501614+0000","last_change":"2024-09-19T04:15:35.511901+0000","last_active":"2024-09-19T04:15:36.501614+0000","last_peered":"2024-09-19T04:15:36.501614+0000","last_clean":"2024-09-19T04:15:36.501614+0000","last_became_active":"2024-09-19T04:15:35.511441+0000","last_became_peered":"2024-09-19T04:15:35.511441+0000","last_unstale":"2024-09-19T04:15:36.501614+0000","last_undegraded":"2024-09-19T04:15:36.501614+0000","last_fullsized":"2024-09-19T04:15:36.501614+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:38:22.481214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506226+0000","last_change":"2024-09-19T04:15:35.498150+0000","last_active":"2024-09-19T04:15:36.506226+0000","last_peered":"2024-09-19T04:15:36.506226+0000","last_clean":"2024-09-19T04:15:36.506226+0000","last_became_active":"2024-09-19T04:15:35.497558+0000","last_became_peered":"2024-09-19T04:15:35.497558+0000","last_unstale":"2024-09-19T04:15:36.506226+0000","last_undegraded":"2024-09-19T04:15:36.506226+0000","last_fullsized":"2024-09-19T04:15:36.506226+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:37:23.060883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475935+0000","last_change":"2024-09-19T04:15:35.485025+0000","last_active":"2024-09-19T04:15:36.475935+0000","last_peered":"2024-09-19T04:15:36.475935+0000","last_clean":"2024-09-19T04:15:36.475935+0000","last_became_active":"2024-09-19T04:15:35.484187+0000","last_became_peered":"2024-09-19T04:15:35.484187+0000","last_unstale":"2024-09-19T04:15:36.475935+0000","last_undegraded":"2024-09-19T04:15:36.475935+0000","last_fullsized":"2024-09-19T04:15:36.475935+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:25.484595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475338+0000","last_change":"2024-09-19T04:15:35.490056+0000","last_active":"2024-09-19T04:15:36.475338+0000","last_peered":"2024-09-19T04:15:36.475338+0000","last_clean":"2024-09-19T04:15:36.475338+0000","last_became_active":"2024-09-19T04:15:35.489437+0000","last_became_peered":"2024-09-19T04:15:35.489437+0000","last_unstale":"2024-09-19T04:15:36.475338+0000","last_undegraded":"2024-09-19T04:15:36.475338+0000","last_fullsized":"2024-09-19T04:15:36.475338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:57:21.740917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501287+0000","last_change":"2024-09-19T04:15:35.502675+0000","last_active":"2024-09-19T04:15:36.501287+0000","last_peered":"2024-09-19T04:15:36.501287+0000","last_clean":"2024-09-19T04:15:36.501287+0000","last_became_active":"2024-09-19T04:15:35.502355+0000","last_became_peered":"2024-09-19T04:15:35.502355+0000","last_unstale":"2024-09-19T04:15:36.501287+0000","last_undegraded":"2024-09-19T04:15:36.501287+0000","last_fullsized":"2024-09-19T04:15:36.501287+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:54:21.458201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668235+0000","last_change":"2024-09-19T04:15:35.499099+0000","last_active":"2024-09-19T04:15:36.668235+0000","last_peered":"2024-09-19T04:15:36.668235+0000","last_clean":"2024-09-19T04:15:36.668235+0000","last_became_active":"2024-09-19T04:15:35.496345+0000","last_became_peered":"2024-09-19T04:15:35.496345+0000","last_unstale":"2024-09-19T04:15:36.668235+0000","last_undegraded":"2024-09-19T04:15:36.668235+0000","last_fullsized":"2024-09-19T04:15:36.668235+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:52:54.906971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.670500+0000","last_change":"2024-09-19T04:15:35.496303+0000","last_active":"2024-09-19T04:15:36.670500+0000","last_peered":"2024-09-19T04:15:36.670500+0000","last_clean":"2024-09-19T04:15:36.670500+0000","last_became_active":"2024-09-19T04:15:35.495954+0000","last_became_peered":"2024-09-19T04:15:35.495954+0000","last_unstale":"2024-09-19T04:15:36.670500+0000","last_undegraded":"2024-09-19T04:15:36.670500+0000","last_fullsized":"2024-09-19T04:15:36.670500+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:33:14.143439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476134+0000","last_change":"2024-09-19T04:15:35.477800+0000","last_active":"2024-09-19T04:15:36.476134+0000","last_peered":"2024-09-19T04:15:36.476134+0000","last_clean":"2024-09-19T04:15:36.476134+0000","last_became_active":"2024-09-19T04:15:35.477196+0000","last_became_peered":"2024-09-19T04:15:35.477196+0000","last_unstale":"2024-09-19T04:15:36.476134+0000","last_undegraded":"2024-09-19T04:15:36.476134+0000","last_fullsized":"2024-09-19T04:15:36.476134+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:32.842474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669402+0000","last_change":"2024-09-19T04:15:35.487396+0000","last_active":"2024-09-19T04:15:36.669402+0000","last_peered":"2024-09-19T04:15:36.669402+0000","last_clean":"2024-09-19T04:15:36.669402+0000","last_became_active":"2024-09-19T04:15:35.486649+0000","last_became_peered":"2024-09-19T04:15:35.486649+0000","last_unstale":"2024-09-19T04:15:36.669402+0000","last_undegraded":"2024-09-19T04:15:36.669402+0000","last_fullsized":"2024-09-19T04:15:36.669402+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:26:22.933149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.670446+0000","last_change":"2024-09-19T04:15:35.504080+0000","last_active":"2024-09-19T04:15:36.670446+0000","last_peered":"2024-09-19T04:15:36.670446+0000","last_clean":"2024-09-19T04:15:36.670446+0000","last_became_active":"2024-09-19T04:15:35.503779+0000","last_became_peered":"2024-09-19T04:15:35.503779+0000","last_unstale":"2024-09-19T04:15:36.670446+0000","last_undegraded":"2024-09-19T04:15:36.670446+0000","last_fullsized":"2024-09-19T04:15:36.670446+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:04.785223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475476+0000","last_change":"2024-09-19T04:15:35.481702+0000","last_active":"2024-09-19T04:15:36.475476+0000","last_peered":"2024-09-19T04:15:36.475476+0000","last_clean":"2024-09-19T04:15:36.475476+0000","last_became_active":"2024-09-19T04:15:35.480973+0000","last_became_peered":"2024-09-19T04:15:35.480973+0000","last_unstale":"2024-09-19T04:15:36.475476+0000","last_undegraded":"2024-09-19T04:15:36.475476+0000","last_fullsized":"2024-09-19T04:15:36.475476+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:36.125497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502043+0000","last_change":"2024-09-19T04:15:35.494967+0000","last_active":"2024-09-19T04:15:36.502043+0000","last_peered":"2024-09-19T04:15:36.502043+0000","last_clean":"2024-09-19T04:15:36.502043+0000","last_became_active":"2024-09-19T04:15:35.493817+0000","last_became_peered":"2024-09-19T04:15:35.493817+0000","last_unstale":"2024-09-19T04:15:36.502043+0000","last_undegraded":"2024-09-19T04:15:36.502043+0000","last_fullsized":"2024-09-19T04:15:36.502043+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.159877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.505850+0000","last_change":"2024-09-19T04:15:35.505565+0000","last_active":"2024-09-19T04:15:36.505850+0000","last_peered":"2024-09-19T04:15:36.505850+0000","last_clean":"2024-09-19T04:15:36.505850+0000","last_became_active":"2024-09-19T04:15:35.504660+0000","last_became_peered":"2024-09-19T04:15:35.504660+0000","last_unstale":"2024-09-19T04:15:36.505850+0000","last_undegraded":"2024-09-19T04:15:36.505850+0000","last_fullsized":"2024-09-19T04:15:36.505850+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:27:21.647795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475166+0000","last_change":"2024-09-19T04:15:35.481233+0000","last_active":"2024-09-19T04:15:36.475166+0000","last_peered":"2024-09-19T04:15:36.475166+0000","last_clean":"2024-09-19T04:15:36.475166+0000","last_became_active":"2024-09-19T04:15:35.480385+0000","last_became_peered":"2024-09-19T04:15:35.480385+0000","last_unstale":"2024-09-19T04:15:36.475166+0000","last_undegraded":"2024-09-19T04:15:36.475166+0000","last_fullsized":"2024-09-19T04:15:36.475166+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:28.365203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501651+0000","last_change":"2024-09-19T04:15:35.501821+0000","last_active":"2024-09-19T04:15:36.501651+0000","last_peered":"2024-09-19T04:15:36.501651+0000","last_clean":"2024-09-19T04:15:36.501651+0000","last_became_active":"2024-09-19T04:15:35.495414+0000","last_became_peered":"2024-09-19T04:15:35.495414+0000","last_unstale":"2024-09-19T04:15:36.501651+0000","last_undegraded":"2024-09-19T04:15:36.501651+0000","last_fullsized":"2024-09-19T04:15:36.501651+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:05.317264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506792+0000","last_change":"2024-09-19T04:15:35.496238+0000","last_active":"2024-09-19T04:15:36.506792+0000","last_peered":"2024-09-19T04:15:36.506792+0000","last_clean":"2024-09-19T04:15:36.506792+0000","last_became_active":"2024-09-19T04:15:35.495558+0000","last_became_peered":"2024-09-19T04:15:35.495558+0000","last_unstale":"2024-09-19T04:15:36.506792+0000","last_undegraded":"2024-09-19T04:15:36.506792+0000","last_fullsized":"2024-09-19T04:15:36.506792+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:40:15.874685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502521+0000","last_change":"2024-09-19T04:15:35.511142+0000","last_active":"2024-09-19T04:15:36.502521+0000","last_peered":"2024-09-19T04:15:36.502521+0000","last_clean":"2024-09-19T04:15:36.502521+0000","last_became_active":"2024-09-19T04:15:35.501821+0000","last_became_peered":"2024-09-19T04:15:35.501821+0000","last_unstale":"2024-09-19T04:15:36.502521+0000","last_undegraded":"2024-09-19T04:15:36.502521+0000","last_fullsized":"2024-09-19T04:15:36.502521+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:22.294320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668643+0000","last_change":"2024-09-19T04:15:35.523138+0000","last_active":"2024-09-19T04:15:36.668643+0000","last_peered":"2024-09-19T04:15:36.668643+0000","last_clean":"2024-09-19T04:15:36.668643+0000","last_became_active":"2024-09-19T04:15:35.522403+0000","last_became_peered":"2024-09-19T04:15:35.522403+0000","last_unstale":"2024-09-19T04:15:36.668643+0000","last_undegraded":"2024-09-19T04:15:36.668643+0000","last_fullsized":"2024-09-19T04:15:36.668643+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:57:21.740917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668482+0000","last_change":"2024-09-19T04:15:35.515485+0000","last_active":"2024-09-19T04:15:36.668482+0000","last_peered":"2024-09-19T04:15:36.668482+0000","last_clean":"2024-09-19T04:15:36.668482+0000","last_became_active":"2024-09-19T04:15:35.509562+0000","last_became_peered":"2024-09-19T04:15:35.509562+0000","last_unstale":"2024-09-19T04:15:36.668482+0000","last_undegraded":"2024-09-19T04:15:36.668482+0000","last_fullsized":"2024-09-19T04:15:36.668482+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:31.328898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669525+0000","last_change":"2024-09-19T04:15:35.496542+0000","last_active":"2024-09-19T04:15:36.669525+0000","last_peered":"2024-09-19T04:15:36.669525+0000","last_clean":"2024-09-19T04:15:36.669525+0000","last_became_active":"2024-09-19T04:15:35.496191+0000","last_became_peered":"2024-09-19T04:15:35.496191+0000","last_unstale":"2024-09-19T04:15:36.669525+0000","last_undegraded":"2024-09-19T04:15:36.669525+0000","last_fullsized":"2024-09-19T04:15:36.669525+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:45.939674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668673+0000","last_change":"2024-09-19T04:15:35.487809+0000","last_active":"2024-09-19T04:15:36.668673+0000","last_peered":"2024-09-19T04:15:36.668673+0000","last_clean":"2024-09-19T04:15:36.668673+0000","last_became_active":"2024-09-19T04:15:35.487121+0000","last_became_peered":"2024-09-19T04:15:35.487121+0000","last_unstale":"2024-09-19T04:15:36.668673+0000","last_undegraded":"2024-09-19T04:15:36.668673+0000","last_fullsized":"2024-09-19T04:15:36.668673+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:09:14.491514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501006+0000","last_change":"2024-09-19T04:15:35.494879+0000","last_active":"2024-09-19T04:15:36.501006+0000","last_peered":"2024-09-19T04:15:36.501006+0000","last_clean":"2024-09-19T04:15:36.501006+0000","last_became_active":"2024-09-19T04:15:35.494519+0000","last_became_peered":"2024-09-19T04:15:35.494519+0000","last_unstale":"2024-09-19T04:15:36.501006+0000","last_undegraded":"2024-09-19T04:15:36.501006+0000","last_fullsized":"2024-09-19T04:15:36.501006+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:45:21.594713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502403+0000","last_change":"2024-09-19T04:15:35.514821+0000","last_active":"2024-09-19T04:15:36.502403+0000","last_peered":"2024-09-19T04:15:36.502403+0000","last_clean":"2024-09-19T04:15:36.502403+0000","last_became_active":"2024-09-19T04:15:35.513574+0000","last_became_peered":"2024-09-19T04:15:35.513574+0000","last_unstale":"2024-09-19T04:15:36.502403+0000","last_undegraded":"2024-09-19T04:15:36.502403+0000","last_fullsized":"2024-09-19T04:15:36.502403+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:17.266434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669927+0000","last_change":"2024-09-19T04:15:35.499614+0000","last_active":"2024-09-19T04:15:36.669927+0000","last_peered":"2024-09-19T04:15:36.669927+0000","last_clean":"2024-09-19T04:15:36.669927+0000","last_became_active":"2024-09-19T04:15:35.499344+0000","last_became_peered":"2024-09-19T04:15:35.499344+0000","last_unstale":"2024-09-19T04:15:36.669927+0000","last_undegraded":"2024-09-19T04:15:36.669927+0000","last_fullsized":"2024-09-19T04:15:36.669927+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:49.962739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476163+0000","last_change":"2024-09-19T04:15:35.494583+0000","last_active":"2024-09-19T04:15:36.476163+0000","last_peered":"2024-09-19T04:15:36.476163+0000","last_clean":"2024-09-19T04:15:36.476163+0000","last_became_active":"2024-09-19T04:15:35.494168+0000","last_became_peered":"2024-09-19T04:15:35.494168+0000","last_unstale":"2024-09-19T04:15:36.476163+0000","last_undegraded":"2024-09-19T04:15:36.476163+0000","last_fullsized":"2024-09-19T04:15:36.476163+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:23.373342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668366+0000","last_change":"2024-09-19T04:15:35.502323+0000","last_active":"2024-09-19T04:15:36.668366+0000","last_peered":"2024-09-19T04:15:36.668366+0000","last_clean":"2024-09-19T04:15:36.668366+0000","last_became_active":"2024-09-19T04:15:35.501535+0000","last_became_peered":"2024-09-19T04:15:35.501535+0000","last_unstale":"2024-09-19T04:15:36.668366+0000","last_undegraded":"2024-09-19T04:15:36.668366+0000","last_fullsized":"2024-09-19T04:15:36.668366+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:43.709884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502102+0000","last_change":"2024-09-19T04:15:35.510010+0000","last_active":"2024-09-19T04:15:36.502102+0000","last_peered":"2024-09-19T04:15:36.502102+0000","last_clean":"2024-09-19T04:15:36.502102+0000","last_became_active":"2024-09-19T04:15:35.503491+0000","last_became_peered":"2024-09-19T04:15:35.503491+0000","last_unstale":"2024-09-19T04:15:36.502102+0000","last_undegraded":"2024-09-19T04:15:36.502102+0000","last_fullsized":"2024-09-19T04:15:36.502102+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:49.962739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506011+0000","last_change":"2024-09-19T04:15:35.507246+0000","last_active":"2024-09-19T04:15:36.506011+0000","last_peered":"2024-09-19T04:15:36.506011+0000","last_clean":"2024-09-19T04:15:36.506011+0000","last_became_active":"2024-09-19T04:15:35.505926+0000","last_became_peered":"2024-09-19T04:15:35.505926+0000","last_unstale":"2024-09-19T04:15:36.506011+0000","last_undegraded":"2024-09-19T04:15:36.506011+0000","last_fullsized":"2024-09-19T04:15:36.506011+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:31:24.794628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501294+0000","last_change":"2024-09-19T04:15:35.494378+0000","last_active":"2024-09-19T04:15:36.501294+0000","last_peered":"2024-09-19T04:15:36.501294+0000","last_clean":"2024-09-19T04:15:36.501294+0000","last_became_active":"2024-09-19T04:15:35.493438+0000","last_became_peered":"2024-09-19T04:15:35.493438+0000","last_unstale":"2024-09-19T04:15:36.501294+0000","last_undegraded":"2024-09-19T04:15:36.501294+0000","last_fullsized":"2024-09-19T04:15:36.501294+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:01:02.580915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476895+0000","last_change":"2024-09-19T04:15:35.492488+0000","last_active":"2024-09-19T04:15:36.476895+0000","last_peered":"2024-09-19T04:15:36.476895+0000","last_clean":"2024-09-19T04:15:36.476895+0000","last_became_active":"2024-09-19T04:15:35.491778+0000","last_became_peered":"2024-09-19T04:15:35.491778+0000","last_unstale":"2024-09-19T04:15:36.476895+0000","last_undegraded":"2024-09-19T04:15:36.476895+0000","last_fullsized":"2024-09-19T04:15:36.476895+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:37.610383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.505632+0000","last_change":"2024-09-19T04:15:35.494553+0000","last_active":"2024-09-19T04:15:36.505632+0000","last_peered":"2024-09-19T04:15:36.505632+0000","last_clean":"2024-09-19T04:15:36.505632+0000","last_became_active":"2024-09-19T04:15:35.493844+0000","last_became_peered":"2024-09-19T04:15:35.493844+0000","last_unstale":"2024-09-19T04:15:36.505632+0000","last_undegraded":"2024-09-19T04:15:36.505632+0000","last_fullsized":"2024-09-19T04:15:36.505632+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:43:43.021282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"52'4","reported_seq":18,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668459+0000","last_change":"2024-09-19T04:15:34.495140+0000","last_active":"2024-09-19T04:15:36.668459+0000","last_peered":"2024-09-19T04:15:36.668459+0000","last_clean":"2024-09-19T04:15:36.668459+0000","last_became_active":"2024-09-19T04:15:34.494396+0000","last_became_peered":"2024-09-19T04:15:34.494396+0000","last_unstale":"2024-09-19T04:15:36.668459+0000","last_undegraded":"2024-09-19T04:15:36.668459+0000","last_fullsized":"2024-09-19T04:15:36.668459+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:40.535985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.477781+0000","last_change":"2024-09-19T04:15:35.482268+0000","last_active":"2024-09-19T04:15:36.477781+0000","last_peered":"2024-09-19T04:15:36.477781+0000","last_clean":"2024-09-19T04:15:36.477781+0000","last_became_active":"2024-09-19T04:15:35.481439+0000","last_became_peered":"2024-09-19T04:15:35.481439+0000","last_unstale":"2024-09-19T04:15:36.477781+0000","last_undegraded":"2024-09-19T04:15:36.477781+0000","last_fullsized":"2024-09-19T04:15:36.477781+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:30:08.505205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476380+0000","last_change":"2024-09-19T04:15:34.498757+0000","last_active":"2024-09-19T04:15:36.476380+0000","last_peered":"2024-09-19T04:15:36.476380+0000","last_clean":"2024-09-19T04:15:36.476380+0000","last_became_active":"2024-09-19T04:15:34.498426+0000","last_became_peered":"2024-09-19T04:15:34.498426+0000","last_unstale":"2024-09-19T04:15:36.476380+0000","last_undegraded":"2024-09-19T04:15:36.476380+0000","last_fullsized":"2024-09-19T04:15:36.476380+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:40.535985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668098+0000","last_change":"2024-09-19T04:15:35.509583+0000","last_active":"2024-09-19T04:15:36.668098+0000","last_peered":"2024-09-19T04:15:36.668098+0000","last_clean":"2024-09-19T04:15:36.668098+0000","last_became_active":"2024-09-19T04:15:35.509008+0000","last_became_peered":"2024-09-19T04:15:35.509008+0000","last_unstale":"2024-09-19T04:15:36.668098+0000","last_undegraded":"2024-09-19T04:15:36.668098+0000","last_fullsized":"2024-09-19T04:15:36.668098+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:38.558469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476394+0000","last_change":"2024-09-19T04:15:34.500264+0000","last_active":"2024-09-19T04:15:36.476394+0000","last_peered":"2024-09-19T04:15:36.476394+0000","last_clean":"2024-09-19T04:15:36.476394+0000","last_became_active":"2024-09-19T04:15:34.499851+0000","last_became_peered":"2024-09-19T04:15:34.499851+0000","last_unstale":"2024-09-19T04:15:36.476394+0000","last_undegraded":"2024-09-19T04:15:36.476394+0000","last_fullsized":"2024-09-19T04:15:36.476394+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:13.930894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669441+0000","last_change":"2024-09-19T04:15:35.500216+0000","last_active":"2024-09-19T04:15:36.669441+0000","last_peered":"2024-09-19T04:15:36.669441+0000","last_clean":"2024-09-19T04:15:36.669441+0000","last_became_active":"2024-09-19T04:15:35.497183+0000","last_became_peered":"2024-09-19T04:15:35.497183+0000","last_unstale":"2024-09-19T04:15:36.669441+0000","last_undegraded":"2024-09-19T04:15:36.669441+0000","last_fullsized":"2024-09-19T04:15:36.669441+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:46:05.317264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.478118+0000","last_change":"2024-09-19T04:15:34.490750+0000","last_active":"2024-09-19T04:15:36.478118+0000","last_peered":"2024-09-19T04:15:36.478118+0000","last_clean":"2024-09-19T04:15:36.478118+0000","last_became_active":"2024-09-19T04:15:34.489944+0000","last_became_peered":"2024-09-19T04:15:34.489944+0000","last_unstale":"2024-09-19T04:15:36.478118+0000","last_undegraded":"2024-09-19T04:15:36.478118+0000","last_fullsized":"2024-09-19T04:15:36.478118+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:27.355033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476219+0000","last_change":"2024-09-19T04:15:35.496537+0000","last_active":"2024-09-19T04:15:36.476219+0000","last_peered":"2024-09-19T04:15:36.476219+0000","last_clean":"2024-09-19T04:15:36.476219+0000","last_became_active":"2024-09-19T04:15:35.495798+0000","last_became_peered":"2024-09-19T04:15:35.495798+0000","last_unstale":"2024-09-19T04:15:36.476219+0000","last_undegraded":"2024-09-19T04:15:36.476219+0000","last_fullsized":"2024-09-19T04:15:36.476219+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:19.097387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501584+0000","last_change":"2024-09-19T04:15:34.490792+0000","last_active":"2024-09-19T04:15:36.501584+0000","last_peered":"2024-09-19T04:15:36.501584+0000","last_clean":"2024-09-19T04:15:36.501584+0000","last_became_active":"2024-09-19T04:15:34.490424+0000","last_became_peered":"2024-09-19T04:15:34.490424+0000","last_unstale":"2024-09-19T04:15:36.501584+0000","last_undegraded":"2024-09-19T04:15:36.501584+0000","last_fullsized":"2024-09-19T04:15:36.501584+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:13:42.699714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506408+0000","last_change":"2024-09-19T04:15:35.503832+0000","last_active":"2024-09-19T04:15:36.506408+0000","last_peered":"2024-09-19T04:15:36.506408+0000","last_clean":"2024-09-19T04:15:36.506408+0000","last_became_active":"2024-09-19T04:15:35.499550+0000","last_became_peered":"2024-09-19T04:15:35.499550+0000","last_unstale":"2024-09-19T04:15:36.506408+0000","last_undegraded":"2024-09-19T04:15:36.506408+0000","last_fullsized":"2024-09-19T04:15:36.506408+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:38:10.129072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475338+0000","last_change":"2024-09-19T04:15:34.506986+0000","last_active":"2024-09-19T04:15:36.475338+0000","last_peered":"2024-09-19T04:15:36.475338+0000","last_clean":"2024-09-19T04:15:36.475338+0000","last_became_active":"2024-09-19T04:15:34.506206+0000","last_became_peered":"2024-09-19T04:15:34.506206+0000","last_unstale":"2024-09-19T04:15:36.475338+0000","last_undegraded":"2024-09-19T04:15:36.475338+0000","last_fullsized":"2024-09-19T04:15:36.475338+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:01.529754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.670373+0000","last_change":"2024-09-19T04:15:35.498803+0000","last_active":"2024-09-19T04:15:36.670373+0000","last_peered":"2024-09-19T04:15:36.670373+0000","last_clean":"2024-09-19T04:15:36.670373+0000","last_became_active":"2024-09-19T04:15:35.497151+0000","last_became_peered":"2024-09-19T04:15:35.497151+0000","last_unstale":"2024-09-19T04:15:36.670373+0000","last_undegraded":"2024-09-19T04:15:36.670373+0000","last_fullsized":"2024-09-19T04:15:36.670373+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:17.266434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668469+0000","last_change":"2024-09-19T04:15:34.500494+0000","last_active":"2024-09-19T04:15:36.668469+0000","last_peered":"2024-09-19T04:15:36.668469+0000","last_clean":"2024-09-19T04:15:36.668469+0000","last_became_active":"2024-09-19T04:15:34.499595+0000","last_became_peered":"2024-09-19T04:15:34.499595+0000","last_unstale":"2024-09-19T04:15:36.668469+0000","last_undegraded":"2024-09-19T04:15:36.668469+0000","last_fullsized":"2024-09-19T04:15:36.668469+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.803393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506738+0000","last_change":"2024-09-19T04:15:35.518369+0000","last_active":"2024-09-19T04:15:36.506738+0000","last_peered":"2024-09-19T04:15:36.506738+0000","last_clean":"2024-09-19T04:15:36.506738+0000","last_became_active":"2024-09-19T04:15:35.516692+0000","last_became_peered":"2024-09-19T04:15:35.516692+0000","last_unstale":"2024-09-19T04:15:36.506738+0000","last_undegraded":"2024-09-19T04:15:36.506738+0000","last_fullsized":"2024-09-19T04:15:36.506738+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:04:11.253895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501070+0000","last_change":"2024-09-19T04:15:34.503131+0000","last_active":"2024-09-19T04:15:36.501070+0000","last_peered":"2024-09-19T04:15:36.501070+0000","last_clean":"2024-09-19T04:15:36.501070+0000","last_became_active":"2024-09-19T04:15:34.502758+0000","last_became_peered":"2024-09-19T04:15:34.502758+0000","last_unstale":"2024-09-19T04:15:36.501070+0000","last_undegraded":"2024-09-19T04:15:36.501070+0000","last_fullsized":"2024-09-19T04:15:36.501070+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:15:00.538377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475233+0000","last_change":"2024-09-19T04:15:35.490718+0000","last_active":"2024-09-19T04:15:36.475233+0000","last_peered":"2024-09-19T04:15:36.475233+0000","last_clean":"2024-09-19T04:15:36.475233+0000","last_became_active":"2024-09-19T04:15:35.490004+0000","last_became_peered":"2024-09-19T04:15:35.490004+0000","last_unstale":"2024-09-19T04:15:36.475233+0000","last_undegraded":"2024-09-19T04:15:36.475233+0000","last_fullsized":"2024-09-19T04:15:36.475233+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:28.244131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506989+0000","last_change":"2024-09-19T04:15:34.498668+0000","last_active":"2024-09-19T04:15:36.506989+0000","last_peered":"2024-09-19T04:15:36.506989+0000","last_clean":"2024-09-19T04:15:36.506989+0000","last_became_active":"2024-09-19T04:15:34.497658+0000","last_became_peered":"2024-09-19T04:15:34.497658+0000","last_unstale":"2024-09-19T04:15:36.506989+0000","last_undegraded":"2024-09-19T04:15:36.506989+0000","last_fullsized":"2024-09-19T04:15:36.506989+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:04:26.537405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476526+0000","last_change":"2024-09-19T04:15:35.475931+0000","last_active":"2024-09-19T04:15:36.476526+0000","last_peered":"2024-09-19T04:15:36.476526+0000","last_clean":"2024-09-19T04:15:36.476526+0000","last_became_active":"2024-09-19T04:15:35.475161+0000","last_became_peered":"2024-09-19T04:15:35.475161+0000","last_unstale":"2024-09-19T04:15:36.476526+0000","last_undegraded":"2024-09-19T04:15:36.476526+0000","last_fullsized":"2024-09-19T04:15:36.476526+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:16.117299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476096+0000","last_change":"2024-09-19T04:15:34.490610+0000","last_active":"2024-09-19T04:15:36.476096+0000","last_peered":"2024-09-19T04:15:36.476096+0000","last_clean":"2024-09-19T04:15:36.476096+0000","last_became_active":"2024-09-19T04:15:34.489820+0000","last_became_peered":"2024-09-19T04:15:34.489820+0000","last_unstale":"2024-09-19T04:15:36.476096+0000","last_undegraded":"2024-09-19T04:15:36.476096+0000","last_fullsized":"2024-09-19T04:15:36.476096+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:05.905546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502438+0000","last_change":"2024-09-19T04:15:35.513551+0000","last_active":"2024-09-19T04:15:36.502438+0000","last_peered":"2024-09-19T04:15:36.502438+0000","last_clean":"2024-09-19T04:15:36.502438+0000","last_became_active":"2024-09-19T04:15:35.512435+0000","last_became_peered":"2024-09-19T04:15:35.512435+0000","last_unstale":"2024-09-19T04:15:36.502438+0000","last_undegraded":"2024-09-19T04:15:36.502438+0000","last_fullsized":"2024-09-19T04:15:36.502438+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:45:04.785223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"52'3","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506470+0000","last_change":"2024-09-19T04:15:34.496490+0000","last_active":"2024-09-19T04:15:36.506470+0000","last_peered":"2024-09-19T04:15:36.506470+0000","last_clean":"2024-09-19T04:15:36.506470+0000","last_became_active":"2024-09-19T04:15:34.494953+0000","last_became_peered":"2024-09-19T04:15:34.494953+0000","last_unstale":"2024-09-19T04:15:36.506470+0000","last_undegraded":"2024-09-19T04:15:36.506470+0000","last_fullsized":"2024-09-19T04:15:36.506470+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:52:25.591068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476001+0000","last_change":"2024-09-19T04:15:35.489083+0000","last_active":"2024-09-19T04:15:36.476001+0000","last_peered":"2024-09-19T04:15:36.476001+0000","last_clean":"2024-09-19T04:15:36.476001+0000","last_became_active":"2024-09-19T04:15:35.487960+0000","last_became_peered":"2024-09-19T04:15:35.487960+0000","last_unstale":"2024-09-19T04:15:36.476001+0000","last_undegraded":"2024-09-19T04:15:36.476001+0000","last_fullsized":"2024-09-19T04:15:36.476001+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:25.130555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"52'6","reported_seq":20,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.670302+0000","last_change":"2024-09-19T04:15:34.495155+0000","last_active":"2024-09-19T04:15:36.670302+0000","last_peered":"2024-09-19T04:15:36.670302+0000","last_clean":"2024-09-19T04:15:36.670302+0000","last_became_active":"2024-09-19T04:15:34.494775+0000","last_became_peered":"2024-09-19T04:15:34.494775+0000","last_unstale":"2024-09-19T04:15:36.670302+0000","last_undegraded":"2024-09-19T04:15:36.670302+0000","last_fullsized":"2024-09-19T04:15:36.670302+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:22.363172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.477828+0000","last_change":"2024-09-19T04:15:35.483699+0000","last_active":"2024-09-19T04:15:36.477828+0000","last_peered":"2024-09-19T04:15:36.477828+0000","last_clean":"2024-09-19T04:15:36.477828+0000","last_became_active":"2024-09-19T04:15:35.483024+0000","last_became_peered":"2024-09-19T04:15:35.483024+0000","last_unstale":"2024-09-19T04:15:36.477828+0000","last_undegraded":"2024-09-19T04:15:36.477828+0000","last_fullsized":"2024-09-19T04:15:36.477828+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:53.506681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501872+0000","last_change":"2024-09-19T04:15:34.500072+0000","last_active":"2024-09-19T04:15:36.501872+0000","last_peered":"2024-09-19T04:15:36.501872+0000","last_clean":"2024-09-19T04:15:36.501872+0000","last_became_active":"2024-09-19T04:15:34.498610+0000","last_became_peered":"2024-09-19T04:15:34.498610+0000","last_unstale":"2024-09-19T04:15:36.501872+0000","last_undegraded":"2024-09-19T04:15:36.501872+0000","last_fullsized":"2024-09-19T04:15:36.501872+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:30.318728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502200+0000","last_change":"2024-09-19T04:15:35.511695+0000","last_active":"2024-09-19T04:15:36.502200+0000","last_peered":"2024-09-19T04:15:36.502200+0000","last_clean":"2024-09-19T04:15:36.502200+0000","last_became_active":"2024-09-19T04:15:35.502324+0000","last_became_peered":"2024-09-19T04:15:35.502324+0000","last_unstale":"2024-09-19T04:15:36.502200+0000","last_undegraded":"2024-09-19T04:15:36.502200+0000","last_fullsized":"2024-09-19T04:15:36.502200+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:13.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668265+0000","last_change":"2024-09-19T04:15:34.500317+0000","last_active":"2024-09-19T04:15:36.668265+0000","last_peered":"2024-09-19T04:15:36.668265+0000","last_clean":"2024-09-19T04:15:36.668265+0000","last_became_active":"2024-09-19T04:15:34.499498+0000","last_became_peered":"2024-09-19T04:15:34.499498+0000","last_unstale":"2024-09-19T04:15:36.668265+0000","last_undegraded":"2024-09-19T04:15:36.668265+0000","last_fullsized":"2024-09-19T04:15:36.668265+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:24:13.930894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.670114+0000","last_change":"2024-09-19T04:15:35.503212+0000","last_active":"2024-09-19T04:15:36.670114+0000","last_peered":"2024-09-19T04:15:36.670114+0000","last_clean":"2024-09-19T04:15:36.670114+0000","last_became_active":"2024-09-19T04:15:35.500722+0000","last_became_peered":"2024-09-19T04:15:35.500722+0000","last_unstale":"2024-09-19T04:15:36.670114+0000","last_undegraded":"2024-09-19T04:15:36.670114+0000","last_fullsized":"2024-09-19T04:15:36.670114+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:46:13.036328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.505677+0000","last_change":"2024-09-19T04:15:34.494542+0000","last_active":"2024-09-19T04:15:36.505677+0000","last_peered":"2024-09-19T04:15:36.505677+0000","last_clean":"2024-09-19T04:15:36.505677+0000","last_became_active":"2024-09-19T04:15:34.493758+0000","last_became_peered":"2024-09-19T04:15:34.493758+0000","last_unstale":"2024-09-19T04:15:36.505677+0000","last_undegraded":"2024-09-19T04:15:36.505677+0000","last_fullsized":"2024-09-19T04:15:36.505677+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:23:55.021296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502012+0000","last_change":"2024-09-19T04:15:35.494364+0000","last_active":"2024-09-19T04:15:36.502012+0000","last_peered":"2024-09-19T04:15:36.502012+0000","last_clean":"2024-09-19T04:15:36.502012+0000","last_became_active":"2024-09-19T04:15:35.494025+0000","last_became_peered":"2024-09-19T04:15:35.494025+0000","last_unstale":"2024-09-19T04:15:36.502012+0000","last_undegraded":"2024-09-19T04:15:36.502012+0000","last_fullsized":"2024-09-19T04:15:36.502012+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:45.939674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"52'8","reported_seq":22,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669237+0000","last_change":"2024-09-19T04:15:34.498119+0000","last_active":"2024-09-19T04:15:36.669237+0000","last_peered":"2024-09-19T04:15:36.669237+0000","last_clean":"2024-09-19T04:15:36.669237+0000","last_became_active":"2024-09-19T04:15:34.497139+0000","last_became_peered":"2024-09-19T04:15:34.497139+0000","last_unstale":"2024-09-19T04:15:36.669237+0000","last_undegraded":"2024-09-19T04:15:36.669237+0000","last_fullsized":"2024-09-19T04:15:36.669237+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:15.107129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668942+0000","last_change":"2024-09-19T04:15:35.503480+0000","last_active":"2024-09-19T04:15:36.668942+0000","last_peered":"2024-09-19T04:15:36.668942+0000","last_clean":"2024-09-19T04:15:36.668942+0000","last_became_active":"2024-09-19T04:15:35.503218+0000","last_became_peered":"2024-09-19T04:15:35.503218+0000","last_unstale":"2024-09-19T04:15:36.668942+0000","last_undegraded":"2024-09-19T04:15:36.668942+0000","last_fullsized":"2024-09-19T04:15:36.668942+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:38:22.481214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502602+0000","last_change":"2024-09-19T04:15:34.502315+0000","last_active":"2024-09-19T04:15:36.502602+0000","last_peered":"2024-09-19T04:15:36.502602+0000","last_clean":"2024-09-19T04:15:36.502602+0000","last_became_active":"2024-09-19T04:15:34.501968+0000","last_became_peered":"2024-09-19T04:15:34.501968+0000","last_unstale":"2024-09-19T04:15:36.502602+0000","last_undegraded":"2024-09-19T04:15:36.502602+0000","last_fullsized":"2024-09-19T04:15:36.502602+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:48.264206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.474978+0000","last_change":"2024-09-19T04:15:35.502706+0000","last_active":"2024-09-19T04:15:36.474978+0000","last_peered":"2024-09-19T04:15:36.474978+0000","last_clean":"2024-09-19T04:15:36.474978+0000","last_became_active":"2024-09-19T04:15:35.502182+0000","last_became_peered":"2024-09-19T04:15:35.502182+0000","last_unstale":"2024-09-19T04:15:36.474978+0000","last_undegraded":"2024-09-19T04:15:36.474978+0000","last_fullsized":"2024-09-19T04:15:36.474978+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:31.328898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669158+0000","last_change":"2024-09-19T04:15:34.502631+0000","last_active":"2024-09-19T04:15:36.669158+0000","last_peered":"2024-09-19T04:15:36.669158+0000","last_clean":"2024-09-19T04:15:36.669158+0000","last_became_active":"2024-09-19T04:15:34.502039+0000","last_became_peered":"2024-09-19T04:15:34.502039+0000","last_unstale":"2024-09-19T04:15:36.669158+0000","last_undegraded":"2024-09-19T04:15:36.669158+0000","last_fullsized":"2024-09-19T04:15:36.669158+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:15:20.632965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475499+0000","last_change":"2024-09-19T04:15:35.496671+0000","last_active":"2024-09-19T04:15:36.475499+0000","last_peered":"2024-09-19T04:15:36.475499+0000","last_clean":"2024-09-19T04:15:36.475499+0000","last_became_active":"2024-09-19T04:15:35.495942+0000","last_became_peered":"2024-09-19T04:15:35.495942+0000","last_unstale":"2024-09-19T04:15:36.475499+0000","last_undegraded":"2024-09-19T04:15:36.475499+0000","last_fullsized":"2024-09-19T04:15:36.475499+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:33.755646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"52'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668489+0000","last_change":"2024-09-19T04:15:34.500954+0000","last_active":"2024-09-19T04:15:36.668489+0000","last_peered":"2024-09-19T04:15:36.668489+0000","last_clean":"2024-09-19T04:15:36.668489+0000","last_became_active":"2024-09-19T04:15:34.500170+0000","last_became_peered":"2024-09-19T04:15:34.500170+0000","last_unstale":"2024-09-19T04:15:36.668489+0000","last_undegraded":"2024-09-19T04:15:36.668489+0000","last_fullsized":"2024-09-19T04:15:36.668489+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:32:36.874338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476710+0000","last_change":"2024-09-19T04:15:35.504235+0000","last_active":"2024-09-19T04:15:36.476710+0000","last_peered":"2024-09-19T04:15:36.476710+0000","last_clean":"2024-09-19T04:15:36.476710+0000","last_became_active":"2024-09-19T04:15:35.502159+0000","last_became_peered":"2024-09-19T04:15:35.502159+0000","last_unstale":"2024-09-19T04:15:36.476710+0000","last_undegraded":"2024-09-19T04:15:36.476710+0000","last_fullsized":"2024-09-19T04:15:36.476710+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:16.117299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668638+0000","last_change":"2024-09-19T04:15:34.503068+0000","last_active":"2024-09-19T04:15:36.668638+0000","last_peered":"2024-09-19T04:15:36.668638+0000","last_clean":"2024-09-19T04:15:36.668638+0000","last_became_active":"2024-09-19T04:15:34.502332+0000","last_became_peered":"2024-09-19T04:15:34.502332+0000","last_unstale":"2024-09-19T04:15:36.668638+0000","last_undegraded":"2024-09-19T04:15:36.668638+0000","last_fullsized":"2024-09-19T04:15:36.668638+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:18:01.529754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.478563+0000","last_change":"2024-09-19T04:15:35.499236+0000","last_active":"2024-09-19T04:15:36.478563+0000","last_peered":"2024-09-19T04:15:36.478563+0000","last_clean":"2024-09-19T04:15:36.478563+0000","last_became_active":"2024-09-19T04:15:35.498537+0000","last_became_peered":"2024-09-19T04:15:35.498537+0000","last_unstale":"2024-09-19T04:15:36.478563+0000","last_undegraded":"2024-09-19T04:15:36.478563+0000","last_fullsized":"2024-09-19T04:15:36.478563+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:40.969029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475731+0000","last_change":"2024-09-19T04:15:34.497471+0000","last_active":"2024-09-19T04:15:36.475731+0000","last_peered":"2024-09-19T04:15:36.475731+0000","last_clean":"2024-09-19T04:15:36.475731+0000","last_became_active":"2024-09-19T04:15:34.497206+0000","last_became_peered":"2024-09-19T04:15:34.497206+0000","last_unstale":"2024-09-19T04:15:36.475731+0000","last_undegraded":"2024-09-19T04:15:36.475731+0000","last_fullsized":"2024-09-19T04:15:36.475731+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:20.803393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668600+0000","last_change":"2024-09-19T04:15:35.523389+0000","last_active":"2024-09-19T04:15:36.668600+0000","last_peered":"2024-09-19T04:15:36.668600+0000","last_clean":"2024-09-19T04:15:36.668600+0000","last_became_active":"2024-09-19T04:15:35.522199+0000","last_became_peered":"2024-09-19T04:15:35.522199+0000","last_unstale":"2024-09-19T04:15:36.668600+0000","last_undegraded":"2024-09-19T04:15:36.668600+0000","last_fullsized":"2024-09-19T04:15:36.668600+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:28.365203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669711+0000","last_change":"2024-09-19T04:15:34.495285+0000","last_active":"2024-09-19T04:15:36.669711+0000","last_peered":"2024-09-19T04:15:36.669711+0000","last_clean":"2024-09-19T04:15:36.669711+0000","last_became_active":"2024-09-19T04:15:34.494887+0000","last_became_peered":"2024-09-19T04:15:34.494887+0000","last_unstale":"2024-09-19T04:15:36.669711+0000","last_undegraded":"2024-09-19T04:15:36.669711+0000","last_fullsized":"2024-09-19T04:15:36.669711+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:27:09.358061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501591+0000","last_change":"2024-09-19T04:15:35.510165+0000","last_active":"2024-09-19T04:15:36.501591+0000","last_peered":"2024-09-19T04:15:36.501591+0000","last_clean":"2024-09-19T04:15:36.501591+0000","last_became_active":"2024-09-19T04:15:35.509777+0000","last_became_peered":"2024-09-19T04:15:35.509777+0000","last_unstale":"2024-09-19T04:15:36.501591+0000","last_undegraded":"2024-09-19T04:15:36.501591+0000","last_fullsized":"2024-09-19T04:15:36.501591+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:52:54.906971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.502476+0000","last_change":"2024-09-19T04:15:34.497293+0000","last_active":"2024-09-19T04:15:36.502476+0000","last_peered":"2024-09-19T04:15:36.502476+0000","last_clean":"2024-09-19T04:15:36.502476+0000","last_became_active":"2024-09-19T04:15:34.496609+0000","last_became_peered":"2024-09-19T04:15:34.496609+0000","last_unstale":"2024-09-19T04:15:36.502476+0000","last_undegraded":"2024-09-19T04:15:36.502476+0000","last_fullsized":"2024-09-19T04:15:36.502476+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:02:37.548299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.476004+0000","last_change":"2024-09-19T04:15:35.501183+0000","last_active":"2024-09-19T04:15:36.476004+0000","last_peered":"2024-09-19T04:15:36.476004+0000","last_clean":"2024-09-19T04:15:36.476004+0000","last_became_active":"2024-09-19T04:15:35.500918+0000","last_became_peered":"2024-09-19T04:15:35.500918+0000","last_unstale":"2024-09-19T04:15:36.476004+0000","last_undegraded":"2024-09-19T04:15:36.476004+0000","last_fullsized":"2024-09-19T04:15:36.476004+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:04.480520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668132+0000","last_change":"2024-09-19T04:15:34.499089+0000","last_active":"2024-09-19T04:15:36.668132+0000","last_peered":"2024-09-19T04:15:36.668132+0000","last_clean":"2024-09-19T04:15:36.668132+0000","last_became_active":"2024-09-19T04:15:34.498691+0000","last_became_peered":"2024-09-19T04:15:34.498691+0000","last_unstale":"2024-09-19T04:15:36.668132+0000","last_undegraded":"2024-09-19T04:15:36.668132+0000","last_fullsized":"2024-09-19T04:15:36.668132+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:36:54.565804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.477115+0000","last_change":"2024-09-19T04:15:35.492266+0000","last_active":"2024-09-19T04:15:36.477115+0000","last_peered":"2024-09-19T04:15:36.477115+0000","last_clean":"2024-09-19T04:15:36.477115+0000","last_became_active":"2024-09-19T04:15:35.491625+0000","last_became_peered":"2024-09-19T04:15:35.491625+0000","last_unstale":"2024-09-19T04:15:36.477115+0000","last_undegraded":"2024-09-19T04:15:36.477115+0000","last_fullsized":"2024-09-19T04:15:36.477115+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:42:29.652305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"52'2","reported_seq":16,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668618+0000","last_change":"2024-09-19T04:15:34.502107+0000","last_active":"2024-09-19T04:15:36.668618+0000","last_peered":"2024-09-19T04:15:36.668618+0000","last_clean":"2024-09-19T04:15:36.668618+0000","last_became_active":"2024-09-19T04:15:34.501387+0000","last_became_peered":"2024-09-19T04:15:34.501387+0000","last_unstale":"2024-09-19T04:15:36.668618+0000","last_undegraded":"2024-09-19T04:15:36.668618+0000","last_fullsized":"2024-09-19T04:15:36.668618+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:14:06.890512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475578+0000","last_change":"2024-09-19T04:15:35.489714+0000","last_active":"2024-09-19T04:15:36.475578+0000","last_peered":"2024-09-19T04:15:36.475578+0000","last_clean":"2024-09-19T04:15:36.475578+0000","last_became_active":"2024-09-19T04:15:35.488954+0000","last_became_peered":"2024-09-19T04:15:35.488954+0000","last_unstale":"2024-09-19T04:15:36.475578+0000","last_undegraded":"2024-09-19T04:15:36.475578+0000","last_fullsized":"2024-09-19T04:15:36.475578+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:20:02.093430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.506685+0000","last_change":"2024-09-19T04:15:34.497076+0000","last_active":"2024-09-19T04:15:36.506685+0000","last_peered":"2024-09-19T04:15:36.506685+0000","last_clean":"2024-09-19T04:15:36.506685+0000","last_became_active":"2024-09-19T04:15:34.495768+0000","last_became_peered":"2024-09-19T04:15:34.495768+0000","last_unstale":"2024-09-19T04:15:36.506685+0000","last_undegraded":"2024-09-19T04:15:36.506685+0000","last_fullsized":"2024-09-19T04:15:36.506685+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:26:00.789775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669641+0000","last_change":"2024-09-19T04:15:35.485265+0000","last_active":"2024-09-19T04:15:36.669641+0000","last_peered":"2024-09-19T04:15:36.669641+0000","last_clean":"2024-09-19T04:15:36.669641+0000","last_became_active":"2024-09-19T04:15:35.484517+0000","last_became_peered":"2024-09-19T04:15:35.484517+0000","last_unstale":"2024-09-19T04:15:36.669641+0000","last_undegraded":"2024-09-19T04:15:36.669641+0000","last_fullsized":"2024-09-19T04:15:36.669641+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:13.159877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501808+0000","last_change":"2024-09-19T04:15:34.500442+0000","last_active":"2024-09-19T04:15:36.501808+0000","last_peered":"2024-09-19T04:15:36.501808+0000","last_clean":"2024-09-19T04:15:36.501808+0000","last_became_active":"2024-09-19T04:15:34.499887+0000","last_became_peered":"2024-09-19T04:15:34.499887+0000","last_unstale":"2024-09-19T04:15:36.501808+0000","last_undegraded":"2024-09-19T04:15:36.501808+0000","last_fullsized":"2024-09-19T04:15:36.501808+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:18.087217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501548+0000","last_change":"2024-09-19T04:15:35.509790+0000","last_active":"2024-09-19T04:15:36.501548+0000","last_peered":"2024-09-19T04:15:36.501548+0000","last_clean":"2024-09-19T04:15:36.501548+0000","last_became_active":"2024-09-19T04:15:35.503286+0000","last_became_peered":"2024-09-19T04:15:35.503286+0000","last_unstale":"2024-09-19T04:15:36.501548+0000","last_undegraded":"2024-09-19T04:15:36.501548+0000","last_fullsized":"2024-09-19T04:15:36.501548+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:06.013531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669711+0000","last_change":"2024-09-19T04:15:34.496074+0000","last_active":"2024-09-19T04:15:36.669711+0000","last_peered":"2024-09-19T04:15:36.669711+0000","last_clean":"2024-09-19T04:15:36.669711+0000","last_became_active":"2024-09-19T04:15:34.495301+0000","last_became_peered":"2024-09-19T04:15:34.495301+0000","last_unstale":"2024-09-19T04:15:36.669711+0000","last_undegraded":"2024-09-19T04:15:36.669711+0000","last_fullsized":"2024-09-19T04:15:36.669711+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:46:31.832303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.477927+0000","last_change":"2024-09-19T04:15:35.481125+0000","last_active":"2024-09-19T04:15:36.477927+0000","last_peered":"2024-09-19T04:15:36.477927+0000","last_clean":"2024-09-19T04:15:36.477927+0000","last_became_active":"2024-09-19T04:15:35.480407+0000","last_became_peered":"2024-09-19T04:15:35.480407+0000","last_unstale":"2024-09-19T04:15:36.477927+0000","last_undegraded":"2024-09-19T04:15:36.477927+0000","last_fullsized":"2024-09-19T04:15:36.477927+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:20:15.219264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.669809+0000","last_change":"2024-09-19T04:15:34.495587+0000","last_active":"2024-09-19T04:15:36.669809+0000","last_peered":"2024-09-19T04:15:36.669809+0000","last_clean":"2024-09-19T04:15:36.669809+0000","last_became_active":"2024-09-19T04:15:34.495193+0000","last_became_peered":"2024-09-19T04:15:34.495193+0000","last_unstale":"2024-09-19T04:15:36.669809+0000","last_undegraded":"2024-09-19T04:15:36.669809+0000","last_fullsized":"2024-09-19T04:15:36.669809+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:05:03.470350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.501331+0000","last_change":"2024-09-19T04:15:35.490280+0000","last_active":"2024-09-19T04:15:36.501331+0000","last_peered":"2024-09-19T04:15:36.501331+0000","last_clean":"2024-09-19T04:15:36.501331+0000","last_became_active":"2024-09-19T04:15:35.489595+0000","last_became_peered":"2024-09-19T04:15:35.489595+0000","last_unstale":"2024-09-19T04:15:36.501331+0000","last_undegraded":"2024-09-19T04:15:36.501331+0000","last_fullsized":"2024-09-19T04:15:36.501331+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:12:28.688996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.475178+0000","last_change":"2024-09-19T04:15:34.499535+0000","last_active":"2024-09-19T04:15:36.475178+0000","last_peered":"2024-09-19T04:15:36.475178+0000","last_clean":"2024-09-19T04:15:36.475178+0000","last_became_active":"2024-09-19T04:15:34.498704+0000","last_became_peered":"2024-09-19T04:15:34.498704+0000","last_unstale":"2024-09-19T04:15:36.475178+0000","last_undegraded":"2024-09-19T04:15:36.475178+0000","last_fullsized":"2024-09-19T04:15:36.475178+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:53:19.008661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.670501+0000","last_change":"2024-09-19T04:15:35.500268+0000","last_active":"2024-09-19T04:15:36.670501+0000","last_peered":"2024-09-19T04:15:36.670501+0000","last_clean":"2024-09-19T04:15:36.670501+0000","last_became_active":"2024-09-19T04:15:35.499796+0000","last_became_peered":"2024-09-19T04:15:35.499796+0000","last_unstale":"2024-09-19T04:15:36.670501+0000","last_undegraded":"2024-09-19T04:15:36.670501+0000","last_fullsized":"2024-09-19T04:15:36.670501+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:44:22.294320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.477259+0000","last_change":"2024-09-19T04:15:34.496246+0000","last_active":"2024-09-19T04:15:36.477259+0000","last_peered":"2024-09-19T04:15:36.477259+0000","last_clean":"2024-09-19T04:15:36.477259+0000","last_became_active":"2024-09-19T04:15:34.495515+0000","last_became_peered":"2024-09-19T04:15:34.495515+0000","last_unstale":"2024-09-19T04:15:36.477259+0000","last_undegraded":"2024-09-19T04:15:36.477259+0000","last_fullsized":"2024-09-19T04:15:36.477259+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:30.318728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.668156+0000","last_change":"2024-09-19T04:15:35.514179+0000","last_active":"2024-09-19T04:15:36.668156+0000","last_peered":"2024-09-19T04:15:36.668156+0000","last_clean":"2024-09-19T04:15:36.668156+0000","last_became_active":"2024-09-19T04:15:35.513473+0000","last_became_peered":"2024-09-19T04:15:35.513473+0000","last_unstale":"2024-09-19T04:15:36.668156+0000","last_undegraded":"2024-09-19T04:15:36.668156+0000","last_fullsized":"2024-09-19T04:15:36.668156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:34.451892+0000","last_clean_scrub_stamp":"2024-09-19T04:15:34.451892+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:55:49.274377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"53'1","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.504712+0000","last_change":"2024-09-19T04:15:34.498473+0000","last_active":"2024-09-19T04:15:36.504712+0000","last_peered":"2024-09-19T04:15:36.504712+0000","last_clean":"2024-09-19T04:15:36.504712+0000","last_became_active":"2024-09-19T04:15:34.498203+0000","last_became_peered":"2024-09-19T04:15:34.498203+0000","last_unstale":"2024-09-19T04:15:36.504712+0000","last_undegraded":"2024-09-19T04:15:36.504712+0000","last_fullsized":"2024-09-19T04:15:36.504712+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":50,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:15:33.441722+0000","last_clean_scrub_stamp":"2024-09-19T04:15:33.441722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:26:24.120385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":229,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-19T04:15:36.505501+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:15:36.505501+0000","last_peered":"2024-09-19T04:15:36.505501+0000","last_clean":"2024-09-19T04:15:36.505501+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:15:36.505501+0000","last_undegraded":"2024-09-19T04:15:36.505501+0000","last_fullsized":"2024-09-19T04:15:36.505501+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542176,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1228800,"data_stored":871426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542177,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":634880,"data_stored":281586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27644,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":626688,"data_stored":280934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542176,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":630784,"data_stored":281516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542176,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27664,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344176,"statfs":{"total":96636764160,"available":96608436224,"internally_reserved":0,"allocated":643072,"data_stored":282568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542177,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":282546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":634880,"data_stored":281094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542177,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1228800,"data_stored":871358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:15:39.840 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:15:39.840 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T04:15:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7407748640 1 Processor -- start 2024-09-19T04:15:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7407748640 1 -- start start 2024-09-19T04:15:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7407748640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74000c7c80 0x7f74000b3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7407748640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000b0450 0x7f74000b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7407748640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f740006bf10 con 0x7f740015c3e0 2024-09-19T04:15:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7407748640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f740015c7c0 con 0x7f74000c7c80 2024-09-19T04:15:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7407748640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f740006bcc0 con 0x7f74000b0450 2024-09-19T04:15:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f74054e4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f740015c3e0 0x7f74000c7570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47292/0 (socket says 172.21.15.76:47292) 2024-09-19T04:15:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f74054e4640 1 -- 172.21.15.76:0/1878817660 learned_addr learned my addr 172.21.15.76:0/1878817660 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:40.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7404ce3640 1 --2- 172.21.15.76:0/1878817660 >> v2:172.21.15.76:3300/0 conn(0x7f74000c7c80 0x7f74000b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7405ce5640 1 --2- 172.21.15.76:0/1878817660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000b0450 0x7f74000b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7404ce3640 1 -- 172.21.15.76:0/1878817660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000b0450 msgr2=0x7f74000b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7404ce3640 1 --2- 172.21.15.76:0/1878817660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000b0450 0x7f74000b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7404ce3640 1 -- 172.21.15.76:0/1878817660 >> v1:172.21.15.76:6789/0 conn(0x7f740015c3e0 legacy=0x7f74000c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7404ce3640 1 -- 172.21.15.76:0/1878817660 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f740006c1f0 con 0x7f74000c7c80 2024-09-19T04:15:40.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.005+0000 7f7405ce5640 1 --2- 172.21.15.76:0/1878817660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000b0450 0x7f74000b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7404ce3640 1 --2- 172.21.15.76:0/1878817660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000c7c80 0x7f74000b3540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f73f40097c0 tx=0x7f73f401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a73300bb71be5f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f73fffff640 1 -- 172.21.15.76:0/1878817660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40b0070 con 0x7f74000c7c80 2024-09-19T04:15:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f73fffff640 1 -- 172.21.15.76:0/1878817660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f40ab600 con 0x7f74000c7c80 2024-09-19T04:15:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f73fffff640 1 -- 172.21.15.76:0/1878817660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40ab9d0 con 0x7f74000c7c80 2024-09-19T04:15:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- 172.21.15.76:0/1878817660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000c7c80 msgr2=0x7f74000b3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 --2- 172.21.15.76:0/1878817660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000c7c80 0x7f74000b3540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f73f40097c0 tx=0x7f73f401c410 comp rx=0 tx=0).stop 2024-09-19T04:15:40.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- 172.21.15.76:0/1878817660 shutdown_connections 2024-09-19T04:15:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 --2- 172.21.15.76:0/1878817660 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000b0450 0x7f74000b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 --2- 172.21.15.76:0/1878817660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000c7c80 0x7f74000b3540 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- 172.21.15.76:0/1878817660 >> 172.21.15.76:0/1878817660 conn(0x7f740009aa00 msgr2=0x7f74000b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- 172.21.15.76:0/1878817660 shutdown_connections 2024-09-19T04:15:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- 172.21.15.76:0/1878817660 wait complete. 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 Processor -- start 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- start start 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000b0450 0x7f7400157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000c7c80 0x7f740014bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f740015c3e0 0x7f74001544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74000b1b00 con 0x7f74000b0450 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f74000b1980 con 0x7f740015c3e0 2024-09-19T04:15:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7400168eb0 con 0x7f74000c7c80 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000b0450 0x7f7400157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000b0450 0x7f7400157670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35146/0 (socket says 172.21.15.76:35146) 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 -- 172.21.15.76:0/2411179204 learned_addr learned my addr 172.21.15.76:0/2411179204 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 -- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000c7c80 msgr2=0x7f740014bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7404ce3640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000c7c80 0x7f740014bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7405ce5640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f740015c3e0 0x7f74001544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000c7c80 0x7f740014bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 -- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f740015c3e0 msgr2=0x7f74001544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f740015c3e0 0x7f74001544d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 -- 172.21.15.76:0/2411179204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7400156970 con 0x7f74000b0450 2024-09-19T04:15:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f74054e4640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000b0450 0x7f7400157670 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f73f800cb10 tx=0x7f73f800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f73fdffb640 1 -- 172.21.15.76:0/2411179204 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8017070 con 0x7f74000b0450 2024-09-19T04:15:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f73fdffb640 1 -- 172.21.15.76:0/2411179204 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f800ac70 con 0x7f74000b0450 2024-09-19T04:15:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f73fdffb640 1 -- 172.21.15.76:0/2411179204 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f8012620 con 0x7f74000b0450 2024-09-19T04:15:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.009+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7400159a40 con 0x7f74000b0450 2024-09-19T04:15:40.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.013+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f740015a000 con 0x7f74000b0450 2024-09-19T04:15:40.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.013+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74000b09a0 con 0x7f74000b0450 2024-09-19T04:15:40.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.013+0000 7f73fdffb640 1 -- 172.21.15.76:0/2411179204 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f73f801a070 con 0x7f74000b0450 2024-09-19T04:15:40.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.013+0000 7f73fdffb640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f73ec0678c0 0x7f73ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.013+0000 7f7404ce3640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f73ec0678c0 0x7f73ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.013+0000 7f73fdffb640 1 -- 172.21.15.76:0/2411179204 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f73f8085b80 con 0x7f74000b0450 2024-09-19T04:15:40.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.013+0000 7f7404ce3640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f73ec0678c0 0x7f73ec069d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73f40ad000 tx=0x7f73f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:40.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.017+0000 7f73fdffb640 1 -- 172.21.15.76:0/2411179204 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f73f80565c0 con 0x7f74000b0450 2024-09-19T04:15:40.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.237+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7400155e70 con 0x7f74000b0450 2024-09-19T04:15:40.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.241+0000 7f73fdffb640 1 -- 172.21.15.76:0/2411179204 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f73f8055ec0 con 0x7f74000b0450 2024-09-19T04:15:40.245 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-19T04:15:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:15:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2024-09-19T04:15:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2024-09-19T04:15:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T04:15:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f73ec0678c0 msgr2=0x7f73ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f73ec0678c0 0x7f73ec069d80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f73f40ad000 tx=0x7f73f4008040 comp rx=0 tx=0).stop 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000b0450 msgr2=0x7f7400157670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000b0450 0x7f7400157670 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f73f800cb10 tx=0x7f73f800a060 comp rx=0 tx=0).stop 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 shutdown_connections 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f73ec0678c0 0x7f73ec069d80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f740015c3e0 0x7f74001544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74000c7c80 0x7f740014bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 --2- 172.21.15.76:0/2411179204 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74000b0450 0x7f7400157670 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 >> 172.21.15.76:0/2411179204 conn(0x7f740009aa00 msgr2=0x7f74000b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 shutdown_connections 2024-09-19T04:15:40.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.245+0000 7f7407748640 1 -- 172.21.15.76:0/2411179204 wait complete. 2024-09-19T04:15:40.264 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.373+0000 7f9f68c3a640 1 Processor -- start 2024-09-19T04:15:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.373+0000 7f9f68c3a640 1 -- start start 2024-09-19T04:15:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f68c3a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f6415bec0 0x7f9f640ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f68c3a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9f6415c2a0 0x7f9f640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f68c3a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f640b3290 con 0x7f9f640af2a0 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f68c3a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9f640b3570 con 0x7f9f6415bec0 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f68c3a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9f640b3850 con 0x7f9f6415c2a0 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f6415bec0 0x7f9f640ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9f6415bec0 0x7f9f640ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35160/0 (socket says 172.21.15.76:35160) 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 -- 172.21.15.76:0/3157681070 learned_addr learned my addr 172.21.15.76:0/3157681070 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 -- 172.21.15.76:0/3157681070 >> v2:172.21.15.116:3300/0 conn(0x7f9f6415c2a0 msgr2=0x7f9f640aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f59d9b640 1 --2- 172.21.15.76:0/3157681070 >> v2:172.21.15.116:3300/0 conn(0x7f9f6415c2a0 0x7f9f640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 --2- 172.21.15.76:0/3157681070 >> v2:172.21.15.116:3300/0 conn(0x7f9f6415c2a0 0x7f9f640aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 -- 172.21.15.76:0/3157681070 >> v1:172.21.15.116:6789/0 conn(0x7f9f640af2a0 legacy=0x7f9f6415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:40.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 -- 172.21.15.76:0/3157681070 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f640b3b30 con 0x7f9f6415bec0 2024-09-19T04:15:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f6259c640 1 --2- 172.21.15.76:0/3157681070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f640ad200 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9f640ae510 tx=0x7f9f4c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee8b1a9022e3dcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f61d9b640 1 -- 172.21.15.76:0/3157681070 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0aa610 con 0x7f9f6415bec0 2024-09-19T04:15:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f61d9b640 1 -- 172.21.15.76:0/3157681070 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f4c0af070 con 0x7f9f6415bec0 2024-09-19T04:15:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.377+0000 7f9f61d9b640 1 -- 172.21.15.76:0/3157681070 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0aa9d0 con 0x7f9f6415bec0 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- 172.21.15.76:0/3157681070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 msgr2=0x7f9f640ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/3157681070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f640ad200 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9f640ae510 tx=0x7f9f4c01c4b0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- 172.21.15.76:0/3157681070 shutdown_connections 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/3157681070 >> v2:172.21.15.116:3300/0 conn(0x7f9f6415c2a0 0x7f9f640aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/3157681070 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f640ad200 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- 172.21.15.76:0/3157681070 >> 172.21.15.76:0/3157681070 conn(0x7f9f640b1f90 msgr2=0x7f9f640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- 172.21.15.76:0/3157681070 shutdown_connections 2024-09-19T04:15:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- 172.21.15.76:0/3157681070 wait complete. 2024-09-19T04:15:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 Processor -- start 2024-09-19T04:15:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- start start 2024-09-19T04:15:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f640af2a0 0x7f9f6415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f6415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f6415c2a0 0x7f9f641400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640ad740 con 0x7f9f6415bec0 2024-09-19T04:15:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9f640af710 con 0x7f9f640af2a0 2024-09-19T04:15:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9f64169000 con 0x7f9f6415c2a0 2024-09-19T04:15:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f6415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f6415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35172/0 (socket says 172.21.15.76:35172) 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 -- 172.21.15.76:0/1520488784 learned_addr learned my addr 172.21.15.76:0/1520488784 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f6259c640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f640af2a0 0x7f9f6415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f62d9d640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f6415c2a0 0x7f9f641400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 -- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f6415c2a0 msgr2=0x7f9f641400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f6415c2a0 0x7f9f641400a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 -- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f640af2a0 msgr2=0x7f9f6415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f640af2a0 0x7f9f6415c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 -- 172.21.15.76:0/1520488784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f6414fbc0 con 0x7f9f6415bec0 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f62d9d640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f6415c2a0 0x7f9f641400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:40.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f59d9b640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f6415d630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c5e0 tx=0x7f9f5400caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f5b7fe640 1 -- 172.21.15.76:0/1520488784 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54015070 con 0x7f9f6415bec0 2024-09-19T04:15:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f5b7fe640 1 -- 172.21.15.76:0/1520488784 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f54004220 con 0x7f9f6415bec0 2024-09-19T04:15:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f6415cc20 con 0x7f9f6415bec0 2024-09-19T04:15:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f5b7fe640 1 -- 172.21.15.76:0/1520488784 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f54003620 con 0x7f9f6415bec0 2024-09-19T04:15:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.381+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f6414f8e0 con 0x7f9f6415bec0 2024-09-19T04:15:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.385+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f640ad200 con 0x7f9f6415bec0 2024-09-19T04:15:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.385+0000 7f9f5b7fe640 1 -- 172.21.15.76:0/1520488784 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f540037e0 con 0x7f9f6415bec0 2024-09-19T04:15:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.385+0000 7f9f5b7fe640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f440678c0 0x7f9f44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.385+0000 7f9f5b7fe640 1 -- 172.21.15.76:0/1520488784 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9f54085bb0 con 0x7f9f6415bec0 2024-09-19T04:15:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.385+0000 7f9f6259c640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f440678c0 0x7f9f44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.385+0000 7f9f6259c640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f440678c0 0x7f9f44069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0ae000 tx=0x7f9f4c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.389+0000 7f9f5b7fe640 1 -- 172.21.15.76:0/1520488784 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9f540565f0 con 0x7f9f6415bec0 2024-09-19T04:15:40.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f6406fcd0 con 0x7f9f6415bec0 2024-09-19T04:15:40.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f5b7fe640 1 -- 172.21.15.76:0/1520488784 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+5229 (secure 0 0 0) 0x7f9f54055ef0 con 0x7f9f6415bec0 2024-09-19T04:15:40.579 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:40.579 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":29,"btime":"2024-09-19T04:15:38:482457+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":5496,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:15:37.839123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502]},"id":3}]} 2024-09-19T04:15:40.579 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 29 2024-09-19T04:15:40.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f440678c0 msgr2=0x7f9f44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f440678c0 0x7f9f44069d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0ae000 tx=0x7f9f4c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:15:40.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 msgr2=0x7f9f6415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f6415d630 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9f5400c5e0 tx=0x7f9f5400caa0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 shutdown_connections 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f440678c0 0x7f9f44069d80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9f6415c2a0 0x7f9f641400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9f6415bec0 0x7f9f6415d630 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 --2- 172.21.15.76:0/1520488784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f640af2a0 0x7f9f6415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 >> 172.21.15.76:0/1520488784 conn(0x7f9f640b1f90 msgr2=0x7f9f640cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 shutdown_connections 2024-09-19T04:15:40.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:40.577+0000 7f9f68c3a640 1 -- 172.21.15.76:0/1520488784 wait complete. 2024-09-19T04:15:40.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:15:34.831676+0000', 'modified': '2024-09-19T04:15:37.839123+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5502': {'gid': 5502, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3635635274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3635635274}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3635635274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5502, 'qdb_cluster': [5502]} 2024-09-19T04:15:40.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:15:40.999 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:15:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f178dff1640 1 Processor -- start 2024-09-19T04:15:41.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f178dff1640 1 -- start start 2024-09-19T04:15:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f178dff1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 0x7f17880d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f178dff1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f17880cf620 0x7f178806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f178dff1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880d32b0 con 0x7f17880d2e90 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f178dff1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f17880d0300 con 0x7f17880cf620 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f178dff1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f17880b0f60 con 0x7f1788161a60 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f17880d2e90 0x7f17880cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47296/0 (socket says 172.21.15.76:47296) 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 -- 172.21.15.76:0/3426003554 learned_addr learned my addr 172.21.15.76:0/3426003554 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1787fff640 1 --2- 172.21.15.76:0/3426003554 >> v2:172.21.15.116:3300/0 conn(0x7f17880cf620 0x7f178806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17877fe640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 0x7f17880d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 11384628 0 0) 0x7f17880d32b0 con 0x7f17880d2e90 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1770003530 con 0x7f17880d2e90 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 486814287 0 0) 0x7f1770003530 con 0x7f17880d2e90 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 msgr2=0x7f17880d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 0x7f17880d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 >> v2:172.21.15.116:3300/0 conn(0x7f17880cf620 msgr2=0x7f178806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 --2- 172.21.15.76:0/3426003554 >> v2:172.21.15.116:3300/0 conn(0x7f17880cf620 0x7f178806d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880cfad0 con 0x7f17880d2e90 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f177c001680 con 0x7f17880d2e90 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1787fff640 1 --2- 172.21.15.76:0/3426003554 >> v2:172.21.15.116:3300/0 conn(0x7f17880cf620 0x7f178806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 >> v1:172.21.15.76:6789/0 conn(0x7f17880d2e90 legacy=0x7f17880cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 0x7f17700039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f1770004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1770004d50 0x7f17700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17880d0300 con 0x7f1770003f40 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f17880b0f60 con 0x7f1770004d50 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f177000bad0 con 0x7f1788161a60 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f1770004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f17877fe640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 0x7f17700039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1787fff640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1770004d50 0x7f17700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 -- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 msgr2=0x7f17700039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1788161a60 0x7f17700039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 -- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1770004d50 msgr2=0x7f17700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1770004d50 0x7f17700051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.129+0000 7f1786ffd640 1 -- 172.21.15.76:0/3426003554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1770003530 con 0x7f1770003f40 2024-09-19T04:15:41.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f1786ffd640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f1770004810 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f177c00c210 tx=0x7f177c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1572a78ddf1fbab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c007d60 con 0x7f1770003f40 2024-09-19T04:15:41.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f177c0a9020 con 0x7f1770003f40 2024-09-19T04:15:41.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f17867fc640 1 -- 172.21.15.76:0/3426003554 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f177c005b20 con 0x7f1770003f40 2024-09-19T04:15:41.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- 172.21.15.76:0/3426003554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 msgr2=0x7f1770004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 --2- 172.21.15.76:0/3426003554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f1770004810 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f177c00c210 tx=0x7f177c01cc30 comp rx=0 tx=0).stop 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f1787fff640 1 -- 172.21.15.76:0/3426003554 reap_dead start 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- 172.21.15.76:0/3426003554 shutdown_connections 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- 172.21.15.76:0/3426003554 >> 172.21.15.76:0/3426003554 conn(0x7f178808bf90 msgr2=0x7f178809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- 172.21.15.76:0/3426003554 shutdown_connections 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- 172.21.15.76:0/3426003554 wait complete. 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 Processor -- start 2024-09-19T04:15:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- start start 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f178815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17881f7980 0x7f17880cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f17880ccdf0 0x7f178815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f178806e400 con 0x7f1770003f40 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f178806d190 con 0x7f17881f7980 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.133+0000 7f178dff1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f178806e580 con 0x7f17880ccdf0 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f178815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f178815a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35204/0 (socket says 172.21.15.76:35204) 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 -- 172.21.15.76:0/2564600258 learned_addr learned my addr 172.21.15.76:0/2564600258 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f1787fff640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f17880ccdf0 0x7f178815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f1786ffd640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17881f7980 0x7f17880cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 -- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f17880ccdf0 msgr2=0x7f178815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f17880ccdf0 0x7f178815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 -- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17881f7980 msgr2=0x7f17880cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17881f7980 0x7f17880cc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 -- 172.21.15.76:0/2564600258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880d0ca0 con 0x7f1770003f40 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f1787fff640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f17880ccdf0 0x7f178815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f1786ffd640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17881f7980 0x7f17880cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f17877fe640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f178815a9f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f17780115e0 tx=0x7f1778011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f176ffff640 1 -- 172.21.15.76:0/2564600258 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778015870 con 0x7f1770003f40 2024-09-19T04:15:41.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f176ffff640 1 -- 172.21.15.76:0/2564600258 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778015a10 con 0x7f1770003f40 2024-09-19T04:15:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f176ffff640 1 -- 172.21.15.76:0/2564600258 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778015dc0 con 0x7f1770003f40 2024-09-19T04:15:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17880cd300 con 0x7f1770003f40 2024-09-19T04:15:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17880b8440 con 0x7f1770003f40 2024-09-19T04:15:41.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f176ffff640 1 -- 172.21.15.76:0/2564600258 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f177800f480 con 0x7f1770003f40 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f176ffff640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17740678c0 0x7f1774069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f1786ffd640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17740678c0 0x7f1774069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f176ffff640 1 -- 172.21.15.76:0/2564600258 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f17780866c0 con 0x7f1770003f40 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.137+0000 7f1786ffd640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17740678c0 0x7f1774069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f177c0aa000 tx=0x7f177c000ff0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 Processor -- start 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- start start 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b6ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b6b290 0x562da7b6b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562da7a52b60 con 0x562da7b6b290 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562da79b9c70 con 0x562da7bb4b20 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562da795fed0 con 0x562da7bb5440 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b6b290 0x562da7b6b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b6b290 0x562da7b6b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35208/0 (socket says 172.21.15.76:35208) 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 -- 172.21.15.76:0/496897639 learned_addr learned my addr 172.21.15.76:0/496897639 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17877fe640 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1786ffd640 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b6ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 -- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 msgr2=0x562da7b6ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b6ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 -- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 msgr2=0x562da7bb4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 -- 172.21.15.76:0/496897639 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da7971e50 con 0x562da7b6b290 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17877fe640 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b6b290 0x562da7b6b670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f17880b10e0 tx=0x7f1780003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27b7ce0089c2124 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f176d7fa640 1 -- 172.21.15.76:0/496897639 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17800044d0 con 0x562da7b6b290 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f176d7fa640 1 -- 172.21.15.76:0/496897639 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178001a070 con 0x562da7b6b290 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f176d7fa640 1 -- 172.21.15.76:0/496897639 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780015420 con 0x562da7b6b290 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- 172.21.15.76:0/496897639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b6b290 msgr2=0x562da7b6b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b6b290 0x562da7b6b670 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f17880b10e0 tx=0x7f1780003f90 comp rx=0 tx=0).stop 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- 172.21.15.76:0/496897639 shutdown_connections 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b6b290 0x562da7b6b670 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b6ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- 172.21.15.76:0/496897639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- 172.21.15.76:0/496897639 >> 172.21.15.76:0/496897639 conn(0x562da7b71600 msgr2=0x562da7b69630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- 172.21.15.76:0/496897639 shutdown_connections 2024-09-19T04:15:41.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- 172.21.15.76:0/496897639 wait complete. 2024-09-19T04:15:41.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- start start 2024-09-19T04:15:41.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 Processor -- start 2024-09-19T04:15:41.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7b6fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b75cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7b76230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562da7a1bb00 con 0x562da7b70390 2024-09-19T04:15:41.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562da7994360 con 0x562da7bb4b20 2024-09-19T04:15:41.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562da79912d0 con 0x562da7bb5440 2024-09-19T04:15:41.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f17910ad000 10 client.0 _ll_register_callbacks cb 0x562da7b56820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7b76230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7b76230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35214/0 (socket says 172.21.15.76:35214) 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.141+0000 7f1787fff640 1 -- 172.21.15.76:0/2224947248 learned_addr learned my addr 172.21.15.76:0/2224947248 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17877fe640 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7b6fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f1786ffd640 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b75cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f1787fff640 1 -- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 msgr2=0x562da7b75cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f1787fff640 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b75cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f1787fff640 1 -- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 msgr2=0x562da7b6fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f1787fff640 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7b6fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f1787fff640 1 -- 172.21.15.76:0/2224947248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da7960120 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f1787fff640 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7b76230 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1780003bc0 tx=0x7f1780004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3a691758a98e81d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 1 -- 172.21.15.76:0/2224947248 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1780015830 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 1 -- 172.21.15.76:0/2224947248 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f17800159d0 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f178001a070 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 1 -- 172.21.15.76:0/2224947248 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1780031050 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 10 client.5619 fetch_fsmap learned FSMap version 29 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x562da7959450 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 1 -- 172.21.15.76:0/2224947248 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f178001a070 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 1 -- 172.21.15.76:0/2224947248 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) ==== 3502+0+0 (secure 0 0 0) 0x7f1780034020 con 0x562da7b70390 2024-09-19T04:15:41.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 10 client.5619 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 10 client.5619 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 20 client.5619 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7bc3280 0x562da7bc5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 4 client.5619 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562da7953c00 con 0x562da7bc3280 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17877fe640 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7bc3280 0x562da7bc5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 10 client.5619 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17877fe640 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7bc3280 0x562da7bc5670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38de1fa79a689a8c server_cookie=9bc736ebf8b30f8d in_seq=0 out_seq=0 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 1 -- 172.21.15.76:0/2224947248 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562da7953c00 con 0x562da7bc3280 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 10 client.5619 handle_command_reply: tid=0 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 client.5619 shutdown 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 msgr2=0x562da7b76230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7b76230 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1780003bc0 tx=0x7f1780004a50 comp rx=0 tx=0).stop 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 shutdown_connections 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7b76230 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7b75cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7bc3280 0x562da7bc5670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 --2- 172.21.15.76:0/2224947248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7b6fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17667fc640 0 client.5619 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 >> 172.21.15.76:0/2224947248 conn(0x562da7b6b290 msgr2=0x562da7b69d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 shutdown_connections 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 1 -- 172.21.15.76:0/2224947248 wait complete. 2024-09-19T04:15:41.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.145+0000 7f17910ad000 20 client.5619 trim_cache size 0 max 16384 2024-09-19T04:15:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 Processor -- start 2024-09-19T04:15:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- start start 2024-09-19T04:15:41.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7bb3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7bc5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f17882036f0 con 0x562da7b70390 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f17880b96f0 con 0x562da7bb4b20 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1788203140 con 0x562da7bb5440 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7bb3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7bb3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35218/0 (socket says 172.21.15.76:35218) 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 -- 172.21.15.76:0/1413818554 learned_addr learned my addr 172.21.15.76:0/1413818554 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 -- 172.21.15.76:0/1413818554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 msgr2=0x562da7bc5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 --2- 172.21.15.76:0/1413818554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7bc5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 -- 172.21.15.76:0/1413818554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 msgr2=0x562da7bb4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 --2- 172.21.15.76:0/1413818554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 -- 172.21.15.76:0/1413818554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da7940fd0 con 0x562da7b70390 2024-09-19T04:15:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17877fe640 1 --2- 172.21.15.76:0/1413818554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7bb3cf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1778056700 tx=0x7f177801f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ce4db2ad64fb0e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f176cff9640 1 -- 172.21.15.76:0/1413818554 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778054d00 con 0x562da7b70390 2024-09-19T04:15:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f176cff9640 1 -- 172.21.15.76:0/1413818554 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778083070 con 0x562da7b70390 2024-09-19T04:15:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f176cff9640 1 -- 172.21.15.76:0/1413818554 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1778056c60 con 0x562da7b70390 2024-09-19T04:15:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- 172.21.15.76:0/1413818554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 msgr2=0x562da7bb3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- 172.21.15.76:0/1413818554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7bb3cf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1778056700 tx=0x7f177801f040 comp rx=0 tx=0).stop 2024-09-19T04:15:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- 172.21.15.76:0/1413818554 shutdown_connections 2024-09-19T04:15:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- 172.21.15.76:0/1413818554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb5440 0x562da7bc5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- 172.21.15.76:0/1413818554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7bb4b20 0x562da7bb4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- 172.21.15.76:0/1413818554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7b70390 0x562da7bb3cf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- 172.21.15.76:0/1413818554 >> 172.21.15.76:0/1413818554 conn(0x562da7b6b290 msgr2=0x562da7b6f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- 172.21.15.76:0/1413818554 shutdown_connections 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- 172.21.15.76:0/1413818554 wait complete. 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- start start 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 Processor -- start 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7b70390 0x562da7bc8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb4b20 0x562da7be3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7bb5440 0x562da7be3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562da79b8f80 con 0x562da7bb5440 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562da7971ca0 con 0x562da7b70390 2024-09-19T04:15:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562da79be2a0 con 0x562da7bb4b20 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f17910ad000 10 client.0 _ll_register_callbacks cb 0x7f1780003ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f1787fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7bb5440 0x562da7be3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f1787fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7bb5440 0x562da7be3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35222/0 (socket says 172.21.15.76:35222) 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.165+0000 7f1787fff640 1 -- 172.21.15.76:0/3717828338 learned_addr learned my addr 172.21.15.76:0/3717828338 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f1786ffd640 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb4b20 0x562da7be3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17877fe640 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7b70390 0x562da7bc8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f1787fff640 1 -- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb4b20 msgr2=0x562da7be3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f1787fff640 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb4b20 0x562da7be3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f1787fff640 1 -- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7b70390 msgr2=0x562da7bc8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f1787fff640 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7b70390 0x562da7bc8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f1787fff640 1 -- 172.21.15.76:0/3717828338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da7a3c750 con 0x562da7bb5440 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f1787fff640 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7bb5440 0x562da7be3ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1780015c60 tx=0x7f1780004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f43e4b487dc045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 1 -- 172.21.15.76:0/3717828338 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f178003a060 con 0x562da7bb5440 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 1 -- 172.21.15.76:0/3717828338 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1780029680 con 0x562da7bb5440 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1778083070 con 0x562da7bb5440 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 1 -- 172.21.15.76:0/3717828338 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1780040050 con 0x562da7bb5440 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 10 client.5625 fetch_fsmap learned FSMap version 29 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) -- 0x562da7947a30 con 0x562da7bb5440 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 1 -- 172.21.15.76:0/3717828338 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1778083070 con 0x562da7bb5440 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 1 -- 172.21.15.76:0/3717828338 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 29) ==== 3502+0+0 (secure 0 0 0) 0x7f1780043020 con 0x562da7bb5440 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 10 client.5625 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 10 client.5625 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 20 client.5625 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:41.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7be43e0 0x562da7c27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 4 client.5625 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "status"}) -- 0x562da79c73a0 con 0x562da7be43e0 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17877fe640 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7be43e0 0x562da7c27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 10 client.5625 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17877fe640 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7be43e0 0x562da7c27c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7885008ae5bb0d0c server_cookie=2fafed62cf54fbd5 in_seq=0 out_seq=0 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 1 -- 172.21.15.76:0/3717828338 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x562da79c73a0 con 0x562da7be43e0 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 10 client.5625 handle_command_reply: tid=0 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 client.5625 shutdown 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7bb5440 msgr2=0x562da7be3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7bb5440 0x562da7be3ea0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1780015c60 tx=0x7f1780004ac0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 shutdown_connections 2024-09-19T04:15:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562da7bb5440 0x562da7be3ea0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562da7bb4b20 0x562da7be3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x562da7be43e0 0x562da7c27c70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 --2- 172.21.15.76:0/3717828338 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562da7b70390 0x562da7bc8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 >> 172.21.15.76:0/3717828338 conn(0x562da7b6b290 msgr2=0x562da7bb2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17667fc640 0 client.5625 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 shutdown_connections 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 1 -- 172.21.15.76:0/3717828338 wait complete. 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f17910ad000 20 client.5625 trim_cache size 0 max 16384 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:15:41.178 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5502, 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 5.6781601119999996, 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 29, 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 53, 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 53, 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 10.67394019 2024-09-19T04:15:41.179 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:15:41.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17740678c0 msgr2=0x7f1774069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17740678c0 0x7f1774069d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f177c0aa000 tx=0x7f177c000ff0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 msgr2=0x7f178815a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f178815a9f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f17780115e0 tx=0x7f1778011aa0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 shutdown_connections 2024-09-19T04:15:41.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17740678c0 0x7f1774069d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f17880ccdf0 0x7f178815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17881f7980 0x7f17880cc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 --2- 172.21.15.76:0/2564600258 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1770003f40 0x7f178815a9f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 >> 172.21.15.76:0/2564600258 conn(0x7f178808bf90 msgr2=0x7f17880b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 shutdown_connections 2024-09-19T04:15:41.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.169+0000 7f178dff1640 1 -- 172.21.15.76:0/2564600258 wait complete. 2024-09-19T04:15:41.188 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:15:41.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.293+0000 7fc4d7441640 1 Processor -- start 2024-09-19T04:15:41.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.293+0000 7fc4d7441640 1 -- start start 2024-09-19T04:15:41.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d7441640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 0x7fc4d00d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d7441640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4d00cf800 0x7fc4d00b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d7441640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d00d27a0 con 0x7fc4d00b4340 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d7441640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d00d1720 con 0x7fc4d00cf800 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d7441640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d00b4790 con 0x7fc4d01457a0 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4d00cf800 0x7fc4d00b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc4d00cf800 0x7fc4d00b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35226/0 (socket says 172.21.15.76:35226) 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 -- 172.21.15.76:0/2901660736 learned_addr learned my addr 172.21.15.76:0/2901660736 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 -- 172.21.15.76:0/2901660736 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 msgr2=0x7fc4d00d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 --2- 172.21.15.76:0/2901660736 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 0x7fc4d00d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 -- 172.21.15.76:0/2901660736 >> v1:172.21.15.116:6789/0 conn(0x7fc4d00b4340 legacy=0x7fc4d015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:41.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 -- 172.21.15.76:0/2901660736 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d00d2a80 con 0x7fc4d00cf800 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d49dc640 1 --2- 172.21.15.76:0/2901660736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d00b3e00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009870 tx=0x7fc4c001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b214434dda2b16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4c7fff640 1 -- 172.21.15.76:0/2901660736 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b0070 con 0x7fc4d00cf800 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4c7fff640 1 -- 172.21.15.76:0/2901660736 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c00ab400 con 0x7fc4d00cf800 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4c7fff640 1 -- 172.21.15.76:0/2901660736 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ab7d0 con 0x7fc4d00cf800 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d7441640 1 -- 172.21.15.76:0/2901660736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 msgr2=0x7fc4d00b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.297+0000 7fc4d7441640 1 --2- 172.21.15.76:0/2901660736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d00b3e00 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009870 tx=0x7fc4c001c490 comp rx=0 tx=0).stop 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- 172.21.15.76:0/2901660736 shutdown_connections 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 --2- 172.21.15.76:0/2901660736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d00b3e00 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 --2- 172.21.15.76:0/2901660736 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 0x7fc4d00d1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- 172.21.15.76:0/2901660736 >> 172.21.15.76:0/2901660736 conn(0x7fc4d006bcc0 msgr2=0x7fc4d006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- 172.21.15.76:0/2901660736 shutdown_connections 2024-09-19T04:15:41.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- 172.21.15.76:0/2901660736 wait complete. 2024-09-19T04:15:41.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 Processor -- start 2024-09-19T04:15:41.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- start start 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d00b4340 0x7fc4d015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d0164600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 0x7fc4d0140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d00b52f0 con 0x7fc4d00cf800 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d00b5170 con 0x7fc4d00b4340 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d01690b0 con 0x7fc4d01457a0 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d0164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d0164600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35240/0 (socket says 172.21.15.76:35240) 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 -- 172.21.15.76:0/78855588 learned_addr learned my addr 172.21.15.76:0/78855588 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:41.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d51dd640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d00b4340 0x7fc4d015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d59de640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 0x7fc4d0140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 -- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 msgr2=0x7fc4d0140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 0x7fc4d0140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 -- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d00b4340 msgr2=0x7fc4d015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d00b4340 0x7fc4d015bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 -- 172.21.15.76:0/78855588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d0145d20 con 0x7fc4d00cf800 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d51dd640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d00b4340 0x7fc4d015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d49dc640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d0164600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4c5ffb640 1 -- 172.21.15.76:0/78855588 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ba020 con 0x7fc4d00cf800 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4c5ffb640 1 -- 172.21.15.76:0/78855588 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c00b6640 con 0x7fc4d00cf800 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d0164d00 con 0x7fc4d00cf800 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4c5ffb640 1 -- 172.21.15.76:0/78855588 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b0040 con 0x7fc4d00cf800 2024-09-19T04:15:41.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.301+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d01652c0 con 0x7fc4d00cf800 2024-09-19T04:15:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.305+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d0075670 con 0x7fc4d00cf800 2024-09-19T04:15:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.305+0000 7fc4c5ffb640 1 -- 172.21.15.76:0/78855588 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc4c00bd020 con 0x7fc4d00cf800 2024-09-19T04:15:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.305+0000 7fc4c5ffb640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4b40678c0 0x7fc4b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:41.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.305+0000 7fc4d51dd640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4b40678c0 0x7fc4b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.305+0000 7fc4c5ffb640 1 -- 172.21.15.76:0/78855588 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fc4c0120090 con 0x7fc4d00cf800 2024-09-19T04:15:41.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.305+0000 7fc4d51dd640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4b40678c0 0x7fc4b4069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc006fd0 tx=0x7fc4cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:41.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.309+0000 7fc4c5ffb640 1 -- 172.21.15.76:0/78855588 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc4c00c4030 con 0x7fc4d00cf800 2024-09-19T04:15:41.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.505+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc4d0075390 con 0x7fc4d00cf800 2024-09-19T04:15:41.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.505+0000 7fc4c5ffb640 1 -- 172.21.15.76:0/78855588 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+73 (secure 0 0 0) 0x7fc4c0016110 con 0x7fc4d00cf800 2024-09-19T04:15:41.508 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:15:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4b40678c0 msgr2=0x7fc4b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4b40678c0 0x7fc4b4069d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc006fd0 tx=0x7fc4cc008040 comp rx=0 tx=0).stop 2024-09-19T04:15:41.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 msgr2=0x7fc4d0164600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d0164600 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ae040 tx=0x7fc4c001acd0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 shutdown_connections 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4b40678c0 0x7fc4b4069d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc4d01457a0 0x7fc4d0140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc4d00cf800 0x7fc4d0164600 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 --2- 172.21.15.76:0/78855588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d00b4340 0x7fc4d015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 >> 172.21.15.76:0/78855588 conn(0x7fc4d006bcc0 msgr2=0x7fc4d00b7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 shutdown_connections 2024-09-19T04:15:41.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41.509+0000 7fc4d7441640 1 -- 172.21.15.76:0/78855588 wait complete. 2024-09-19T04:15:41.526 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:15:41.526 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:15:41.526 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:15:41.526 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:15:41.527 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:41.527 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:15:41.527 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:15:41.527 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:15:41.527 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:15:41.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:41.527 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:15:41.533 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:15:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:15:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:15:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:15:41.533 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.533 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42379sec preferred_lft 42379sec 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:15:41.534 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:15:41.535 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:15:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.536 INFO:teuthology.orchestra.run.smithi076.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:15:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:15:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T04:15:41.536 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:15:41.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:41.536 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:15:41.584 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:15:41.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:41.586 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:15:41.639 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:15:41.640 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:15:41.640 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:15:41.640 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:15:41.640 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:15:41.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:15:41.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:15:41.640 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:15:41.640 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:15:41.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:15:41.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:15:41.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:41.761 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:15:41.806 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T04:15:41.807 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:15:41.807 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:15:41.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:41.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:15:41.862 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T04:15:41.862 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:15:41.863 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:41.910 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:15:41.910 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:15:41.911 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:41.958 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:15:42.016 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:15:42.063 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:15:42.064 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:15:42.065 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:15:42.066 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:15:42.066 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:15:42.066 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:15:42.066 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:15:42.066 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:15:42.066 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:15:42.066 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:15:42.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:42.067 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:15:42.117 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:15:42.117 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T04:15:42.159 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:15:42.173 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:15:42.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:15:42.219 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:15:42.219 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:15:42.219 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:15:42.219 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:15:42.219 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:15:42.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:15:42.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:15:42.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:15:42.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:42.222 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:15:42.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-09-19T04:15:42.237+0000 7f4de2e90580 -1 init, newargv = 0x55eb57130ef0 newargc=15 2024-09-19T04:15:42.238 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[40200]: starting ceph client 2024-09-19T04:15:42.243 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[40200]: starting fuse 2024-09-19T04:15:43.344 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:15:43.360 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:15:43.412 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:15:43.413 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:15:43.414 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:43.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:43.416 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:15:43.463 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:15:43.463 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:15:43.463 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:15:43.463 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:15:43.464 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-09-19T04:15:43.465 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:15:43.587 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.40200.asok 2024-09-19T04:15:43.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:15:43.600 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40200.asok 2024-09-19T04:15:43.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:43.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40200.asok status 2024-09-19T04:15:43.756 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:15:43.756 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:15:43.756 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "40200", 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5637, 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:15:43.757 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 5637 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 307465888 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:15:43.758 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 307465888 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.5637 192.168.0.1:0/307465888", 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/307465888", 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 29, 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 53, 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:15:43.759 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:15:43.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:43.770 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:15:43.770 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:43.777 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:15:43.777 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:43.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:43.778 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:15:43.831 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-19T04:15:44.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe78b897640 1 Processor -- start 2024-09-19T04:15:44.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe78b897640 1 -- start start 2024-09-19T04:15:44.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe78b897640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78415bec0 0x7fe7840ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe78b897640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7840b3290 con 0x7fe78415c2a0 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe78b897640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7840b3570 con 0x7fe7840af470 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe78b897640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe7840b3850 con 0x7fe78415bec0 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe788e32640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe78415c2a0 0x7fe7840aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54476/0 (socket says 172.21.15.76:54476) 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe788e32640 1 -- 172.21.15.76:0/2342956928 learned_addr learned my addr 172.21.15.76:0/2342956928 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe789633640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78415bec0 0x7fe7840ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1112139430 0 0) 0x7fe7840b3290 con 0x7fe78415c2a0 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe770003530 con 0x7fe78415c2a0 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3429752161 0 0) 0x7fe770003530 con 0x7fe78415c2a0 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78415bec0 msgr2=0x7fe7840ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78415bec0 0x7fe7840ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 >> v1:172.21.15.116:6789/0 conn(0x7fe7840af470 legacy=0x7fe78406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7840b3b30 con 0x7fe78415c2a0 2024-09-19T04:15:44.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fe780001680 con 0x7fe78415c2a0 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 >> v1:172.21.15.76:6789/0 conn(0x7fe78415c2a0 legacy=0x7fe7840aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.005+0000 7fe777fff640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe770003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78415bec0 0x7fe770004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe770004d50 0x7fe7700051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7840b3850 con 0x7fe7700039f0 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe77000ba80 con 0x7fe770004d50 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe77000bc80 con 0x7fe78415bec0 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789633640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe770003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789633640 1 -- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78415bec0 msgr2=0x7fe770004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789633640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78415bec0 0x7fe770004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789633640 1 -- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe770004d50 msgr2=0x7fe7700051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789633640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe770004d50 0x7fe7700051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789633640 1 -- 172.21.15.76:0/2342956928 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe770003530 con 0x7fe7700039f0 2024-09-19T04:15:44.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789633640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe770003e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe778009a90 tx=0x7fe77801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d408373afea69209 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780b2070 con 0x7fe7700039f0 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7780a5610 con 0x7fe7700039f0 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe777fff640 1 -- 172.21.15.76:0/2342956928 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778009690 con 0x7fe7700039f0 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe78b897640 1 -- 172.21.15.76:0/2342956928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 msgr2=0x7fe770003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe78b897640 1 --2- 172.21.15.76:0/2342956928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe770003e10 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe778009a90 tx=0x7fe77801ca40 comp rx=0 tx=0).stop 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe789e34640 1 -- 172.21.15.76:0/2342956928 reap_dead start 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe78b897640 1 -- 172.21.15.76:0/2342956928 shutdown_connections 2024-09-19T04:15:44.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe78b897640 1 -- 172.21.15.76:0/2342956928 >> 172.21.15.76:0/2342956928 conn(0x7fe7840b1f90 msgr2=0x7fe7840b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe78b897640 1 -- 172.21.15.76:0/2342956928 shutdown_connections 2024-09-19T04:15:44.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.009+0000 7fe78b897640 1 -- 172.21.15.76:0/2342956928 wait complete. 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 Processor -- start 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 -- start start 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe7841529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78414d190 0x7fe784152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe78414d670 0x7fe78409c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7840ad790 con 0x7fe7700039f0 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7840ad610 con 0x7fe78414d670 2024-09-19T04:15:44.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe78406ef20 con 0x7fe78414d190 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe7841529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe7841529f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36010/0 (socket says 172.21.15.76:36010) 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 -- 172.21.15.76:0/2584946052 learned_addr learned my addr 172.21.15.76:0/2584946052 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe788e32640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78414d190 0x7fe784152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789e34640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe78414d670 0x7fe78409c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 -- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78414d190 msgr2=0x7fe784152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78414d190 0x7fe784152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 -- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe78414d670 msgr2=0x7fe78409c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe78414d670 0x7fe78409c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 -- 172.21.15.76:0/2584946052 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe784150f50 con 0x7fe7700039f0 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe788e32640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78414d190 0x7fe784152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789e34640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe78414d670 0x7fe78409c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe789633640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe7841529f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe77801c870 tx=0x7fe770005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe775ffb640 1 -- 172.21.15.76:0/2584946052 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780b2040 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe775ffb640 1 -- 172.21.15.76:0/2584946052 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe77800f000 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe784151230 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe775ffb640 1 -- 172.21.15.76:0/2584946052 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780b76a0 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78409a720 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe78406e000 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.013+0000 7fe775ffb640 1 -- 172.21.15.76:0/2584946052 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7780c0020 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.017+0000 7fe775ffb640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7680678c0 0x7fe768069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.017+0000 7fe775ffb640 1 -- 172.21.15.76:0/2584946052 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fe77811fe10 con 0x7fe7700039f0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.017+0000 7fe788e32640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7680678c0 0x7fe768069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.017+0000 7fe788e32640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7680678c0 0x7fe768069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe780001ec0 tx=0x7fe780009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.021+0000 7fe775ffb640 1 -- 172.21.15.76:0/2584946052 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe7780c7020 con 0x7fe7700039f0 2024-09-19T04:15:44.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.225+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fe78406fcd0 con 0x7fe7700039f0 2024-09-19T04:15:44.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.837+0000 7fe775ffb640 1 -- 172.21.15.76:0/2584946052 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v30) ==== 105+0+0 (secure 0 0 0) 0x7fe778016780 con 0x7fe7700039f0 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7680678c0 msgr2=0x7fe768069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7680678c0 0x7fe768069d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe780001ec0 tx=0x7fe780009040 comp rx=0 tx=0).stop 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 msgr2=0x7fe7841529f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe7841529f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe77801c870 tx=0x7fe770005840 comp rx=0 tx=0).stop 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 shutdown_connections 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7680678c0 0x7fe768069d80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe78414d670 0x7fe78409c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe78414d190 0x7fe784152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 --2- 172.21.15.76:0/2584946052 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7700039f0 0x7fe7841529f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 >> 172.21.15.76:0/2584946052 conn(0x7fe7840b1f90 msgr2=0x7fe78415aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 shutdown_connections 2024-09-19T04:15:44.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.841+0000 7fe78b897640 1 -- 172.21.15.76:0/2584946052 wait complete. 2024-09-19T04:15:44.870 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:44.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.981+0000 7f3fef7db640 1 Processor -- start 2024-09-19T04:15:44.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.981+0000 7f3fef7db640 1 -- start start 2024-09-19T04:15:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fef7db640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe815bec0 0x7f3fe80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fef7db640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3fe815c2a0 0x7f3fe80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fef7db640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3fe80af2a0 0x7f3fe815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fef7db640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe80b3290 con 0x7f3fe80af2a0 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fef7db640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3fe80b3570 con 0x7f3fe815c2a0 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fef7db640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe80b3850 con 0x7f3fe815bec0 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3fe80af2a0 0x7f3fe815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3fe80af2a0 0x7f3fe815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36026/0 (socket says 172.21.15.76:36026) 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 -- 172.21.15.76:0/1140310111 learned_addr learned my addr 172.21.15.76:0/1140310111 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 -- 172.21.15.76:0/1140310111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe815bec0 msgr2=0x7f3fe80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fed577640 1 --2- 172.21.15.76:0/1140310111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe815bec0 0x7f3fe80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fecd76640 1 --2- 172.21.15.76:0/1140310111 >> v2:172.21.15.116:3300/0 conn(0x7f3fe815c2a0 0x7f3fe80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 --2- 172.21.15.76:0/1140310111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe815bec0 0x7f3fe80ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 -- 172.21.15.76:0/1140310111 >> v2:172.21.15.116:3300/0 conn(0x7f3fe815c2a0 msgr2=0x7f3fe80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:44.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 --2- 172.21.15.76:0/1140310111 >> v2:172.21.15.116:3300/0 conn(0x7f3fe815c2a0 0x7f3fe80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 -- 172.21.15.76:0/1140310111 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe80b3b30 con 0x7f3fe80af2a0 2024-09-19T04:15:44.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fedd78640 1 --2- 172.21.15.76:0/1140310111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe80af2a0 0x7f3fe815fc60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3fe806de20 tx=0x7f3fe001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0f245511f96392 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fdffff640 1 -- 172.21.15.76:0/1140310111 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00a5580 con 0x7f3fe80af2a0 2024-09-19T04:15:44.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fdffff640 1 -- 172.21.15.76:0/1140310111 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe00a5720 con 0x7f3fe80af2a0 2024-09-19T04:15:44.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.985+0000 7f3fdffff640 1 -- 172.21.15.76:0/1140310111 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe00a5af0 con 0x7f3fe80af2a0 2024-09-19T04:15:44.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- 172.21.15.76:0/1140310111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe80af2a0 msgr2=0x7f3fe815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 --2- 172.21.15.76:0/1140310111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe80af2a0 0x7f3fe815fc60 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3fe806de20 tx=0x7f3fe001c700 comp rx=0 tx=0).stop 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- 172.21.15.76:0/1140310111 shutdown_connections 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 --2- 172.21.15.76:0/1140310111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe80af2a0 0x7f3fe815fc60 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 --2- 172.21.15.76:0/1140310111 >> v2:172.21.15.116:3300/0 conn(0x7f3fe815c2a0 0x7f3fe80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 --2- 172.21.15.76:0/1140310111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe815bec0 0x7f3fe80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- 172.21.15.76:0/1140310111 >> 172.21.15.76:0/1140310111 conn(0x7f3fe80b1f90 msgr2=0x7f3fe80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- 172.21.15.76:0/1140310111 shutdown_connections 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- 172.21.15.76:0/1140310111 wait complete. 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 Processor -- start 2024-09-19T04:15:44.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- start start 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe80af2a0 0x7f3fe81538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe815bec0 0x7f3fe80b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fe815c2a0 0x7f3fe80b0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe80ad740 con 0x7f3fe815bec0 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe80afc90 con 0x7f3fe815c2a0 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3fe806e000 con 0x7f3fe80af2a0 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe815bec0 0x7f3fe80b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe815bec0 0x7f3fe80b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36030/0 (socket says 172.21.15.76:36030) 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 -- 172.21.15.76:0/4082932022 learned_addr learned my addr 172.21.15.76:0/4082932022 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:44.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fed577640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe80af2a0 0x7f3fe81538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 -- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe80af2a0 msgr2=0x7f3fe81538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe80af2a0 0x7f3fe81538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fedd78640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fe815c2a0 0x7f3fe80b0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 -- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fe815c2a0 msgr2=0x7f3fe80b0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fe815c2a0 0x7f3fe80b0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 -- 172.21.15.76:0/4082932022 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe80b0e90 con 0x7f3fe815bec0 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fecd76640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe815bec0 0x7f3fe80b0320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c5a0 tx=0x7f3fe400ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fddffb640 1 -- 172.21.15.76:0/4082932022 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4015070 con 0x7f3fe815bec0 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fddffb640 1 -- 172.21.15.76:0/4082932022 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fe4004220 con 0x7f3fe815bec0 2024-09-19T04:15:44.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe80b1170 con 0x7f3fe815bec0 2024-09-19T04:15:44.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fddffb640 1 -- 172.21.15.76:0/4082932022 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe4003620 con 0x7f3fe815bec0 2024-09-19T04:15:44.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.989+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe80b1a10 con 0x7f3fe815bec0 2024-09-19T04:15:44.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.993+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe80b1c10 con 0x7f3fe815bec0 2024-09-19T04:15:44.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.993+0000 7f3fddffb640 1 -- 172.21.15.76:0/4082932022 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3fe40037c0 con 0x7f3fe815bec0 2024-09-19T04:15:44.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.993+0000 7f3fddffb640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fcc0678c0 0x7f3fcc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:44.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.993+0000 7f3fddffb640 1 -- 172.21.15.76:0/4082932022 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f3fe4085b80 con 0x7f3fe815bec0 2024-09-19T04:15:44.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.993+0000 7f3fed577640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fcc0678c0 0x7f3fcc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:44.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.993+0000 7f3fed577640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fcc0678c0 0x7f3fcc069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b800 tx=0x7f3fd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:44.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:44.997+0000 7f3fddffb640 1 -- 172.21.15.76:0/4082932022 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3fe80b1c10 con 0x7f3fe815bec0 2024-09-19T04:15:45.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fe80ad200 con 0x7f3fe815bec0 2024-09-19T04:15:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fddffb640 1 -- 172.21.15.76:0/4082932022 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+5245 (secure 0 0 0) 0x7f3fe4056530 con 0x7f3fe815bec0 2024-09-19T04:15:45.195 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:45.195 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":31,"btime":"2024-09-19T04:15:44:845148+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":4754,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:15:44.845104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":5502,"mds_1":5496},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502]},"id":3}]} 2024-09-19T04:15:45.196 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 31 2024-09-19T04:15:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fcc0678c0 msgr2=0x7f3fcc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fcc0678c0 0x7f3fcc069d80 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3fd800b800 tx=0x7f3fd8008040 comp rx=0 tx=0).stop 2024-09-19T04:15:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe815bec0 msgr2=0x7f3fe80b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe815bec0 0x7f3fe80b0320 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3fe400c5a0 tx=0x7f3fe400ca60 comp rx=0 tx=0).stop 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 shutdown_connections 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fcc0678c0 0x7f3fcc069d80 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fe815c2a0 0x7f3fe80b0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fe815bec0 0x7f3fe80b0320 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 --2- 172.21.15.76:0/4082932022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fe80af2a0 0x7f3fe81538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 >> 172.21.15.76:0/4082932022 conn(0x7f3fe80b1f90 msgr2=0x7f3fe81666f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 shutdown_connections 2024-09-19T04:15:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:45.193+0000 7f3fef7db640 1 -- 172.21.15.76:0/4082932022 wait complete. 2024-09-19T04:15:45.615 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:15:34.831676+0000', 'modified': '2024-09-19T04:15:44.845104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 5502, 'mds_1': 5496}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5496': {'gid': 5496, 'name': 'c', 'rank': 1, 'incarnation': 31, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.76:6835/3259378986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3259378986}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3259378986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5502': {'gid': 5502, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3635635274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3635635274}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3635635274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5502, 'qdb_cluster': [5502]} 2024-09-19T04:15:45.615 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5496:up:creating 2024-09-19T04:15:46.444 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:15:46.441+0000 7fccdf431640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T04:15:46.617 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:46.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.741+0000 7f2efe082640 1 Processor -- start 2024-09-19T04:15:46.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2efe082640 1 -- start start 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2efe082640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2ef8070020 0x7f2ef815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2efe082640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef80aede0 con 0x7f2ef815f1d0 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2efe082640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef815fa10 con 0x7f2ef81520a0 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2efe082640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef8152480 con 0x7f2ef8070020 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2ef815f1d0 0x7f2ef80adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54490/0 (socket says 172.21.15.76:54490) 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 -- 172.21.15.76:0/3979687741 learned_addr learned my addr 172.21.15.76:0/3979687741 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef6ffd640 1 --2- 172.21.15.76:0/3979687741 >> v2:172.21.15.116:3300/0 conn(0x7f2ef8070020 0x7f2ef815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1989515546 0 0) 0x7f2ef80aede0 con 0x7f2ef815f1d0 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ee0005740 con 0x7f2ef815f1d0 2024-09-19T04:15:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3865927304 0 0) 0x7f2ee0005740 con 0x7f2ef815f1d0 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 >> v2:172.21.15.116:3300/0 conn(0x7f2ef8070020 msgr2=0x7f2ef815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 --2- 172.21.15.76:0/3979687741 >> v2:172.21.15.116:3300/0 conn(0x7f2ef8070020 0x7f2ef815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 >> v1:172.21.15.116:6789/0 conn(0x7f2ef81520a0 legacy=0x7f2ef806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef8074250 con 0x7f2ef815f1d0 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2ef0001a00 con 0x7f2ef815f1d0 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 >> v1:172.21.15.76:6789/0 conn(0x7f2ef815f1d0 legacy=0x7f2ef80adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0005bd0 0x7f2ee0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ee0006a20 0x7f2ee0006ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ee00073e0 0x7f2ee000dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee000e250 con 0x7f2ee00073e0 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee000e400 con 0x7f2ee0005bd0 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee000e600 con 0x7f2ee0006a20 2024-09-19T04:15:46.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ee00073e0 0x7f2ee000dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef77fe640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0005bd0 0x7f2ee0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef6ffd640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ee0006a20 0x7f2ee0006ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 -- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ee0006a20 msgr2=0x7f2ee0006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ee0006a20 0x7f2ee0006ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 -- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0005bd0 msgr2=0x7f2ee0005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee0005bd0 0x7f2ee0005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.745+0000 7f2ef7fff640 1 -- 172.21.15.76:0/3979687741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee0005740 con 0x7f2ee00073e0 2024-09-19T04:15:46.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef7fff640 1 -- 172.21.15.76:0/3979687741 reap_dead start 2024-09-19T04:15:46.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef7fff640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ee00073e0 0x7f2ee000dbe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000a860 tx=0x7f2ef001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e2a6ff90fcbf38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000a480 con 0x7f2ee00073e0 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ef00ad630 con 0x7f2ee00073e0 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef67fc640 1 -- 172.21.15.76:0/3979687741 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00ada00 con 0x7f2ee00073e0 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- 172.21.15.76:0/3979687741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ee00073e0 msgr2=0x7f2ee000dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ee00073e0 0x7f2ee000dbe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2ef000a860 tx=0x7f2ef001d380 comp rx=0 tx=0).stop 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- 172.21.15.76:0/3979687741 shutdown_connections 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 --2- 172.21.15.76:0/3979687741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ee00073e0 0x7f2ee000dbe0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- 172.21.15.76:0/3979687741 >> 172.21.15.76:0/3979687741 conn(0x7f2ef80b1f90 msgr2=0x7f2ef80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- 172.21.15.76:0/3979687741 shutdown_connections 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- 172.21.15.76:0/3979687741 wait complete. 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 Processor -- start 2024-09-19T04:15:46.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- start start 2024-09-19T04:15:46.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee00073e0 0x7f2ef815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef815cf60 0x7f2ef809c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef809c7a0 0x7f2ef814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8073a50 con 0x7f2ef815cf60 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef8073770 con 0x7f2ee00073e0 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2efe082640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef8073d30 con 0x7f2ef809c7a0 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef815cf60 0x7f2ef809c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef815cf60 0x7f2ef809c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36044/0 (socket says 172.21.15.76:36044) 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef6ffd640 1 -- 172.21.15.76:0/1089016834 learned_addr learned my addr 172.21.15.76:0/1089016834 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef77fe640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee00073e0 0x7f2ef815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef7fff640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef809c7a0 0x7f2ef814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef6ffd640 1 -- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef809c7a0 msgr2=0x7f2ef814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef6ffd640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef809c7a0 0x7f2ef814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef6ffd640 1 -- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee00073e0 msgr2=0x7f2ef815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.749+0000 7f2ef6ffd640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee00073e0 0x7f2ef815dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2ef6ffd640 1 -- 172.21.15.76:0/1089016834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef815c1a0 con 0x7f2ef815cf60 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2ef77fe640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee00073e0 0x7f2ef815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2ef6ffd640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef815cf60 0x7f2ef809c260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8011590 tx=0x7f2ee8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2edffff640 1 -- 172.21.15.76:0/1089016834 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8015870 con 0x7f2ef815cf60 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2edffff640 1 -- 172.21.15.76:0/1089016834 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee8015a10 con 0x7f2ef815cf60 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef815c480 con 0x7f2ef815cf60 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2edffff640 1 -- 172.21.15.76:0/1089016834 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8015dc0 con 0x7f2ef815cf60 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef815e320 con 0x7f2ef815cf60 2024-09-19T04:15:46.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef809cbb0 con 0x7f2ef815cf60 2024-09-19T04:15:46.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2edffff640 1 -- 172.21.15.76:0/1089016834 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ee8013400 con 0x7f2ef815cf60 2024-09-19T04:15:46.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2edffff640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee40678c0 0x7f2ee4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:46.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2ef77fe640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee40678c0 0x7f2ee4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:46.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2edffff640 1 -- 172.21.15.76:0/1089016834 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2ee8086620 con 0x7f2ef815cf60 2024-09-19T04:15:46.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.753+0000 7f2ef77fe640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee40678c0 0x7f2ee4069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ef815dd10 tx=0x7f2eec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:46.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.757+0000 7f2edffff640 1 -- 172.21.15.76:0/1089016834 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2ef809cbb0 con 0x7f2ef815cf60 2024-09-19T04:15:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.945+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ef8068e50 con 0x7f2ef815cf60 2024-09-19T04:15:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.945+0000 7f2edffff640 1 -- 172.21.15.76:0/1089016834 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5267 (secure 0 0 0) 0x7f2ee8019030 con 0x7f2ef815cf60 2024-09-19T04:15:46.949 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:46.949 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":34,"btime":"2024-09-19T04:15:46:858706+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:15:46.858697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[4754,5496,5502]},"id":3}]} 2024-09-19T04:15:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 34 2024-09-19T04:15:46.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee40678c0 msgr2=0x7f2ee4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:46.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee40678c0 0x7f2ee4069d80 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2ef815dd10 tx=0x7f2eec009040 comp rx=0 tx=0).stop 2024-09-19T04:15:46.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef815cf60 msgr2=0x7f2ef809c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:46.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef815cf60 0x7f2ef809c260 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8011590 tx=0x7f2ee8011a50 comp rx=0 tx=0).stop 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 shutdown_connections 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ee40678c0 0x7f2ee4069d80 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef809c7a0 0x7f2ef814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef815cf60 0x7f2ef809c260 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 --2- 172.21.15.76:0/1089016834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ee00073e0 0x7f2ef815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 >> 172.21.15.76:0/1089016834 conn(0x7f2ef80b1f90 msgr2=0x7f2ef80b0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 shutdown_connections 2024-09-19T04:15:46.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:46.949+0000 7f2efe082640 1 -- 172.21.15.76:0/1089016834 wait complete. 2024-09-19T04:15:47.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:15:34.831676+0000', 'modified': '2024-09-19T04:15:46.858697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 5502, 'mds_1': 5496, 'mds_2': 4754}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4754': {'gid': 4754, 'name': 'b', 'rank': 2, 'incarnation': 33, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6835/3863430206', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3863430206}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3863430206}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5496': {'gid': 5496, 'name': 'c', 'rank': 1, 'incarnation': 31, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6835/3259378986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 3259378986}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 3259378986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_5502': {'gid': 5502, 'name': 'a', 'rank': 0, 'incarnation': 26, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3635635274', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3635635274}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3635635274}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5502, 'qdb_cluster': [4754, 5496, 5502]} 2024-09-19T04:15:47.370 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:15:47.370 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-19T04:15:47.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.497+0000 7fdd95731640 1 Processor -- start 2024-09-19T04:15:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.497+0000 7fdd95731640 1 -- start start 2024-09-19T04:15:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd95731640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 0x7fdd9015c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd95731640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdd9006d3e0 0x7fdd900ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd95731640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd900cd5b0 con 0x7fdd9015cae0 2024-09-19T04:15:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd95731640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fdd900cd890 con 0x7fdd9006d3e0 2024-09-19T04:15:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd95731640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdd900cdb70 con 0x7fdd9015c1c0 2024-09-19T04:15:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8e7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdd9015cae0 0x7fdd9006ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54500/0 (socket says 172.21.15.76:54500) 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/3946469168 learned_addr learned my addr 172.21.15.76:0/3946469168 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8effd640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 0x7fdd9015c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/3946469168 >> v2:172.21.15.116:3300/0 conn(0x7fdd9006d3e0 0x7fdd900ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1815043253 0 0) 0x7fdd900cd5b0 con 0x7fdd9015cae0 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd7c003530 con 0x7fdd9015cae0 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 426872492 0 0) 0x7fdd7c003530 con 0x7fdd9015cae0 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 msgr2=0x7fdd9015c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 0x7fdd9015c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 >> v2:172.21.15.116:3300/0 conn(0x7fdd9006d3e0 msgr2=0x7fdd900ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 --2- 172.21.15.76:0/3946469168 >> v2:172.21.15.116:3300/0 conn(0x7fdd9006d3e0 0x7fdd900ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd900cde50 con 0x7fdd9015cae0 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fdd780016a0 con 0x7fdd9015cae0 2024-09-19T04:15:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 >> v1:172.21.15.76:6789/0 conn(0x7fdd9015cae0 legacy=0x7fdd9006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd7c0039c0 0x7fdd7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 0x7fdd7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd900cd890 con 0x7fdd7c004d50 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdd900cdb70 con 0x7fdd7c0039c0 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c00b9d0 con 0x7fdd9015c1c0 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 0x7fdd7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8effd640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd7c0039c0 0x7fdd7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 msgr2=0x7fdd7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd9015c1c0 0x7fdd7c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd7c0039c0 msgr2=0x7fdd7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd7c0039c0 0x7fdd7c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/3946469168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c003530 con 0x7fdd7c004d50 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009f90 tx=0x7fdd8801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa870fdf682cb137 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880b2070 con 0x7fdd7c004d50 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880a5c90 con 0x7fdd7c004d50 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.501+0000 7fdd8dffb640 1 -- 172.21.15.76:0/3946469168 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88009c10 con 0x7fdd7c004d50 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- 172.21.15.76:0/3946469168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd7c004d50 msgr2=0x7fdd7c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 --2- 172.21.15.76:0/3946469168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd7c004d50 0x7fdd7c0051d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdd88009f90 tx=0x7fdd8801cc80 comp rx=0 tx=0).stop 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/3946469168 reap_dead start 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- 172.21.15.76:0/3946469168 shutdown_connections 2024-09-19T04:15:47.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- 172.21.15.76:0/3946469168 >> 172.21.15.76:0/3946469168 conn(0x7fdd90072670 msgr2=0x7fdd90072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- 172.21.15.76:0/3946469168 shutdown_connections 2024-09-19T04:15:47.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- 172.21.15.76:0/3946469168 wait complete. 2024-09-19T04:15:47.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 Processor -- start 2024-09-19T04:15:47.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- start start 2024-09-19T04:15:47.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd7c004d50 0x7fdd90146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd900aee50 0x7fdd901461f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd90151680 0x7fdd9006bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd900b4c20 con 0x7fdd90151680 2024-09-19T04:15:47.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdd900b4aa0 con 0x7fdd900aee50 2024-09-19T04:15:47.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdd900b5880 con 0x7fdd7c004d50 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd90151680 0x7fdd9006bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd90151680 0x7fdd9006bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36070/0 (socket says 172.21.15.76:36070) 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/1663129644 learned_addr learned my addr 172.21.15.76:0/1663129644 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd900aee50 0x7fdd901461f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8effd640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd7c004d50 0x7fdd90146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd7c004d50 msgr2=0x7fdd90146b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd7c004d50 0x7fdd90146b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd900aee50 msgr2=0x7fdd901461f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd900aee50 0x7fdd901461f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/1663129644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9015b340 con 0x7fdd90151680 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd90151680 0x7fdd9006bcc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdd88002240 tx=0x7fdd88009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd877fe640 1 -- 172.21.15.76:0/1663129644 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880b2040 con 0x7fdd90151680 2024-09-19T04:15:47.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd877fe640 1 -- 172.21.15.76:0/1663129644 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880a5a90 con 0x7fdd90151680 2024-09-19T04:15:47.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9006e8d0 con 0x7fdd90151680 2024-09-19T04:15:47.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd877fe640 1 -- 172.21.15.76:0/1663129644 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd88009c10 con 0x7fdd90151680 2024-09-19T04:15:47.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.505+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9006ee90 con 0x7fdd90151680 2024-09-19T04:15:47.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd877fe640 1 -- 172.21.15.76:0/1663129644 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdd880be020 con 0x7fdd90151680 2024-09-19T04:15:47.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd877fe640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd700678f0 0x7fdd70069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd877fe640 1 -- 172.21.15.76:0/1663129644 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdd88120420 con 0x7fdd90151680 2024-09-19T04:15:47.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8effd640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd700678f0 0x7fdd70069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8effd640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd700678f0 0x7fdd70069db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd800099b0 tx=0x7fdd80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 Processor -- start 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 -- start start 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270bb20 0x559cb270bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 0x559cb26c1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb26c2290 0x559cb26c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559cb25a9b60 con 0x559cb26c2290 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559cb2510c70 con 0x559cb270bb20 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559cb24b6ed0 con 0x559cb270c440 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb26c2290 0x559cb26c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb26c2290 0x559cb26c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36084/0 (socket says 172.21.15.76:36084) 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/2226603049 learned_addr learned my addr 172.21.15.76:0/2226603049 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8effd640 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270bb20 0x559cb270bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 0x559cb26c1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 msgr2=0x559cb26c1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 0x559cb26c1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270bb20 msgr2=0x559cb270bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270bb20 0x559cb270bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 -- 172.21.15.76:0/2226603049 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb24c8e50 con 0x559cb26c2290 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb26c2290 0x559cb26c2670 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdd880b0040 tx=0x7fdd88004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc6e107aca63d96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.509+0000 7fdd84ff9640 1 -- 172.21.15.76:0/2226603049 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880f0dd0 con 0x559cb26c2290 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd84ff9640 1 -- 172.21.15.76:0/2226603049 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880f06d0 con 0x559cb26c2290 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd84ff9640 1 -- 172.21.15.76:0/2226603049 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd880f4a80 con 0x559cb26c2290 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/2226603049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb26c2290 msgr2=0x559cb26c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb26c2290 0x559cb26c2670 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdd880b0040 tx=0x7fdd88004a50 comp rx=0 tx=0).stop 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/2226603049 shutdown_connections 2024-09-19T04:15:47.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb26c2290 0x559cb26c2670 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 0x559cb26c1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- 172.21.15.76:0/2226603049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270bb20 0x559cb270bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/2226603049 >> 172.21.15.76:0/2226603049 conn(0x559cb26c8600 msgr2=0x559cb26c0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/2226603049 shutdown_connections 2024-09-19T04:15:47.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/2226603049 wait complete. 2024-09-19T04:15:47.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- start start 2024-09-19T04:15:47.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 Processor -- start 2024-09-19T04:15:47.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270bb20 0x559cb26c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270c440 0x559cb26c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 0x559cb26cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559cb2572b00 con 0x559cb270c440 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559cb24eb360 con 0x559cb26cccf0 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559cb24e82d0 con 0x559cb270bb20 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 10 client.0 _ll_register_callbacks cb 0x559cb26ad820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270c440 0x559cb26c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270c440 0x559cb26c71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36100/0 (socket says 172.21.15.76:36100) 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/4141271490 learned_addr learned my addr 172.21.15.76:0/4141271490 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8effd640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270bb20 0x559cb26c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 0x559cb26cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270bb20 msgr2=0x559cb26c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270bb20 0x559cb26c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 msgr2=0x559cb26cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 0x559cb26cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/4141271490 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb24b7120 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8effd640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270bb20 0x559cb26c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270c440 0x559cb26c71b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdd900b0470 tx=0x7fdd780149c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489d09c15963e5de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd6dffb640 1 -- 172.21.15.76:0/4141271490 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78010040 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd6dffb640 1 -- 172.21.15.76:0/4141271490 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdd78016620 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd880f06d0 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd6dffb640 1 -- 172.21.15.76:0/4141271490 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd78024050 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 10 client.5667 fetch_fsmap learned FSMap version 34 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x559cb24b0450 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd6dffb640 1 -- 172.21.15.76:0/4141271490 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd880f06d0 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd6dffb640 1 -- 172.21.15.76:0/4141271490 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7fdd78027020 con 0x559cb270c440 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 10 client.5667 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:15:47.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 10 client.5667 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 20 client.5667 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2719db0 0x559cb271c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 4 client.5667 mds_command: new command op to 4754 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 --> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cb24aac00 con 0x559cb2719db0 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.513+0000 7fdd8effd640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2719db0 0x559cb271c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd6dffb640 10 client.5667 ms_handle_connect on v2:172.21.15.116:6834/3863430206 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd8effd640 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2719db0 0x559cb271c1a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=72f8ba6e8351c4c server_cookie=5e318a53141c1ddb in_seq=0 out_seq=0 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd6dffb640 1 -- 172.21.15.76:0/4141271490 <== mds.2 v2:172.21.15.116:6834/3863430206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559cb24aac00 con 0x559cb2719db0 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd6dffb640 10 client.5667 handle_command_reply: tid=0 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 client.5667 shutdown 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270c440 msgr2=0x559cb26c71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270c440 0x559cb26c71b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fdd900b0470 tx=0x7fdd780149c0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 shutdown_connections 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 0x559cb26cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270c440 0x559cb26c71b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2719db0 0x559cb271c1a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 --2- 172.21.15.76:0/4141271490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270bb20 0x559cb26c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 >> 172.21.15.76:0/4141271490 conn(0x559cb26c2290 msgr2=0x559cb26c0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd6dffb640 0 client.5667 ms_handle_reset on v2:172.21.15.116:6834/3863430206 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 shutdown_connections 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 1 -- 172.21.15.76:0/4141271490 wait complete. 2024-09-19T04:15:47.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.517+0000 7fdd987ed000 20 client.5667 trim_cache size 0 max 16384 2024-09-19T04:15:47.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 Processor -- start 2024-09-19T04:15:47.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- start start 2024-09-19T04:15:47.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 0x559cb26c67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb26c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 0x559cb271f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90200800 con 0x559cb270bb20 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdd90200ed0 con 0x559cb26cccf0 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdd901ff910 con 0x559cb270c440 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb26c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb26c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36116/0 (socket says 172.21.15.76:36116) 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/3169973716 learned_addr learned my addr 172.21.15.76:0/3169973716 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/3169973716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 msgr2=0x559cb271f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/3169973716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 0x559cb271f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/3169973716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 msgr2=0x559cb26c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/3169973716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 0x559cb26c67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/3169973716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb2497fd0 con 0x559cb270bb20 2024-09-19T04:15:47.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/3169973716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb26c6d10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdd78015010 tx=0x7fdd78001020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d377ebed7e68248e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd6ffff640 1 -- 172.21.15.76:0/3169973716 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78016c80 con 0x559cb270bb20 2024-09-19T04:15:47.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd6ffff640 1 -- 172.21.15.76:0/3169973716 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7801c070 con 0x559cb270bb20 2024-09-19T04:15:47.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd6ffff640 1 -- 172.21.15.76:0/3169973716 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78016620 con 0x559cb270bb20 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- 172.21.15.76:0/3169973716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 msgr2=0x559cb26c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 --2- 172.21.15.76:0/3169973716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb26c6d10 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fdd78015010 tx=0x7fdd78001020 comp rx=0 tx=0).stop 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- 172.21.15.76:0/3169973716 shutdown_connections 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 --2- 172.21.15.76:0/3169973716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb270c440 0x559cb271f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 --2- 172.21.15.76:0/3169973716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb26c6d10 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 --2- 172.21.15.76:0/3169973716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb26cccf0 0x559cb26c67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- 172.21.15.76:0/3169973716 >> 172.21.15.76:0/3169973716 conn(0x559cb26c2290 msgr2=0x559cb270ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- 172.21.15.76:0/3169973716 shutdown_connections 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- 172.21.15.76:0/3169973716 wait complete. 2024-09-19T04:15:47.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.537+0000 7fdd987ed000 1 -- start start 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 Processor -- start 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb26cccf0 0x559cb2785da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb271ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270c440 0x559cb271efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559cb250ff80 con 0x559cb270bb20 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559cb24c8ca0 con 0x559cb270c440 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559cb25152a0 con 0x559cb26cccf0 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 10 client.0 _ll_register_callbacks cb 0x7fdd780057b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb271ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb271ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36126/0 (socket says 172.21.15.76:36126) 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/1757140896 learned_addr learned my addr 172.21.15.76:0/1757140896 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8effd640 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb26cccf0 0x559cb2785da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb26cccf0 msgr2=0x559cb2785da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8f7fe640 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270c440 0x559cb271efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb26cccf0 0x559cb2785da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270c440 msgr2=0x559cb271efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270c440 0x559cb271efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 -- 172.21.15.76:0/1757140896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb2593750 con 0x559cb270bb20 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8e7fc640 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb271ea70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdd78005550 tx=0x7fdd900ce020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14438098667c1db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 1 -- 172.21.15.76:0/1757140896 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78016630 con 0x559cb270bb20 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 1 -- 172.21.15.76:0/1757140896 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fdd78033ca0 con 0x559cb270bb20 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd7801c070 con 0x559cb270bb20 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 1 -- 172.21.15.76:0/1757140896 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdd78033ea0 con 0x559cb270bb20 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 10 client.5673 fetch_fsmap learned FSMap version 34 2024-09-19T04:15:47.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x559cb249ea30 con 0x559cb270bb20 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 1 -- 172.21.15.76:0/1757140896 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd7801c070 con 0x559cb270bb20 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 1 -- 172.21.15.76:0/1757140896 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7fdd78027040 con 0x559cb270bb20 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 10 client.5673 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 10 client.5673 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 20 client.5673 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2742ca0 0x559cb2743080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 4 client.5673 mds_command: new command op to 4754 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 --> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] -- command(tid 0: {"prefix": "status"}) -- 0x559cb251e3a0 con 0x559cb2742ca0 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8effd640 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2742ca0 0x559cb2743080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 10 client.5673 ms_handle_connect on v2:172.21.15.116:6834/3863430206 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd8effd640 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2742ca0 0x559cb2743080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=334c63656af263c2 server_cookie=db3a8639519daa7c in_seq=0 out_seq=0 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 1 -- 172.21.15.76:0/1757140896 <== mds.2 v2:172.21.15.116:6834/3863430206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x559cb251e3a0 con 0x559cb2742ca0 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 10 client.5673 handle_command_reply: tid=0 2024-09-19T04:15:47.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 client.5673 shutdown 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 msgr2=0x559cb271ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb271ea70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fdd78005550 tx=0x7fdd900ce020 comp rx=0 tx=0).stop 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 shutdown_connections 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559cb270c440 0x559cb271efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559cb270bb20 0x559cb271ea70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x559cb2742ca0 0x559cb2743080 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 --2- 172.21.15.76:0/1757140896 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559cb26cccf0 0x559cb2785da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd6dffb640 0 client.5673 ms_handle_reset on v2:172.21.15.116:6834/3863430206 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 >> 172.21.15.76:0/1757140896 conn(0x559cb26c2290 msgr2=0x559cb270b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 shutdown_connections 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 1 -- 172.21.15.76:0/1757140896 wait complete. 2024-09-19T04:15:47.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.541+0000 7fdd987ed000 20 client.5673 trim_cache size 0 max 16384 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4754, 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 1.685052794, 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 34, 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 53, 2024-09-19T04:15:47.551 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 53, 2024-09-19T04:15:47.552 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 17.040046434000001 2024-09-19T04:15:47.552 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:15:47.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd700678f0 msgr2=0x7fdd70069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd700678f0 0x7fdd70069db0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd800099b0 tx=0x7fdd80008040 comp rx=0 tx=0).stop 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd90151680 msgr2=0x7fdd9006bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd90151680 0x7fdd9006bcc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fdd88002240 tx=0x7fdd88009750 comp rx=0 tx=0).stop 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 shutdown_connections 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd700678f0 0x7fdd70069db0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdd90151680 0x7fdd9006bcc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd900aee50 0x7fdd901461f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 --2- 172.21.15.76:0/1663129644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdd7c004d50 0x7fdd90146b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 >> 172.21.15.76:0/1663129644 conn(0x7fdd90072670 msgr2=0x7fdd90072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 shutdown_connections 2024-09-19T04:15:47.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.545+0000 7fdd95731640 1 -- 172.21.15.76:0/1663129644 wait complete. 2024-09-19T04:15:47.563 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:15:47.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.673+0000 7f2d3f86f640 1 Processor -- start 2024-09-19T04:15:47.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.673+0000 7f2d3f86f640 1 -- start start 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3f86f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2d3815f840 0x7f2d380cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3f86f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d38159070 con 0x7f2d380b3e00 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3f86f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d3806d2c0 con 0x7f2d38158c90 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3f86f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2d3806d070 con 0x7f2d3815f840 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3de0c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2d380b3e00 0x7f2d380cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54506/0 (socket says 172.21.15.76:54506) 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3de0c640 1 -- 172.21.15.76:0/594607773 learned_addr learned my addr 172.21.15.76:0/594607773 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/594607773 >> v2:172.21.15.116:3300/0 conn(0x7f2d3815f840 0x7f2d380cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 201072845 0 0) 0x7f2d38159070 con 0x7f2d380b3e00 2024-09-19T04:15:47.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2d24003530 con 0x7f2d380b3e00 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3989353112 0 0) 0x7f2d24003530 con 0x7f2d380b3e00 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 >> v2:172.21.15.116:3300/0 conn(0x7f2d3815f840 msgr2=0x7f2d380cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 --2- 172.21.15.76:0/594607773 >> v2:172.21.15.116:3300/0 conn(0x7f2d3815f840 0x7f2d380cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 >> v1:172.21.15.116:6789/0 conn(0x7f2d38158c90 legacy=0x7f2d3806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d3806d5a0 con 0x7f2d380b3e00 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2d30001a00 con 0x7f2d380b3e00 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 >> v1:172.21.15.76:6789/0 conn(0x7f2d380b3e00 legacy=0x7f2d380cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d240039c0 0x7f2d24003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d24004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d240051d0 0x7f2d2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3806d070 con 0x7f2d24004810 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2400c070 con 0x7f2d240051d0 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2400c270 con 0x7f2d240039c0 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d24004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d240051d0 0x7f2d2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d240039c0 0x7f2d24003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3ce0a640 1 -- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d240039c0 msgr2=0x7f2d24003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d240039c0 0x7f2d24003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3ce0a640 1 -- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d240051d0 msgr2=0x7f2d2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d240051d0 0x7f2d2400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3ce0a640 1 -- 172.21.15.76:0/594607773 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d24003530 con 0x7f2d24004810 2024-09-19T04:15:47.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.677+0000 7f2d3de0c640 1 -- 172.21.15.76:0/594607773 reap_dead start 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d24004c90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ae0 tx=0x7f2d2801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef546e6b62f77a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d280b2070 con 0x7f2d24004810 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d280a5550 con 0x7f2d24004810 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d2ffff640 1 -- 172.21.15.76:0/594607773 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d280096b0 con 0x7f2d24004810 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- 172.21.15.76:0/594607773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 msgr2=0x7f2d24004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d24004c90 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ae0 tx=0x7f2d2801ca70 comp rx=0 tx=0).stop 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- 172.21.15.76:0/594607773 shutdown_connections 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/594607773 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d24004c90 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- 172.21.15.76:0/594607773 >> 172.21.15.76:0/594607773 conn(0x7f2d3806bcc0 msgr2=0x7f2d3806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- 172.21.15.76:0/594607773 shutdown_connections 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- 172.21.15.76:0/594607773 wait complete. 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 Processor -- start 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- start start 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d380b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d380ce180 0x7f2d38153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d38155920 0x7f2d38155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3806de40 con 0x7f2d24004810 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2d3806db60 con 0x7f2d38155920 2024-09-19T04:15:47.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3f86f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380cb9c0 con 0x7f2d380ce180 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d380b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d380b77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36134/0 (socket says 172.21.15.76:36134) 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 -- 172.21.15.76:0/208066001 learned_addr learned my addr 172.21.15.76:0/208066001 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d38155920 0x7f2d38155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d380ce180 0x7f2d38153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 -- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d380ce180 msgr2=0x7f2d38153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d380ce180 0x7f2d38153ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 -- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d38155920 msgr2=0x7f2d38155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d38155920 0x7f2d38155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 -- 172.21.15.76:0/208066001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d380b8290 con 0x7f2d24004810 2024-09-19T04:15:47.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d380b77f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e4d0 tx=0x7f2d3400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d2dffb640 1 -- 172.21.15.76:0/208066001 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d340053a0 con 0x7f2d24004810 2024-09-19T04:15:47.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d2dffb640 1 -- 172.21.15.76:0/208066001 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d34018070 con 0x7f2d24004810 2024-09-19T04:15:47.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.681+0000 7f2d2dffb640 1 -- 172.21.15.76:0/208066001 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34003820 con 0x7f2d24004810 2024-09-19T04:15:47.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d380b7d30 con 0x7f2d24004810 2024-09-19T04:15:47.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d38154e40 con 0x7f2d24004810 2024-09-19T04:15:47.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d2dffb640 1 -- 172.21.15.76:0/208066001 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2d340039c0 con 0x7f2d24004810 2024-09-19T04:15:47.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d2dffb640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2d1c0678c0 0x7f2d1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d2dffb640 1 -- 172.21.15.76:0/208066001 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2d34085a90 con 0x7f2d24004810 2024-09-19T04:15:47.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2d1c0678c0 0x7f2d1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2d1c0678c0 0x7f2d1c069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d280af000 tx=0x7f2d28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 Processor -- start 2024-09-19T04:15:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 -- start start 2024-09-19T04:15:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 0x560e10ae8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae9440 0x560e10a9ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10a9f290 0x560e10a9f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560e10986b60 con 0x560e10a9f290 2024-09-19T04:15:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560e108edc70 con 0x560e10ae8b20 2024-09-19T04:15:47.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560e10893ed0 con 0x560e10ae9440 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3de0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10a9f290 0x560e10a9f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3de0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10a9f290 0x560e10a9f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36140/0 (socket says 172.21.15.76:36140) 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3de0c640 1 -- 172.21.15.76:0/753115278 learned_addr learned my addr 172.21.15.76:0/753115278 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 0x560e10ae8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.685+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae9440 0x560e10a9ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3de0c640 1 -- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae9440 msgr2=0x560e10a9ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae9440 0x560e10a9ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3de0c640 1 -- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 msgr2=0x560e10ae8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 0x560e10ae8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3de0c640 1 -- 172.21.15.76:0/753115278 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e108a5e50 con 0x560e10a9f290 2024-09-19T04:15:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10a9f290 0x560e10a9f670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2d380cc7a0 tx=0x7f2d30005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea373995f6a5b52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d1affd640 1 -- 172.21.15.76:0/753115278 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3001c070 con 0x560e10a9f290 2024-09-19T04:15:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d1affd640 1 -- 172.21.15.76:0/753115278 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d30012990 con 0x560e10a9f290 2024-09-19T04:15:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d1affd640 1 -- 172.21.15.76:0/753115278 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d30012d60 con 0x560e10a9f290 2024-09-19T04:15:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- 172.21.15.76:0/753115278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10a9f290 msgr2=0x560e10a9f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10a9f290 0x560e10a9f670 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2d380cc7a0 tx=0x7f2d30005980 comp rx=0 tx=0).stop 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- 172.21.15.76:0/753115278 shutdown_connections 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10a9f290 0x560e10a9f670 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae9440 0x560e10a9ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 --2- 172.21.15.76:0/753115278 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 0x560e10ae8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- 172.21.15.76:0/753115278 >> 172.21.15.76:0/753115278 conn(0x560e10aa5600 msgr2=0x560e10a9d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- 172.21.15.76:0/753115278 shutdown_connections 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- 172.21.15.76:0/753115278 wait complete. 2024-09-19T04:15:47.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- start start 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 Processor -- start 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae8b20 0x560e10aa3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10aa9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 0x560e10aaa230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560e1094fb00 con 0x560e10ae8b20 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560e108c8360 con 0x560e10ae9440 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560e108c52d0 con 0x560e10aa4390 2024-09-19T04:15:47.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae8b20 0x560e10aa3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 10 client.0 _ll_register_callbacks cb 0x560e10a8a820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae8b20 0x560e10aa3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36152/0 (socket says 172.21.15.76:36152) 2024-09-19T04:15:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 -- 172.21.15.76:0/2134501616 learned_addr learned my addr 172.21.15.76:0/2134501616 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 0x560e10aaa230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10aa9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 -- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 msgr2=0x560e10aaa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 0x560e10aaa230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 -- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 msgr2=0x560e10aa9cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10aa9cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 -- 172.21.15.76:0/2134501616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e10894120 con 0x560e10ae8b20 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10aa9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae8b20 0x560e10aa3e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e4a0 tx=0x7f2d34003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2ad2eb8c534209 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d13fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d13fff640 1 -- 172.21.15.76:0/2134501616 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34081070 con 0x560e10ae8b20 2024-09-19T04:15:47.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d13fff640 1 -- 172.21.15.76:0/2134501616 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2d34055220 con 0x560e10ae8b20 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d30012990 con 0x560e10ae8b20 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d13fff640 1 -- 172.21.15.76:0/2134501616 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2d3405a180 con 0x560e10ae8b20 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 10 client.5685 fetch_fsmap learned FSMap version 34 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560e1088d450 con 0x560e10ae8b20 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d13fff640 1 -- 172.21.15.76:0/2134501616 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d30012990 con 0x560e10ae8b20 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.689+0000 7f2d13fff640 1 -- 172.21.15.76:0/2134501616 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f2d3405c380 con 0x560e10ae8b20 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 10 client.5685 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 10 client.5685 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:15:47.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 20 client.5685 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:47.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10af7280 0x560e10af9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 4 client.5685 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:47.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e10887c00 con 0x560e10af7280 2024-09-19T04:15:47.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10af7280 0x560e10af9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d13fff640 10 client.5685 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10af7280 0x560e10af9670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=88f74781a11447f8 server_cookie=27890b528e692933 in_seq=0 out_seq=0 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d13fff640 1 -- 172.21.15.76:0/2134501616 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e10887c00 con 0x560e10af7280 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d13fff640 10 client.5685 handle_command_reply: tid=0 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 client.5685 shutdown 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae8b20 msgr2=0x560e10aa3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae8b20 0x560e10aa3e50 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e4a0 tx=0x7f2d34003d70 comp rx=0 tx=0).stop 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 shutdown_connections 2024-09-19T04:15:47.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10af7280 0x560e10af9670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 0x560e10aaa230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10aa9cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d13fff640 0 client.5685 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2134501616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae8b20 0x560e10aa3e50 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 >> 172.21.15.76:0/2134501616 conn(0x560e10a9f290 msgr2=0x560e10a9dd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 shutdown_connections 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 1 -- 172.21.15.76:0/2134501616 wait complete. 2024-09-19T04:15:47.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.693+0000 7f2d4292b000 20 client.5685 trim_cache size 0 max 16384 2024-09-19T04:15:47.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.709+0000 7f2d4292b000 1 Processor -- start 2024-09-19T04:15:47.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.709+0000 7f2d4292b000 1 -- start start 2024-09-19T04:15:47.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 0x560e10ae7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 0x560e10ae8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae9440 0x560e10af9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38200890 con 0x560e10ae9440 2024-09-19T04:15:47.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2d38200f60 con 0x560e10ae8b20 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2d380cd0e0 con 0x560e10aa4390 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae9440 0x560e10af9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae9440 0x560e10af9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36156/0 (socket says 172.21.15.76:36156) 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 -- 172.21.15.76:0/2038862345 learned_addr learned my addr 172.21.15.76:0/2038862345 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 -- 172.21.15.76:0/2038862345 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 msgr2=0x560e10ae7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/2038862345 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 0x560e10ae7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 -- 172.21.15.76:0/2038862345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 msgr2=0x560e10ae8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/2038862345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 0x560e10ae8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 -- 172.21.15.76:0/2038862345 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e10af48e0 con 0x560e10ae9440 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/2038862345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae9440 0x560e10af9670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2d38162180 tx=0x7f2d30015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4d5bff4e711889 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d1a7fc640 1 -- 172.21.15.76:0/2038862345 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3002a070 con 0x560e10ae9440 2024-09-19T04:15:47.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d1a7fc640 1 -- 172.21.15.76:0/2038862345 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3001d040 con 0x560e10ae9440 2024-09-19T04:15:47.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d1a7fc640 1 -- 172.21.15.76:0/2038862345 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3001c070 con 0x560e10ae9440 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- 172.21.15.76:0/2038862345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae9440 msgr2=0x560e10af9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2038862345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae9440 0x560e10af9670 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2d38162180 tx=0x7f2d30015000 comp rx=0 tx=0).stop 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- 172.21.15.76:0/2038862345 shutdown_connections 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2038862345 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10ae9440 0x560e10af9670 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2038862345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae8b20 0x560e10ae8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- 172.21.15.76:0/2038862345 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10aa4390 0x560e10ae7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- 172.21.15.76:0/2038862345 >> 172.21.15.76:0/2038862345 conn(0x560e10a9f290 msgr2=0x560e10aa3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- 172.21.15.76:0/2038862345 shutdown_connections 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- 172.21.15.76:0/2038862345 wait complete. 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- start start 2024-09-19T04:15:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 Processor -- start 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10aa4390 0x560e10afcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae8b20 0x560e10b5c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10b5c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560e108ecf80 con 0x560e10aa4390 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560e108a5ca0 con 0x560e10ae9440 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560e108f22a0 con 0x560e10ae8b20 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d4292b000 10 client.0 _ll_register_callbacks cb 0x7f2d34055730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3d60b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10aa4390 0x560e10afcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3d60b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10aa4390 0x560e10afcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36170/0 (socket says 172.21.15.76:36170) 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3d60b640 1 -- 172.21.15.76:0/3803120878 learned_addr learned my addr 172.21.15.76:0/3803120878 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10b5c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.713+0000 7f2d3d60b640 1 -- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae8b20 msgr2=0x560e10b5c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae8b20 0x560e10b5c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3d60b640 1 -- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 msgr2=0x560e10b5c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10b5c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3d60b640 1 -- 172.21.15.76:0/3803120878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e10ae9270 con 0x560e10aa4390 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3de0c640 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10b5c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3d60b640 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10aa4390 0x560e10afcaa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2d3405c850 tx=0x7f2d34083d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d97f1d30ed3e55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 1 -- 172.21.15.76:0/3803120878 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d34081030 con 0x560e10aa4390 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 1 -- 172.21.15.76:0/3803120878 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2d3407d430 con 0x560e10aa4390 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d3001d040 con 0x560e10aa4390 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 1 -- 172.21.15.76:0/3803120878 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2d34055220 con 0x560e10aa4390 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 10 client.5691 fetch_fsmap learned FSMap version 34 2024-09-19T04:15:47.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x560e10874fd0 con 0x560e10aa4390 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 1 -- 172.21.15.76:0/3803120878 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d3001d040 con 0x560e10aa4390 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 1 -- 172.21.15.76:0/3803120878 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f2d340553c0 con 0x560e10aa4390 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 10 client.5691 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 10 client.5691 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 20 client.5691 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10b5cc00 0x560e10b58b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 4 client.5691 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "status"}) -- 0x560e108fb3a0 con 0x560e10b5cc00 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10b5cc00 0x560e10b58b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 10 client.5691 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3ce0a640 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10b5cc00 0x560e10b58b70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a539fbd8eea58e66 server_cookie=f8bc1a0827b5b13c in_seq=0 out_seq=0 2024-09-19T04:15:47.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 1 -- 172.21.15.76:0/3803120878 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x560e108fb3a0 con 0x560e10b5cc00 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 10 client.5691 handle_command_reply: tid=0 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 client.5691 shutdown 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10aa4390 msgr2=0x560e10afcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10aa4390 0x560e10afcaa0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2d3405c850 tx=0x7f2d34083d90 comp rx=0 tx=0).stop 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 shutdown_connections 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560e10b5cc00 0x560e10b58b70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560e10ae9440 0x560e10b5c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d13fff640 0 client.5691 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560e10ae8b20 0x560e10b5c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 --2- 172.21.15.76:0/3803120878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560e10aa4390 0x560e10afcaa0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 >> 172.21.15.76:0/3803120878 conn(0x560e10a9f290 msgr2=0x560e10ae6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 shutdown_connections 2024-09-19T04:15:47.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 1 -- 172.21.15.76:0/3803120878 wait complete. 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d4292b000 20 client.5691 trim_cache size 0 max 16384 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5496, 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:15:47.726 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 2.8717140689999998, 2024-09-19T04:15:47.727 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 34, 2024-09-19T04:15:47.727 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 53, 2024-09-19T04:15:47.727 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 53, 2024-09-19T04:15:47.727 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 17.230450332 2024-09-19T04:15:47.727 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2d1c0678c0 msgr2=0x7f2d1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2d1c0678c0 0x7f2d1c069d80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2d280af000 tx=0x7f2d28008040 comp rx=0 tx=0).stop 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 msgr2=0x7f2d380b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d380b77f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2d3400e4d0 tx=0x7f2d3400e990 comp rx=0 tx=0).stop 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 shutdown_connections 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2d1c0678c0 0x7f2d1c069d80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d38155920 0x7f2d38155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2d380ce180 0x7f2d38153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 --2- 172.21.15.76:0/208066001 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2d24004810 0x7f2d380b77f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 >> 172.21.15.76:0/208066001 conn(0x7f2d3806bcc0 msgr2=0x7f2d3806e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 shutdown_connections 2024-09-19T04:15:47.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.717+0000 7f2d3f86f640 1 -- 172.21.15.76:0/208066001 wait complete. 2024-09-19T04:15:47.736 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:15:47.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.849+0000 7f8ba92e5640 1 Processor -- start 2024-09-19T04:15:47.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.849+0000 7f8ba92e5640 1 -- start start 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.849+0000 7f8ba92e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba415b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.849+0000 7f8ba92e5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ba40b4e50 0x7f8ba40b52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.849+0000 7f8ba92e5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba415c040 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.849+0000 7f8ba92e5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba40cfe00 con 0x7f8ba40b4e50 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.849+0000 7f8ba92e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba415c320 con 0x7f8ba415b5a0 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba2ffd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8ba40ce0a0 0x7f8ba40d0290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:40560/0 (socket says 172.21.15.76:40560) 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/1811686832 learned_addr learned my addr 172.21.15.76:0/1811686832 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba37fe640 1 --2- 172.21.15.76:0/1811686832 >> v2:172.21.15.116:3300/0 conn(0x7f8ba40b4e50 0x7f8ba40b52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba415b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba27fc640 1 -- 172.21.15.76:0/1811686832 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2659590321 0 0) 0x7f8ba415c040 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba27fc640 1 -- 172.21.15.76:0/1811686832 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b90003530 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba37fe640 1 -- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 msgr2=0x7f8ba415b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba37fe640 1 --2- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba415b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba37fe640 1 -- 172.21.15.76:0/1811686832 >> v1:172.21.15.116:6789/0 conn(0x7f8ba40ce0a0 legacy=0x7f8ba40d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba37fe640 1 -- 172.21.15.76:0/1811686832 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba40cfa90 con 0x7f8ba40b4e50 2024-09-19T04:15:47.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:15:47.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba37fe640 1 --2- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 0x7f8ba40b52e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009d90 tx=0x7f8b9c01c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74ee2f17af28b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba27fc640 1 -- 172.21.15.76:0/1811686832 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0b2070 con 0x7f8ba40b4e50 2024-09-19T04:15:47.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba27fc640 1 -- 172.21.15.76:0/1811686832 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9c0a5830 con 0x7f8ba40b4e50 2024-09-19T04:15:47.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba27fc640 1 -- 172.21.15.76:0/1811686832 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0a5be0 con 0x7f8ba40b4e50 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba92e5640 1 -- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 msgr2=0x7f8ba40b52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 0x7f8ba40b52e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c009d90 tx=0x7f8b9c01c970 comp rx=0 tx=0).stop 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.853+0000 7f8ba92e5640 1 -- 172.21.15.76:0/1811686832 shutdown_connections 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 0x7f8ba40b52e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/1811686832 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba415b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- 172.21.15.76:0/1811686832 >> 172.21.15.76:0/1811686832 conn(0x7f8ba4073580 msgr2=0x7f8ba406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- 172.21.15.76:0/1811686832 shutdown_connections 2024-09-19T04:15:47.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- 172.21.15.76:0/1811686832 wait complete. 2024-09-19T04:15:47.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 Processor -- start 2024-09-19T04:15:47.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- start start 2024-09-19T04:15:47.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 0x7f8ba413fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba40ce0a0 0x7f8ba40b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba40b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba415d1a0 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba415d020 con 0x7f8ba40b4e50 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba4167ce0 con 0x7f8ba415b5a0 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba40ce0a0 0x7f8ba40b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba40ce0a0 0x7f8ba40b3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36182/0 (socket says 172.21.15.76:36182) 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/210883954 learned_addr learned my addr 172.21.15.76:0/210883954 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 0x7f8ba413fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba37fe640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba40b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 msgr2=0x7f8ba40b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba40b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 msgr2=0x7f8ba413fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 0x7f8ba413fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/210883954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba4157280 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba40ce0a0 0x7f8ba40b3d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ebe0 tx=0x7f8b8c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9bfff640 1 -- 172.21.15.76:0/210883954 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c017070 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9bfff640 1 -- 172.21.15.76:0/210883954 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c00cc30 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8b9bfff640 1 -- 172.21.15.76:0/210883954 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c013470 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba413e590 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.857+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba4157800 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8b9bfff640 1 -- 172.21.15.76:0/210883954 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8b8c01a070 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8b9bfff640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b840678f0 0x7f8b84069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8b9bfff640 1 -- 172.21.15.76:0/210883954 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f8b8c085d00 con 0x7f8ba40ce0a0 2024-09-19T04:15:47.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b840678f0 0x7f8b84069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b840678f0 0x7f8b84069db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005890 tx=0x7f8b94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 Processor -- start 2024-09-19T04:15:47.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 -- start start 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8cedf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8ca3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca4290 0x5572d8ca4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5572d8b8bb60 con 0x5572d8cedb20 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5572d8af2c70 con 0x5572d8ca4290 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5572d8a98ed0 con 0x5572d8cee440 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8cedf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8cedf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36196/0 (socket says 172.21.15.76:36196) 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/4110668322 learned_addr learned my addr 172.21.15.76:0/4110668322 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8ca3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba37fe640 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca4290 0x5572d8ca4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 msgr2=0x5572d8ca3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8ca3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.861+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca4290 msgr2=0x5572d8ca4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca4290 0x5572d8ca4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/4110668322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d8aaae50 con 0x5572d8cedb20 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8ca3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8cedf00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8b940058c0 tx=0x7f8b94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fc558282b4d84e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b98ff9640 1 -- 172.21.15.76:0/4110668322 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94005e40 con 0x5572d8cedb20 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b98ff9640 1 -- 172.21.15.76:0/4110668322 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9400e7b0 con 0x5572d8cedb20 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b98ff9640 1 -- 172.21.15.76:0/4110668322 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9400eb60 con 0x5572d8cedb20 2024-09-19T04:15:47.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- 172.21.15.76:0/4110668322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 msgr2=0x5572d8cedf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8cedf00 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f8b940058c0 tx=0x7f8b94011000 comp rx=0 tx=0).stop 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- 172.21.15.76:0/4110668322 shutdown_connections 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca4290 0x5572d8ca4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8ca3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/4110668322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8cedf00 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- 172.21.15.76:0/4110668322 >> 172.21.15.76:0/4110668322 conn(0x5572d8caa600 msgr2=0x5572d8ca2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- 172.21.15.76:0/4110668322 shutdown_connections 2024-09-19T04:15:47.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- 172.21.15.76:0/4110668322 wait complete. 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- start start 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 Processor -- start 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 0x5572d8ca8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cee440 0x5572d8caecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8ca9350 0x5572d8caf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5572d8b54b00 con 0x5572d8cee440 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5572d8acd360 con 0x5572d8cedb20 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5572d8aca2d0 con 0x5572d8ca9350 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8bac3a1000 10 client.0 _ll_register_callbacks cb 0x5572d8c8f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cee440 0x5572d8caecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cee440 0x5572d8caecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36206/0 (socket says 172.21.15.76:36206) 2024-09-19T04:15:47.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/3323546120 learned_addr learned my addr 172.21.15.76:0/3323546120 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8ba37fe640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8ca9350 0x5572d8caf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 0x5572d8ca8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8ca9350 msgr2=0x5572d8caf230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8ca9350 0x5572d8caf230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 msgr2=0x5572d8ca8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 0x5572d8ca8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/3323546120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d8a99120 con 0x5572d8cee440 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8ba37fe640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8ca9350 0x5572d8caf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cee440 0x5572d8caecf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ebb0 tx=0x7f8b8c0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b01f0464cbb07b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.865+0000 7f8b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 1 -- 172.21.15.76:0/3323546120 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c082060 con 0x5572d8cee440 2024-09-19T04:15:47.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 1 -- 172.21.15.76:0/3323546120 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f8b8c055690 con 0x5572d8cee440 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9400e7b0 con 0x5572d8cee440 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 1 -- 172.21.15.76:0/3323546120 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b8c06e070 con 0x5572d8cee440 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 10 client.5700 fetch_fsmap learned FSMap version 34 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5572d8a92450 con 0x5572d8cee440 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 1 -- 172.21.15.76:0/3323546120 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9400e7b0 con 0x5572d8cee440 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 1 -- 172.21.15.76:0/3323546120 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f8b8c070030 con 0x5572d8cee440 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 10 client.5700 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 10 client.5700 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 20 client.5700 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8cfc280 0x5572d8cfe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 4 client.5700 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572d8a8cc00 con 0x5572d8cfc280 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8cfc280 0x5572d8cfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 10 client.5700 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8cfc280 0x5572d8cfe670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b67c7d1e740860e server_cookie=1dd672dddcc52e6e in_seq=0 out_seq=0 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 1 -- 172.21.15.76:0/3323546120 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572d8a8cc00 con 0x5572d8cfc280 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 10 client.5700 handle_command_reply: tid=0 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 client.5700 shutdown 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cee440 msgr2=0x5572d8caecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cee440 0x5572d8caecf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ebb0 tx=0x7f8b8c0094c0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 shutdown_connections 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8ca9350 0x5572d8caf230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cee440 0x5572d8caecf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8cfc280 0x5572d8cfe670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/3323546120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 0x5572d8ca8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 >> 172.21.15.76:0/3323546120 conn(0x5572d8ca4290 msgr2=0x5572d8ca2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8b81ffb640 0 client.5700 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 shutdown_connections 2024-09-19T04:15:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 1 -- 172.21.15.76:0/3323546120 wait complete. 2024-09-19T04:15:47.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.869+0000 7f8bac3a1000 20 client.5700 trim_cache size 0 max 16384 2024-09-19T04:15:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 Processor -- start 2024-09-19T04:15:47.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- start start 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca9350 0x5572d8ceccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8ced230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8cfe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba4200dc0 con 0x5572d8cedb20 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba4201490 con 0x5572d8ca9350 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba4158b30 con 0x5572d8cee440 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8ced230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8ced230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36212/0 (socket says 172.21.15.76:36212) 2024-09-19T04:15:47.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/1643659721 learned_addr learned my addr 172.21.15.76:0/1643659721 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/1643659721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 msgr2=0x5572d8cfe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/1643659721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8cfe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/1643659721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca9350 msgr2=0x5572d8ceccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/1643659721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca9350 0x5572d8ceccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 -- 172.21.15.76:0/1643659721 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572d8a79fd0 con 0x5572d8cedb20 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/1643659721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8ced230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c086020 tx=0x7f8b8c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb66538908560e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b83fff640 1 -- 172.21.15.76:0/1643659721 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c021040 con 0x5572d8cedb20 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b83fff640 1 -- 172.21.15.76:0/1643659721 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c055690 con 0x5572d8cedb20 2024-09-19T04:15:47.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8b83fff640 1 -- 172.21.15.76:0/1643659721 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c051fd0 con 0x5572d8cedb20 2024-09-19T04:15:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1643659721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 msgr2=0x5572d8ced230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1643659721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8ced230 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c086020 tx=0x7f8b8c01e000 comp rx=0 tx=0).stop 2024-09-19T04:15:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1643659721 shutdown_connections 2024-09-19T04:15:47.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1643659721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8cfe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1643659721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8cedb20 0x5572d8ced230 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1643659721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8ca9350 0x5572d8ceccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1643659721 >> 172.21.15.76:0/1643659721 conn(0x5572d8ca4290 msgr2=0x5572d8ca8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1643659721 shutdown_connections 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.889+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1643659721 wait complete. 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- start start 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 Processor -- start 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8ca9350 0x5572d8d01500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 0x5572d8d1c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8d1cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5572d8af1f80 con 0x5572d8ca9350 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5572d8aaaca0 con 0x5572d8cedb20 2024-09-19T04:15:47.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5572d8af72a0 con 0x5572d8cee440 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 10 client.0 _ll_register_callbacks cb 0x7f8b8c004690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8ca9350 0x5572d8d01500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8ca9350 0x5572d8d01500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36222/0 (socket says 172.21.15.76:36222) 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/1141541081 learned_addr learned my addr 172.21.15.76:0/1141541081 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/1141541081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 msgr2=0x5572d8d1cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8d1cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/1141541081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 msgr2=0x5572d8d1c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 0x5572d8d1c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 -- 172.21.15.76:0/1141541081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572d8b75750 con 0x5572d8ca9350 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8ba2ffd640 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8ca9350 0x5572d8d01500 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000f70 tx=0x7f8b94000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12d43d160cc91fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 1 -- 172.21.15.76:0/1141541081 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9400eba0 con 0x5572d8ca9350 2024-09-19T04:15:47.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 1 -- 172.21.15.76:0/1141541081 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f8b9402ec40 con 0x5572d8ca9350 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8c055690 con 0x5572d8ca9350 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 1 -- 172.21.15.76:0/1141541081 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f8b9400ed40 con 0x5572d8ca9350 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 10 client.5706 fetch_fsmap learned FSMap version 34 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) -- 0x5572d8a80a30 con 0x5572d8ca9350 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 1 -- 172.21.15.76:0/1141541081 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b8c055690 con 0x5572d8ca9350 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 1 -- 172.21.15.76:0/1141541081 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 34) ==== 3542+0+0 (secure 0 0 0) 0x7f8b94021450 con 0x5572d8ca9350 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 10 client.5706 fetch_fsmap finished waiting for FSMap version 34 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 10 client.5706 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 20 client.5706 populate_metadata read hostname 'smithi076' 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8d1d3e0 0x5572d8d60c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 4 client.5706 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "status"}) -- 0x5572d8b003a0 con 0x5572d8d1d3e0 2024-09-19T04:15:47.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8d1d3e0 0x5572d8d60c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b9a7fc640 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8d1d3e0 0x5572d8d60c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8144368645a0a4df server_cookie=838e03a032ea3ebc in_seq=0 out_seq=0 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 10 client.5706 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 1 -- 172.21.15.76:0/1141541081 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5572d8b003a0 con 0x5572d8d1d3e0 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 10 client.5706 handle_command_reply: tid=0 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 client.5706 shutdown 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8ca9350 msgr2=0x5572d8d01500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8ca9350 0x5572d8d01500 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000f70 tx=0x7f8b94000bd0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 shutdown_connections 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5572d8cee440 0x5572d8d1cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5572d8d1d3e0 0x5572d8d60c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5572d8cedb20 0x5572d8d1c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8b81ffb640 0 client.5706 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:15:47.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 --2- 172.21.15.76:0/1141541081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5572d8ca9350 0x5572d8d01500 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 >> 172.21.15.76:0/1141541081 conn(0x5572d8ca4290 msgr2=0x5572d8cebce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 shutdown_connections 2024-09-19T04:15:47.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 1 -- 172.21.15.76:0/1141541081 wait complete. 2024-09-19T04:15:47.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.893+0000 7f8bac3a1000 20 client.5706 trim_cache size 0 max 16384 2024-09-19T04:15:47.903 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:15:47.903 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:15:47.903 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5502, 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 12.403603593, 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 34, 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 53, 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 53, 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 17.399384742999999 2024-09-19T04:15:47.904 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b840678f0 msgr2=0x7f8b84069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b840678f0 0x7f8b84069db0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005890 tx=0x7f8b94009040 comp rx=0 tx=0).stop 2024-09-19T04:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba40ce0a0 msgr2=0x7f8ba40b3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba40ce0a0 0x7f8ba40b3d40 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c00ebe0 tx=0x7f8b8c00c520 comp rx=0 tx=0).stop 2024-09-19T04:15:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 shutdown_connections 2024-09-19T04:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b840678f0 0x7f8b84069db0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ba415b5a0 0x7f8ba40b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ba40ce0a0 0x7f8ba40b3d40 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 --2- 172.21.15.76:0/210883954 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ba40b4e50 0x7f8ba413fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 >> 172.21.15.76:0/210883954 conn(0x7f8ba4073580 msgr2=0x7f8ba4159110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 shutdown_connections 2024-09-19T04:15:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:47.897+0000 7f8ba92e5640 1 -- 172.21.15.76:0/210883954 wait complete. 2024-09-19T04:15:47.914 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:15:48.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6c2bc640 1 Processor -- start 2024-09-19T04:15:48.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6c2bc640 1 -- start start 2024-09-19T04:15:48.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6c2bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d640ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:48.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6c2bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5d640af470 0x7f5d6406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:48.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6c2bc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d640b3290 con 0x7f5d6415c2a0 2024-09-19T04:15:48.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6c2bc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5d640b3570 con 0x7f5d640af470 2024-09-19T04:15:48.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6c2bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5d640b3850 con 0x7f5d6415bec0 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a859640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5d640af470 0x7f5d6406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a859640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5d640af470 0x7f5d6406e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36238/0 (socket says 172.21.15.76:36238) 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a859640 1 -- 172.21.15.76:0/811668162 learned_addr learned my addr 172.21.15.76:0/811668162 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a058640 1 --2- 172.21.15.76:0/811668162 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d640ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a859640 1 -- 172.21.15.76:0/811668162 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 msgr2=0x7f5d640ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a859640 1 --2- 172.21.15.76:0/811668162 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d640ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a859640 1 -- 172.21.15.76:0/811668162 >> v1:172.21.15.116:6789/0 conn(0x7f5d6415c2a0 legacy=0x7f5d640aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:15:48.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.021+0000 7f5d6a859640 1 -- 172.21.15.76:0/811668162 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d640b3b30 con 0x7f5d640af470 2024-09-19T04:15:48.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a859640 1 --2- 172.21.15.76:0/811668162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6406e000 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009e50 tx=0x7f5d5c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104c84355dc5ea56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:48.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d69056640 1 -- 172.21.15.76:0/811668162 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0a5840 con 0x7f5d640af470 2024-09-19T04:15:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d69056640 1 -- 172.21.15.76:0/811668162 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d5c0a59e0 con 0x7f5d640af470 2024-09-19T04:15:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d69056640 1 -- 172.21.15.76:0/811668162 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5c0a5db0 con 0x7f5d640af470 2024-09-19T04:15:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/811668162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 msgr2=0x7f5d6406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:48.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/811668162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6406e000 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5d5c009e50 tx=0x7f5d5c01c9c0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/811668162 shutdown_connections 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/811668162 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6406e000 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/811668162 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d640ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/811668162 >> 172.21.15.76:0/811668162 conn(0x7f5d640b1f90 msgr2=0x7f5d640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/811668162 shutdown_connections 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/811668162 wait complete. 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 Processor -- start 2024-09-19T04:15:48.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- start start 2024-09-19T04:15:48.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6414ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:48.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d6414dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:48.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6415c2a0 0x7f5d64155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:48.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5d6406ed50 con 0x7f5d640af470 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5d640ad740 con 0x7f5d6415c2a0 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6c2bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5d6406eed0 con 0x7f5d6415bec0 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6414ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6414ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36240/0 (socket says 172.21.15.76:36240) 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 -- 172.21.15.76:0/661113805 learned_addr learned my addr 172.21.15.76:0/661113805 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d69857640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d6414dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a859640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6415c2a0 0x7f5d64155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 -- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 msgr2=0x7f5d6414dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d6414dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 -- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6415c2a0 msgr2=0x7f5d64155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6415c2a0 0x7f5d64155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 -- 172.21.15.76:0/661113805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d6414d150 con 0x7f5d640af470 2024-09-19T04:15:48.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a859640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6415c2a0 0x7f5d64155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:15:48.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.025+0000 7f5d6a058640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6414ab90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d800 tx=0x7f5d5400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:48.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d5affd640 1 -- 172.21.15.76:0/661113805 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400b6c0 con 0x7f5d640af470 2024-09-19T04:15:48.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d5affd640 1 -- 172.21.15.76:0/661113805 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d54019070 con 0x7f5d640af470 2024-09-19T04:15:48.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d5affd640 1 -- 172.21.15.76:0/661113805 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d5400ba60 con 0x7f5d640af470 2024-09-19T04:15:48.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d6414e3e0 con 0x7f5d640af470 2024-09-19T04:15:48.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d6414d6a0 con 0x7f5d640af470 2024-09-19T04:15:48.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d6406f220 con 0x7f5d640af470 2024-09-19T04:15:48.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d5affd640 1 -- 172.21.15.76:0/661113805 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5d5400bc00 con 0x7f5d640af470 2024-09-19T04:15:48.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d5affd640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5d480678c0 0x7f5d48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:15:48.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d5affd640 1 -- 172.21.15.76:0/661113805 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f5d540862e0 con 0x7f5d640af470 2024-09-19T04:15:48.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d69857640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5d480678c0 0x7f5d48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:15:48.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.029+0000 7f5d69857640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5d480678c0 0x7f5d48069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5d6414ce70 tx=0x7f5d60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:15:48.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.033+0000 7f5d5affd640 1 -- 172.21.15.76:0/661113805 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5d54056c90 con 0x7f5d640af470 2024-09-19T04:15:48.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.233+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d6406fcd0 con 0x7f5d640af470 2024-09-19T04:15:48.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.233+0000 7f5d5affd640 1 -- 172.21.15.76:0/661113805 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+5267 (secure 0 0 0) 0x7f5d54056590 con 0x7f5d640af470 2024-09-19T04:15:48.236 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:15:48.237 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":34,"btime":"2024-09-19T04:15:46:858706+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:15:46.858697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[4754,5496,5502]},"id":3}]} 2024-09-19T04:15:48.237 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 34 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5d480678c0 msgr2=0x7f5d48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5d480678c0 0x7f5d48069d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5d6414ce70 tx=0x7f5d60008040 comp rx=0 tx=0).stop 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 msgr2=0x7f5d6414ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6414ab90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400d800 tx=0x7f5d5400dcc0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 shutdown_connections 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5d480678c0 0x7f5d48069d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d6415c2a0 0x7f5d64155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5d6415bec0 0x7f5d6414dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 --2- 172.21.15.76:0/661113805 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5d640af470 0x7f5d6414ab90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:15:48.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 >> 172.21.15.76:0/661113805 conn(0x7f5d640b1f90 msgr2=0x7f5d640b1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:15:48.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 shutdown_connections 2024-09-19T04:15:48.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:15:48.237+0000 7f5d6c2bc640 1 -- 172.21.15.76:0/661113805 wait complete. 2024-09-19T04:15:48.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:48.255 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:15:48.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:48.278 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:15:48.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:48.343 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-19T04:15:48.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:48.393 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-19T04:15:48.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:15:48.441 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-19T04:15:48.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:15:48.676 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:15:48.676 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:15:48.676 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:15:50.677 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:15:50.823 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:15:50.823 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:15:50.823 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:15:52.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:15:52.974 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:15:52.974 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:15:52.974 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:15:53.387 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:15:53.391 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:15:54.975 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:15:55.122 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:15:55.122 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:15:55.123 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:15:57.124 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:15:57.270 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-19T04:15:57.271 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:15:57.271 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:15:57.271 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:15:57.406 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:15:57.406 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:15:57.407 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:15:59.408 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:15:59.553 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:15:59.554 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:15:59.554 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:16:01.555 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:16:01.705 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-19T04:16:01.705 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:16:01.706 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:16:01.706 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-19T04:16:01.852 INFO:teuthology.orchestra.run.smithi116.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:16:01.853 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:16:01.853 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:16:01.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 flush journal 2024-09-19T04:16:01.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e2dba640 1 Processor -- start 2024-09-19T04:16:01.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e2dba640 1 -- start start 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e2dba640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01dc1630c0 0x7f01dc06f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e2dba640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e2dba640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01dc163bb0 con 0x7f01dc06df70 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e2dba640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f01dc06e470 con 0x7f01dc1630c0 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e2dba640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc06e750 con 0x7f01dc0b3290 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e0b56640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01dc1630c0 0x7f01dc06f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e0b56640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01dc1630c0 0x7f01dc06f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52764/0 (socket says 172.21.15.76:52764) 2024-09-19T04:16:01.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.980+0000 7f01e0b56640 1 -- 172.21.15.76:0/1834753221 learned_addr learned my addr 172.21.15.76:0/1834753221 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:01.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e1357640 1 --2- 172.21.15.76:0/1834753221 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e0b56640 1 -- 172.21.15.76:0/1834753221 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 msgr2=0x7f01dc0b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:01.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e0b56640 1 --2- 172.21.15.76:0/1834753221 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc0b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:01.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e0b56640 1 -- 172.21.15.76:0/1834753221 >> v1:172.21.15.116:6789/0 conn(0x7f01dc06df70 legacy=0x7f01dc1634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:01.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e0b56640 1 -- 172.21.15.76:0/1834753221 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc06ea30 con 0x7f01dc1630c0 2024-09-19T04:16:01.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e0b56640 1 --2- 172.21.15.76:0/1834753221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc06f510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f01dc071710 tx=0x7f01cc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c1b6c8128789e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:01.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01db7fe640 1 -- 172.21.15.76:0/1834753221 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0b0070 con 0x7f01dc1630c0 2024-09-19T04:16:01.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01db7fe640 1 -- 172.21.15.76:0/1834753221 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc0ab600 con 0x7f01dc1630c0 2024-09-19T04:16:01.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01db7fe640 1 -- 172.21.15.76:0/1834753221 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0ab9d0 con 0x7f01dc1630c0 2024-09-19T04:16:01.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 -- 172.21.15.76:0/1834753221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 msgr2=0x7f01dc06f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:01.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 --2- 172.21.15.76:0/1834753221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc06f510 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f01dc071710 tx=0x7f01cc01c500 comp rx=0 tx=0).stop 2024-09-19T04:16:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 -- 172.21.15.76:0/1834753221 shutdown_connections 2024-09-19T04:16:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 --2- 172.21.15.76:0/1834753221 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc0b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 --2- 172.21.15.76:0/1834753221 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc06f510 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 -- 172.21.15.76:0/1834753221 >> 172.21.15.76:0/1834753221 conn(0x7f01dc0b1f90 msgr2=0x7f01dc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 -- 172.21.15.76:0/1834753221 shutdown_connections 2024-09-19T04:16:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.984+0000 7f01e2dba640 1 -- 172.21.15.76:0/1834753221 wait complete. 2024-09-19T04:16:01.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 Processor -- start 2024-09-19T04:16:01.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 -- start start 2024-09-19T04:16:01.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc06df70 0x7f01dc09ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc09c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc09c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc16f770 con 0x7f01dc1630c0 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc16f5f0 con 0x7f01dc06df70 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc16f8f0 con 0x7f01dc0b3290 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc09c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc09c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52778/0 (socket says 172.21.15.76:52778) 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 -- 172.21.15.76:0/886573054 learned_addr learned my addr 172.21.15.76:0/886573054 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e0b56640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc06df70 0x7f01dc09ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01dbfff640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc09c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 -- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 msgr2=0x7f01dc09c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc09c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:01.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 -- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc06df70 msgr2=0x7f01dc09ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc06df70 0x7f01dc09ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 -- 172.21.15.76:0/886573054 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc0cead0 con 0x7f01dc1630c0 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e0b56640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc06df70 0x7f01dc09ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e1357640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc09c5a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f01d400c820 tx=0x7f01d400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01d97fa640 1 -- 172.21.15.76:0/886573054 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d4015070 con 0x7f01dc1630c0 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01d97fa640 1 -- 172.21.15.76:0/886573054 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4004220 con 0x7f01dc1630c0 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01d97fa640 1 -- 172.21.15.76:0/886573054 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d400ce10 con 0x7f01dc1630c0 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc0cedb0 con 0x7f01dc1630c0 2024-09-19T04:16:01.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.988+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc0cf370 con 0x7f01dc1630c0 2024-09-19T04:16:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01d97fa640 1 -- 172.21.15.76:0/886573054 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01d4007830 con 0x7f01dc1630c0 2024-09-19T04:16:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01d97fa640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01bc0678c0 0x7f01bc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01d97fa640 1 -- 172.21.15.76:0/886573054 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f01d4085bd0 con 0x7f01dc1630c0 2024-09-19T04:16:01.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01bc0678c0 0x7f01bc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01bc0678c0 0x7f01bc069d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0ad000 tx=0x7f01cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 Processor -- start 2024-09-19T04:16:01.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 -- start start 2024-09-19T04:16:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd62a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd62a690 0x560bbd5dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560bbd4c7960 con 0x560bbd629d70 2024-09-19T04:16:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560bbd40b900 con 0x560bbd62a690 2024-09-19T04:16:01.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560bbd42ed40 con 0x560bbd5dddb0 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd62a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd62a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52792/0 (socket says 172.21.15.76:52792) 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 -- 172.21.15.76:0/1784530366 learned_addr learned my addr 172.21.15.76:0/1784530366 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e1357640 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01dbfff640 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd62a690 0x560bbd5dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 -- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 msgr2=0x560bbd5de190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5de190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 -- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd62a690 msgr2=0x560bbd5dd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd62a690 0x560bbd5dd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 -- 172.21.15.76:0/1784530366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bbd3d5580 con 0x560bbd629d70 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01e0b56640 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd62a150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f01cc01abb0 tx=0x7f01cc0a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba790977d712690 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:01.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01d27fc640 1 -- 172.21.15.76:0/1784530366 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0c6070 con 0x560bbd629d70 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01d27fc640 1 -- 172.21.15.76:0/1784530366 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc0b5660 con 0x560bbd629d70 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.992+0000 7f01d27fc640 1 -- 172.21.15.76:0/1784530366 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc0b0040 con 0x560bbd629d70 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/1784530366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 msgr2=0x560bbd62a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd62a150 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f01cc01abb0 tx=0x7f01cc0a6000 comp rx=0 tx=0).stop 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/1784530366 shutdown_connections 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd62a690 0x560bbd5dd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1784530366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd62a150 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/1784530366 >> 172.21.15.76:0/1784530366 conn(0x560bbd60b040 msgr2=0x560bbd60d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/1784530366 shutdown_connections 2024-09-19T04:16:02.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/1784530366 wait complete. 2024-09-19T04:16:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- start start 2024-09-19T04:16:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 Processor -- start 2024-09-19T04:16:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd630ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd631220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd62a690 0x560bbd631760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560bbd42ad00 con 0x560bbd62a690 2024-09-19T04:16:02.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560bbd3e9aa0 con 0x560bbd629d70 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560bbd3c8340 con 0x560bbd5dddb0 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 10 client.0 _ll_register_callbacks cb 0x560bbd5cb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd62a690 0x560bbd631760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd62a690 0x560bbd631760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52796/0 (socket says 172.21.15.76:52796) 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 -- 172.21.15.76:0/2534972756 learned_addr learned my addr 172.21.15.76:0/2534972756 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01dbfff640 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd631220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e0b56640 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd630ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 -- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 msgr2=0x560bbd630ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd630ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 -- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 msgr2=0x560bbd631220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd631220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 -- 172.21.15.76:0/2534972756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bbd3d1dc0 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e1357640 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd62a690 0x560bbd631760 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f01d4083fa0 tx=0x7f01d4055fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5788316bfedc30c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2534972756 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d406f020 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2534972756 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f01d4055620 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01cc0b5660 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2534972756 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f01d4081020 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 10 client.5724 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x560bbd3ce7f0 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2534972756 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01cc0b5660 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2534972756 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f01d4007b20 con 0x560bbd62a690 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 10 client.5724 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:02.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 10 client.5724 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 20 client.5724 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd6390b0 0x560bbd63b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 4 client.5724 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bbd3c8e10 con 0x560bbd6390b0 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e0b56640 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd6390b0 0x560bbd63b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01bb7fe640 10 client.5724 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:01.996+0000 7f01e0b56640 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd6390b0 0x560bbd63b4a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb976e75374fe2ac server_cookie=7b1ab8fe42657b96 in_seq=0 out_seq=0 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2534972756 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560bbd3c8e10 con 0x560bbd6390b0 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01bb7fe640 10 client.5724 handle_command_reply: tid=0 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 client.5724 shutdown 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd62a690 msgr2=0x560bbd631760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd62a690 0x560bbd631760 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f01d4083fa0 tx=0x7f01d4055fd0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 shutdown_connections 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd62a690 0x560bbd631760 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd6390b0 0x560bbd63b4a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd631220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2534972756 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd630ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01bb7fe640 0 client.5724 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 >> 172.21.15.76:0/2534972756 conn(0x560bbd60b040 msgr2=0x560bbd5dcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 shutdown_connections 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 1 -- 172.21.15.76:0/2534972756 wait complete. 2024-09-19T04:16:02.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.000+0000 7f01e5e76000 20 client.5724 trim_cache size 0 max 16384 2024-09-19T04:16:02.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 Processor -- start 2024-09-19T04:16:02.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- start start 2024-09-19T04:16:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd60b040 0x560bbd63ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd5e6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc208850 con 0x560bbd629d70 2024-09-19T04:16:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc207a20 con 0x560bbd60b040 2024-09-19T04:16:02.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc208120 con 0x560bbd5dddb0 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd5e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd5e6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52810/0 (socket says 172.21.15.76:52810) 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 -- 172.21.15.76:0/1439869734 learned_addr learned my addr 172.21.15.76:0/1439869734 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e0b56640 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd60b040 0x560bbd63ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01dbfff640 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 -- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 msgr2=0x560bbd5e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5e5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 -- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd60b040 msgr2=0x560bbd63ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd60b040 0x560bbd63ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 -- 172.21.15.76:0/1439869734 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bbd3ec470 con 0x560bbd629d70 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e0b56640 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd60b040 0x560bbd63ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:02.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e1357640 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd5e6140 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f01d4055f60 tx=0x7f01d401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcf034aa089478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01d1ffb640 1 -- 172.21.15.76:0/1439869734 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d406f040 con 0x560bbd629d70 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01d1ffb640 1 -- 172.21.15.76:0/1439869734 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4055640 con 0x560bbd629d70 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01d1ffb640 1 -- 172.21.15.76:0/1439869734 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d401e070 con 0x560bbd629d70 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- 172.21.15.76:0/1439869734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 msgr2=0x560bbd5e6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd5e6140 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f01d4055f60 tx=0x7f01d401e000 comp rx=0 tx=0).stop 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- 172.21.15.76:0/1439869734 shutdown_connections 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd629d70 0x560bbd5e6140 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd5e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- 172.21.15.76:0/1439869734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd60b040 0x560bbd63ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- 172.21.15.76:0/1439869734 >> 172.21.15.76:0/1439869734 conn(0x560bbd5e2890 msgr2=0x560bbd636ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- 172.21.15.76:0/1439869734 shutdown_connections 2024-09-19T04:16:02.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- 172.21.15.76:0/1439869734 wait complete. 2024-09-19T04:16:02.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- start start 2024-09-19T04:16:02.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 Processor -- start 2024-09-19T04:16:02.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd63de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd60b040 0x560bbd659790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd659cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc09b9b0 con 0x560bbd60b040 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560bbd433370 con 0x560bbd629d70 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560bbd5e93f0 con 0x560bbd5dddb0 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01e5e76000 10 client.0 _ll_register_callbacks cb 0x7f01d400a470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd60b040 0x560bbd659790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd60b040 0x560bbd659790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52822/0 (socket says 172.21.15.76:52822) 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.016+0000 7f01dbfff640 1 -- 172.21.15.76:0/2901337325 learned_addr learned my addr 172.21.15.76:0/2901337325 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e1357640 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd659cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e0b56640 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd63de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01dbfff640 1 -- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 msgr2=0x560bbd63de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01dbfff640 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd63de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01dbfff640 1 -- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 msgr2=0x560bbd659cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01dbfff640 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd659cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01dbfff640 1 -- 172.21.15.76:0/2901337325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bbd3c2640 con 0x560bbd60b040 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01dbfff640 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd60b040 0x560bbd659790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f01c40115b0 tx=0x7f01c4011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f408ef7f1d39394 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2901337325 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c4018020 con 0x560bbd60b040 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2901337325 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f01c4003d90 con 0x560bbd60b040 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d4055640 con 0x560bbd60b040 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2901337325 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f01c4022050 con 0x560bbd60b040 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 10 client.5730 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x560bbd3c63b0 con 0x560bbd60b040 2024-09-19T04:16:02.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2901337325 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d4055640 con 0x560bbd60b040 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2901337325 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f01c4025020 con 0x560bbd60b040 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 10 client.5730 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 10 client.5730 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 20 client.5730 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd62a690 0x560bbd65a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 4 client.5730 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bbd43c470 con 0x560bbd62a690 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e0b56640 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd62a690 0x560bbd65a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01e0b56640 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd62a690 0x560bbd65a210 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=599ee35bb6472f1f server_cookie=bc0266be93483e41 in_seq=0 out_seq=0 2024-09-19T04:16:02.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.020+0000 7f01bb7fe640 10 client.5730 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01bb7fe640 1 -- 172.21.15.76:0/2901337325 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bbd43c470 con 0x560bbd62a690 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01bb7fe640 10 client.5730 handle_command_reply: tid=0 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 client.5730 shutdown 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd60b040 msgr2=0x560bbd659790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd60b040 0x560bbd659790 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f01c40115b0 tx=0x7f01c4011a70 comp rx=0 tx=0).stop 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 shutdown_connections 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x560bbd62a690 0x560bbd65a210 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560bbd629d70 0x560bbd659cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560bbd60b040 0x560bbd659790 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01bb7fe640 0 client.5730 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 --2- 172.21.15.76:0/2901337325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560bbd5dddb0 0x560bbd63de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 >> 172.21.15.76:0/2901337325 conn(0x560bbd5e2890 msgr2=0x560bbd6372f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 shutdown_connections 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 1 -- 172.21.15.76:0/2901337325 wait complete. 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e5e76000 20 client.5730 trim_cache size 0 max 16384 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:16:02.041 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:02.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01bc0678c0 msgr2=0x7f01bc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01bc0678c0 0x7f01bc069d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f01cc0ad000 tx=0x7f01cc008040 comp rx=0 tx=0).stop 2024-09-19T04:16:02.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 msgr2=0x7f01dc09c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc09c5a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f01d400c820 tx=0x7f01d400cce0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 shutdown_connections 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01bc0678c0 0x7f01bc069d80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01dc1630c0 0x7f01dc09c5a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01dc0b3290 0x7f01dc09c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 --2- 172.21.15.76:0/886573054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc06df70 0x7f01dc09ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 >> 172.21.15.76:0/886573054 conn(0x7f01dc0b1f90 msgr2=0x7f01dc1634a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 shutdown_connections 2024-09-19T04:16:02.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.036+0000 7f01e2dba640 1 -- 172.21.15.76:0/886573054 wait complete. 2024-09-19T04:16:02.056 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:1 flush journal 2024-09-19T04:16:02.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.168+0000 7f862a71c640 1 Processor -- start 2024-09-19T04:16:02.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.168+0000 7f862a71c640 1 -- start start 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f862a71c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f862415e470 0x7f86240d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f862a71c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f86240b38e0 0x7f86240d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f862a71c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f8624161e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f862a71c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f86240d3bc0 con 0x7f862415e470 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f862a71c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f86240d3730 con 0x7f86240b38e0 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f862a71c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f86240d2b20 con 0x7f86240d4750 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f862415e470 0x7f86240d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f862415e470 0x7f86240d15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52832/0 (socket says 172.21.15.76:52832) 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 -- 172.21.15.76:0/232171730 learned_addr learned my addr 172.21.15.76:0/232171730 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8628cb9640 1 --2- 172.21.15.76:0/232171730 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f8624161e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f861b7fe640 1 --2- 172.21.15.76:0/232171730 >> v2:172.21.15.116:3300/0 conn(0x7f86240b38e0 0x7f86240d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 -- 172.21.15.76:0/232171730 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 msgr2=0x7f8624161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 --2- 172.21.15.76:0/232171730 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f8624161e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 -- 172.21.15.76:0/232171730 >> v2:172.21.15.116:3300/0 conn(0x7f86240b38e0 msgr2=0x7f86240d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 --2- 172.21.15.76:0/232171730 >> v2:172.21.15.116:3300/0 conn(0x7f86240b38e0 0x7f86240d4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 -- 172.21.15.76:0/232171730 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86240d2e00 con 0x7f862415e470 2024-09-19T04:16:02.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f8623fff640 1 --2- 172.21.15.76:0/232171730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240d15d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f862409dda0 tx=0x7f861401c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81aaf28ff5bfa99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f86237fe640 1 -- 172.21.15.76:0/232171730 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140b0070 con 0x7f862415e470 2024-09-19T04:16:02.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f86237fe640 1 -- 172.21.15.76:0/232171730 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86140ab600 con 0x7f862415e470 2024-09-19T04:16:02.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.172+0000 7f86237fe640 1 -- 172.21.15.76:0/232171730 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140ab9d0 con 0x7f862415e470 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- 172.21.15.76:0/232171730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 msgr2=0x7f86240d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 --2- 172.21.15.76:0/232171730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240d15d0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f862409dda0 tx=0x7f861401c1d0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- 172.21.15.76:0/232171730 shutdown_connections 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 --2- 172.21.15.76:0/232171730 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f8624161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 --2- 172.21.15.76:0/232171730 >> v2:172.21.15.116:3300/0 conn(0x7f86240b38e0 0x7f86240d4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 --2- 172.21.15.76:0/232171730 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240d15d0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- 172.21.15.76:0/232171730 >> 172.21.15.76:0/232171730 conn(0x7f862409d730 msgr2=0x7f862408bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- 172.21.15.76:0/232171730 shutdown_connections 2024-09-19T04:16:02.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- 172.21.15.76:0/232171730 wait complete. 2024-09-19T04:16:02.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 Processor -- start 2024-09-19T04:16:02.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- start start 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86240b38e0 0x7f86240a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f862409e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f862406d330 con 0x7f862415e470 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f862406d1b0 con 0x7f86240b38e0 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f862406d4b0 con 0x7f86240d4750 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52842/0 (socket says 172.21.15.76:52842) 2024-09-19T04:16:02.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 -- 172.21.15.76:0/633883573 learned_addr learned my addr 172.21.15.76:0/633883573 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8623fff640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86240b38e0 0x7f86240a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f861b7fe640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f862409e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 -- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 msgr2=0x7f862409e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f862409e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 -- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86240b38e0 msgr2=0x7f86240a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86240b38e0 0x7f86240a0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 -- 172.21.15.76:0/633883573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862409e810 con 0x7f862415e470 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f8628cb9640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240ca430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f861c007e40 tx=0x7f861c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f86217fa640 1 -- 172.21.15.76:0/633883573 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c014620 con 0x7f862415e470 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f86217fa640 1 -- 172.21.15.76:0/633883573 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f861c019070 con 0x7f862415e470 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86240a17c0 con 0x7f862415e470 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f86217fa640 1 -- 172.21.15.76:0/633883573 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c014a20 con 0x7f862415e470 2024-09-19T04:16:02.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.176+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86240a1200 con 0x7f862415e470 2024-09-19T04:16:02.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f86217fa640 1 -- 172.21.15.76:0/633883573 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f861c014bc0 con 0x7f862415e470 2024-09-19T04:16:02.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f86217fa640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f86040678c0 0x7f8604069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 Processor -- start 2024-09-19T04:16:02.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f8623fff640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f86040678c0 0x7f8604069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f86217fa640 1 -- 172.21.15.76:0/633883573 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f861c086770 con 0x7f862415e470 2024-09-19T04:16:02.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 -- start start 2024-09-19T04:16:02.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f8623fff640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f86040678c0 0x7f8604069d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86140ad000 tx=0x7f86140a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc714150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc714690 0x5592cc6c7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc6c8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5592cc5b1960 con 0x5592cc713d70 2024-09-19T04:16:02.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5592cc4f5900 con 0x5592cc6c7db0 2024-09-19T04:16:02.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5592cc518d40 con 0x5592cc714690 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f8623fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc714150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.180+0000 7f8623fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc714150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52850/0 (socket says 172.21.15.76:52850) 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 -- 172.21.15.76:0/361445887 learned_addr learned my addr 172.21.15.76:0/361445887 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc6c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f861b7fe640 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc714690 0x5592cc6c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 -- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc714690 msgr2=0x5592cc6c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc714690 0x5592cc6c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 -- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 msgr2=0x5592cc6c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc6c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 -- 172.21.15.76:0/361445887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cc4bf580 con 0x5592cc713d70 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc6c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc714150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f861401a880 tx=0x7f8614004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cea9583e65430a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8619ffb640 1 -- 172.21.15.76:0/361445887 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140c7070 con 0x5592cc713d70 2024-09-19T04:16:02.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8619ffb640 1 -- 172.21.15.76:0/361445887 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86140abd30 con 0x5592cc713d70 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8619ffb640 1 -- 172.21.15.76:0/361445887 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86140b0040 con 0x5592cc713d70 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- 172.21.15.76:0/361445887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 msgr2=0x5592cc714150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc714150 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f861401a880 tx=0x7f8614004740 comp rx=0 tx=0).stop 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- 172.21.15.76:0/361445887 shutdown_connections 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc6c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc714690 0x5592cc6c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 --2- 172.21.15.76:0/361445887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc714150 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- 172.21.15.76:0/361445887 >> 172.21.15.76:0/361445887 conn(0x5592cc6f5040 msgr2=0x5592cc6f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- 172.21.15.76:0/361445887 shutdown_connections 2024-09-19T04:16:02.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- 172.21.15.76:0/361445887 wait complete. 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- start start 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 Processor -- start 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc71ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 0x5592cc71b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc714690 0x5592cc71b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5592cc514d00 con 0x5592cc714690 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5592cc4d3aa0 con 0x5592cc6c7db0 2024-09-19T04:16:02.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5592cc4b2340 con 0x5592cc713d70 2024-09-19T04:16:02.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 10 client.0 _ll_register_callbacks cb 0x5592cc6b5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:02.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc714690 0x5592cc71b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc714690 0x5592cc71b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52864/0 (socket says 172.21.15.76:52864) 2024-09-19T04:16:02.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 -- 172.21.15.76:0/744428111 learned_addr learned my addr 172.21.15.76:0/744428111 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f861b7fe640 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 0x5592cc71b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8623fff640 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc71ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 -- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 msgr2=0x5592cc71b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 0x5592cc71b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 -- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 msgr2=0x5592cc71ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc71ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 -- 172.21.15.76:0/744428111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cc4bbdc0 con 0x5592cc714690 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8628cb9640 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc714690 0x5592cc71b760 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f861c056680 tx=0x7f861c056a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14abe001ed52efa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8602ffd640 1 -- 172.21.15.76:0/744428111 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c056be0 con 0x5592cc714690 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8602ffd640 1 -- 172.21.15.76:0/744428111 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f861c056d80 con 0x5592cc714690 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86140abd30 con 0x5592cc714690 2024-09-19T04:16:02.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f8602ffd640 1 -- 172.21.15.76:0/744428111 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f861c05f0d0 con 0x5592cc714690 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 10 client.5742 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.184+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5592cc4b87f0 con 0x5592cc714690 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8602ffd640 1 -- 172.21.15.76:0/744428111 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86140abd30 con 0x5592cc714690 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8602ffd640 1 -- 172.21.15.76:0/744428111 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f861c055dd0 con 0x5592cc714690 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 10 client.5742 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 10 client.5742 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 20 client.5742 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc7230b0 0x5592cc7254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 4 client.5742 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592cc4b2e10 con 0x5592cc7230b0 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8623fff640 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc7230b0 0x5592cc7254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8602ffd640 10 client.5742 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:02.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8623fff640 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc7230b0 0x5592cc7254a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9bd6542bb93ac25 server_cookie=1ee2f353901eb39e in_seq=0 out_seq=0 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8602ffd640 1 -- 172.21.15.76:0/744428111 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5592cc4b2e10 con 0x5592cc7230b0 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8602ffd640 10 client.5742 handle_command_reply: tid=0 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 client.5742 shutdown 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc714690 msgr2=0x5592cc71b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc714690 0x5592cc71b760 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f861c056680 tx=0x7f861c056a40 comp rx=0 tx=0).stop 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 shutdown_connections 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc7230b0 0x5592cc7254a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc714690 0x5592cc71b760 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 0x5592cc71b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 --2- 172.21.15.76:0/744428111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc71ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f8602ffd640 0 client.5742 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:02.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 >> 172.21.15.76:0/744428111 conn(0x5592cc6f5040 msgr2=0x5592cc6c6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 shutdown_connections 2024-09-19T04:16:02.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 1 -- 172.21.15.76:0/744428111 wait complete. 2024-09-19T04:16:02.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.188+0000 7f862d7d8000 20 client.5742 trim_cache size 0 max 16384 2024-09-19T04:16:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 Processor -- start 2024-09-19T04:16:02.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- start start 2024-09-19T04:16:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6f5040 0x5592cc6cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc6c7db0 0x5592cc6d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 0x5592cc71dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f86242041e0 con 0x5592cc6c7db0 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f86242033b0 con 0x5592cc6f5040 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8624203ab0 con 0x5592cc713d70 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc6c7db0 0x5592cc6d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc6c7db0 0x5592cc6d0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52876/0 (socket says 172.21.15.76:52876) 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 -- 172.21.15.76:0/278234486 learned_addr learned my addr 172.21.15.76:0/278234486 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 -- 172.21.15.76:0/278234486 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 msgr2=0x5592cc71dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 --2- 172.21.15.76:0/278234486 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 0x5592cc71dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 -- 172.21.15.76:0/278234486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6f5040 msgr2=0x5592cc6cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 --2- 172.21.15.76:0/278234486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6f5040 0x5592cc6cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 -- 172.21.15.76:0/278234486 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592cc4d6470 con 0x5592cc6c7db0 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f861b7fe640 1 --2- 172.21.15.76:0/278234486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc6c7db0 0x5592cc6d0140 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f860c006fd0 tx=0x7f860c00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a4b281b4f6e50a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f86197fa640 1 -- 172.21.15.76:0/278234486 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c019070 con 0x5592cc6c7db0 2024-09-19T04:16:02.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f86197fa640 1 -- 172.21.15.76:0/278234486 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c005bc0 con 0x5592cc6c7db0 2024-09-19T04:16:02.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f86197fa640 1 -- 172.21.15.76:0/278234486 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860c00f040 con 0x5592cc6c7db0 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- 172.21.15.76:0/278234486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc6c7db0 msgr2=0x5592cc6d0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 --2- 172.21.15.76:0/278234486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc6c7db0 0x5592cc6d0140 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f860c006fd0 tx=0x7f860c00cd10 comp rx=0 tx=0).stop 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- 172.21.15.76:0/278234486 shutdown_connections 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 --2- 172.21.15.76:0/278234486 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc713d70 0x5592cc71dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 --2- 172.21.15.76:0/278234486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc6c7db0 0x5592cc6d0140 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 --2- 172.21.15.76:0/278234486 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6f5040 0x5592cc6cfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- 172.21.15.76:0/278234486 >> 172.21.15.76:0/278234486 conn(0x5592cc6cc770 msgr2=0x5592cc71aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- 172.21.15.76:0/278234486 shutdown_connections 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- 172.21.15.76:0/278234486 wait complete. 2024-09-19T04:16:02.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 -- start start 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.204+0000 7f862d7d8000 1 Processor -- start 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc720950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc6f5040 0x5592cc787f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc7884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f862415d9e0 con 0x5592cc713d70 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5592cc4d73f0 con 0x5592cc6c7db0 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5592cc5d50c0 con 0x5592cc6f5040 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 10 client.0 _ll_register_callbacks cb 0x7f861c00ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc7884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc7884c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52892/0 (socket says 172.21.15.76:52892) 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 -- 172.21.15.76:0/2470813537 learned_addr learned my addr 172.21.15.76:0/2470813537 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f861b7fe640 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc6f5040 0x5592cc787f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 -- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc6f5040 msgr2=0x5592cc787f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8623fff640 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc720950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc6f5040 0x5592cc787f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 -- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 msgr2=0x5592cc720950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc720950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 -- 172.21.15.76:0/2470813537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592cc4ac640 con 0x5592cc713d70 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8628cb9640 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc7884c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f861c007e10 tx=0x7f861c084940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7c48a6e11c2abf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8602ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8602ffd640 1 -- 172.21.15.76:0/2470813537 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c0562e0 con 0x5592cc713d70 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8602ffd640 1 -- 172.21.15.76:0/2470813537 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f861c06ede0 con 0x5592cc713d70 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f860c005bc0 con 0x5592cc713d70 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8602ffd640 1 -- 172.21.15.76:0/2470813537 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f861c07e170 con 0x5592cc713d70 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 10 client.5748 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5592cc4b03b0 con 0x5592cc713d70 2024-09-19T04:16:02.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8602ffd640 1 -- 172.21.15.76:0/2470813537 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f860c005bc0 con 0x5592cc713d70 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8602ffd640 1 -- 172.21.15.76:0/2470813537 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f861c077070 con 0x5592cc713d70 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 10 client.5748 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 10 client.5748 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 20 client.5748 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc714690 0x5592cc788a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 4 client.5748 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5592cc526470 con 0x5592cc714690 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8623fff640 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc714690 0x5592cc788a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8602ffd640 10 client.5748 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:02.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.208+0000 7f8623fff640 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc714690 0x5592cc788a00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5022ecc30941922a server_cookie=c0d208d0b73c940b in_seq=0 out_seq=0 2024-09-19T04:16:02.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f8602ffd640 1 -- 172.21.15.76:0/2470813537 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5592cc526470 con 0x5592cc714690 2024-09-19T04:16:02.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f8602ffd640 10 client.5748 handle_command_reply: tid=0 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 client.5748 shutdown 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 msgr2=0x5592cc7884c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc7884c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f861c007e10 tx=0x7f861c084940 comp rx=0 tx=0).stop 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 shutdown_connections 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5592cc714690 0x5592cc788a00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5592cc713d70 0x5592cc7884c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5592cc6f5040 0x5592cc787f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 --2- 172.21.15.76:0/2470813537 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5592cc6c7db0 0x5592cc720950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f8602ffd640 0 client.5748 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 >> 172.21.15.76:0/2470813537 conn(0x5592cc6cc770 msgr2=0x5592cc6cf410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 shutdown_connections 2024-09-19T04:16:02.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 1 -- 172.21.15.76:0/2470813537 wait complete. 2024-09-19T04:16:02.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862d7d8000 20 client.5748 trim_cache size 0 max 16384 2024-09-19T04:16:02.233 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:02.233 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:16:02.233 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:16:02.233 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f86040678c0 msgr2=0x7f8604069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f86040678c0 0x7f8604069d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f86140ad000 tx=0x7f86140a6000 comp rx=0 tx=0).stop 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 msgr2=0x7f86240ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240ca430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f861c007e40 tx=0x7f861c00e4b0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 shutdown_connections 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f86040678c0 0x7f8604069d80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f862415e470 0x7f86240ca430 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f86240d4750 0x7f862409e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 --2- 172.21.15.76:0/633883573 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86240b38e0 0x7f86240a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 >> 172.21.15.76:0/633883573 conn(0x7f862409d730 msgr2=0x7f8624075240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 shutdown_connections 2024-09-19T04:16:02.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.228+0000 7f862a71c640 1 -- 172.21.15.76:0/633883573 wait complete. 2024-09-19T04:16:02.249 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:2 flush journal 2024-09-19T04:16:02.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.360+0000 7f00ce963640 1 Processor -- start 2024-09-19T04:16:02.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00ce963640 1 -- start start 2024-09-19T04:16:02.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00ce963640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00c80d3060 0x7f00c80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00ce963640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c80d0300 con 0x7f00c80cf450 2024-09-19T04:16:02.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00ce963640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00c80b0f60 con 0x7f00c8161a60 2024-09-19T04:16:02.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00ce963640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f00c80cfd40 con 0x7f00c80d3060 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00bf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00c80d3060 0x7f00c80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00bf7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f00c80d3060 0x7f00c80cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52906/0 (socket says 172.21.15.76:52906) 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00bf7fe640 1 -- 172.21.15.76:0/2473443688 learned_addr learned my addr 172.21.15.76:0/2473443688 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00ccf00640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f00c80cf450 0x7f00c806d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47278/0 (socket says 172.21.15.76:47278) 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00bf7fe640 1 -- 172.21.15.76:0/2473443688 >> v1:172.21.15.116:6789/0 conn(0x7f00c8161a60 legacy=0x7f00c80d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00bf7fe640 1 -- 172.21.15.76:0/2473443688 >> v1:172.21.15.76:6789/0 conn(0x7f00c80cf450 legacy=0x7f00c806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00bf7fe640 1 -- 172.21.15.76:0/2473443688 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c80d0020 con 0x7f00c80d3060 2024-09-19T04:16:02.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00c77fe640 1 -- 172.21.15.76:0/2473443688 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4168872570 0 0) 0x7f00c00014a0 con 0x7f00c80cf450 2024-09-19T04:16:02.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00bf7fe640 1 --2- 172.21.15.76:0/2473443688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c80d3060 0x7f00c80cef10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009870 tx=0x7f00b001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166bf84ebb468c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.364+0000 7f00c77fe640 1 -- 172.21.15.76:0/2473443688 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00aa600 con 0x7f00c80d3060 2024-09-19T04:16:02.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00c77fe640 1 -- 172.21.15.76:0/2473443688 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b00af070 con 0x7f00c80d3060 2024-09-19T04:16:02.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00c77fe640 1 -- 172.21.15.76:0/2473443688 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b00aa9c0 con 0x7f00c80d3060 2024-09-19T04:16:02.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- 172.21.15.76:0/2473443688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c80d3060 msgr2=0x7f00c80cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 --2- 172.21.15.76:0/2473443688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c80d3060 0x7f00c80cef10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f00b0009870 tx=0x7f00b001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- 172.21.15.76:0/2473443688 shutdown_connections 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 --2- 172.21.15.76:0/2473443688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c80d3060 0x7f00c80cef10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- 172.21.15.76:0/2473443688 >> 172.21.15.76:0/2473443688 conn(0x7f00c808bf90 msgr2=0x7f00c809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- 172.21.15.76:0/2473443688 shutdown_connections 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- 172.21.15.76:0/2473443688 wait complete. 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 Processor -- start 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- start start 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c80cf450 0x7f00c80b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f00c80d3060 0x7f00c815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8161a60 0x7f00c815de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00c80cfa00 con 0x7f00c8161a60 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00c80cf880 con 0x7f00c80cf450 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f00c816bf30 con 0x7f00c80d3060 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8161a60 0x7f00c815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8161a60 0x7f00c815de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52916/0 (socket says 172.21.15.76:52916) 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 -- 172.21.15.76:0/2844200185 learned_addr learned my addr 172.21.15.76:0/2844200185 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00c7fff640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c80cf450 0x7f00c80b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00bf7fe640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f00c80d3060 0x7f00c815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 -- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f00c80d3060 msgr2=0x7f00c815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f00c80d3060 0x7f00c815d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 -- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c80cf450 msgr2=0x7f00c80b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c80cf450 0x7f00c80b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 -- 172.21.15.76:0/2844200185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c8155d10 con 0x7f00c8161a60 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ccf00640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8161a60 0x7f00c815de40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c700 tx=0x7f00c000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00c57fa640 1 -- 172.21.15.76:0/2844200185 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0018070 con 0x7f00c8161a60 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00c57fa640 1 -- 172.21.15.76:0/2844200185 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00c000ce10 con 0x7f00c8161a60 2024-09-19T04:16:02.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c8155fb0 con 0x7f00c8161a60 2024-09-19T04:16:02.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00c57fa640 1 -- 172.21.15.76:0/2844200185 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c000a800 con 0x7f00c8161a60 2024-09-19T04:16:02.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.368+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c81584f0 con 0x7f00c8161a60 2024-09-19T04:16:02.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c57fa640 1 -- 172.21.15.76:0/2844200185 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f00c000a9a0 con 0x7f00c8161a60 2024-09-19T04:16:02.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c57fa640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00a80678f0 0x7f00a8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00a80678f0 0x7f00a8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c57fa640 1 -- 172.21.15.76:0/2844200185 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f00c00852c0 con 0x7f00c8161a60 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00a80678f0 0x7f00a8069db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f00b8007fd0 tx=0x7f00b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 Processor -- start 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 -- start start 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6f9d70 0x56416a6fa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6fa690 0x56416a6ad870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6addb0 0x56416a6ae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56416a597960 con 0x56416a6f9d70 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56416a4db900 con 0x56416a6addb0 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56416a4fed40 con 0x56416a6fa690 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6f9d70 0x56416a6fa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6f9d70 0x56416a6fa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52932/0 (socket says 172.21.15.76:52932) 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 -- 172.21.15.76:0/509050550 learned_addr learned my addr 172.21.15.76:0/509050550 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 -- 172.21.15.76:0/509050550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6fa690 msgr2=0x56416a6ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 --2- 172.21.15.76:0/509050550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6fa690 0x56416a6ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 -- 172.21.15.76:0/509050550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6addb0 msgr2=0x56416a6ae190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.372+0000 7f00c7fff640 1 --2- 172.21.15.76:0/509050550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6addb0 0x56416a6ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00c7fff640 1 -- 172.21.15.76:0/509050550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416a4a5580 con 0x56416a6f9d70 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00c7fff640 1 --2- 172.21.15.76:0/509050550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6f9d70 0x56416a6fa150 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f00b800fb70 tx=0x7f00b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd4404690440a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00bdffb640 1 -- 172.21.15.76:0/509050550 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8025070 con 0x56416a6f9d70 2024-09-19T04:16:02.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00bdffb640 1 -- 172.21.15.76:0/509050550 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b801b050 con 0x56416a6f9d70 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00bdffb640 1 -- 172.21.15.76:0/509050550 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8014de0 con 0x56416a6f9d70 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/509050550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6f9d70 msgr2=0x56416a6fa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/509050550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6f9d70 0x56416a6fa150 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f00b800fb70 tx=0x7f00b8012000 comp rx=0 tx=0).stop 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/509050550 shutdown_connections 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/509050550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6addb0 0x56416a6ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/509050550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6fa690 0x56416a6ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/509050550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6f9d70 0x56416a6fa150 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/509050550 >> 172.21.15.76:0/509050550 conn(0x56416a6db040 msgr2=0x56416a6dd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/509050550 shutdown_connections 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/509050550 wait complete. 2024-09-19T04:16:02.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- start start 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 Processor -- start 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 0x56416a700ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 0x56416a701220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6fa690 0x56416a701760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56416a4fad00 con 0x56416a6fa690 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56416a4b9aa0 con 0x56416a6f9d70 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56416a498340 con 0x56416a6addb0 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 10 client.0 _ll_register_callbacks cb 0x56416a69b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:02.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6fa690 0x56416a701760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6fa690 0x56416a701760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52936/0 (socket says 172.21.15.76:52936) 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 -- 172.21.15.76:0/98485701 learned_addr learned my addr 172.21.15.76:0/98485701 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 -- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 msgr2=0x56416a700ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 0x56416a700ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 -- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 msgr2=0x56416a701220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 0x56416a701220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 -- 172.21.15.76:0/98485701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416a4a1dc0 con 0x56416a6fa690 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00ccf00640 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6fa690 0x56416a701760 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f00c0055200 tx=0x7f00c0055630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f360b65ed764837 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:02.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00a6ffd640 1 -- 172.21.15.76:0/98485701 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0054950 con 0x56416a6fa690 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00a6ffd640 1 -- 172.21.15.76:0/98485701 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f00c0054af0 con 0x56416a6fa690 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b801b050 con 0x56416a6fa690 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00a6ffd640 1 -- 172.21.15.76:0/98485701 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f00c0072050 con 0x56416a6fa690 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 10 client.5763 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x56416a49e7f0 con 0x56416a6fa690 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00a6ffd640 1 -- 172.21.15.76:0/98485701 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b801b050 con 0x56416a6fa690 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00a6ffd640 1 -- 172.21.15.76:0/98485701 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f00c00721f0 con 0x56416a6fa690 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 10 client.5763 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:02.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 10 client.5763 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 20 client.5763 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a7090b0 0x56416a70b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 4 client.5763 mds_command: new command op to 4754 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.376+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 --> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56416a498e10 con 0x56416a7090b0 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00c7fff640 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a7090b0 0x56416a70b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00c7fff640 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a7090b0 0x56416a70b4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74fe6e1e83e73091 server_cookie=c781a4a002bad362 in_seq=0 out_seq=0 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00a6ffd640 10 client.5763 ms_handle_connect on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00a6ffd640 1 -- 172.21.15.76:0/98485701 <== mds.2 v2:172.21.15.116:6834/3863430206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56416a498e10 con 0x56416a7090b0 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00a6ffd640 10 client.5763 handle_command_reply: tid=0 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 client.5763 shutdown 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6fa690 msgr2=0x56416a701760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6fa690 0x56416a701760 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f00c0055200 tx=0x7f00c0055630 comp rx=0 tx=0).stop 2024-09-19T04:16:02.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 shutdown_connections 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6fa690 0x56416a701760 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 0x56416a701220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a7090b0 0x56416a70b4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/98485701 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 0x56416a700ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00a6ffd640 0 client.5763 ms_handle_reset on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 >> 172.21.15.76:0/98485701 conn(0x56416a6db040 msgr2=0x56416a6acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 shutdown_connections 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 1 -- 172.21.15.76:0/98485701 wait complete. 2024-09-19T04:16:02.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.380+0000 7f00d1a1f000 20 client.5763 trim_cache size 0 max 16384 2024-09-19T04:16:02.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 Processor -- start 2024-09-19T04:16:02.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 -- start start 2024-09-19T04:16:02.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6db040 0x56416a7071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 0x56416a707720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 0x56416a70b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8203fc0 con 0x56416a6db040 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8203190 con 0x56416a6f9d70 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f00c8203890 con 0x56416a6addb0 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6db040 0x56416a7071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6db040 0x56416a7071e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52940/0 (socket says 172.21.15.76:52940) 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 -- 172.21.15.76:0/552406230 learned_addr learned my addr 172.21.15.76:0/552406230 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 -- 172.21.15.76:0/552406230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 msgr2=0x56416a707720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 --2- 172.21.15.76:0/552406230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 0x56416a707720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 -- 172.21.15.76:0/552406230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 msgr2=0x56416a70b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 --2- 172.21.15.76:0/552406230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 0x56416a70b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 -- 172.21.15.76:0/552406230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56416a4bc470 con 0x56416a6db040 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00c7fff640 1 --2- 172.21.15.76:0/552406230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6db040 0x56416a7071e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f00b8023040 tx=0x7f00b8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f41b80e4a2f5377 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00bd7fa640 1 -- 172.21.15.76:0/552406230 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8025040 con 0x56416a6db040 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00bd7fa640 1 -- 172.21.15.76:0/552406230 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b8031030 con 0x56416a6db040 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.396+0000 7f00bd7fa640 1 -- 172.21.15.76:0/552406230 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b802d8a0 con 0x56416a6db040 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/552406230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6db040 msgr2=0x56416a7071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/552406230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6db040 0x56416a7071e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f00b8023040 tx=0x7f00b8000ee0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/552406230 shutdown_connections 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/552406230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6f9d70 0x56416a70b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/552406230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6addb0 0x56416a707720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/552406230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6db040 0x56416a7071e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/552406230 >> 172.21.15.76:0/552406230 conn(0x56416a6b2890 msgr2=0x56416a706ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/552406230 shutdown_connections 2024-09-19T04:16:02.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/552406230 wait complete. 2024-09-19T04:16:02.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- start start 2024-09-19T04:16:02.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 Processor -- start 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6addb0 0x56416a70e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6db040 0x56416a76d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6f9d70 0x56416a76d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f00c80b2a10 con 0x56416a6addb0 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56416a53e780 con 0x56416a6db040 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56416a4bd7c0 con 0x56416a6f9d70 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 10 client.0 _ll_register_callbacks cb 0x7f00b00096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6addb0 0x56416a70e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6addb0 0x56416a70e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52948/0 (socket says 172.21.15.76:52948) 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 -- 172.21.15.76:0/1912297655 learned_addr learned my addr 172.21.15.76:0/1912297655 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 -- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6f9d70 msgr2=0x56416a76d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6f9d70 0x56416a76d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 -- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6db040 msgr2=0x56416a76d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6db040 0x56416a76d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 -- 172.21.15.76:0/1912297655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56416a492640 con 0x56416a6addb0 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00c7fff640 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6addb0 0x56416a70e330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x56416a4c2c90 tx=0x7f00b8001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d9f61ce3f1f61a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00a6ffd640 1 -- 172.21.15.76:0/1912297655 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b8044060 con 0x56416a6addb0 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00a6ffd640 1 -- 172.21.15.76:0/1912297655 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f00b802d6d0 con 0x56416a6addb0 2024-09-19T04:16:02.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00b8031030 con 0x56416a6addb0 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00a6ffd640 1 -- 172.21.15.76:0/1912297655 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f00b804a050 con 0x56416a6addb0 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 10 client.5769 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x56416a4963b0 con 0x56416a6addb0 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00a6ffd640 1 -- 172.21.15.76:0/1912297655 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00b8031030 con 0x56416a6addb0 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00a6ffd640 1 -- 172.21.15.76:0/1912297655 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f00b804d020 con 0x56416a6addb0 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 10 client.5769 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 10 client.5769 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 20 client.5769 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a6fa690 0x56416a729790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 4 client.5769 mds_command: new command op to 4754 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 --> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56416a50c470 con 0x56416a6fa690 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00bf7fe640 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a6fa690 0x56416a729790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00a6ffd640 10 client.5769 ms_handle_connect on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:02.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.400+0000 7f00bf7fe640 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a6fa690 0x56416a729790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e9382ce12943eae server_cookie=5d522ea2079fc726 in_seq=0 out_seq=0 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00a6ffd640 1 -- 172.21.15.76:0/1912297655 <== mds.2 v2:172.21.15.116:6834/3863430206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56416a50c470 con 0x56416a6fa690 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00a6ffd640 10 client.5769 handle_command_reply: tid=0 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 client.5769 shutdown 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6addb0 msgr2=0x56416a70e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6addb0 0x56416a70e330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x56416a4c2c90 tx=0x7f00b8001130 comp rx=0 tx=0).stop 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 shutdown_connections 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x56416a6fa690 0x56416a729790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56416a6f9d70 0x56416a76d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56416a6db040 0x56416a76d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00a6ffd640 0 client.5769 ms_handle_reset on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:02.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 --2- 172.21.15.76:0/1912297655 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56416a6addb0 0x56416a70e330 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 >> 172.21.15.76:0/1912297655 conn(0x56416a6b2890 msgr2=0x56416a7072f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 shutdown_connections 2024-09-19T04:16:02.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 1 -- 172.21.15.76:0/1912297655 wait complete. 2024-09-19T04:16:02.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00d1a1f000 20 client.5769 trim_cache size 0 max 16384 2024-09-19T04:16:02.424 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:02.425 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:16:02.425 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:16:02.425 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:02.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00a80678f0 msgr2=0x7f00a8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00a80678f0 0x7f00a8069db0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f00b8007fd0 tx=0x7f00b8009040 comp rx=0 tx=0).stop 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8161a60 msgr2=0x7f00c815de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8161a60 0x7f00c815de40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c700 tx=0x7f00c000cbc0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 shutdown_connections 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00a80678f0 0x7f00a8069db0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f00c8161a60 0x7f00c815de40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f00c80d3060 0x7f00c815d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 --2- 172.21.15.76:0/2844200185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00c80cf450 0x7f00c80b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 >> 172.21.15.76:0/2844200185 conn(0x7f00c808bf90 msgr2=0x7f00c8158870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 shutdown_connections 2024-09-19T04:16:02.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02.420+0000 7f00ce963640 1 -- 172.21.15.76:0/2844200185 wait complete. 2024-09-19T04:16:02.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:02.440 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-19T04:16:02.469 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/ 2024-09-19T04:16:02.469 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4 2024-09-19T04:16:02.469 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5 2024-09-19T04:16:02.469 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6 2024-09-19T04:16:02.469 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-19T04:16:02.469 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-19T04:16:02.470 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:16:02.470 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:16:02.470 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:16:02.470 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:02.470 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-19T04:16:02.470 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:16:02.576 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627778 2024-09-19T04:16:02.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:02.586 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:16:02.586 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:16:02.586 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:16:02.586 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:02.586 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-19T04:16:02.586 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:16:02.692 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627779 2024-09-19T04:16:02.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:02.702 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:16:02.702 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:16:02.702 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:16:02.702 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:02.702 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-19T04:16:02.702 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:16:02.810 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627780 2024-09-19T04:16:02.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:02.820 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:16:02.820 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:16:02.820 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:16:02.820 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:02.820 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-19T04:16:02.821 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:16:02.922 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627781 2024-09-19T04:16:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:02.932 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:16:02.932 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:16:02.932 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:16:02.932 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:02.932 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-19T04:16:02.933 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:16:03.035 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627782 2024-09-19T04:16:03.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:03.046 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:16:03.046 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:16:03.046 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:16:03.047 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:03.047 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-19T04:16:03.047 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:16:03.151 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627783 2024-09-19T04:16:03.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:03.160 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-19T04:16:03.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 Processor -- start 2024-09-19T04:16:03.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- start start 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a8639e0 0x556c0a907280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a87a230 0x556c0a909b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a9081b0 0x556c0a90a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556c0a909170 con 0x556c0a9081b0 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556c0a908ff0 con 0x556c0a87a230 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556c0a9092f0 con 0x556c0a8639e0 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a9081b0 0x556c0a90a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a9081b0 0x556c0a90a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36862/0 (socket says 172.21.15.76:36862) 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 -- 172.21.15.76:0/3527780253 learned_addr learned my addr 172.21.15.76:0/3527780253 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afd492640 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a8639e0 0x556c0a907280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afcc91640 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a87a230 0x556c0a909b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 -- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a8639e0 msgr2=0x556c0a907280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a8639e0 0x556c0a907280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 -- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a87a230 msgr2=0x556c0a909b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a87a230 0x556c0a909b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 -- 172.21.15.76:0/3527780253 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0a90b380 con 0x556c0a9081b0 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1afdc93640 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a9081b0 0x556c0a90a070 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1af800a650 tx=0x7f1af800ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdfe559011731bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aef7fe640 1 -- 172.21.15.76:0/3527780253 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8008c00 con 0x556c0a9081b0 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aef7fe640 1 -- 172.21.15.76:0/3527780253 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af8008da0 con 0x556c0a9081b0 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aef7fe640 1 -- 172.21.15.76:0/3527780253 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af80135f0 con 0x556c0a9081b0 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- 172.21.15.76:0/3527780253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a9081b0 msgr2=0x556c0a90a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a9081b0 0x556c0a90a070 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f1af800a650 tx=0x7f1af800ab10 comp rx=0 tx=0).stop 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- 172.21.15.76:0/3527780253 shutdown_connections 2024-09-19T04:16:03.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a9081b0 0x556c0a90a070 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a87a230 0x556c0a909b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 --2- 172.21.15.76:0/3527780253 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a8639e0 0x556c0a907280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- 172.21.15.76:0/3527780253 >> 172.21.15.76:0/3527780253 conn(0x556c0a85e220 msgr2=0x556c0a87d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- 172.21.15.76:0/3527780253 shutdown_connections 2024-09-19T04:16:03.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.240+0000 7f1aff97f080 1 -- 172.21.15.76:0/3527780253 wait complete. 2024-09-19T04:16:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 Processor -- start 2024-09-19T04:16:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 -- start start 2024-09-19T04:16:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a8639e0 0x556c0a8790e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a87a230 0x556c0a9001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a9081b0 0x556c0a874da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556c0a90b620 con 0x556c0a8639e0 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556c0a901410 con 0x556c0a9081b0 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556c0a90b8c0 con 0x556c0a87a230 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a8639e0 0x556c0a8790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a8639e0 0x556c0a8790e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36872/0 (socket says 172.21.15.76:36872) 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 -- 172.21.15.76:0/1280984386 learned_addr learned my addr 172.21.15.76:0/1280984386 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afcc91640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a87a230 0x556c0a9001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afdc93640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a9081b0 0x556c0a874da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 -- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a87a230 msgr2=0x556c0a9001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556c0a87a230 0x556c0a9001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 -- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a9081b0 msgr2=0x556c0a874da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a9081b0 0x556c0a874da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 -- 172.21.15.76:0/1280984386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c0a874120 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afdc93640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556c0a9081b0 0x556c0a874da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1afd492640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a8639e0 0x556c0a8790e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1af0019820 tx=0x7f1af0017f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aee7fc640 1 -- 172.21.15.76:0/1280984386 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af00af040 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aee7fc640 1 -- 172.21.15.76:0/1280984386 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1af00b5630 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c0a8743c0 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aee7fc640 1 -- 172.21.15.76:0/1280984386 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af00b5a20 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c0a874900 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.244+0000 7f1aee7fc640 1 -- 172.21.15.76:0/1280984386 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1af00bd020 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aee7fc640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ae00662a0 0x7f1ae0068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aee7fc640 1 -- 172.21.15.76:0/1280984386 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f1af011fe30 con 0x556c0a8639e0 2024-09-19T04:16:03.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1afcc91640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ae00662a0 0x7f1ae0068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x556c0a900710 0x556c0a876950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- osd_op(unknown.0.0:1 7.12 7:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) -- 0x556c0a876e90 con 0x556c0a900710 2024-09-19T04:16:03.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1afcc91640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ae00662a0 0x7f1ae0068760 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1af400c770 tx=0x7f1af4007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1afdc93640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x556c0a900710 0x556c0a876950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1afdc93640 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x556c0a900710 0x556c0a876950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1afdc93640 1 -- 172.21.15.76:0/1280984386 <== osd.6 v2:172.21.15.116:6816/2407229160 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v53'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1af80134f0 con 0x556c0a900710 2024-09-19T04:16:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x556c0a900710 msgr2=0x556c0a876950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x556c0a900710 0x556c0a876950 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ae00662a0 msgr2=0x7f1ae0068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ae00662a0 0x7f1ae0068760 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1af400c770 tx=0x7f1af4007040 comp rx=0 tx=0).stop 2024-09-19T04:16:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a8639e0 msgr2=0x556c0a8790e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 --2- 172.21.15.76:0/1280984386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x556c0a8639e0 0x556c0a8790e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f1af0019820 tx=0x7f1af0017f10 comp rx=0 tx=0).stop 2024-09-19T04:16:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1afdc93640 1 -- 172.21.15.76:0/1280984386 reap_dead start 2024-09-19T04:16:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 shutdown_connections 2024-09-19T04:16:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 >> 172.21.15.76:0/1280984386 conn(0x556c0a85e220 msgr2=0x556c0a87d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 shutdown_connections 2024-09-19T04:16:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.248+0000 7f1aff97f080 1 -- 172.21.15.76:0/1280984386 wait complete. 2024-09-19T04:16:03.257 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 Processor -- start 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 -- start start 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0500022b0 0x5563f5884e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 0x5563f59b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5563f59b7be0 0x5563f58961f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563f57a61b0 con 0x7ff0500022b0 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5563f57ff7b0 con 0x5563f59b72c0 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06453d080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5563f592aa70 con 0x5563f59b7be0 2024-09-19T04:16:03.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0500022b0 0x5563f5884e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0500022b0 0x5563f5884e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36880/0 (socket says 172.21.15.76:36880) 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 -- 172.21.15.76:0/2831329965 learned_addr learned my addr 172.21.15.76:0/2831329965 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062851640 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5563f59b7be0 0x5563f58961f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06184f640 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 0x5563f59b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 -- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5563f59b7be0 msgr2=0x5563f58961f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5563f59b7be0 0x5563f58961f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 -- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 msgr2=0x5563f59b76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 0x5563f59b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 -- 172.21.15.76:0/2831329965 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f5926d20 con 0x7ff0500022b0 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff062050640 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0500022b0 0x5563f5884e40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009770 tx=0x7ff05c019e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7029973f346cb081 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06084d640 1 -- 172.21.15.76:0/2831329965 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0a9880 con 0x7ff0500022b0 2024-09-19T04:16:03.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06084d640 1 -- 172.21.15.76:0/2831329965 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5563f58a09e0 con 0x7ff0500022b0 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.328+0000 7ff06084d640 1 -- 172.21.15.76:0/2831329965 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05c0a9c20 con 0x7ff0500022b0 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- 172.21.15.76:0/2831329965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0500022b0 msgr2=0x5563f5884e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0500022b0 0x5563f5884e40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009770 tx=0x7ff05c019e80 comp rx=0 tx=0).stop 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- 172.21.15.76:0/2831329965 shutdown_connections 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5563f59b7be0 0x5563f58961f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 0x5563f59b76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 --2- 172.21.15.76:0/2831329965 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff0500022b0 0x5563f5884e40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- 172.21.15.76:0/2831329965 >> 172.21.15.76:0/2831329965 conn(0x5563f5854680 msgr2=0x5563f5884280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- 172.21.15.76:0/2831329965 shutdown_connections 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- 172.21.15.76:0/2831329965 wait complete. 2024-09-19T04:16:03.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 Processor -- start 2024-09-19T04:16:03.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- start start 2024-09-19T04:16:03.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 0x5563f5925c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563f59b7be0 0x5563f59b6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0500022b0 0x5563f59b6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5563f5882070 con 0x5563f59b7be0 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5563f58980d0 con 0x5563f59b72c0 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5563f5898370 con 0x7ff0500022b0 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563f59b7be0 0x5563f59b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563f59b7be0 0x5563f59b6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36886/0 (socket says 172.21.15.76:36886) 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 -- 172.21.15.76:0/905928567 learned_addr learned my addr 172.21.15.76:0/905928567 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff062050640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 0x5563f5925c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff062851640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0500022b0 0x5563f59b6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 -- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0500022b0 msgr2=0x5563f59b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0500022b0 0x5563f59b6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 -- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 msgr2=0x5563f5925c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5563f59b72c0 0x5563f5925c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 -- 172.21.15.76:0/905928567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563f59b5aa0 con 0x5563f59b7be0 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff062851640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff0500022b0 0x5563f59b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06184f640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563f59b7be0 0x5563f59b6790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00e400 tx=0x7ff04c00e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff05b7fe640 1 -- 172.21.15.76:0/905928567 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c018070 con 0x5563f59b7be0 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff05b7fe640 1 -- 172.21.15.76:0/905928567 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff04c0135f0 con 0x5563f59b7be0 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5563f59b5d40 con 0x5563f59b7be0 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff05b7fe640 1 -- 172.21.15.76:0/905928567 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04c00f040 con 0x5563f59b7be0 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.332+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563f59b6280 con 0x5563f59b7be0 2024-09-19T04:16:03.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff05b7fe640 1 -- 172.21.15.76:0/905928567 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff04c0139e0 con 0x5563f59b7be0 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff05b7fe640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0440662d0 0x7ff044068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff05b7fe640 1 -- 172.21.15.76:0/905928567 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7ff04c083b60 con 0x5563f59b7be0 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff062050640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0440662d0 0x7ff044068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5563f59261b0 0x5563f59b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- osd_op(unknown.0.0:1 7.12 7:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) -- 0x5563f59b0fc0 con 0x5563f59261b0 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff062050640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0440662d0 0x7ff044068790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff05c01b470 tx=0x7ff05c002fc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff062851640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5563f59261b0 0x5563f59b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff062851640 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5563f59261b0 0x5563f59b49c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff062851640 1 -- 172.21.15.76:0/905928567 <== osd.6 v2:172.21.15.116:6816/2407229160 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v53'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff05400f3b0 con 0x5563f59261b0 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5563f59261b0 msgr2=0x5563f59b49c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5563f59261b0 0x5563f59b49c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0440662d0 msgr2=0x7ff044068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0440662d0 0x7ff044068790 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff05c01b470 tx=0x7ff05c002fc0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563f59b7be0 msgr2=0x5563f59b6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff06453d080 1 --2- 172.21.15.76:0/905928567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5563f59b7be0 0x5563f59b6790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00e400 tx=0x7ff04c00e8c0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.336+0000 7ff062851640 1 -- 172.21.15.76:0/905928567 reap_dead start 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.340+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 shutdown_connections 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.340+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 >> 172.21.15.76:0/905928567 conn(0x5563f5854680 msgr2=0x5563f589c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.340+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 shutdown_connections 2024-09-19T04:16:03.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.340+0000 7ff06453d080 1 -- 172.21.15.76:0/905928567 wait complete. 2024-09-19T04:16:03.346 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-19T04:16:03.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 Processor -- start 2024-09-19T04:16:03.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- start start 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0bcef0 0x555cac0d34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac0d72d0 0x555cac15b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac1610c0 0x555cac1616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555cabfde1b0 con 0x555cac0d72d0 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555cac0377b0 con 0x555cac1610c0 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555cac0bca50 con 0x555cac0bcef0 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac0d72d0 0x555cac15b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac0d72d0 0x555cac15b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36890/0 (socket says 172.21.15.76:36890) 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 -- 172.21.15.76:0/1788949098 learned_addr learned my addr 172.21.15.76:0/1788949098 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f2925960640 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac1610c0 0x555cac1616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 -- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0bcef0 msgr2=0x555cac0d34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292515f640 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0bcef0 0x555cac0d34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0bcef0 0x555cac0d34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 -- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac1610c0 msgr2=0x555cac1616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac1610c0 0x555cac1616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 -- 172.21.15.76:0/1788949098 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cac15bc90 con 0x555cac0d72d0 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292515f640 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0bcef0 0x555cac0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f2925960640 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac1610c0 0x555cac1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292495e640 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac0d72d0 0x555cac15b2a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f291000a500 tx=0x7f2910003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab39230bbb9cfce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f291f7fe640 1 -- 172.21.15.76:0/1788949098 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29100af040 con 0x555cac0d72d0 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f291f7fe640 1 -- 172.21.15.76:0/1788949098 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29100b9070 con 0x555cac0d72d0 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f291f7fe640 1 -- 172.21.15.76:0/1788949098 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29100b4410 con 0x555cac0d72d0 2024-09-19T04:16:03.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- 172.21.15.76:0/1788949098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac0d72d0 msgr2=0x555cac15b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac0d72d0 0x555cac15b2a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f291000a500 tx=0x7f2910003030 comp rx=0 tx=0).stop 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- 172.21.15.76:0/1788949098 shutdown_connections 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac1610c0 0x555cac1616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac0d72d0 0x555cac15b2a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 --2- 172.21.15.76:0/1788949098 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0bcef0 0x555cac0d34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- 172.21.15.76:0/1788949098 >> 172.21.15.76:0/1788949098 conn(0x555cac0b8220 msgr2=0x555cac0d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- 172.21.15.76:0/1788949098 shutdown_connections 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.416+0000 7f292764c080 1 -- 172.21.15.76:0/1788949098 wait complete. 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 Processor -- start 2024-09-19T04:16:03.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 -- start start 2024-09-19T04:16:03.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac0bcef0 0x555cac15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0d72d0 0x555cac200710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac1610c0 0x555cac200c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555cac08eb30 con 0x555cac1610c0 2024-09-19T04:16:03.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555cac15e0a0 con 0x555cac0bcef0 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555cac15b9f0 con 0x555cac0d72d0 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac1610c0 0x555cac200c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac1610c0 0x555cac200c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36906/0 (socket says 172.21.15.76:36906) 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 -- 172.21.15.76:0/3136171030 learned_addr learned my addr 172.21.15.76:0/3136171030 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292495e640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0d72d0 0x555cac200710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292515f640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac0bcef0 0x555cac15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 -- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0d72d0 msgr2=0x555cac200710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0d72d0 0x555cac200710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 -- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac0bcef0 msgr2=0x555cac15acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555cac0bcef0 0x555cac15acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 -- 172.21.15.76:0/3136171030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cac162160 con 0x555cac1610c0 2024-09-19T04:16:03.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292495e640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555cac0d72d0 0x555cac200710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f2925960640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac1610c0 0x555cac200c50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f291800ceb0 tx=0x7f291800a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f291e7fc640 1 -- 172.21.15.76:0/3136171030 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800f410 con 0x555cac1610c0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f291e7fc640 1 -- 172.21.15.76:0/3136171030 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2918014020 con 0x555cac1610c0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555cac162400 con 0x555cac1610c0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f291e7fc640 1 -- 172.21.15.76:0/3136171030 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291800f7f0 con 0x555cac1610c0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cac162940 con 0x555cac1610c0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f291e7fc640 1 -- 172.21.15.76:0/3136171030 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f291800f990 con 0x555cac1610c0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f291e7fc640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080662a0 0x7f2908068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f291e7fc640 1 -- 172.21.15.76:0/3136171030 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f2918084f70 con 0x555cac1610c0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292515f640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080662a0 0x7f2908068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x555cac15b200 0x555cac163080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.420+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- osd_op(unknown.0.0:1 7.5 7:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) -- 0x555cac15fa20 con 0x555cac15b200 2024-09-19T04:16:03.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292515f640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080662a0 0x7f2908068760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292495e640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x555cac15b200 0x555cac163080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292495e640 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x555cac15b200 0x555cac163080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292495e640 1 -- 172.21.15.76:0/3136171030 <== osd.7 v2:172.21.15.116:6824/2361616500 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v53'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f29100b2040 con 0x555cac15b200 2024-09-19T04:16:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x555cac15b200 msgr2=0x555cac163080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x555cac15b200 0x555cac163080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080662a0 msgr2=0x7f2908068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080662a0 0x7f2908068760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920008040 comp rx=0 tx=0).stop 2024-09-19T04:16:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac1610c0 msgr2=0x555cac200c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 --2- 172.21.15.76:0/3136171030 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555cac1610c0 0x555cac200c50 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f291800ceb0 tx=0x7f291800a5e0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f2925960640 1 -- 172.21.15.76:0/3136171030 reap_dead start 2024-09-19T04:16:03.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 shutdown_connections 2024-09-19T04:16:03.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 >> 172.21.15.76:0/3136171030 conn(0x555cac0b8220 msgr2=0x555cac0d4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 shutdown_connections 2024-09-19T04:16:03.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.424+0000 7f292764c080 1 -- 172.21.15.76:0/3136171030 wait complete. 2024-09-19T04:16:03.433 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-19T04:16:03.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 Processor -- start 2024-09-19T04:16:03.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- start start 2024-09-19T04:16:03.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd590660 0x55b3bd5931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd54ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd6226c0 0x55b3bd57bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd595340 con 0x55b3bd592df0 2024-09-19T04:16:03.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd49c1b0 con 0x55b3bd590660 2024-09-19T04:16:03.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b3bd5954c0 con 0x55b3bd6226c0 2024-09-19T04:16:03.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd54ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd54ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36912/0 (socket says 172.21.15.76:36912) 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 -- 172.21.15.76:0/2534299072 learned_addr learned my addr 172.21.15.76:0/2534299072 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde30f8e640 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd6226c0 0x55b3bd57bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde3078d640 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd590660 0x55b3bd5931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 -- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd6226c0 msgr2=0x55b3bd57bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd6226c0 0x55b3bd57bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 -- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd590660 msgr2=0x55b3bd5931d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd590660 0x55b3bd5931d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 -- 172.21.15.76:0/2534299072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bd57d0b0 con 0x55b3bd592df0 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ff8c640 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd54ada0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fde180ad040 tx=0x7fde18003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df1f6836116b789d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ef8a640 1 -- 172.21.15.76:0/2534299072 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180af040 con 0x55b3bd592df0 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ef8a640 1 -- 172.21.15.76:0/2534299072 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde180b9070 con 0x55b3bd592df0 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde2ef8a640 1 -- 172.21.15.76:0/2534299072 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180aacd0 con 0x55b3bd592df0 2024-09-19T04:16:03.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- 172.21.15.76:0/2534299072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 msgr2=0x55b3bd54ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd54ada0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fde180ad040 tx=0x7fde18003030 comp rx=0 tx=0).stop 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- 172.21.15.76:0/2534299072 shutdown_connections 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd6226c0 0x55b3bd57bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd54ada0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 --2- 172.21.15.76:0/2534299072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd590660 0x55b3bd5931d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- 172.21.15.76:0/2534299072 >> 172.21.15.76:0/2534299072 conn(0x55b3bd577ef0 msgr2=0x55b3bd57a2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- 172.21.15.76:0/2534299072 shutdown_connections 2024-09-19T04:16:03.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.508+0000 7fde32c7a080 1 -- 172.21.15.76:0/2534299072 wait complete. 2024-09-19T04:16:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 Processor -- start 2024-09-19T04:16:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 -- start start 2024-09-19T04:16:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd590660 0x55b3bd617f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd621040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd6226c0 0x55b3bd61c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd57b5a0 con 0x55b3bd592df0 2024-09-19T04:16:03.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd57d350 con 0x55b3bd6226c0 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b3bd57b840 con 0x55b3bd590660 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd621040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd621040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36924/0 (socket says 172.21.15.76:36924) 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 -- 172.21.15.76:0/3324082336 learned_addr learned my addr 172.21.15.76:0/3324082336 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde3078d640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd590660 0x55b3bd617f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde30f8e640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd6226c0 0x55b3bd61c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 -- 172.21.15.76:0/3324082336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd590660 msgr2=0x55b3bd617f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b3bd590660 0x55b3bd617f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 -- 172.21.15.76:0/3324082336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd6226c0 msgr2=0x55b3bd61c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b3bd6226c0 0x55b3bd61c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 -- 172.21.15.76:0/3324082336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bd61d730 con 0x55b3bd592df0 2024-09-19T04:16:03.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2ff8c640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd621040 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fde18017ff0 tx=0x7fde18018180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2df88640 1 -- 172.21.15.76:0/3324082336 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180af040 con 0x55b3bd592df0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2df88640 1 -- 172.21.15.76:0/3324082336 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde180c7070 con 0x55b3bd592df0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b3bd621b50 con 0x55b3bd592df0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2df88640 1 -- 172.21.15.76:0/3324082336 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180b4870 con 0x55b3bd592df0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3bd54c350 con 0x55b3bd592df0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2df88640 1 -- 172.21.15.76:0/3324082336 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fde180b4a10 con 0x55b3bd592df0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.512+0000 7fde2df88640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fde100662a0 0x7fde10068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.516+0000 7fde2df88640 1 -- 172.21.15.76:0/3324082336 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fde18120b20 con 0x55b3bd592df0 2024-09-19T04:16:03.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.516+0000 7fde3078d640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fde100662a0 0x7fde10068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.516+0000 7fde32c7a080 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b3bd54ca90 0x55b3bd54ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.516+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 7.9 7:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) -- 0x55b3bd54d3b0 con 0x55b3bd54ca90 2024-09-19T04:16:03.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.516+0000 7fde30f8e640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b3bd54ca90 0x55b3bd54ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.516+0000 7fde3078d640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fde100662a0 0x7fde10068760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde20006fd0 tx=0x7fde20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.516+0000 7fde30f8e640 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b3bd54ca90 0x55b3bd54ce70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde30f8e640 1 -- 172.21.15.76:0/3324082336 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v53'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fde280096e0 con 0x55b3bd54ca90 2024-09-19T04:16:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b3bd54ca90 msgr2=0x55b3bd54ce70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55b3bd54ca90 0x55b3bd54ce70 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fde100662a0 msgr2=0x7fde10068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fde100662a0 0x7fde10068760 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde20006fd0 tx=0x7fde20008040 comp rx=0 tx=0).stop 2024-09-19T04:16:03.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 msgr2=0x55b3bd621040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 --2- 172.21.15.76:0/3324082336 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b3bd592df0 0x55b3bd621040 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fde18017ff0 tx=0x7fde18018180 comp rx=0 tx=0).stop 2024-09-19T04:16:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde30f8e640 1 -- 172.21.15.76:0/3324082336 reap_dead start 2024-09-19T04:16:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 shutdown_connections 2024-09-19T04:16:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 >> 172.21.15.76:0/3324082336 conn(0x55b3bd577ef0 msgr2=0x55b3bd61bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 shutdown_connections 2024-09-19T04:16:03.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.520+0000 7fde32c7a080 1 -- 172.21.15.76:0/3324082336 wait complete. 2024-09-19T04:16:03.527 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-19T04:16:03.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 Processor -- start 2024-09-19T04:16:03.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- start start 2024-09-19T04:16:03.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae23f25f0 0x557ae23ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae2466140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae2480c90 0x557ae2466680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557ae23d9970 con 0x557ae23f29d0 2024-09-19T04:16:03.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ae22fa1b0 con 0x557ae23f25f0 2024-09-19T04:16:03.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x557ae23d8810 con 0x557ae2480c90 2024-09-19T04:16:03.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae2466140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae2466140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36942/0 (socket says 172.21.15.76:36942) 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 -- 172.21.15.76:0/282124426 learned_addr learned my addr 172.21.15.76:0/282124426 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7eaa9640 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae2480c90 0x557ae2466680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 -- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae2480c90 msgr2=0x557ae2466680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae2480c90 0x557ae2466680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 -- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae23f25f0 msgr2=0x557ae23ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae23f25f0 0x557ae23ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 -- 172.21.15.76:0/282124426 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae23eccd0 con 0x557ae23f29d0 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7eaa9640 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae2480c90 0x557ae2466680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7daa7640 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae2466140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f680ad040 tx=0x7f9f68003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41001e59d9bb251 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7caa5640 1 -- 172.21.15.76:0/282124426 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680af040 con 0x557ae23f29d0 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7caa5640 1 -- 172.21.15.76:0/282124426 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f680b9070 con 0x557ae23f29d0 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f7caa5640 1 -- 172.21.15.76:0/282124426 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680b4410 con 0x557ae23f29d0 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- 172.21.15.76:0/282124426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 msgr2=0x557ae2466140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae2466140 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f9f680ad040 tx=0x7f9f68003030 comp rx=0 tx=0).stop 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- 172.21.15.76:0/282124426 shutdown_connections 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae2480c90 0x557ae2466680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae2466140 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 --2- 172.21.15.76:0/282124426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae23f25f0 0x557ae23ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- 172.21.15.76:0/282124426 >> 172.21.15.76:0/282124426 conn(0x557ae23d3940 msgr2=0x557ae2483370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- 172.21.15.76:0/282124426 shutdown_connections 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- 172.21.15.76:0/282124426 wait complete. 2024-09-19T04:16:03.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 Processor -- start 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.608+0000 7f9f80795080 1 -- start start 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae23f25f0 0x557ae245ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae23aa110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae2480c90 0x557ae23aa650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x557ae23ed210 con 0x557ae23f29d0 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x557ae23ecf70 con 0x557ae2480c90 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x557ae23ed4b0 con 0x557ae23f25f0 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae23aa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae23aa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36954/0 (socket says 172.21.15.76:36954) 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 -- 172.21.15.76:0/538515008 learned_addr learned my addr 172.21.15.76:0/538515008 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7e2a8640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae23f25f0 0x557ae245ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7eaa9640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae2480c90 0x557ae23aa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 -- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae23f25f0 msgr2=0x557ae245ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x557ae23f25f0 0x557ae245ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 -- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae2480c90 msgr2=0x557ae23aa650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x557ae2480c90 0x557ae23aa650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 -- 172.21.15.76:0/538515008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ae2470980 con 0x557ae23f29d0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7daa7640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae23aa110 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9f680b6040 tx=0x7f9f68018030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f777fe640 1 -- 172.21.15.76:0/538515008 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680af040 con 0x557ae23f29d0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f777fe640 1 -- 172.21.15.76:0/538515008 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f680b4d50 con 0x557ae23f29d0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557ae2470c20 con 0x557ae23f29d0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f777fe640 1 -- 172.21.15.76:0/538515008 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f680c6680 con 0x557ae23f29d0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ae2471160 con 0x557ae23f29d0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f777fe640 1 -- 172.21.15.76:0/538515008 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9f680be400 con 0x557ae23f29d0 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f777fe640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f600662a0 0x7f9f60068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f777fe640 1 -- 172.21.15.76:0/538515008 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9f68121890 con 0x557ae23f29d0 2024-09-19T04:16:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7e2a8640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f600662a0 0x7f9f60068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x557ae24662a0 0x557ae2466680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- osd_op(unknown.0.0:1 7.5 7:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) -- 0x557ae2466bc0 con 0x557ae24662a0 2024-09-19T04:16:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7e2a8640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f600662a0 0x7f9f60068760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f78001650 tx=0x7f9f78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.612+0000 7f9f7eaa9640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x557ae24662a0 0x557ae2466680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f7eaa9640 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x557ae24662a0 0x557ae2466680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f7eaa9640 1 -- 172.21.15.76:0/538515008 <== osd.7 v2:172.21.15.116:6824/2361616500 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v53'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9f70009650 con 0x557ae24662a0 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x557ae24662a0 msgr2=0x557ae2466680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x557ae24662a0 0x557ae2466680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f600662a0 msgr2=0x7f9f60068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9f600662a0 0x7f9f60068760 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9f78001650 tx=0x7f9f78008040 comp rx=0 tx=0).stop 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 msgr2=0x557ae23aa110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 --2- 172.21.15.76:0/538515008 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x557ae23f29d0 0x557ae23aa110 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9f680b6040 tx=0x7f9f68018030 comp rx=0 tx=0).stop 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f7eaa9640 1 -- 172.21.15.76:0/538515008 reap_dead start 2024-09-19T04:16:03.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 shutdown_connections 2024-09-19T04:16:03.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 >> 172.21.15.76:0/538515008 conn(0x557ae23d3940 msgr2=0x557ae23d7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 shutdown_connections 2024-09-19T04:16:03.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.616+0000 7f9f80795080 1 -- 172.21.15.76:0/538515008 wait complete. 2024-09-19T04:16:03.626 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-19T04:16:03.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.700+0000 7f275c9f6080 1 Processor -- start 2024-09-19T04:16:03.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.700+0000 7f275c9f6080 1 -- start start 2024-09-19T04:16:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8b79c0 0x56221b8b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b8b1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8d5c50 0x56221b8b1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56221b8bbd20 con 0x56221b8b79c0 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56221b7dd1b0 con 0x56221b8d5870 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56221b8bbea0 con 0x56221b8d5c50 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8b79c0 0x56221b8b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8b79c0 0x56221b8b8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36968/0 (socket says 172.21.15.76:36968) 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 -- 172.21.15.76:0/1828253140 learned_addr learned my addr 172.21.15.76:0/1828253140 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f2759d08640 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b8b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275ad0a640 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8d5c50 0x56221b8b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 -- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8d5c50 msgr2=0x56221b8b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8d5c50 0x56221b8b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 -- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 msgr2=0x56221b8b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b8b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 -- 172.21.15.76:0/1828253140 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221b88f860 con 0x56221b8b79c0 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f2759d08640 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b8b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8b79c0 0x56221b8b8fd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f275000b330 tx=0x7f275000b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df59ab11f2791ecb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f2758d06640 1 -- 172.21.15.76:0/1828253140 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2750012020 con 0x56221b8b79c0 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f2758d06640 1 -- 172.21.15.76:0/1828253140 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275000d6b0 con 0x56221b8b79c0 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f2758d06640 1 -- 172.21.15.76:0/1828253140 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275000da80 con 0x56221b8b79c0 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- 172.21.15.76:0/1828253140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8b79c0 msgr2=0x56221b8b8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8b79c0 0x56221b8b8fd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f275000b330 tx=0x7f275000b7f0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- 172.21.15.76:0/1828253140 shutdown_connections 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8d5c50 0x56221b8b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b8b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- 172.21.15.76:0/1828253140 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8b79c0 0x56221b8b8fd0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- 172.21.15.76:0/1828253140 >> 172.21.15.76:0/1828253140 conn(0x56221b88b680 msgr2=0x56221b8b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- 172.21.15.76:0/1828253140 shutdown_connections 2024-09-19T04:16:03.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- 172.21.15.76:0/1828253140 wait complete. 2024-09-19T04:16:03.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 Processor -- start 2024-09-19T04:16:03.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- start start 2024-09-19T04:16:03.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8b79c0 0x56221b9439f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b9431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8d5c50 0x56221b8b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56221b95e810 con 0x56221b8d5c50 2024-09-19T04:16:03.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56221b95f730 con 0x56221b8d5870 2024-09-19T04:16:03.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275c9f6080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56221b95eab0 con 0x56221b8b79c0 2024-09-19T04:16:03.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275ad0a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8d5c50 0x56221b8b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275ad0a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8d5c50 0x56221b8b2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36982/0 (socket says 172.21.15.76:36982) 2024-09-19T04:16:03.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275ad0a640 1 -- 172.21.15.76:0/492165741 learned_addr learned my addr 172.21.15.76:0/492165741 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f2759d08640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b9431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.704+0000 7f275a509640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8b79c0 0x56221b9439f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275ad0a640 1 -- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8b79c0 msgr2=0x56221b9439f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275ad0a640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8b79c0 0x56221b9439f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275ad0a640 1 -- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 msgr2=0x56221b9431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275ad0a640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b9431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275ad0a640 1 -- 172.21.15.76:0/492165741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221b943700 con 0x56221b8d5c50 2024-09-19T04:16:03.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f2759d08640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56221b8d5870 0x56221b9431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275a509640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56221b8b79c0 0x56221b9439f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275ad0a640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8d5c50 0x56221b8b2eb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2754019720 tx=0x7f2754017e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f274b7fe640 1 -- 172.21.15.76:0/492165741 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540b9020 con 0x56221b8d5c50 2024-09-19T04:16:03.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f274b7fe640 1 -- 172.21.15.76:0/492165741 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27540b4670 con 0x56221b8d5c50 2024-09-19T04:16:03.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56221b944090 con 0x56221b8d5c50 2024-09-19T04:16:03.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f274b7fe640 1 -- 172.21.15.76:0/492165741 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27540af040 con 0x56221b8d5c50 2024-09-19T04:16:03.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221b9445d0 con 0x56221b8d5c50 2024-09-19T04:16:03.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f274b7fe640 1 -- 172.21.15.76:0/492165741 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f27540bc020 con 0x56221b8d5c50 2024-09-19T04:16:03.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f274b7fe640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f273c0662d0 0x7f273c068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f274b7fe640 1 -- 172.21.15.76:0/492165741 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f275411f8f0 con 0x56221b8d5c50 2024-09-19T04:16:03.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275a509640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f273c0662d0 0x7f273c068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275c9f6080 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x56221b946e80 0x56221b947260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- osd_op(unknown.0.0:1 7.1d 7:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53) -- 0x56221b95d4b0 con 0x56221b946e80 2024-09-19T04:16:03.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f2759d08640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x56221b946e80 0x56221b947260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f275a509640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f273c0662d0 0x7f273c068790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2750013000 tx=0x7f2750009000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.708+0000 7f2759d08640 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x56221b946e80 0x56221b947260 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f2759d08640 1 -- 172.21.15.76:0/492165741 <== osd.1 v2:172.21.15.76:6808/1010487110 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v53'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f274c009a20 con 0x56221b946e80 2024-09-19T04:16:03.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x56221b946e80 msgr2=0x56221b947260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x56221b946e80 0x56221b947260 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f273c0662d0 msgr2=0x7f273c068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f273c0662d0 0x7f273c068790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f2750013000 tx=0x7f2750009000 comp rx=0 tx=0).stop 2024-09-19T04:16:03.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8d5c50 msgr2=0x56221b8b2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 --2- 172.21.15.76:0/492165741 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56221b8d5c50 0x56221b8b2eb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2754019720 tx=0x7f2754017e10 comp rx=0 tx=0).stop 2024-09-19T04:16:03.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275ad0a640 1 -- 172.21.15.76:0/492165741 reap_dead start 2024-09-19T04:16:03.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 shutdown_connections 2024-09-19T04:16:03.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 >> 172.21.15.76:0/492165741 conn(0x56221b88b680 msgr2=0x56221b8b1740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 shutdown_connections 2024-09-19T04:16:03.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.712+0000 7f275c9f6080 1 -- 172.21.15.76:0/492165741 wait complete. 2024-09-19T04:16:03.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:03.725 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start /d1/d2/d3 recursive,force 2024-09-19T04:16:03.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9decea9640 1 Processor -- start 2024-09-19T04:16:03.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9decea9640 1 -- start start 2024-09-19T04:16:03.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9decea9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9de815bec0 0x7f9de80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9decea9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de80af470 0x7f9de806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9decea9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9de80b3290 con 0x7f9de815c2a0 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9decea9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9de80b3570 con 0x7f9de815bec0 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9decea9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9de80b3850 con 0x7f9de80af470 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9de815bec0 0x7f9de80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de5d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9de815c2a0 0x7f9de80aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47298/0 (socket says 172.21.15.76:47298) 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9de815bec0 0x7f9de80ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:36996/0 (socket says 172.21.15.76:36996) 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de5d9b640 1 -- 172.21.15.76:0/1434905373 learned_addr learned my addr 172.21.15.76:0/1434905373 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/1434905373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de80af470 0x7f9de806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de659c640 1 -- 172.21.15.76:0/1434905373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de80af470 msgr2=0x7f9de806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de659c640 1 --2- 172.21.15.76:0/1434905373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de80af470 0x7f9de806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de659c640 1 -- 172.21.15.76:0/1434905373 >> v1:172.21.15.76:6789/0 conn(0x7f9de815c2a0 legacy=0x7f9de80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de659c640 1 -- 172.21.15.76:0/1434905373 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de80b3b30 con 0x7f9de815bec0 2024-09-19T04:16:03.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.892+0000 7f9de559a640 1 -- 172.21.15.76:0/1434905373 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2543794934 0 0) 0x7f9de80b3290 con 0x7f9de815c2a0 2024-09-19T04:16:03.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9de659c640 1 --2- 172.21.15.76:0/1434905373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815bec0 0x7f9de80ad200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009d90 tx=0x7f9de001c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5889db00234908 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9de559a640 1 -- 172.21.15.76:0/1434905373 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b2070 con 0x7f9de815bec0 2024-09-19T04:16:03.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9de559a640 1 -- 172.21.15.76:0/1434905373 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de00a5970 con 0x7f9de815bec0 2024-09-19T04:16:03.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9de559a640 1 -- 172.21.15.76:0/1434905373 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00a5d40 con 0x7f9de815bec0 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- 172.21.15.76:0/1434905373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815bec0 msgr2=0x7f9de80ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 --2- 172.21.15.76:0/1434905373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815bec0 0x7f9de80ad200 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009d90 tx=0x7f9de001c8b0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- 172.21.15.76:0/1434905373 shutdown_connections 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 --2- 172.21.15.76:0/1434905373 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de80af470 0x7f9de806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 --2- 172.21.15.76:0/1434905373 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815bec0 0x7f9de80ad200 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- 172.21.15.76:0/1434905373 >> 172.21.15.76:0/1434905373 conn(0x7f9de80b1f90 msgr2=0x7f9de80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- 172.21.15.76:0/1434905373 shutdown_connections 2024-09-19T04:16:03.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- 172.21.15.76:0/1434905373 wait complete. 2024-09-19T04:16:03.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 Processor -- start 2024-09-19T04:16:03.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- start start 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9de80af470 0x7f9de814ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de815bec0 0x7f9de80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815c2a0 0x7f9de80d0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9de806ed50 con 0x7f9de815c2a0 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9de80ad740 con 0x7f9de80af470 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.896+0000 7f9decea9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9de806eed0 con 0x7f9de815bec0 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815c2a0 0x7f9de80d0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815c2a0 0x7f9de80d0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37004/0 (socket says 172.21.15.76:37004) 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 -- 172.21.15.76:0/1947816462 learned_addr learned my addr 172.21.15.76:0/1947816462 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de659c640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9de80af470 0x7f9de814ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de815bec0 0x7f9de80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 -- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de815bec0 msgr2=0x7f9de80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de815bec0 0x7f9de80cfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 -- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9de80af470 msgr2=0x7f9de814ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9de80af470 0x7f9de814ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 -- 172.21.15.76:0/1947816462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9de8157760 con 0x7f9de815c2a0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de659c640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9de80af470 0x7f9de814ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815c2a0 0x7f9de80d0150 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d800 tx=0x7f9dd800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9ddeffd640 1 -- 172.21.15.76:0/1947816462 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800b6c0 con 0x7f9de815c2a0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9ddeffd640 1 -- 172.21.15.76:0/1947816462 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd8019070 con 0x7f9de815c2a0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9de8164be0 con 0x7f9de815c2a0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9ddeffd640 1 -- 172.21.15.76:0/1947816462 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd800baa0 con 0x7f9de815c2a0 2024-09-19T04:16:03.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9de81651a0 con 0x7f9de815c2a0 2024-09-19T04:16:03.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9ddeffd640 1 -- 172.21.15.76:0/1947816462 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9dd800bc40 con 0x7f9de815c2a0 2024-09-19T04:16:03.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9ddeffd640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9dc80678c0 0x7f9dc8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9ddeffd640 1 -- 172.21.15.76:0/1947816462 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9dd80861b0 con 0x7f9de815c2a0 2024-09-19T04:16:03.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de659c640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9dc80678c0 0x7f9dc8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9de659c640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9dc80678c0 0x7f9dc8069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9de00af000 tx=0x7f9de00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9deff65000 1 Processor -- start 2024-09-19T04:16:03.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.900+0000 7f9deff65000 1 -- start start 2024-09-19T04:16:03.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c7b40 0x5570ee7c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee77e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee77ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5570ee5ccbd0 con 0x5570ee7c7b40 2024-09-19T04:16:03.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5570ee5722f0 con 0x5570ee7c8460 2024-09-19T04:16:03.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5570ee5c8b90 con 0x5570ee77e830 2024-09-19T04:16:03.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c7b40 0x5570ee7c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c7b40 0x5570ee7c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37018/0 (socket says 172.21.15.76:37018) 2024-09-19T04:16:03.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 -- 172.21.15.76:0/3055544316 learned_addr learned my addr 172.21.15.76:0/3055544316 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee77e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee77ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 -- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 msgr2=0x5570ee77ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee77ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 -- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 msgr2=0x5570ee77e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee77e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 -- 172.21.15.76:0/3055544316 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ee598b10 con 0x5570ee7c7b40 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee77ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c7b40 0x5570ee7c7f20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9de00095c0 tx=0x7f9de0004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e493480b7099e32 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9dc7fff640 1 -- 172.21.15.76:0/3055544316 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b2040 con 0x5570ee7c7b40 2024-09-19T04:16:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9dc7fff640 1 -- 172.21.15.76:0/3055544316 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de00b87b0 con 0x5570ee7c7b40 2024-09-19T04:16:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9dc7fff640 1 -- 172.21.15.76:0/3055544316 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b8b80 con 0x5570ee7c7b40 2024-09-19T04:16:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- 172.21.15.76:0/3055544316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c7b40 msgr2=0x5570ee7c7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c7b40 0x5570ee7c7f20 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9de00095c0 tx=0x7f9de0004ac0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- 172.21.15.76:0/3055544316 shutdown_connections 2024-09-19T04:16:03.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee77ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee77e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- 172.21.15.76:0/3055544316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c7b40 0x5570ee7c7f20 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- 172.21.15.76:0/3055544316 >> 172.21.15.76:0/3055544316 conn(0x5570ee77a7e0 msgr2=0x5570ee77cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- 172.21.15.76:0/3055544316 shutdown_connections 2024-09-19T04:16:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- 172.21.15.76:0/3055544316 wait complete. 2024-09-19T04:16:03.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- start start 2024-09-19T04:16:03.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 Processor -- start 2024-09-19T04:16:03.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7888f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c7b40 0x5570ee788e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee789370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5570ee64e700 con 0x5570ee77e830 2024-09-19T04:16:03.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5570ee5665c0 con 0x5570ee7c8460 2024-09-19T04:16:03.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5570ee564020 con 0x5570ee7c7b40 2024-09-19T04:16:03.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9deff65000 10 client.0 _ll_register_callbacks cb 0x5570ee6805a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:03.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7888f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7888f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37034/0 (socket says 172.21.15.76:37034) 2024-09-19T04:16:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de659c640 1 -- 172.21.15.76:0/1704904265 learned_addr learned my addr 172.21.15.76:0/1704904265 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.904+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c7b40 0x5570ee788e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de659c640 1 -- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c7b40 msgr2=0x5570ee788e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de659c640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c7b40 0x5570ee788e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee789370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de659c640 1 -- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 msgr2=0x5570ee789370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de659c640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee789370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de659c640 1 -- 172.21.15.76:0/1704904265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ee572500 con 0x5570ee77e830 2024-09-19T04:16:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c7b40 0x5570ee788e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de659c640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7888f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9de00095c0 tx=0x7f9de001b0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bad59fab527d199 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 1 -- 172.21.15.76:0/1704904265 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00b2040 con 0x5570ee77e830 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 1 -- 172.21.15.76:0/1704904265 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9de00b8960 con 0x5570ee77e830 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9de00b87b0 con 0x5570ee77e830 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 1 -- 172.21.15.76:0/1704904265 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9de00da050 con 0x5570ee77e830 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 10 client.5844 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5570ee56c550 con 0x5570ee77e830 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 1 -- 172.21.15.76:0/1704904265 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de00b87b0 con 0x5570ee77e830 2024-09-19T04:16:03.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 1 -- 172.21.15.76:0/1704904265 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f9de00dd020 con 0x5570ee77e830 2024-09-19T04:16:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 10 client.5844 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 10 client.5844 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 20 client.5844 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7d6bd0 0x5570ee7d8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 4 client.5844 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570ee566d00 con 0x5570ee7d6bd0 2024-09-19T04:16:03.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7d6bd0 0x5570ee7d8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7d6bd0 0x5570ee7d8fc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41dc99fbec917f5a server_cookie=8c2ae10f7592584d in_seq=0 out_seq=0 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 10 client.5844 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 1 -- 172.21.15.76:0/1704904265 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570ee566d00 con 0x5570ee7d6bd0 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 10 client.5844 handle_command_reply: tid=0 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 client.5844 shutdown 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 msgr2=0x5570ee7888f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7888f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f9de00095c0 tx=0x7f9de001b0f0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 shutdown_connections 2024-09-19T04:16:03.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7d6bd0 0x5570ee7d8fc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c8460 0x5570ee789370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c7b40 0x5570ee788e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9dc57fa640 0 client.5844 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 --2- 172.21.15.76:0/1704904265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7888f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 >> 172.21.15.76:0/1704904265 conn(0x5570ee77a7e0 msgr2=0x5570ee77d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 shutdown_connections 2024-09-19T04:16:03.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 1 -- 172.21.15.76:0/1704904265 wait complete. 2024-09-19T04:16:03.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.908+0000 7f9deff65000 20 client.5844 trim_cache size 0 max 16384 2024-09-19T04:16:03.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 Processor -- start 2024-09-19T04:16:03.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- start start 2024-09-19T04:16:03.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee7dc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7dc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c8460 0x5570ee77cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8203b50 con 0x5570ee7c8460 2024-09-19T04:16:03.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8202d20 con 0x5570ee7c7b40 2024-09-19T04:16:03.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9de8203420 con 0x5570ee77e830 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c8460 0x5570ee77cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c8460 0x5570ee77cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37038/0 (socket says 172.21.15.76:37038) 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 -- 172.21.15.76:0/2481101184 learned_addr learned my addr 172.21.15.76:0/2481101184 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 -- 172.21.15.76:0/2481101184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 msgr2=0x5570ee7dc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee7dc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7dc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 -- 172.21.15.76:0/2481101184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 msgr2=0x5570ee7dc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7dc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 -- 172.21.15.76:0/2481101184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570ee64eb90 con 0x5570ee7c8460 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c8460 0x5570ee77cbd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8052950 tx=0x7f9dd8055e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5251b19f9986a72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9dc77fe640 1 -- 172.21.15.76:0/2481101184 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd8082070 con 0x5570ee7c8460 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9dc77fe640 1 -- 172.21.15.76:0/2481101184 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd805a8a0 con 0x5570ee7c8460 2024-09-19T04:16:03.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9dc77fe640 1 -- 172.21.15.76:0/2481101184 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dd806e070 con 0x5570ee7c8460 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- 172.21.15.76:0/2481101184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c8460 msgr2=0x5570ee77cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c8460 0x5570ee77cbd0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f9dd8052950 tx=0x7f9dd8055e50 comp rx=0 tx=0).stop 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- 172.21.15.76:0/2481101184 shutdown_connections 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee7c8460 0x5570ee77cbd0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7dc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 --2- 172.21.15.76:0/2481101184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee77e830 0x5570ee7dc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- 172.21.15.76:0/2481101184 >> 172.21.15.76:0/2481101184 conn(0x5570ee77a7e0 msgr2=0x5570ee782ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- 172.21.15.76:0/2481101184 shutdown_connections 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- 172.21.15.76:0/2481101184 wait complete. 2024-09-19T04:16:03.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 -- start start 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.928+0000 7f9deff65000 1 Processor -- start 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c8460 0x5570ee7f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5570ee5cbee0 con 0x5570ee77e830 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9de8151120 con 0x5570ee7c7b40 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5570ee597fa0 con 0x5570ee7c8460 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 10 client.0 _ll_register_callbacks cb 0x7f9de0009c90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7db940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37052/0 (socket says 172.21.15.76:37052) 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 -- 172.21.15.76:0/2626672882 learned_addr learned my addr 172.21.15.76:0/2626672882 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:03.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de6d9d640 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c8460 0x5570ee7f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 -- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c8460 msgr2=0x5570ee7f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c8460 0x5570ee7f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 -- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 msgr2=0x5570ee7f72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7f72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 -- 172.21.15.76:0/2626672882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570ee625ca0 con 0x5570ee77e830 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de659c640 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7db940 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9de001c8f0 tx=0x7f9de0019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6202de96ccbd07c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 1 -- 172.21.15.76:0/2626672882 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0003cb0 con 0x5570ee77e830 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 1 -- 172.21.15.76:0/2626672882 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f9de0004100 con 0x5570ee77e830 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dd805a8a0 con 0x5570ee77e830 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 1 -- 172.21.15.76:0/2626672882 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9de00ed050 con 0x5570ee77e830 2024-09-19T04:16:03.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 10 client.5850 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5570ee2874c0 con 0x5570ee77e830 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 1 -- 172.21.15.76:0/2626672882 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9dd805a8a0 con 0x5570ee77e830 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 1 -- 172.21.15.76:0/2626672882 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f9de00f0020 con 0x5570ee77e830 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 10 client.5850 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 10 client.5850 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 20 client.5850 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7f7d30 0x5570ee83a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 4 client.5850 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x5570ee5da300 con 0x5570ee7f7d30 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7f7d30 0x5570ee83a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:03.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 10 client.5850 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9de5d9b640 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7f7d30 0x5570ee83a9d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=616f4fb11ff6485a server_cookie=4e0fd87cac5da028 in_seq=0 out_seq=0 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 1 -- 172.21.15.76:0/2626672882 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5570ee5da300 con 0x5570ee7f7d30 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 10 client.5850 handle_command_reply: tid=0 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 client.5850 shutdown 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 msgr2=0x5570ee7db940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7db940 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f9de001c8f0 tx=0x7f9de0019650 comp rx=0 tx=0).stop 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 shutdown_connections 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5570ee7c8460 0x5570ee7f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x5570ee7f7d30 0x5570ee83a9d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9dc57fa640 0 client.5850 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5570ee7c7b40 0x5570ee7f72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 --2- 172.21.15.76:0/2626672882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5570ee77e830 0x5570ee7db940 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 >> 172.21.15.76:0/2626672882 conn(0x5570ee77a7e0 msgr2=0x5570ee783500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.945 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:16:03.932+0000 7fccdb429640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:15:48.270101+0000 1=0+1) n(v3 rc2024-09-19T04:15:48.488399+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5637=pAsLsXsFsx/AsLsXsFsx@10},l=5637 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5647964c4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.270101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.488399+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.270101+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:15:48.488399+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:16:03.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 shutdown_connections 2024-09-19T04:16:03.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 1 -- 172.21.15.76:0/2626672882 wait complete. 2024-09-19T04:16:03.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.932+0000 7f9deff65000 20 client.5850 trim_cache size 0 max 16384 2024-09-19T04:16:03.946 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:03.947 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-09-19T04:16:03.947 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "2735132f-48f6-4722-bdad-c4d08d9e3230", 2024-09-19T04:16:03.947 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-09-19T04:16:03.947 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:03.948 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f61506c0640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:15:48.273429+0000 1=0+1) n(v2 rc2024-09-19T04:15:48.488399+0000 4=0+4) (inest lock) caps={5637=pAsLsXsFs/-@7},l=5637 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55fc16beb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.273429+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.488399+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.273429+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:15:48.488399+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:16:03.949 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-09-19T04:16:03.935+0000 7f8194c1e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:15:48.276603+0000 1=0+1) n(v0 rc2024-09-19T04:15:48.276603+0000 2=0+2) (inest lock) (ifile excl) caps={5637=pAsLsXsFsx/-@6},l=5637 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x562b06aaa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.276603+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.276603+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.276603+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:15:48.276603+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:16:03.950 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-09-19T04:16:03.935+0000 7f8194c1e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={5637=pAsLsXsFs/-@3},l=5637 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562b06aaa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:16:03.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9dc80678c0 msgr2=0x7f9dc8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9dc80678c0 0x7f9dc8069d80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f9de00af000 tx=0x7f9de00a7070 comp rx=0 tx=0).stop 2024-09-19T04:16:03.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815c2a0 msgr2=0x7f9de80d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:03.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815c2a0 0x7f9de80d0150 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9dd800d800 tx=0x7f9dd800dcc0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 shutdown_connections 2024-09-19T04:16:03.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9dc80678c0 0x7f9dc8069d80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9de815c2a0 0x7f9de80d0150 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9de815bec0 0x7f9de80cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 --2- 172.21.15.76:0/1947816462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9de80af470 0x7f9de814ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:03.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 >> 172.21.15.76:0/1947816462 conn(0x7f9de80b1f90 msgr2=0x7f9de81597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:03.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 shutdown_connections 2024-09-19T04:16:03.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f9decea9640 1 -- 172.21.15.76:0/1947816462 wait complete. 2024-09-19T04:16:03.953 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:16:03.936+0000 7fcce1c36640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:15:48.271783+0000 1=0+1) n(v4 rc2024-09-19T04:15:48.488399+0000 5=0+5) (inest sync->mix g=1,2) caps={5637=pAsLsXsFs/-@11} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x564796546100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.271783+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.488399+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.271783+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:15:48.488399+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:16:03.953 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:16:03.936+0000 7f6156ecd640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:15:48.275018+0000 1=0+1) n(v1 rc2024-09-19T04:15:48.488399+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={5637=pAsLsXsFs/-@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55fc17c56000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.275018+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.276603+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.275018+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:15:48.488399+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:16:03.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:03.956 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub abort 2024-09-19T04:16:04.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7d490f640 1 Processor -- start 2024-09-19T04:16:04.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7d490f640 1 -- start start 2024-09-19T04:16:04.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7d490f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d00b3c40 0x7fd7d015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7d490f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd7d00b40d0 0x7fd7d0160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7d490f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7d00b4620 con 0x7fd7d01567e0 2024-09-19T04:16:04.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7d490f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd7d00b4900 con 0x7fd7d00b40d0 2024-09-19T04:16:04.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7d490f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d00b4be0 con 0x7fd7d00b3c40 2024-09-19T04:16:04.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7ce59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd7d01567e0 0x7fd7d00b56c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47308/0 (socket says 172.21.15.76:47308) 2024-09-19T04:16:04.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7ce59c640 1 -- 172.21.15.76:0/2338952386 learned_addr learned my addr 172.21.15.76:0/2338952386 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:04.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d00b3c40 0x7fd7d015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/2338952386 >> v2:172.21.15.116:3300/0 conn(0x7fd7d00b40d0 0x7fd7d0160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755269774 0 0) 0x7fd7d00b4620 con 0x7fd7d01567e0 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd7b8003530 con 0x7fd7d01567e0 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4037220487 0 0) 0x7fd7b8003530 con 0x7fd7d01567e0 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d00b3c40 msgr2=0x7fd7d015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d00b3c40 0x7fd7d015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 >> v2:172.21.15.116:3300/0 conn(0x7fd7d00b40d0 msgr2=0x7fd7d0160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 --2- 172.21.15.76:0/2338952386 >> v2:172.21.15.116:3300/0 conn(0x7fd7d00b40d0 0x7fd7d0160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d00b4ec0 con 0x7fd7d01567e0 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fd7c0001680 con 0x7fd7d01567e0 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 >> v1:172.21.15.76:6789/0 conn(0x7fd7d01567e0 legacy=0x7fd7d00b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b80039c0 0x7fd7b8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d00b3c40 0x7fd7b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b8004d50 0x7fd7b80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d00b4900 con 0x7fd7b80039c0 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d00b4be0 con 0x7fd7b8004d50 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b800ba80 con 0x7fd7d00b3c40 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.068+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b80039c0 0x7fd7b8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7ce59c640 1 -- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d00b3c40 msgr2=0x7fd7b8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d00b3c40 0x7fd7b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7ce59c640 1 -- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b8004d50 msgr2=0x7fd7b80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b8004d50 0x7fd7b80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7ce59c640 1 -- 172.21.15.76:0/2338952386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b8003530 con 0x7fd7b80039c0 2024-09-19T04:16:04.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b80039c0 0x7fd7b8003de0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c240 tx=0x7fd7c001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb102dda7a277171 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0007d60 con 0x7fd7b80039c0 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c00a9070 con 0x7fd7b80039c0 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7cd59a640 1 -- 172.21.15.76:0/2338952386 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c0005d20 con 0x7fd7b80039c0 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7d490f640 1 -- 172.21.15.76:0/2338952386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b80039c0 msgr2=0x7fd7b8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7d490f640 1 --2- 172.21.15.76:0/2338952386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7b80039c0 0x7fd7b8003de0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd7c000c240 tx=0x7fd7c001cc90 comp rx=0 tx=0).stop 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7ced9d640 1 -- 172.21.15.76:0/2338952386 reap_dead start 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7d490f640 1 -- 172.21.15.76:0/2338952386 shutdown_connections 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7d490f640 1 -- 172.21.15.76:0/2338952386 >> 172.21.15.76:0/2338952386 conn(0x7fd7d0073580 msgr2=0x7fd7d006bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7d490f640 1 -- 172.21.15.76:0/2338952386 shutdown_connections 2024-09-19T04:16:04.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.072+0000 7fd7d490f640 1 -- 172.21.15.76:0/2338952386 wait complete. 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 Processor -- start 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 -- start start 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b80039c0 0x7fd7d014cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d015b210 0x7fd7d015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7d014fdc0 0x7fd7d0150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d015e130 con 0x7fd7d014fdc0 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0160be0 con 0x7fd7b80039c0 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d015e2b0 con 0x7fd7d015b210 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7d014fdc0 0x7fd7d0150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7d014fdc0 0x7fd7d0150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37070/0 (socket says 172.21.15.76:37070) 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 -- 172.21.15.76:0/3626541089 learned_addr learned my addr 172.21.15.76:0/3626541089 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b80039c0 0x7fd7d014cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d015b210 0x7fd7d015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 -- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d015b210 msgr2=0x7fd7d015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d015b210 0x7fd7d015b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 -- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b80039c0 msgr2=0x7fd7d014cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b80039c0 0x7fd7d014cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 -- 172.21.15.76:0/3626541089 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d014e130 con 0x7fd7d014fdc0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7d014fdc0 0x7fd7d0150250 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00d0ac0 tx=0x7fd7c800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7beffd640 1 -- 172.21.15.76:0/3626541089 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800bae0 con 0x7fd7d014fdc0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7beffd640 1 -- 172.21.15.76:0/3626541089 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c801a070 con 0x7fd7d014fdc0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d014e3a0 con 0x7fd7d014fdc0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7beffd640 1 -- 172.21.15.76:0/3626541089 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8014400 con 0x7fd7d014fdc0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d014e960 con 0x7fd7d014fdc0 2024-09-19T04:16:04.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7beffd640 1 -- 172.21.15.76:0/3626541089 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd7c8013070 con 0x7fd7d014fdc0 2024-09-19T04:16:04.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7beffd640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7b00678f0 0x7fd7b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7beffd640 1 -- 172.21.15.76:0/3626541089 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fd7c8085db0 con 0x7fd7d014fdc0 2024-09-19T04:16:04.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.076+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7b00678f0 0x7fd7b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7b00678f0 0x7fd7b0069db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00ab000 tx=0x7fd7c00b1040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 Processor -- start 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- start start 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831da3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 0x55d831d57240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 0x55d831d57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d831ba8cc0 con 0x55d831da3c10 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d831b63a20 con 0x55d831da4530 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d831ba4c80 con 0x55d831d57780 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831da3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831da3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37082/0 (socket says 172.21.15.76:37082) 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 -- 172.21.15.76:0/1689390919 learned_addr learned my addr 172.21.15.76:0/1689390919 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 -- 172.21.15.76:0/1689390919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 msgr2=0x55d831d57b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/1689390919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 0x55d831d57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 -- 172.21.15.76:0/1689390919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 msgr2=0x55d831d57240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/1689390919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 0x55d831d57240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 -- 172.21.15.76:0/1689390919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d831b4bd40 con 0x55d831da3c10 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/1689390919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831da3ff0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd7c001ccd0 tx=0x7fd7d00b5f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d67937b80693403 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7affff640 1 -- 172.21.15.76:0/1689390919 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c00a5e50 con 0x55d831da3c10 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7affff640 1 -- 172.21.15.76:0/1689390919 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c00a9070 con 0x55d831da3c10 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7affff640 1 -- 172.21.15.76:0/1689390919 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c00049f0 con 0x55d831da3c10 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1689390919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 msgr2=0x55d831da3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1689390919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831da3ff0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd7c001ccd0 tx=0x7fd7d00b5f60 comp rx=0 tx=0).stop 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1689390919 shutdown_connections 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1689390919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 0x55d831d57b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1689390919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 0x55d831d57240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1689390919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831da3ff0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1689390919 >> 172.21.15.76:0/1689390919 conn(0x55d831d849b0 msgr2=0x55d831d86da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1689390919 shutdown_connections 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1689390919 wait complete. 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 -- start start 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.080+0000 7fd7d79cb000 1 Processor -- start 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 0x55d831d5dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831d5e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da4530 0x55d831d5ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d831b422c0 con 0x55d831da3c10 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d831b67370 con 0x55d831d57780 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d831bb6f50 con 0x55d831da4530 2024-09-19T04:16:04.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 10 client.0 _ll_register_callbacks cb 0x55d831c09db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831d5e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831d5e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37098/0 (socket says 172.21.15.76:37098) 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/328882058 learned_addr learned my addr 172.21.15.76:0/328882058 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da4530 0x55d831d5ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da4530 msgr2=0x55d831d5ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da4530 0x55d831d5ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 0x55d831d5dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 msgr2=0x55d831d5dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 0x55d831d5dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/328882058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d831b48770 con 0x55d831da3c10 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831d5e4f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009fb0 tx=0x7fd7c40116f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0206f09623630a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/328882058 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4019070 con 0x55d831da3c10 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c00a9070 con 0x55d831da3c10 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/328882058 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fd7c4015870 con 0x55d831da3c10 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/328882058 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd7c4024050 con 0x55d831da3c10 2024-09-19T04:16:04.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 10 client.5862 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:04.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d831b663f0 con 0x55d831da3c10 2024-09-19T04:16:04.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/328882058 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c00a9070 con 0x55d831da3c10 2024-09-19T04:16:04.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/328882058 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7fd7c4027020 con 0x55d831da3c10 2024-09-19T04:16:04.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 10 client.5862 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:04.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 10 client.5862 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:04.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 20 client.5862 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:04.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831db3cb0 0x55d831db60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 4 client.5862 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d831b42d90 con 0x55d831db3cb0 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831db3cb0 0x55d831db60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 10 client.5862 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831db3cb0 0x55d831db60a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b36cfb1b22022c0a server_cookie=eb9b9ab43b2897d7 in_seq=0 out_seq=0 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/328882058 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d831b42d90 con 0x55d831db3cb0 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 10 client.5862 handle_command_reply: tid=0 2024-09-19T04:16:04.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 client.5862 shutdown 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 msgr2=0x55d831d5e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831d5e4f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009fb0 tx=0x7fd7c40116f0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 shutdown_connections 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da4530 0x55d831d5ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831d5e4f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831db3cb0 0x55d831db60a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/328882058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 0x55d831d5dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7ad7fa640 0 client.5862 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:04.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 >> 172.21.15.76:0/328882058 conn(0x55d831d849b0 msgr2=0x55d831d56940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 shutdown_connections 2024-09-19T04:16:04.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 1 -- 172.21.15.76:0/328882058 wait complete. 2024-09-19T04:16:04.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.084+0000 7fd7d79cb000 20 client.5862 trim_cache size 0 max 16384 2024-09-19T04:16:04.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 Processor -- start 2024-09-19T04:16:04.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- start start 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 0x55d831e133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da3c10 0x55d831e13920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da4530 0x55d831e13e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0203950 con 0x55d831da4530 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d0204050 con 0x55d831d57780 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d0202a30 con 0x55d831da3c10 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da4530 0x55d831e13e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da4530 0x55d831e13e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37106/0 (socket says 172.21.15.76:37106) 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 -- 172.21.15.76:0/1237780417 learned_addr learned my addr 172.21.15.76:0/1237780417 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 -- 172.21.15.76:0/1237780417 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da3c10 msgr2=0x55d831e13920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/1237780417 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da3c10 0x55d831e13920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 -- 172.21.15.76:0/1237780417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 msgr2=0x55d831e133e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/1237780417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 0x55d831e133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 -- 172.21.15.76:0/1237780417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d831c2a320 con 0x55d831da4530 2024-09-19T04:16:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/1237780417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da4530 0x55d831e13e60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7c805e710 tx=0x7fd7c8056820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1154649ab1e07796 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7af7fe640 1 -- 172.21.15.76:0/1237780417 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c805ec40 con 0x55d831da4530 2024-09-19T04:16:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7af7fe640 1 -- 172.21.15.76:0/1237780417 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c806e020 con 0x55d831da4530 2024-09-19T04:16:04.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7af7fe640 1 -- 172.21.15.76:0/1237780417 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c805a0e0 con 0x55d831da4530 2024-09-19T04:16:04.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1237780417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da4530 msgr2=0x55d831e13e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1237780417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da4530 0x55d831e13e60 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fd7c805e710 tx=0x7fd7c8056820 comp rx=0 tx=0).stop 2024-09-19T04:16:04.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1237780417 shutdown_connections 2024-09-19T04:16:04.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1237780417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da4530 0x55d831e13e60 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1237780417 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831da3c10 0x55d831e13920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/1237780417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831d57780 0x55d831e133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1237780417 >> 172.21.15.76:0/1237780417 conn(0x55d831d849b0 msgr2=0x55d831d55a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1237780417 shutdown_connections 2024-09-19T04:16:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- 172.21.15.76:0/1237780417 wait complete. 2024-09-19T04:16:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 -- start start 2024-09-19T04:16:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.104+0000 7fd7d79cb000 1 Processor -- start 2024-09-19T04:16:04.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 0x55d831e16de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831dd2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 0x55d831dd28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d831b50120 con 0x55d831da3c10 2024-09-19T04:16:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d831c29e90 con 0x55d831da4530 2024-09-19T04:16:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d831b52b90 con 0x55d831d57780 2024-09-19T04:16:04.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 10 client.0 _ll_register_callbacks cb 0x7fd7c40094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831dd2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831dd2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37108/0 (socket says 172.21.15.76:37108) 2024-09-19T04:16:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/2085460886 learned_addr learned my addr 172.21.15.76:0/2085460886 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 msgr2=0x55d831e16de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 0x55d831e16de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 0x55d831e16de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ced9d640 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 0x55d831dd28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 msgr2=0x55d831dd28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 0x55d831dd28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 -- 172.21.15.76:0/2085460886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d831b3c5c0 con 0x55d831da3c10 2024-09-19T04:16:04.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7cdd9b640 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831dd2380 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4015d20 tx=0x7fd7c40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f435d7cf25031b78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/2085460886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c4019070 con 0x55d831da3c10 2024-09-19T04:16:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/2085460886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7fd7c4004030 con 0x55d831da3c10 2024-09-19T04:16:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7c806e020 con 0x55d831da3c10 2024-09-19T04:16:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/2085460886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd7c4035050 con 0x55d831da3c10 2024-09-19T04:16:04.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 10 client.5868 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d831b40330 con 0x55d831da3c10 2024-09-19T04:16:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/2085460886 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7c806e020 con 0x55d831da3c10 2024-09-19T04:16:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/2085460886 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7fd7c4038020 con 0x55d831da3c10 2024-09-19T04:16:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 10 client.5868 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 10 client.5868 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:04.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 20 client.5868 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831e17320 0x55d831dd2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 4 client.5868 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-19T04:16:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x55d831bb63f0 con 0x55d831e17320 2024-09-19T04:16:04.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831e17320 0x55d831dd2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ce59c640 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831e17320 0x55d831dd2e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ad5e5b576620d35 server_cookie=9f7c13e4980e5e55 in_seq=0 out_seq=0 2024-09-19T04:16:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 10 client.5868 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 1 -- 172.21.15.76:0/2085460886 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d831bb63f0 con 0x55d831e17320 2024-09-19T04:16:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 10 client.5868 handle_command_reply: tid=0 2024-09-19T04:16:04.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 client.5868 shutdown 2024-09-19T04:16:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 msgr2=0x55d831dd2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831dd2380 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4015d20 tx=0x7fd7c40046d0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 shutdown_connections 2024-09-19T04:16:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55d831e17320 0x55d831dd2e00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d831da4530 0x55d831dd28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d831da3c10 0x55d831dd2380 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7ad7fa640 0 client.5868 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:04.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 --2- 172.21.15.76:0/2085460886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d831d57780 0x55d831e16de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 >> 172.21.15.76:0/2085460886 conn(0x55d831d849b0 msgr2=0x55d831d56230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 shutdown_connections 2024-09-19T04:16:04.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 1 -- 172.21.15.76:0/2085460886 wait complete. 2024-09-19T04:16:04.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d79cb000 20 client.5868 trim_cache size 0 max 16384 2024-09-19T04:16:04.125 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:04.125 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:16:04.126 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:16:04.126 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:04.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7b00678f0 msgr2=0x7fd7b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7b00678f0 0x7fd7b0069db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd7c00ab000 tx=0x7fd7c00b1040 comp rx=0 tx=0).stop 2024-09-19T04:16:04.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7d014fdc0 msgr2=0x7fd7d0150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7d014fdc0 0x7fd7d0150250 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7d00d0ac0 tx=0x7fd7c800b480 comp rx=0 tx=0).stop 2024-09-19T04:16:04.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 shutdown_connections 2024-09-19T04:16:04.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7b00678f0 0x7fd7b0069db0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd7d014fdc0 0x7fd7d0150250 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd7d015b210 0x7fd7d015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 --2- 172.21.15.76:0/3626541089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7b80039c0 0x7fd7d014cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 >> 172.21.15.76:0/3626541089 conn(0x7fd7d0073580 msgr2=0x7fd7d00b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 shutdown_connections 2024-09-19T04:16:04.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.108+0000 7fd7d490f640 1 -- 172.21.15.76:0/3626541089 wait complete. 2024-09-19T04:16:04.131 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:16:04.131 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-19T04:16:04.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.232+0000 7f64503cb640 1 Processor -- start 2024-09-19T04:16:04.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.232+0000 7f64503cb640 1 -- start start 2024-09-19T04:16:04.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f64503cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 0x7f64480d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f64503cb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64480d32b0 con 0x7f64480d2e90 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f64503cb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64480cfad0 con 0x7f64480cf620 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f64503cb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f64480d0300 con 0x7f6448161a60 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d966640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f64480d2e90 0x7f64480cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47312/0 (socket says 172.21.15.76:47312) 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d966640 1 -- 172.21.15.76:0/87800794 learned_addr learned my addr 172.21.15.76:0/87800794 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e167640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 0x7f64480d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 550604720 0 0) 0x7f64480d32b0 con 0x7f64480d2e90 2024-09-19T04:16:04.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6434003530 con 0x7f64480d2e90 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 189405348 0 0) 0x7f6434003530 con 0x7f64480d2e90 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 msgr2=0x7f64480d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 0x7f64480d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 >> v1:172.21.15.116:6789/0 conn(0x7f64480cf620 legacy=0x7f644806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64480b0f60 con 0x7f64480d2e90 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6444001680 con 0x7f64480d2e90 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 >> v1:172.21.15.76:6789/0 conn(0x7f64480d2e90 legacy=0x7f64480cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 0x7f64340039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6434003f70 0x7f6434004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6434004d50 0x7f64340051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f64480d0300 con 0x7f6434004d50 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f643400bb60 con 0x7f6434003f70 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f643400bd60 con 0x7f6448161a60 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e968640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6434004d50 0x7f64340051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e167640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 0x7f64340039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d966640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6434003f70 0x7f6434004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e968640 1 -- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 msgr2=0x7f64340039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e968640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 0x7f64340039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e968640 1 -- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6434003f70 msgr2=0x7f6434004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e968640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6434003f70 0x7f6434004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e968640 1 -- 172.21.15.76:0/87800794 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6434003530 con 0x7f6434004d50 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e167640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6448161a60 0x7f64340039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:04.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d966640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6434003f70 0x7f6434004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644e968640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6434004d50 0x7f64340051d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6440001e70 tx=0x7f644001c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f23e598dee8d01 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440001a90 con 0x7f6434004d50 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6440003b80 con 0x7f6434004d50 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.236+0000 7f644d165640 1 -- 172.21.15.76:0/87800794 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440005080 con 0x7f6434004d50 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- 172.21.15.76:0/87800794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6434004d50 msgr2=0x7f64340051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 --2- 172.21.15.76:0/87800794 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6434004d50 0x7f64340051d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6440001e70 tx=0x7f644001c780 comp rx=0 tx=0).stop 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644e968640 1 -- 172.21.15.76:0/87800794 reap_dead start 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- 172.21.15.76:0/87800794 shutdown_connections 2024-09-19T04:16:04.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- 172.21.15.76:0/87800794 >> 172.21.15.76:0/87800794 conn(0x7f644808bf90 msgr2=0x7f644809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- 172.21.15.76:0/87800794 shutdown_connections 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- 172.21.15.76:0/87800794 wait complete. 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 Processor -- start 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- start start 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6434004d50 0x7f644814b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64480cb440 0x7f64480cb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64480c8ed0 0x7f64480c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f644806ceb0 con 0x7f64480cb440 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64480d0020 con 0x7f64480c8ed0 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f644806d190 con 0x7f6434004d50 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64480cb440 0x7f64480cb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64480cb440 0x7f64480cb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37128/0 (socket says 172.21.15.76:37128) 2024-09-19T04:16:04.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 -- 172.21.15.76:0/288412627 learned_addr learned my addr 172.21.15.76:0/288412627 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644e167640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6434004d50 0x7f644814b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 -- 172.21.15.76:0/288412627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6434004d50 msgr2=0x7f644814b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6434004d50 0x7f644814b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 -- 172.21.15.76:0/288412627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64480c8ed0 msgr2=0x7f64480c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64480c8ed0 0x7f64480c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 -- 172.21.15.76:0/288412627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64480c9c20 con 0x7f64480cb440 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f644d966640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64480cb440 0x7f64480cb8c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6444010c80 tx=0x7f644400e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f643effd640 1 -- 172.21.15.76:0/288412627 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644401b070 con 0x7f64480cb440 2024-09-19T04:16:04.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f643effd640 1 -- 172.21.15.76:0/288412627 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6444011040 con 0x7f64480cb440 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64480c9f00 con 0x7f64480cb440 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f643effd640 1 -- 172.21.15.76:0/288412627 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64440145a0 con 0x7f64480cb440 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.240+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64480ca4c0 con 0x7f64480cb440 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f643effd640 1 -- 172.21.15.76:0/288412627 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f644401e070 con 0x7f64480cb440 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f643effd640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f642c0678c0 0x7f642c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f643effd640 1 -- 172.21.15.76:0/288412627 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f6444088a40 con 0x7f64480cb440 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f642c0678c0 0x7f642c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f642c0678c0 0x7f642c069d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64380099e0 tx=0x7f6438008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 Processor -- start 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- start start 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590b5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 0x556759068870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 0x556759069190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556758f52960 con 0x5567590b4d70 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556758e96900 con 0x556759068db0 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556758eb9d40 con 0x5567590b5690 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590b5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590b5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37144/0 (socket says 172.21.15.76:37144) 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 -- 172.21.15.76:0/3743517181 learned_addr learned my addr 172.21.15.76:0/3743517181 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 -- 172.21.15.76:0/3743517181 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 msgr2=0x556759068870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 --2- 172.21.15.76:0/3743517181 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 0x556759068870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 -- 172.21.15.76:0/3743517181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 msgr2=0x556759069190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 --2- 172.21.15.76:0/3743517181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 0x556759069190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 -- 172.21.15.76:0/3743517181 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556758e60580 con 0x5567590b4d70 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f644e167640 1 --2- 172.21.15.76:0/3743517181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590b5150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6438006a60 tx=0x7f6438014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8978674259de9d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f642bfff640 1 -- 172.21.15.76:0/3743517181 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438011d00 con 0x5567590b4d70 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f642bfff640 1 -- 172.21.15.76:0/3743517181 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6438026070 con 0x5567590b4d70 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f642bfff640 1 -- 172.21.15.76:0/3743517181 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438011ea0 con 0x5567590b4d70 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- 172.21.15.76:0/3743517181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 msgr2=0x5567590b5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 --2- 172.21.15.76:0/3743517181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590b5150 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6438006a60 tx=0x7f6438014000 comp rx=0 tx=0).stop 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- 172.21.15.76:0/3743517181 shutdown_connections 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 --2- 172.21.15.76:0/3743517181 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 0x556759069190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 --2- 172.21.15.76:0/3743517181 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 0x556759068870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 --2- 172.21.15.76:0/3743517181 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590b5150 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- 172.21.15.76:0/3743517181 >> 172.21.15.76:0/3743517181 conn(0x556759096040 msgr2=0x556759098430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- 172.21.15.76:0/3743517181 shutdown_connections 2024-09-19T04:16:04.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.244+0000 7f6453487000 1 -- 172.21.15.76:0/3743517181 wait complete. 2024-09-19T04:16:04.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- start start 2024-09-19T04:16:04.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 Processor -- start 2024-09-19T04:16:04.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 0x5567590bbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b4d70 0x5567590bc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b5690 0x5567590bc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x556758eb5d00 con 0x5567590b5690 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556758e74aa0 con 0x5567590b4d70 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556758e53340 con 0x556759068db0 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 10 client.0 _ll_register_callbacks cb 0x556759056760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b5690 0x5567590bc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b5690 0x5567590bc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37152/0 (socket says 172.21.15.76:37152) 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 -- 172.21.15.76:0/865243216 learned_addr learned my addr 172.21.15.76:0/865243216 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 -- 172.21.15.76:0/865243216 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 msgr2=0x5567590bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 0x5567590bbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 -- 172.21.15.76:0/865243216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b4d70 msgr2=0x5567590bc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b4d70 0x5567590bc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 -- 172.21.15.76:0/865243216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556758e5cdc0 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e968640 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b5690 0x5567590bc760 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f644001cc60 tx=0x7f6440003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bc815d5cb2019e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 1 -- 172.21.15.76:0/865243216 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64400bb060 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 1 -- 172.21.15.76:0/865243216 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f6440006b60 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6438026070 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 1 -- 172.21.15.76:0/865243216 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f64400c1050 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 10 client.5880 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x556758e597f0 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 1 -- 172.21.15.76:0/865243216 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6438026070 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 1 -- 172.21.15.76:0/865243216 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f64400c4020 con 0x5567590b5690 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 10 client.5880 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 10 client.5880 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:04.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 20 client.5880 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590c40b0 0x5567590c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 4 client.5880 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556758e53e10 con 0x5567590c40b0 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e167640 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590c40b0 0x5567590c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f644e167640 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590c40b0 0x5567590c64a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b4d90f3559a479c server_cookie=c249a63e595f1fb3 in_seq=0 out_seq=0 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 10 client.5880 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 1 -- 172.21.15.76:0/865243216 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556758e53e10 con 0x5567590c40b0 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 10 client.5880 handle_command_reply: tid=0 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 client.5880 shutdown 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b5690 msgr2=0x5567590bc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b5690 0x5567590bc760 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f644001cc60 tx=0x7f6440003d10 comp rx=0 tx=0).stop 2024-09-19T04:16:04.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 shutdown_connections 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590c40b0 0x5567590c64a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b5690 0x5567590bc760 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b4d70 0x5567590bc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f64297fa640 0 client.5880 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 --2- 172.21.15.76:0/865243216 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 0x5567590bbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 >> 172.21.15.76:0/865243216 conn(0x556759096040 msgr2=0x556759067f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 shutdown_connections 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 1 -- 172.21.15.76:0/865243216 wait complete. 2024-09-19T04:16:04.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.248+0000 7f6453487000 20 client.5880 trim_cache size 0 max 16384 2024-09-19T04:16:04.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 Processor -- start 2024-09-19T04:16:04.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- start start 2024-09-19T04:16:04.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 0x5567590c2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x556759070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 0x556759071130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6448201710 con 0x5567590b4d70 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64482008e0 con 0x556759068db0 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6448200fe0 con 0x5567590b5690 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x556759070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x556759070bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37166/0 (socket says 172.21.15.76:37166) 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 -- 172.21.15.76:0/911117664 learned_addr learned my addr 172.21.15.76:0/911117664 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 -- 172.21.15.76:0/911117664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 msgr2=0x556759071130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 --2- 172.21.15.76:0/911117664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 0x556759071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 -- 172.21.15.76:0/911117664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 msgr2=0x5567590c2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 --2- 172.21.15.76:0/911117664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 0x5567590c2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 -- 172.21.15.76:0/911117664 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556758e77470 con 0x5567590b4d70 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 --2- 172.21.15.76:0/911117664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x556759070bf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f64440613a0 tx=0x7f6444058650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3298c1e14b5f26e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f642b7fe640 1 -- 172.21.15.76:0/911117664 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64440593f0 con 0x5567590b4d70 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f642b7fe640 1 -- 172.21.15.76:0/911117664 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644406e070 con 0x5567590b4d70 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f642b7fe640 1 -- 172.21.15.76:0/911117664 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444058cf0 con 0x5567590b4d70 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- 172.21.15.76:0/911117664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 msgr2=0x556759070bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- 172.21.15.76:0/911117664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x556759070bf0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f64440613a0 tx=0x7f6444058650 comp rx=0 tx=0).stop 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- 172.21.15.76:0/911117664 shutdown_connections 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- 172.21.15.76:0/911117664 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5567590b5690 0x556759071130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- 172.21.15.76:0/911117664 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x556759070bf0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- 172.21.15.76:0/911117664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556759068db0 0x5567590c2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- 172.21.15.76:0/911117664 >> 172.21.15.76:0/911117664 conn(0x556759096040 msgr2=0x5567590bbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- 172.21.15.76:0/911117664 shutdown_connections 2024-09-19T04:16:04.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- 172.21.15.76:0/911117664 wait complete. 2024-09-19T04:16:04.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- start start 2024-09-19T04:16:04.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 Processor -- start 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 0x5567590c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590e4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b5690 0x5567590e4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f64481ffa40 con 0x5567590b4d70 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x556758e5ff20 con 0x5567590b5690 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x556758f760c0 con 0x556759068db0 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f6453487000 10 client.0 _ll_register_callbacks cb 0x7f6434003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590e4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590e4790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37178/0 (socket says 172.21.15.76:37178) 2024-09-19T04:16:04.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.268+0000 7f644d966640 1 -- 172.21.15.76:0/4164615845 learned_addr learned my addr 172.21.15.76:0/4164615845 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644e167640 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 0x5567590c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644d966640 1 -- 172.21.15.76:0/4164615845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 msgr2=0x5567590c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644d966640 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 0x5567590c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644d966640 1 -- 172.21.15.76:0/4164615845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b5690 msgr2=0x5567590e4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644d966640 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b5690 0x5567590e4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644d966640 1 -- 172.21.15.76:0/4164615845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556758e4d640 con 0x5567590b4d70 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644d966640 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590e4790 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f644400e9d0 tx=0x7f6444001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3005bb50d0834cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 1 -- 172.21.15.76:0/4164615845 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444077340 con 0x5567590b4d70 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 1 -- 172.21.15.76:0/4164615845 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f6444061640 con 0x5567590b4d70 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f644406e070 con 0x5567590b4d70 2024-09-19T04:16:04.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 1 -- 172.21.15.76:0/4164615845 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f644407f050 con 0x5567590b4d70 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 10 client.5886 fetch_fsmap learned FSMap version 36 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x556758e513b0 con 0x5567590b4d70 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 1 -- 172.21.15.76:0/4164615845 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644406e070 con 0x5567590b4d70 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 1 -- 172.21.15.76:0/4164615845 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 36) ==== 3554+0+0 (secure 0 0 0) 0x7f6444080240 con 0x5567590b4d70 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 10 client.5886 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 10 client.5886 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 20 client.5886 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590e5210 0x556759128ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 4 client.5886 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556758ec7470 con 0x5567590e5210 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644e167640 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590e5210 0x556759128ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f644e167640 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590e5210 0x556759128ba0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f2a44a4a358a851 server_cookie=b8d5fb20ed5f4553 in_seq=0 out_seq=0 2024-09-19T04:16:04.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 10 client.5886 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 1 -- 172.21.15.76:0/4164615845 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x556758ec7470 con 0x5567590e5210 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 10 client.5886 handle_command_reply: tid=0 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 client.5886 shutdown 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 msgr2=0x5567590e4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590e4790 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f644400e9d0 tx=0x7f6444001a50 comp rx=0 tx=0).stop 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 shutdown_connections 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x5567590e5210 0x556759128ba0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5567590b5690 0x5567590e4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5567590b4d70 0x5567590e4790 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64297fa640 0 client.5886 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 --2- 172.21.15.76:0/4164615845 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556759068db0 0x5567590c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 >> 172.21.15.76:0/4164615845 conn(0x556759096040 msgr2=0x556759070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 shutdown_connections 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 1 -- 172.21.15.76:0/4164615845 wait complete. 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f6453487000 20 client.5886 trim_cache size 0 max 16384 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stdout: "2735132f-48f6-4722-bdad-c4d08d9e3230": { 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:16:04.283 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2735132f-48f6-4722-bdad-c4d08d9e3230", 2024-09-19T04:16:04.284 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:16:04.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:04.284 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:04.284 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:04.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f642c0678c0 msgr2=0x7f642c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f642c0678c0 0x7f642c069d80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f64380099e0 tx=0x7f6438008040 comp rx=0 tx=0).stop 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64480cb440 msgr2=0x7f64480cb8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64480cb440 0x7f64480cb8c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6444010c80 tx=0x7f644400e4d0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 shutdown_connections 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f642c0678c0 0x7f642c069d80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64480c8ed0 0x7f64480c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64480cb440 0x7f64480cb8c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 --2- 172.21.15.76:0/288412627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6434004d50 0x7f644814b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 >> 172.21.15.76:0/288412627 conn(0x7f644808bf90 msgr2=0x7f6448160ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 shutdown_connections 2024-09-19T04:16:04.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:04.272+0000 7f64503cb640 1 -- 172.21.15.76:0/288412627 wait complete. 2024-09-19T04:16:05.932 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:16:05.928+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:16:09.292 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:16:09.292 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-19T04:16:09.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb61593640 1 Processor -- start 2024-09-19T04:16:09.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb61593640 1 -- start start 2024-09-19T04:16:09.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb61593640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb5c070020 0x7fdb5c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb61593640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c15f1d0 0x7fdb5c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb61593640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb5c0aede0 con 0x7fdb5c1520a0 2024-09-19T04:16:09.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb61593640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdb5c152480 con 0x7fdb5c070020 2024-09-19T04:16:09.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb61593640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c15fa10 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb527fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdb5c070020 0x7fdb5c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb5affd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdb5c1520a0 0x7fdb5c06f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47320/0 (socket says 172.21.15.76:47320) 2024-09-19T04:16:09.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.424+0000 7fdb5affd640 1 -- 172.21.15.76:0/3255808076 learned_addr learned my addr 172.21.15.76:0/3255808076 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:09.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb527fc640 1 -- 172.21.15.76:0/3255808076 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c15f1d0 msgr2=0x7fdb5c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb527fc640 1 --2- 172.21.15.76:0/3255808076 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c15f1d0 0x7fdb5c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb5a7fc640 1 -- 172.21.15.76:0/3255808076 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275838427 0 0) 0x7fdb5c0aede0 con 0x7fdb5c1520a0 2024-09-19T04:16:09.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb527fc640 1 -- 172.21.15.76:0/3255808076 >> v1:172.21.15.76:6789/0 conn(0x7fdb5c1520a0 legacy=0x7fdb5c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb527fc640 1 -- 172.21.15.76:0/3255808076 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c074250 con 0x7fdb5c070020 2024-09-19T04:16:09.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb527fc640 1 --2- 172.21.15.76:0/3255808076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c070020 0x7fdb5c15ec90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097c0 tx=0x7fdb4401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20243246d0314025 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:09.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb5a7fc640 1 -- 172.21.15.76:0/3255808076 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a7070 con 0x7fdb5c070020 2024-09-19T04:16:09.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb5a7fc640 1 -- 172.21.15.76:0/3255808076 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb440b0070 con 0x7fdb5c070020 2024-09-19T04:16:09.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb5a7fc640 1 -- 172.21.15.76:0/3255808076 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440ab890 con 0x7fdb5c070020 2024-09-19T04:16:09.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 -- 172.21.15.76:0/3255808076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c070020 msgr2=0x7fdb5c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 --2- 172.21.15.76:0/3255808076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c070020 0x7fdb5c15ec90 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fdb440097c0 tx=0x7fdb4401c470 comp rx=0 tx=0).stop 2024-09-19T04:16:09.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 -- 172.21.15.76:0/3255808076 shutdown_connections 2024-09-19T04:16:09.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 --2- 172.21.15.76:0/3255808076 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c15f1d0 0x7fdb5c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 --2- 172.21.15.76:0/3255808076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c070020 0x7fdb5c15ec90 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 -- 172.21.15.76:0/3255808076 >> 172.21.15.76:0/3255808076 conn(0x7fdb5c0b1f90 msgr2=0x7fdb5c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:09.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 -- 172.21.15.76:0/3255808076 shutdown_connections 2024-09-19T04:16:09.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.428+0000 7fdb61593640 1 -- 172.21.15.76:0/3255808076 wait complete. 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 Processor -- start 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 -- start start 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb5c070020 0x7fdb5c146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c1520a0 0x7fdb5c1477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c15f1d0 0x7fdb5c15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c15f660 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c073d30 con 0x7fdb5c070020 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c14f5c0 con 0x7fdb5c1520a0 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c15f1d0 0x7fdb5c15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c15f1d0 0x7fdb5c15d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37196/0 (socket says 172.21.15.76:37196) 2024-09-19T04:16:09.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1535704234 learned_addr learned my addr 172.21.15.76:0/1535704234 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5affd640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb5c070020 0x7fdb5c146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb527fc640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c1520a0 0x7fdb5c1477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c1520a0 msgr2=0x7fdb5c1477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c1520a0 0x7fdb5c1477d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb5c070020 msgr2=0x7fdb5c146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb5c070020 0x7fdb5c146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1535704234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c146980 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c15f1d0 0x7fdb5c15d700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b970 tx=0x7fdb5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:09.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb53fff640 1 -- 172.21.15.76:0/1535704234 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54019070 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb53fff640 1 -- 172.21.15.76:0/1535704234 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb5400cb60 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb53fff640 1 -- 172.21.15.76:0/1535704234 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54014440 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c14a3f0 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.432+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c146550 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb53fff640 1 -- 172.21.15.76:0/1535704234 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdb5401c070 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb53fff640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdb3c0678f0 0x7fdb3c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5affd640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdb3c0678f0 0x7fdb3c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb53fff640 1 -- 172.21.15.76:0/1535704234 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7fdb54085ba0 con 0x7fdb5c15f1d0 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5affd640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdb3c0678f0 0x7fdb3c069db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000f70 tx=0x7fdb4c007b60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 Processor -- start 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 -- start start 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac61b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61b690 0x55f5ac5ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac5cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f5ac4b8960 con 0x55f5ac5cedb0 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f5ac3fc900 con 0x55f5ac61b690 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f5ac41fd40 con 0x55f5ac61ad70 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac5cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac5cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37204/0 (socket says 172.21.15.76:37204) 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1314792681 learned_addr learned my addr 172.21.15.76:0/1314792681 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5affd640 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac61b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb527fc640 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61b690 0x55f5ac5ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 msgr2=0x55f5ac61b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac61b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61b690 msgr2=0x55f5ac5ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61b690 0x55f5ac5ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/1314792681 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ac3c6580 con 0x55f5ac5cedb0 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb527fc640 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61b690 0x55f5ac5ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac5cf190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdb5405e4c0 tx=0x7fdb54055a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e070f22eac97854c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb50ff9640 1 -- 172.21.15.76:0/1314792681 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5405e9f0 con 0x55f5ac5cedb0 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb50ff9640 1 -- 172.21.15.76:0/1314792681 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb54053eb0 con 0x55f5ac5cedb0 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.436+0000 7fdb50ff9640 1 -- 172.21.15.76:0/1314792681 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54052480 con 0x55f5ac5cedb0 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/1314792681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 msgr2=0x55f5ac5cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac5cf190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fdb5405e4c0 tx=0x7fdb54055a80 comp rx=0 tx=0).stop 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/1314792681 shutdown_connections 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac5cf190 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61b690 0x55f5ac5ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- 172.21.15.76:0/1314792681 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac61b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/1314792681 >> 172.21.15.76:0/1314792681 conn(0x55f5ac5fc040 msgr2=0x55f5ac5fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/1314792681 shutdown_connections 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/1314792681 wait complete. 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- start start 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 Processor -- start 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 0x55f5ac621ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac622220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac622760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f5ac41bd00 con 0x55f5ac61b690 2024-09-19T04:16:09.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f5ac3daaa0 con 0x55f5ac5cedb0 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f5ac3b9340 con 0x55f5ac61ad70 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 10 client.0 _ll_register_callbacks cb 0x55f5ac5bc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac622760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac622760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37212/0 (socket says 172.21.15.76:37212) 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2222684230 learned_addr learned my addr 172.21.15.76:0/2222684230 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5affd640 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 0x55f5ac621ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb527fc640 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac622220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 msgr2=0x55f5ac622220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac622220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 msgr2=0x55f5ac621ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 0x55f5ac621ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2222684230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ac3c2dc0 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac622760 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b940 tx=0x7fdb54004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2175a1e9dddbef9d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb39ffb640 1 -- 172.21.15.76:0/2222684230 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54079060 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb39ffb640 1 -- 172.21.15.76:0/2222684230 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7fdb540743d0 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb54053eb0 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb39ffb640 1 -- 172.21.15.76:0/2222684230 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdb540541d0 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 10 client.5901 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55f5ac3bf7f0 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb39ffb640 1 -- 172.21.15.76:0/2222684230 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb54053eb0 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb39ffb640 1 -- 172.21.15.76:0/2222684230 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7fdb54055260 con 0x55f5ac61b690 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 10 client.5901 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:09.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 10 client.5901 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 20 client.5901 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac62a0b0 0x55f5ac62c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 4 client.5901 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5ac3b9e10 con 0x55f5ac62a0b0 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5affd640 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac62a0b0 0x55f5ac62c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb39ffb640 10 client.5901 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.440+0000 7fdb5affd640 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac62a0b0 0x55f5ac62c4a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=afadc1b4b25c1d11 server_cookie=653b763cb0b2d312 in_seq=0 out_seq=0 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb39ffb640 1 -- 172.21.15.76:0/2222684230 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5ac3b9e10 con 0x55f5ac62a0b0 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb39ffb640 10 client.5901 handle_command_reply: tid=0 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 client.5901 shutdown 2024-09-19T04:16:09.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 msgr2=0x55f5ac622760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac622760 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b940 tx=0x7fdb54004a50 comp rx=0 tx=0).stop 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 shutdown_connections 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac62a0b0 0x55f5ac62c4a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac622760 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac622220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2222684230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 0x55f5ac621ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb39ffb640 0 client.5901 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 >> 172.21.15.76:0/2222684230 conn(0x55f5ac5fc040 msgr2=0x55f5ac5cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 shutdown_connections 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 1 -- 172.21.15.76:0/2222684230 wait complete. 2024-09-19T04:16:09.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.444+0000 7fdb6464f000 20 client.5901 trim_cache size 0 max 16384 2024-09-19T04:16:09.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 Processor -- start 2024-09-19T04:16:09.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 -- start start 2024-09-19T04:16:09.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac5d6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac5d7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c201f80 con 0x55f5ac61b690 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c201120 con 0x55f5ac5cedb0 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdb5c201850 con 0x55f5ac61ad70 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac5d7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac5d7140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37222/0 (socket says 172.21.15.76:37222) 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2605622240 learned_addr learned my addr 172.21.15.76:0/2605622240 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2605622240 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 msgr2=0x55f5ac5d6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/2605622240 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac5d6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2605622240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 msgr2=0x55f5ac62fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/2605622240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 -- 172.21.15.76:0/2605622240 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5ac3dd470 con 0x55f5ac61b690 2024-09-19T04:16:09.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.460+0000 7fdb5b7fe640 1 --2- 172.21.15.76:0/2605622240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac5d7140 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdb5405e4c0 tx=0x7fdb54071000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89ab64d28057784 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb3bfff640 1 -- 172.21.15.76:0/2605622240 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540743e0 con 0x55f5ac61b690 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb3bfff640 1 -- 172.21.15.76:0/2605622240 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb54053eb0 con 0x55f5ac61b690 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb3bfff640 1 -- 172.21.15.76:0/2605622240 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5405c420 con 0x55f5ac61b690 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/2605622240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 msgr2=0x55f5ac5d7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2605622240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac5d7140 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fdb5405e4c0 tx=0x7fdb54071000 comp rx=0 tx=0).stop 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/2605622240 shutdown_connections 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2605622240 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac61b690 0x55f5ac5d7140 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2605622240 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61ad70 0x55f5ac5d6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- 172.21.15.76:0/2605622240 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/2605622240 >> 172.21.15.76:0/2605622240 conn(0x55f5ac5fc040 msgr2=0x55f5ac627ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:09.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/2605622240 shutdown_connections 2024-09-19T04:16:09.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/2605622240 wait complete. 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- start start 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 Processor -- start 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61ad70 0x55f5ac62f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61b690 0x55f5ac693120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c14b390 con 0x55f5ac5cedb0 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f5ac45f780 con 0x55f5ac61ad70 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f5ac4dc0c0 con 0x55f5ac61b690 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 10 client.0 _ll_register_callbacks cb 0x7fdb54055800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:09.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:37228/0 (socket says 172.21.15.76:37228) 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 -- 172.21.15.76:0/324963475 learned_addr learned my addr 172.21.15.76:0/324963475 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb527fc640 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61ad70 0x55f5ac62f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 -- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61b690 msgr2=0x55f5ac693120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61b690 0x55f5ac693120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 -- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61ad70 msgr2=0x55f5ac62f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61ad70 0x55f5ac62f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 -- 172.21.15.76:0/324963475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5ac3b3640 con 0x55f5ac5cedb0 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb527fc640 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61ad70 0x55f5ac62f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:09.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb5affd640 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62f150 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0072e0 tx=0x7fdb4c001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9580dfeba9d8a178 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb39ffb640 1 -- 172.21.15.76:0/324963475 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c031060 con 0x55f5ac5cedb0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb39ffb640 1 -- 172.21.15.76:0/324963475 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7fdb4c010040 con 0x55f5ac5cedb0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb54053eb0 con 0x55f5ac5cedb0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb39ffb640 1 -- 172.21.15.76:0/324963475 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c037050 con 0x55f5ac5cedb0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 10 client.5907 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55f5ac3b73b0 con 0x55f5ac5cedb0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb39ffb640 1 -- 172.21.15.76:0/324963475 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdb54053eb0 con 0x55f5ac5cedb0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb39ffb640 1 -- 172.21.15.76:0/324963475 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7fdb4c03a020 con 0x55f5ac5cedb0 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 10 client.5907 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 10 client.5907 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:09.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 20 client.5907 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac693fd0 0x55f5ac69f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 4 client.5907 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f5ac42d470 con 0x55f5ac693fd0 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.464+0000 7fdb527fc640 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac693fd0 0x55f5ac69f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb39ffb640 10 client.5907 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb527fc640 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac693fd0 0x55f5ac69f880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2121db73e7162f2b server_cookie=b6606bc80f094813 in_seq=0 out_seq=0 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb39ffb640 1 -- 172.21.15.76:0/324963475 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f5ac42d470 con 0x55f5ac693fd0 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb39ffb640 10 client.5907 handle_command_reply: tid=0 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 client.5907 shutdown 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 msgr2=0x55f5ac62f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62f150 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0072e0 tx=0x7fdb4c001090 comp rx=0 tx=0).stop 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 shutdown_connections 2024-09-19T04:16:09.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55f5ac693fd0 0x55f5ac69f880 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f5ac61b690 0x55f5ac693120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f5ac61ad70 0x55f5ac62f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb39ffb640 0 client.5907 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 --2- 172.21.15.76:0/324963475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f5ac5cedb0 0x55f5ac62f150 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 >> 172.21.15.76:0/324963475 conn(0x55f5ac5fc040 msgr2=0x55f5ac6282f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 shutdown_connections 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 1 -- 172.21.15.76:0/324963475 wait complete. 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb6464f000 20 client.5907 trim_cache size 0 max 16384 2024-09-19T04:16:09.477 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:16:09.478 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:16:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "2735132f-48f6-4722-bdad-c4d08d9e3230": { 2024-09-19T04:16:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:16:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2735132f-48f6-4722-bdad-c4d08d9e3230", 2024-09-19T04:16:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:16:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:09.479 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:09.479 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:09.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdb3c0678f0 msgr2=0x7fdb3c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdb3c0678f0 0x7fdb3c069db0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c000f70 tx=0x7fdb4c007b60 comp rx=0 tx=0).stop 2024-09-19T04:16:09.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c15f1d0 msgr2=0x7fdb5c15d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:09.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c15f1d0 0x7fdb5c15d700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400b970 tx=0x7fdb5400be30 comp rx=0 tx=0).stop 2024-09-19T04:16:09.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 shutdown_connections 2024-09-19T04:16:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdb3c0678f0 0x7fdb3c069db0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdb5c15f1d0 0x7fdb5c15d700 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdb5c1520a0 0x7fdb5c1477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 --2- 172.21.15.76:0/1535704234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb5c070020 0x7fdb5c146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 >> 172.21.15.76:0/1535704234 conn(0x7fdb5c0b1f90 msgr2=0x7fdb5c15b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 shutdown_connections 2024-09-19T04:16:09.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:09.468+0000 7fdb61593640 1 -- 172.21.15.76:0/1535704234 wait complete. 2024-09-19T04:16:14.488 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:16:14.489 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-19T04:16:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6adbf640 1 Processor -- start 2024-09-19T04:16:14.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6adbf640 1 -- start start 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6adbf640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4d640cf620 0x7f4d64168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6adbf640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d640d0300 con 0x7f4d64161a60 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6adbf640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d6406e6c0 con 0x7f4d640d3060 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6adbf640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4d640cfa90 con 0x7f4d640cf620 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4d64161a60 0x7f4d640d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60844/0 (socket says 172.21.15.76:60844) 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 -- 172.21.15.76:0/1351216722 learned_addr learned my addr 172.21.15.76:0/1351216722 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4278824410 0 0) 0x7f4d54001120 con 0x7f4d64161a60 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d4c003530 con 0x7f4d64161a60 2024-09-19T04:16:14.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6935c640 1 --2- 172.21.15.76:0/1351216722 >> v2:172.21.15.116:3300/0 conn(0x7f4d640cf620 0x7f4d64168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3529131806 0 0) 0x7f4d4c003530 con 0x7f4d64161a60 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 >> v2:172.21.15.116:3300/0 conn(0x7f4d640cf620 msgr2=0x7f4d64168f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 --2- 172.21.15.76:0/1351216722 >> v2:172.21.15.116:3300/0 conn(0x7f4d640cf620 0x7f4d64168f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 >> v1:172.21.15.116:6789/0 conn(0x7f4d640d3060 legacy=0x7f4d640cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d640b0f60 con 0x7f4d64161a60 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f4d54001850 con 0x7f4d64161a60 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 >> v1:172.21.15.76:6789/0 conn(0x7f4d64161a60 legacy=0x7f4d640d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0039c0 0x7f4d4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d4c004810 0x7f4d4c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0051d0 0x7f4d4c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d640cfa90 con 0x7f4d4c0039c0 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4c00c170 con 0x7f4d4c0051d0 2024-09-19T04:16:14.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4d4c00c370 con 0x7f4d4c004810 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0039c0 0x7f4d4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6935c640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0051d0 0x7f4d4c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 -- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d4c004810 msgr2=0x7f4d4c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d4c004810 0x7f4d4c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 -- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0051d0 msgr2=0x7f4d4c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0051d0 0x7f4d4c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d68b5b640 1 -- 172.21.15.76:0/1351216722 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4c003530 con 0x7f4d4c0039c0 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6935c640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0051d0 0x7f4d4c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:14.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.624+0000 7f4d6935c640 1 -- 172.21.15.76:0/1351216722 reap_dead start 2024-09-19T04:16:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0039c0 0x7f4d4c003de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c4c0 tx=0x7f4d5401cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d537e46bbc0eb66b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400c0e0 con 0x7f4d4c0039c0 2024-09-19T04:16:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d540a9070 con 0x7f4d4c0039c0 2024-09-19T04:16:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d637fe640 1 -- 172.21.15.76:0/1351216722 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54005d80 con 0x7f4d4c0039c0 2024-09-19T04:16:14.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 -- 172.21.15.76:0/1351216722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0039c0 msgr2=0x7f4d4c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0039c0 0x7f4d4c003de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c4c0 tx=0x7f4d5401cc30 comp rx=0 tx=0).stop 2024-09-19T04:16:14.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 -- 172.21.15.76:0/1351216722 shutdown_connections 2024-09-19T04:16:14.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/1351216722 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d4c0039c0 0x7f4d4c003de0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 -- 172.21.15.76:0/1351216722 >> 172.21.15.76:0/1351216722 conn(0x7f4d6408bf90 msgr2=0x7f4d6409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:14.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 -- 172.21.15.76:0/1351216722 shutdown_connections 2024-09-19T04:16:14.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 -- 172.21.15.76:0/1351216722 wait complete. 2024-09-19T04:16:14.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 Processor -- start 2024-09-19T04:16:14.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 -- start start 2024-09-19T04:16:14.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.628+0000 7f4d6adbf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0039c0 0x7f4d641f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6adbf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d641526b0 0x7f4d640cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6adbf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d64152b30 0x7f4d641600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6adbf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6406d190 con 0x7f4d641526b0 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6adbf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d6406ceb0 con 0x7f4d4c0039c0 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6adbf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4d640d2d60 con 0x7f4d64152b30 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d641526b0 0x7f4d640cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d641526b0 0x7f4d640cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38198/0 (socket says 172.21.15.76:38198) 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 -- 172.21.15.76:0/2776458822 learned_addr learned my addr 172.21.15.76:0/2776458822 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0039c0 0x7f4d641f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6935c640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d64152b30 0x7f4d641600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 -- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d64152b30 msgr2=0x7f4d641600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d64152b30 0x7f4d641600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 -- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0039c0 msgr2=0x7f4d641f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0039c0 0x7f4d641f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 -- 172.21.15.76:0/2776458822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d640b8300 con 0x7f4d641526b0 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0039c0 0x7f4d641f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d63fff640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d641526b0 0x7f4d640cdd60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c4d0 tx=0x7f4d5800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d617fa640 1 -- 172.21.15.76:0/2776458822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d580052d0 con 0x7f4d641526b0 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d617fa640 1 -- 172.21.15.76:0/2776458822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d58005470 con 0x7f4d641526b0 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d640b85a0 con 0x7f4d641526b0 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d617fa640 1 -- 172.21.15.76:0/2776458822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58012870 con 0x7f4d641526b0 2024-09-19T04:16:14.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d640b8b60 con 0x7f4d641526b0 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d617fa640 1 -- 172.21.15.76:0/2776458822 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d58012a10 con 0x7f4d641526b0 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d617fa640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d500678c0 0x7f4d50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d617fa640 1 -- 172.21.15.76:0/2776458822 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f4d58085af0 con 0x7f4d641526b0 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d500678c0 0x7f4d50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d500678c0 0x7f4d50069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c490 tx=0x7f4d540b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6de7b000 1 Processor -- start 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.632+0000 7f4d6de7b000 1 -- start start 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8833d70 0x55dbf8834150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf87e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87e7db0 0x55dbf87e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dbf86d1960 con 0x55dbf8833d70 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8615900 con 0x55dbf8834690 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55dbf8638d40 con 0x55dbf87e7db0 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8833d70 0x55dbf8834150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8833d70 0x55dbf8834150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38200/0 (socket says 172.21.15.76:38200) 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3905782231 learned_addr learned my addr 172.21.15.76:0/3905782231 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d63fff640 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf87e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6935c640 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87e7db0 0x55dbf87e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87e7db0 msgr2=0x55dbf87e8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87e7db0 0x55dbf87e8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 msgr2=0x55dbf87e7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf87e7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3905782231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf85df580 con 0x55dbf8833d70 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8833d70 0x55dbf8834150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4d540982d0 tx=0x7f4d54001c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d188574956173f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3905782231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d54007050 con 0x55dbf8833d70 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3905782231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d540a9040 con 0x55dbf8833d70 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d4a7fc640 1 -- 172.21.15.76:0/3905782231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540073f0 con 0x55dbf8833d70 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3905782231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8833d70 msgr2=0x55dbf8834150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8833d70 0x55dbf8834150 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4d540982d0 tx=0x7f4d54001c20 comp rx=0 tx=0).stop 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3905782231 shutdown_connections 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87e7db0 0x55dbf87e8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf87e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3905782231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8833d70 0x55dbf8834150 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3905782231 >> 172.21.15.76:0/3905782231 conn(0x55dbf8815040 msgr2=0x55dbf8817430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:14.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3905782231 shutdown_connections 2024-09-19T04:16:14.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3905782231 wait complete. 2024-09-19T04:16:14.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- start start 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 Processor -- start 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf87e7db0 0x55dbf883ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf8833d70 0x55dbf883b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf883b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8634d00 con 0x55dbf87e7db0 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dbf85f3aa0 con 0x55dbf8834690 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55dbf85d2340 con 0x55dbf8833d70 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d6de7b000 10 client.0 _ll_register_callbacks cb 0x55dbf87d5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf87e7db0 0x55dbf883ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf87e7db0 0x55dbf883ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38210/0 (socket says 172.21.15.76:38210) 2024-09-19T04:16:14.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.636+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3890327108 learned_addr learned my addr 172.21.15.76:0/3890327108 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6935c640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf883b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d63fff640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf8833d70 0x55dbf883b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf8833d70 msgr2=0x55dbf883b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf8833d70 0x55dbf883b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 msgr2=0x55dbf883b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf883b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d68b5b640 1 -- 172.21.15.76:0/3890327108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf85dbdc0 con 0x55dbf87e7db0 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d63fff640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf8833d70 0x55dbf883b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6935c640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf883b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf87e7db0 0x55dbf883ace0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4d540982d0 tx=0x7f4d540199d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c9519f00f61fd2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 1 -- 172.21.15.76:0/3890327108 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d540072a0 con 0x55dbf87e7db0 2024-09-19T04:16:14.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 1 -- 172.21.15.76:0/3890327108 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f4d54007440 con 0x55dbf87e7db0 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d540a9040 con 0x55dbf87e7db0 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 1 -- 172.21.15.76:0/3890327108 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4d54007640 con 0x55dbf87e7db0 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 10 client.5919 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55dbf85d87f0 con 0x55dbf87e7db0 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 1 -- 172.21.15.76:0/3890327108 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d540a9040 con 0x55dbf87e7db0 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 1 -- 172.21.15.76:0/3890327108 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f4d540a55b0 con 0x55dbf87e7db0 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 10 client.5919 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 10 client.5919 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 20 client.5919 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf88430b0 0x55dbf88454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 4 client.5919 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbf85d2e10 con 0x55dbf88430b0 2024-09-19T04:16:14.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d63fff640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf88430b0 0x55dbf88454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 10 client.5919 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d63fff640 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf88430b0 0x55dbf88454a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbf41c3b3f0569af server_cookie=48b68c75eea64d4b in_seq=0 out_seq=0 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 1 -- 172.21.15.76:0/3890327108 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbf85d2e10 con 0x55dbf88430b0 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 10 client.5919 handle_command_reply: tid=0 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 client.5919 shutdown 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf87e7db0 msgr2=0x55dbf883ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf87e7db0 0x55dbf883ace0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4d540982d0 tx=0x7f4d540199d0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 shutdown_connections 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf88430b0 0x55dbf88454a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf8834690 0x55dbf883b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf8833d70 0x55dbf883b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d437fe640 0 client.5919 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:14.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/3890327108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf87e7db0 0x55dbf883ace0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 >> 172.21.15.76:0/3890327108 conn(0x55dbf8815040 msgr2=0x55dbf87e6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:14.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 shutdown_connections 2024-09-19T04:16:14.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 1 -- 172.21.15.76:0/3890327108 wait complete. 2024-09-19T04:16:14.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.640+0000 7f4d6de7b000 20 client.5919 trim_cache size 0 max 16384 2024-09-19T04:16:14.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 Processor -- start 2024-09-19T04:16:14.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- start start 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 0x55dbf88a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d64205060 con 0x55dbf8815040 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d64204220 con 0x55dbf87e7db0 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4d64204930 con 0x55dbf87ec770 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38226/0 (socket says 172.21.15.76:38226) 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 -- 172.21.15.76:0/1326277023 learned_addr learned my addr 172.21.15.76:0/1326277023 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 -- 172.21.15.76:0/1326277023 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 msgr2=0x55dbf88a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 --2- 172.21.15.76:0/1326277023 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 0x55dbf88a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 -- 172.21.15.76:0/1326277023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 msgr2=0x55dbf88a4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 --2- 172.21.15.76:0/1326277023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 -- 172.21.15.76:0/1326277023 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbf85f6470 con 0x55dbf8815040 2024-09-19T04:16:14.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d63fff640 1 --2- 172.21.15.76:0/1326277023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a4700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4d58007fd0 tx=0x7f4d58012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb689b7733cde478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:14.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d49ffb640 1 -- 172.21.15.76:0/1326277023 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58072070 con 0x55dbf8815040 2024-09-19T04:16:14.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d49ffb640 1 -- 172.21.15.76:0/1326277023 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d58053e10 con 0x55dbf8815040 2024-09-19T04:16:14.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d49ffb640 1 -- 172.21.15.76:0/1326277023 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d58010dc0 con 0x55dbf8815040 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- 172.21.15.76:0/1326277023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 msgr2=0x55dbf88a4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/1326277023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a4700 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f4d58007fd0 tx=0x7f4d58012000 comp rx=0 tx=0).stop 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- 172.21.15.76:0/1326277023 shutdown_connections 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/1326277023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/1326277023 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a4700 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/1326277023 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 0x55dbf88a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- 172.21.15.76:0/1326277023 >> 172.21.15.76:0/1326277023 conn(0x55dbf88335c0 msgr2=0x55dbf87eebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- 172.21.15.76:0/1326277023 shutdown_connections 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- 172.21.15.76:0/1326277023 wait complete. 2024-09-19T04:16:14.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 -- start start 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.660+0000 7f4d6de7b000 1 Processor -- start 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 0x55dbf88a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dbf87f33f0 con 0x55dbf8815040 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dbf8600400 con 0x55dbf87e7db0 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55dbf8678780 con 0x55dbf87ec770 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 10 client.0 _ll_register_callbacks cb 0x7f4d4c00bff0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38240/0 (socket says 172.21.15.76:38240) 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 -- 172.21.15.76:0/989808090 learned_addr learned my addr 172.21.15.76:0/989808090 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:14.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d63fff640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 0x55dbf88a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 -- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 msgr2=0x55dbf88a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 0x55dbf88a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 -- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 msgr2=0x55dbf88a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 -- 172.21.15.76:0/989808090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbf85cc640 con 0x55dbf8815040 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6935c640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a58e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4d6408dd40 tx=0x7f4d5c0156a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8eb19c6c22a19fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 1 -- 172.21.15.76:0/989808090 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c01c030 con 0x55dbf8815040 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 1 -- 172.21.15.76:0/989808090 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f4d5c012040 con 0x55dbf8815040 2024-09-19T04:16:14.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d58053e10 con 0x55dbf8815040 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 1 -- 172.21.15.76:0/989808090 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4d5c015c20 con 0x55dbf8815040 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 10 client.5925 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55dbf85d03b0 con 0x55dbf8815040 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 1 -- 172.21.15.76:0/989808090 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d58053e10 con 0x55dbf8815040 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 1 -- 172.21.15.76:0/989808090 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f4d5c018450 con 0x55dbf8815040 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 10 client.5925 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 10 client.5925 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 20 client.5925 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf8833d70 0x55dbf88b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 4 client.5925 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:14.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbf8646470 con 0x55dbf8833d70 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf8833d70 0x55dbf88b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 10 client.5925 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d68b5b640 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf8833d70 0x55dbf88b0bc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7464a13f239b98d8 server_cookie=29d6f23e9618afc8 in_seq=0 out_seq=0 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 1 -- 172.21.15.76:0/989808090 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55dbf8646470 con 0x55dbf8833d70 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 10 client.5925 handle_command_reply: tid=0 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 client.5925 shutdown 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 msgr2=0x55dbf88a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a58e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4d6408dd40 tx=0x7f4d5c0156a0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 shutdown_connections 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x55dbf8833d70 0x55dbf88b0bc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbf8815040 0x55dbf88a58e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d437fe640 0 client.5925 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:14.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbf87ec770 0x55dbf88a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 --2- 172.21.15.76:0/989808090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbf87e7db0 0x55dbf88a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 >> 172.21.15.76:0/989808090 conn(0x55dbf88335c0 msgr2=0x55dbf87ef3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:14.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 shutdown_connections 2024-09-19T04:16:14.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 1 -- 172.21.15.76:0/989808090 wait complete. 2024-09-19T04:16:14.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6de7b000 20 client.5925 trim_cache size 0 max 16384 2024-09-19T04:16:14.675 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: "2735132f-48f6-4722-bdad-c4d08d9e3230": { 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2735132f-48f6-4722-bdad-c4d08d9e3230", 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:14.676 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:14.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d500678c0 msgr2=0x7f4d50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d500678c0 0x7f4d50069d80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400c490 tx=0x7f4d540b0040 comp rx=0 tx=0).stop 2024-09-19T04:16:14.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d641526b0 msgr2=0x7f4d640cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:14.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d641526b0 0x7f4d640cdd60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800c4d0 tx=0x7f4d5800c990 comp rx=0 tx=0).stop 2024-09-19T04:16:14.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 shutdown_connections 2024-09-19T04:16:14.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d500678c0 0x7f4d50069d80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d64152b30 0x7f4d641600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d641526b0 0x7f4d640cdd60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 --2- 172.21.15.76:0/2776458822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d4c0039c0 0x7f4d641f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:14.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 >> 172.21.15.76:0/2776458822 conn(0x7f4d6408bf90 msgr2=0x7f4d640b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:14.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 shutdown_connections 2024-09-19T04:16:14.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:14.664+0000 7f4d6adbf640 1 -- 172.21.15.76:0/2776458822 wait complete. 2024-09-19T04:16:15.748 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:16:15.744+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:16:19.686 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:16:19.686 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:1 scrub status 2024-09-19T04:16:19.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c6e32640 1 Processor -- start 2024-09-19T04:16:19.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c6e32640 1 -- start start 2024-09-19T04:16:19.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c6e32640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f62c01630c0 0x7f62c006f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c6e32640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c006df70 0x7f62c01634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c6e32640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62c006e470 con 0x7f62c00b3290 2024-09-19T04:16:19.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c6e32640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f62c006e750 con 0x7f62c01630c0 2024-09-19T04:16:19.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c6e32640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f62c006ea30 con 0x7f62c006df70 2024-09-19T04:16:19.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c4bce640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f62c01630c0 0x7f62c006f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62bffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c006df70 0x7f62c01634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c53cf640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f62c00b3290 0x7f62c00ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:38126/0 (socket says 172.21.15.76:38126) 2024-09-19T04:16:19.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c53cf640 1 -- 172.21.15.76:0/1744750828 learned_addr learned my addr 172.21.15.76:0/1744750828 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:19.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c4bce640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f62c01630c0 0x7f62c006f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:52738/0 (socket says 172.21.15.76:52738) 2024-09-19T04:16:19.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c4bce640 1 -- 172.21.15.76:0/1744750828 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c006df70 msgr2=0x7f62c01634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1744750828 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c006df70 0x7f62c01634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c4bce640 1 -- 172.21.15.76:0/1744750828 >> v1:172.21.15.116:6789/0 conn(0x7f62c00b3290 legacy=0x7f62c00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.820+0000 7f62c4bce640 1 -- 172.21.15.76:0/1744750828 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c006ed10 con 0x7f62c01630c0 2024-09-19T04:16:19.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1744750828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 0x7f62c006f510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f62b00097c0 tx=0x7f62b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1e8e8e61c84cec2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:19.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62bf7fe640 1 -- 172.21.15.76:0/1744750828 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00b1070 con 0x7f62c01630c0 2024-09-19T04:16:19.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62bf7fe640 1 -- 172.21.15.76:0/1744750828 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b00ac410 con 0x7f62c01630c0 2024-09-19T04:16:19.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62bf7fe640 1 -- 172.21.15.76:0/1744750828 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00ac7e0 con 0x7f62c01630c0 2024-09-19T04:16:19.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62c6e32640 1 -- 172.21.15.76:0/1744750828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 msgr2=0x7f62c006f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1744750828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 0x7f62c006f510 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f62b00097c0 tx=0x7f62b001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62c6e32640 1 -- 172.21.15.76:0/1744750828 shutdown_connections 2024-09-19T04:16:19.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1744750828 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c006df70 0x7f62c01634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1744750828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 0x7f62c006f510 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.824+0000 7f62c6e32640 1 -- 172.21.15.76:0/1744750828 >> 172.21.15.76:0/1744750828 conn(0x7f62c00b1f90 msgr2=0x7f62c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:19.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- 172.21.15.76:0/1744750828 shutdown_connections 2024-09-19T04:16:19.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- 172.21.15.76:0/1744750828 wait complete. 2024-09-19T04:16:19.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 Processor -- start 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- start start 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62c006df70 0x7f62c0153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c00b3290 0x7f62c0154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 0x7f62c015bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0163b60 con 0x7f62c006df70 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f62c01639e0 con 0x7f62c01630c0 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f62c016fe00 con 0x7f62c00b3290 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62c006df70 0x7f62c0153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62c006df70 0x7f62c0153ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38254/0 (socket says 172.21.15.76:38254) 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 -- 172.21.15.76:0/1160973366 learned_addr learned my addr 172.21.15.76:0/1160973366 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c53cf640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 0x7f62c015bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62bffff640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c00b3290 0x7f62c0154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 -- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c00b3290 msgr2=0x7f62c0154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c00b3290 0x7f62c0154330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 -- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 msgr2=0x7f62c015bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 0x7f62c015bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 -- 172.21.15.76:0/1160973366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c0155e10 con 0x7f62c006df70 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62c006df70 0x7f62c0153ad0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f62b00af040 tx=0x7f62b0019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62bd7fa640 1 -- 172.21.15.76:0/1160973366 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00b1040 con 0x7f62c006df70 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62bd7fa640 1 -- 172.21.15.76:0/1160973366 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b00bb070 con 0x7f62c006df70 2024-09-19T04:16:19.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62bd7fa640 1 -- 172.21.15.76:0/1160973366 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00b5410 con 0x7f62c006df70 2024-09-19T04:16:19.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c01560f0 con 0x7f62c006df70 2024-09-19T04:16:19.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.828+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c01566b0 con 0x7f62c006df70 2024-09-19T04:16:19.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62bd7fa640 1 -- 172.21.15.76:0/1160973366 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f62b00b55d0 con 0x7f62c006df70 2024-09-19T04:16:19.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62bd7fa640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62a40678c0 0x7f62a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62bd7fa640 1 -- 172.21.15.76:0/1160973366 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f62b011f9e0 con 0x7f62c006df70 2024-09-19T04:16:19.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62bffff640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62a40678c0 0x7f62a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62bffff640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62a40678c0 0x7f62a4069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62c009e4d0 tx=0x7f62a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:19.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 Processor -- start 2024-09-19T04:16:19.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 -- start start 2024-09-19T04:16:19.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608c67150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c67690 0x560608c1a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c1b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560608b04960 con 0x560608c66d70 2024-09-19T04:16:19.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560608a48900 con 0x560608c67690 2024-09-19T04:16:19.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560608a6bd40 con 0x560608c1adb0 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608c67150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608c67150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38260/0 (socket says 172.21.15.76:38260) 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 -- 172.21.15.76:0/3988125503 learned_addr learned my addr 172.21.15.76:0/3988125503 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c53cf640 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c1b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62bffff640 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c67690 0x560608c1a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 -- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 msgr2=0x560608c1b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c1b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 -- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c67690 msgr2=0x560608c1a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c67690 0x560608c1a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.832+0000 7f62c4bce640 1 -- 172.21.15.76:0/3988125503 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560608a12580 con 0x560608c66d70 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c4bce640 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608c67150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f62b00ec160 tx=0x7f62b00ef660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55236ee3be357bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:19.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62b67fc640 1 -- 172.21.15.76:0/3988125503 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b011f040 con 0x560608c66d70 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62b67fc640 1 -- 172.21.15.76:0/3988125503 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b00f40b0 con 0x560608c66d70 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62b67fc640 1 -- 172.21.15.76:0/3988125503 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b00f88e0 con 0x560608c66d70 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/3988125503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 msgr2=0x560608c67150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608c67150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f62b00ec160 tx=0x7f62b00ef660 comp rx=0 tx=0).stop 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/3988125503 shutdown_connections 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c1b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c67690 0x560608c1a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3988125503 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608c67150 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/3988125503 >> 172.21.15.76:0/3988125503 conn(0x560608c48040 msgr2=0x560608c4a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/3988125503 shutdown_connections 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/3988125503 wait complete. 2024-09-19T04:16:19.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- start start 2024-09-19T04:16:19.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 Processor -- start 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c6dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c66d70 0x560608c6e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c67690 0x560608c6e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560608a67d00 con 0x560608c67690 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560608a26aa0 con 0x560608c66d70 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560608a05340 con 0x560608c1adb0 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 10 client.0 _ll_register_callbacks cb 0x560608c08760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c67690 0x560608c6e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c67690 0x560608c6e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38272/0 (socket says 172.21.15.76:38272) 2024-09-19T04:16:19.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 -- 172.21.15.76:0/1200140132 learned_addr learned my addr 172.21.15.76:0/1200140132 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62bffff640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c66d70 0x560608c6e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c6dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 -- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 msgr2=0x560608c6dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c6dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 -- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c66d70 msgr2=0x560608c6e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c66d70 0x560608c6e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 -- 172.21.15.76:0/1200140132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560608a0edc0 con 0x560608c67690 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c6dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62bffff640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c66d70 0x560608c6e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c53cf640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c67690 0x560608c6e760 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f62b800f820 tx=0x7f62b800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a19d3c7fad6b6e52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:19.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62af7fe640 1 -- 172.21.15.76:0/1200140132 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8017020 con 0x560608c67690 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62af7fe640 1 -- 172.21.15.76:0/1200140132 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f62b800d940 con 0x560608c67690 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b00f40b0 con 0x560608c67690 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62af7fe640 1 -- 172.21.15.76:0/1200140132 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62b8021050 con 0x560608c67690 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 10 client.5934 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x560608a0b7f0 con 0x560608c67690 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62af7fe640 1 -- 172.21.15.76:0/1200140132 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b00f40b0 con 0x560608c67690 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62af7fe640 1 -- 172.21.15.76:0/1200140132 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f62b8024020 con 0x560608c67690 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 10 client.5934 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 10 client.5934 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 20 client.5934 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:19.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c760b0 0x560608c784a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 4 client.5934 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560608a05e10 con 0x560608c760b0 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.836+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c760b0 0x560608c784a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62af7fe640 10 client.5934 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c4bce640 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c760b0 0x560608c784a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d4a00a495b960dd server_cookie=7cf7c14de78a92fa in_seq=0 out_seq=0 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62af7fe640 1 -- 172.21.15.76:0/1200140132 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560608a05e10 con 0x560608c760b0 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62af7fe640 10 client.5934 handle_command_reply: tid=0 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 client.5934 shutdown 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c67690 msgr2=0x560608c6e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c67690 0x560608c6e760 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f62b800f820 tx=0x7f62b800fce0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 shutdown_connections 2024-09-19T04:16:19.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c760b0 0x560608c784a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c67690 0x560608c6e760 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c66d70 0x560608c6e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62af7fe640 0 client.5934 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 --2- 172.21.15.76:0/1200140132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c1adb0 0x560608c6dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 >> 172.21.15.76:0/1200140132 conn(0x560608c48040 msgr2=0x560608c19f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 shutdown_connections 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 1 -- 172.21.15.76:0/1200140132 wait complete. 2024-09-19T04:16:19.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.840+0000 7f62c9eee000 20 client.5934 trim_cache size 0 max 16384 2024-09-19T04:16:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.856+0000 7f62c9eee000 1 Processor -- start 2024-09-19T04:16:19.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.856+0000 7f62c9eee000 1 -- start start 2024-09-19T04:16:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c48040 0x560608cd71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 0x560608cd7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608cd7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0208b00 con 0x560608c66d70 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f62c0207cc0 con 0x560608c1adb0 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f62c02083d0 con 0x560608c48040 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608cd7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608cd7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38278/0 (socket says 172.21.15.76:38278) 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 -- 172.21.15.76:0/3452808109 learned_addr learned my addr 172.21.15.76:0/3452808109 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 -- 172.21.15.76:0/3452808109 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c48040 msgr2=0x560608cd71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 --2- 172.21.15.76:0/3452808109 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c48040 0x560608cd71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 -- 172.21.15.76:0/3452808109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 msgr2=0x560608cd7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 --2- 172.21.15.76:0/3452808109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 0x560608cd7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 -- 172.21.15.76:0/3452808109 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560608a29470 con 0x560608c66d70 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c53cf640 1 --2- 172.21.15.76:0/3452808109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608cd7c40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f62b801f040 tx=0x7f62b8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d74e915b728c49 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62b5ffb640 1 -- 172.21.15.76:0/3452808109 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b8017030 con 0x560608c66d70 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62b5ffb640 1 -- 172.21.15.76:0/3452808109 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b800d520 con 0x560608c66d70 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62b5ffb640 1 -- 172.21.15.76:0/3452808109 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b801c3f0 con 0x560608c66d70 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- 172.21.15.76:0/3452808109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 msgr2=0x560608cd7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3452808109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608cd7c40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f62b801f040 tx=0x7f62b8013000 comp rx=0 tx=0).stop 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- 172.21.15.76:0/3452808109 shutdown_connections 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3452808109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c66d70 0x560608cd7c40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3452808109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 0x560608cd7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- 172.21.15.76:0/3452808109 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c48040 0x560608cd71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- 172.21.15.76:0/3452808109 >> 172.21.15.76:0/3452808109 conn(0x560608c665c0 msgr2=0x560608c21c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- 172.21.15.76:0/3452808109 shutdown_connections 2024-09-19T04:16:19.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- 172.21.15.76:0/3452808109 wait complete. 2024-09-19T04:16:19.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- start start 2024-09-19T04:16:19.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 Processor -- start 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 0x560608cd9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c48040 0x560608cde3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c66d70 0x560608ce0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560608a6b050 con 0x560608c48040 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560608a33400 con 0x560608c1adb0 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560608a6bd40 con 0x560608c66d70 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 10 client.0 _ll_register_callbacks cb 0x7f62b80019c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c48040 0x560608cde3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c48040 0x560608cde3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38286/0 (socket says 172.21.15.76:38286) 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 -- 172.21.15.76:0/2887267231 learned_addr learned my addr 172.21.15.76:0/2887267231 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 -- 172.21.15.76:0/2887267231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c66d70 msgr2=0x560608ce0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c66d70 0x560608ce0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 -- 172.21.15.76:0/2887267231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 msgr2=0x560608cd9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 0x560608cd9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 -- 172.21.15.76:0/2887267231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606089ff640 con 0x560608c48040 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62bffff640 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c48040 0x560608cde3f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f62a8015670 tx=0x7f62a8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11155a86ddb738f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62af7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62af7fe640 1 -- 172.21.15.76:0/2887267231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a8025070 con 0x560608c48040 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62af7fe640 1 -- 172.21.15.76:0/2887267231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f62a8021620 con 0x560608c48040 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62b800d520 con 0x560608c48040 2024-09-19T04:16:19.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62af7fe640 1 -- 172.21.15.76:0/2887267231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f62a8030050 con 0x560608c48040 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 10 client.5940 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.860+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x560608a033b0 con 0x560608c48040 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62af7fe640 1 -- 172.21.15.76:0/2887267231 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b800d520 con 0x560608c48040 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62af7fe640 1 -- 172.21.15.76:0/2887267231 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f62a8033020 con 0x560608c48040 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 10 client.5940 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 10 client.5940 resolve_mds: resolved 3:1 to role '3:1' aka daemon mds.c 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 20 client.5940 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c67690 0x560608ce30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 4 client.5940 mds_command: new command op to 5496 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 --> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560608a79470 con 0x560608c67690 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c4bce640 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c67690 0x560608ce30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:19.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c4bce640 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c67690 0x560608ce30a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d12387bca2be1900 server_cookie=35fd5f13c2e57276 in_seq=0 out_seq=0 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62af7fe640 10 client.5940 ms_handle_connect on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62af7fe640 1 -- 172.21.15.76:0/2887267231 <== mds.1 v2:172.21.15.76:6834/3259378986 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560608a79470 con 0x560608c67690 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62af7fe640 10 client.5940 handle_command_reply: tid=0 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 client.5940 shutdown 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c48040 msgr2=0x560608cde3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c48040 0x560608cde3f0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f62a8015670 tx=0x7f62a8009190 comp rx=0 tx=0).stop 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 shutdown_connections 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:6834/3259378986,v1:172.21.15.76:6835/3259378986] conn(0x560608c67690 0x560608ce30a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560608c66d70 0x560608ce0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560608c48040 0x560608cde3f0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 --2- 172.21.15.76:0/2887267231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560608c1adb0 0x560608cd9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62af7fe640 0 client.5940 ms_handle_reset on v2:172.21.15.76:6834/3259378986 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 >> 172.21.15.76:0/2887267231 conn(0x560608c665c0 msgr2=0x560608c22440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 shutdown_connections 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 1 -- 172.21.15.76:0/2887267231 wait complete. 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c9eee000 20 client.5940 trim_cache size 0 max 16384 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-09-19T04:16:19.873 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:19.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62a40678c0 msgr2=0x7f62a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62a40678c0 0x7f62a4069d80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f62c009e4d0 tx=0x7f62a8008040 comp rx=0 tx=0).stop 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62c006df70 msgr2=0x7f62c0153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62c006df70 0x7f62c0153ad0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f62b00af040 tx=0x7f62b0019240 comp rx=0 tx=0).stop 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 shutdown_connections 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62a40678c0 0x7f62a4069d80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62c01630c0 0x7f62c015bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f62c00b3290 0x7f62c0154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 --2- 172.21.15.76:0/1160973366 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62c006df70 0x7f62c0153ad0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 >> 172.21.15.76:0/1160973366 conn(0x7f62c00b1f90 msgr2=0x7f62c00ad200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 shutdown_connections 2024-09-19T04:16:19.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.864+0000 7f62c6e32640 1 -- 172.21.15.76:0/1160973366 wait complete. 2024-09-19T04:16:19.884 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:16:19.885 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:16:19.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:2 scrub status 2024-09-19T04:16:20.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.996+0000 7f639f3b8640 1 Processor -- start 2024-09-19T04:16:20.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.996+0000 7f639f3b8640 1 -- start start 2024-09-19T04:16:20.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.996+0000 7f639f3b8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f639815c2a0 0x7f63980aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.996+0000 7f639f3b8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 0x7f639806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.996+0000 7f639f3b8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63980b3290 con 0x7f639815bec0 2024-09-19T04:16:20.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.996+0000 7f639f3b8640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f63980b3570 con 0x7f639815c2a0 2024-09-19T04:16:20.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:19.996+0000 7f639f3b8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f63980b3850 con 0x7f63980af2a0 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639d154640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f639815bec0 0x7f63980ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60858/0 (socket says 172.21.15.76:60858) 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639d154640 1 -- 172.21.15.76:0/1372746478 learned_addr learned my addr 172.21.15.76:0/1372746478 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 --2- 172.21.15.76:0/1372746478 >> v2:172.21.15.116:3300/0 conn(0x7f639815c2a0 0x7f63980aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639d955640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 0x7f639806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677656868 0 0) 0x7f63980b3290 con 0x7f639815bec0 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6384003530 con 0x7f639815bec0 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1802473766 0 0) 0x7f6384003530 con 0x7f639815bec0 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 msgr2=0x7f639806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 0x7f639806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 >> v2:172.21.15.116:3300/0 conn(0x7f639815c2a0 msgr2=0x7f63980aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 --2- 172.21.15.76:0/1372746478 >> v2:172.21.15.116:3300/0 conn(0x7f639815c2a0 0x7f63980aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63980b3b30 con 0x7f639815bec0 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6394001680 con 0x7f639815bec0 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 >> v1:172.21.15.76:6789/0 conn(0x7f639815bec0 legacy=0x7f63980ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63840039c0 0x7f6384003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f6384004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 0x7f63840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63980b3570 con 0x7f6384004810 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63980b3850 con 0x7f63840039c0 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f638400fde0 con 0x7f63980af2a0 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f6384004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639d154640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63840039c0 0x7f6384003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639d955640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 0x7f63840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 -- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 msgr2=0x7f63840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63980af2a0 0x7f63840051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 -- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63840039c0 msgr2=0x7f6384003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63840039c0 0x7f6384003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 -- 172.21.15.76:0/1372746478 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6384003530 con 0x7f6384004810 2024-09-19T04:16:20.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639d154640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63840039c0 0x7f6384003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:20.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f639c953640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f6384004c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6388009ab0 tx=0x7f638801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea95a53a62f96fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63880b2070 con 0x7f6384004810 2024-09-19T04:16:20.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63880a55a0 con 0x7f6384004810 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.000+0000 7f638ffff640 1 -- 172.21.15.76:0/1372746478 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63880096b0 con 0x7f6384004810 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- 172.21.15.76:0/1372746478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 msgr2=0x7f6384004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 --2- 172.21.15.76:0/1372746478 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f6384004c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6388009ab0 tx=0x7f638801ca30 comp rx=0 tx=0).stop 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d955640 1 -- 172.21.15.76:0/1372746478 reap_dead start 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- 172.21.15.76:0/1372746478 shutdown_connections 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- 172.21.15.76:0/1372746478 >> 172.21.15.76:0/1372746478 conn(0x7f63980b1f90 msgr2=0x7f63980b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- 172.21.15.76:0/1372746478 shutdown_connections 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- 172.21.15.76:0/1372746478 wait complete. 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 Processor -- start 2024-09-19T04:16:20.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- start start 2024-09-19T04:16:20.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f63980998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6398156490 0x7f6398072f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398156910 0x7f63980734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63980af730 con 0x7f6384004810 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63980b40f0 con 0x7f6398156910 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f63980afc90 con 0x7f6398156490 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f63980998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f63980998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38312/0 (socket says 172.21.15.76:38312) 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 -- 172.21.15.76:0/98100859 learned_addr learned my addr 172.21.15.76:0/98100859 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d955640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398156910 0x7f63980734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639c953640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6398156490 0x7f6398072f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 -- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6398156490 msgr2=0x7f6398072f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6398156490 0x7f6398072f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 -- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398156910 msgr2=0x7f63980734c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398156910 0x7f63980734c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 -- 172.21.15.76:0/98100859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6398152560 con 0x7f6384004810 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d955640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398156910 0x7f63980734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639d154640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f63980998b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6394010b10 tx=0x7f6394005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f638dffb640 1 -- 172.21.15.76:0/98100859 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639401b070 con 0x7f6384004810 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f638dffb640 1 -- 172.21.15.76:0/98100859 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6394011040 con 0x7f6384004810 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f638dffb640 1 -- 172.21.15.76:0/98100859 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639400ede0 con 0x7f6384004810 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6398152840 con 0x7f6384004810 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.004+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6398152e00 con 0x7f6384004810 2024-09-19T04:16:20.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f638dffb640 1 -- 172.21.15.76:0/98100859 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f639401e070 con 0x7f6384004810 2024-09-19T04:16:20.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f638dffb640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f637c0678f0 0x7f637c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f638dffb640 1 -- 172.21.15.76:0/98100859 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f63940888d0 con 0x7f6384004810 2024-09-19T04:16:20.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639c953640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f637c0678f0 0x7f637c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639c953640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f637c0678f0 0x7f637c069db0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63880af000 tx=0x7f63880046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 Processor -- start 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 -- start start 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f95150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f48870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485f49190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c485e32960 con 0x55c485f48db0 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c485d76900 con 0x55c485f95690 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55c485d99d40 con 0x55c485f94d70 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485f49190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485f49190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38324/0 (socket says 172.21.15.76:38324) 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 -- 172.21.15.76:0/3221048386 learned_addr learned my addr 172.21.15.76:0/3221048386 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639c953640 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f48870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d154640 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f95150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 -- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 msgr2=0x55c485f95150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f95150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 -- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 msgr2=0x55c485f48870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f48870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.008+0000 7f639d955640 1 -- 172.21.15.76:0/3221048386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c485d40580 con 0x55c485f48db0 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485f49190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f639815c680 tx=0x7f6390009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530d3b7cc74584ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f637affd640 1 -- 172.21.15.76:0/3221048386 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639001b070 con 0x55c485f48db0 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f637affd640 1 -- 172.21.15.76:0/3221048386 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6390016400 con 0x55c485f48db0 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f637affd640 1 -- 172.21.15.76:0/3221048386 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900167f0 con 0x55c485f48db0 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/3221048386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 msgr2=0x55c485f49190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485f49190 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f639815c680 tx=0x7f6390009750 comp rx=0 tx=0).stop 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/3221048386 shutdown_connections 2024-09-19T04:16:20.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485f49190 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f48870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- 172.21.15.76:0/3221048386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f95150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/3221048386 >> 172.21.15.76:0/3221048386 conn(0x55c485f76040 msgr2=0x55c485f78430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/3221048386 shutdown_connections 2024-09-19T04:16:20.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/3221048386 wait complete. 2024-09-19T04:16:20.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- start start 2024-09-19T04:16:20.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 Processor -- start 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c485f9bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f94d70 0x55c485f9c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f95690 0x55c485f9c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c485d95d00 con 0x55c485f95690 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55c485d54aa0 con 0x55c485f94d70 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55c485d33340 con 0x55c485f48db0 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 10 client.0 _ll_register_callbacks cb 0x55c485f36760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f95690 0x55c485f9c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f95690 0x55c485f9c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38328/0 (socket says 172.21.15.76:38328) 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 -- 172.21.15.76:0/520777217 learned_addr learned my addr 172.21.15.76:0/520777217 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d154640 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c485f9bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639c953640 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f94d70 0x55c485f9c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 -- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 msgr2=0x55c485f9bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c485f9bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 -- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f94d70 msgr2=0x55c485f9c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f94d70 0x55c485f9c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 -- 172.21.15.76:0/520777217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c485d3cdc0 con 0x55c485f95690 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f639d955640 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f95690 0x55c485f9c760 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f639000b010 tx=0x7f6390004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c229f8c89a73db1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f6373fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f6373fff640 1 -- 172.21.15.76:0/520777217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6390026020 con 0x55c485f95690 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f6373fff640 1 -- 172.21.15.76:0/520777217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f6390016d80 con 0x55c485f95690 2024-09-19T04:16:20.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6390016400 con 0x55c485f95690 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f6373fff640 1 -- 172.21.15.76:0/520777217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6390030050 con 0x55c485f95690 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 10 client.5952 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55c485d397f0 con 0x55c485f95690 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f6373fff640 1 -- 172.21.15.76:0/520777217 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6390016400 con 0x55c485f95690 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f6373fff640 1 -- 172.21.15.76:0/520777217 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f6390033020 con 0x55c485f95690 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 10 client.5952 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 10 client.5952 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 20 client.5952 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa40b0 0x55c485fa64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 4 client.5952 mds_command: new command op to 4754 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.012+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 --> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c485d33e10 con 0x55c485fa40b0 2024-09-19T04:16:20.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f639d154640 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa40b0 0x55c485fa64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f6373fff640 10 client.5952 ms_handle_connect on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f639d154640 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa40b0 0x55c485fa64a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9f5a1f332e06ce63 server_cookie=2fbf4581addc1035 in_seq=0 out_seq=0 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f6373fff640 1 -- 172.21.15.76:0/520777217 <== mds.2 v2:172.21.15.116:6834/3863430206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c485d33e10 con 0x55c485fa40b0 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f6373fff640 10 client.5952 handle_command_reply: tid=0 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 client.5952 shutdown 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f95690 msgr2=0x55c485f9c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f95690 0x55c485f9c760 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f639000b010 tx=0x7f6390004a50 comp rx=0 tx=0).stop 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 shutdown_connections 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f95690 0x55c485f9c760 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f94d70 0x55c485f9c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa40b0 0x55c485fa64a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 --2- 172.21.15.76:0/520777217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c485f9bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f6373fff640 0 client.5952 ms_handle_reset on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:20.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 >> 172.21.15.76:0/520777217 conn(0x55c485f76040 msgr2=0x55c485f47f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 shutdown_connections 2024-09-19T04:16:20.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 1 -- 172.21.15.76:0/520777217 wait complete. 2024-09-19T04:16:20.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.016+0000 7f63a2474000 20 client.5952 trim_cache size 0 max 16384 2024-09-19T04:16:20.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 Processor -- start 2024-09-19T04:16:20.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 -- start start 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485fa98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f50c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f51140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63981fd4f0 con 0x55c485f48db0 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63981fc690 con 0x55c485f95690 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f63981fcdc0 con 0x55c485f94d70 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485fa98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485fa98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38334/0 (socket says 172.21.15.76:38334) 2024-09-19T04:16:20.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 -- 172.21.15.76:0/2287265787 learned_addr learned my addr 172.21.15.76:0/2287265787 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d955640 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f51140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639c953640 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f50c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 -- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 msgr2=0x55c485f50c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f50c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 -- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 msgr2=0x55c485f51140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f51140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 -- 172.21.15.76:0/2287265787 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c485d57470 con 0x55c485f48db0 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d955640 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.036+0000 7f639d154640 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485fa98f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f63940588c0 tx=0x7f6394088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2b7863412c18e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f637a7fc640 1 -- 172.21.15.76:0/2287265787 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63940617a0 con 0x55c485f48db0 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f637a7fc640 1 -- 172.21.15.76:0/2287265787 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6394085070 con 0x55c485f48db0 2024-09-19T04:16:20.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f637a7fc640 1 -- 172.21.15.76:0/2287265787 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6394056ec0 con 0x55c485f48db0 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/2287265787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 msgr2=0x55c485fa98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485fa98f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f63940588c0 tx=0x7f6394088040 comp rx=0 tx=0).stop 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/2287265787 shutdown_connections 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485f51140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f94d70 0x55c485f50c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- 172.21.15.76:0/2287265787 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f48db0 0x55c485fa98f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/2287265787 >> 172.21.15.76:0/2287265787 conn(0x55c485f76040 msgr2=0x55c485fa1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/2287265787 shutdown_connections 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/2287265787 wait complete. 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- start start 2024-09-19T04:16:20.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 Processor -- start 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c486008a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f94d70 0x55c486008fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485fa87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55c485d3ff20 con 0x55c485f94d70 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63981fb710 con 0x55c485f95690 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55c485d61400 con 0x55c485f48db0 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 10 client.0 _ll_register_callbacks cb 0x7f6390019070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f94d70 0x55c486008fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f94d70 0x55c486008fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38340/0 (socket says 172.21.15.76:38340) 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 -- 172.21.15.76:0/1283618431 learned_addr learned my addr 172.21.15.76:0/1283618431 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639d154640 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c486008a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 -- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 msgr2=0x55c486008a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c486008a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639d955640 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485fa87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 -- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 msgr2=0x55c485fa87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485fa87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 -- 172.21.15.76:0/1283618431 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c485d2d640 con 0x55c485f94d70 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f639c953640 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f94d70 0x55c486008fa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f638801d2d0 tx=0x7f6388019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1887d92c9b99ada server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f6373fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f6373fff640 1 -- 172.21.15.76:0/1283618431 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63880b2040 con 0x55c485f94d70 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f6373fff640 1 -- 172.21.15.76:0/1283618431 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f6388003af0 con 0x55c485f94d70 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6394085070 con 0x55c485f94d70 2024-09-19T04:16:20.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f6373fff640 1 -- 172.21.15.76:0/1283618431 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f63880d1050 con 0x55c485f94d70 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 10 client.5958 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55c485d313b0 con 0x55c485f94d70 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f6373fff640 1 -- 172.21.15.76:0/1283618431 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6394085070 con 0x55c485f94d70 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f6373fff640 1 -- 172.21.15.76:0/1283618431 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f63880d4020 con 0x55c485f94d70 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 10 client.5958 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 10 client.5958 resolve_mds: resolved 3:2 to role '3:2' aka daemon mds.b 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 20 client.5958 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa8d20 0x55c485fa9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 4 client.5958 mds_command: new command op to 4754 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.040+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 --> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c485da7470 con 0x55c485fa8d20 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639d154640 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa8d20 0x55c485fa9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f6373fff640 10 client.5958 ms_handle_connect on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639d154640 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa8d20 0x55c485fa9130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47c5914a58762d9b server_cookie=403311adb0d02d1a in_seq=0 out_seq=0 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f6373fff640 1 -- 172.21.15.76:0/1283618431 <== mds.2 v2:172.21.15.116:6834/3863430206 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c485da7470 con 0x55c485fa8d20 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f6373fff640 10 client.5958 handle_command_reply: tid=0 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 client.5958 shutdown 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f94d70 msgr2=0x55c486008fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f94d70 0x55c486008fa0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f638801d2d0 tx=0x7f6388019830 comp rx=0 tx=0).stop 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 shutdown_connections 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55c485f95690 0x55c485fa87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55c485f94d70 0x55c486008fa0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:6834/3863430206,v1:172.21.15.116:6835/3863430206] conn(0x55c485fa8d20 0x55c485fa9130 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 --2- 172.21.15.76:0/1283618431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55c485f48db0 0x55c486008a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f6373fff640 0 client.5958 ms_handle_reset on v2:172.21.15.116:6834/3863430206 2024-09-19T04:16:20.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 >> 172.21.15.76:0/1283618431 conn(0x55c485f76040 msgr2=0x55c485fa22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 shutdown_connections 2024-09-19T04:16:20.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 1 -- 172.21.15.76:0/1283618431 wait complete. 2024-09-19T04:16:20.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f63a2474000 20 client.5958 trim_cache size 0 max 16384 2024-09-19T04:16:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:20.053 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-09-19T04:16:20.053 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-09-19T04:16:20.053 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f637c0678f0 msgr2=0x7f637c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f637c0678f0 0x7f637c069db0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f63880af000 tx=0x7f63880046d0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 msgr2=0x7f63980998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f63980998b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6394010b10 tx=0x7f6394005810 comp rx=0 tx=0).stop 2024-09-19T04:16:20.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 shutdown_connections 2024-09-19T04:16:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f637c0678f0 0x7f637c069db0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6398156910 0x7f63980734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6398156490 0x7f6398072f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 --2- 172.21.15.76:0/98100859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6384004810 0x7f63980998b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 >> 172.21.15.76:0/98100859 conn(0x7f63980b1f90 msgr2=0x7f639806df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 shutdown_connections 2024-09-19T04:16:20.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.044+0000 7f639f3b8640 1 -- 172.21.15.76:0/98100859 wait complete. 2024-09-19T04:16:20.064 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:16:20.064 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:16:20.064 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-19T04:16:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268cf5d640 1 Processor -- start 2024-09-19T04:16:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268cf5d640 1 -- start start 2024-09-19T04:16:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268cf5d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f268815e5f0 0x7f268806d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268cf5d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268806d9c0 con 0x7f26880ce9d0 2024-09-19T04:16:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268cf5d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f268806dca0 con 0x7f26880cfaf0 2024-09-19T04:16:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268cf5d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f268806df80 con 0x7f268815e5f0 2024-09-19T04:16:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f26880ce9d0 0x7f26880cf3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60870/0 (socket says 172.21.15.76:60870) 2024-09-19T04:16:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 -- 172.21.15.76:0/3199132809 learned_addr learned my addr 172.21.15.76:0/3199132809 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1769026958 0 0) 0x7f268806d9c0 con 0x7f26880ce9d0 2024-09-19T04:16:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268659c640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f268815e5f0 0x7f268806d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2670003530 con 0x7f26880ce9d0 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3298456343 0 0) 0x7f2670003530 con 0x7f26880ce9d0 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f268815e5f0 msgr2=0x7f268806d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f268815e5f0 0x7f268806d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 >> v1:172.21.15.116:6789/0 conn(0x7f26880cfaf0 legacy=0x7f26880d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268806e260 con 0x7f26880ce9d0 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2678001680 con 0x7f26880ce9d0 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 >> v1:172.21.15.76:6789/0 conn(0x7f26880ce9d0 legacy=0x7f26880cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26700039f0 0x7f2670003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2670004350 0x7f2670004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2670005030 0x7f267000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f268806df80 con 0x7f2670004350 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f267000bf10 con 0x7f2670005030 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f267000c0c0 con 0x7f26700039f0 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2670004350 0x7f2670004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f268659c640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26700039f0 0x7f2670003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 -- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26700039f0 msgr2=0x7f2670003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26700039f0 0x7f2670003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 -- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2670005030 msgr2=0x7f267000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2670005030 0x7f267000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.176+0000 7f2685d9b640 1 -- 172.21.15.76:0/3199132809 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2670003530 con 0x7f2670004350 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f2685d9b640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2670004350 0x7f2670004830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f267800c210 tx=0x7f267801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c4944d9ab7d0f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678007d20 con 0x7f2670004350 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2678005ae0 con 0x7f2670004350 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268559a640 1 -- 172.21.15.76:0/3199132809 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678007070 con 0x7f2670004350 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 -- 172.21.15.76:0/3199132809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2670004350 msgr2=0x7f2670004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 --2- 172.21.15.76:0/3199132809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2670004350 0x7f2670004830 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f267800c210 tx=0x7f267801f3b0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f2686d9d640 1 -- 172.21.15.76:0/3199132809 reap_dead start 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 -- 172.21.15.76:0/3199132809 shutdown_connections 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 -- 172.21.15.76:0/3199132809 >> 172.21.15.76:0/3199132809 conn(0x7f2688074030 msgr2=0x7f268809d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 -- 172.21.15.76:0/3199132809 shutdown_connections 2024-09-19T04:16:20.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 -- 172.21.15.76:0/3199132809 wait complete. 2024-09-19T04:16:20.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 Processor -- start 2024-09-19T04:16:20.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.180+0000 7f268cf5d640 1 -- start start 2024-09-19T04:16:20.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2688158d30 0x7f26880ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268815d530 0x7f26880cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26880cb450 0x7f26880cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f268806d420 con 0x7f268815d530 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f268806d2a0 con 0x7f26880cb450 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f268806d5a0 con 0x7f2688158d30 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268815d530 0x7f26880cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268815d530 0x7f26880cc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38352/0 (socket says 172.21.15.76:38352) 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 -- 172.21.15.76:0/3960612946 learned_addr learned my addr 172.21.15.76:0/3960612946 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268659c640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2688158d30 0x7f26880ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 -- 172.21.15.76:0/3960612946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2688158d30 msgr2=0x7f26880ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2688158d30 0x7f26880ce130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 -- 172.21.15.76:0/3960612946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26880cb450 msgr2=0x7f26880cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26880cb450 0x7f26880cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 -- 172.21.15.76:0/3960612946 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26881f7350 con 0x7f268815d530 2024-09-19T04:16:20.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2685d9b640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268815d530 0x7f26880cc0e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f267800c210 tx=0x7f2670005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2676ffd640 1 -- 172.21.15.76:0/3960612946 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780047f0 con 0x7f268815d530 2024-09-19T04:16:20.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2676ffd640 1 -- 172.21.15.76:0/3960612946 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2678004990 con 0x7f268815d530 2024-09-19T04:16:20.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26881f7630 con 0x7f268815d530 2024-09-19T04:16:20.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2676ffd640 1 -- 172.21.15.76:0/3960612946 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2678004d40 con 0x7f268815d530 2024-09-19T04:16:20.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26881f3b50 con 0x7f268815d530 2024-09-19T04:16:20.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2676ffd640 1 -- 172.21.15.76:0/3960612946 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2678007070 con 0x7f268815d530 2024-09-19T04:16:20.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2676ffd640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26680678f0 0x7f2668069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f2676ffd640 1 -- 172.21.15.76:0/3960612946 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f2678123310 con 0x7f268815d530 2024-09-19T04:16:20.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268659c640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26680678f0 0x7f2668069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.184+0000 7f268659c640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26680678f0 0x7f2668069db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f267c009930 tx=0x7f267c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 Processor -- start 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- start start 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d464d70 0x55bf9d465150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d418870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d418db0 0x55bf9d419190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf9d302960 con 0x55bf9d418db0 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf9d246900 con 0x55bf9d465690 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bf9d269d40 con 0x55bf9d464d70 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d418db0 0x55bf9d419190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d418db0 0x55bf9d419190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38368/0 (socket says 172.21.15.76:38368) 2024-09-19T04:16:20.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 -- 172.21.15.76:0/2184805750 learned_addr learned my addr 172.21.15.76:0/2184805750 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2685d9b640 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d418870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f268659c640 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d464d70 0x55bf9d465150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 -- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d464d70 msgr2=0x55bf9d465150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d464d70 0x55bf9d465150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 -- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 msgr2=0x55bf9d418870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d418870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 -- 172.21.15.76:0/2184805750 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9d210580 con 0x55bf9d418db0 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2686d9d640 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d418db0 0x55bf9d419190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f268806ce60 tx=0x7f2680001f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b7a74b35b30cd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2667fff640 1 -- 172.21.15.76:0/2184805750 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680017070 con 0x55bf9d418db0 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2667fff640 1 -- 172.21.15.76:0/2184805750 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268000c690 con 0x55bf9d418db0 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2667fff640 1 -- 172.21.15.76:0/2184805750 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268000ca80 con 0x55bf9d418db0 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- 172.21.15.76:0/2184805750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d418db0 msgr2=0x55bf9d419190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d418db0 0x55bf9d419190 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f268806ce60 tx=0x7f2680001f50 comp rx=0 tx=0).stop 2024-09-19T04:16:20.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- 172.21.15.76:0/2184805750 shutdown_connections 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d418db0 0x55bf9d419190 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d418870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- 172.21.15.76:0/2184805750 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d464d70 0x55bf9d465150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- 172.21.15.76:0/2184805750 >> 172.21.15.76:0/2184805750 conn(0x55bf9d446040 msgr2=0x55bf9d448430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- 172.21.15.76:0/2184805750 shutdown_connections 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- 172.21.15.76:0/2184805750 wait complete. 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- start start 2024-09-19T04:16:20.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 Processor -- start 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d418db0 0x55bf9d4205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d464d70 0x55bf9d420ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d465690 0x55bf9d446420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf9d265d00 con 0x55bf9d464d70 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf9d224aa0 con 0x55bf9d418db0 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.188+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bf9d203340 con 0x55bf9d465690 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 10 client.0 _ll_register_callbacks cb 0x55bf9d406760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d464d70 0x55bf9d420ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d464d70 0x55bf9d420ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38376/0 (socket says 172.21.15.76:38376) 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 -- 172.21.15.76:0/2900504529 learned_addr learned my addr 172.21.15.76:0/2900504529 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f268659c640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d418db0 0x55bf9d4205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2686d9d640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d465690 0x55bf9d446420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 -- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d465690 msgr2=0x55bf9d446420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d465690 0x55bf9d446420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 -- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d418db0 msgr2=0x55bf9d4205a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d418db0 0x55bf9d4205a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 -- 172.21.15.76:0/2900504529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9d20cdc0 con 0x55bf9d464d70 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2686d9d640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d465690 0x55bf9d446420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2685d9b640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d464d70 0x55bf9d420ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f267809bb00 tx=0x7f26780f31d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54552028f011a3f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 1 -- 172.21.15.76:0/2900504529 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780c2040 con 0x55bf9d464d70 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 1 -- 172.21.15.76:0/2900504529 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f26780f18a0 con 0x55bf9d464d70 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f268000c690 con 0x55bf9d464d70 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 1 -- 172.21.15.76:0/2900504529 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f26780f7950 con 0x55bf9d464d70 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 10 client.5970 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55bf9d2097f0 con 0x55bf9d464d70 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 1 -- 172.21.15.76:0/2900504529 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268000c690 con 0x55bf9d464d70 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 1 -- 172.21.15.76:0/2900504529 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f26780efc10 con 0x55bf9d464d70 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 10 client.5970 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 10 client.5970 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 20 client.5970 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:20.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d446e30 0x55bf9d476020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 4 client.5970 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf9d203e10 con 0x55bf9d446e30 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f268659c640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d446e30 0x55bf9d476020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f268659c640 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d446e30 0x55bf9d476020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=376ea7cf25dc6ca4 server_cookie=31427fc3c49af07c in_seq=0 out_seq=0 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 10 client.5970 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 1 -- 172.21.15.76:0/2900504529 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf9d203e10 con 0x55bf9d446e30 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 10 client.5970 handle_command_reply: tid=0 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 client.5970 shutdown 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d464d70 msgr2=0x55bf9d420ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d464d70 0x55bf9d420ae0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f267809bb00 tx=0x7f26780f31d0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 shutdown_connections 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d465690 0x55bf9d446420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d464d70 0x55bf9d420ae0 secure :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f267809bb00 tx=0x7f26780f31d0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d446e30 0x55bf9d476020 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 --2- 172.21.15.76:0/2900504529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d418db0 0x55bf9d4205a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f26657fa640 0 client.5970 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:20.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 >> 172.21.15.76:0/2900504529 conn(0x55bf9d446040 msgr2=0x55bf9d417f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 shutdown_connections 2024-09-19T04:16:20.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 1 -- 172.21.15.76:0/2900504529 wait complete. 2024-09-19T04:16:20.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.192+0000 7f2690019000 20 client.5970 trim_cache size 0 max 16384 2024-09-19T04:16:20.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 Processor -- start 2024-09-19T04:16:20.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- start start 2024-09-19T04:16:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 0x55bf9d444d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d465690 0x55bf9d445270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d420330 0x55bf9d420710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2688201490 con 0x55bf9d465690 2024-09-19T04:16:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2688200630 con 0x55bf9d420330 2024-09-19T04:16:20.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2688200d60 con 0x55bf9d418db0 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d465690 0x55bf9d445270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d465690 0x55bf9d445270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38378/0 (socket says 172.21.15.76:38378) 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 -- 172.21.15.76:0/1924965969 learned_addr learned my addr 172.21.15.76:0/1924965969 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 -- 172.21.15.76:0/1924965969 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 msgr2=0x55bf9d444d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 --2- 172.21.15.76:0/1924965969 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 0x55bf9d444d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 -- 172.21.15.76:0/1924965969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d420330 msgr2=0x55bf9d420710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 --2- 172.21.15.76:0/1924965969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d420330 0x55bf9d420710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 -- 172.21.15.76:0/1924965969 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9d227470 con 0x55bf9d465690 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2685d9b640 1 --2- 172.21.15.76:0/1924965969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d465690 0x55bf9d445270 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f26780f3530 tx=0x7f26780bd000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75ee2bc6f77bf74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f26677fe640 1 -- 172.21.15.76:0/1924965969 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780f3ca0 con 0x55bf9d465690 2024-09-19T04:16:20.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f26677fe640 1 -- 172.21.15.76:0/1924965969 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26780f18a0 con 0x55bf9d465690 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f26677fe640 1 -- 172.21.15.76:0/1924965969 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780efc10 con 0x55bf9d465690 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- 172.21.15.76:0/1924965969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d465690 msgr2=0x55bf9d445270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 --2- 172.21.15.76:0/1924965969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d465690 0x55bf9d445270 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f26780f3530 tx=0x7f26780bd000 comp rx=0 tx=0).stop 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- 172.21.15.76:0/1924965969 shutdown_connections 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 --2- 172.21.15.76:0/1924965969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d420330 0x55bf9d420710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 --2- 172.21.15.76:0/1924965969 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d465690 0x55bf9d445270 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 --2- 172.21.15.76:0/1924965969 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 0x55bf9d444d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- 172.21.15.76:0/1924965969 >> 172.21.15.76:0/1924965969 conn(0x55bf9d446040 msgr2=0x55bf9d463b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- 172.21.15.76:0/1924965969 shutdown_connections 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- 172.21.15.76:0/1924965969 wait complete. 2024-09-19T04:16:20.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 -- start start 2024-09-19T04:16:20.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.212+0000 7f2690019000 1 Processor -- start 2024-09-19T04:16:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 0x55bf9d4794b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d420330 0x55bf9d477580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d477ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf9d2283f0 con 0x55bf9d420330 2024-09-19T04:16:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf9d265d00 con 0x55bf9d465690 2024-09-19T04:16:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bf9d277fd0 con 0x55bf9d418db0 2024-09-19T04:16:20.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 10 client.0 _ll_register_callbacks cb 0x7f2678007e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d420330 0x55bf9d477580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d420330 0x55bf9d477580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38386/0 (socket says 172.21.15.76:38386) 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 -- 172.21.15.76:0/4195975565 learned_addr learned my addr 172.21.15.76:0/4195975565 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268659c640 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 0x55bf9d4794b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2686d9d640 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d477ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 -- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 msgr2=0x55bf9d4794b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 0x55bf9d4794b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 -- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 msgr2=0x55bf9d477ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d477ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 -- 172.21.15.76:0/4195975565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9d1fd640 con 0x55bf9d420330 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2685d9b640 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d420330 0x55bf9d477580 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2678001210 tx=0x7f26781215e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b990a466aa52ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 1 -- 172.21.15.76:0/4195975565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26780c2070 con 0x55bf9d420330 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 1 -- 172.21.15.76:0/4195975565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f26780f2bf0 con 0x55bf9d420330 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26780f18a0 con 0x55bf9d420330 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 1 -- 172.21.15.76:0/4195975565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f26780f3ca0 con 0x55bf9d420330 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 10 client.5976 fetch_fsmap learned FSMap version 39 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) -- 0x55bf9d2013b0 con 0x55bf9d420330 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 1 -- 172.21.15.76:0/4195975565 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26780f18a0 con 0x55bf9d420330 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 1 -- 172.21.15.76:0/4195975565 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 39) ==== 3558+0+0 (secure 0 0 0) 0x7f26780f8ae0 con 0x55bf9d420330 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 10 client.5976 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 10 client.5976 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 20 client.5976 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d478000 0x55bf9d4d9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 4 client.5976 mds_command: new command op to 5502 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 --> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bf9d277470 con 0x55bf9d478000 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268659c640 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d478000 0x55bf9d4d9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268659c640 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d478000 0x55bf9d4d9620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657a5ea473cf3963 server_cookie=4a423965da851c41 in_seq=0 out_seq=0 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 10 client.5976 ms_handle_connect on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 1 -- 172.21.15.76:0/4195975565 <== mds.0 v2:172.21.15.76:6836/3635635274 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55bf9d277470 con 0x55bf9d478000 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 10 client.5976 handle_command_reply: tid=0 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 client.5976 shutdown 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d420330 msgr2=0x55bf9d477580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d420330 0x55bf9d477580 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2678001210 tx=0x7f26781215e0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 shutdown_connections 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:6836/3635635274,v1:172.21.15.76:6837/3635635274] conn(0x55bf9d478000 0x55bf9d4d9620 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf9d465690 0x55bf9d477ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f26657fa640 0 client.5976 ms_handle_reset on v2:172.21.15.76:6836/3635635274 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf9d420330 0x55bf9d477580 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 --2- 172.21.15.76:0/4195975565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf9d418db0 0x55bf9d4794b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 >> 172.21.15.76:0/4195975565 conn(0x55bf9d446040 msgr2=0x55bf9d41f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 shutdown_connections 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 1 -- 172.21.15.76:0/4195975565 wait complete. 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f2690019000 20 client.5976 trim_cache size 0 max 16384 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-09-19T04:16:20.227 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-09-19T04:16:20.228 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:20.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26680678f0 msgr2=0x7f2668069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26680678f0 0x7f2668069db0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f267c009930 tx=0x7f267c008040 comp rx=0 tx=0).stop 2024-09-19T04:16:20.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268815d530 msgr2=0x7f26880cc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268815d530 0x7f26880cc0e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f267800c210 tx=0x7f2670005840 comp rx=0 tx=0).stop 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 shutdown_connections 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26680678f0 0x7f2668069db0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26880cb450 0x7f26880cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f268815d530 0x7f26880cc0e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 --2- 172.21.15.76:0/3960612946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2688158d30 0x7f26880ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 >> 172.21.15.76:0/3960612946 conn(0x7f2670004350 msgr2=0x7f268806e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 shutdown_connections 2024-09-19T04:16:20.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.216+0000 7f268cf5d640 1 -- 172.21.15.76:0/3960612946 wait complete. 2024-09-19T04:16:20.237 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:16:20.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfe56b640 1 Processor -- start 2024-09-19T04:16:20.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfe56b640 1 -- start start 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfe56b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0df8161a60 0x7f0df80d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfe56b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0df80cf620 0x7f0df806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfe56b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0df80d32b0 con 0x7f0df80d2e90 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfe56b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0df80d0300 con 0x7f0df80cf620 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfe56b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0df80cfa90 con 0x7f0df8161a60 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfcb08640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0df80cf620 0x7f0df806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfcb08640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0df80cf620 0x7f0df806d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38390/0 (socket says 172.21.15.76:38390) 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2673981227 learned_addr learned my addr 172.21.15.76:0/2673981227 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0df7fff640 1 --2- 172.21.15.76:0/2673981227 >> v2:172.21.15.116:3300/0 conn(0x7f0df8161a60 0x7f0df80d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2673981227 >> v2:172.21.15.116:3300/0 conn(0x7f0df8161a60 msgr2=0x7f0df80d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfcb08640 1 --2- 172.21.15.76:0/2673981227 >> v2:172.21.15.116:3300/0 conn(0x7f0df8161a60 0x7f0df80d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2673981227 >> v1:172.21.15.116:6789/0 conn(0x7f0df80d2e90 legacy=0x7f0df80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.348+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2673981227 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df80b0f60 con 0x7f0df80cf620 2024-09-19T04:16:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfcb08640 1 --2- 172.21.15.76:0/2673981227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df80cf620 0x7f0df806d6b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0df808e010 tx=0x7f0df001c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4336532a3a78240 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0df6ffd640 1 -- 172.21.15.76:0/2673981227 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00a57b0 con 0x7f0df80cf620 2024-09-19T04:16:20.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0df6ffd640 1 -- 172.21.15.76:0/2673981227 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df00a5950 con 0x7f0df80cf620 2024-09-19T04:16:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0df6ffd640 1 -- 172.21.15.76:0/2673981227 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00a5d20 con 0x7f0df80cf620 2024-09-19T04:16:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2673981227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df80cf620 msgr2=0x7f0df806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2673981227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df80cf620 0x7f0df806d6b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0df808e010 tx=0x7f0df001c8f0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2673981227 shutdown_connections 2024-09-19T04:16:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2673981227 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df80cf620 0x7f0df806d6b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2673981227 >> v2:172.21.15.116:3300/0 conn(0x7f0df8161a60 0x7f0df80d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2673981227 >> 172.21.15.76:0/2673981227 conn(0x7f0df808bf90 msgr2=0x7f0df809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2673981227 shutdown_connections 2024-09-19T04:16:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2673981227 wait complete. 2024-09-19T04:16:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 Processor -- start 2024-09-19T04:16:20.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- start start 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0df80cf620 0x7f0df80cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0df80d2e90 0x7f0df814d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df8161a60 0x7f0df81535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0df806e400 con 0x7f0df8161a60 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0df806d190 con 0x7f0df80cf620 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.352+0000 7f0dfe56b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0df806e580 con 0x7f0df80d2e90 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df8161a60 0x7f0df81535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df8161a60 0x7f0df81535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38402/0 (socket says 172.21.15.76:38402) 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2703997043 learned_addr learned my addr 172.21.15.76:0/2703997043 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df7fff640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0df80cf620 0x7f0df80cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df77fe640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0df80d2e90 0x7f0df814d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0df80d2e90 msgr2=0x7f0df814d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0df80d2e90 0x7f0df814d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0df80cf620 msgr2=0x7f0df80cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0df80cf620 0x7f0df80cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 -- 172.21.15.76:0/2703997043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df8152960 con 0x7f0df8161a60 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df7fff640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0df80cf620 0x7f0df80cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfcb08640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df8161a60 0x7f0df81535c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0df00af040 tx=0x7f0df0019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df4ff9640 1 -- 172.21.15.76:0/2703997043 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00b8670 con 0x7f0df8161a60 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df4ff9640 1 -- 172.21.15.76:0/2703997043 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0df00be070 con 0x7f0df8161a60 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df80cb980 con 0x7f0df8161a60 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df4ff9640 1 -- 172.21.15.76:0/2703997043 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df00b8a70 con 0x7f0df8161a60 2024-09-19T04:16:20.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df8152680 con 0x7f0df8161a60 2024-09-19T04:16:20.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df806d6b0 con 0x7f0df8161a60 2024-09-19T04:16:20.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df4ff9640 1 -- 172.21.15.76:0/2703997043 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0df00b8c10 con 0x7f0df8161a60 2024-09-19T04:16:20.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df4ff9640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0de40678c0 0x7f0de4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df4ff9640 1 -- 172.21.15.76:0/2703997043 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f0df0121060 con 0x7f0df8161a60 2024-09-19T04:16:20.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df7fff640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0de40678c0 0x7f0de4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.356+0000 7f0df7fff640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0de40678c0 0x7f0de4069d80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009840 tx=0x7f0de8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.360+0000 7f0df4ff9640 1 -- 172.21.15.76:0/2703997043 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0df00165a0 con 0x7f0df8161a60 2024-09-19T04:16:20.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.556+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0df80b67a0 con 0x7f0df8161a60 2024-09-19T04:16:20.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.556+0000 7f0df4ff9640 1 -- 172.21.15.76:0/2703997043 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5276 (secure 0 0 0) 0x7f0df00f1a70 con 0x7f0df8161a60 2024-09-19T04:16:20.561 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:20.561 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:16:08.863653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502,5496,4754]},"id":3}]} 2024-09-19T04:16:20.561 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 39 2024-09-19T04:16:20.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0de40678c0 msgr2=0x7f0de4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0de40678c0 0x7f0de4069d80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0de8009840 tx=0x7f0de8008040 comp rx=0 tx=0).stop 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df8161a60 msgr2=0x7f0df81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df8161a60 0x7f0df81535c0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0df00af040 tx=0x7f0df0019690 comp rx=0 tx=0).stop 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 shutdown_connections 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0de40678c0 0x7f0de4069d80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0df8161a60 0x7f0df81535c0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0df80d2e90 0x7f0df814d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 --2- 172.21.15.76:0/2703997043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0df80cf620 0x7f0df80cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 >> 172.21.15.76:0/2703997043 conn(0x7f0df808bf90 msgr2=0x7f0df80cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 shutdown_connections 2024-09-19T04:16:20.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.560+0000 7f0dfe56b640 1 -- 172.21.15.76:0/2703997043 wait complete. 2024-09-19T04:16:20.579 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:16:20.579 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:16:20.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef24550640 1 Processor -- start 2024-09-19T04:16:20.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef24550640 1 -- start start 2024-09-19T04:16:20.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef24550640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fef1c15bec0 0x7fef1c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef24550640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fef1c15c2a0 0x7fef1c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef24550640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef1c0b3290 con 0x7fef1c0af2a0 2024-09-19T04:16:20.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef24550640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fef1c0b3570 con 0x7fef1c15c2a0 2024-09-19T04:16:20.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef24550640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fef1c0b3850 con 0x7fef1c15bec0 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef21aeb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fef1c15c2a0 0x7fef1c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef22aed640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fef1c0af2a0 0x7fef1c15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:60886/0 (socket says 172.21.15.76:60886) 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef22aed640 1 -- 172.21.15.76:0/2685458316 learned_addr learned my addr 172.21.15.76:0/2685458316 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef21aeb640 1 -- 172.21.15.76:0/2685458316 >> v2:172.21.15.116:3300/0 conn(0x7fef1c15bec0 msgr2=0x7fef1c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef222ec640 1 --2- 172.21.15.76:0/2685458316 >> v2:172.21.15.116:3300/0 conn(0x7fef1c15bec0 0x7fef1c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef212ea640 1 -- 172.21.15.76:0/2685458316 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1133400239 0 0) 0x7fef1c0b3290 con 0x7fef1c0af2a0 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef21aeb640 1 --2- 172.21.15.76:0/2685458316 >> v2:172.21.15.116:3300/0 conn(0x7fef1c15bec0 0x7fef1c0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef21aeb640 1 -- 172.21.15.76:0/2685458316 >> v1:172.21.15.76:6789/0 conn(0x7fef1c0af2a0 legacy=0x7fef1c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.684+0000 7fef21aeb640 1 -- 172.21.15.76:0/2685458316 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c0b3b30 con 0x7fef1c15c2a0 2024-09-19T04:16:20.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef21aeb640 1 --2- 172.21.15.76:0/2685458316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c0aed60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fef1c071710 tx=0x7fef0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62f2017a70f0b8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef212ea640 1 -- 172.21.15.76:0/2685458316 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c0ab650 con 0x7fef1c15c2a0 2024-09-19T04:16:20.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef212ea640 1 -- 172.21.15.76:0/2685458316 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c0b0070 con 0x7fef1c15c2a0 2024-09-19T04:16:20.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef212ea640 1 -- 172.21.15.76:0/2685458316 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c0aba10 con 0x7fef1c15c2a0 2024-09-19T04:16:20.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 -- 172.21.15.76:0/2685458316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 msgr2=0x7fef1c0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 --2- 172.21.15.76:0/2685458316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c0aed60 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fef1c071710 tx=0x7fef0c01c450 comp rx=0 tx=0).stop 2024-09-19T04:16:20.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 -- 172.21.15.76:0/2685458316 shutdown_connections 2024-09-19T04:16:20.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 --2- 172.21.15.76:0/2685458316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c0aed60 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 --2- 172.21.15.76:0/2685458316 >> v2:172.21.15.116:3300/0 conn(0x7fef1c15bec0 0x7fef1c0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 -- 172.21.15.76:0/2685458316 >> 172.21.15.76:0/2685458316 conn(0x7fef1c0b1f90 msgr2=0x7fef1c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 -- 172.21.15.76:0/2685458316 shutdown_connections 2024-09-19T04:16:20.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 -- 172.21.15.76:0/2685458316 wait complete. 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 Processor -- start 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.688+0000 7fef24550640 1 -- start start 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef1c0af2a0 0x7fef1c1477a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c15bec0 0x7fef1c15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c1553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0af6d0 con 0x7fef1c15c2a0 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0ad740 con 0x7fef1c15bec0 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fef1c169000 con 0x7fef1c0af2a0 2024-09-19T04:16:20.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c1553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c1553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38410/0 (socket says 172.21.15.76:38410) 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 -- 172.21.15.76:0/1698343487 learned_addr learned my addr 172.21.15.76:0/1698343487 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef21aeb640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c15bec0 0x7fef1c15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 -- 172.21.15.76:0/1698343487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef1c0af2a0 msgr2=0x7fef1c1477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef1c0af2a0 0x7fef1c1477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 -- 172.21.15.76:0/1698343487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c15bec0 msgr2=0x7fef1c15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c15bec0 0x7fef1c15a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 -- 172.21.15.76:0/1698343487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c15aeb0 con 0x7fef1c15c2a0 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef22aed640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c1553f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c7d0 tx=0x7fef1400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef12ffd640 1 -- 172.21.15.76:0/1698343487 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef14018070 con 0x7fef1c15c2a0 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef12ffd640 1 -- 172.21.15.76:0/1698343487 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1400a640 con 0x7fef1c15c2a0 2024-09-19T04:16:20.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c15d430 con 0x7fef1c15c2a0 2024-09-19T04:16:20.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef12ffd640 1 -- 172.21.15.76:0/1698343487 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1400aa70 con 0x7fef1c15c2a0 2024-09-19T04:16:20.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c15d9f0 con 0x7fef1c15c2a0 2024-09-19T04:16:20.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1c0ad200 con 0x7fef1c15c2a0 2024-09-19T04:16:20.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef12ffd640 1 -- 172.21.15.76:0/1698343487 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fef1401b070 con 0x7fef1c15c2a0 2024-09-19T04:16:20.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef12ffd640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef000678f0 0x7fef00069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:20.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef12ffd640 1 -- 172.21.15.76:0/1698343487 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7fef140853a0 con 0x7fef1c15c2a0 2024-09-19T04:16:20.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef222ec640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef000678f0 0x7fef00069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:20.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.692+0000 7fef222ec640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef000678f0 0x7fef00069db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef1800a8b0 tx=0x7fef18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:20.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.696+0000 7fef12ffd640 1 -- 172.21.15.76:0/1698343487 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fef14055db0 con 0x7fef1c15c2a0 2024-09-19T04:16:20.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fef1c06fcd0 con 0x7fef1c15c2a0 2024-09-19T04:16:20.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef12ffd640 1 -- 172.21.15.76:0/1698343487 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1959 (secure 0 0 0) 0x7fef140556b0 con 0x7fef1c15c2a0 2024-09-19T04:16:20.916 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:20.916 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700801,"bytes_used":240963584,"bytes_avail":772853149696,"bytes_total":773094113280,"write_bytes_sec":691,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"b","status":"up:active","gid":4754},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5496},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5502}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":10,"modified":"2024-09-19T04:16:17.795220+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef000678f0 msgr2=0x7fef00069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef000678f0 0x7fef00069db0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fef1800a8b0 tx=0x7fef18008040 comp rx=0 tx=0).stop 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 msgr2=0x7fef1c1553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c1553f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fef1400c7d0 tx=0x7fef1400cc90 comp rx=0 tx=0).stop 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 shutdown_connections 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fef000678f0 0x7fef00069db0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fef1c15c2a0 0x7fef1c1553f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fef1c15bec0 0x7fef1c15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 --2- 172.21.15.76:0/1698343487 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fef1c0af2a0 0x7fef1c1477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.912+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 >> 172.21.15.76:0/1698343487 conn(0x7fef1c0b1f90 msgr2=0x7fef1c0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:20.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.916+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 shutdown_connections 2024-09-19T04:16:20.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:20.916+0000 7fef24550640 1 -- 172.21.15.76:0/1698343487 wait complete. 2024-09-19T04:16:20.933 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-19T04:16:21.934 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:22.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9759339640 1 Processor -- start 2024-09-19T04:16:22.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9759339640 1 -- start start 2024-09-19T04:16:22.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9759339640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9754161a60 0x7f97540d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9759339640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97540cf620 0x7f975406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9759339640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97540d32b0 con 0x7f97540d2e90 2024-09-19T04:16:22.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9759339640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f97540d0300 con 0x7f9754161a60 2024-09-19T04:16:22.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9759339640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f97540cfab0 con 0x7f97540cf620 2024-09-19T04:16:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f97537fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97540cf620 0x7f975406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f9752ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9754161a60 0x7f97540d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f97537fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97540cf620 0x7f975406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:33964/0 (socket says 172.21.15.76:33964) 2024-09-19T04:16:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f97537fe640 1 -- 172.21.15.76:0/2636656478 learned_addr learned my addr 172.21.15.76:0/2636656478 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f97537fe640 1 -- 172.21.15.76:0/2636656478 >> v2:172.21.15.116:3300/0 conn(0x7f9754161a60 msgr2=0x7f97540d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f97537fe640 1 --2- 172.21.15.76:0/2636656478 >> v2:172.21.15.116:3300/0 conn(0x7f9754161a60 0x7f97540d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f97537fe640 1 -- 172.21.15.76:0/2636656478 >> v1:172.21.15.116:6789/0 conn(0x7f97540d2e90 legacy=0x7f97540cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.064+0000 7f97537fe640 1 -- 172.21.15.76:0/2636656478 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97540b0f60 con 0x7f97540cf620 2024-09-19T04:16:22.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f97537fe640 1 --2- 172.21.15.76:0/2636656478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97540cf620 0x7f975406d6b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f974c009e50 tx=0x7f974c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ceec6362c739eff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9751ffb640 1 -- 172.21.15.76:0/2636656478 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0a5830 con 0x7f97540cf620 2024-09-19T04:16:22.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9751ffb640 1 -- 172.21.15.76:0/2636656478 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f974c0a59d0 con 0x7f97540cf620 2024-09-19T04:16:22.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9751ffb640 1 -- 172.21.15.76:0/2636656478 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974c0a5da0 con 0x7f97540cf620 2024-09-19T04:16:22.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- 172.21.15.76:0/2636656478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97540cf620 msgr2=0x7f975406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 --2- 172.21.15.76:0/2636656478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97540cf620 0x7f975406d6b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f974c009e50 tx=0x7f974c01c970 comp rx=0 tx=0).stop 2024-09-19T04:16:22.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- 172.21.15.76:0/2636656478 shutdown_connections 2024-09-19T04:16:22.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 --2- 172.21.15.76:0/2636656478 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f97540cf620 0x7f975406d6b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 --2- 172.21.15.76:0/2636656478 >> v2:172.21.15.116:3300/0 conn(0x7f9754161a60 0x7f97540d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- 172.21.15.76:0/2636656478 >> 172.21.15.76:0/2636656478 conn(0x7f975408bf90 msgr2=0x7f975409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:22.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- 172.21.15.76:0/2636656478 shutdown_connections 2024-09-19T04:16:22.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- 172.21.15.76:0/2636656478 wait complete. 2024-09-19T04:16:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 Processor -- start 2024-09-19T04:16:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- start start 2024-09-19T04:16:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97540cf620 0x7f97541566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97540d2e90 0x7f97540cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9754161a60 0x7f97540cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f975406e400 con 0x7f97540cf620 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f975406d190 con 0x7f97540d2e90 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9759339640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f975406e580 con 0x7f9754161a60 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9752ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97540cf620 0x7f97541566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9752ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97540cf620 0x7f97541566f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38420/0 (socket says 172.21.15.76:38420) 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.068+0000 7f9752ffd640 1 -- 172.21.15.76:0/1749514097 learned_addr learned my addr 172.21.15.76:0/1749514097 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f97537fe640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9754161a60 0x7f97540cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f97527fc640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97540d2e90 0x7f97540cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9752ffd640 1 -- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9754161a60 msgr2=0x7f97540cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9752ffd640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9754161a60 0x7f97540cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9752ffd640 1 -- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97540d2e90 msgr2=0x7f97540cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9752ffd640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97540d2e90 0x7f97540cb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9752ffd640 1 -- 172.21.15.76:0/1749514097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975415b5a0 con 0x7f97540cf620 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f97537fe640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9754161a60 0x7f97540cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9752ffd640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97540cf620 0x7f97541566f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f973c00e870 tx=0x7f973c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f974b7fe640 1 -- 172.21.15.76:0/1749514097 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c017070 con 0x7f97540cf620 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975415b880 con 0x7f97540cf620 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f974b7fe640 1 -- 172.21.15.76:0/1749514097 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c00c940 con 0x7f97540cf620 2024-09-19T04:16:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f974b7fe640 1 -- 172.21.15.76:0/1749514097 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c00cd10 con 0x7f97540cf620 2024-09-19T04:16:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f975415be40 con 0x7f97540cf620 2024-09-19T04:16:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975406d6b0 con 0x7f97540cf620 2024-09-19T04:16:22.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f974b7fe640 1 -- 172.21.15.76:0/1749514097 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f973c01a070 con 0x7f97540cf620 2024-09-19T04:16:22.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f974b7fe640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97340678f0 0x7f9734069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f97527fc640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97340678f0 0x7f9734069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f974b7fe640 1 -- 172.21.15.76:0/1749514097 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f973c0860f0 con 0x7f97540cf620 2024-09-19T04:16:22.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.072+0000 7f97527fc640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97340678f0 0x7f9734069db0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97540ccea0 tx=0x7f9744009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.076+0000 7f974b7fe640 1 -- 172.21.15.76:0/1749514097 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f973c056a80 con 0x7f97540cf620 2024-09-19T04:16:22.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.260+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97540d38a0 con 0x7f97540cf620 2024-09-19T04:16:22.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.260+0000 7f974b7fe640 1 -- 172.21.15.76:0/1749514097 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5276 (secure 0 0 0) 0x7f973c056380 con 0x7f97540cf620 2024-09-19T04:16:22.264 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:22.264 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:16:08.863653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502,5496,4754]},"id":3}]} 2024-09-19T04:16:22.264 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 39 2024-09-19T04:16:22.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97340678f0 msgr2=0x7f9734069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97340678f0 0x7f9734069db0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f97540ccea0 tx=0x7f9744009040 comp rx=0 tx=0).stop 2024-09-19T04:16:22.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97540cf620 msgr2=0x7f97541566f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97540cf620 0x7f97541566f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f973c00e870 tx=0x7f973c00ed30 comp rx=0 tx=0).stop 2024-09-19T04:16:22.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 shutdown_connections 2024-09-19T04:16:22.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f97340678f0 0x7f9734069db0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9754161a60 0x7f97540cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97540d2e90 0x7f97540cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 --2- 172.21.15.76:0/1749514097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f97540cf620 0x7f97541566f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 >> 172.21.15.76:0/1749514097 conn(0x7f975408bf90 msgr2=0x7f975415d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:22.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 shutdown_connections 2024-09-19T04:16:22.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.264+0000 7f9759339640 1 -- 172.21.15.76:0/1749514097 wait complete. 2024-09-19T04:16:22.281 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:16:22.281 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:16:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 Processor -- start 2024-09-19T04:16:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 -- start start 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe3801520a0 0x7fe38006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe38006fe50 0x7fe38015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe38015f1d0 0x7fe380071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe380152480 con 0x7fe38006fe50 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe380070250 con 0x7fe3801520a0 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe38558b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe3800aede0 con 0x7fe38015f1d0 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe37e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe38006fe50 0x7fe38015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe37e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe38006fe50 0x7fe38015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38422/0 (socket says 172.21.15.76:38422) 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.392+0000 7fe37e7fc640 1 -- 172.21.15.76:0/3074405184 learned_addr learned my addr 172.21.15.76:0/3074405184 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:22.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37f7fe640 1 --2- 172.21.15.76:0/3074405184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe38015f1d0 0x7fe380071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37effd640 1 --2- 172.21.15.76:0/3074405184 >> v2:172.21.15.116:3300/0 conn(0x7fe3801520a0 0x7fe38006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37e7fc640 1 -- 172.21.15.76:0/3074405184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe38015f1d0 msgr2=0x7fe380071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:22.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37e7fc640 1 --2- 172.21.15.76:0/3074405184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe38015f1d0 0x7fe380071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37e7fc640 1 -- 172.21.15.76:0/3074405184 >> v2:172.21.15.116:3300/0 conn(0x7fe3801520a0 msgr2=0x7fe38006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:22.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37e7fc640 1 --2- 172.21.15.76:0/3074405184 >> v2:172.21.15.116:3300/0 conn(0x7fe3801520a0 0x7fe38006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37e7fc640 1 -- 172.21.15.76:0/3074405184 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38015fa10 con 0x7fe38006fe50 2024-09-19T04:16:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37e7fc640 1 --2- 172.21.15.76:0/3074405184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38015ec90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe380071710 tx=0x7fe36801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c2db6b2c9efa1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37dffb640 1 -- 172.21.15.76:0/3074405184 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680b0070 con 0x7fe38006fe50 2024-09-19T04:16:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37dffb640 1 -- 172.21.15.76:0/3074405184 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3680ab600 con 0x7fe38006fe50 2024-09-19T04:16:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe37dffb640 1 -- 172.21.15.76:0/3074405184 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680ab9d0 con 0x7fe38006fe50 2024-09-19T04:16:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 -- 172.21.15.76:0/3074405184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 msgr2=0x7fe38015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 --2- 172.21.15.76:0/3074405184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38015ec90 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe380071710 tx=0x7fe36801c240 comp rx=0 tx=0).stop 2024-09-19T04:16:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 -- 172.21.15.76:0/3074405184 shutdown_connections 2024-09-19T04:16:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 --2- 172.21.15.76:0/3074405184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe38015f1d0 0x7fe380071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 --2- 172.21.15.76:0/3074405184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38015ec90 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 --2- 172.21.15.76:0/3074405184 >> v2:172.21.15.116:3300/0 conn(0x7fe3801520a0 0x7fe38006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 -- 172.21.15.76:0/3074405184 >> 172.21.15.76:0/3074405184 conn(0x7fe3800b1f90 msgr2=0x7fe3800b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 -- 172.21.15.76:0/3074405184 shutdown_connections 2024-09-19T04:16:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.396+0000 7fe38558b640 1 -- 172.21.15.76:0/3074405184 wait complete. 2024-09-19T04:16:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 Processor -- start 2024-09-19T04:16:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 -- start start 2024-09-19T04:16:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38014de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe3801520a0 0x7fe38014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38015f1d0 0x7fe38014e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe380073d30 con 0x7fe38006fe50 2024-09-19T04:16:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe380073a50 con 0x7fe38015f1d0 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe38015f5e0 con 0x7fe3801520a0 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38014de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38014de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:38432/0 (socket says 172.21.15.76:38432) 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 -- 172.21.15.76:0/2224633949 learned_addr learned my addr 172.21.15.76:0/2224633949 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37f7fe640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38015f1d0 0x7fe38014e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37e7fc640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe3801520a0 0x7fe38014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 -- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe3801520a0 msgr2=0x7fe38014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe3801520a0 0x7fe38014e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 -- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38015f1d0 msgr2=0x7fe38014e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38015f1d0 0x7fe38014e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 -- 172.21.15.76:0/2224633949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe38015aa80 con 0x7fe38006fe50 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe37effd640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38014de80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe378007e90 tx=0x7fe37800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe3777fe640 1 -- 172.21.15.76:0/2224633949 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378014620 con 0x7fe38006fe50 2024-09-19T04:16:22.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe3777fe640 1 -- 172.21.15.76:0/2224633949 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe378019070 con 0x7fe38006fe50 2024-09-19T04:16:22.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe380164410 con 0x7fe38006fe50 2024-09-19T04:16:22.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe3777fe640 1 -- 172.21.15.76:0/2224633949 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe378014a00 con 0x7fe38006fe50 2024-09-19T04:16:22.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe38015dbc0 con 0x7fe38006fe50 2024-09-19T04:16:22.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38006f910 con 0x7fe38006fe50 2024-09-19T04:16:22.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.400+0000 7fe3777fe640 1 -- 172.21.15.76:0/2224633949 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe378014ba0 con 0x7fe38006fe50 2024-09-19T04:16:22.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.404+0000 7fe3777fe640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe3600678c0 0x7fe360069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.404+0000 7fe3777fe640 1 -- 172.21.15.76:0/2224633949 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7fe378086730 con 0x7fe38006fe50 2024-09-19T04:16:22.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.404+0000 7fe37e7fc640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe3600678c0 0x7fe360069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.404+0000 7fe37e7fc640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe3600678c0 0x7fe360069d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3680ad000 tx=0x7fe368008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.408+0000 7fe3777fe640 1 -- 172.21.15.76:0/2224633949 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe3780570b0 con 0x7fe38006fe50 2024-09-19T04:16:22.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.628+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe38006e540 con 0x7fe38006fe50 2024-09-19T04:16:22.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.628+0000 7fe3777fe640 1 -- 172.21.15.76:0/2224633949 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1865 (secure 0 0 0) 0x7fe3780569b0 con 0x7fe38006fe50 2024-09-19T04:16:22.633 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:22.633 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":201,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700801,"bytes_used":240963584,"bytes_avail":772853149696,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","id":3,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":3,"rank":2,"name":"b","status":"up:active","gid":4754},{"filesystem_id":3,"rank":1,"name":"c","status":"up:active","gid":5496},{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5502}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2024-09-19T04:16:21.796508+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe3600678c0 msgr2=0x7fe360069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe3600678c0 0x7fe360069d80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3680ad000 tx=0x7fe368008040 comp rx=0 tx=0).stop 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 msgr2=0x7fe38014de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38014de80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe378007e90 tx=0x7fe37800e4b0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 shutdown_connections 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe3600678c0 0x7fe360069d80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe38015f1d0 0x7fe38014e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe3801520a0 0x7fe38014e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 --2- 172.21.15.76:0/2224633949 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe38006fe50 0x7fe38014de80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 >> 172.21.15.76:0/2224633949 conn(0x7fe3800b1f90 msgr2=0x7fe3801507d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:22.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 shutdown_connections 2024-09-19T04:16:22.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.632+0000 7fe38558b640 1 -- 172.21.15.76:0/2224633949 wait complete. 2024-09-19T04:16:22.652 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-19T04:16:22.652 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:16:22.652 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:16:22.727 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:16:22.800 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:16:22.800 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:16:22.800 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:16:22.801 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:22.807 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:16:22.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.972+0000 7f4f2544c640 1 Processor -- start 2024-09-19T04:16:22.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.972+0000 7f4f2544c640 1 -- start start 2024-09-19T04:16:22.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.972+0000 7f4f2544c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f20070020 0x7f4f2015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.972+0000 7f4f2544c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4f2015f1d0 0x7f4f200adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.972+0000 7f4f2544c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f200aede0 con 0x7f4f201520a0 2024-09-19T04:16:22.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.972+0000 7f4f2544c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4f20152480 con 0x7f4f2015f1d0 2024-09-19T04:16:22.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.972+0000 7f4f2544c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2015fa10 con 0x7f4f20070020 2024-09-19T04:16:22.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1f7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4f2015f1d0 0x7f4f200adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f20070020 0x7f4f2015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1effd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4f201520a0 0x7f4f2006f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:43558/0 (socket says 172.21.15.76:43558) 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f17fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f20070020 0x7f4f2015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:54082/0 (socket says 172.21.15.76:54082) 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1f7fe640 1 --2- 172.21.15.76:0/1037112834 >> v2:172.21.15.116:3300/0 conn(0x7f4f2015f1d0 0x7f4f200adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:55974/0 (socket says 172.21.15.76:55974) 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1effd640 1 -- 172.21.15.76:0/1037112834 learned_addr learned my addr 172.21.15.76:0/1037112834 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1e7fc640 1 -- 172.21.15.76:0/1037112834 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1034344782 0 0) 0x7f4f200aede0 con 0x7f4f201520a0 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1e7fc640 1 -- 172.21.15.76:0/1037112834 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f0c003530 con 0x7f4f201520a0 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f17fff640 1 -- 172.21.15.76:0/1037112834 >> v2:172.21.15.116:3300/0 conn(0x7f4f2015f1d0 msgr2=0x7f4f200adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f17fff640 1 --2- 172.21.15.76:0/1037112834 >> v2:172.21.15.116:3300/0 conn(0x7f4f2015f1d0 0x7f4f200adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f17fff640 1 -- 172.21.15.76:0/1037112834 >> v1:172.21.15.116:6789/0 conn(0x7f4f201520a0 legacy=0x7f4f2006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f17fff640 1 -- 172.21.15.76:0/1037112834 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f20074250 con 0x7f4f20070020 2024-09-19T04:16:22.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1f7fe640 1 --2- 172.21.15.76:0/1037112834 >> v2:172.21.15.116:3300/0 conn(0x7f4f2015f1d0 0x7f4f200adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:16:22.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f17fff640 1 --2- 172.21.15.76:0/1037112834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f20070020 0x7f4f2015ec90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097f0 tx=0x7f4f0801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6db10e24cf6cbe75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.976+0000 7f4f1e7fc640 1 -- 172.21.15.76:0/1037112834 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080a7070 con 0x7f4f20070020 2024-09-19T04:16:22.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f1e7fc640 1 -- 172.21.15.76:0/1037112834 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f080b0070 con 0x7f4f20070020 2024-09-19T04:16:22.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f1e7fc640 1 -- 172.21.15.76:0/1037112834 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f080ab870 con 0x7f4f20070020 2024-09-19T04:16:22.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- 172.21.15.76:0/1037112834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f20070020 msgr2=0x7f4f2015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1037112834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f20070020 0x7f4f2015ec90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f080097f0 tx=0x7f4f0801c470 comp rx=0 tx=0).stop 2024-09-19T04:16:22.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- 172.21.15.76:0/1037112834 shutdown_connections 2024-09-19T04:16:22.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1037112834 >> v2:172.21.15.116:3300/0 conn(0x7f4f2015f1d0 0x7f4f200adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1037112834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f20070020 0x7f4f2015ec90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- 172.21.15.76:0/1037112834 >> 172.21.15.76:0/1037112834 conn(0x7f4f200b1f90 msgr2=0x7f4f200b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:22.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- 172.21.15.76:0/1037112834 shutdown_connections 2024-09-19T04:16:22.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- 172.21.15.76:0/1037112834 wait complete. 2024-09-19T04:16:22.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 Processor -- start 2024-09-19T04:16:22.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- start start 2024-09-19T04:16:22.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f20070020 0x7f4f200998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f201520a0 0x7f4f200cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2015f1d0 0x7f4f200cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2015f5e0 con 0x7f4f20070020 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20073d30 con 0x7f4f2015f1d0 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f2544c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4f2015f760 con 0x7f4f201520a0 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f1effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f20070020 0x7f4f200998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f1effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f20070020 0x7f4f200998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51182/0 (socket says 172.21.15.76:51182) 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f1effd640 1 -- 172.21.15.76:0/1092423873 learned_addr learned my addr 172.21.15.76:0/1092423873 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f17fff640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f201520a0 0x7f4f200cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.980+0000 7f4f1f7fe640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2015f1d0 0x7f4f200cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f1effd640 1 -- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f201520a0 msgr2=0x7f4f200cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f1effd640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f201520a0 0x7f4f200cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f1effd640 1 -- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2015f1d0 msgr2=0x7f4f200cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f1effd640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2015f1d0 0x7f4f200cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:22.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f1effd640 1 -- 172.21.15.76:0/1092423873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f200cf250 con 0x7f4f20070020 2024-09-19T04:16:22.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f1effd640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f20070020 0x7f4f200998d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c500 tx=0x7f4f1000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f177fe640 1 -- 172.21.15.76:0/1092423873 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f10017070 con 0x7f4f20070020 2024-09-19T04:16:22.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f177fe640 1 -- 172.21.15.76:0/1092423873 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f10005cf0 con 0x7f4f20070020 2024-09-19T04:16:22.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f200cf8f0 con 0x7f4f20070020 2024-09-19T04:16:22.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f177fe640 1 -- 172.21.15.76:0/1092423873 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f100072f0 con 0x7f4f20070020 2024-09-19T04:16:22.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f200cea50 con 0x7f4f20070020 2024-09-19T04:16:22.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2015ec90 con 0x7f4f20070020 2024-09-19T04:16:22.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f177fe640 1 -- 172.21.15.76:0/1092423873 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4f1001a070 con 0x7f4f20070020 2024-09-19T04:16:22.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f177fe640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f000678c0 0x7f4f00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:22.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f177fe640 1 -- 172.21.15.76:0/1092423873 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f4f10085580 con 0x7f4f20070020 2024-09-19T04:16:22.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f17fff640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f000678c0 0x7f4f00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:22.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.984+0000 7f4f17fff640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f000678c0 0x7f4f00069d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4f080af000 tx=0x7f4f080b6040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:22.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:22.988+0000 7f4f177fe640 1 -- 172.21.15.76:0/1092423873 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4f200cf8f0 con 0x7f4f20070020 2024-09-19T04:16:23.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.148+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4f200d0030 con 0x7f4f20070020 2024-09-19T04:16:23.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f177fe640 1 -- 172.21.15.76:0/1092423873 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v55) ==== 80+0+0 (secure 0 0 0) 0x7f4f10055a20 con 0x7f4f20070020 2024-09-19T04:16:23.154 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:16:23.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f000678c0 msgr2=0x7f4f00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f000678c0 0x7f4f00069d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4f080af000 tx=0x7f4f080b6040 comp rx=0 tx=0).stop 2024-09-19T04:16:23.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f20070020 msgr2=0x7f4f200998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f20070020 0x7f4f200998d0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000c500 tx=0x7f4f1000c9c0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 shutdown_connections 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f000678c0 0x7f4f00069d80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f2015f1d0 0x7f4f200cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f201520a0 0x7f4f200cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 --2- 172.21.15.76:0/1092423873 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f20070020 0x7f4f200998d0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 >> 172.21.15.76:0/1092423873 conn(0x7f4f200b1f90 msgr2=0x7f4f200b3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 shutdown_connections 2024-09-19T04:16:23.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.152+0000 7f4f2544c640 1 -- 172.21.15.76:0/1092423873 wait complete. 2024-09-19T04:16:23.171 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:16:23.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:23.172 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:23.182 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[40200]: fuse finished with error 0 and tester_r 0 2024-09-19T04:16:23.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:16:23.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:23.183 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:16:23.282 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:16:23.283 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:16:23.283 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:23.289 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:16:23.289 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:16:23.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:23.290 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T04:16:23.334 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T04:16:23.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:16:23.336 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:16:23.336 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:16:23.336 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:23.341 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:16:23.341 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:16:23.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:23.341 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T04:16:23.390 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T04:16:23.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:16:23.391 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:16:23.391 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:16:23.391 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:23.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:16:23.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:16:23.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:23.398 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T04:16:23.447 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T04:16:23.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:16:23.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:23.449 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:16:23.452 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:16:23.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.568+0000 7f8c1b430640 1 Processor -- start 2024-09-19T04:16:23.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.568+0000 7f8c1b430640 1 -- start start 2024-09-19T04:16:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c1b430640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8c14161a60 0x7f8c140d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c1b430640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140d2e90 0x7f8c140cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c1b430640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c140d32b0 con 0x7f8c140cf450 2024-09-19T04:16:23.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c1b430640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8c140d0300 con 0x7f8c14161a60 2024-09-19T04:16:23.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c1b430640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1406e6c0 con 0x7f8c140d2e90 2024-09-19T04:16:23.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c191cc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8c14161a60 0x7f8c140d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c189cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140d2e90 0x7f8c140cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c189cb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140d2e90 0x7f8c140cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:54102/0 (socket says 172.21.15.76:54102) 2024-09-19T04:16:23.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c189cb640 1 -- 172.21.15.76:0/1902844524 learned_addr learned my addr 172.21.15.76:0/1902844524 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:23.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c189cb640 1 -- 172.21.15.76:0/1902844524 >> v2:172.21.15.116:3300/0 conn(0x7f8c14161a60 msgr2=0x7f8c140d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:23.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c189cb640 1 --2- 172.21.15.76:0/1902844524 >> v2:172.21.15.116:3300/0 conn(0x7f8c14161a60 0x7f8c140d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c189cb640 1 -- 172.21.15.76:0/1902844524 >> v1:172.21.15.116:6789/0 conn(0x7f8c140cf450 legacy=0x7f8c140d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.572+0000 7f8c189cb640 1 -- 172.21.15.76:0/1902844524 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c140b0f60 con 0x7f8c140d2e90 2024-09-19T04:16:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c189cb640 1 --2- 172.21.15.76:0/1902844524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140d2e90 0x7f8c140cef10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009870 tx=0x7f8c0401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78be65f055354d1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c0bfff640 1 -- 172.21.15.76:0/1902844524 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c140d1930 con 0x7f8c140d2e90 2024-09-19T04:16:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c0bfff640 1 -- 172.21.15.76:0/1902844524 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c040af070 con 0x7f8c140d2e90 2024-09-19T04:16:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c0bfff640 1 -- 172.21.15.76:0/1902844524 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c040aa820 con 0x7f8c140d2e90 2024-09-19T04:16:23.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- 172.21.15.76:0/1902844524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140d2e90 msgr2=0x7f8c140cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 --2- 172.21.15.76:0/1902844524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140d2e90 0x7f8c140cef10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009870 tx=0x7f8c0401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- 172.21.15.76:0/1902844524 shutdown_connections 2024-09-19T04:16:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 --2- 172.21.15.76:0/1902844524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140d2e90 0x7f8c140cef10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 --2- 172.21.15.76:0/1902844524 >> v2:172.21.15.116:3300/0 conn(0x7f8c14161a60 0x7f8c140d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- 172.21.15.76:0/1902844524 >> 172.21.15.76:0/1902844524 conn(0x7f8c1408bf90 msgr2=0x7f8c1409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- 172.21.15.76:0/1902844524 shutdown_connections 2024-09-19T04:16:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- 172.21.15.76:0/1902844524 wait complete. 2024-09-19T04:16:23.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 Processor -- start 2024-09-19T04:16:23.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- start start 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140cf450 0x7f8c140b7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c140d2e90 0x7f8c14158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c14161a60 0x7f8c14158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1416af20 con 0x7f8c14161a60 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c1406d190 con 0x7f8c140d2e90 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c1b430640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8c1416b0a0 con 0x7f8c140cf450 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c199cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c14161a60 0x7f8c14158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c199cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c14161a60 0x7f8c14158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51196/0 (socket says 172.21.15.76:51196) 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c199cd640 1 -- 172.21.15.76:0/452731781 learned_addr learned my addr 172.21.15.76:0/452731781 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.576+0000 7f8c191cc640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140cf450 0x7f8c140b7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c189cb640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c140d2e90 0x7f8c14158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c199cd640 1 -- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140cf450 msgr2=0x7f8c140b7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c199cd640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140cf450 0x7f8c140b7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c199cd640 1 -- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c140d2e90 msgr2=0x7f8c14158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c199cd640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c140d2e90 0x7f8c14158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c199cd640 1 -- 172.21.15.76:0/452731781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c140b8520 con 0x7f8c14161a60 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c189cb640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c140d2e90 0x7f8c14158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c199cd640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c14161a60 0x7f8c14158aa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00c770 tx=0x7f8c0c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c09ffb640 1 -- 172.21.15.76:0/452731781 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c003b80 con 0x7f8c14161a60 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c09ffb640 1 -- 172.21.15.76:0/452731781 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c0c003d20 con 0x7f8c14161a60 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c14151d20 con 0x7f8c14161a60 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c09ffb640 1 -- 172.21.15.76:0/452731781 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00aa30 con 0x7f8c14161a60 2024-09-19T04:16:23.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c141522e0 con 0x7f8c14161a60 2024-09-19T04:16:23.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c140cef10 con 0x7f8c14161a60 2024-09-19T04:16:23.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c09ffb640 1 -- 172.21.15.76:0/452731781 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c0c00abd0 con 0x7f8c14161a60 2024-09-19T04:16:23.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c09ffb640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bf80678f0 0x7f8bf8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c09ffb640 1 -- 172.21.15.76:0/452731781 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f8c0c085c10 con 0x7f8c14161a60 2024-09-19T04:16:23.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c191cc640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bf80678f0 0x7f8bf8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.580+0000 7f8c191cc640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bf80678f0 0x7f8bf8069db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c1415d810 tx=0x7f8c10009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:23.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.584+0000 7f8c09ffb640 1 -- 172.21.15.76:0/452731781 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8c0c056620 con 0x7f8c14161a60 2024-09-19T04:16:23.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.768+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c1406db60 con 0x7f8c14161a60 2024-09-19T04:16:23.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c09ffb640 1 -- 172.21.15.76:0/452731781 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5276 (secure 0 0 0) 0x7f8c0c055f20 con 0x7f8c14161a60 2024-09-19T04:16:23.774 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:23.774 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:16:08.863653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502,5496,4754]},"id":3}]} 2024-09-19T04:16:23.774 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 39 2024-09-19T04:16:23.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bf80678f0 msgr2=0x7f8bf8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bf80678f0 0x7f8bf8069db0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8c1415d810 tx=0x7f8c10009190 comp rx=0 tx=0).stop 2024-09-19T04:16:23.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c14161a60 msgr2=0x7f8c14158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c14161a60 0x7f8c14158aa0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00c770 tx=0x7f8c0c00cc30 comp rx=0 tx=0).stop 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 shutdown_connections 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bf80678f0 0x7f8bf8069db0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c14161a60 0x7f8c14158aa0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c140d2e90 0x7f8c14158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 --2- 172.21.15.76:0/452731781 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c140cf450 0x7f8c140b7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 >> 172.21.15.76:0/452731781 conn(0x7f8c1408bf90 msgr2=0x7f8c140d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 shutdown_connections 2024-09-19T04:16:23.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.772+0000 7f8c1b430640 1 -- 172.21.15.76:0/452731781 wait complete. 2024-09-19T04:16:23.792 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:23.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a83a76640 1 Processor -- start 2024-09-19T04:16:23.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a83a76640 1 -- start start 2024-09-19T04:16:23.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a83a76640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c145600 0x7f2a7c145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a83a76640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2a7c0b15b0 0x7f2a7c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a83a76640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a7c0b0510 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a83a76640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2a7c146020 con 0x7f2a7c0b15b0 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a83a76640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b07f0 con 0x7f2a7c145600 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a82013640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2a7c0b19f0 0x7f2a7c15c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36598/0 (socket says 172.21.15.76:36598) 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a82013640 1 -- 172.21.15.76:0/2841674540 learned_addr learned my addr 172.21.15.76:0/2841674540 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a81011640 1 --2- 172.21.15.76:0/2841674540 >> v2:172.21.15.76:3300/0 conn(0x7f2a7c0b15b0 0x7f2a7c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a81812640 1 --2- 172.21.15.76:0/2841674540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c145600 0x7f2a7c145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a80810640 1 -- 172.21.15.76:0/2841674540 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 496317957 0 0) 0x7f2a7c0b0510 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a80810640 1 -- 172.21.15.76:0/2841674540 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a64003530 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a81011640 1 -- 172.21.15.76:0/2841674540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c145600 msgr2=0x7f2a7c145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:23.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a81011640 1 --2- 172.21.15.76:0/2841674540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c145600 0x7f2a7c145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a81011640 1 -- 172.21.15.76:0/2841674540 >> v1:172.21.15.76:6789/0 conn(0x7f2a7c0b19f0 legacy=0x7f2a7c15c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.948+0000 7f2a81011640 1 -- 172.21.15.76:0/2841674540 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c0b0ad0 con 0x7f2a7c0b15b0 2024-09-19T04:16:23.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a81011640 1 --2- 172.21.15.76:0/2841674540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b15b0 0x7f2a7c0b2030 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2a700097c0 tx=0x7f2a7001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950915b13abc7e3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:23.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a80810640 1 -- 172.21.15.76:0/2841674540 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b2070 con 0x7f2a7c0b15b0 2024-09-19T04:16:23.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a80810640 1 -- 172.21.15.76:0/2841674540 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a700ad680 con 0x7f2a7c0b15b0 2024-09-19T04:16:23.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a80810640 1 -- 172.21.15.76:0/2841674540 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700ada50 con 0x7f2a7c0b15b0 2024-09-19T04:16:23.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- 172.21.15.76:0/2841674540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b15b0 msgr2=0x7f2a7c0b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 --2- 172.21.15.76:0/2841674540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b15b0 0x7f2a7c0b2030 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f2a700097c0 tx=0x7f2a7001c470 comp rx=0 tx=0).stop 2024-09-19T04:16:23.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- 172.21.15.76:0/2841674540 shutdown_connections 2024-09-19T04:16:23.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 --2- 172.21.15.76:0/2841674540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b15b0 0x7f2a7c0b2030 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 --2- 172.21.15.76:0/2841674540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c145600 0x7f2a7c145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- 172.21.15.76:0/2841674540 >> 172.21.15.76:0/2841674540 conn(0x7f2a7c09aa00 msgr2=0x7f2a7c0b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:23.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- 172.21.15.76:0/2841674540 shutdown_connections 2024-09-19T04:16:23.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- 172.21.15.76:0/2841674540 wait complete. 2024-09-19T04:16:23.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 Processor -- start 2024-09-19T04:16:23.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- start start 2024-09-19T04:16:23.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c0b15b0 0x7f2a7c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b19f0 0x7f2a7c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a7c145600 0x7f2a7c158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b6f20 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b2980 con 0x7f2a7c145600 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a83a76640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2a7c0b70a0 con 0x7f2a7c0b15b0 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a81011640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b19f0 0x7f2a7c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a81011640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b19f0 0x7f2a7c158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51214/0 (socket says 172.21.15.76:51214) 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a81011640 1 -- 172.21.15.76:0/1844499893 learned_addr learned my addr 172.21.15.76:0/1844499893 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.952+0000 7f2a81812640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c0b15b0 0x7f2a7c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a82013640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a7c145600 0x7f2a7c158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81011640 1 -- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c0b15b0 msgr2=0x7f2a7c15a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81011640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c0b15b0 0x7f2a7c15a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81011640 1 -- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a7c145600 msgr2=0x7f2a7c158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81011640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a7c145600 0x7f2a7c158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81011640 1 -- 172.21.15.76:0/1844499893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c1594b0 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81011640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b19f0 0x7f2a7c158a30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001a0a0 tx=0x7f2a7001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a7a7fc640 1 -- 172.21.15.76:0/1844499893 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700bd070 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a7a7fc640 1 -- 172.21.15.76:0/1844499893 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a700adb30 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c15a8b0 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a7a7fc640 1 -- 172.21.15.76:0/1844499893 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700b2040 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c159720 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a7c0b2030 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a7a7fc640 1 -- 172.21.15.76:0/1844499893 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2a700c0070 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a7a7fc640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2a680678c0 0x7f2a68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:23.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a7a7fc640 1 -- 172.21.15.76:0/1844499893 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f2a7011f3a0 con 0x7f2a7c0b19f0 2024-09-19T04:16:23.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81812640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2a680678c0 0x7f2a68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:23.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.956+0000 7f2a81812640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2a680678c0 0x7f2a68069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a740073f0 tx=0x7f2a7400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:23.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:23.960+0000 7f2a7a7fc640 1 -- 172.21.15.76:0/1844499893 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2a70016090 con 0x7f2a7c0b19f0 2024-09-19T04:16:24.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.156+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a7c0753e0 con 0x7f2a7c0b19f0 2024-09-19T04:16:24.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a7a7fc640 1 -- 172.21.15.76:0/1844499893 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5276 (secure 0 0 0) 0x7f2a700ef760 con 0x7f2a7c0b19f0 2024-09-19T04:16:24.162 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:24.162 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:16:08.863653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502,5496,4754]},"id":3}]} 2024-09-19T04:16:24.162 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 39 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2a680678c0 msgr2=0x7f2a68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2a680678c0 0x7f2a68069d80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f2a740073f0 tx=0x7f2a7400b040 comp rx=0 tx=0).stop 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b19f0 msgr2=0x7f2a7c158a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b19f0 0x7f2a7c158a30 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f2a7001a0a0 tx=0x7f2a7001acb0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 shutdown_connections 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2a680678c0 0x7f2a68069d80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a7c145600 0x7f2a7c158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a7c0b19f0 0x7f2a7c158a30 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 --2- 172.21.15.76:0/1844499893 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a7c0b15b0 0x7f2a7c15a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 >> 172.21.15.76:0/1844499893 conn(0x7f2a7c09aa00 msgr2=0x7f2a7c06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 shutdown_connections 2024-09-19T04:16:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.160+0000 7f2a83a76640 1 -- 172.21.15.76:0/1844499893 wait complete. 2024-09-19T04:16:24.181 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:16:24.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1ad66ae640 1 Processor -- start 2024-09-19T04:16:24.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1ad66ae640 1 -- start start 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1ad66ae640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ad015bec0 0x7f1ad00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1ad66ae640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad00b3290 con 0x7f1ad00af470 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1ad66ae640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ad00b3570 con 0x7f1ad015c2a0 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1ad66ae640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1ad00b3850 con 0x7f1ad015bec0 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1acffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ad015bec0 0x7f1ad00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1acffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ad015bec0 0x7f1ad00ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51230/0 (socket says 172.21.15.76:51230) 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1ad4c4b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1ad00af470 0x7f1ad006e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36614/0 (socket says 172.21.15.76:36614) 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.300+0000 7f1acffff640 1 -- 172.21.15.76:0/428357561 learned_addr learned my addr 172.21.15.76:0/428357561 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1acffff640 1 -- 172.21.15.76:0/428357561 >> v1:172.21.15.116:6789/0 conn(0x7f1ad015c2a0 legacy=0x7f1ad00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1acffff640 1 -- 172.21.15.76:0/428357561 >> v1:172.21.15.76:6789/0 conn(0x7f1ad00af470 legacy=0x7f1ad006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1acffff640 1 -- 172.21.15.76:0/428357561 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad00b3b30 con 0x7f1ad015bec0 2024-09-19T04:16:24.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1acffff640 1 --2- 172.21.15.76:0/428357561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad015bec0 0x7f1ad00ad200 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009870 tx=0x7f1ac001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c0b842e001fdd4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1aceffd640 1 -- 172.21.15.76:0/428357561 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00ab600 con 0x7f1ad015bec0 2024-09-19T04:16:24.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1aceffd640 1 -- 172.21.15.76:0/428357561 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac00b0070 con 0x7f1ad015bec0 2024-09-19T04:16:24.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1aceffd640 1 -- 172.21.15.76:0/428357561 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00ab9c0 con 0x7f1ad015bec0 2024-09-19T04:16:24.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1ad66ae640 1 -- 172.21.15.76:0/428357561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad015bec0 msgr2=0x7f1ad00ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/428357561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad015bec0 0x7f1ad00ad200 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009870 tx=0x7f1ac001c4f0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1ad66ae640 1 -- 172.21.15.76:0/428357561 shutdown_connections 2024-09-19T04:16:24.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/428357561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad015bec0 0x7f1ad00ad200 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1ad66ae640 1 -- 172.21.15.76:0/428357561 >> 172.21.15.76:0/428357561 conn(0x7f1ad00b1f90 msgr2=0x7f1ad00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:24.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1ad66ae640 1 -- 172.21.15.76:0/428357561 shutdown_connections 2024-09-19T04:16:24.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.304+0000 7f1ad66ae640 1 -- 172.21.15.76:0/428357561 wait complete. 2024-09-19T04:16:24.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 Processor -- start 2024-09-19T04:16:24.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 -- start start 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad00af470 0x7f1ad0147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ad015bec0 0x7f1ad014a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad015c2a0 0x7f1ad015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad006ef20 con 0x7f1ad00af470 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad00ad740 con 0x7f1ad015c2a0 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad006f0a0 con 0x7f1ad015bec0 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad00af470 0x7f1ad0147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad00af470 0x7f1ad0147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51242/0 (socket says 172.21.15.76:51242) 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 -- 172.21.15.76:0/152931785 learned_addr learned my addr 172.21.15.76:0/152931785 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:24.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acf7fe640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ad015bec0 0x7f1ad014a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad4c4b640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad015c2a0 0x7f1ad015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 -- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ad015bec0 msgr2=0x7f1ad014a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ad015bec0 0x7f1ad014a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 -- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad015c2a0 msgr2=0x7f1ad015efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad015c2a0 0x7f1ad015efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 -- 172.21.15.76:0/152931785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad0137ee0 con 0x7f1ad00af470 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad4c4b640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad015c2a0 0x7f1ad015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1acffff640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad00af470 0x7f1ad0147750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0019100 tx=0x7f1ac001ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1accff9640 1 -- 172.21.15.76:0/152931785 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00b4800 con 0x7f1ad00af470 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1accff9640 1 -- 172.21.15.76:0/152931785 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ac00b49c0 con 0x7f1ad00af470 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1accff9640 1 -- 172.21.15.76:0/152931785 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00b4db0 con 0x7f1ad00af470 2024-09-19T04:16:24.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad01381c0 con 0x7f1ad00af470 2024-09-19T04:16:24.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad0138780 con 0x7f1ad00af470 2024-09-19T04:16:24.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad00ad200 con 0x7f1ad00af470 2024-09-19T04:16:24.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.308+0000 7f1accff9640 1 -- 172.21.15.76:0/152931785 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1ac00aa070 con 0x7f1ad00af470 2024-09-19T04:16:24.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.312+0000 7f1accff9640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1abc0678c0 0x7f1abc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.312+0000 7f1accff9640 1 -- 172.21.15.76:0/152931785 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f1ac011fd50 con 0x7f1ad00af470 2024-09-19T04:16:24.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.312+0000 7f1acf7fe640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1abc0678c0 0x7f1abc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.312+0000 7f1acf7fe640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1abc0678c0 0x7f1abc069d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ad01f8560 tx=0x7f1ac4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.316+0000 7f1accff9640 1 -- 172.21.15.76:0/152931785 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1ac0123030 con 0x7f1ad00af470 2024-09-19T04:16:24.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.468+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ad00ad200 con 0x7f1ad00af470 2024-09-19T04:16:24.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.468+0000 7f1accff9640 1 -- 172.21.15.76:0/152931785 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+17065 (secure 0 0 0) 0x7f1ac00c4060 con 0x7f1ad00af470 2024-09-19T04:16:24.473 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:24.473 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":55,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:16:07.941982+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:15:32.810605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T04:15:33.811920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.1d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:16:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1abc0678c0 msgr2=0x7f1abc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1abc0678c0 0x7f1abc069d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1ad01f8560 tx=0x7f1ac4009040 comp rx=0 tx=0).stop 2024-09-19T04:16:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad00af470 msgr2=0x7f1ad0147750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad00af470 0x7f1ad0147750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0019100 tx=0x7f1ac001ad30 comp rx=0 tx=0).stop 2024-09-19T04:16:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 shutdown_connections 2024-09-19T04:16:24.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1abc0678c0 0x7f1abc069d80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ad015c2a0 0x7f1ad015efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ad015bec0 0x7f1ad014a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 --2- 172.21.15.76:0/152931785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ad00af470 0x7f1ad0147750 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 >> 172.21.15.76:0/152931785 conn(0x7f1ad00b1f90 msgr2=0x7f1ad00b1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 shutdown_connections 2024-09-19T04:16:24.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.472+0000 7f1ad66ae640 1 -- 172.21.15.76:0/152931785 wait complete. 2024-09-19T04:16:24.491 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:16:24.492 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63eca29640 1 Processor -- start 2024-09-19T04:16:24.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63eca29640 1 -- start start 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63eca29640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63e806f010 0x7f63e8166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63eca29640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e80b3290 con 0x7f63e815c2a0 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63eca29640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63e80b3570 con 0x7f63e815bec0 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63eca29640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f63e80b3850 con 0x7f63e806f010 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f63e815c2a0 0x7f63e80aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36622/0 (socket says 172.21.15.76:36622) 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 -- 172.21.15.76:0/2794262151 learned_addr learned my addr 172.21.15.76:0/2794262151 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e6d9d640 1 --2- 172.21.15.76:0/2794262151 >> v2:172.21.15.116:3300/0 conn(0x7f63e806f010 0x7f63e8166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1539943508 0 0) 0x7f63e80b3290 con 0x7f63e815c2a0 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f63d0003530 con 0x7f63e815c2a0 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 133044619 0 0) 0x7f63d0003530 con 0x7f63e815c2a0 2024-09-19T04:16:24.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 >> v2:172.21.15.116:3300/0 conn(0x7f63e806f010 msgr2=0x7f63e8166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 --2- 172.21.15.76:0/2794262151 >> v2:172.21.15.116:3300/0 conn(0x7f63e806f010 0x7f63e8166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 >> v1:172.21.15.116:6789/0 conn(0x7f63e815bec0 legacy=0x7f63e80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e80b3b30 con 0x7f63e815c2a0 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f63dc001680 con 0x7f63e815c2a0 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 >> v1:172.21.15.76:6789/0 conn(0x7f63e815c2a0 legacy=0x7f63e80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d00039c0 0x7f63d0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d0004810 0x7f63d0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d00051d0 0x7f63d000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80b3850 con 0x7f63d0004810 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63d000c070 con 0x7f63d00039c0 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f63d000c270 con 0x7f63d00051d0 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d0004810 0x7f63d0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e659c640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d00039c0 0x7f63d0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 -- 172.21.15.76:0/2794262151 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d00051d0 msgr2=0x7f63d000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d00051d0 0x7f63d000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 -- 172.21.15.76:0/2794262151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d00039c0 msgr2=0x7f63d0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63d00039c0 0x7f63d0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e5d9b640 1 -- 172.21.15.76:0/2794262151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d0003530 con 0x7f63d0004810 2024-09-19T04:16:24.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.596+0000 7f63e6d9d640 1 -- 172.21.15.76:0/2794262151 reap_dead start 2024-09-19T04:16:24.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d0004810 0x7f63d0004c90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00c190 tx=0x7f63dc01f350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7906231b1ff660f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc007d00 con 0x7f63d0004810 2024-09-19T04:16:24.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc005b00 con 0x7f63d0004810 2024-09-19T04:16:24.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e559a640 1 -- 172.21.15.76:0/2794262151 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc007070 con 0x7f63d0004810 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- 172.21.15.76:0/2794262151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d0004810 msgr2=0x7f63d0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d0004810 0x7f63d0004c90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00c190 tx=0x7f63dc01f350 comp rx=0 tx=0).stop 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- 172.21.15.76:0/2794262151 shutdown_connections 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 --2- 172.21.15.76:0/2794262151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63d0004810 0x7f63d0004c90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- 172.21.15.76:0/2794262151 >> 172.21.15.76:0/2794262151 conn(0x7f63e80b1f90 msgr2=0x7f63e80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- 172.21.15.76:0/2794262151 shutdown_connections 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- 172.21.15.76:0/2794262151 wait complete. 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 Processor -- start 2024-09-19T04:16:24.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- start start 2024-09-19T04:16:24.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d0004810 0x7f63e815c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e815f3b0 0x7f63e809a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63e81f5b70 0x7f63e81f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80afc90 con 0x7f63e815f3b0 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80b40f0 con 0x7f63e81f5b70 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63eca29640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f63e80ad200 con 0x7f63d0004810 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e815f3b0 0x7f63e809a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e815f3b0 0x7f63e809a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51272/0 (socket says 172.21.15.76:51272) 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 -- 172.21.15.76:0/1271488666 learned_addr learned my addr 172.21.15.76:0/1271488666 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e659c640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d0004810 0x7f63e815c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e6d9d640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63e81f5b70 0x7f63e81f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 -- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d0004810 msgr2=0x7f63e815c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d0004810 0x7f63e815c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 -- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63e81f5b70 msgr2=0x7f63e81f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63e81f5b70 0x7f63e81f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.600+0000 7f63e5d9b640 1 -- 172.21.15.76:0/1271488666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e80757d0 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63e5d9b640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e815f3b0 0x7f63e809a280 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f63dc001980 tx=0x7f63d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63d6ffd640 1 -- 172.21.15.76:0/1271488666 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc007d00 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63d6ffd640 1 -- 172.21.15.76:0/1271488666 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc000ee0 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63d6ffd640 1 -- 172.21.15.76:0/1271488666 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc001480 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e80b4a60 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e815dc40 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e80ad530 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63d6ffd640 1 -- 172.21.15.76:0/1271488666 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f63dc0047f0 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63d6ffd640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63c80678c0 0x7f63c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63d6ffd640 1 -- 172.21.15.76:0/1271488666 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f63dc122e60 con 0x7f63e815f3b0 2024-09-19T04:16:24.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63e659c640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63c80678c0 0x7f63c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.604+0000 7f63e659c640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63c80678c0 0x7f63c8069d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c10 tx=0x7f63d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.608+0000 7f63d6ffd640 1 -- 172.21.15.76:0/1271488666 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f63dc0c2030 con 0x7f63e815f3b0 2024-09-19T04:16:24.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.796+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63e806fcd0 con 0x7f63e815f3b0 2024-09-19T04:16:24.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.796+0000 7f63d6ffd640 1 -- 172.21.15.76:0/1271488666 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5276 (secure 0 0 0) 0x7f63dc019060 con 0x7f63e815f3b0 2024-09-19T04:16:24.800 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:24.800 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:16:08.863653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502,5496,4754]},"id":3}]} 2024-09-19T04:16:24.801 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 39 2024-09-19T04:16:24.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63c80678c0 msgr2=0x7f63c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63c80678c0 0x7f63c8069d80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f63d8000c10 tx=0x7f63d8009040 comp rx=0 tx=0).stop 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e815f3b0 msgr2=0x7f63e809a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e815f3b0 0x7f63e809a280 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f63dc001980 tx=0x7f63d0005840 comp rx=0 tx=0).stop 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 shutdown_connections 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63c80678c0 0x7f63c8069d80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63e81f5b70 0x7f63e81f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f63e815f3b0 0x7f63e809a280 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 --2- 172.21.15.76:0/1271488666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f63d0004810 0x7f63e815c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 >> 172.21.15.76:0/1271488666 conn(0x7f63e80b1f90 msgr2=0x7f63e815aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 shutdown_connections 2024-09-19T04:16:24.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.800+0000 7f63eca29640 1 -- 172.21.15.76:0/1271488666 wait complete. 2024-09-19T04:16:24.818 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:24.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0cd3a1640 1 Processor -- start 2024-09-19T04:16:24.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0cd3a1640 1 -- start start 2024-09-19T04:16:24.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0cd3a1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0c8073e60 0x7fd0c806e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0cd3a1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0c806fef0 0x7fd0c815b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0cd3a1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0c815bda0 con 0x7fd0c80b3d50 2024-09-19T04:16:24.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0cd3a1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c806ee50 con 0x7fd0c8073e60 2024-09-19T04:16:24.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0cd3a1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd0c806f130 con 0x7fd0c806fef0 2024-09-19T04:16:24.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0c8073e60 0x7fd0c806e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0c8073e60 0x7fd0c806e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51280/0 (socket says 172.21.15.76:51280) 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 -- 172.21.15.76:0/3655959723 learned_addr learned my addr 172.21.15.76:0/3655959723 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c67fc640 1 --2- 172.21.15.76:0/3655959723 >> v2:172.21.15.116:3300/0 conn(0x7fd0c806fef0 0x7fd0c815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 -- 172.21.15.76:0/3655959723 >> v2:172.21.15.116:3300/0 conn(0x7fd0c806fef0 msgr2=0x7fd0c815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 --2- 172.21.15.76:0/3655959723 >> v2:172.21.15.116:3300/0 conn(0x7fd0c806fef0 0x7fd0c815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 -- 172.21.15.76:0/3655959723 >> v1:172.21.15.116:6789/0 conn(0x7fd0c80b3d50 legacy=0x7fd0c80b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 -- 172.21.15.76:0/3655959723 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c806f410 con 0x7fd0c8073e60 2024-09-19T04:16:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c67fc640 1 --2- 172.21.15.76:0/3655959723 >> v2:172.21.15.116:3300/0 conn(0x7fd0c806fef0 0x7fd0c815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c6ffd640 1 --2- 172.21.15.76:0/3655959723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c8073e60 0x7fd0c806e800 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd0c81589e0 tx=0x7fd0b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44f04884e2c9339 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c5ffb640 1 -- 172.21.15.76:0/3655959723 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80b0070 con 0x7fd0c8073e60 2024-09-19T04:16:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c5ffb640 1 -- 172.21.15.76:0/3655959723 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0b80ab600 con 0x7fd0c8073e60 2024-09-19T04:16:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.924+0000 7fd0c5ffb640 1 -- 172.21.15.76:0/3655959723 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0b80ab9d0 con 0x7fd0c8073e60 2024-09-19T04:16:24.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/3655959723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c8073e60 msgr2=0x7fd0c806e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/3655959723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c8073e60 0x7fd0c806e800 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd0c81589e0 tx=0x7fd0b801c450 comp rx=0 tx=0).stop 2024-09-19T04:16:24.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/3655959723 shutdown_connections 2024-09-19T04:16:24.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/3655959723 >> v2:172.21.15.116:3300/0 conn(0x7fd0c806fef0 0x7fd0c815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/3655959723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c8073e60 0x7fd0c806e800 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/3655959723 >> 172.21.15.76:0/3655959723 conn(0x7fd0c80b1f90 msgr2=0x7fd0c80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:24.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/3655959723 shutdown_connections 2024-09-19T04:16:24.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/3655959723 wait complete. 2024-09-19T04:16:24.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 Processor -- start 2024-09-19T04:16:24.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- start start 2024-09-19T04:16:24.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c806fef0 0x7fd0c814a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0c8073e60 0x7fd0c8155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c80b3d50 0x7fd0c8155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c80731b0 con 0x7fd0c80b3d50 2024-09-19T04:16:24.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd0c8073030 con 0x7fd0c806fef0 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd0c8073330 con 0x7fd0c8073e60 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c80b3d50 0x7fd0c8155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c80b3d50 0x7fd0c8155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51294/0 (socket says 172.21.15.76:51294) 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 -- 172.21.15.76:0/729702653 learned_addr learned my addr 172.21.15.76:0/729702653 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c6ffd640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c806fef0 0x7fd0c814a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 -- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0c8073e60 msgr2=0x7fd0c8155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c67fc640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0c8073e60 0x7fd0c8155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0c8073e60 0x7fd0c8155430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 -- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c806fef0 msgr2=0x7fd0c814a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c806fef0 0x7fd0c814a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 -- 172.21.15.76:0/729702653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0c81f4b30 con 0x7fd0c80b3d50 2024-09-19T04:16:24.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c67fc640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0c8073e60 0x7fd0c8155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c6ffd640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c806fef0 0x7fd0c814a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0c77fe640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c80b3d50 0x7fd0c8155970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c780 tx=0x7fd0c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0bf7fe640 1 -- 172.21.15.76:0/729702653 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0004b70 con 0x7fd0c80b3d50 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0bf7fe640 1 -- 172.21.15.76:0/729702653 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0c0004d10 con 0x7fd0c80b3d50 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0c8157680 con 0x7fd0c80b3d50 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0bf7fe640 1 -- 172.21.15.76:0/729702653 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c000aa90 con 0x7fd0c80b3d50 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.928+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0c8157c40 con 0x7fd0c80b3d50 2024-09-19T04:16:24.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.932+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0c815b860 con 0x7fd0c80b3d50 2024-09-19T04:16:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.932+0000 7fd0bf7fe640 1 -- 172.21.15.76:0/729702653 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd0c000ac30 con 0x7fd0c80b3d50 2024-09-19T04:16:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.932+0000 7fd0bf7fe640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0ac0678c0 0x7fd0ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.932+0000 7fd0bf7fe640 1 -- 172.21.15.76:0/729702653 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7fd0c0086200 con 0x7fd0c80b3d50 2024-09-19T04:16:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.932+0000 7fd0c6ffd640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0ac0678c0 0x7fd0ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:24.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.932+0000 7fd0c6ffd640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0ac0678c0 0x7fd0ac069d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80ad000 tx=0x7fd0b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:24.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:24.936+0000 7fd0bf7fe640 1 -- 172.21.15.76:0/729702653 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd0c0020070 con 0x7fd0c80b3d50 2024-09-19T04:16:25.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.116+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0c80723f0 con 0x7fd0c80b3d50 2024-09-19T04:16:25.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.116+0000 7fd0bf7fe640 1 -- 172.21.15.76:0/729702653 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+5276 (secure 0 0 0) 0x7fd0c0056ae0 con 0x7fd0c80b3d50 2024-09-19T04:16:25.121 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:25.122 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":39,"btime":"2024-09-19T04:16:08:994849+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:15:34.831676+0000","modified":"2024-09-19T04:16:08.863653+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":5502,"mds_1":5496,"mds_2":4754},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4754":{"gid":4754,"name":"b","rank":2,"incarnation":33,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6835/3863430206","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3863430206},{"type":"v1","addr":"172.21.15.116:6835","nonce":3863430206}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5496":{"gid":5496,"name":"c","rank":1,"incarnation":31,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/3259378986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":3259378986},{"type":"v1","addr":"172.21.15.76:6835","nonce":3259378986}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_5502":{"gid":5502,"name":"a","rank":0,"incarnation":26,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3635635274","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3635635274},{"type":"v1","addr":"172.21.15.76:6837","nonce":3635635274}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5502,"qdb_cluster":[5502,5496,4754]},"id":3}]} 2024-09-19T04:16:25.122 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 39 2024-09-19T04:16:25.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0ac0678c0 msgr2=0x7fd0ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0ac0678c0 0x7fd0ac069d80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80ad000 tx=0x7fd0b8008040 comp rx=0 tx=0).stop 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c80b3d50 msgr2=0x7fd0c8155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c80b3d50 0x7fd0c8155970 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000c780 tx=0x7fd0c000cc40 comp rx=0 tx=0).stop 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 shutdown_connections 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0ac0678c0 0x7fd0ac069d80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0c80b3d50 0x7fd0c8155970 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0c8073e60 0x7fd0c8155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 --2- 172.21.15.76:0/729702653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c806fef0 0x7fd0c814a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 >> 172.21.15.76:0/729702653 conn(0x7fd0c80b1f90 msgr2=0x7fd0c814f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 shutdown_connections 2024-09-19T04:16:25.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.120+0000 7fd0cd3a1640 1 -- 172.21.15.76:0/729702653 wait complete. 2024-09-19T04:16:25.139 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:16:25.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52b61b4640 1 Processor -- start 2024-09-19T04:16:25.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52b61b4640 1 -- start start 2024-09-19T04:16:25.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52b61b4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 0x7f52b0168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52b61b4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b00d0300 con 0x7f52b00d3060 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52b61b4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52b00cfab0 con 0x7f52b0161a60 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52b61b4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f52b006e6c0 con 0x7f52b00cf620 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f52b00d3060 0x7f52b00cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36626/0 (socket says 172.21.15.76:36626) 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 -- 172.21.15.76:0/3849196472 learned_addr learned my addr 172.21.15.76:0/3849196472 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52affff640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 0x7f52b0168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2686066595 0 0) 0x7f5298001120 con 0x7f52b00d3060 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f529c003530 con 0x7f52b00d3060 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1759440804 0 0) 0x7f529c003530 con 0x7f52b00d3060 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 msgr2=0x7f52b0168f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 0x7f52b0168f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 >> v1:172.21.15.116:6789/0 conn(0x7f52b0161a60 legacy=0x7f52b00d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b00b0f60 con 0x7f52b00d3060 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5298001850 con 0x7f52b00d3060 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 >> v1:172.21.15.76:6789/0 conn(0x7f52b00d3060 legacy=0x7f52b00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 0x7f529c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c003f70 0x7f529c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c004d50 0x7f529c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f52b006e6c0 con 0x7f529c003f70 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f529c00bb60 con 0x7f529c004d50 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f529c00bd60 con 0x7f52b00cf620 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c003f70 0x7f529c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52af7fe640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 0x7f529c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52affff640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c004d50 0x7f529c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 -- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 msgr2=0x7f529c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:25.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 0x7f529c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 -- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c004d50 msgr2=0x7f529c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:25.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c004d50 0x7f529c0051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52a7fff640 1 -- 172.21.15.76:0/3849196472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f529c003530 con 0x7f529c003f70 2024-09-19T04:16:25.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.256+0000 7f52af7fe640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cf620 0x7f529c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:25.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52a7fff640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c003f70 0x7f529c004810 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f529800c4c0 tx=0x7f529801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ad269c4ad2a85b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529800c0e0 con 0x7f529c003f70 2024-09-19T04:16:25.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52980a9070 con 0x7f529c003f70 2024-09-19T04:16:25.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52aeffd640 1 -- 172.21.15.76:0/3849196472 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52980a5db0 con 0x7f529c003f70 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- 172.21.15.76:0/3849196472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c003f70 msgr2=0x7f529c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 --2- 172.21.15.76:0/3849196472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f529c003f70 0x7f529c004810 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f529800c4c0 tx=0x7f529801cc70 comp rx=0 tx=0).stop 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52affff640 1 -- 172.21.15.76:0/3849196472 reap_dead start 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- 172.21.15.76:0/3849196472 shutdown_connections 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- 172.21.15.76:0/3849196472 >> 172.21.15.76:0/3849196472 conn(0x7f52b008bf90 msgr2=0x7f52b009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- 172.21.15.76:0/3849196472 shutdown_connections 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- 172.21.15.76:0/3849196472 wait complete. 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 Processor -- start 2024-09-19T04:16:25.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- start start 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c003f70 0x7f52b00cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cbe90 0x7f52b00b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52b00cc2f0 0x7f52b00b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0169720 con 0x7f52b00cc2f0 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f52b006d190 con 0x7f529c003f70 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52b61b4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f52b01698a0 con 0x7f52b00cbe90 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52b00cc2f0 0x7f52b00b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52b00cc2f0 0x7f52b00b4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51310/0 (socket says 172.21.15.76:51310) 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52affff640 1 -- 172.21.15.76:0/848522131 learned_addr learned my addr 172.21.15.76:0/848522131 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52af7fe640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c003f70 0x7f52b00cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52a7fff640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cbe90 0x7f52b00b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.260+0000 7f52affff640 1 -- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cbe90 msgr2=0x7f52b00b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52affff640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cbe90 0x7f52b00b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52affff640 1 -- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c003f70 msgr2=0x7f52b00cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52affff640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c003f70 0x7f52b00cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52affff640 1 -- 172.21.15.76:0/848522131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b00ccdc0 con 0x7f52b00cc2f0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52affff640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52b00cc2f0 0x7f52b00b4210 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009fd0 tx=0x7f52a800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52acff9640 1 -- 172.21.15.76:0/848522131 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a801c070 con 0x7f52b00cc2f0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52acff9640 1 -- 172.21.15.76:0/848522131 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52a8017400 con 0x7f52b00cc2f0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b00cd060 con 0x7f52b00cc2f0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52acff9640 1 -- 172.21.15.76:0/848522131 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a80177f0 con 0x7f52b00cc2f0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b00cb690 con 0x7f52b00cc2f0 2024-09-19T04:16:25.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b00cef10 con 0x7f52b00cc2f0 2024-09-19T04:16:25.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52acff9640 1 -- 172.21.15.76:0/848522131 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52a801f070 con 0x7f52b00cc2f0 2024-09-19T04:16:25.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52acff9640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52900678c0 0x7f5290069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52acff9640 1 -- 172.21.15.76:0/848522131 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f52a8085df0 con 0x7f52b00cc2f0 2024-09-19T04:16:25.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52af7fe640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52900678c0 0x7f5290069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.264+0000 7f52af7fe640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52900678c0 0x7f5290069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f52a0001760 tx=0x7f52a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.268+0000 7f52acff9640 1 -- 172.21.15.76:0/848522131 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f52a8012070 con 0x7f52b00cc2f0 2024-09-19T04:16:25.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.420+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52b015c6e0 con 0x7f52b00cc2f0 2024-09-19T04:16:25.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.420+0000 7f52acff9640 1 -- 172.21.15.76:0/848522131 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+17065 (secure 0 0 0) 0x7f52a8056800 con 0x7f52b00cc2f0 2024-09-19T04:16:25.424 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:25.425 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":55,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:16:07.941982+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:15:32.810605+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-09-19T04:15:33.811920+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.2","mappings":[{"from":0,"to":6}]},{"pgid":"8.1d","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52900678c0 msgr2=0x7f5290069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52900678c0 0x7f5290069d80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f52a0001760 tx=0x7f52a0009040 comp rx=0 tx=0).stop 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52b00cc2f0 msgr2=0x7f52b00b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52b00cc2f0 0x7f52b00b4210 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009fd0 tx=0x7f52a800f4a0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 shutdown_connections 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52900678c0 0x7f5290069d80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f52b00cc2f0 0x7f52b00b4210 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f52b00cbe90 0x7f52b00b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 --2- 172.21.15.76:0/848522131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f529c003f70 0x7f52b00cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 >> 172.21.15.76:0/848522131 conn(0x7f52b008bf90 msgr2=0x7f52b00d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:25.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 shutdown_connections 2024-09-19T04:16:25.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.424+0000 7f52b61b4640 1 -- 172.21.15.76:0/848522131 wait complete. 2024-09-19T04:16:25.443 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T04:16:25.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc8fac640 1 Processor -- start 2024-09-19T04:16:25.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc8fac640 1 -- start start 2024-09-19T04:16:25.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc8fac640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7dc415e5f0 0x7f7dc406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc8fac640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7dc40cfaf0 0x7f7dc40d2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc8fac640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7dc406d9c0 con 0x7f7dc40ce9d0 2024-09-19T04:16:25.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc8fac640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc406dca0 con 0x7f7dc415e5f0 2024-09-19T04:16:25.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc8fac640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7dc406df80 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7dc415e5f0 0x7f7dc406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc259c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7dc415e5f0 0x7f7dc406d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51314/0 (socket says 172.21.15.76:51314) 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7db9d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7dc40ce9d0 0x7f7dc40cf3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36630/0 (socket says 172.21.15.76:36630) 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc259c640 1 -- 172.21.15.76:0/3112119582 learned_addr learned my addr 172.21.15.76:0/3112119582 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc2d9d640 1 --2- 172.21.15.76:0/3112119582 >> v2:172.21.15.116:3300/0 conn(0x7f7dc40cfaf0 0x7f7dc40d2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc259c640 1 -- 172.21.15.76:0/3112119582 >> v2:172.21.15.116:3300/0 conn(0x7f7dc40cfaf0 msgr2=0x7f7dc40d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.556+0000 7f7dc259c640 1 --2- 172.21.15.76:0/3112119582 >> v2:172.21.15.116:3300/0 conn(0x7f7dc40cfaf0 0x7f7dc40d2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc259c640 1 -- 172.21.15.76:0/3112119582 >> v1:172.21.15.76:6789/0 conn(0x7f7dc40ce9d0 legacy=0x7f7dc40cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc259c640 1 -- 172.21.15.76:0/3112119582 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc406e260 con 0x7f7dc415e5f0 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc1d9b640 1 -- 172.21.15.76:0/3112119582 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4075224327 0 0) 0x7f7dc406d9c0 con 0x7f7dc40ce9d0 2024-09-19T04:16:25.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc2d9d640 1 --2- 172.21.15.76:0/3112119582 >> v2:172.21.15.116:3300/0 conn(0x7f7dc40cfaf0 0x7f7dc40d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:25.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc259c640 1 --2- 172.21.15.76:0/3112119582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc415e5f0 0x7f7dc406d090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40b3d50 tx=0x7f7db401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b402558f3a816cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc1d9b640 1 -- 172.21.15.76:0/3112119582 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db40ab430 con 0x7f7dc415e5f0 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc1d9b640 1 -- 172.21.15.76:0/3112119582 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db40b0070 con 0x7f7dc415e5f0 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc1d9b640 1 -- 172.21.15.76:0/3112119582 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db40ab7f0 con 0x7f7dc415e5f0 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 -- 172.21.15.76:0/3112119582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc415e5f0 msgr2=0x7f7dc406d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/3112119582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc415e5f0 0x7f7dc406d090 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7dc40b3d50 tx=0x7f7db401c520 comp rx=0 tx=0).stop 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 -- 172.21.15.76:0/3112119582 shutdown_connections 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/3112119582 >> v2:172.21.15.116:3300/0 conn(0x7f7dc40cfaf0 0x7f7dc40d2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/3112119582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc415e5f0 0x7f7dc406d090 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 -- 172.21.15.76:0/3112119582 >> 172.21.15.76:0/3112119582 conn(0x7f7dc4074030 msgr2=0x7f7dc409d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:25.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 -- 172.21.15.76:0/3112119582 shutdown_connections 2024-09-19T04:16:25.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.560+0000 7f7dc8fac640 1 -- 172.21.15.76:0/3112119582 wait complete. 2024-09-19T04:16:25.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 Processor -- start 2024-09-19T04:16:25.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 -- start start 2024-09-19T04:16:25.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc40cfaf0 0x7f7dc40b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7dc414d8f0 0x7f7dc415a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc414cc10 0x7f7dc414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc40d3c40 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7dc40d3ac0 con 0x7f7dc414cc10 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7dc416afa0 con 0x7f7dc414d8f0 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc40cfaf0 0x7f7dc40b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc40cfaf0 0x7f7dc40b2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51316/0 (socket says 172.21.15.76:51316) 2024-09-19T04:16:25.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 -- 172.21.15.76:0/2528328879 learned_addr learned my addr 172.21.15.76:0/2528328879 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7db9d9b640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7dc414d8f0 0x7f7dc415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc2d9d640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc414cc10 0x7f7dc414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 -- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7dc414d8f0 msgr2=0x7f7dc415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7dc414d8f0 0x7f7dc415a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 -- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc414cc10 msgr2=0x7f7dc414d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc414cc10 0x7f7dc414d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 -- 172.21.15.76:0/2528328879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dc40c9de0 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc2d9d640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc414cc10 0x7f7dc414d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc259c640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc40cfaf0 0x7f7dc40b2630 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7db4019130 tx=0x7f7db401ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dbb7fe640 1 -- 172.21.15.76:0/2528328879 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db40ba070 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dbb7fe640 1 -- 172.21.15.76:0/2528328879 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7db4004030 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dbb7fe640 1 -- 172.21.15.76:0/2528328879 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db40043e0 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dc40ca080 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dc40ca640 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dc40d2b60 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.564+0000 7f7dbb7fe640 1 -- 172.21.15.76:0/2528328879 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7db4003620 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.568+0000 7f7dbb7fe640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7da40678f0 0x7f7da4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.568+0000 7f7dbb7fe640 1 -- 172.21.15.76:0/2528328879 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 6159+0+0 (secure 0 0 0) 0x7f7db40a8070 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.568+0000 7f7db9d9b640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7da40678f0 0x7f7da4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.568+0000 7f7db9d9b640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7da40678f0 0x7f7da4069db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4150870 tx=0x7f7dac007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.572+0000 7f7dbb7fe640 1 -- 172.21.15.76:0/2528328879 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7db4016110 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.760+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7dc40cf6a0 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.763 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:16:25.760+0000 7fa674384640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:16:25.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.764+0000 7f7dbb7fe640 1 -- 172.21.15.76:0/2528328879 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) ==== 161+0+0 (secure 0 0 0) 0x7f7db40f03f0 con 0x7f7dc40cfaf0 2024-09-19T04:16:25.766 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7da40678f0 msgr2=0x7f7da4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7da40678f0 0x7f7da4069db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7dc4150870 tx=0x7f7dac007b80 comp rx=0 tx=0).stop 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc40cfaf0 msgr2=0x7f7dc40b2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc40cfaf0 0x7f7dc40b2630 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7db4019130 tx=0x7f7db401ad60 comp rx=0 tx=0).stop 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 shutdown_connections 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7da40678f0 0x7f7da4069db0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc414cc10 0x7f7dc414d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7dc414d8f0 0x7f7dc415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 --2- 172.21.15.76:0/2528328879 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7dc40cfaf0 0x7f7dc40b2630 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 >> 172.21.15.76:0/2528328879 conn(0x7f7dc40ce9d0 msgr2=0x7f7dc40b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:25.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 shutdown_connections 2024-09-19T04:16:25.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.768+0000 7f7dc8fac640 1 -- 172.21.15.76:0/2528328879 wait complete. 2024-09-19T04:16:25.775 INFO:tasks.ceph.mds.c.smithi076.stderr: -1282> 2024-09-19T04:16:03.936+0000 7f61506c0640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:15:48.273429+0000 1=0+1) n(v2 rc2024-09-19T04:15:48.488399+0000 4=0+4) (inest lock) caps={5637=pAsLsXsFs/-@7},l=5637 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55fc16beb080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.273429+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.488399+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.273429+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:15:48.488399+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:16:25.776 INFO:tasks.ceph.mds.c.smithi076.stderr: -1135> 2024-09-19T04:16:03.936+0000 7f6156ecd640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:15:48.275018+0000 1=0+1) n(v1 rc2024-09-19T04:15:48.488399+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={5637=pAsLsXsFs/-@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55fc17c56000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.275018+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.276603+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.275018+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:15:48.488399+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:16:25.785 INFO:tasks.ceph.mds.a.smithi076.stderr: -2487> 2024-09-19T04:16:03.932+0000 7fccdb429640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:15:48.270101+0000 1=0+1) n(v3 rc2024-09-19T04:15:48.488399+0000 6=0+6) (inest mix dirty) (ifile excl) caps={5637=pAsLsXsFsx/AsLsXsFsx@10},l=5637 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x5647964c4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.270101+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.488399+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.270101+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:15:48.488399+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:16:25.786 INFO:tasks.ceph.mds.a.smithi076.stderr: -2389> 2024-09-19T04:16:03.936+0000 7fcce1c36640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:15:48.271783+0000 1=0+1) n(v4 rc2024-09-19T04:15:48.488399+0000 5=0+5) (inest sync->mix g=1,2) caps={5637=pAsLsXsFs/-@11} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x564796546100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.271783+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.488399+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.271783+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:15:48.488399+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:16:25.786 INFO:tasks.ceph.mds.b.smithi116.stderr: -1043> 2024-09-19T04:16:03.935+0000 7f8194c1e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:15:48.276603+0000 1=0+1) n(v0 rc2024-09-19T04:15:48.276603+0000 2=0+2) (inest lock) (ifile excl) caps={5637=pAsLsXsFsx/-@6},l=5637 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x562b06aaa000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:15:48.276603+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:15:48.276603+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:15:48.276603+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:15:48.276603+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:16:25.787 INFO:tasks.ceph.mds.b.smithi116.stderr: -1022> 2024-09-19T04:16:03.935+0000 7f8194c1e640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={5637=pAsLsXsFs/-@3},l=5637 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562b06aaa580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:16:25.789 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:16:25.803 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:16:25.819 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:16:25.830 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:16:25.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.896+0000 7f6948b8c640 1 Processor -- start 2024-09-19T04:16:25.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.896+0000 7f6948b8c640 1 -- start start 2024-09-19T04:16:25.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6948b8c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69440ce9d0 0x7f69440cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6948b8c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694406d9c0 con 0x7f694415e5f0 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6948b8c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f694406dca0 con 0x7f69440cf920 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6948b8c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f694406d7a0 con 0x7f69440ce9d0 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f694415e5f0 0x7f694406d090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36648/0 (socket says 172.21.15.76:36648) 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694259c640 1 -- 172.21.15.76:0/2152381491 learned_addr learned my addr 172.21.15.76:0/2152381491 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6941d9b640 1 --2- 172.21.15.76:0/2152381491 >> v2:172.21.15.116:3300/0 conn(0x7f69440ce9d0 0x7f69440cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430458577 0 0) 0x7f694406d9c0 con 0x7f694415e5f0 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f692c003530 con 0x7f694415e5f0 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1641269966 0 0) 0x7f692c003530 con 0x7f694415e5f0 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 >> v2:172.21.15.116:3300/0 conn(0x7f69440ce9d0 msgr2=0x7f69440cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 --2- 172.21.15.76:0/2152381491 >> v2:172.21.15.116:3300/0 conn(0x7f69440ce9d0 0x7f69440cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 >> v1:172.21.15.116:6789/0 conn(0x7f69440cf920 legacy=0x7f69440d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694406df80 con 0x7f694415e5f0 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6934001680 con 0x7f694415e5f0 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 >> v1:172.21.15.76:6789/0 conn(0x7f694415e5f0 legacy=0x7f694406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f692c0039c0 0x7f692c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c004810 0x7f692c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f692c0051d0 0x7f692c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f694406d7a0 con 0x7f692c0051d0 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f692c00c070 con 0x7f692c004810 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f692c00c270 con 0x7f692c0039c0 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6942d9d640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f692c0051d0 0x7f692c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f694259c640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f692c0039c0 0x7f692c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6941d9b640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c004810 0x7f692c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6942d9d640 1 -- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f692c0039c0 msgr2=0x7f692c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:25.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6942d9d640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f692c0039c0 0x7f692c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6942d9d640 1 -- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c004810 msgr2=0x7f692c004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:25.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6942d9d640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c004810 0x7f692c004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.900+0000 7f6942d9d640 1 -- 172.21.15.76:0/2152381491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692c003530 con 0x7f692c0051d0 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6942d9d640 1 -- 172.21.15.76:0/2152381491 reap_dead start 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6942d9d640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f692c0051d0 0x7f692c00b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f693c001e70 tx=0x7f693c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea3c03f66b8e198 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c001a90 con 0x7f692c0051d0 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f693c003bc0 con 0x7f692c0051d0 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694159a640 1 -- 172.21.15.76:0/2152381491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c005020 con 0x7f692c0051d0 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- 172.21.15.76:0/2152381491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f692c0051d0 msgr2=0x7f692c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f692c0051d0 0x7f692c00b9d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f693c001e70 tx=0x7f693c01c780 comp rx=0 tx=0).stop 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- 172.21.15.76:0/2152381491 shutdown_connections 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 --2- 172.21.15.76:0/2152381491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f692c0051d0 0x7f692c00b9d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- 172.21.15.76:0/2152381491 >> 172.21.15.76:0/2152381491 conn(0x7f6944074030 msgr2=0x7f694409d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:25.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- 172.21.15.76:0/2152381491 shutdown_connections 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- 172.21.15.76:0/2152381491 wait complete. 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 Processor -- start 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- start start 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69441559b0 0x7f69440b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6944160bc0 0x7f694415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f694415b1c0 0x7f694415ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f69440cf3e0 con 0x7f69441559b0 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f69440cfd90 con 0x7f694415b1c0 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f69440cf560 con 0x7f6944160bc0 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69441559b0 0x7f69440b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69441559b0 0x7f69440b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51372/0 (socket says 172.21.15.76:51372) 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 -- 172.21.15.76:0/1464632100 learned_addr learned my addr 172.21.15.76:0/1464632100 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6941d9b640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6944160bc0 0x7f694415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 -- 172.21.15.76:0/1464632100 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6944160bc0 msgr2=0x7f694415ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6944160bc0 0x7f694415ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 -- 172.21.15.76:0/1464632100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f694415b1c0 msgr2=0x7f694415ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f694415b1c0 0x7f694415ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 -- 172.21.15.76:0/1464632100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69440b7800 con 0x7f69441559b0 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f694259c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69441559b0 0x7f69440b71b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f693400e7b0 tx=0x7f693400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6932ffd640 1 -- 172.21.15.76:0/1464632100 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693400ee60 con 0x7f69441559b0 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6932ffd640 1 -- 172.21.15.76:0/1464632100 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6934018070 con 0x7f69441559b0 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69440b7ae0 con 0x7f69441559b0 2024-09-19T04:16:25.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6932ffd640 1 -- 172.21.15.76:0/1464632100 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693400ca90 con 0x7f69441559b0 2024-09-19T04:16:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.904+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69440b80a0 con 0x7f69441559b0 2024-09-19T04:16:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.908+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694415b5e0 con 0x7f69441559b0 2024-09-19T04:16:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.908+0000 7f6932ffd640 1 -- 172.21.15.76:0/1464632100 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f693400cc30 con 0x7f69441559b0 2024-09-19T04:16:25.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.908+0000 7f6932ffd640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69240678c0 0x7f6924069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:25.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.908+0000 7f6932ffd640 1 -- 172.21.15.76:0/1464632100 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6417+0+0 (secure 0 0 0) 0x7f6934085d40 con 0x7f69441559b0 2024-09-19T04:16:25.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.908+0000 7f6941d9b640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69240678c0 0x7f6924069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:25.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.908+0000 7f6941d9b640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69240678c0 0x7f6924069d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f694415b930 tx=0x7f6938008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:25.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:25.912+0000 7f6932ffd640 1 -- 172.21.15.76:0/1464632100 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f69340565b0 con 0x7f69441559b0 2024-09-19T04:16:26.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.096+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f694406d920 con 0x7f69441559b0 2024-09-19T04:16:26.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.764+0000 7f6932ffd640 1 -- 172.21.15.76:0/1464632100 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) ==== 104+0+0 (secure 0 0 0) 0x7f6934055eb0 con 0x7f69441559b0 2024-09-19T04:16:26.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69240678c0 msgr2=0x7f6924069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69240678c0 0x7f6924069d80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f694415b930 tx=0x7f6938008040 comp rx=0 tx=0).stop 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69441559b0 msgr2=0x7f69440b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69441559b0 0x7f69440b71b0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f693400e7b0 tx=0x7f693400ec70 comp rx=0 tx=0).stop 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 shutdown_connections 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69240678c0 0x7f6924069d80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f694415b1c0 0x7f694415ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6944160bc0 0x7f694415ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 --2- 172.21.15.76:0/1464632100 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69441559b0 0x7f69440b71b0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 >> 172.21.15.76:0/1464632100 conn(0x7f692c0051d0 msgr2=0x7f694415bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 shutdown_connections 2024-09-19T04:16:26.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.768+0000 7f6948b8c640 1 -- 172.21.15.76:0/1464632100 wait complete. 2024-09-19T04:16:26.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:16:26.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facec4ca640 1 Processor -- start 2024-09-19T04:16:26.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facec4ca640 1 -- start start 2024-09-19T04:16:26.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facec4ca640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7face4073e60 0x7face406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facec4ca640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face406fef0 0x7face415b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facec4ca640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7face415bda0 con 0x7face40b3d50 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facec4ca640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7face406ee50 con 0x7face4073e60 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facec4ca640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7face406f130 con 0x7face406fef0 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7faceaa67640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7face40b3d50 0x7face40b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36658/0 (socket says 172.21.15.76:36658) 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7faceaa67640 1 -- 172.21.15.76:0/13953060 learned_addr learned my addr 172.21.15.76:0/13953060 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7face9a65640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face406fef0 0x7face415b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7face9264640 1 -- 172.21.15.76:0/13953060 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1239581375 0 0) 0x7face415bda0 con 0x7face40b3d50 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7facea266640 1 --2- 172.21.15.76:0/13953060 >> v2:172.21.15.116:3300/0 conn(0x7face4073e60 0x7face406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7face9264640 1 -- 172.21.15.76:0/13953060 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7facd0004640 con 0x7face40b3d50 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.960+0000 7face9264640 1 -- 172.21.15.76:0/13953060 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 832130366 0 0) 0x7facd0004640 con 0x7face40b3d50 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face406fef0 msgr2=0x7face415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:26.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face406fef0 0x7face415b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 >> v2:172.21.15.116:3300/0 conn(0x7face4073e60 msgr2=0x7face406e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 --2- 172.21.15.76:0/13953060 >> v2:172.21.15.116:3300/0 conn(0x7face4073e60 0x7face406e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face406f410 con 0x7face40b3d50 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7facdc001a00 con 0x7face40b3d50 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 >> v1:172.21.15.76:6789/0 conn(0x7face40b3d50 legacy=0x7face40b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0004ad0 0x7facd0004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0005920 0x7facd0005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face406fef0 0x7facd00062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7face406ee50 con 0x7facd0004ad0 2024-09-19T04:16:26.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7face406f130 con 0x7facd0005920 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7facd000cb10 con 0x7face406fef0 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facea266640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0004ad0 0x7facd0004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9a65640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0005920 0x7facd0005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facea266640 1 -- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face406fef0 msgr2=0x7facd00062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facea266640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face406fef0 0x7facd00062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facea266640 1 -- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0005920 msgr2=0x7facd0005da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facea266640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0005920 0x7facd0005da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facea266640 1 -- 172.21.15.76:0/13953060 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd0004640 con 0x7facd0004ad0 2024-09-19T04:16:26.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9a65640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0005920 0x7facd0005da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facea266640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0004ad0 0x7facd0004ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7face0009ae0 tx=0x7face001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d4efb7d43bd9e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00096b0 con 0x7facd0004ad0 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face00a9430 con 0x7facd0004ad0 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7face9264640 1 -- 172.21.15.76:0/13953060 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face00a9820 con 0x7facd0004ad0 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facec4ca640 1 -- 172.21.15.76:0/13953060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0004ad0 msgr2=0x7facd0004ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facec4ca640 1 --2- 172.21.15.76:0/13953060 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7facd0004ad0 0x7facd0004ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7face0009ae0 tx=0x7face001ca50 comp rx=0 tx=0).stop 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7faceaa67640 1 -- 172.21.15.76:0/13953060 reap_dead start 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facec4ca640 1 -- 172.21.15.76:0/13953060 shutdown_connections 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facec4ca640 1 -- 172.21.15.76:0/13953060 >> 172.21.15.76:0/13953060 conn(0x7face40b1f90 msgr2=0x7face40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facec4ca640 1 -- 172.21.15.76:0/13953060 shutdown_connections 2024-09-19T04:16:26.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.964+0000 7facec4ca640 1 -- 172.21.15.76:0/13953060 wait complete. 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 Processor -- start 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 -- start start 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0004ad0 0x7face414abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face40ade50 0x7face40ae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face415a4d0 0x7face415a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7face40729a0 con 0x7face415a4d0 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7face4072820 con 0x7facd0004ad0 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7face4072b20 con 0x7face40ade50 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face415a4d0 0x7face415a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face415a4d0 0x7face415a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51396/0 (socket says 172.21.15.76:51396) 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 -- 172.21.15.76:0/4001351944 learned_addr learned my addr 172.21.15.76:0/4001351944 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:26.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facea266640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0004ad0 0x7face414abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 -- 172.21.15.76:0/4001351944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face40ade50 msgr2=0x7face40ae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face40ade50 0x7face40ae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 -- 172.21.15.76:0/4001351944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0004ad0 msgr2=0x7face414abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0004ad0 0x7face414abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 -- 172.21.15.76:0/4001351944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face414daf0 con 0x7face415a4d0 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7faceaa67640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face415a4d0 0x7face415a950 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7face40b4130 tx=0x7facdc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facdaffd640 1 -- 172.21.15.76:0/4001351944 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc00cea0 con 0x7face415a4d0 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facdaffd640 1 -- 172.21.15.76:0/4001351944 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7facdc00a5b0 con 0x7face415a4d0 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face414ca50 con 0x7face415a4d0 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facdaffd640 1 -- 172.21.15.76:0/4001351944 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facdc00a9a0 con 0x7face415a4d0 2024-09-19T04:16:26.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face414d010 con 0x7face415a4d0 2024-09-19T04:16:26.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face40aede0 con 0x7face415a4d0 2024-09-19T04:16:26.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facdaffd640 1 -- 172.21.15.76:0/4001351944 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7facdc011070 con 0x7face415a4d0 2024-09-19T04:16:26.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.968+0000 7facdaffd640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7facc80678f0 0x7facc8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:26.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.972+0000 7facdaffd640 1 -- 172.21.15.76:0/4001351944 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 6417+0+0 (secure 0 0 0) 0x7facdc0858e0 con 0x7face415a4d0 2024-09-19T04:16:26.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.972+0000 7facea266640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7facc80678f0 0x7facc8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:26.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.972+0000 7facea266640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7facc80678f0 0x7facc8069db0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7face00ae000 tx=0x7face00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:26.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:26.976+0000 7facdaffd640 1 -- 172.21.15.76:0/4001351944 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7face40aede0 con 0x7face415a4d0 2024-09-19T04:16:27.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:27.140+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7face406e800 con 0x7face415a4d0 2024-09-19T04:16:28.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.060+0000 7facdaffd640 1 -- 172.21.15.76:0/4001351944 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v57) ==== 181+0+0 (secure 0 0 0) 0x7facdc01a060 con 0x7face415a4d0 2024-09-19T04:16:28.065 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:16:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7facc80678f0 msgr2=0x7facc8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7facc80678f0 0x7facc8069db0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7face00ae000 tx=0x7face00046d0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face415a4d0 msgr2=0x7face415a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face415a4d0 0x7face415a950 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7face40b4130 tx=0x7facdc00cc50 comp rx=0 tx=0).stop 2024-09-19T04:16:28.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 shutdown_connections 2024-09-19T04:16:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7facc80678f0 0x7facc8069db0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7face415a4d0 0x7face415a950 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7face40ade50 0x7face40ae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 --2- 172.21.15.76:0/4001351944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7facd0004ad0 0x7face414abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 >> 172.21.15.76:0/4001351944 conn(0x7face40b1f90 msgr2=0x7face4158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 shutdown_connections 2024-09-19T04:16:28.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.068+0000 7facec4ca640 1 -- 172.21.15.76:0/4001351944 wait complete. 2024-09-19T04:16:28.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:16:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 Processor -- start 2024-09-19T04:16:28.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 -- start start 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff49815bec0 0x7ff4980ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff49815c2a0 0x7ff4980aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff49815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff4980b3290 con 0x7ff49815c2a0 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff4980b3570 con 0x7ff49815bec0 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.224+0000 7ff49e920640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff4980b3850 con 0x7ff4980af2a0 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff49815c2a0 0x7ff4980aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:28.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff49815c2a0 0x7ff4980aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51402/0 (socket says 172.21.15.76:51402) 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 -- 172.21.15.76:0/2725906865 learned_addr learned my addr 172.21.15.76:0/2725906865 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff497fff640 1 --2- 172.21.15.76:0/2725906865 >> v2:172.21.15.116:3300/0 conn(0x7ff49815bec0 0x7ff4980ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49cebd640 1 --2- 172.21.15.76:0/2725906865 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff49815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 -- 172.21.15.76:0/2725906865 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 msgr2=0x7ff49815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 --2- 172.21.15.76:0/2725906865 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff49815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 -- 172.21.15.76:0/2725906865 >> v2:172.21.15.116:3300/0 conn(0x7ff49815bec0 msgr2=0x7ff4980ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 --2- 172.21.15.76:0/2725906865 >> v2:172.21.15.116:3300/0 conn(0x7ff49815bec0 0x7ff4980ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 -- 172.21.15.76:0/2725906865 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4980b3b30 con 0x7ff49815c2a0 2024-09-19T04:16:28.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff4977fe640 1 --2- 172.21.15.76:0/2725906865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815c2a0 0x7ff4980aed60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff498071710 tx=0x7ff48c01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6f9f5776d093dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:28.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff496ffd640 1 -- 172.21.15.76:0/2725906865 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0b0070 con 0x7ff49815c2a0 2024-09-19T04:16:28.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff496ffd640 1 -- 172.21.15.76:0/2725906865 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c0ab600 con 0x7ff49815c2a0 2024-09-19T04:16:28.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff496ffd640 1 -- 172.21.15.76:0/2725906865 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0ab9d0 con 0x7ff49815c2a0 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 -- 172.21.15.76:0/2725906865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815c2a0 msgr2=0x7ff4980aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 --2- 172.21.15.76:0/2725906865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815c2a0 0x7ff4980aed60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff498071710 tx=0x7ff48c01c230 comp rx=0 tx=0).stop 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 -- 172.21.15.76:0/2725906865 shutdown_connections 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 --2- 172.21.15.76:0/2725906865 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff49815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 --2- 172.21.15.76:0/2725906865 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815c2a0 0x7ff4980aed60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 --2- 172.21.15.76:0/2725906865 >> v2:172.21.15.116:3300/0 conn(0x7ff49815bec0 0x7ff4980ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 -- 172.21.15.76:0/2725906865 >> 172.21.15.76:0/2725906865 conn(0x7ff4980b1f90 msgr2=0x7ff4980b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 -- 172.21.15.76:0/2725906865 shutdown_connections 2024-09-19T04:16:28.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.228+0000 7ff49e920640 1 -- 172.21.15.76:0/2725906865 wait complete. 2024-09-19T04:16:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 Processor -- start 2024-09-19T04:16:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 -- start start 2024-09-19T04:16:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff498157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:28.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815bec0 0x7ff498148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff49815c2a0 0x7ff49815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4980af6b0 con 0x7ff49815bec0 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4980ad740 con 0x7ff49815c2a0 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff49806e000 con 0x7ff4980af2a0 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815bec0 0x7ff498148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815bec0 0x7ff498148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51414/0 (socket says 172.21.15.76:51414) 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 -- 172.21.15.76:0/3201398406 learned_addr learned my addr 172.21.15.76:0/3201398406 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff497fff640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff498157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 -- 172.21.15.76:0/3201398406 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 msgr2=0x7ff498157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff498157c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 -- 172.21.15.76:0/3201398406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff49815c2a0 msgr2=0x7ff49815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:28.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff49815c2a0 0x7ff49815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 -- 172.21.15.76:0/3201398406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff49815c7e0 con 0x7ff49815bec0 2024-09-19T04:16:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff4977fe640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815bec0 0x7ff498148740 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff48c01a8e0 tx=0x7ff48c018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff494ff9640 1 -- 172.21.15.76:0/3201398406 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0ba020 con 0x7ff49815bec0 2024-09-19T04:16:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff494ff9640 1 -- 172.21.15.76:0/3201398406 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff48c0b6620 con 0x7ff49815bec0 2024-09-19T04:16:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff494ff9640 1 -- 172.21.15.76:0/3201398406 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0b0040 con 0x7ff49815bec0 2024-09-19T04:16:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff498158180 con 0x7ff49815bec0 2024-09-19T04:16:28.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff49815ca80 con 0x7ff49815bec0 2024-09-19T04:16:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.232+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4980ad200 con 0x7ff49815bec0 2024-09-19T04:16:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.236+0000 7ff494ff9640 1 -- 172.21.15.76:0/3201398406 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff48c0bd020 con 0x7ff49815bec0 2024-09-19T04:16:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.236+0000 7ff494ff9640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4840678c0 0x7ff484069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.236+0000 7ff497fff640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4840678c0 0x7ff484069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.236+0000 7ff494ff9640 1 -- 172.21.15.76:0/3201398406 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5979+0+0 (secure 0 0 0) 0x7ff48c11ffc0 con 0x7ff49815bec0 2024-09-19T04:16:28.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.236+0000 7ff497fff640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4840678c0 0x7ff484069d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff49815f820 tx=0x7ff488008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:28.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.240+0000 7ff494ff9640 1 -- 172.21.15.76:0/3201398406 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff48c0c4030 con 0x7ff49815bec0 2024-09-19T04:16:28.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:28.408+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff49806fcd0 con 0x7ff49815bec0 2024-09-19T04:16:29.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.072+0000 7ff494ff9640 1 -- 172.21.15.76:0/3201398406 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v58) ==== 169+0+0 (secure 0 0 0) 0x7ff48c016110 con 0x7ff49815bec0 2024-09-19T04:16:29.074 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-09-19T04:16:29.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4840678c0 msgr2=0x7ff484069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4840678c0 0x7ff484069d80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff49815f820 tx=0x7ff488008040 comp rx=0 tx=0).stop 2024-09-19T04:16:29.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815bec0 msgr2=0x7ff498148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815bec0 0x7ff498148740 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7ff48c01a8e0 tx=0x7ff48c018fd0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 shutdown_connections 2024-09-19T04:16:29.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4840678c0 0x7ff484069d80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff49815c2a0 0x7ff49815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff49815bec0 0x7ff498148740 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 --2- 172.21.15.76:0/3201398406 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4980af2a0 0x7ff498157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 >> 172.21.15.76:0/3201398406 conn(0x7ff4980b1f90 msgr2=0x7ff4980cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:29.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 shutdown_connections 2024-09-19T04:16:29.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.076+0000 7ff49e920640 1 -- 172.21.15.76:0/3201398406 wait complete. 2024-09-19T04:16:29.105 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T04:16:29.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 Processor -- start 2024-09-19T04:16:29.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 -- start start 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a81520a0 0x7fa8a806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8a806fe50 0x7fa8a815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8a815f1d0 0x7fa8a8071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a8152480 con 0x7fa8a806fe50 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a8070250 con 0x7fa8a815f1d0 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8b042c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a80aede0 con 0x7fa8a81520a0 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8ad9c7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8a806fe50 0x7fa8a815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8ad9c7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa8a806fe50 0x7fa8a815ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51426/0 (socket says 172.21.15.76:51426) 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.228+0000 7fa8ad9c7640 1 -- 172.21.15.76:0/521679046 learned_addr learned my addr 172.21.15.76:0/521679046 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ae9c9640 1 --2- 172.21.15.76:0/521679046 >> v2:172.21.15.116:3300/0 conn(0x7fa8a815f1d0 0x7fa8a8071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad9c7640 1 -- 172.21.15.76:0/521679046 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a81520a0 msgr2=0x7fa8a806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:29.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ae1c8640 1 --2- 172.21.15.76:0/521679046 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a81520a0 0x7fa8a806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad9c7640 1 --2- 172.21.15.76:0/521679046 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a81520a0 0x7fa8a806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad9c7640 1 -- 172.21.15.76:0/521679046 >> v2:172.21.15.116:3300/0 conn(0x7fa8a815f1d0 msgr2=0x7fa8a8071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:29.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad9c7640 1 --2- 172.21.15.76:0/521679046 >> v2:172.21.15.116:3300/0 conn(0x7fa8a815f1d0 0x7fa8a8071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad9c7640 1 -- 172.21.15.76:0/521679046 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a815fa10 con 0x7fa8a806fe50 2024-09-19T04:16:29.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ae9c9640 1 --2- 172.21.15.76:0/521679046 >> v2:172.21.15.116:3300/0 conn(0x7fa8a815f1d0 0x7fa8a8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:29.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ae1c8640 1 --2- 172.21.15.76:0/521679046 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a81520a0 0x7fa8a806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:29.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad9c7640 1 --2- 172.21.15.76:0/521679046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a806fe50 0x7fa8a815ec90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa8a806e250 tx=0x7fa89801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aaaaeadccba47c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:29.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad1c6640 1 -- 172.21.15.76:0/521679046 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8980b0070 con 0x7fa8a806fe50 2024-09-19T04:16:29.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad1c6640 1 -- 172.21.15.76:0/521679046 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8980ab600 con 0x7fa8a806fe50 2024-09-19T04:16:29.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8ad1c6640 1 -- 172.21.15.76:0/521679046 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8980ab9d0 con 0x7fa8a806fe50 2024-09-19T04:16:29.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 -- 172.21.15.76:0/521679046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a806fe50 msgr2=0x7fa8a815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 --2- 172.21.15.76:0/521679046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a806fe50 0x7fa8a815ec90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fa8a806e250 tx=0x7fa89801c240 comp rx=0 tx=0).stop 2024-09-19T04:16:29.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 -- 172.21.15.76:0/521679046 shutdown_connections 2024-09-19T04:16:29.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 --2- 172.21.15.76:0/521679046 >> v2:172.21.15.116:3300/0 conn(0x7fa8a815f1d0 0x7fa8a8071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 --2- 172.21.15.76:0/521679046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a806fe50 0x7fa8a815ec90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 --2- 172.21.15.76:0/521679046 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a81520a0 0x7fa8a806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 -- 172.21.15.76:0/521679046 >> 172.21.15.76:0/521679046 conn(0x7fa8a80b1f90 msgr2=0x7fa8a80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:29.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 -- 172.21.15.76:0/521679046 shutdown_connections 2024-09-19T04:16:29.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.232+0000 7fa8b042c640 1 -- 172.21.15.76:0/521679046 wait complete. 2024-09-19T04:16:29.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 Processor -- start 2024-09-19T04:16:29.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 -- start start 2024-09-19T04:16:29.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a806fe50 0x7fa8a814e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a81520a0 0x7fa8a8151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a815f1d0 0x7fa8a8151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8073d30 con 0x7fa8a815f1d0 2024-09-19T04:16:29.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a8073a50 con 0x7fa8a81520a0 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a815f5e0 con 0x7fa8a806fe50 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a815f1d0 0x7fa8a8151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a815f1d0 0x7fa8a8151980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51432/0 (socket says 172.21.15.76:51432) 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 -- 172.21.15.76:0/188337488 learned_addr learned my addr 172.21.15.76:0/188337488 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ad9c7640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a81520a0 0x7fa8a8151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae1c8640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a806fe50 0x7fa8a814e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 -- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a806fe50 msgr2=0x7fa8a814e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a806fe50 0x7fa8a814e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 -- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a81520a0 msgr2=0x7fa8a8151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a81520a0 0x7fa8a8151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 -- 172.21.15.76:0/188337488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a80b4cf0 con 0x7fa8a815f1d0 2024-09-19T04:16:29.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ad9c7640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a81520a0 0x7fa8a8151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:29.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8ae9c9640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a815f1d0 0x7fa8a8151980 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000cef0 tx=0x7fa8a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:29.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa89effd640 1 -- 172.21.15.76:0/188337488 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a001a070 con 0x7fa8a815f1d0 2024-09-19T04:16:29.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa89effd640 1 -- 172.21.15.76:0/188337488 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8a000d860 con 0x7fa8a815f1d0 2024-09-19T04:16:29.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a80cce50 con 0x7fa8a815f1d0 2024-09-19T04:16:29.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa89effd640 1 -- 172.21.15.76:0/188337488 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000dc50 con 0x7fa8a815f1d0 2024-09-19T04:16:29.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a80cd410 con 0x7fa8a815f1d0 2024-09-19T04:16:29.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a806f910 con 0x7fa8a815f1d0 2024-09-19T04:16:29.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.236+0000 7fa89effd640 1 -- 172.21.15.76:0/188337488 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa8a001d070 con 0x7fa8a815f1d0 2024-09-19T04:16:29.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.240+0000 7fa89effd640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa88c0678c0 0x7fa88c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.240+0000 7fa89effd640 1 -- 172.21.15.76:0/188337488 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5507+0+0 (secure 0 0 0) 0x7fa8a0086430 con 0x7fa8a815f1d0 2024-09-19T04:16:29.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.240+0000 7fa8ae1c8640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa88c0678c0 0x7fa88c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.240+0000 7fa8ae1c8640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa88c0678c0 0x7fa88c069d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4006b20 tx=0x7fa8a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:29.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.244+0000 7fa89effd640 1 -- 172.21.15.76:0/188337488 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa8a00570f0 con 0x7fa8a815f1d0 2024-09-19T04:16:29.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.404+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fa8a806e540 con 0x7fa8a815f1d0 2024-09-19T04:16:29.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.408+0000 7fa89effd640 1 -- 172.21.15.76:0/188337488 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fa8a00569f0 con 0x7fa8a815f1d0 2024-09-19T04:16:29.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa88c0678c0 msgr2=0x7fa88c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa88c0678c0 0x7fa88c069d80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4006b20 tx=0x7fa8a4008040 comp rx=0 tx=0).stop 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a815f1d0 msgr2=0x7fa8a8151980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a815f1d0 0x7fa8a8151980 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000cef0 tx=0x7fa8a000a710 comp rx=0 tx=0).stop 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 shutdown_connections 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa88c0678c0 0x7fa88c069d80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a815f1d0 0x7fa8a8151980 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a81520a0 0x7fa8a8151440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 --2- 172.21.15.76:0/188337488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a806fe50 0x7fa8a814e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 >> 172.21.15.76:0/188337488 conn(0x7fa8a80b1f90 msgr2=0x7fa8a80ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 shutdown_connections 2024-09-19T04:16:29.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.412+0000 7fa8b042c640 1 -- 172.21.15.76:0/188337488 wait complete. 2024-09-19T04:16:29.432 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0' 2024-09-19T04:16:29.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.544+0000 7fd2803bd640 1 Processor -- start 2024-09-19T04:16:29.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.544+0000 7fd2803bd640 1 -- start start 2024-09-19T04:16:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd2803bd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815c2a0 0x7fd2780aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd2803bd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2780af2a0 0x7fd27806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd2803bd640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2780b3290 con 0x7fd27815bec0 2024-09-19T04:16:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd2803bd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd2780b3570 con 0x7fd2780af2a0 2024-09-19T04:16:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd2803bd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd2780b3850 con 0x7fd27815c2a0 2024-09-19T04:16:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2780af2a0 0x7fd27806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd2780af2a0 0x7fd27806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51434/0 (socket says 172.21.15.76:51434) 2024-09-19T04:16:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 -- 172.21.15.76:0/3828175872 learned_addr learned my addr 172.21.15.76:0/3828175872 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27d958640 1 --2- 172.21.15.76:0/3828175872 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815c2a0 0x7fd2780aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 -- 172.21.15.76:0/3828175872 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815c2a0 msgr2=0x7fd2780aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 --2- 172.21.15.76:0/3828175872 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815c2a0 0x7fd2780aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 -- 172.21.15.76:0/3828175872 >> v1:172.21.15.116:6789/0 conn(0x7fd27815bec0 legacy=0x7fd2780ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:29.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 -- 172.21.15.76:0/3828175872 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2780b3b30 con 0x7fd2780af2a0 2024-09-19T04:16:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27e95a640 1 --2- 172.21.15.76:0/3828175872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2780af2a0 0x7fd27806e000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd274009870 tx=0x7fd27401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982ea811dd8032d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27d157640 1 -- 172.21.15.76:0/3828175872 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740aa610 con 0x7fd2780af2a0 2024-09-19T04:16:29.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27d157640 1 -- 172.21.15.76:0/3828175872 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2740af070 con 0x7fd2780af2a0 2024-09-19T04:16:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.548+0000 7fd27d157640 1 -- 172.21.15.76:0/3828175872 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740aa9d0 con 0x7fd2780af2a0 2024-09-19T04:16:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- 172.21.15.76:0/3828175872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2780af2a0 msgr2=0x7fd27806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 --2- 172.21.15.76:0/3828175872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2780af2a0 0x7fd27806e000 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd274009870 tx=0x7fd27401c4f0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- 172.21.15.76:0/3828175872 shutdown_connections 2024-09-19T04:16:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 --2- 172.21.15.76:0/3828175872 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2780af2a0 0x7fd27806e000 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 --2- 172.21.15.76:0/3828175872 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815c2a0 0x7fd2780aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- 172.21.15.76:0/3828175872 >> 172.21.15.76:0/3828175872 conn(0x7fd2780b1f90 msgr2=0x7fd2780b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- 172.21.15.76:0/3828175872 shutdown_connections 2024-09-19T04:16:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- 172.21.15.76:0/3828175872 wait complete. 2024-09-19T04:16:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 Processor -- start 2024-09-19T04:16:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- start start 2024-09-19T04:16:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2780af2a0 0x7fd27814a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815bec0 0x7fd278073a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd27815c2a0 0x7fd2780729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd27806ed50 con 0x7fd27815c2a0 2024-09-19T04:16:29.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2780afc90 con 0x7fd2780af2a0 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd2803bd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd27806eed0 con 0x7fd27815bec0 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd27815c2a0 0x7fd2780729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd27815c2a0 0x7fd2780729b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51446/0 (socket says 172.21.15.76:51446) 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 -- 172.21.15.76:0/294140203 learned_addr learned my addr 172.21.15.76:0/294140203 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e159640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2780af2a0 0x7fd27814a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27d958640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815bec0 0x7fd278073a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 -- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815bec0 msgr2=0x7fd278073a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815bec0 0x7fd278073a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 -- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2780af2a0 msgr2=0x7fd27814a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2780af2a0 0x7fd27814a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 -- 172.21.15.76:0/294140203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd278071f10 con 0x7fd27815c2a0 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e159640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2780af2a0 0x7fd27814a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27d958640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815bec0 0x7fd278073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.552+0000 7fd27e95a640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd27815c2a0 0x7fd2780729b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd27401aba0 tx=0x7fd27401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd26affd640 1 -- 172.21.15.76:0/294140203 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740bd5f0 con 0x7fd27815c2a0 2024-09-19T04:16:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd26affd640 1 -- 172.21.15.76:0/294140203 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2740bd790 con 0x7fd27815c2a0 2024-09-19T04:16:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd26affd640 1 -- 172.21.15.76:0/294140203 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2740bdb40 con 0x7fd27815c2a0 2024-09-19T04:16:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd278150760 con 0x7fd27815c2a0 2024-09-19T04:16:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd278150d20 con 0x7fd27815c2a0 2024-09-19T04:16:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27806f280 con 0x7fd27815c2a0 2024-09-19T04:16:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd26affd640 1 -- 172.21.15.76:0/294140203 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd2740b33f0 con 0x7fd27815c2a0 2024-09-19T04:16:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd26affd640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd25c0678c0 0x7fd25c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd27e159640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd25c0678c0 0x7fd25c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd26affd640 1 -- 172.21.15.76:0/294140203 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5507+0+0 (secure 0 0 0) 0x7fd27411fea0 con 0x7fd27815c2a0 2024-09-19T04:16:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.556+0000 7fd27e159640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd25c0678c0 0x7fd25c069d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd26c003e40 tx=0x7fd26c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:29.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.560+0000 7fd26affd640 1 -- 172.21.15.76:0/294140203 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd2740a7070 con 0x7fd27815c2a0 2024-09-19T04:16:29.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:29.772+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]} v 0) -- 0x7fd27806f280 con 0x7fd27815c2a0 2024-09-19T04:16:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.072+0000 7fd26affd640 1 -- 172.21.15.76:0/294140203 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_abort_mds0"]}]=0 v167) ==== 143+0+0 (secure 0 0 0) 0x7fd274016110 con 0x7fd27815c2a0 2024-09-19T04:16:30.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd25c0678c0 msgr2=0x7fd25c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd25c0678c0 0x7fd25c069d80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd26c003e40 tx=0x7fd26c009040 comp rx=0 tx=0).stop 2024-09-19T04:16:30.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd27815c2a0 msgr2=0x7fd2780729b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd27815c2a0 0x7fd2780729b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd27401aba0 tx=0x7fd27401ad30 comp rx=0 tx=0).stop 2024-09-19T04:16:30.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 shutdown_connections 2024-09-19T04:16:30.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd25c0678c0 0x7fd25c069d80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd27815c2a0 0x7fd2780729b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd27815bec0 0x7fd278073a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 --2- 172.21.15.76:0/294140203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2780af2a0 0x7fd27814a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 >> 172.21.15.76:0/294140203 conn(0x7fd2780b1f90 msgr2=0x7fd2780afe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:30.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 shutdown_connections 2024-09-19T04:16:30.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.076+0000 7fd2803bd640 1 -- 172.21.15.76:0/294140203 wait complete. 2024-09-19T04:16:30.102 INFO:tasks.cephfs_test_runner:test_scrub_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-19T04:16:30.102 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-19T04:16:30.103 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:16:30.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.220+0000 7f0b6f371640 1 Processor -- start 2024-09-19T04:16:30.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.220+0000 7f0b6f371640 1 -- start start 2024-09-19T04:16:30.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6f371640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b681520a0 0x7f0b6806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6f371640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b6806fe50 0x7f0b6815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6f371640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0b6815f1d0 0x7f0b68071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6f371640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0b68152480 con 0x7f0b6806fe50 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6f371640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0b68070250 con 0x7f0b6815f1d0 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6f371640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0b680aede0 con 0x7f0b681520a0 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b6806fe50 0x7f0b6815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0b6806fe50 0x7f0b6815ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51450/0 (socket says 172.21.15.76:51450) 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 -- 172.21.15.76:0/3472297790 learned_addr learned my addr 172.21.15.76:0/3472297790 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6d10d640 1 --2- 172.21.15.76:0/3472297790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b681520a0 0x7f0b6806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6d90e640 1 --2- 172.21.15.76:0/3472297790 >> v2:172.21.15.116:3300/0 conn(0x7f0b6815f1d0 0x7f0b68071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 -- 172.21.15.76:0/3472297790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b681520a0 msgr2=0x7f0b6806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 --2- 172.21.15.76:0/3472297790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b681520a0 0x7f0b6806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 -- 172.21.15.76:0/3472297790 >> v2:172.21.15.116:3300/0 conn(0x7f0b6815f1d0 msgr2=0x7f0b68071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:30.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 --2- 172.21.15.76:0/3472297790 >> v2:172.21.15.116:3300/0 conn(0x7f0b6815f1d0 0x7f0b68071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 -- 172.21.15.76:0/3472297790 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b6815fa10 con 0x7f0b6806fe50 2024-09-19T04:16:30.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b6c90c640 1 --2- 172.21.15.76:0/3472297790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6806fe50 0x7f0b6815ec90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806e250 tx=0x7f0b5801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d2bffdfd36acb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b5ffff640 1 -- 172.21.15.76:0/3472297790 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580b0070 con 0x7f0b6806fe50 2024-09-19T04:16:30.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b5ffff640 1 -- 172.21.15.76:0/3472297790 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b580ab600 con 0x7f0b6806fe50 2024-09-19T04:16:30.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.224+0000 7f0b5ffff640 1 -- 172.21.15.76:0/3472297790 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b580ab9d0 con 0x7f0b6806fe50 2024-09-19T04:16:30.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- 172.21.15.76:0/3472297790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6806fe50 msgr2=0x7f0b6815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 --2- 172.21.15.76:0/3472297790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6806fe50 0x7f0b6815ec90 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0b6806e250 tx=0x7f0b5801c240 comp rx=0 tx=0).stop 2024-09-19T04:16:30.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- 172.21.15.76:0/3472297790 shutdown_connections 2024-09-19T04:16:30.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 --2- 172.21.15.76:0/3472297790 >> v2:172.21.15.116:3300/0 conn(0x7f0b6815f1d0 0x7f0b68071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 --2- 172.21.15.76:0/3472297790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6806fe50 0x7f0b6815ec90 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 --2- 172.21.15.76:0/3472297790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b681520a0 0x7f0b6806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- 172.21.15.76:0/3472297790 >> 172.21.15.76:0/3472297790 conn(0x7f0b680b1f90 msgr2=0x7f0b680b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:30.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- 172.21.15.76:0/3472297790 shutdown_connections 2024-09-19T04:16:30.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- 172.21.15.76:0/3472297790 wait complete. 2024-09-19T04:16:30.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 Processor -- start 2024-09-19T04:16:30.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- start start 2024-09-19T04:16:30.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b6806fe50 0x7f0b6809cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b681520a0 0x7f0b6815dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6815f1d0 0x7f0b6813ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68073d30 con 0x7f0b6815f1d0 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0b68073a50 con 0x7f0b681520a0 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0b6815f5e0 con 0x7f0b6806fe50 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6815f1d0 0x7f0b6813ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6815f1d0 0x7f0b6813ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51458/0 (socket says 172.21.15.76:51458) 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 -- 172.21.15.76:0/2369180968 learned_addr learned my addr 172.21.15.76:0/2369180968 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d10d640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b6806fe50 0x7f0b6809cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6c90c640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b681520a0 0x7f0b6815dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 -- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b6806fe50 msgr2=0x7f0b6809cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b6806fe50 0x7f0b6809cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 -- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b681520a0 msgr2=0x7f0b6815dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b681520a0 0x7f0b6815dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 -- 172.21.15.76:0/2369180968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b680b08b0 con 0x7f0b6815f1d0 2024-09-19T04:16:30.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6d90e640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6815f1d0 0x7f0b6813ff80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000efc0 tx=0x7f0b6000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b5dffb640 1 -- 172.21.15.76:0/2369180968 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60018020 con 0x7f0b6815f1d0 2024-09-19T04:16:30.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b5dffb640 1 -- 172.21.15.76:0/2369180968 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b60014480 con 0x7f0b6815f1d0 2024-09-19T04:16:30.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b5dffb640 1 -- 172.21.15.76:0/2369180968 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b60014870 con 0x7f0b6815f1d0 2024-09-19T04:16:30.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b6814e780 con 0x7f0b6815f1d0 2024-09-19T04:16:30.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.228+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b680b0b20 con 0x7f0b6815f1d0 2024-09-19T04:16:30.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.232+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b6806f910 con 0x7f0b6815f1d0 2024-09-19T04:16:30.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.232+0000 7f0b5dffb640 1 -- 172.21.15.76:0/2369180968 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0b6001b020 con 0x7f0b6815f1d0 2024-09-19T04:16:30.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.232+0000 7f0b5dffb640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0b4c0678c0 0x7f0b4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.232+0000 7f0b5dffb640 1 -- 172.21.15.76:0/2369180968 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5507+0+0 (secure 0 0 0) 0x7f0b600865b0 con 0x7f0b6815f1d0 2024-09-19T04:16:30.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.232+0000 7f0b6d10d640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0b4c0678c0 0x7f0b4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.232+0000 7f0b6d10d640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0b4c0678c0 0x7f0b4c069d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009ae0 tx=0x7f0b64009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.236+0000 7f0b5dffb640 1 -- 172.21.15.76:0/2369180968 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0b60022070 con 0x7f0b6815f1d0 2024-09-19T04:16:30.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.388+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b6806e540 con 0x7f0b6815f1d0 2024-09-19T04:16:30.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.388+0000 7f0b5dffb640 1 -- 172.21.15.76:0/2369180968 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+13591 (secure 0 0 0) 0x7f0b60010040 con 0x7f0b6815f1d0 2024-09-19T04:16:30.392 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:30.392 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:16:29.066076+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6837/3635635274":"2024-09-20T04:16:25.763783+0000","172.21.15.76:6836/3635635274":"2024-09-20T04:16:25.763783+0000","172.21.15.116:6835/3863430206":"2024-09-20T04:16:25.763769+0000","172.21.15.116:6834/3863430206":"2024-09-20T04:16:25.763769+0000","172.21.15.76:6835/3259378986":"2024-09-20T04:16:25.763778+0000","172.21.15.76:6834/3259378986":"2024-09-20T04:16:25.763778+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:16:30.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0b4c0678c0 msgr2=0x7f0b4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0b4c0678c0 0x7f0b4c069d80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0b64009ae0 tx=0x7f0b64009190 comp rx=0 tx=0).stop 2024-09-19T04:16:30.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6815f1d0 msgr2=0x7f0b6813ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6815f1d0 0x7f0b6813ff80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f0b6000efc0 tx=0x7f0b6000c490 comp rx=0 tx=0).stop 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 shutdown_connections 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0b4c0678c0 0x7f0b4c069d80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0b6815f1d0 0x7f0b6813ff80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0b681520a0 0x7f0b6815dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 --2- 172.21.15.76:0/2369180968 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0b6806fe50 0x7f0b6809cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 >> 172.21.15.76:0/2369180968 conn(0x7f0b680b1f90 msgr2=0x7f0b6814eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 shutdown_connections 2024-09-19T04:16:30.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.392+0000 7f0b6f371640 1 -- 172.21.15.76:0/2369180968 wait complete. 2024-09-19T04:16:30.410 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:13.812655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:16:30.410 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:16:30.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc348c15640 1 Processor -- start 2024-09-19T04:16:30.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc348c15640 1 -- start start 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc348c15640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc344070020 0x7fc34415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc348c15640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3440aede0 con 0x7fc34415f1d0 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc348c15640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc34415fa10 con 0x7fc3441520a0 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc348c15640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc344152480 con 0x7fc344070020 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc341d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc344070020 0x7fc34415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc341d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc344070020 0x7fc34415ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51472/0 (socket says 172.21.15.76:51472) 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc341d9b640 1 -- 172.21.15.76:0/1914684180 learned_addr learned my addr 172.21.15.76:0/1914684180 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc342d9d640 1 --1- 172.21.15.76:0/1914684180 >> v1:172.21.15.76:6789/0 conn(0x7fc34415f1d0 0x7fc3440adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36662/0 (socket says 172.21.15.76:36662) 2024-09-19T04:16:30.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc341d9b640 1 -- 172.21.15.76:0/1914684180 >> v1:172.21.15.116:6789/0 conn(0x7fc3441520a0 legacy=0x7fc34406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:30.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc341d9b640 1 -- 172.21.15.76:0/1914684180 >> v1:172.21.15.76:6789/0 conn(0x7fc34415f1d0 legacy=0x7fc3440adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc34159a640 1 -- 172.21.15.76:0/1914684180 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1089502609 0 0) 0x7fc3440aede0 con 0x7fc34415f1d0 2024-09-19T04:16:30.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.520+0000 7fc341d9b640 1 -- 172.21.15.76:0/1914684180 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc344074250 con 0x7fc344070020 2024-09-19T04:16:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 --2- 172.21.15.76:0/1914684180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc344070020 0x7fc34415ec90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc33800bb10 tx=0x7fc33801ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caff13cbb4e03c09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc34159a640 1 -- 172.21.15.76:0/1914684180 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3380b4070 con 0x7fc344070020 2024-09-19T04:16:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc34159a640 1 -- 172.21.15.76:0/1914684180 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3380a7c90 con 0x7fc344070020 2024-09-19T04:16:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc34159a640 1 -- 172.21.15.76:0/1914684180 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3440aeba0 con 0x7fc344070020 2024-09-19T04:16:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- 172.21.15.76:0/1914684180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc344070020 msgr2=0x7fc34415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 --2- 172.21.15.76:0/1914684180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc344070020 0x7fc34415ec90 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc33800bb10 tx=0x7fc33801ebc0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- 172.21.15.76:0/1914684180 shutdown_connections 2024-09-19T04:16:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 --2- 172.21.15.76:0/1914684180 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc344070020 0x7fc34415ec90 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- 172.21.15.76:0/1914684180 >> 172.21.15.76:0/1914684180 conn(0x7fc3440b1f90 msgr2=0x7fc3440b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- 172.21.15.76:0/1914684180 shutdown_connections 2024-09-19T04:16:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- 172.21.15.76:0/1914684180 wait complete. 2024-09-19T04:16:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 Processor -- start 2024-09-19T04:16:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- start start 2024-09-19T04:16:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc344070020 0x7fc344147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3441520a0 0x7fc34414c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc34415f1d0 0x7fc3440cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc34415f640 con 0x7fc3441520a0 2024-09-19T04:16:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc344073d30 con 0x7fc34415f1d0 2024-09-19T04:16:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc34415f7c0 con 0x7fc344070020 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3441520a0 0x7fc34414c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3441520a0 0x7fc34414c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51478/0 (socket says 172.21.15.76:51478) 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 -- 172.21.15.76:0/899740791 learned_addr learned my addr 172.21.15.76:0/899740791 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 -- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc344070020 msgr2=0x7fc344147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc34259c640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc344070020 0x7fc344147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc342d9d640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc34415f1d0 0x7fc3440cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc344070020 0x7fc344147210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 -- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc34415f1d0 msgr2=0x7fc3440cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc34415f1d0 0x7fc3440cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 -- 172.21.15.76:0/899740791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc344147910 con 0x7fc3441520a0 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc342d9d640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc34415f1d0 0x7fc3440cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc341d9b640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3441520a0 0x7fc34414c2d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc33800b2c0 tx=0x7fc33801b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc332ffd640 1 -- 172.21.15.76:0/899740791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3380b4040 con 0x7fc3441520a0 2024-09-19T04:16:30.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc332ffd640 1 -- 172.21.15.76:0/899740791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3380c1070 con 0x7fc3441520a0 2024-09-19T04:16:30.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc332ffd640 1 -- 172.21.15.76:0/899740791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3380b9460 con 0x7fc3441520a0 2024-09-19T04:16:30.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.524+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc344147bf0 con 0x7fc3441520a0 2024-09-19T04:16:30.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.528+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3441481b0 con 0x7fc3441520a0 2024-09-19T04:16:30.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.528+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34415ec90 con 0x7fc3441520a0 2024-09-19T04:16:30.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.528+0000 7fc332ffd640 1 -- 172.21.15.76:0/899740791 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc3380b7070 con 0x7fc3441520a0 2024-09-19T04:16:30.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.528+0000 7fc332ffd640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3240678c0 0x7fc324069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.528+0000 7fc332ffd640 1 -- 172.21.15.76:0/899740791 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5507+0+0 (secure 0 0 0) 0x7fc338122cb0 con 0x7fc3441520a0 2024-09-19T04:16:30.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.528+0000 7fc34259c640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3240678c0 0x7fc324069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.528+0000 7fc34259c640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3240678c0 0x7fc324069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc334003d60 tx=0x7fc334009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.532+0000 7fc332ffd640 1 -- 172.21.15.76:0/899740791 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc3380187e0 con 0x7fc3441520a0 2024-09-19T04:16:30.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.700+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc34406e540 con 0x7fc3441520a0 2024-09-19T04:16:30.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.700+0000 7fc332ffd640 1 -- 172.21.15.76:0/899740791 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7fc3380ad340 con 0x7fc3441520a0 2024-09-19T04:16:30.704 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3240678c0 msgr2=0x7fc324069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3240678c0 0x7fc324069d80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc334003d60 tx=0x7fc334009040 comp rx=0 tx=0).stop 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3441520a0 msgr2=0x7fc34414c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3441520a0 0x7fc34414c2d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc33800b2c0 tx=0x7fc33801b960 comp rx=0 tx=0).stop 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 shutdown_connections 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc3240678c0 0x7fc324069d80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc34415f1d0 0x7fc3440cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc3441520a0 0x7fc34414c2d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 --2- 172.21.15.76:0/899740791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc344070020 0x7fc344147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 >> 172.21.15.76:0/899740791 conn(0x7fc3440b1f90 msgr2=0x7fc344071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:30.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 shutdown_connections 2024-09-19T04:16:30.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.704+0000 7fc348c15640 1 -- 172.21.15.76:0/899740791 wait complete. 2024-09-19T04:16:30.724 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-09-19T04:16:30.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.828+0000 7fe27231d640 1 Processor -- start 2024-09-19T04:16:30.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.828+0000 7fe27231d640 1 -- start start 2024-09-19T04:16:30.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe27231d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe26c15bec0 0x7fe26c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe27231d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26c0b3290 con 0x7fe26c15c2a0 2024-09-19T04:16:30.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe27231d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe26c0b3570 con 0x7fe26c0af470 2024-09-19T04:16:30.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe27231d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe26c0b3850 con 0x7fe26c15bec0 2024-09-19T04:16:30.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe26c15bec0 0x7fe26c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26b7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe26c15c2a0 0x7fe26c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36666/0 (socket says 172.21.15.76:36666) 2024-09-19T04:16:30.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26b7fe640 1 -- 172.21.15.76:0/1035775406 learned_addr learned my addr 172.21.15.76:0/1035775406 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:30.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26affd640 1 -- 172.21.15.76:0/1035775406 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3191589721 0 0) 0x7fe26c0b3290 con 0x7fe26c15c2a0 2024-09-19T04:16:30.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26affd640 1 -- 172.21.15.76:0/1035775406 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe254003530 con 0x7fe26c15c2a0 2024-09-19T04:16:30.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26bfff640 1 -- 172.21.15.76:0/1035775406 >> v1:172.21.15.116:6789/0 conn(0x7fe26c0af470 legacy=0x7fe26c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26bfff640 1 -- 172.21.15.76:0/1035775406 >> v1:172.21.15.76:6789/0 conn(0x7fe26c15c2a0 legacy=0x7fe26c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26bfff640 1 -- 172.21.15.76:0/1035775406 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c0b3b30 con 0x7fe26c15bec0 2024-09-19T04:16:30.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.832+0000 7fe26bfff640 1 --2- 172.21.15.76:0/1035775406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15bec0 0x7fe26c0ad200 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe2600097c0 tx=0x7fe26001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d783375a714b267 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe26affd640 1 -- 172.21.15.76:0/1035775406 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2600b2070 con 0x7fe26c15bec0 2024-09-19T04:16:30.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe26affd640 1 -- 172.21.15.76:0/1035775406 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2600ad680 con 0x7fe26c15bec0 2024-09-19T04:16:30.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe26affd640 1 -- 172.21.15.76:0/1035775406 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2600ada50 con 0x7fe26c15bec0 2024-09-19T04:16:30.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- 172.21.15.76:0/1035775406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15bec0 msgr2=0x7fe26c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 --2- 172.21.15.76:0/1035775406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15bec0 0x7fe26c0ad200 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fe2600097c0 tx=0x7fe26001c470 comp rx=0 tx=0).stop 2024-09-19T04:16:30.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- 172.21.15.76:0/1035775406 shutdown_connections 2024-09-19T04:16:30.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 --2- 172.21.15.76:0/1035775406 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15bec0 0x7fe26c0ad200 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- 172.21.15.76:0/1035775406 >> 172.21.15.76:0/1035775406 conn(0x7fe26c0b1f90 msgr2=0x7fe26c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:30.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- 172.21.15.76:0/1035775406 shutdown_connections 2024-09-19T04:16:30.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- 172.21.15.76:0/1035775406 wait complete. 2024-09-19T04:16:30.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 Processor -- start 2024-09-19T04:16:30.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- start start 2024-09-19T04:16:30.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe26c0af470 0x7fe26c1444b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe26c15bec0 0x7fe26c1649f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15c2a0 0x7fe26c15a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c06ef20 con 0x7fe26c15c2a0 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe26c0ad740 con 0x7fe26c0af470 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe26c06f0a0 con 0x7fe26c15bec0 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15c2a0 0x7fe26c15a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15c2a0 0x7fe26c15a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51494/0 (socket says 172.21.15.76:51494) 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 -- 172.21.15.76:0/1324387600 learned_addr learned my addr 172.21.15.76:0/1324387600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe26b7fe640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe26c15bec0 0x7fe26c1649f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe26bfff640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe26c0af470 0x7fe26c1444b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 -- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe26c15bec0 msgr2=0x7fe26c1649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe26c15bec0 0x7fe26c1649f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 -- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe26c0af470 msgr2=0x7fe26c1444b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe26c0af470 0x7fe26c1444b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 -- 172.21.15.76:0/1324387600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe26c165360 con 0x7fe26c15c2a0 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe2708ba640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15c2a0 0x7fe26c15a400 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe26400c770 tx=0x7fe26400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe268ff9640 1 -- 172.21.15.76:0/1324387600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe264003b80 con 0x7fe26c15c2a0 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe268ff9640 1 -- 172.21.15.76:0/1324387600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe264003d20 con 0x7fe26c15c2a0 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe26c165640 con 0x7fe26c15c2a0 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe268ff9640 1 -- 172.21.15.76:0/1324387600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26400aa90 con 0x7fe26c15c2a0 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.836+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe26c165c00 con 0x7fe26c15c2a0 2024-09-19T04:16:30.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.840+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe26c0ad200 con 0x7fe26c15c2a0 2024-09-19T04:16:30.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.840+0000 7fe268ff9640 1 -- 172.21.15.76:0/1324387600 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe26400ac30 con 0x7fe26c15c2a0 2024-09-19T04:16:30.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.840+0000 7fe268ff9640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe2580678c0 0x7fe258069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:30.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.840+0000 7fe268ff9640 1 -- 172.21.15.76:0/1324387600 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5507+0+0 (secure 0 0 0) 0x7fe2640859b0 con 0x7fe26c15c2a0 2024-09-19T04:16:30.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.840+0000 7fe26bfff640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe2580678c0 0x7fe258069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:30.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.840+0000 7fe26bfff640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe2580678c0 0x7fe258069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2600af000 tx=0x7fe2600a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:30.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:30.844+0000 7fe268ff9640 1 -- 172.21.15.76:0/1324387600 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe264055f60 con 0x7fe26c15c2a0 2024-09-19T04:16:31.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.060+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7fe26c0b2bd0 con 0x7fe26c15c2a0 2024-09-19T04:16:31.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe268ff9640 1 -- 172.21.15.76:0/1324387600 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v168) ==== 145+0+0 (secure 0 0 0) 0x7fe26405a260 con 0x7fe26c15c2a0 2024-09-19T04:16:31.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe2580678c0 msgr2=0x7fe258069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe2580678c0 0x7fe258069d80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2600af000 tx=0x7fe2600a7000 comp rx=0 tx=0).stop 2024-09-19T04:16:31.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15c2a0 msgr2=0x7fe26c15a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15c2a0 0x7fe26c15a400 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fe26400c770 tx=0x7fe26400cc30 comp rx=0 tx=0).stop 2024-09-19T04:16:31.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 shutdown_connections 2024-09-19T04:16:31.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe2580678c0 0x7fe258069d80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe26c15c2a0 0x7fe26c15a400 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe26c15bec0 0x7fe26c1649f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 --2- 172.21.15.76:0/1324387600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe26c0af470 0x7fe26c1444b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 >> 172.21.15.76:0/1324387600 conn(0x7fe26c0b1f90 msgr2=0x7fe26c159920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 shutdown_connections 2024-09-19T04:16:31.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.076+0000 7fe27231d640 1 -- 172.21.15.76:0/1324387600 wait complete. 2024-09-19T04:16:31.097 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T04:16:31.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d33a3640 1 Processor -- start 2024-09-19T04:16:31.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d33a3640 1 -- start start 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d33a3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f04cc070020 0x7f04cc15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d33a3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04cc15f1d0 0x7f04cc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d33a3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04cc0aede0 con 0x7f04cc1520a0 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d33a3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f04cc152480 con 0x7f04cc15f1d0 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d33a3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f04cc15fa10 con 0x7f04cc070020 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d1940640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04cc15f1d0 0x7f04cc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d113f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f04cc1520a0 0x7f04cc06f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36682/0 (socket says 172.21.15.76:36682) 2024-09-19T04:16:31.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d113f640 1 -- 172.21.15.76:0/243711252 learned_addr learned my addr 172.21.15.76:0/243711252 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:31.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d093e640 1 --2- 172.21.15.76:0/243711252 >> v2:172.21.15.116:3300/0 conn(0x7f04cc070020 0x7f04cc15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d1940640 1 -- 172.21.15.76:0/243711252 >> v2:172.21.15.116:3300/0 conn(0x7f04cc070020 msgr2=0x7f04cc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d1940640 1 --2- 172.21.15.76:0/243711252 >> v2:172.21.15.116:3300/0 conn(0x7f04cc070020 0x7f04cc15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d1940640 1 -- 172.21.15.76:0/243711252 >> v1:172.21.15.76:6789/0 conn(0x7f04cc1520a0 legacy=0x7f04cc06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04c3fff640 1 -- 172.21.15.76:0/243711252 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2848485072 0 0) 0x7f04cc0aede0 con 0x7f04cc1520a0 2024-09-19T04:16:31.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.212+0000 7f04d1940640 1 -- 172.21.15.76:0/243711252 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc074250 con 0x7f04cc15f1d0 2024-09-19T04:16:31.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d1940640 1 --2- 172.21.15.76:0/243711252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc15f1d0 0x7f04cc0adc80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f04cc06e510 tx=0x7f04c401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1de79e422cd8ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04c3fff640 1 -- 172.21.15.76:0/243711252 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40b2070 con 0x7f04cc15f1d0 2024-09-19T04:16:31.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04c3fff640 1 -- 172.21.15.76:0/243711252 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c40a59c0 con 0x7f04cc15f1d0 2024-09-19T04:16:31.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04c3fff640 1 -- 172.21.15.76:0/243711252 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c40a5d90 con 0x7f04cc15f1d0 2024-09-19T04:16:31.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- 172.21.15.76:0/243711252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc15f1d0 msgr2=0x7f04cc0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 --2- 172.21.15.76:0/243711252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc15f1d0 0x7f04cc0adc80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f04cc06e510 tx=0x7f04c401c900 comp rx=0 tx=0).stop 2024-09-19T04:16:31.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- 172.21.15.76:0/243711252 shutdown_connections 2024-09-19T04:16:31.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 --2- 172.21.15.76:0/243711252 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc15f1d0 0x7f04cc0adc80 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 --2- 172.21.15.76:0/243711252 >> v2:172.21.15.116:3300/0 conn(0x7f04cc070020 0x7f04cc15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- 172.21.15.76:0/243711252 >> 172.21.15.76:0/243711252 conn(0x7f04cc0b1f90 msgr2=0x7f04cc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- 172.21.15.76:0/243711252 shutdown_connections 2024-09-19T04:16:31.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- 172.21.15.76:0/243711252 wait complete. 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 Processor -- start 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- start start 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc070020 0x7f04cc1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04cc1520a0 0x7f04cc1f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04cc15f1d0 0x7f04cc0cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc15f680 con 0x7f04cc070020 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04cc073d30 con 0x7f04cc1520a0 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d33a3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f04cc167f10 con 0x7f04cc15f1d0 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc070020 0x7f04cc1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc070020 0x7f04cc1f5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51514/0 (socket says 172.21.15.76:51514) 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 -- 172.21.15.76:0/573691129 learned_addr learned my addr 172.21.15.76:0/573691129 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d093e640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04cc1520a0 0x7f04cc1f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d1940640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04cc15f1d0 0x7f04cc0cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 -- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04cc15f1d0 msgr2=0x7f04cc0cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04cc15f1d0 0x7f04cc0cf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 -- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04cc1520a0 msgr2=0x7f04cc1f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04cc1520a0 0x7f04cc1f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 -- 172.21.15.76:0/573691129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04cc0cd7f0 con 0x7f04cc070020 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04d113f640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc070020 0x7f04cc1f5570 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f04c80011f0 tx=0x7f04c800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.216+0000 7f04c1ffb640 1 -- 172.21.15.76:0/573691129 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8018070 con 0x7f04cc070020 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04c1ffb640 1 -- 172.21.15.76:0/573691129 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c80074e0 con 0x7f04cc070020 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04c1ffb640 1 -- 172.21.15.76:0/573691129 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8004a10 con 0x7f04cc070020 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04cc0cdad0 con 0x7f04cc070020 2024-09-19T04:16:31.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04cc0ce090 con 0x7f04cc070020 2024-09-19T04:16:31.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04cc15ec90 con 0x7f04cc070020 2024-09-19T04:16:31.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04c1ffb640 1 -- 172.21.15.76:0/573691129 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f04c801b070 con 0x7f04cc070020 2024-09-19T04:16:31.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04c1ffb640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04b00678c0 0x7f04b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04c1ffb640 1 -- 172.21.15.76:0/573691129 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5507+0+0 (secure 0 0 0) 0x7f04c8084f70 con 0x7f04cc070020 2024-09-19T04:16:31.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04d093e640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04b00678c0 0x7f04b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.220+0000 7f04d093e640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04b00678c0 0x7f04b0069d80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f04cc15b010 tx=0x7f04bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.224+0000 7f04c1ffb640 1 -- 172.21.15.76:0/573691129 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f04c8055b70 con 0x7f04cc070020 2024-09-19T04:16:31.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.372+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f04cc06e540 con 0x7f04cc070020 2024-09-19T04:16:31.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.376+0000 7f04c1ffb640 1 -- 172.21.15.76:0/573691129 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f04c8055470 con 0x7f04cc070020 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04b00678c0 msgr2=0x7f04b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04b00678c0 0x7f04b0069d80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f04cc15b010 tx=0x7f04bc008040 comp rx=0 tx=0).stop 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc070020 msgr2=0x7f04cc1f5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc070020 0x7f04cc1f5570 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f04c80011f0 tx=0x7f04c800c9b0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 shutdown_connections 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04b00678c0 0x7f04b0069d80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04cc15f1d0 0x7f04cc0cf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04cc1520a0 0x7f04cc1f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 --2- 172.21.15.76:0/573691129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04cc070020 0x7f04cc1f5570 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 >> 172.21.15.76:0/573691129 conn(0x7f04cc0b1f90 msgr2=0x7f04cc071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 shutdown_connections 2024-09-19T04:16:31.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.380+0000 7f04d33a3640 1 -- 172.21.15.76:0/573691129 wait complete. 2024-09-19T04:16:31.399 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:16:31.399 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:16:31.479 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:16:31.556 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:16:31.556 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:16:31.556 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:31.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:16:31.562 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:16:31.562 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:31.568 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:16:31.568 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:16:31.568 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:31.610 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:16:31.611 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:16:31.611 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:16:31.618 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:16:31.619 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T04:16:31.619 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:31.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff7244640 1 Processor -- start 2024-09-19T04:16:31.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff7244640 1 -- start start 2024-09-19T04:16:31.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff7244640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 0x7f0ff00b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff7244640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0ff006e680 0x7f0ff0072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff7244640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ff006ec60 con 0x7f0ff00b37d0 2024-09-19T04:16:31.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff7244640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0ff006f870 con 0x7f0ff006e680 2024-09-19T04:16:31.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff7244640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff00b3c10 con 0x7f0ff01587a0 2024-09-19T04:16:31.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0feffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0ff00b37d0 0x7f0ff006df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36692/0 (socket says 172.21.15.76:36692) 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0feffff640 1 -- 172.21.15.76:0/289859073 learned_addr learned my addr 172.21.15.76:0/289859073 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2610861628 0 0) 0x7f0ff006ec60 con 0x7f0ff00b37d0 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff4fe0640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 0x7f0ff00b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0fe0003530 con 0x7f0ff00b37d0 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff57e1640 1 --2- 172.21.15.76:0/289859073 >> v2:172.21.15.116:3300/0 conn(0x7f0ff006e680 0x7f0ff0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1262964957 0 0) 0x7f0fe0003530 con 0x7f0ff00b37d0 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 msgr2=0x7f0ff00b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 0x7f0ff00b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 >> v2:172.21.15.116:3300/0 conn(0x7f0ff006e680 msgr2=0x7f0ff0072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 --2- 172.21.15.76:0/289859073 >> v2:172.21.15.116:3300/0 conn(0x7f0ff006e680 0x7f0ff0072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff006ef40 con 0x7f0ff00b37d0 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f0fdc001680 con 0x7f0ff00b37d0 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 >> v1:172.21.15.76:6789/0 conn(0x7f0ff00b37d0 legacy=0x7f0ff006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fe00039c0 0x7f0fe0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 0x7f0fe0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe0004d50 0x7f0fe00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff006f870 con 0x7f0fe0004d50 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff00b3c10 con 0x7f0fe00039c0 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe000b9d0 con 0x7f0ff01587a0 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff57e1640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe0004d50 0x7f0fe00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff4fe0640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fe00039c0 0x7f0fe0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0feffff640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 0x7f0fe0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff57e1640 1 -- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 msgr2=0x7f0fe0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff57e1640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 0x7f0fe0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff57e1640 1 -- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fe00039c0 msgr2=0x7f0fe0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:31.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff57e1640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0fe00039c0 0x7f0fe0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0ff57e1640 1 -- 172.21.15.76:0/289859073 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe0003530 con 0x7f0fe0004d50 2024-09-19T04:16:31.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.776+0000 7f0feffff640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01587a0 0x7f0fe0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:31.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff57e1640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe0004d50 0x7f0fe00051d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009fc0 tx=0x7f0fe801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a837e0b9b8289bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80b2070 con 0x7f0fe0004d50 2024-09-19T04:16:31.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe80a5cd0 con 0x7f0fe0004d50 2024-09-19T04:16:31.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0fef7fe640 1 -- 172.21.15.76:0/289859073 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8009c10 con 0x7f0fe0004d50 2024-09-19T04:16:31.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- 172.21.15.76:0/289859073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe0004d50 msgr2=0x7f0fe00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 --2- 172.21.15.76:0/289859073 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0fe0004d50 0x7f0fe00051d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009fc0 tx=0x7f0fe801ccc0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff57e1640 1 -- 172.21.15.76:0/289859073 reap_dead start 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- 172.21.15.76:0/289859073 shutdown_connections 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- 172.21.15.76:0/289859073 >> 172.21.15.76:0/289859073 conn(0x7f0ff00b1f90 msgr2=0x7f0ff00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- 172.21.15.76:0/289859073 shutdown_connections 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- 172.21.15.76:0/289859073 wait complete. 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 Processor -- start 2024-09-19T04:16:31.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- start start 2024-09-19T04:16:31.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff0099940 0x7f0ff01501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ff01ea730 0x7f0ff00b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01eab90 0x7f0ff00b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0158b80 con 0x7f0ff0099940 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff006f500 con 0x7f0ff01ea730 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff7244640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0ff01546e0 con 0x7f0ff01eab90 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff0099940 0x7f0ff01501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff0099940 0x7f0ff01501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51522/0 (socket says 172.21.15.76:51522) 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 -- 172.21.15.76:0/907203330 learned_addr learned my addr 172.21.15.76:0/907203330 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0feffff640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ff01ea730 0x7f0ff00b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 -- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01eab90 msgr2=0x7f0ff00b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff57e1640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01eab90 0x7f0ff00b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01eab90 0x7f0ff00b0d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 -- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ff01ea730 msgr2=0x7f0ff00b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ff01ea730 0x7f0ff00b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.780+0000 7f0ff4fe0640 1 -- 172.21.15.76:0/907203330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff006e390 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0ff4fe0640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff0099940 0x7f0ff01501b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4011590 tx=0x7f0fe4011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0fed7fa640 1 -- 172.21.15.76:0/907203330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe4015930 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0fed7fa640 1 -- 172.21.15.76:0/907203330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fe4015ad0 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff00d0890 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0fed7fa640 1 -- 172.21.15.76:0/907203330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe400f460 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff0153db0 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff00b3290 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0fed7fa640 1 -- 172.21.15.76:0/907203330 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0fe400f600 con 0x7f0ff0099940 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0fed7fa640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0fd40678c0 0x7f0fd4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:31.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0feffff640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0fd40678c0 0x7f0fd4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:31.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0fed7fa640 1 -- 172.21.15.76:0/907203330 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5507+0+0 (secure 0 0 0) 0x7f0fe40864d0 con 0x7f0ff0099940 2024-09-19T04:16:31.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.784+0000 7f0feffff640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0fd40678c0 0x7f0fd4069d80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc001ec0 tx=0x7f0fdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.788+0000 7f0fed7fa640 1 -- 172.21.15.76:0/907203330 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0fe4019030 con 0x7f0ff0099940 2024-09-19T04:16:31.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.984+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ff0074ba0 con 0x7f0ff0099940 2024-09-19T04:16:31.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0fed7fa640 1 -- 172.21.15.76:0/907203330 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+3917 (secure 0 0 0) 0x7f0fe40570d0 con 0x7f0ff0099940 2024-09-19T04:16:31.991 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:31.991 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":41,"btime":"2024-09-19T04:16:26:766155+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3682762522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3682762522},{"type":"v1","addr":"172.21.15.116:6833","nonce":3682762522}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":4982,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/1084923567","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1084923567},{"type":"v1","addr":"172.21.15.116:6835","nonce":1084923567}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1570942891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1570942891},{"type":"v1","addr":"172.21.15.76:6835","nonce":1570942891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6066,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1595871837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1595871837},{"type":"v1","addr":"172.21.15.76:6837","nonce":1595871837}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[]} 2024-09-19T04:16:31.991 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 41 2024-09-19T04:16:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0fd40678c0 msgr2=0x7f0fd4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0fd40678c0 0x7f0fd4069d80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc001ec0 tx=0x7f0fdc009040 comp rx=0 tx=0).stop 2024-09-19T04:16:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff0099940 msgr2=0x7f0ff01501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff0099940 0x7f0ff01501b0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4011590 tx=0x7f0fe4011a50 comp rx=0 tx=0).stop 2024-09-19T04:16:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 shutdown_connections 2024-09-19T04:16:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0fd40678c0 0x7f0fd4069d80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ff01eab90 0x7f0ff00b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ff01ea730 0x7f0ff00b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 --2- 172.21.15.76:0/907203330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0ff0099940 0x7f0ff01501b0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 >> 172.21.15.76:0/907203330 conn(0x7f0ff00b1f90 msgr2=0x7f0ff0160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 shutdown_connections 2024-09-19T04:16:31.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:31.988+0000 7f0ff7244640 1 -- 172.21.15.76:0/907203330 wait complete. 2024-09-19T04:16:32.010 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T04:16:32.010 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T04:16:32.011 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T04:16:32.011 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:16:32.011 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T04:16:32.011 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T04:16:32.012 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T04:16:32.012 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:16:32.012 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T04:16:32.012 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T04:16:32.012 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T04:16:32.012 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:16:32.012 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T04:16:32.013 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T04:16:32.013 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T04:16:32.013 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:16:32.023 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T04:16:32.023 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T04:16:32.025 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T04:16:32.025 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T04:16:32.025 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T04:16:32.027 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T04:16:32.067 INFO:tasks.ceph.mds.c:Started 2024-09-19T04:16:32.067 DEBUG:teuthology.parallel:result is None 2024-09-19T04:16:32.068 INFO:tasks.ceph.mds.b:Started 2024-09-19T04:16:32.068 DEBUG:teuthology.parallel:result is None 2024-09-19T04:16:32.068 INFO:tasks.ceph.mds.d:Started 2024-09-19T04:16:32.069 DEBUG:teuthology.parallel:result is None 2024-09-19T04:16:32.134 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:16:32.137 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:16:32.142 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:16:36.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-09-19T04:16:38.024 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T04:16:38.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T04:16:38.027 INFO:tasks.ceph.mds.a:Started 2024-09-19T04:16:38.027 DEBUG:teuthology.parallel:result is None 2024-09-19T04:16:38.027 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T04:16:38.099 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:16:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 Processor -- start 2024-09-19T04:16:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 -- start start 2024-09-19T04:16:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f21c01520a0 0x7f21c006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21c015f1d0 0x7f21c0071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f21c0152480 con 0x7f21c015f1d0 2024-09-19T04:16:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f21c0070250 con 0x7f21c01520a0 2024-09-19T04:16:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c798c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f21c00aede0 con 0x7f21c006fe50 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21c015f1d0 0x7f21c0071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f21c015f1d0 0x7f21c0071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59512/0 (socket says 172.21.15.76:59512) 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 -- 172.21.15.76:0/1462976847 learned_addr learned my addr 172.21.15.76:0/1462976847 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c4f27640 1 --2- 172.21.15.76:0/1462976847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5728640 1 --2- 172.21.15.76:0/1462976847 >> v2:172.21.15.116:3300/0 conn(0x7f21c01520a0 0x7f21c006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 -- 172.21.15.76:0/1462976847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 msgr2=0x7f21c015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 --2- 172.21.15.76:0/1462976847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 -- 172.21.15.76:0/1462976847 >> v2:172.21.15.116:3300/0 conn(0x7f21c01520a0 msgr2=0x7f21c006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:38.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 --2- 172.21.15.76:0/1462976847 >> v2:172.21.15.116:3300/0 conn(0x7f21c01520a0 0x7f21c006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.156+0000 7f21c5f29640 1 -- 172.21.15.76:0/1462976847 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c015fa10 con 0x7f21c015f1d0 2024-09-19T04:16:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c5f29640 1 --2- 172.21.15.76:0/1462976847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c0071f10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f21c0071710 tx=0x7f21b801e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc22d1697343e3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21b7fff640 1 -- 172.21.15.76:0/1462976847 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800b2d0 con 0x7f21c015f1d0 2024-09-19T04:16:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21b7fff640 1 -- 172.21.15.76:0/1462976847 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b800b470 con 0x7f21c015f1d0 2024-09-19T04:16:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21b7fff640 1 -- 172.21.15.76:0/1462976847 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b80a79c0 con 0x7f21c015f1d0 2024-09-19T04:16:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 -- 172.21.15.76:0/1462976847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 msgr2=0x7f21c0071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 --2- 172.21.15.76:0/1462976847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c0071f10 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f21c0071710 tx=0x7f21b801e910 comp rx=0 tx=0).stop 2024-09-19T04:16:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 -- 172.21.15.76:0/1462976847 shutdown_connections 2024-09-19T04:16:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 --2- 172.21.15.76:0/1462976847 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c0071f10 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 --2- 172.21.15.76:0/1462976847 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:38.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 --2- 172.21.15.76:0/1462976847 >> v2:172.21.15.116:3300/0 conn(0x7f21c01520a0 0x7f21c006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:38.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 -- 172.21.15.76:0/1462976847 >> 172.21.15.76:0/1462976847 conn(0x7f21c00b1f90 msgr2=0x7f21c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:38.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 -- 172.21.15.76:0/1462976847 shutdown_connections 2024-09-19T04:16:38.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 -- 172.21.15.76:0/1462976847 wait complete. 2024-09-19T04:16:38.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 Processor -- start 2024-09-19T04:16:38.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.160+0000 7f21c798c640 1 -- start start 2024-09-19T04:16:38.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c0151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:38.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01520a0 0x7f21c00cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:38.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c00cf800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:38.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0073d30 con 0x7f21c015f1d0 2024-09-19T04:16:38.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f21c0073a50 con 0x7f21c01520a0 2024-09-19T04:16:38.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f21c015f680 con 0x7f21c006fe50 2024-09-19T04:16:38.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c00cf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:38.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c00cf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59528/0 (socket says 172.21.15.76:59528) 2024-09-19T04:16:38.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 -- 172.21.15.76:0/3236749634 learned_addr learned my addr 172.21.15.76:0/3236749634 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:38.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5728640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c0151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:38.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 -- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 msgr2=0x7f21c0151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:38.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c0151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:38.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 -- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01520a0 msgr2=0x7f21c00cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:38.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c4f27640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01520a0 0x7f21c00cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:38.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01520a0 0x7f21c00cf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:38.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 -- 172.21.15.76:0/3236749634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c00cdd30 con 0x7f21c015f1d0 2024-09-19T04:16:38.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5728640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c0151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:38.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5f29640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c00cf800 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f21b80b1040 tx=0x7f21b801b6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:38.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21b5ffb640 1 -- 172.21.15.76:0/3236749634 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800b720 con 0x7f21c015f1d0 2024-09-19T04:16:38.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21b5ffb640 1 -- 172.21.15.76:0/3236749634 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21b8009600 con 0x7f21c015f1d0 2024-09-19T04:16:38.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c014cc80 con 0x7f21c015f1d0 2024-09-19T04:16:38.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21b5ffb640 1 -- 172.21.15.76:0/3236749634 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8009a30 con 0x7f21c015f1d0 2024-09-19T04:16:38.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c014d240 con 0x7f21c015f1d0 2024-09-19T04:16:38.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c006e540 con 0x7f21c015f1d0 2024-09-19T04:16:38.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21b5ffb640 1 -- 172.21.15.76:0/3236749634 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f21b80c3020 con 0x7f21c015f1d0 2024-09-19T04:16:38.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21b5ffb640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f21a40678c0 0x7f21a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:38.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21b5ffb640 1 -- 172.21.15.76:0/3236749634 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f21b81220b0 con 0x7f21c015f1d0 2024-09-19T04:16:38.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5728640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f21a40678c0 0x7f21a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:38.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.164+0000 7f21c5728640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f21a40678c0 0x7f21a4069d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21c009deb0 tx=0x7f21bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:38.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.168+0000 7f21b5ffb640 1 -- 172.21.15.76:0/3236749634 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f21b80ca020 con 0x7f21c015f1d0 2024-09-19T04:16:38.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:38.344+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f21c00ae680 con 0x7f21c015f1d0 2024-09-19T04:16:39.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.856+0000 7f21b5ffb640 1 -- 172.21.15.76:0/3236749634 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7f21b80187c0 con 0x7f21c015f1d0 2024-09-19T04:16:39.860 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2024-09-19T04:16:39.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f21a40678c0 msgr2=0x7f21a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:39.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f21a40678c0 0x7f21a4069d80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f21c009deb0 tx=0x7f21bc008040 comp rx=0 tx=0).stop 2024-09-19T04:16:39.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 msgr2=0x7f21c00cf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:39.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c00cf800 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f21b80b1040 tx=0x7f21b801b6b0 comp rx=0 tx=0).stop 2024-09-19T04:16:39.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 shutdown_connections 2024-09-19T04:16:39.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f21a40678c0 0x7f21a4069d80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:39.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f21c015f1d0 0x7f21c00cf800 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:39.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f21c01520a0 0x7f21c00cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:39.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 --2- 172.21.15.76:0/3236749634 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f21c006fe50 0x7f21c0151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:39.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 >> 172.21.15.76:0/3236749634 conn(0x7f21c00b1f90 msgr2=0x7f21c00b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:39.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 shutdown_connections 2024-09-19T04:16:39.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:39.860+0000 7f21c798c640 1 -- 172.21.15.76:0/3236749634 wait complete. 2024-09-19T04:16:39.887 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T04:16:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f6c06c640 1 Processor -- start 2024-09-19T04:16:40.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f6c06c640 1 -- start start 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f6c06c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7f641457a0 0x7f7f640d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f6c06c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f640d27a0 con 0x7f7f640cf800 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f6c06c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f640d2a80 con 0x7f7f640b4510 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f6c06c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7f640d2d60 con 0x7f7f641457a0 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f69e08640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7f641457a0 0x7f7f640d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f69607640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7f640cf800 0x7f7f640b3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34722/0 (socket says 172.21.15.76:34722) 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f69607640 1 -- 172.21.15.76:0/1980615662 learned_addr learned my addr 172.21.15.76:0/1980615662 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f69e08640 1 -- 172.21.15.76:0/1980615662 >> v1:172.21.15.116:6789/0 conn(0x7f7f640b4510 legacy=0x7f7f640d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f69e08640 1 -- 172.21.15.76:0/1980615662 >> v1:172.21.15.76:6789/0 conn(0x7f7f640cf800 legacy=0x7f7f640b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f68e06640 1 -- 172.21.15.76:0/1980615662 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4259751729 0 0) 0x7f7f640d27a0 con 0x7f7f640cf800 2024-09-19T04:16:40.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.008+0000 7f7f69e08640 1 -- 172.21.15.76:0/1980615662 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640d01f0 con 0x7f7f641457a0 2024-09-19T04:16:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69e08640 1 --2- 172.21.15.76:0/1980615662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f641457a0 0x7f7f640d1ea0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097f0 tx=0x7f7f6001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af8fff0f6f30e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f68e06640 1 -- 172.21.15.76:0/1980615662 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f600b0070 con 0x7f7f641457a0 2024-09-19T04:16:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f68e06640 1 -- 172.21.15.76:0/1980615662 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f600ab600 con 0x7f7f641457a0 2024-09-19T04:16:40.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f68e06640 1 -- 172.21.15.76:0/1980615662 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f600a4e60 con 0x7f7f641457a0 2024-09-19T04:16:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- 172.21.15.76:0/1980615662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f641457a0 msgr2=0x7f7f640d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/1980615662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f641457a0 0x7f7f640d1ea0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f7f600097f0 tx=0x7f7f6001c3d0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- 172.21.15.76:0/1980615662 shutdown_connections 2024-09-19T04:16:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/1980615662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f641457a0 0x7f7f640d1ea0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- 172.21.15.76:0/1980615662 >> 172.21.15.76:0/1980615662 conn(0x7f7f6406bcc0 msgr2=0x7f7f6406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- 172.21.15.76:0/1980615662 shutdown_connections 2024-09-19T04:16:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- 172.21.15.76:0/1980615662 wait complete. 2024-09-19T04:16:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 Processor -- start 2024-09-19T04:16:40.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- start start 2024-09-19T04:16:40.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f640b4510 0x7f7f640c8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f640cf800 0x7f7f641414f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7f641457a0 0x7f7f6413d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6416a450 con 0x7f7f640cf800 2024-09-19T04:16:40.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6416a2d0 con 0x7f7f640b4510 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f6c06c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6416a5d0 con 0x7f7f641457a0 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69607640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f640cf800 0x7f7f641414f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69607640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f640cf800 0x7f7f641414f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59550/0 (socket says 172.21.15.76:59550) 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69607640 1 -- 172.21.15.76:0/2773136556 learned_addr learned my addr 172.21.15.76:0/2773136556 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69607640 1 -- 172.21.15.76:0/2773136556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7f641457a0 msgr2=0x7f7f6413d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69607640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7f641457a0 0x7f7f6413d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69e08640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f640b4510 0x7f7f640c8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69607640 1 -- 172.21.15.76:0/2773136556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f640b4510 msgr2=0x7f7f640c8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.012+0000 7f7f69607640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f640b4510 0x7f7f640c8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f69607640 1 -- 172.21.15.76:0/2773136556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f640ca2e0 con 0x7f7f640cf800 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f69607640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f640cf800 0x7f7f641414f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7f540011f0 tx=0x7f7f5400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f5a7fc640 1 -- 172.21.15.76:0/2773136556 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54018070 con 0x7f7f640cf800 2024-09-19T04:16:40.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f5a7fc640 1 -- 172.21.15.76:0/2773136556 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f540074e0 con 0x7f7f640cf800 2024-09-19T04:16:40.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f640ca830 con 0x7f7f640cf800 2024-09-19T04:16:40.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f5a7fc640 1 -- 172.21.15.76:0/2773136556 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f54004a10 con 0x7f7f640cf800 2024-09-19T04:16:40.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f640c88c0 con 0x7f7f640cf800 2024-09-19T04:16:40.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f640d1ea0 con 0x7f7f640cf800 2024-09-19T04:16:40.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f5a7fc640 1 -- 172.21.15.76:0/2773136556 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7f5401b070 con 0x7f7f640cf800 2024-09-19T04:16:40.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f5a7fc640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f480678c0 0x7f7f48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f5a7fc640 1 -- 172.21.15.76:0/2773136556 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7f7f54084ea0 con 0x7f7f640cf800 2024-09-19T04:16:40.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f69e08640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f480678c0 0x7f7f48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.016+0000 7f7f69e08640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f480678c0 0x7f7f48069d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7f600ad000 tx=0x7f7f60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.020+0000 7f7f5a7fc640 1 -- 172.21.15.76:0/2773136556 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7f54055b10 con 0x7f7f640cf800 2024-09-19T04:16:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.240+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7f640cafc0 con 0x7f7f640cf800 2024-09-19T04:16:40.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.240+0000 7f7f5a7fc640 1 -- 172.21.15.76:0/2773136556 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f7f54055410 con 0x7f7f640cf800 2024-09-19T04:16:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:40.245 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2024-09-19T04:16:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2024-09-19T04:16:40.245 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w==", 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA==", 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.246 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw==", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:16:40.247 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA==", 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.248 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ==", 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w==", 2024-09-19T04:16:40.249 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ==", 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.250 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw==", 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA==", 2024-09-19T04:16:40.251 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.252 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.252 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.252 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.252 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw==", 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.253 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg==", 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA==", 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:16:40.254 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg==", 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.255 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg==", 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ==", 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.256 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA==", 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:16:40.257 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCopOtmGhUPLBAA9AkjLm1xF81Mjf1IYlN6Fg==", 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.258 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A==", 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg==", 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:16:40.259 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:16:40.260 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:16:40.260 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.260 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.260 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-19T04:16:40.260 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f480678c0 msgr2=0x7f7f48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f480678c0 0x7f7f48069d80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7f600ad000 tx=0x7f7f60008040 comp rx=0 tx=0).stop 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f640cf800 msgr2=0x7f7f641414f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f640cf800 0x7f7f641414f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f7f540011f0 tx=0x7f7f5400c9b0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 shutdown_connections 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f480678c0 0x7f7f48069d80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7f641457a0 0x7f7f6413d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7f640cf800 0x7f7f641414f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 --2- 172.21.15.76:0/2773136556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f640b4510 0x7f7f640c8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 >> 172.21.15.76:0/2773136556 conn(0x7f7f6406bcc0 msgr2=0x7f7f640b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 shutdown_connections 2024-09-19T04:16:40.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.244+0000 7f7f6c06c640 1 -- 172.21.15.76:0/2773136556 wait complete. 2024-09-19T04:16:40.264 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T04:16:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.368+0000 7fd86dcae640 1 Processor -- start 2024-09-19T04:16:40.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.368+0000 7fd86dcae640 1 -- start start 2024-09-19T04:16:40.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd86dcae640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd86815bec0 0x7fd8680ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd86dcae640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd86815c2a0 0x7fd8680aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd86dcae640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8680af2a0 0x7fd86815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd86dcae640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd8680b3290 con 0x7fd86815bec0 2024-09-19T04:16:40.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd86dcae640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd8680b3570 con 0x7fd86815c2a0 2024-09-19T04:16:40.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd86dcae640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd8680b3850 con 0x7fd8680af2a0 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd86815bec0 0x7fd8680ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd86815bec0 0x7fd8680ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59556/0 (socket says 172.21.15.76:59556) 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 -- 172.21.15.76:0/377160610 learned_addr learned my addr 172.21.15.76:0/377160610 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd866ffd640 1 --2- 172.21.15.76:0/377160610 >> v2:172.21.15.116:3300/0 conn(0x7fd86815c2a0 0x7fd8680aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd867fff640 1 --2- 172.21.15.76:0/377160610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8680af2a0 0x7fd86815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 -- 172.21.15.76:0/377160610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8680af2a0 msgr2=0x7fd86815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 --2- 172.21.15.76:0/377160610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8680af2a0 0x7fd86815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 -- 172.21.15.76:0/377160610 >> v2:172.21.15.116:3300/0 conn(0x7fd86815c2a0 msgr2=0x7fd8680aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 --2- 172.21.15.76:0/377160610 >> v2:172.21.15.116:3300/0 conn(0x7fd86815c2a0 0x7fd8680aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 -- 172.21.15.76:0/377160610 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8680b3b30 con 0x7fd86815bec0 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd866ffd640 1 --2- 172.21.15.76:0/377160610 >> v2:172.21.15.116:3300/0 conn(0x7fd86815c2a0 0x7fd8680aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:40.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd867fff640 1 --2- 172.21.15.76:0/377160610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8680af2a0 0x7fd86815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8677fe640 1 --2- 172.21.15.76:0/377160610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd86815bec0 0x7fd8680ad200 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd8680ae510 tx=0x7fd85801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a328e81718ad90d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8667fc640 1 -- 172.21.15.76:0/377160610 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8580b0070 con 0x7fd86815bec0 2024-09-19T04:16:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8667fc640 1 -- 172.21.15.76:0/377160610 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8580ab600 con 0x7fd86815bec0 2024-09-19T04:16:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.372+0000 7fd8667fc640 1 -- 172.21.15.76:0/377160610 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8580ab9d0 con 0x7fd86815bec0 2024-09-19T04:16:40.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/377160610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd86815bec0 msgr2=0x7fd8680ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 --2- 172.21.15.76:0/377160610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd86815bec0 0x7fd8680ad200 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd8680ae510 tx=0x7fd85801c230 comp rx=0 tx=0).stop 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/377160610 shutdown_connections 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 --2- 172.21.15.76:0/377160610 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8680af2a0 0x7fd86815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 --2- 172.21.15.76:0/377160610 >> v2:172.21.15.116:3300/0 conn(0x7fd86815c2a0 0x7fd8680aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 --2- 172.21.15.76:0/377160610 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd86815bec0 0x7fd8680ad200 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/377160610 >> 172.21.15.76:0/377160610 conn(0x7fd8680b1f90 msgr2=0x7fd8680b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/377160610 shutdown_connections 2024-09-19T04:16:40.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/377160610 wait complete. 2024-09-19T04:16:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 Processor -- start 2024-09-19T04:16:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- start start 2024-09-19T04:16:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8680af2a0 0x7fd86814c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd86815bec0 0x7fd868152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd86815c2a0 0x7fd86815f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8680af6b0 con 0x7fd8680af2a0 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd8680ad740 con 0x7fd86815bec0 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd86806e000 con 0x7fd86815c2a0 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8680af2a0 0x7fd86814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8680af2a0 0x7fd86814c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59566/0 (socket says 172.21.15.76:59566) 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 -- 172.21.15.76:0/3660801082 learned_addr learned my addr 172.21.15.76:0/3660801082 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd867fff640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd86815c2a0 0x7fd86815f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 -- 172.21.15.76:0/3660801082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd86815c2a0 msgr2=0x7fd86815f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd86815c2a0 0x7fd86815f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 -- 172.21.15.76:0/3660801082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd86815bec0 msgr2=0x7fd868152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd86815bec0 0x7fd868152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 -- 172.21.15.76:0/3660801082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8681513a0 con 0x7fd8680af2a0 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd8677fe640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8680af2a0 0x7fd86814c980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8580ae040 tx=0x7fd858018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd84ffff640 1 -- 172.21.15.76:0/3660801082 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8580b0040 con 0x7fd8680af2a0 2024-09-19T04:16:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd84ffff640 1 -- 172.21.15.76:0/3660801082 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8580b9070 con 0x7fd8680af2a0 2024-09-19T04:16:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd868151680 con 0x7fd8680af2a0 2024-09-19T04:16:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd84ffff640 1 -- 172.21.15.76:0/3660801082 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8580b48a0 con 0x7fd8680af2a0 2024-09-19T04:16:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd868151c40 con 0x7fd8680af2a0 2024-09-19T04:16:40.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.376+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8680ad200 con 0x7fd8680af2a0 2024-09-19T04:16:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.380+0000 7fd84ffff640 1 -- 172.21.15.76:0/3660801082 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8580ab4c0 con 0x7fd8680af2a0 2024-09-19T04:16:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.380+0000 7fd84ffff640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8540678c0 0x7fd854069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.380+0000 7fd84ffff640 1 -- 172.21.15.76:0/3660801082 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fd85811f930 con 0x7fd8680af2a0 2024-09-19T04:16:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.380+0000 7fd866ffd640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8540678c0 0x7fd854069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.380+0000 7fd866ffd640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8540678c0 0x7fd854069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd85c006b20 tx=0x7fd85c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.384+0000 7fd84ffff640 1 -- 172.21.15.76:0/3660801082 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd858016110 con 0x7fd8680af2a0 2024-09-19T04:16:40.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd86806fcd0 con 0x7fd8680af2a0 2024-09-19T04:16:40.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd84ffff640 1 -- 172.21.15.76:0/3660801082 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v63) ==== 84+0+65 (secure 0 0 0) 0x7fd8580f0620 con 0x7fd8680af2a0 2024-09-19T04:16:40.551 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:40.551 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:16:40.552 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:16:40.552 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-09-19T04:16:40.552 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-09-19T04:16:40.552 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:40.552 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:16:40.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8540678c0 msgr2=0x7fd854069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8540678c0 0x7fd854069d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd85c006b20 tx=0x7fd85c008040 comp rx=0 tx=0).stop 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8680af2a0 msgr2=0x7fd86814c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8680af2a0 0x7fd86814c980 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd8580ae040 tx=0x7fd858018fd0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 shutdown_connections 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8540678c0 0x7fd854069d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd86815c2a0 0x7fd86815f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd86815bec0 0x7fd868152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 --2- 172.21.15.76:0/3660801082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8680af2a0 0x7fd86814c980 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 >> 172.21.15.76:0/3660801082 conn(0x7fd8680b1f90 msgr2=0x7fd8680cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 shutdown_connections 2024-09-19T04:16:40.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.548+0000 7fd86dcae640 1 -- 172.21.15.76:0/3660801082 wait complete. 2024-09-19T04:16:40.570 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:16:40.570 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:16:40.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbccc1e4640 1 Processor -- start 2024-09-19T04:16:40.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbccc1e4640 1 -- start start 2024-09-19T04:16:40.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbccc1e4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc40d3060 0x7fbcc40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbccc1e4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcc40cf450 0x7fbcc406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbccc1e4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcc40d0300 con 0x7fbcc4161a60 2024-09-19T04:16:40.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbccc1e4640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc40b0f60 con 0x7fbcc40cf450 2024-09-19T04:16:40.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbccc1e4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc40cfd40 con 0x7fbcc40d3060 2024-09-19T04:16:40.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcca781640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcc40cf450 0x7fbcc406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcca781640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbcc40cf450 0x7fbcc406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59574/0 (socket says 172.21.15.76:59574) 2024-09-19T04:16:40.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcca781640 1 -- 172.21.15.76:0/3589513683 learned_addr learned my addr 172.21.15.76:0/3589513683 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:40.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcc9f80640 1 --1- 172.21.15.76:0/3589513683 >> v1:172.21.15.76:6789/0 conn(0x7fbcc4161a60 0x7fbcc40d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34738/0 (socket says 172.21.15.76:34738) 2024-09-19T04:16:40.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcc977f640 1 --2- 172.21.15.76:0/3589513683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc40d3060 0x7fbcc40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcca781640 1 -- 172.21.15.76:0/3589513683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc40d3060 msgr2=0x7fbcc40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:40.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcca781640 1 --2- 172.21.15.76:0/3589513683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc40d3060 0x7fbcc40cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcca781640 1 -- 172.21.15.76:0/3589513683 >> v1:172.21.15.76:6789/0 conn(0x7fbcc4161a60 legacy=0x7fbcc40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcc8f7e640 1 -- 172.21.15.76:0/3589513683 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2751746263 0 0) 0x7fbcbc001120 con 0x7fbcc4161a60 2024-09-19T04:16:40.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcca781640 1 -- 172.21.15.76:0/3589513683 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc40d0020 con 0x7fbcc40cf450 2024-09-19T04:16:40.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.692+0000 7fbcc977f640 1 --2- 172.21.15.76:0/3589513683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc40d3060 0x7fbcc40cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:40.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcca781640 1 --2- 172.21.15.76:0/3589513683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40cf450 0x7fbcc406d6b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab711acd6f6ba4f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcc8f7e640 1 -- 172.21.15.76:0/3589513683 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00a5830 con 0x7fbcc40cf450 2024-09-19T04:16:40.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcc8f7e640 1 -- 172.21.15.76:0/3589513683 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcc00a59d0 con 0x7fbcc40cf450 2024-09-19T04:16:40.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcc8f7e640 1 -- 172.21.15.76:0/3589513683 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc00a5da0 con 0x7fbcc40cf450 2024-09-19T04:16:40.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3589513683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40cf450 msgr2=0x7fbcc406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3589513683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40cf450 0x7fbcc406d6b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0009e50 tx=0x7fbcc001c970 comp rx=0 tx=0).stop 2024-09-19T04:16:40.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3589513683 shutdown_connections 2024-09-19T04:16:40.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3589513683 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40cf450 0x7fbcc406d6b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3589513683 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc40d3060 0x7fbcc40cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3589513683 >> 172.21.15.76:0/3589513683 conn(0x7fbcc408bf90 msgr2=0x7fbcc409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3589513683 shutdown_connections 2024-09-19T04:16:40.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3589513683 wait complete. 2024-09-19T04:16:40.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 Processor -- start 2024-09-19T04:16:40.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- start start 2024-09-19T04:16:40.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcc40cf450 0x7fbcc4153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40d3060 0x7fbcc40cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc4161a60 0x7fbcc4157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc406e400 con 0x7fbcc40d3060 2024-09-19T04:16:40.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc40cfaf0 con 0x7fbcc40cf450 2024-09-19T04:16:40.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbccc1e4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc406e580 con 0x7fbcc4161a60 2024-09-19T04:16:40.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcc977f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40d3060 0x7fbcc40cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcc977f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40d3060 0x7fbcc40cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59588/0 (socket says 172.21.15.76:59588) 2024-09-19T04:16:40.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcc977f640 1 -- 172.21.15.76:0/3602313859 learned_addr learned my addr 172.21.15.76:0/3602313859 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:40.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.696+0000 7fbcc9f80640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcc40cf450 0x7fbcc4153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcca781640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc4161a60 0x7fbcc4157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc977f640 1 -- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc4161a60 msgr2=0x7fbcc4157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc977f640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc4161a60 0x7fbcc4157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc977f640 1 -- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcc40cf450 msgr2=0x7fbcc4153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc977f640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcc40cf450 0x7fbcc4153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc977f640 1 -- 172.21.15.76:0/3602313859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc4158080 con 0x7fbcc40d3060 2024-09-19T04:16:40.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc977f640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40d3060 0x7fbcc40cb440 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c7f0 tx=0x7fbcb400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcba7fc640 1 -- 172.21.15.76:0/3602313859 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4015070 con 0x7fbcc40d3060 2024-09-19T04:16:40.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcba7fc640 1 -- 172.21.15.76:0/3602313859 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb4007820 con 0x7fbcc40d3060 2024-09-19T04:16:40.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc4158360 con 0x7fbcc40d3060 2024-09-19T04:16:40.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcba7fc640 1 -- 172.21.15.76:0/3602313859 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4007c10 con 0x7fbcc40d3060 2024-09-19T04:16:40.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc40b90f0 con 0x7fbcc40d3060 2024-09-19T04:16:40.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc406d6b0 con 0x7fbcc40d3060 2024-09-19T04:16:40.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcba7fc640 1 -- 172.21.15.76:0/3602313859 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbcb4018070 con 0x7fbcc40d3060 2024-09-19T04:16:40.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcba7fc640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbca80678c0 0x7fbca8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:40.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc9f80640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbca80678c0 0x7fbca8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:40.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcba7fc640 1 -- 172.21.15.76:0/3602313859 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5249+0+0 (secure 0 0 0) 0x7fbcb4085c40 con 0x7fbcc40d3060 2024-09-19T04:16:40.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.700+0000 7fbcc9f80640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbca80678c0 0x7fbca8069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc000f70 tx=0x7fbcbc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:40.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.704+0000 7fbcba7fc640 1 -- 172.21.15.76:0/3602313859 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbcb4056950 con 0x7fbcc40d3060 2024-09-19T04:16:40.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.868+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fbcc40cb980 con 0x7fbcc40d3060 2024-09-19T04:16:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbcba7fc640 1 -- 172.21.15.76:0/3602313859 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v64) ==== 138+0+0 (secure 0 0 0) 0x7fbcb4056250 con 0x7fbcc40d3060 2024-09-19T04:16:40.962 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbca80678c0 msgr2=0x7fbca8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbca80678c0 0x7fbca8069d80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc000f70 tx=0x7fbcbc009040 comp rx=0 tx=0).stop 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40d3060 msgr2=0x7fbcc40cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40d3060 0x7fbcc40cb440 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400c7f0 tx=0x7fbcb400ccb0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 shutdown_connections 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbca80678c0 0x7fbca8069d80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbcc4161a60 0x7fbcc4157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbcc40d3060 0x7fbcc40cb440 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 --2- 172.21.15.76:0/3602313859 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbcc40cf450 0x7fbcc4153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 >> 172.21.15.76:0/3602313859 conn(0x7fbcc408bf90 msgr2=0x7fbcc415d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:40.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 shutdown_connections 2024-09-19T04:16:40.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:40.960+0000 7fbccc1e4640 1 -- 172.21.15.76:0/3602313859 wait complete. 2024-09-19T04:16:40.991 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:16:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f968a797640 1 Processor -- start 2024-09-19T04:16:41.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f968a797640 1 -- start start 2024-09-19T04:16:41.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f968a797640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f96840b43a0 0x7f96840b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f968a797640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9684072230 0x7f9684072680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f968a797640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96840cf0f0 con 0x7f968415e640 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f968a797640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f968415ea20 con 0x7f96840b43a0 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f968a797640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f96840cf3c0 con 0x7f9684072230 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f9683fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f968415e640 0x7f96840ce9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34740/0 (socket says 172.21.15.76:34740) 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f9683fff640 1 -- 172.21.15.76:0/4143374164 learned_addr learned my addr 172.21.15.76:0/4143374164 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f96837fe640 1 --2- 172.21.15.76:0/4143374164 >> v2:172.21.15.76:3300/0 conn(0x7f96840b43a0 0x7f96840b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f9688d34640 1 --2- 172.21.15.76:0/4143374164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9684072230 0x7f9684072680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f9682ffd640 1 -- 172.21.15.76:0/4143374164 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3370365095 0 0) 0x7f96840cf0f0 con 0x7f968415e640 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f9682ffd640 1 -- 172.21.15.76:0/4143374164 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f966c003530 con 0x7f968415e640 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f96837fe640 1 -- 172.21.15.76:0/4143374164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9684072230 msgr2=0x7f9684072680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:41.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f96837fe640 1 --2- 172.21.15.76:0/4143374164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9684072230 0x7f9684072680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f96837fe640 1 -- 172.21.15.76:0/4143374164 >> v1:172.21.15.76:6789/0 conn(0x7f968415e640 legacy=0x7f96840ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:41.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.120+0000 7f96837fe640 1 -- 172.21.15.76:0/4143374164 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96840cf6a0 con 0x7f96840b43a0 2024-09-19T04:16:41.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f96837fe640 1 --2- 172.21.15.76:0/4143374164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96840b43a0 0x7f96840b47e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f96780097c0 tx=0x7f967801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420e5bebf1315eed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:41.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f9682ffd640 1 -- 172.21.15.76:0/4143374164 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96780b1070 con 0x7f96840b43a0 2024-09-19T04:16:41.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f9682ffd640 1 -- 172.21.15.76:0/4143374164 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96780ac410 con 0x7f96840b43a0 2024-09-19T04:16:41.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f9682ffd640 1 -- 172.21.15.76:0/4143374164 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96780ac7e0 con 0x7f96840b43a0 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 -- 172.21.15.76:0/4143374164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96840b43a0 msgr2=0x7f96840b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 --2- 172.21.15.76:0/4143374164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96840b43a0 0x7f96840b47e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f96780097c0 tx=0x7f967801c4a0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 -- 172.21.15.76:0/4143374164 shutdown_connections 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 --2- 172.21.15.76:0/4143374164 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9684072230 0x7f9684072680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 --2- 172.21.15.76:0/4143374164 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f96840b43a0 0x7f96840b47e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 -- 172.21.15.76:0/4143374164 >> 172.21.15.76:0/4143374164 conn(0x7f96840b1c80 msgr2=0x7f96840b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 -- 172.21.15.76:0/4143374164 shutdown_connections 2024-09-19T04:16:41.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.124+0000 7f968a797640 1 -- 172.21.15.76:0/4143374164 wait complete. 2024-09-19T04:16:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 Processor -- start 2024-09-19T04:16:41.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 -- start start 2024-09-19T04:16:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9684072230 0x7f9684152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f96840b43a0 0x7f96840ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968415e640 0x7f96840ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f968406f0e0 con 0x7f968415e640 2024-09-19T04:16:41.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f968406ef60 con 0x7f9684072230 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f968416b0f0 con 0x7f96840b43a0 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968415e640 0x7f96840ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968415e640 0x7f96840ad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59608/0 (socket says 172.21.15.76:59608) 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 -- 172.21.15.76:0/2995654597 learned_addr learned my addr 172.21.15.76:0/2995654597 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f96837fe640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f96840b43a0 0x7f96840ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9683fff640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9684072230 0x7f9684152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 -- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f96840b43a0 msgr2=0x7f96840ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f96840b43a0 0x7f96840ae6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 -- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9684072230 msgr2=0x7f9684152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9684072230 0x7f9684152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 -- 172.21.15.76:0/2995654597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9684156140 con 0x7f968415e640 2024-09-19T04:16:41.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f96837fe640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f96840b43a0 0x7f96840ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9683fff640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9684072230 0x7f9684152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9688d34640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968415e640 0x7f96840ad760 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f967c00e820 tx=0x7f967c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9680ff9640 1 -- 172.21.15.76:0/2995654597 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c017070 con 0x7f968415e640 2024-09-19T04:16:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9680ff9640 1 -- 172.21.15.76:0/2995654597 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f967c00c940 con 0x7f968415e640 2024-09-19T04:16:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9684156420 con 0x7f968415e640 2024-09-19T04:16:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f9680ff9640 1 -- 172.21.15.76:0/2995654597 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f967c00cd10 con 0x7f968415e640 2024-09-19T04:16:41.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96841569e0 con 0x7f968415e640 2024-09-19T04:16:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.128+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9684072680 con 0x7f968415e640 2024-09-19T04:16:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.132+0000 7f9680ff9640 1 -- 172.21.15.76:0/2995654597 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f967c01a070 con 0x7f968415e640 2024-09-19T04:16:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.132+0000 7f9680ff9640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f96700678c0 0x7f9670069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.132+0000 7f9680ff9640 1 -- 172.21.15.76:0/2995654597 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5619+0+0 (secure 0 0 0) 0x7f967c0862d0 con 0x7f968415e640 2024-09-19T04:16:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.132+0000 7f9683fff640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f96700678c0 0x7f9670069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:41.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.132+0000 7f9683fff640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f96700678c0 0x7f9670069d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9674005c10 tx=0x7f9674009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:41.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.136+0000 7f9680ff9640 1 -- 172.21.15.76:0/2995654597 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f967c056e40 con 0x7f968415e640 2024-09-19T04:16:41.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.300+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f96840cea40 con 0x7f968415e640 2024-09-19T04:16:41.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.968+0000 7f9680ff9640 1 -- 172.21.15.76:0/2995654597 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v65) ==== 170+0+0 (secure 0 0 0) 0x7f967c056740 con 0x7f968415e640 2024-09-19T04:16:41.971 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-09-19T04:16:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.972+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f96700678c0 msgr2=0x7f9670069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.972+0000 7f968a797640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f96700678c0 0x7f9670069d80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9674005c10 tx=0x7f9674009040 comp rx=0 tx=0).stop 2024-09-19T04:16:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.972+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968415e640 msgr2=0x7f96840ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:41.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.972+0000 7f968a797640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968415e640 0x7f96840ad760 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f967c00e820 tx=0x7f967c00ece0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.980+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 shutdown_connections 2024-09-19T04:16:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.980+0000 7f968a797640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f96700678c0 0x7f9670069d80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.980+0000 7f968a797640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f968415e640 0x7f96840ad760 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.980+0000 7f968a797640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f96840b43a0 0x7f96840ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.980+0000 7f968a797640 1 --2- 172.21.15.76:0/2995654597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9684072230 0x7f9684152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:41.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.980+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 >> 172.21.15.76:0/2995654597 conn(0x7f96840b1c80 msgr2=0x7f9684154720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:41.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.984+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 shutdown_connections 2024-09-19T04:16:41.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:41.984+0000 7f968a797640 1 -- 172.21.15.76:0/2995654597 wait complete. 2024-09-19T04:16:42.016 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:16:42.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f6680cdb640 1 Processor -- start 2024-09-19T04:16:42.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f6680cdb640 1 -- start start 2024-09-19T04:16:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f6680cdb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15e5f0 0x7f667c06d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f6680cdb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c06d9c0 con 0x7f667c0cfaf0 2024-09-19T04:16:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f6680cdb640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f667c06dca0 con 0x7f667c0ce9d0 2024-09-19T04:16:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f6680cdb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f667c06df80 con 0x7f667c15e5f0 2024-09-19T04:16:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f667c0cfaf0 0x7f667c0d2b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:34744/0 (socket says 172.21.15.76:34744) 2024-09-19T04:16:42.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667ad9d640 1 -- 172.21.15.76:0/1126952712 learned_addr learned my addr 172.21.15.76:0/1126952712 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667a59c640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15e5f0 0x7f667c06d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1602848922 0 0) 0x7f667c06d9c0 con 0x7f667c0cfaf0 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6664003530 con 0x7f667c0cfaf0 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2909071386 0 0) 0x7f6664003530 con 0x7f667c0cfaf0 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15e5f0 msgr2=0x7f667c06d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15e5f0 0x7f667c06d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 >> v1:172.21.15.116:6789/0 conn(0x7f667c0ce9d0 legacy=0x7f667c0cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c06e260 con 0x7f667c0cfaf0 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6674001a00 con 0x7f667c0cfaf0 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 >> v1:172.21.15.76:6789/0 conn(0x7f667c0cfaf0 legacy=0x7f667c0d2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66640039f0 0x7f6664003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6664004810 0x7f6664004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6664005190 0x7f666400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f667c06df80 con 0x7f66640039f0 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f666400bf10 con 0x7f6664005190 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f666400c090 con 0x7f6664004810 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667a59c640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66640039f0 0x7f6664003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667a59c640 1 -- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6664004810 msgr2=0x7f6664004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667a59c640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6664004810 0x7f6664004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667a59c640 1 -- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6664005190 msgr2=0x7f666400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667ad9d640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6664005190 0x7f666400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667a59c640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6664005190 0x7f666400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.140+0000 7f667a59c640 1 -- 172.21.15.76:0/1126952712 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6664003530 con 0x7f66640039f0 2024-09-19T04:16:42.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f667a59c640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66640039f0 0x7f6664003e10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f666c009ae0 tx=0x7f666c01ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939663541e4f1536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:42.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c0b2070 con 0x7f66640039f0 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f666c0a5510 con 0x7f66640039f0 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f667959a640 1 -- 172.21.15.76:0/1126952712 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f666c0096b0 con 0x7f66640039f0 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- 172.21.15.76:0/1126952712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66640039f0 msgr2=0x7f6664003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 --2- 172.21.15.76:0/1126952712 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f66640039f0 0x7f6664003e10 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f666c009ae0 tx=0x7f666c01ca50 comp rx=0 tx=0).stop 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f667ad9d640 1 -- 172.21.15.76:0/1126952712 reap_dead start 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- 172.21.15.76:0/1126952712 shutdown_connections 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- 172.21.15.76:0/1126952712 >> 172.21.15.76:0/1126952712 conn(0x7f667c074030 msgr2=0x7f667c09d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- 172.21.15.76:0/1126952712 shutdown_connections 2024-09-19T04:16:42.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- 172.21.15.76:0/1126952712 wait complete. 2024-09-19T04:16:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 Processor -- start 2024-09-19T04:16:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- start start 2024-09-19T04:16:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15d910 0x7f667c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f667c0b9d20 0x7f667c15fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f667c160060 0x7f667c0b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f667c169fa0 con 0x7f667c0b9d20 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f667c0d3c90 con 0x7f667c160060 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6680cdb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f667c16a120 con 0x7f667c15d910 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6679d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f667c0b9d20 0x7f667c15fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6679d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f667c0b9d20 0x7f667c15fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59622/0 (socket says 172.21.15.76:59622) 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.144+0000 7f6679d9b640 1 -- 172.21.15.76:0/3824388559 learned_addr learned my addr 172.21.15.76:0/3824388559 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f667ad9d640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f667c160060 0x7f667c0b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f667a59c640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15d910 0x7f667c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6679d9b640 1 -- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15d910 msgr2=0x7f667c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6679d9b640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15d910 0x7f667c0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6679d9b640 1 -- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f667c160060 msgr2=0x7f667c0b71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6679d9b640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f667c160060 0x7f667c0b71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6679d9b640 1 -- 172.21.15.76:0/3824388559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667c0cba30 con 0x7f667c0b9d20 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f667ad9d640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f667c160060 0x7f667c0b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6679d9b640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f667c0b9d20 0x7f667c15fb20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f667000c490 tx=0x7f667000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f666affd640 1 -- 172.21.15.76:0/3824388559 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670017070 con 0x7f667c0b9d20 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f666affd640 1 -- 172.21.15.76:0/3824388559 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66700052f0 con 0x7f667c0b9d20 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667c0cbd10 con 0x7f667c0b9d20 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f666affd640 1 -- 172.21.15.76:0/3824388559 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6670003840 con 0x7f667c0b9d20 2024-09-19T04:16:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667c1f74e0 con 0x7f667c0b9d20 2024-09-19T04:16:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f667c15e5f0 con 0x7f667c0b9d20 2024-09-19T04:16:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f666affd640 1 -- 172.21.15.76:0/3824388559 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f667001a070 con 0x7f667c0b9d20 2024-09-19T04:16:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f666affd640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f665c0678c0 0x7f665c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f666affd640 1 -- 172.21.15.76:0/3824388559 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 6001+0+0 (secure 0 0 0) 0x7f6670085190 con 0x7f667c0b9d20 2024-09-19T04:16:42.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f667a59c640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f665c0678c0 0x7f665c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:42.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.148+0000 7f667a59c640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f665c0678c0 0x7f665c069d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f666c0af000 tx=0x7f666c0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.152+0000 7f666affd640 1 -- 172.21.15.76:0/3824388559 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6670055ba0 con 0x7f667c0b9d20 2024-09-19T04:16:42.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.352+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f667c160440 con 0x7f667c0b9d20 2024-09-19T04:16:42.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.976+0000 7f666affd640 1 -- 172.21.15.76:0/3824388559 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v44) ==== 173+0+0 (secure 0 0 0) 0x7f66700554a0 con 0x7f667c0b9d20 2024-09-19T04:16:42.982 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 9 and data pool 10 2024-09-19T04:16:42.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f665c0678c0 msgr2=0x7f665c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f665c0678c0 0x7f665c069d80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f666c0af000 tx=0x7f666c0046d0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f667c0b9d20 msgr2=0x7f667c15fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f667c0b9d20 0x7f667c15fb20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f667000c490 tx=0x7f667000c950 comp rx=0 tx=0).stop 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 shutdown_connections 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f665c0678c0 0x7f665c069d80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f667c160060 0x7f667c0b71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f667c0b9d20 0x7f667c15fb20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 --2- 172.21.15.76:0/3824388559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f667c15d910 0x7f667c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 >> 172.21.15.76:0/3824388559 conn(0x7f66640039f0 msgr2=0x7f667c06e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 shutdown_connections 2024-09-19T04:16:42.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:42.984+0000 7f6680cdb640 1 -- 172.21.15.76:0/3824388559 wait complete. 2024-09-19T04:16:43.007 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:16:43.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.116+0000 7f79ed4ad640 1 Processor -- start 2024-09-19T04:16:43.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.116+0000 7f79ed4ad640 1 -- start start 2024-09-19T04:16:43.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.116+0000 7f79ed4ad640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f79e809ac60 0x7f79e809ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.116+0000 7f79ed4ad640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f79e80af730 0x7f79e815d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.116+0000 7f79ed4ad640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79e80afd80 con 0x7f79e809b040 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.116+0000 7f79ed4ad640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f79e80afb60 con 0x7f79e80af730 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.116+0000 7f79ed4ad640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f79e80b0060 con 0x7f79e809ac60 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f79e80af730 0x7f79e815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e67fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f79e809b040 0x7f79e80b0e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33274/0 (socket says 172.21.15.76:33274) 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f79e80af730 0x7f79e815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44564/0 (socket says 172.21.15.76:44564) 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e67fc640 1 -- 172.21.15.76:0/3649970554 learned_addr learned my addr 172.21.15.76:0/3649970554 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:43.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e6ffd640 1 --2- 172.21.15.76:0/3649970554 >> v2:172.21.15.116:3300/0 conn(0x7f79e809ac60 0x7f79e809ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e77fe640 1 -- 172.21.15.76:0/3649970554 >> v2:172.21.15.116:3300/0 conn(0x7f79e809ac60 msgr2=0x7f79e809ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e77fe640 1 --2- 172.21.15.76:0/3649970554 >> v2:172.21.15.116:3300/0 conn(0x7f79e809ac60 0x7f79e809ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e77fe640 1 -- 172.21.15.76:0/3649970554 >> v1:172.21.15.76:6789/0 conn(0x7f79e809b040 legacy=0x7f79e80b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e5ffb640 1 -- 172.21.15.76:0/3649970554 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2091832764 0 0) 0x7f79e80afd80 con 0x7f79e809b040 2024-09-19T04:16:43.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e77fe640 1 -- 172.21.15.76:0/3649970554 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e80b0340 con 0x7f79e80af730 2024-09-19T04:16:43.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e77fe640 1 --2- 172.21.15.76:0/3649970554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e80af730 0x7f79e815d010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b7d70 tx=0x7f79e001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2127d052751cd302 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e5ffb640 1 -- 172.21.15.76:0/3649970554 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e00b2070 con 0x7f79e80af730 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e5ffb640 1 -- 172.21.15.76:0/3649970554 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79e00a5850 con 0x7f79e80af730 2024-09-19T04:16:43.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79e5ffb640 1 -- 172.21.15.76:0/3649970554 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e00a5c20 con 0x7f79e80af730 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79ed4ad640 1 -- 172.21.15.76:0/3649970554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e80af730 msgr2=0x7f79e815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/3649970554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e80af730 0x7f79e815d010 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b7d70 tx=0x7f79e001c8a0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79ed4ad640 1 -- 172.21.15.76:0/3649970554 shutdown_connections 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/3649970554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e80af730 0x7f79e815d010 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/3649970554 >> v2:172.21.15.116:3300/0 conn(0x7f79e809ac60 0x7f79e809ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.120+0000 7f79ed4ad640 1 -- 172.21.15.76:0/3649970554 >> 172.21.15.76:0/3649970554 conn(0x7f79e80b78d0 msgr2=0x7f79e80b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- 172.21.15.76:0/3649970554 shutdown_connections 2024-09-19T04:16:43.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- 172.21.15.76:0/3649970554 wait complete. 2024-09-19T04:16:43.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 Processor -- start 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- start start 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e809ac60 0x7f79e814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f79e809b040 0x7f79e8151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80af730 0x7f79e81640a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f79e80b23c0 con 0x7f79e809ac60 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f79e815d550 con 0x7f79e80af730 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f79e80b2540 con 0x7f79e809b040 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e809ac60 0x7f79e814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e809ac60 0x7f79e814ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44580/0 (socket says 172.21.15.76:44580) 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 -- 172.21.15.76:0/313307957 learned_addr learned my addr 172.21.15.76:0/313307957 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 -- 172.21.15.76:0/313307957 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f79e809b040 msgr2=0x7f79e8151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f79e809b040 0x7f79e8151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 -- 172.21.15.76:0/313307957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80af730 msgr2=0x7f79e81640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80af730 0x7f79e81640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 -- 172.21.15.76:0/313307957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e81507d0 con 0x7f79e809ac60 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79e6ffd640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e809ac60 0x7f79e814ebb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f79d800d830 tx=0x7f79d800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79cf7fe640 1 -- 172.21.15.76:0/313307957 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d800b6c0 con 0x7f79e809ac60 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79cf7fe640 1 -- 172.21.15.76:0/313307957 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d8019070 con 0x7f79e809ac60 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e8150a90 con 0x7f79e809ac60 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79cf7fe640 1 -- 172.21.15.76:0/313307957 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d800ba60 con 0x7f79e809ac60 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e8151050 con 0x7f79e809ac60 2024-09-19T04:16:43.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.124+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e80b6480 con 0x7f79e809ac60 2024-09-19T04:16:43.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.128+0000 7f79cf7fe640 1 -- 172.21.15.76:0/313307957 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f79d801e460 con 0x7f79e809ac60 2024-09-19T04:16:43.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.128+0000 7f79cf7fe640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f79d40678c0 0x7f79d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.128+0000 7f79cf7fe640 1 -- 172.21.15.76:0/313307957 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f79d8086510 con 0x7f79e809ac60 2024-09-19T04:16:43.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.128+0000 7f79e67fc640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f79d40678c0 0x7f79d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.128+0000 7f79e67fc640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f79d40678c0 0x7f79d4069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0011f0 tx=0x7f79dc007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.132+0000 7f79cf7fe640 1 -- 172.21.15.76:0/313307957 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f79d8056ec0 con 0x7f79e809ac60 2024-09-19T04:16:43.184 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-09-19T04:16:43.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.296+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79e809b980 con 0x7f79e809ac60 2024-09-19T04:16:43.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.296+0000 7f79cf7fe640 1 -- 172.21.15.76:0/313307957 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16981 (secure 0 0 0) 0x7f79d80567c0 con 0x7f79e809ac60 2024-09-19T04:16:43.300 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:43.300 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":66,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:16:42.355647+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:16:40.908065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T04:16:41.340553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:16:43.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f79d40678c0 msgr2=0x7f79d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f79d40678c0 0x7f79d4069d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0011f0 tx=0x7f79dc007930 comp rx=0 tx=0).stop 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e809ac60 msgr2=0x7f79e814ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e809ac60 0x7f79e814ebb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f79d800d830 tx=0x7f79d800dcf0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 shutdown_connections 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f79d40678c0 0x7f79d4069d80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79e80af730 0x7f79e81640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f79e809b040 0x7f79e8151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 --2- 172.21.15.76:0/313307957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f79e809ac60 0x7f79e814ebb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 >> 172.21.15.76:0/313307957 conn(0x7f79e80b78d0 msgr2=0x7f79e80b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 shutdown_connections 2024-09-19T04:16:43.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.300+0000 7f79ed4ad640 1 -- 172.21.15.76:0/313307957 wait complete. 2024-09-19T04:16:43.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:16:43.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.432+0000 7f8ae0efd640 1 Processor -- start 2024-09-19T04:16:43.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.432+0000 7f8ae0efd640 1 -- start start 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ae0efd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8adc1587a0 0x7f8adc0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ae0efd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc06e680 0x7f8adc072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ae0efd640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8adc06ec60 con 0x7f8adc0b37d0 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ae0efd640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8adc0b3c40 con 0x7f8adc1587a0 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ae0efd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8adc06f870 con 0x7f8adc06e680 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8adad9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc06e680 0x7f8adc072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ada59c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8adc1587a0 0x7f8adc0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8adad9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc06e680 0x7f8adc072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:50180/0 (socket says 172.21.15.76:50180) 2024-09-19T04:16:43.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8adad9d640 1 -- 172.21.15.76:0/2161859631 learned_addr learned my addr 172.21.15.76:0/2161859631 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ada59c640 1 -- 172.21.15.76:0/2161859631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc06e680 msgr2=0x7f8adc072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ada59c640 1 --2- 172.21.15.76:0/2161859631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc06e680 0x7f8adc072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ada59c640 1 -- 172.21.15.76:0/2161859631 >> v1:172.21.15.116:6789/0 conn(0x7f8adc0b37d0 legacy=0x7f8adc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.436+0000 7f8ada59c640 1 -- 172.21.15.76:0/2161859631 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc06ef40 con 0x7f8adc1587a0 2024-09-19T04:16:43.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ada59c640 1 --2- 172.21.15.76:0/2161859631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc1587a0 0x7f8adc0b3290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00097c0 tx=0x7f8ad001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7afd76f94127d706 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ad959a640 1 -- 172.21.15.76:0/2161859631 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b2070 con 0x7f8adc1587a0 2024-09-19T04:16:43.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ad959a640 1 -- 172.21.15.76:0/2161859631 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad00ad680 con 0x7f8adc1587a0 2024-09-19T04:16:43.443 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ad959a640 1 -- 172.21.15.76:0/2161859631 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00ada50 con 0x7f8adc1587a0 2024-09-19T04:16:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- 172.21.15.76:0/2161859631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc1587a0 msgr2=0x7f8adc0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/2161859631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc1587a0 0x7f8adc0b3290 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00097c0 tx=0x7f8ad001c470 comp rx=0 tx=0).stop 2024-09-19T04:16:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- 172.21.15.76:0/2161859631 shutdown_connections 2024-09-19T04:16:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/2161859631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc06e680 0x7f8adc072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/2161859631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc1587a0 0x7f8adc0b3290 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- 172.21.15.76:0/2161859631 >> 172.21.15.76:0/2161859631 conn(0x7f8adc0b1f90 msgr2=0x7f8adc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- 172.21.15.76:0/2161859631 shutdown_connections 2024-09-19T04:16:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- 172.21.15.76:0/2161859631 wait complete. 2024-09-19T04:16:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 Processor -- start 2024-09-19T04:16:43.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- start start 2024-09-19T04:16:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc06e680 0x7f8adc0999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc0b37d0 0x7f8adc146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8adc1587a0 0x7f8adc147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc073030 con 0x7f8adc1587a0 2024-09-19T04:16:43.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8adc072eb0 con 0x7f8adc06e680 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ae0efd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8adc0731b0 con 0x7f8adc0b37d0 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8adc1587a0 0x7f8adc147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8adc1587a0 0x7f8adc147490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44596/0 (socket says 172.21.15.76:44596) 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 -- 172.21.15.76:0/3341261994 learned_addr learned my addr 172.21.15.76:0/3341261994 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ad9d9b640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc0b37d0 0x7f8adc146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8ada59c640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc06e680 0x7f8adc0999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 -- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc0b37d0 msgr2=0x7f8adc146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc0b37d0 0x7f8adc146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 -- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc06e680 msgr2=0x7f8adc0999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc06e680 0x7f8adc0999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.440+0000 7f8adad9d640 1 -- 172.21.15.76:0/3341261994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8adc155f20 con 0x7f8adc1587a0 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8adad9d640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8adc1587a0 0x7f8adc147490 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400efc0 tx=0x7f8ad400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8acaffd640 1 -- 172.21.15.76:0/3341261994 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad400cd20 con 0x7f8adc1587a0 2024-09-19T04:16:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8acaffd640 1 -- 172.21.15.76:0/3341261994 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ad40039a0 con 0x7f8adc1587a0 2024-09-19T04:16:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8adc0af6b0 con 0x7f8adc1587a0 2024-09-19T04:16:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8acaffd640 1 -- 172.21.15.76:0/3341261994 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad4003d90 con 0x7f8adc1587a0 2024-09-19T04:16:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8adc0afc70 con 0x7f8adc1587a0 2024-09-19T04:16:43.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8adc072160 con 0x7f8adc1587a0 2024-09-19T04:16:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8acaffd640 1 -- 172.21.15.76:0/3341261994 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ad4012050 con 0x7f8adc1587a0 2024-09-19T04:16:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8acaffd640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8abc0678c0 0x7f8abc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8acaffd640 1 -- 172.21.15.76:0/3341261994 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f8ad4086210 con 0x7f8adc1587a0 2024-09-19T04:16:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8ada59c640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8abc0678c0 0x7f8abc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.444+0000 7f8ada59c640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8abc0678c0 0x7f8abc069d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00af000 tx=0x7f8ad00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.448+0000 7f8acaffd640 1 -- 172.21.15.76:0/3341261994 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8ad4022020 con 0x7f8adc1587a0 2024-09-19T04:16:43.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.612+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8adc070080 con 0x7f8adc1587a0 2024-09-19T04:16:43.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8acaffd640 1 -- 172.21.15.76:0/3341261994 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16981 (secure 0 0 0) 0x7f8ad4010040 con 0x7f8adc1587a0 2024-09-19T04:16:43.618 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:43.618 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":66,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:16:42.355647+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:16:40.908065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T04:16:41.340553+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:16:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8abc0678c0 msgr2=0x7f8abc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8abc0678c0 0x7f8abc069d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00af000 tx=0x7f8ad00a7000 comp rx=0 tx=0).stop 2024-09-19T04:16:43.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8adc1587a0 msgr2=0x7f8adc147490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8adc1587a0 0x7f8adc147490 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f8ad400efc0 tx=0x7f8ad400c490 comp rx=0 tx=0).stop 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 shutdown_connections 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8abc0678c0 0x7f8abc069d80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8adc1587a0 0x7f8adc147490 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8adc0b37d0 0x7f8adc146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 --2- 172.21.15.76:0/3341261994 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8adc06e680 0x7f8adc0999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 >> 172.21.15.76:0/3341261994 conn(0x7f8adc0b1f90 msgr2=0x7f8adc0ad290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 shutdown_connections 2024-09-19T04:16:43.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.616+0000 7f8ae0efd640 1 -- 172.21.15.76:0/3341261994 wait complete. 2024-09-19T04:16:43.638 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:16:43.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.756+0000 7f29c52e7640 1 Processor -- start 2024-09-19T04:16:43.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.756+0000 7f29c52e7640 1 -- start start 2024-09-19T04:16:43.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29c52e7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29c015bec0 0x7f29c00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29c52e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 0x7f29c006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29c52e7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29c00b3290 con 0x7f29c015c2a0 2024-09-19T04:16:43.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29c52e7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f29c00b3570 con 0x7f29c015bec0 2024-09-19T04:16:43.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29c52e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f29c00b3850 con 0x7f29c00af470 2024-09-19T04:16:43.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29be7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f29c015c2a0 0x7f29c00aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33280/0 (socket says 172.21.15.76:33280) 2024-09-19T04:16:43.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29be7fc640 1 -- 172.21.15.76:0/1785729498 learned_addr learned my addr 172.21.15.76:0/1785729498 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29beffd640 1 --2- 172.21.15.76:0/1785729498 >> v2:172.21.15.116:3300/0 conn(0x7f29c015bec0 0x7f29c00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bf7fe640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 0x7f29c006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1412159910 0 0) 0x7f29c00b3290 con 0x7f29c015c2a0 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29ac003530 con 0x7f29c015c2a0 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1131665247 0 0) 0x7f29ac003530 con 0x7f29c015c2a0 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 msgr2=0x7f29c006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 0x7f29c006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 >> v2:172.21.15.116:3300/0 conn(0x7f29c015bec0 msgr2=0x7f29c00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 --2- 172.21.15.76:0/1785729498 >> v2:172.21.15.116:3300/0 conn(0x7f29c015bec0 0x7f29c00ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c00b3b30 con 0x7f29c015c2a0 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f29a8001680 con 0x7f29c015c2a0 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 >> v1:172.21.15.76:6789/0 conn(0x7f29c015c2a0 legacy=0x7f29c00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 0x7f29ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29ac004d50 0x7f29ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00b3570 con 0x7f29ac0039c0 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00b3850 con 0x7f29ac004d50 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f29ac00b9d0 con 0x7f29c00af470 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29beffd640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29be7fc640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 0x7f29ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29beffd640 1 -- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 msgr2=0x7f29ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29beffd640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c00af470 0x7f29ac004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bf7fe640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29ac004d50 0x7f29ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29beffd640 1 -- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29ac004d50 msgr2=0x7f29ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:43.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29beffd640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29ac004d50 0x7f29ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29beffd640 1 -- 172.21.15.76:0/1785729498 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ac003530 con 0x7f29ac0039c0 2024-09-19T04:16:43.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.760+0000 7f29bf7fe640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29ac004d50 0x7f29ac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29ac003de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009fc0 tx=0x7f29b001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a7d32d5196ca95 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00b2070 con 0x7f29ac0039c0 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00a5c80 con 0x7f29ac0039c0 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29bdffb640 1 -- 172.21.15.76:0/1785729498 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b0009c10 con 0x7f29ac0039c0 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- 172.21.15.76:0/1785729498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 msgr2=0x7f29ac003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 --2- 172.21.15.76:0/1785729498 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29ac003de0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f29b0009fc0 tx=0x7f29b001cc90 comp rx=0 tx=0).stop 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29bf7fe640 1 -- 172.21.15.76:0/1785729498 reap_dead start 2024-09-19T04:16:43.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- 172.21.15.76:0/1785729498 shutdown_connections 2024-09-19T04:16:43.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- 172.21.15.76:0/1785729498 >> 172.21.15.76:0/1785729498 conn(0x7f29c00b1f90 msgr2=0x7f29c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:43.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- 172.21.15.76:0/1785729498 shutdown_connections 2024-09-19T04:16:43.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- 172.21.15.76:0/1785729498 wait complete. 2024-09-19T04:16:43.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 Processor -- start 2024-09-19T04:16:43.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- start start 2024-09-19T04:16:43.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29c0158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29c0154330 0x7f29c0147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c015cfd0 0x7f29c015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f29c006e810 con 0x7f29ac0039c0 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f29c00ad740 con 0x7f29c0154330 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29c52e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f29c006e990 con 0x7f29c015cfd0 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29c0158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29c0158ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44610/0 (socket says 172.21.15.76:44610) 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 -- 172.21.15.76:0/2053291662 learned_addr learned my addr 172.21.15.76:0/2053291662 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:43.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 -- 172.21.15.76:0/2053291662 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c015cfd0 msgr2=0x7f29c015eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c015cfd0 0x7f29c015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 -- 172.21.15.76:0/2053291662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29c0154330 msgr2=0x7f29c0147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29be7fc640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29c0154330 0x7f29c0147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29c0154330 0x7f29c0147020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.764+0000 7f29beffd640 1 -- 172.21.15.76:0/2053291662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c0147be0 con 0x7f29ac0039c0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29beffd640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29c0158ca0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f29b001c700 tx=0x7f29b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29b77fe640 1 -- 172.21.15.76:0/2053291662 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00ba070 con 0x7f29ac0039c0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29b77fe640 1 -- 172.21.15.76:0/2053291662 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29b00ada20 con 0x7f29ac0039c0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c01535b0 con 0x7f29ac0039c0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29b77fe640 1 -- 172.21.15.76:0/2053291662 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b00b2040 con 0x7f29ac0039c0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c0153b30 con 0x7f29ac0039c0 2024-09-19T04:16:43.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c00ad200 con 0x7f29ac0039c0 2024-09-19T04:16:43.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29b77fe640 1 -- 172.21.15.76:0/2053291662 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f29b00b6410 con 0x7f29ac0039c0 2024-09-19T04:16:43.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29b77fe640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29a00678f0 0x7f29a0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:43.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29b77fe640 1 -- 172.21.15.76:0/2053291662 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 6101+0+0 (secure 0 0 0) 0x7f29b0120410 con 0x7f29ac0039c0 2024-09-19T04:16:43.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29be7fc640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29a00678f0 0x7f29a0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:43.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.768+0000 7f29be7fc640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29a00678f0 0x7f29a0069db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29a8000f70 tx=0x7f29a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:43.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.772+0000 7f29b77fe640 1 -- 172.21.15.76:0/2053291662 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f29c0147be0 con 0x7f29ac0039c0 2024-09-19T04:16:43.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:43.972+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f29c006fcd0 con 0x7f29ac0039c0 2024-09-19T04:16:44.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.979+0000 7f29b77fe640 1 -- 172.21.15.76:0/2053291662 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) ==== 118+0+0 (secure 0 0 0) 0x7f29b00169d0 con 0x7f29ac0039c0 2024-09-19T04:16:44.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29a00678f0 msgr2=0x7f29a0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29a00678f0 0x7f29a0069db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f29a8000f70 tx=0x7f29a8009040 comp rx=0 tx=0).stop 2024-09-19T04:16:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 msgr2=0x7f29c0158ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29c0158ca0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f29b001c700 tx=0x7f29b0009750 comp rx=0 tx=0).stop 2024-09-19T04:16:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 shutdown_connections 2024-09-19T04:16:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29a00678f0 0x7f29a0069db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29c015cfd0 0x7f29c015eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29c0154330 0x7f29c0147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 --2- 172.21.15.76:0/2053291662 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29ac0039c0 0x7f29c0158ca0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 >> 172.21.15.76:0/2053291662 conn(0x7f29c00b1f90 msgr2=0x7f29c00751b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 shutdown_connections 2024-09-19T04:16:44.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:44.983+0000 7f29c52e7640 1 -- 172.21.15.76:0/2053291662 wait complete. 2024-09-19T04:16:45.007 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T04:16:45.007 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T04:16:45.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 Processor -- start 2024-09-19T04:16:45.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 -- start start 2024-09-19T04:16:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b815bec0 0x7fe7b80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7b815c2a0 0x7fe7b80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:45.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe7b80af2a0 0x7fe7b815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b80b3290 con 0x7fe7b815c2a0 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe7b80b3570 con 0x7fe7b80af2a0 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bfbe8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b80b3850 con 0x7fe7b815bec0 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7b815c2a0 0x7fe7b80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7b815c2a0 0x7fe7b80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44612/0 (socket says 172.21.15.76:44612) 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 -- 172.21.15.76:0/3988460526 learned_addr learned my addr 172.21.15.76:0/3988460526 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7be185640 1 --2- 172.21.15.76:0/3988460526 >> v2:172.21.15.116:3300/0 conn(0x7fe7b80af2a0 0x7fe7b815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd984640 1 --2- 172.21.15.76:0/3988460526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b815bec0 0x7fe7b80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 -- 172.21.15.76:0/3988460526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b815bec0 msgr2=0x7fe7b80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 --2- 172.21.15.76:0/3988460526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b815bec0 0x7fe7b80ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 -- 172.21.15.76:0/3988460526 >> v2:172.21.15.116:3300/0 conn(0x7fe7b80af2a0 msgr2=0x7fe7b815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:45.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 --2- 172.21.15.76:0/3988460526 >> v2:172.21.15.116:3300/0 conn(0x7fe7b80af2a0 0x7fe7b815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 -- 172.21.15.76:0/3988460526 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b80b3b30 con 0x7fe7b815c2a0 2024-09-19T04:16:45.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7be185640 1 --2- 172.21.15.76:0/3988460526 >> v2:172.21.15.116:3300/0 conn(0x7fe7b80af2a0 0x7fe7b815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:45.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.131+0000 7fe7bd183640 1 --2- 172.21.15.76:0/3988460526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b80aed60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ada40 tx=0x7fe7ac01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42575b2e60ea2ff1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:45.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bc982640 1 -- 172.21.15.76:0/3988460526 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0b0070 con 0x7fe7b815c2a0 2024-09-19T04:16:45.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bc982640 1 -- 172.21.15.76:0/3988460526 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac0ab600 con 0x7fe7b815c2a0 2024-09-19T04:16:45.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bc982640 1 -- 172.21.15.76:0/3988460526 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac0ab9d0 con 0x7fe7b815c2a0 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/3988460526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 msgr2=0x7fe7b80aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/3988460526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b80aed60 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ada40 tx=0x7fe7ac01c230 comp rx=0 tx=0).stop 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/3988460526 shutdown_connections 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/3988460526 >> v2:172.21.15.116:3300/0 conn(0x7fe7b80af2a0 0x7fe7b815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/3988460526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b80aed60 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/3988460526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b815bec0 0x7fe7b80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/3988460526 >> 172.21.15.76:0/3988460526 conn(0x7fe7b80b1f90 msgr2=0x7fe7b80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/3988460526 shutdown_connections 2024-09-19T04:16:45.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/3988460526 wait complete. 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 Processor -- start 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- start start 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b80af2a0 0x7fe7b815dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7b815bec0 0x7fe7b809d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b8154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b80af6b0 con 0x7fe7b815c2a0 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7b80ad740 con 0x7fe7b815bec0 2024-09-19T04:16:45.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe7b806e000 con 0x7fe7b80af2a0 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b8154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b8154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44618/0 (socket says 172.21.15.76:44618) 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 -- 172.21.15.76:0/1022893425 learned_addr learned my addr 172.21.15.76:0/1022893425 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bd984640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b80af2a0 0x7fe7b815dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bd183640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7b815bec0 0x7fe7b809d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 -- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b80af2a0 msgr2=0x7fe7b815dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b80af2a0 0x7fe7b815dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 -- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7b815bec0 msgr2=0x7fe7b809d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7b815bec0 0x7fe7b809d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 -- 172.21.15.76:0/1022893425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7b809da00 con 0x7fe7b815c2a0 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bd183640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7b815bec0 0x7fe7b809d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7be185640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b8154990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000cf20 tx=0x7fe7b000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:45.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7aa7fc640 1 -- 172.21.15.76:0/1022893425 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b001a070 con 0x7fe7b815c2a0 2024-09-19T04:16:45.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7aa7fc640 1 -- 172.21.15.76:0/1022893425 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b000d840 con 0x7fe7b815c2a0 2024-09-19T04:16:45.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7aa7fc640 1 -- 172.21.15.76:0/1022893425 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000dc30 con 0x7fe7b815c2a0 2024-09-19T04:16:45.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7b809dce0 con 0x7fe7b815c2a0 2024-09-19T04:16:45.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.135+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7b8151560 con 0x7fe7b815c2a0 2024-09-19T04:16:45.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.139+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7b809e0d0 con 0x7fe7b815c2a0 2024-09-19T04:16:45.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.139+0000 7fe7aa7fc640 1 -- 172.21.15.76:0/1022893425 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7b001d070 con 0x7fe7b815c2a0 2024-09-19T04:16:45.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.139+0000 7fe7aa7fc640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe79c0678c0 0x7fe79c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:45.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.139+0000 7fe7aa7fc640 1 -- 172.21.15.76:0/1022893425 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fe7b00866d0 con 0x7fe7b815c2a0 2024-09-19T04:16:45.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.139+0000 7fe7bd984640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe79c0678c0 0x7fe79c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:45.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.139+0000 7fe7bd984640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe79c0678c0 0x7fe79c069d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8155c90 tx=0x7fe7b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:45.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.143+0000 7fe7aa7fc640 1 -- 172.21.15.76:0/1022893425 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe7b0057080 con 0x7fe7b815c2a0 2024-09-19T04:16:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.339+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe7b80ad200 con 0x7fe7b815c2a0 2024-09-19T04:16:45.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.987+0000 7fe7aa7fc640 1 -- 172.21.15.76:0/1022893425 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) ==== 122+0+0 (secure 0 0 0) 0x7fe7b0056980 con 0x7fe7b815c2a0 2024-09-19T04:16:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe79c0678c0 msgr2=0x7fe79c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe79c0678c0 0x7fe79c069d80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8155c90 tx=0x7fe7b4008040 comp rx=0 tx=0).stop 2024-09-19T04:16:45.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 msgr2=0x7fe7b8154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b8154990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000cf20 tx=0x7fe7b000a7d0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 shutdown_connections 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe79c0678c0 0x7fe79c069d80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7b815c2a0 0x7fe7b8154990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7b815bec0 0x7fe7b809d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 --2- 172.21.15.76:0/1022893425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7b80af2a0 0x7fe7b815dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 >> 172.21.15.76:0/1022893425 conn(0x7fe7b80b1f90 msgr2=0x7fe7b80b17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 shutdown_connections 2024-09-19T04:16:45.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:45.991+0000 7fe7bfbe8640 1 -- 172.21.15.76:0/1022893425 wait complete. 2024-09-19T04:16:46.021 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T04:16:46.022 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:46.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 Processor -- start 2024-09-19T04:16:46.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 -- start start 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c15e450 0x7fab7c0b7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fab7c0b7670 0x7fab7c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab7c0d4670 0x7fab7c0d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fab7c0d1f70 con 0x7fab7c0d4670 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fab7c0b58d0 con 0x7fab7c0b7670 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab82b6e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fab7c0b55f0 con 0x7fab7c15e450 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab7c0d4670 0x7fab7c0d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fab7c0d4670 0x7fab7c0d2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44624/0 (socket says 172.21.15.76:44624) 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 -- 172.21.15.76:0/1631271810 learned_addr learned my addr 172.21.15.76:0/1631271810 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8090a640 1 --2- 172.21.15.76:0/1631271810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c15e450 0x7fab7c0b7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab7bfff640 1 --2- 172.21.15.76:0/1631271810 >> v2:172.21.15.116:3300/0 conn(0x7fab7c0b7670 0x7fab7c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 -- 172.21.15.76:0/1631271810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c15e450 msgr2=0x7fab7c0b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 --2- 172.21.15.76:0/1631271810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c15e450 0x7fab7c0b7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 -- 172.21.15.76:0/1631271810 >> v2:172.21.15.116:3300/0 conn(0x7fab7c0b7670 msgr2=0x7fab7c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 --2- 172.21.15.76:0/1631271810 >> v2:172.21.15.116:3300/0 conn(0x7fab7c0b7670 0x7fab7c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.143+0000 7fab8110b640 1 -- 172.21.15.76:0/1631271810 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c0d2210 con 0x7fab7c0d4670 2024-09-19T04:16:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab8110b640 1 --2- 172.21.15.76:0/1631271810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c0d4670 0x7fab7c0d2a10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0b6ef0 tx=0x7fab7401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6ea754e3a0a02d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab7b7fe640 1 -- 172.21.15.76:0/1631271810 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740a5800 con 0x7fab7c0d4670 2024-09-19T04:16:46.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab7b7fe640 1 -- 172.21.15.76:0/1631271810 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab740a59a0 con 0x7fab7c0d4670 2024-09-19T04:16:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab7b7fe640 1 -- 172.21.15.76:0/1631271810 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740a5d70 con 0x7fab7c0d4670 2024-09-19T04:16:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- 172.21.15.76:0/1631271810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c0d4670 msgr2=0x7fab7c0d2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 --2- 172.21.15.76:0/1631271810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c0d4670 0x7fab7c0d2a10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fab7c0b6ef0 tx=0x7fab7401c9c0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- 172.21.15.76:0/1631271810 shutdown_connections 2024-09-19T04:16:46.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 --2- 172.21.15.76:0/1631271810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c0d4670 0x7fab7c0d2a10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 --2- 172.21.15.76:0/1631271810 >> v2:172.21.15.116:3300/0 conn(0x7fab7c0b7670 0x7fab7c0d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 --2- 172.21.15.76:0/1631271810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c15e450 0x7fab7c0b7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- 172.21.15.76:0/1631271810 >> 172.21.15.76:0/1631271810 conn(0x7fab7c08bf90 msgr2=0x7fab7c09d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- 172.21.15.76:0/1631271810 shutdown_connections 2024-09-19T04:16:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- 172.21.15.76:0/1631271810 wait complete. 2024-09-19T04:16:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 Processor -- start 2024-09-19T04:16:46.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- start start 2024-09-19T04:16:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c0b7670 0x7fab7c0b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c0d4670 0x7fab7c1509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c15e450 0x7fab7c150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c15e9b0 con 0x7fab7c15e450 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c15e830 con 0x7fab7c0b7670 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab82b6e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fab7c0b5030 con 0x7fab7c0d4670 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab8110b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c15e450 0x7fab7c150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab8110b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c15e450 0x7fab7c150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44628/0 (socket says 172.21.15.76:44628) 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.147+0000 7fab8110b640 1 -- 172.21.15.76:0/2252910274 learned_addr learned my addr 172.21.15.76:0/2252910274 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8090a640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c0b7670 0x7fab7c0b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab7bfff640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c0d4670 0x7fab7c1509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8110b640 1 -- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c0d4670 msgr2=0x7fab7c1509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8110b640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c0d4670 0x7fab7c1509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8110b640 1 -- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c0b7670 msgr2=0x7fab7c0b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8110b640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c0b7670 0x7fab7c0b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8110b640 1 -- 172.21.15.76:0/2252910274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c159130 con 0x7fab7c15e450 2024-09-19T04:16:46.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8110b640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c15e450 0x7fab7c150ee0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fab740af040 tx=0x7fab74019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab797fa640 1 -- 172.21.15.76:0/2252910274 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740b8670 con 0x7fab7c15e450 2024-09-19T04:16:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab797fa640 1 -- 172.21.15.76:0/2252910274 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab740be070 con 0x7fab7c15e450 2024-09-19T04:16:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c156850 con 0x7fab7c15e450 2024-09-19T04:16:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab797fa640 1 -- 172.21.15.76:0/2252910274 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740b8a70 con 0x7fab7c15e450 2024-09-19T04:16:46.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c151740 con 0x7fab7c15e450 2024-09-19T04:16:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab7c156640 con 0x7fab7c15e450 2024-09-19T04:16:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab797fa640 1 -- 172.21.15.76:0/2252910274 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fab740b8c10 con 0x7fab7c15e450 2024-09-19T04:16:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab797fa640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fab680678c0 0x7fab68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab797fa640 1 -- 172.21.15.76:0/2252910274 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fab74121130 con 0x7fab7c15e450 2024-09-19T04:16:46.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8090a640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fab680678c0 0x7fab68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.151+0000 7fab8090a640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fab680678c0 0x7fab68069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b930 tx=0x7fab6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.155+0000 7fab797fa640 1 -- 172.21.15.76:0/2252910274 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fab740165e0 con 0x7fab7c15e450 2024-09-19T04:16:46.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.355+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab7c0d2a10 con 0x7fab7c15e450 2024-09-19T04:16:46.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.355+0000 7fab797fa640 1 -- 172.21.15.76:0/2252910274 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5230 (secure 0 0 0) 0x7fab740f1a50 con 0x7fab7c15e450 2024-09-19T04:16:46.360 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:46.361 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":47,"btime":"2024-09-19T04:16:45:985234+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:16:45.346923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-09-19T04:16:46.361 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 47 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fab680678c0 msgr2=0x7fab68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fab680678c0 0x7fab68069d80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fab6c00b930 tx=0x7fab6c008040 comp rx=0 tx=0).stop 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c15e450 msgr2=0x7fab7c150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c15e450 0x7fab7c150ee0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fab740af040 tx=0x7fab74019760 comp rx=0 tx=0).stop 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 shutdown_connections 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fab680678c0 0x7fab68069d80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fab7c15e450 0x7fab7c150ee0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fab7c0d4670 0x7fab7c1509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 --2- 172.21.15.76:0/2252910274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab7c0b7670 0x7fab7c0b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 >> 172.21.15.76:0/2252910274 conn(0x7fab7c08bf90 msgr2=0x7fab7c15bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 shutdown_connections 2024-09-19T04:16:46.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.359+0000 7fab82b6e640 1 -- 172.21.15.76:0/2252910274 wait complete. 2024-09-19T04:16:46.380 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:16:46.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6c3b2640 1 Processor -- start 2024-09-19T04:16:46.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6c3b2640 1 -- start start 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6c3b2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b6415bec0 0x7f8b640ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6c3b2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8b6415c2a0 0x7f8b640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6c3b2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b640b3290 con 0x7f8b640af2a0 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6c3b2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8b640b3570 con 0x7f8b6415c2a0 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6c3b2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8b640b3850 con 0x7f8b6415bec0 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6a94f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8b640af2a0 0x7f8b6415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33290/0 (socket says 172.21.15.76:33290) 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6a94f640 1 -- 172.21.15.76:0/2170806002 learned_addr learned my addr 172.21.15.76:0/2170806002 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6a14e640 1 --2- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b6415bec0 0x7f8b640ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6994d640 1 --2- 172.21.15.76:0/2170806002 >> v2:172.21.15.116:3300/0 conn(0x7f8b6415c2a0 0x7f8b640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6914c640 1 -- 172.21.15.76:0/2170806002 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3793323593 0 0) 0x7f8b640b3290 con 0x7f8b640af2a0 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6914c640 1 -- 172.21.15.76:0/2170806002 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b50003530 con 0x7f8b640af2a0 2024-09-19T04:16:46.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6994d640 1 -- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b6415bec0 msgr2=0x7f8b640ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:46.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6994d640 1 --2- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b6415bec0 0x7f8b640ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6994d640 1 -- 172.21.15.76:0/2170806002 >> v1:172.21.15.76:6789/0 conn(0x7f8b640af2a0 legacy=0x7f8b6415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6914c640 1 -- 172.21.15.76:0/2170806002 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 343600360 0 0) 0x7f8b50003530 con 0x7f8b640af2a0 2024-09-19T04:16:46.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.491+0000 7f8b6994d640 1 -- 172.21.15.76:0/2170806002 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b640b3b30 con 0x7f8b6415c2a0 2024-09-19T04:16:46.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6994d640 1 --2- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415c2a0 0x7f8b640aed60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b540097c0 tx=0x7f8b5401c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=766e52f4b0591b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6914c640 1 -- 172.21.15.76:0/2170806002 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b540b0070 con 0x7f8b6415c2a0 2024-09-19T04:16:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6914c640 1 -- 172.21.15.76:0/2170806002 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b540ab400 con 0x7f8b6415c2a0 2024-09-19T04:16:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6914c640 1 -- 172.21.15.76:0/2170806002 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c001a00 con 0x7f8b6415c2a0 2024-09-19T04:16:46.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415c2a0 msgr2=0x7f8b640aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415c2a0 0x7f8b640aed60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b540097c0 tx=0x7f8b5401c390 comp rx=0 tx=0).stop 2024-09-19T04:16:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/2170806002 shutdown_connections 2024-09-19T04:16:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415c2a0 0x7f8b640aed60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/2170806002 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b6415bec0 0x7f8b640ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/2170806002 >> 172.21.15.76:0/2170806002 conn(0x7f8b640b1f90 msgr2=0x7f8b640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/2170806002 shutdown_connections 2024-09-19T04:16:46.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/2170806002 wait complete. 2024-09-19T04:16:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 Processor -- start 2024-09-19T04:16:46.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- start start 2024-09-19T04:16:46.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b640af2a0 0x7f8b64156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415bec0 0x7f8b6414a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b6415c2a0 0x7f8b6414eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8b640ad740 con 0x7f8b6415c2a0 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8b640afc90 con 0x7f8b6415bec0 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.495+0000 7f8b6c3b2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8b64169000 con 0x7f8b640af2a0 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b6415c2a0 0x7f8b6414eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b6415c2a0 0x7f8b6414eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44630/0 (socket says 172.21.15.76:44630) 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 -- 172.21.15.76:0/1962639488 learned_addr learned my addr 172.21.15.76:0/1962639488 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a14e640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b640af2a0 0x7f8b64156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6994d640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415bec0 0x7f8b6414a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 -- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b640af2a0 msgr2=0x7f8b64156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b640af2a0 0x7f8b64156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 -- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415bec0 msgr2=0x7f8b6414a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415bec0 0x7f8b6414a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 -- 172.21.15.76:0/1962639488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b640720b0 con 0x7f8b6415c2a0 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6994d640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415bec0 0x7f8b6414a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a94f640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b6415c2a0 0x7f8b6414eeb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0018d0 tx=0x7f8b5c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b5affd640 1 -- 172.21.15.76:0/1962639488 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c012400 con 0x7f8b6415c2a0 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b5affd640 1 -- 172.21.15.76:0/1962639488 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b5c0125c0 con 0x7f8b6415c2a0 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b64072390 con 0x7f8b6415c2a0 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b5affd640 1 -- 172.21.15.76:0/1962639488 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c0129b0 con 0x7f8b6415c2a0 2024-09-19T04:16:46.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b64072950 con 0x7f8b6415c2a0 2024-09-19T04:16:46.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b640ad200 con 0x7f8b6415c2a0 2024-09-19T04:16:46.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b5affd640 1 -- 172.21.15.76:0/1962639488 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8b5c012b50 con 0x7f8b6415c2a0 2024-09-19T04:16:46.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b5affd640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b480678f0 0x7f8b48069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b5affd640 1 -- 172.21.15.76:0/1962639488 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f8b5c086750 con 0x7f8b6415c2a0 2024-09-19T04:16:46.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a14e640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b480678f0 0x7f8b48069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.499+0000 7f8b6a14e640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b480678f0 0x7f8b48069db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8b640cf530 tx=0x7f8b60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.503+0000 7f8b5affd640 1 -- 172.21.15.76:0/1962639488 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8b5c0182b0 con 0x7f8b6415c2a0 2024-09-19T04:16:46.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.667+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b6406fcd0 con 0x7f8b6415c2a0 2024-09-19T04:16:46.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.667+0000 7f8b5affd640 1 -- 172.21.15.76:0/1962639488 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16968 (secure 0 0 0) 0x7f8b5c057130 con 0x7f8b6415c2a0 2024-09-19T04:16:46.673 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:46.673 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":67,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:16:43.915354+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:16:40.908065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T04:16:41.340553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:16:46.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b480678f0 msgr2=0x7f8b48069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b480678f0 0x7f8b48069db0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8b640cf530 tx=0x7f8b60008040 comp rx=0 tx=0).stop 2024-09-19T04:16:46.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b6415c2a0 msgr2=0x7f8b6414eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b6415c2a0 0x7f8b6414eeb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0018d0 tx=0x7f8b5c00a430 comp rx=0 tx=0).stop 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 shutdown_connections 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8b480678f0 0x7f8b48069db0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8b6415c2a0 0x7f8b6414eeb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b6415bec0 0x7f8b6414a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 --2- 172.21.15.76:0/1962639488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8b640af2a0 0x7f8b64156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 >> 172.21.15.76:0/1962639488 conn(0x7f8b640b1f90 msgr2=0x7f8b6415adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 shutdown_connections 2024-09-19T04:16:46.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.671+0000 7f8b6c3b2640 1 -- 172.21.15.76:0/1962639488 wait complete. 2024-09-19T04:16:46.693 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:16:46.693 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:16:46.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.807+0000 7fb377015640 1 Processor -- start 2024-09-19T04:16:46.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.807+0000 7fb377015640 1 -- start start 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb377015640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb3701587a0 0x7fb3700b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb377015640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb37006e680 0x7fb370072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb377015640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb37006ec60 con 0x7fb3700b37d0 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb377015640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb3700b3c40 con 0x7fb3701587a0 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb377015640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb37006f870 con 0x7fb37006e680 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb3701587a0 0x7fb3700b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb3701587a0 0x7fb3700b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44636/0 (socket says 172.21.15.76:44636) 2024-09-19T04:16:46.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 -- 172.21.15.76:0/2281439836 learned_addr learned my addr 172.21.15.76:0/2281439836 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:46.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb3755b2640 1 --2- 172.21.15.76:0/2281439836 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb37006e680 0x7fb370072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 -- 172.21.15.76:0/2281439836 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb37006e680 msgr2=0x7fb370072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:46.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 --2- 172.21.15.76:0/2281439836 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb37006e680 0x7fb370072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 -- 172.21.15.76:0/2281439836 >> v1:172.21.15.116:6789/0 conn(0x7fb3700b37d0 legacy=0x7fb37006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:46.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 -- 172.21.15.76:0/2281439836 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37006ef40 con 0x7fb3701587a0 2024-09-19T04:16:46.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.811+0000 7fb374db1640 1 --2- 172.21.15.76:0/2281439836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3701587a0 0x7fb3700b3290 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009870 tx=0x7fb36c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8447e16105522af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb3677fe640 1 -- 172.21.15.76:0/2281439836 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b1070 con 0x7fb3701587a0 2024-09-19T04:16:46.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb3677fe640 1 -- 172.21.15.76:0/2281439836 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c0ac620 con 0x7fb3701587a0 2024-09-19T04:16:46.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb3677fe640 1 -- 172.21.15.76:0/2281439836 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0ac9f0 con 0x7fb3701587a0 2024-09-19T04:16:46.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- 172.21.15.76:0/2281439836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3701587a0 msgr2=0x7fb3700b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 --2- 172.21.15.76:0/2281439836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3701587a0 0x7fb3700b3290 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009870 tx=0x7fb36c01c4f0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- 172.21.15.76:0/2281439836 shutdown_connections 2024-09-19T04:16:46.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 --2- 172.21.15.76:0/2281439836 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb37006e680 0x7fb370072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 --2- 172.21.15.76:0/2281439836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb3701587a0 0x7fb3700b3290 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- 172.21.15.76:0/2281439836 >> 172.21.15.76:0/2281439836 conn(0x7fb3700b1f90 msgr2=0x7fb3700b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:46.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- 172.21.15.76:0/2281439836 shutdown_connections 2024-09-19T04:16:46.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- 172.21.15.76:0/2281439836 wait complete. 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 Processor -- start 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- start start 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb37006e680 0x7fb37014c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3700b37d0 0x7fb37014c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3701587a0 0x7fb370147250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb370073030 con 0x7fb37006e680 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb370072eb0 con 0x7fb3700b37d0 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb377015640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb3700731b0 con 0x7fb3701587a0 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb37006e680 0x7fb37014c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb37006e680 0x7fb37014c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44642/0 (socket says 172.21.15.76:44642) 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 -- 172.21.15.76:0/3331152989 learned_addr learned my addr 172.21.15.76:0/3331152989 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb3755b2640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3701587a0 0x7fb370147250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 -- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3701587a0 msgr2=0x7fb370147250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3701587a0 0x7fb370147250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb367fff640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3700b37d0 0x7fb37014c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 -- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3700b37d0 msgr2=0x7fb37014c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3700b37d0 0x7fb37014c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 -- 172.21.15.76:0/3331152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3700af890 con 0x7fb37006e680 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb367fff640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3700b37d0 0x7fb37014c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb374db1640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb37006e680 0x7fb37014c380 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0af040 tx=0x7fb36c019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb3657fa640 1 -- 172.21.15.76:0/3331152989 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b1040 con 0x7fb37006e680 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb3657fa640 1 -- 172.21.15.76:0/3331152989 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36c0ba070 con 0x7fb37006e680 2024-09-19T04:16:46.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.815+0000 7fb3657fa640 1 -- 172.21.15.76:0/3331152989 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b5680 con 0x7fb37006e680 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3700afb70 con 0x7fb37006e680 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb370153ff0 con 0x7fb37006e680 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3700b0090 con 0x7fb37006e680 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb3657fa640 1 -- 172.21.15.76:0/3331152989 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb36c0b5820 con 0x7fb37006e680 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb3657fa640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb35c0678c0 0x7fb35c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb3657fa640 1 -- 172.21.15.76:0/3331152989 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fb36c11fb80 con 0x7fb37006e680 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb367fff640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb35c0678c0 0x7fb35c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:46.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.819+0000 7fb367fff640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb35c0678c0 0x7fb35c069d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb370153cf0 tx=0x7fb360009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:46.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.823+0000 7fb3657fa640 1 -- 172.21.15.76:0/3331152989 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb36c016110 con 0x7fb37006e680 2024-09-19T04:16:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.983+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb370072160 con 0x7fb35c0678c0 2024-09-19T04:16:47.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:46.999+0000 7fb3657fa640 1 -- 172.21.15.76:0/3331152989 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246712 (secure 0 0 0) 0x7fb3600543d0 con 0x7fb35c0678c0 2024-09-19T04:16:47.007 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:47.009 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:16:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb35c0678c0 msgr2=0x7fb35c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb35c0678c0 0x7fb35c069d80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb370153cf0 tx=0x7fb360009040 comp rx=0 tx=0).stop 2024-09-19T04:16:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb37006e680 msgr2=0x7fb37014c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb37006e680 0x7fb37014c380 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0af040 tx=0x7fb36c019290 comp rx=0 tx=0).stop 2024-09-19T04:16:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 shutdown_connections 2024-09-19T04:16:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb35c0678c0 0x7fb35c069d80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb3701587a0 0x7fb370147250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3700b37d0 0x7fb37014c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 --2- 172.21.15.76:0/3331152989 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb37006e680 0x7fb37014c380 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 >> 172.21.15.76:0/3331152989 conn(0x7fb3700b1f90 msgr2=0x7fb3700ad200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 shutdown_connections 2024-09-19T04:16:47.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.003+0000 7fb377015640 1 -- 172.21.15.76:0/3331152989 wait complete. 2024-09-19T04:16:47.025 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-19T04:16:45.802592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235996,"kb_used_data":7272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738724,"statfs":{"total":773094113280,"available":772852453376,"internally_reserved":0,"allocated":7446528,"data_stored":4018688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.838052"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.981392+0000","last_change":"2024-09-19T04:16:41.981392+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.981392+0000","last_undegraded":"2024-09-19T04:16:41.981392+0000","last_fullsized":"2024-09-19T04:16:41.981392+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980603+0000","last_change":"2024-09-19T04:16:41.980603+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980603+0000","last_undegraded":"2024-09-19T04:16:41.980603+0000","last_fullsized":"2024-09-19T04:16:41.980603+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982715+0000","last_change":"2024-09-19T04:16:41.982715+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982715+0000","last_undegraded":"2024-09-19T04:16:41.982715+0000","last_fullsized":"2024-09-19T04:16:41.982715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984499+0000","last_change":"2024-09-19T04:16:41.984499+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984499+0000","last_undegraded":"2024-09-19T04:16:41.984499+0000","last_fullsized":"2024-09-19T04:16:41.984499+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.998802+0000","last_change":"2024-09-19T04:16:41.998802+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.998802+0000","last_undegraded":"2024-09-19T04:16:41.998802+0000","last_fullsized":"2024-09-19T04:16:41.998802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.988501+0000","last_change":"2024-09-19T04:16:41.988501+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.988501+0000","last_undegraded":"2024-09-19T04:16:41.988501+0000","last_fullsized":"2024-09-19T04:16:41.988501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984236+0000","last_change":"2024-09-19T04:16:41.984236+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984236+0000","last_undegraded":"2024-09-19T04:16:41.984236+0000","last_fullsized":"2024-09-19T04:16:41.984236+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976138+0000","last_change":"2024-09-19T04:16:41.976138+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976138+0000","last_undegraded":"2024-09-19T04:16:41.976138+0000","last_fullsized":"2024-09-19T04:16:41.976138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984883+0000","last_change":"2024-09-19T04:16:41.984883+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984883+0000","last_undegraded":"2024-09-19T04:16:41.984883+0000","last_fullsized":"2024-09-19T04:16:41.984883+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984199+0000","last_change":"2024-09-19T04:16:41.984199+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984199+0000","last_undegraded":"2024-09-19T04:16:41.984199+0000","last_fullsized":"2024-09-19T04:16:41.984199+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987719+0000","last_change":"2024-09-19T04:16:41.987719+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987719+0000","last_undegraded":"2024-09-19T04:16:41.987719+0000","last_fullsized":"2024-09-19T04:16:41.987719+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976359+0000","last_change":"2024-09-19T04:16:41.976359+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976359+0000","last_undegraded":"2024-09-19T04:16:41.976359+0000","last_fullsized":"2024-09-19T04:16:41.976359+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979859+0000","last_change":"2024-09-19T04:16:41.979859+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979859+0000","last_undegraded":"2024-09-19T04:16:41.979859+0000","last_fullsized":"2024-09-19T04:16:41.979859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.983462+0000","last_change":"2024-09-19T04:16:41.983462+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.983462+0000","last_undegraded":"2024-09-19T04:16:41.983462+0000","last_fullsized":"2024-09-19T04:16:41.983462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982729+0000","last_change":"2024-09-19T04:16:41.982729+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982729+0000","last_undegraded":"2024-09-19T04:16:41.982729+0000","last_fullsized":"2024-09-19T04:16:41.982729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985026+0000","last_change":"2024-09-19T04:16:41.985026+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985026+0000","last_undegraded":"2024-09-19T04:16:41.985026+0000","last_fullsized":"2024-09-19T04:16:41.985026+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980867+0000","last_change":"2024-09-19T04:16:41.980867+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980867+0000","last_undegraded":"2024-09-19T04:16:41.980867+0000","last_fullsized":"2024-09-19T04:16:41.980867+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985372+0000","last_change":"2024-09-19T04:16:41.985372+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985372+0000","last_undegraded":"2024-09-19T04:16:41.985372+0000","last_fullsized":"2024-09-19T04:16:41.985372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999302+0000","last_change":"2024-09-19T04:16:41.999302+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999302+0000","last_undegraded":"2024-09-19T04:16:41.999302+0000","last_fullsized":"2024-09-19T04:16:41.999302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980025+0000","last_change":"2024-09-19T04:16:41.980025+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980025+0000","last_undegraded":"2024-09-19T04:16:41.980025+0000","last_fullsized":"2024-09-19T04:16:41.980025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.977096+0000","last_change":"2024-09-19T04:16:41.977096+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.977096+0000","last_undegraded":"2024-09-19T04:16:41.977096+0000","last_fullsized":"2024-09-19T04:16:41.977096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984660+0000","last_change":"2024-09-19T04:16:41.984660+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984660+0000","last_undegraded":"2024-09-19T04:16:41.984660+0000","last_fullsized":"2024-09-19T04:16:41.984660+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.983716+0000","last_change":"2024-09-19T04:16:41.983716+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.983716+0000","last_undegraded":"2024-09-19T04:16:41.983716+0000","last_fullsized":"2024-09-19T04:16:41.983716+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982552+0000","last_change":"2024-09-19T04:16:41.982552+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982552+0000","last_undegraded":"2024-09-19T04:16:41.982552+0000","last_fullsized":"2024-09-19T04:16:41.982552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.977837+0000","last_change":"2024-09-19T04:16:41.977837+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.977837+0000","last_undegraded":"2024-09-19T04:16:41.977837+0000","last_fullsized":"2024-09-19T04:16:41.977837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979819+0000","last_change":"2024-09-19T04:16:41.979819+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979819+0000","last_undegraded":"2024-09-19T04:16:41.979819+0000","last_fullsized":"2024-09-19T04:16:41.979819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986619+0000","last_change":"2024-09-19T04:16:41.986619+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986619+0000","last_undegraded":"2024-09-19T04:16:41.986619+0000","last_fullsized":"2024-09-19T04:16:41.986619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980172+0000","last_change":"2024-09-19T04:16:41.980172+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980172+0000","last_undegraded":"2024-09-19T04:16:41.980172+0000","last_fullsized":"2024-09-19T04:16:41.980172+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982680+0000","last_change":"2024-09-19T04:16:41.982680+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982680+0000","last_undegraded":"2024-09-19T04:16:41.982680+0000","last_fullsized":"2024-09-19T04:16:41.982680+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978736+0000","last_change":"2024-09-19T04:16:41.978736+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978736+0000","last_undegraded":"2024-09-19T04:16:41.978736+0000","last_fullsized":"2024-09-19T04:16:41.978736+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.989513+0000","last_change":"2024-09-19T04:16:41.989513+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.989513+0000","last_undegraded":"2024-09-19T04:16:41.989513+0000","last_fullsized":"2024-09-19T04:16:41.989513+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.990810+0000","last_change":"2024-09-19T04:16:41.990810+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.990810+0000","last_undegraded":"2024-09-19T04:16:41.990810+0000","last_fullsized":"2024-09-19T04:16:41.990810+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.993046+0000","last_change":"2024-09-19T04:16:41.993046+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.993046+0000","last_undegraded":"2024-09-19T04:16:41.993046+0000","last_fullsized":"2024-09-19T04:16:41.993046+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.017233+0000","last_change":"2024-09-19T04:16:42.013739+0000","last_active":"2024-09-19T04:16:42.017233+0000","last_peered":"2024-09-19T04:16:42.017233+0000","last_clean":"2024-09-19T04:16:42.017233+0000","last_became_active":"2024-09-19T04:16:42.013362+0000","last_became_peered":"2024-09-19T04:16:42.013362+0000","last_unstale":"2024-09-19T04:16:42.017233+0000","last_undegraded":"2024-09-19T04:16:42.017233+0000","last_fullsized":"2024-09-19T04:16:42.017233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:28.800656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985724+0000","last_change":"2024-09-19T04:16:41.985724+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985724+0000","last_undegraded":"2024-09-19T04:16:41.985724+0000","last_fullsized":"2024-09-19T04:16:41.985724+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.019939+0000","last_change":"2024-09-19T04:16:42.018926+0000","last_active":"2024-09-19T04:16:42.019939+0000","last_peered":"2024-09-19T04:16:42.019939+0000","last_clean":"2024-09-19T04:16:42.019939+0000","last_became_active":"2024-09-19T04:16:42.018655+0000","last_became_peered":"2024-09-19T04:16:42.018655+0000","last_unstale":"2024-09-19T04:16:42.019939+0000","last_undegraded":"2024-09-19T04:16:42.019939+0000","last_fullsized":"2024-09-19T04:16:42.019939+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:44.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.988796+0000","last_change":"2024-09-19T04:16:41.988796+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.988796+0000","last_undegraded":"2024-09-19T04:16:41.988796+0000","last_fullsized":"2024-09-19T04:16:41.988796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.999950+0000","last_change":"2024-09-19T04:16:41.992501+0000","last_active":"2024-09-19T04:16:41.999950+0000","last_peered":"2024-09-19T04:16:41.999950+0000","last_clean":"2024-09-19T04:16:41.999950+0000","last_became_active":"2024-09-19T04:16:41.992135+0000","last_became_peered":"2024-09-19T04:16:41.992135+0000","last_unstale":"2024-09-19T04:16:41.999950+0000","last_undegraded":"2024-09-19T04:16:41.999950+0000","last_fullsized":"2024-09-19T04:16:41.999950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.537033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.991338+0000","last_change":"2024-09-19T04:16:41.991338+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.991338+0000","last_undegraded":"2024-09-19T04:16:41.991338+0000","last_fullsized":"2024-09-19T04:16:41.991338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005233+0000","last_change":"2024-09-19T04:16:42.000668+0000","last_active":"2024-09-19T04:16:42.005233+0000","last_peered":"2024-09-19T04:16:42.005233+0000","last_clean":"2024-09-19T04:16:42.005233+0000","last_became_active":"2024-09-19T04:16:42.000249+0000","last_became_peered":"2024-09-19T04:16:42.000249+0000","last_unstale":"2024-09-19T04:16:42.005233+0000","last_undegraded":"2024-09-19T04:16:42.005233+0000","last_fullsized":"2024-09-19T04:16:42.005233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.567498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984416+0000","last_change":"2024-09-19T04:16:41.984416+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984416+0000","last_undegraded":"2024-09-19T04:16:41.984416+0000","last_fullsized":"2024-09-19T04:16:41.984416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.990792+0000","last_change":"2024-09-19T04:16:41.989580+0000","last_active":"2024-09-19T04:16:41.990792+0000","last_peered":"2024-09-19T04:16:41.990792+0000","last_clean":"2024-09-19T04:16:41.990792+0000","last_became_active":"2024-09-19T04:16:41.989078+0000","last_became_peered":"2024-09-19T04:16:41.989078+0000","last_unstale":"2024-09-19T04:16:41.990792+0000","last_undegraded":"2024-09-19T04:16:41.990792+0000","last_fullsized":"2024-09-19T04:16:41.990792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.567498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978924+0000","last_change":"2024-09-19T04:16:41.978924+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978924+0000","last_undegraded":"2024-09-19T04:16:41.978924+0000","last_fullsized":"2024-09-19T04:16:41.978924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997003+0000","last_change":"2024-09-19T04:16:41.995699+0000","last_active":"2024-09-19T04:16:41.997003+0000","last_peered":"2024-09-19T04:16:41.997003+0000","last_clean":"2024-09-19T04:16:41.997003+0000","last_became_active":"2024-09-19T04:16:41.993440+0000","last_became_peered":"2024-09-19T04:16:41.993440+0000","last_unstale":"2024-09-19T04:16:41.997003+0000","last_undegraded":"2024-09-19T04:16:41.997003+0000","last_fullsized":"2024-09-19T04:16:41.997003+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:12.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.015296+0000","last_change":"2024-09-19T04:16:42.015296+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.015296+0000","last_undegraded":"2024-09-19T04:16:42.015296+0000","last_fullsized":"2024-09-19T04:16:42.015296+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024309+0000","last_change":"2024-09-19T04:16:42.022337+0000","last_active":"2024-09-19T04:16:42.024309+0000","last_peered":"2024-09-19T04:16:42.024309+0000","last_clean":"2024-09-19T04:16:42.024309+0000","last_became_active":"2024-09-19T04:16:42.021791+0000","last_became_peered":"2024-09-19T04:16:42.021791+0000","last_unstale":"2024-09-19T04:16:42.024309+0000","last_undegraded":"2024-09-19T04:16:42.024309+0000","last_fullsized":"2024-09-19T04:16:42.024309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:33.295425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980459+0000","last_change":"2024-09-19T04:16:41.980459+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980459+0000","last_undegraded":"2024-09-19T04:16:41.980459+0000","last_fullsized":"2024-09-19T04:16:41.980459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.021722+0000","last_change":"2024-09-19T04:16:42.019556+0000","last_active":"2024-09-19T04:16:42.021722+0000","last_peered":"2024-09-19T04:16:42.021722+0000","last_clean":"2024-09-19T04:16:42.021722+0000","last_became_active":"2024-09-19T04:16:42.018870+0000","last_became_peered":"2024-09-19T04:16:42.018870+0000","last_unstale":"2024-09-19T04:16:42.021722+0000","last_undegraded":"2024-09-19T04:16:42.021722+0000","last_fullsized":"2024-09-19T04:16:42.021722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:19.666213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979749+0000","last_change":"2024-09-19T04:16:41.979749+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979749+0000","last_undegraded":"2024-09-19T04:16:41.979749+0000","last_fullsized":"2024-09-19T04:16:41.979749+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028493+0000","last_change":"2024-09-19T04:16:42.026415+0000","last_active":"2024-09-19T04:16:42.028493+0000","last_peered":"2024-09-19T04:16:42.028493+0000","last_clean":"2024-09-19T04:16:42.028493+0000","last_became_active":"2024-09-19T04:16:42.025740+0000","last_became_peered":"2024-09-19T04:16:42.025740+0000","last_unstale":"2024-09-19T04:16:42.028493+0000","last_undegraded":"2024-09-19T04:16:42.028493+0000","last_fullsized":"2024-09-19T04:16:42.028493+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:36.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979412+0000","last_change":"2024-09-19T04:16:41.979412+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979412+0000","last_undegraded":"2024-09-19T04:16:41.979412+0000","last_fullsized":"2024-09-19T04:16:41.979412+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005588+0000","last_change":"2024-09-19T04:16:42.001820+0000","last_active":"2024-09-19T04:16:42.005588+0000","last_peered":"2024-09-19T04:16:42.005588+0000","last_clean":"2024-09-19T04:16:42.005588+0000","last_became_active":"2024-09-19T04:16:42.001141+0000","last_became_peered":"2024-09-19T04:16:42.001141+0000","last_unstale":"2024-09-19T04:16:42.005588+0000","last_undegraded":"2024-09-19T04:16:42.005588+0000","last_fullsized":"2024-09-19T04:16:42.005588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:28.101048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.994594+0000","last_change":"2024-09-19T04:16:41.994594+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.994594+0000","last_undegraded":"2024-09-19T04:16:41.994594+0000","last_fullsized":"2024-09-19T04:16:41.994594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.025625+0000","last_change":"2024-09-19T04:16:42.018953+0000","last_active":"2024-09-19T04:16:42.025625+0000","last_peered":"2024-09-19T04:16:42.025625+0000","last_clean":"2024-09-19T04:16:42.025625+0000","last_became_active":"2024-09-19T04:16:42.018215+0000","last_became_peered":"2024-09-19T04:16:42.018215+0000","last_unstale":"2024-09-19T04:16:42.025625+0000","last_undegraded":"2024-09-19T04:16:42.025625+0000","last_fullsized":"2024-09-19T04:16:42.025625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:09.582870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987703+0000","last_change":"2024-09-19T04:16:41.987703+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987703+0000","last_undegraded":"2024-09-19T04:16:41.987703+0000","last_fullsized":"2024-09-19T04:16:41.987703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.008487+0000","last_change":"2024-09-19T04:16:42.006482+0000","last_active":"2024-09-19T04:16:42.008487+0000","last_peered":"2024-09-19T04:16:42.008487+0000","last_clean":"2024-09-19T04:16:42.008487+0000","last_became_active":"2024-09-19T04:16:42.006111+0000","last_became_peered":"2024-09-19T04:16:42.006111+0000","last_unstale":"2024-09-19T04:16:42.008487+0000","last_undegraded":"2024-09-19T04:16:42.008487+0000","last_fullsized":"2024-09-19T04:16:42.008487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:11.291559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986477+0000","last_change":"2024-09-19T04:16:41.986477+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986477+0000","last_undegraded":"2024-09-19T04:16:41.986477+0000","last_fullsized":"2024-09-19T04:16:41.986477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028432+0000","last_change":"2024-09-19T04:16:42.024717+0000","last_active":"2024-09-19T04:16:42.028432+0000","last_peered":"2024-09-19T04:16:42.028432+0000","last_clean":"2024-09-19T04:16:42.028432+0000","last_became_active":"2024-09-19T04:16:42.023979+0000","last_became_peered":"2024-09-19T04:16:42.023979+0000","last_unstale":"2024-09-19T04:16:42.028432+0000","last_undegraded":"2024-09-19T04:16:42.028432+0000","last_fullsized":"2024-09-19T04:16:42.028432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:14.632041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978077+0000","last_change":"2024-09-19T04:16:41.978077+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978077+0000","last_undegraded":"2024-09-19T04:16:41.978077+0000","last_fullsized":"2024-09-19T04:16:41.978077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.006392+0000","last_change":"2024-09-19T04:16:42.002996+0000","last_active":"2024-09-19T04:16:42.006392+0000","last_peered":"2024-09-19T04:16:42.006392+0000","last_clean":"2024-09-19T04:16:42.006392+0000","last_became_active":"2024-09-19T04:16:42.001542+0000","last_became_peered":"2024-09-19T04:16:42.001542+0000","last_unstale":"2024-09-19T04:16:42.006392+0000","last_undegraded":"2024-09-19T04:16:42.006392+0000","last_fullsized":"2024-09-19T04:16:42.006392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:08.070747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.014200+0000","last_change":"2024-09-19T04:16:42.014200+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.014200+0000","last_undegraded":"2024-09-19T04:16:42.014200+0000","last_fullsized":"2024-09-19T04:16:42.014200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024770+0000","last_change":"2024-09-19T04:16:42.018769+0000","last_active":"2024-09-19T04:16:42.024770+0000","last_peered":"2024-09-19T04:16:42.024770+0000","last_clean":"2024-09-19T04:16:42.024770+0000","last_became_active":"2024-09-19T04:16:42.017955+0000","last_became_peered":"2024-09-19T04:16:42.017955+0000","last_unstale":"2024-09-19T04:16:42.024770+0000","last_undegraded":"2024-09-19T04:16:42.024770+0000","last_fullsized":"2024-09-19T04:16:42.024770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:19.225233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979070+0000","last_change":"2024-09-19T04:16:41.979070+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979070+0000","last_undegraded":"2024-09-19T04:16:41.979070+0000","last_fullsized":"2024-09-19T04:16:41.979070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.016442+0000","last_change":"2024-09-19T04:16:42.011251+0000","last_active":"2024-09-19T04:16:42.016442+0000","last_peered":"2024-09-19T04:16:42.016442+0000","last_clean":"2024-09-19T04:16:42.016442+0000","last_became_active":"2024-09-19T04:16:42.010096+0000","last_became_peered":"2024-09-19T04:16:42.010096+0000","last_unstale":"2024-09-19T04:16:42.016442+0000","last_undegraded":"2024-09-19T04:16:42.016442+0000","last_fullsized":"2024-09-19T04:16:42.016442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:09.087251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987287+0000","last_change":"2024-09-19T04:16:41.987287+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987287+0000","last_undegraded":"2024-09-19T04:16:41.987287+0000","last_fullsized":"2024-09-19T04:16:41.987287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.001783+0000","last_change":"2024-09-19T04:16:42.000089+0000","last_active":"2024-09-19T04:16:42.001783+0000","last_peered":"2024-09-19T04:16:42.001783+0000","last_clean":"2024-09-19T04:16:42.001783+0000","last_became_active":"2024-09-19T04:16:41.999376+0000","last_became_peered":"2024-09-19T04:16:41.999376+0000","last_unstale":"2024-09-19T04:16:42.001783+0000","last_undegraded":"2024-09-19T04:16:42.001783+0000","last_fullsized":"2024-09-19T04:16:42.001783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:34.395480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.997756+0000","last_change":"2024-09-19T04:16:41.997756+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.997756+0000","last_undegraded":"2024-09-19T04:16:41.997756+0000","last_fullsized":"2024-09-19T04:16:41.997756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.027534+0000","last_change":"2024-09-19T04:16:42.023483+0000","last_active":"2024-09-19T04:16:42.027534+0000","last_peered":"2024-09-19T04:16:42.027534+0000","last_clean":"2024-09-19T04:16:42.027534+0000","last_became_active":"2024-09-19T04:16:42.022787+0000","last_became_peered":"2024-09-19T04:16:42.022787+0000","last_unstale":"2024-09-19T04:16:42.027534+0000","last_undegraded":"2024-09-19T04:16:42.027534+0000","last_fullsized":"2024-09-19T04:16:42.027534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:54.902392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984262+0000","last_change":"2024-09-19T04:16:41.984262+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984262+0000","last_undegraded":"2024-09-19T04:16:41.984262+0000","last_fullsized":"2024-09-19T04:16:41.984262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028461+0000","last_change":"2024-09-19T04:16:42.026255+0000","last_active":"2024-09-19T04:16:42.028461+0000","last_peered":"2024-09-19T04:16:42.028461+0000","last_clean":"2024-09-19T04:16:42.028461+0000","last_became_active":"2024-09-19T04:16:42.025430+0000","last_became_peered":"2024-09-19T04:16:42.025430+0000","last_unstale":"2024-09-19T04:16:42.028461+0000","last_undegraded":"2024-09-19T04:16:42.028461+0000","last_fullsized":"2024-09-19T04:16:42.028461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:18.263858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999999+0000","last_change":"2024-09-19T04:16:41.999999+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999999+0000","last_undegraded":"2024-09-19T04:16:41.999999+0000","last_fullsized":"2024-09-19T04:16:41.999999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.999802+0000","last_change":"2024-09-19T04:16:41.998882+0000","last_active":"2024-09-19T04:16:41.999802+0000","last_peered":"2024-09-19T04:16:41.999802+0000","last_clean":"2024-09-19T04:16:41.999802+0000","last_became_active":"2024-09-19T04:16:41.998612+0000","last_became_peered":"2024-09-19T04:16:41.998612+0000","last_unstale":"2024-09-19T04:16:41.999802+0000","last_undegraded":"2024-09-19T04:16:41.999802+0000","last_fullsized":"2024-09-19T04:16:41.999802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:44.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987405+0000","last_change":"2024-09-19T04:16:41.987405+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987405+0000","last_undegraded":"2024-09-19T04:16:41.987405+0000","last_fullsized":"2024-09-19T04:16:41.987405+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.017443+0000","last_change":"2024-09-19T04:16:42.015741+0000","last_active":"2024-09-19T04:16:42.017443+0000","last_peered":"2024-09-19T04:16:42.017443+0000","last_clean":"2024-09-19T04:16:42.017443+0000","last_became_active":"2024-09-19T04:16:42.015285+0000","last_became_peered":"2024-09-19T04:16:42.015285+0000","last_unstale":"2024-09-19T04:16:42.017443+0000","last_undegraded":"2024-09-19T04:16:42.017443+0000","last_fullsized":"2024-09-19T04:16:42.017443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:12.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976440+0000","last_change":"2024-09-19T04:16:41.976440+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976440+0000","last_undegraded":"2024-09-19T04:16:41.976440+0000","last_fullsized":"2024-09-19T04:16:41.976440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.004913+0000","last_change":"2024-09-19T04:16:42.002264+0000","last_active":"2024-09-19T04:16:42.004913+0000","last_peered":"2024-09-19T04:16:42.004913+0000","last_clean":"2024-09-19T04:16:42.004913+0000","last_became_active":"2024-09-19T04:16:42.000593+0000","last_became_peered":"2024-09-19T04:16:42.000593+0000","last_unstale":"2024-09-19T04:16:42.004913+0000","last_undegraded":"2024-09-19T04:16:42.004913+0000","last_fullsized":"2024-09-19T04:16:42.004913+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:36.127882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999116+0000","last_change":"2024-09-19T04:16:41.999116+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999116+0000","last_undegraded":"2024-09-19T04:16:41.999116+0000","last_fullsized":"2024-09-19T04:16:41.999116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.011261+0000","last_change":"2024-09-19T04:16:42.010194+0000","last_active":"2024-09-19T04:16:42.011261+0000","last_peered":"2024-09-19T04:16:42.011261+0000","last_clean":"2024-09-19T04:16:42.011261+0000","last_became_active":"2024-09-19T04:16:42.009783+0000","last_became_peered":"2024-09-19T04:16:42.009783+0000","last_unstale":"2024-09-19T04:16:42.011261+0000","last_undegraded":"2024-09-19T04:16:42.011261+0000","last_fullsized":"2024-09-19T04:16:42.011261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:02.962114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987074+0000","last_change":"2024-09-19T04:16:41.987074+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987074+0000","last_undegraded":"2024-09-19T04:16:41.987074+0000","last_fullsized":"2024-09-19T04:16:41.987074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.012421+0000","last_change":"2024-09-19T04:16:42.010812+0000","last_active":"2024-09-19T04:16:42.012421+0000","last_peered":"2024-09-19T04:16:42.012421+0000","last_clean":"2024-09-19T04:16:42.012421+0000","last_became_active":"2024-09-19T04:16:42.010051+0000","last_became_peered":"2024-09-19T04:16:42.010051+0000","last_unstale":"2024-09-19T04:16:42.012421+0000","last_undegraded":"2024-09-19T04:16:42.012421+0000","last_fullsized":"2024-09-19T04:16:42.012421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:52:53.450248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986784+0000","last_change":"2024-09-19T04:16:41.986784+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986784+0000","last_undegraded":"2024-09-19T04:16:41.986784+0000","last_fullsized":"2024-09-19T04:16:41.986784+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997616+0000","last_change":"2024-09-19T04:16:41.995074+0000","last_active":"2024-09-19T04:16:41.997616+0000","last_peered":"2024-09-19T04:16:41.997616+0000","last_clean":"2024-09-19T04:16:41.997616+0000","last_became_active":"2024-09-19T04:16:41.994321+0000","last_became_peered":"2024-09-19T04:16:41.994321+0000","last_unstale":"2024-09-19T04:16:41.997616+0000","last_undegraded":"2024-09-19T04:16:41.997616+0000","last_fullsized":"2024-09-19T04:16:41.997616+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.017414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978779+0000","last_change":"2024-09-19T04:16:41.978779+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978779+0000","last_undegraded":"2024-09-19T04:16:41.978779+0000","last_fullsized":"2024-09-19T04:16:41.978779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997258+0000","last_change":"2024-09-19T04:16:41.991513+0000","last_active":"2024-09-19T04:16:41.997258+0000","last_peered":"2024-09-19T04:16:41.997258+0000","last_clean":"2024-09-19T04:16:41.997258+0000","last_became_active":"2024-09-19T04:16:41.990986+0000","last_became_peered":"2024-09-19T04:16:41.990986+0000","last_unstale":"2024-09-19T04:16:41.997258+0000","last_undegraded":"2024-09-19T04:16:41.997258+0000","last_fullsized":"2024-09-19T04:16:41.997258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:47.451247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.998881+0000","last_change":"2024-09-19T04:16:41.998881+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.998881+0000","last_undegraded":"2024-09-19T04:16:41.998881+0000","last_fullsized":"2024-09-19T04:16:41.998881+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005686+0000","last_change":"2024-09-19T04:16:42.003466+0000","last_active":"2024-09-19T04:16:42.005686+0000","last_peered":"2024-09-19T04:16:42.005686+0000","last_clean":"2024-09-19T04:16:42.005686+0000","last_became_active":"2024-09-19T04:16:42.003135+0000","last_became_peered":"2024-09-19T04:16:42.003135+0000","last_unstale":"2024-09-19T04:16:42.005686+0000","last_undegraded":"2024-09-19T04:16:42.005686+0000","last_fullsized":"2024-09-19T04:16:42.005686+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:23.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.014982+0000","last_change":"2024-09-19T04:16:42.014982+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.014982+0000","last_undegraded":"2024-09-19T04:16:42.014982+0000","last_fullsized":"2024-09-19T04:16:42.014982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.026933+0000","last_change":"2024-09-19T04:16:42.019204+0000","last_active":"2024-09-19T04:16:42.026933+0000","last_peered":"2024-09-19T04:16:42.026933+0000","last_clean":"2024-09-19T04:16:42.026933+0000","last_became_active":"2024-09-19T04:16:42.018134+0000","last_became_peered":"2024-09-19T04:16:42.018134+0000","last_unstale":"2024-09-19T04:16:42.026933+0000","last_undegraded":"2024-09-19T04:16:42.026933+0000","last_fullsized":"2024-09-19T04:16:42.026933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:41.461855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987022+0000","last_change":"2024-09-19T04:16:41.987022+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987022+0000","last_undegraded":"2024-09-19T04:16:41.987022+0000","last_fullsized":"2024-09-19T04:16:41.987022+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.996533+0000","last_change":"2024-09-19T04:16:41.994558+0000","last_active":"2024-09-19T04:16:41.996533+0000","last_peered":"2024-09-19T04:16:41.996533+0000","last_clean":"2024-09-19T04:16:41.996533+0000","last_became_active":"2024-09-19T04:16:41.993965+0000","last_became_peered":"2024-09-19T04:16:41.993965+0000","last_unstale":"2024-09-19T04:16:41.996533+0000","last_undegraded":"2024-09-19T04:16:41.996533+0000","last_fullsized":"2024-09-19T04:16:41.996533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:02.962114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980245+0000","last_change":"2024-09-19T04:16:41.980245+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980245+0000","last_undegraded":"2024-09-19T04:16:41.980245+0000","last_fullsized":"2024-09-19T04:16:41.980245+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024381+0000","last_change":"2024-09-19T04:16:42.022760+0000","last_active":"2024-09-19T04:16:42.024381+0000","last_peered":"2024-09-19T04:16:42.024381+0000","last_clean":"2024-09-19T04:16:42.024381+0000","last_became_active":"2024-09-19T04:16:42.022030+0000","last_became_peered":"2024-09-19T04:16:42.022030+0000","last_unstale":"2024-09-19T04:16:42.024381+0000","last_undegraded":"2024-09-19T04:16:42.024381+0000","last_fullsized":"2024-09-19T04:16:42.024381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:59.963896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986891+0000","last_change":"2024-09-19T04:16:41.986891+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986891+0000","last_undegraded":"2024-09-19T04:16:41.986891+0000","last_fullsized":"2024-09-19T04:16:41.986891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024272+0000","last_change":"2024-09-19T04:16:42.022541+0000","last_active":"2024-09-19T04:16:42.024272+0000","last_peered":"2024-09-19T04:16:42.024272+0000","last_clean":"2024-09-19T04:16:42.024272+0000","last_became_active":"2024-09-19T04:16:42.022021+0000","last_became_peered":"2024-09-19T04:16:42.022021+0000","last_unstale":"2024-09-19T04:16:42.024272+0000","last_undegraded":"2024-09-19T04:16:42.024272+0000","last_fullsized":"2024-09-19T04:16:42.024272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:20.649775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.994302+0000","last_change":"2024-09-19T04:16:41.994302+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.994302+0000","last_undegraded":"2024-09-19T04:16:41.994302+0000","last_fullsized":"2024-09-19T04:16:41.994302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.989613+0000","last_change":"2024-09-19T04:16:41.988994+0000","last_active":"2024-09-19T04:16:41.989613+0000","last_peered":"2024-09-19T04:16:41.989613+0000","last_clean":"2024-09-19T04:16:41.989613+0000","last_became_active":"2024-09-19T04:16:41.988755+0000","last_became_peered":"2024-09-19T04:16:41.988755+0000","last_unstale":"2024-09-19T04:16:41.989613+0000","last_undegraded":"2024-09-19T04:16:41.989613+0000","last_fullsized":"2024-09-19T04:16:41.989613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.537033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.009326+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:16:42.009326+0000","last_peered":"2024-09-19T04:16:42.009326+0000","last_clean":"2024-09-19T04:16:42.009326+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:16:42.009326+0000","last_undegraded":"2024-09-19T04:16:42.009326+0000","last_fullsized":"2024-09-19T04:16:42.009326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":945112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":945112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:16:47.030 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:16:47.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4703fb2640 1 Processor -- start 2024-09-19T04:16:47.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4703fb2640 1 -- start start 2024-09-19T04:16:47.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4703fb2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc1520a0 0x7f46fc06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4703fb2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46fc15f3a0 0x7f46fc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4703fb2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46fc070250 con 0x7f46fc06fe50 2024-09-19T04:16:47.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4703fb2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f46fc152480 con 0x7f46fc15f3a0 2024-09-19T04:16:47.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4703fb2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc0aede0 con 0x7f46fc1520a0 2024-09-19T04:16:47.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4701d4e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc1520a0 0x7f46fc06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f470254f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46fc15f3a0 0x7f46fc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4701d4e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc1520a0 0x7f46fc06f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:50308/0 (socket says 172.21.15.76:50308) 2024-09-19T04:16:47.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f4701d4e640 1 -- 172.21.15.76:0/3762752319 learned_addr learned my addr 172.21.15.76:0/3762752319 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:47.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f470254f640 1 -- 172.21.15.76:0/3762752319 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc1520a0 msgr2=0x7f46fc06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f470254f640 1 --2- 172.21.15.76:0/3762752319 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc1520a0 0x7f46fc06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f470254f640 1 -- 172.21.15.76:0/3762752319 >> v1:172.21.15.116:6789/0 conn(0x7f46fc06fe50 legacy=0x7f46fc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.187+0000 7f470254f640 1 -- 172.21.15.76:0/3762752319 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc15fa10 con 0x7f46fc15f3a0 2024-09-19T04:16:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f470254f640 1 --2- 172.21.15.76:0/3762752319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 0x7f46fc0adc80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009d90 tx=0x7f46f401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab4556dbfdac880 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4700d4c640 1 -- 172.21.15.76:0/3762752319 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f40b2070 con 0x7f46fc15f3a0 2024-09-19T04:16:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4700d4c640 1 -- 172.21.15.76:0/3762752319 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46f40a5a30 con 0x7f46fc15f3a0 2024-09-19T04:16:47.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4700d4c640 1 -- 172.21.15.76:0/3762752319 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f40a5e00 con 0x7f46fc15f3a0 2024-09-19T04:16:47.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 -- 172.21.15.76:0/3762752319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 msgr2=0x7f46fc0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 --2- 172.21.15.76:0/3762752319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 0x7f46fc0adc80 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009d90 tx=0x7f46f401c970 comp rx=0 tx=0).stop 2024-09-19T04:16:47.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 -- 172.21.15.76:0/3762752319 shutdown_connections 2024-09-19T04:16:47.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 --2- 172.21.15.76:0/3762752319 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 0x7f46fc0adc80 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 --2- 172.21.15.76:0/3762752319 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc1520a0 0x7f46fc06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 -- 172.21.15.76:0/3762752319 >> 172.21.15.76:0/3762752319 conn(0x7f46fc0b1f90 msgr2=0x7f46fc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:47.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 -- 172.21.15.76:0/3762752319 shutdown_connections 2024-09-19T04:16:47.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 -- 172.21.15.76:0/3762752319 wait complete. 2024-09-19T04:16:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 Processor -- start 2024-09-19T04:16:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.191+0000 7f4703fb2640 1 -- start start 2024-09-19T04:16:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc06fe50 0x7f46fc14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46fc1520a0 0x7f46fc14c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 0x7f46fc14c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc073d30 con 0x7f46fc1520a0 2024-09-19T04:16:47.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc073a50 con 0x7f46fc15f3a0 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f46fc167f10 con 0x7f46fc06fe50 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46fc1520a0 0x7f46fc14c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46fc1520a0 0x7f46fc14c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44648/0 (socket says 172.21.15.76:44648) 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 -- 172.21.15.76:0/1713102245 learned_addr learned my addr 172.21.15.76:0/1713102245 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4701d4e640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc06fe50 0x7f46fc14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470254f640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 0x7f46fc14c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 -- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc06fe50 msgr2=0x7f46fc14aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc06fe50 0x7f46fc14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 -- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 msgr2=0x7f46fc14c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 0x7f46fc14c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 -- 172.21.15.76:0/1713102245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc09a5d0 con 0x7f46fc1520a0 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f470154d640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46fc1520a0 0x7f46fc14c420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c340 tx=0x7f46ec00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f46f27fc640 1 -- 172.21.15.76:0/1713102245 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec0052d0 con 0x7f46fc1520a0 2024-09-19T04:16:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f46f27fc640 1 -- 172.21.15.76:0/1713102245 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46ec005470 con 0x7f46fc1520a0 2024-09-19T04:16:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc09a8b0 con 0x7f46fc1520a0 2024-09-19T04:16:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f46f27fc640 1 -- 172.21.15.76:0/1713102245 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ec007260 con 0x7f46fc1520a0 2024-09-19T04:16:47.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc14afc0 con 0x7f46fc1520a0 2024-09-19T04:16:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46fc06e540 con 0x7f46fc1520a0 2024-09-19T04:16:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f46f27fc640 1 -- 172.21.15.76:0/1713102245 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f46ec007400 con 0x7f46fc1520a0 2024-09-19T04:16:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f46f27fc640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46e00678c0 0x7f46e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f46f27fc640 1 -- 172.21.15.76:0/1713102245 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f46ec0856e0 con 0x7f46fc1520a0 2024-09-19T04:16:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.195+0000 7f4701d4e640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46e00678c0 0x7f46e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.199+0000 7f4701d4e640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46e00678c0 0x7f46e0069d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009ae0 tx=0x7f46f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.199+0000 7f46f27fc640 1 -- 172.21.15.76:0/1713102245 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f46ec056120 con 0x7f46fc1520a0 2024-09-19T04:16:47.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.355+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46fc0ccc30 con 0x7f46e00678c0 2024-09-19T04:16:47.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.371+0000 7f46f27fc640 1 -- 172.21.15.76:0/1713102245 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246712 (secure 0 0 0) 0x7f46fc0ccc30 con 0x7f46e00678c0 2024-09-19T04:16:47.377 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:47.379 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:16:47.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46e00678c0 msgr2=0x7f46e0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46e00678c0 0x7f46e0069d80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009ae0 tx=0x7f46f8008040 comp rx=0 tx=0).stop 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46fc1520a0 msgr2=0x7f46fc14c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46fc1520a0 0x7f46fc14c420 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f46ec00c340 tx=0x7f46ec00c800 comp rx=0 tx=0).stop 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 shutdown_connections 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46e00678c0 0x7f46e0069d80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46fc15f3a0 0x7f46fc14c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46fc1520a0 0x7f46fc14c420 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 --2- 172.21.15.76:0/1713102245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f46fc06fe50 0x7f46fc14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 >> 172.21.15.76:0/1713102245 conn(0x7f46fc0b1f90 msgr2=0x7f46fc0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 shutdown_connections 2024-09-19T04:16:47.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.375+0000 7f4703fb2640 1 -- 172.21.15.76:0/1713102245 wait complete. 2024-09-19T04:16:47.395 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-19T04:16:45.802592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235996,"kb_used_data":7272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738724,"statfs":{"total":773094113280,"available":772852453376,"internally_reserved":0,"allocated":7446528,"data_stored":4018688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.838052"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.981392+0000","last_change":"2024-09-19T04:16:41.981392+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.981392+0000","last_undegraded":"2024-09-19T04:16:41.981392+0000","last_fullsized":"2024-09-19T04:16:41.981392+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980603+0000","last_change":"2024-09-19T04:16:41.980603+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980603+0000","last_undegraded":"2024-09-19T04:16:41.980603+0000","last_fullsized":"2024-09-19T04:16:41.980603+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982715+0000","last_change":"2024-09-19T04:16:41.982715+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982715+0000","last_undegraded":"2024-09-19T04:16:41.982715+0000","last_fullsized":"2024-09-19T04:16:41.982715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984499+0000","last_change":"2024-09-19T04:16:41.984499+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984499+0000","last_undegraded":"2024-09-19T04:16:41.984499+0000","last_fullsized":"2024-09-19T04:16:41.984499+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.998802+0000","last_change":"2024-09-19T04:16:41.998802+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.998802+0000","last_undegraded":"2024-09-19T04:16:41.998802+0000","last_fullsized":"2024-09-19T04:16:41.998802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.988501+0000","last_change":"2024-09-19T04:16:41.988501+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.988501+0000","last_undegraded":"2024-09-19T04:16:41.988501+0000","last_fullsized":"2024-09-19T04:16:41.988501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984236+0000","last_change":"2024-09-19T04:16:41.984236+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984236+0000","last_undegraded":"2024-09-19T04:16:41.984236+0000","last_fullsized":"2024-09-19T04:16:41.984236+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976138+0000","last_change":"2024-09-19T04:16:41.976138+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976138+0000","last_undegraded":"2024-09-19T04:16:41.976138+0000","last_fullsized":"2024-09-19T04:16:41.976138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984883+0000","last_change":"2024-09-19T04:16:41.984883+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984883+0000","last_undegraded":"2024-09-19T04:16:41.984883+0000","last_fullsized":"2024-09-19T04:16:41.984883+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984199+0000","last_change":"2024-09-19T04:16:41.984199+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984199+0000","last_undegraded":"2024-09-19T04:16:41.984199+0000","last_fullsized":"2024-09-19T04:16:41.984199+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987719+0000","last_change":"2024-09-19T04:16:41.987719+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987719+0000","last_undegraded":"2024-09-19T04:16:41.987719+0000","last_fullsized":"2024-09-19T04:16:41.987719+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976359+0000","last_change":"2024-09-19T04:16:41.976359+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976359+0000","last_undegraded":"2024-09-19T04:16:41.976359+0000","last_fullsized":"2024-09-19T04:16:41.976359+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979859+0000","last_change":"2024-09-19T04:16:41.979859+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979859+0000","last_undegraded":"2024-09-19T04:16:41.979859+0000","last_fullsized":"2024-09-19T04:16:41.979859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.983462+0000","last_change":"2024-09-19T04:16:41.983462+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.983462+0000","last_undegraded":"2024-09-19T04:16:41.983462+0000","last_fullsized":"2024-09-19T04:16:41.983462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982729+0000","last_change":"2024-09-19T04:16:41.982729+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982729+0000","last_undegraded":"2024-09-19T04:16:41.982729+0000","last_fullsized":"2024-09-19T04:16:41.982729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985026+0000","last_change":"2024-09-19T04:16:41.985026+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985026+0000","last_undegraded":"2024-09-19T04:16:41.985026+0000","last_fullsized":"2024-09-19T04:16:41.985026+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980867+0000","last_change":"2024-09-19T04:16:41.980867+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980867+0000","last_undegraded":"2024-09-19T04:16:41.980867+0000","last_fullsized":"2024-09-19T04:16:41.980867+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985372+0000","last_change":"2024-09-19T04:16:41.985372+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985372+0000","last_undegraded":"2024-09-19T04:16:41.985372+0000","last_fullsized":"2024-09-19T04:16:41.985372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999302+0000","last_change":"2024-09-19T04:16:41.999302+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999302+0000","last_undegraded":"2024-09-19T04:16:41.999302+0000","last_fullsized":"2024-09-19T04:16:41.999302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980025+0000","last_change":"2024-09-19T04:16:41.980025+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980025+0000","last_undegraded":"2024-09-19T04:16:41.980025+0000","last_fullsized":"2024-09-19T04:16:41.980025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.977096+0000","last_change":"2024-09-19T04:16:41.977096+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.977096+0000","last_undegraded":"2024-09-19T04:16:41.977096+0000","last_fullsized":"2024-09-19T04:16:41.977096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984660+0000","last_change":"2024-09-19T04:16:41.984660+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984660+0000","last_undegraded":"2024-09-19T04:16:41.984660+0000","last_fullsized":"2024-09-19T04:16:41.984660+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.983716+0000","last_change":"2024-09-19T04:16:41.983716+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.983716+0000","last_undegraded":"2024-09-19T04:16:41.983716+0000","last_fullsized":"2024-09-19T04:16:41.983716+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982552+0000","last_change":"2024-09-19T04:16:41.982552+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982552+0000","last_undegraded":"2024-09-19T04:16:41.982552+0000","last_fullsized":"2024-09-19T04:16:41.982552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.977837+0000","last_change":"2024-09-19T04:16:41.977837+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.977837+0000","last_undegraded":"2024-09-19T04:16:41.977837+0000","last_fullsized":"2024-09-19T04:16:41.977837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979819+0000","last_change":"2024-09-19T04:16:41.979819+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979819+0000","last_undegraded":"2024-09-19T04:16:41.979819+0000","last_fullsized":"2024-09-19T04:16:41.979819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986619+0000","last_change":"2024-09-19T04:16:41.986619+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986619+0000","last_undegraded":"2024-09-19T04:16:41.986619+0000","last_fullsized":"2024-09-19T04:16:41.986619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980172+0000","last_change":"2024-09-19T04:16:41.980172+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980172+0000","last_undegraded":"2024-09-19T04:16:41.980172+0000","last_fullsized":"2024-09-19T04:16:41.980172+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982680+0000","last_change":"2024-09-19T04:16:41.982680+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982680+0000","last_undegraded":"2024-09-19T04:16:41.982680+0000","last_fullsized":"2024-09-19T04:16:41.982680+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978736+0000","last_change":"2024-09-19T04:16:41.978736+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978736+0000","last_undegraded":"2024-09-19T04:16:41.978736+0000","last_fullsized":"2024-09-19T04:16:41.978736+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.989513+0000","last_change":"2024-09-19T04:16:41.989513+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.989513+0000","last_undegraded":"2024-09-19T04:16:41.989513+0000","last_fullsized":"2024-09-19T04:16:41.989513+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.990810+0000","last_change":"2024-09-19T04:16:41.990810+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.990810+0000","last_undegraded":"2024-09-19T04:16:41.990810+0000","last_fullsized":"2024-09-19T04:16:41.990810+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.993046+0000","last_change":"2024-09-19T04:16:41.993046+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.993046+0000","last_undegraded":"2024-09-19T04:16:41.993046+0000","last_fullsized":"2024-09-19T04:16:41.993046+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.017233+0000","last_change":"2024-09-19T04:16:42.013739+0000","last_active":"2024-09-19T04:16:42.017233+0000","last_peered":"2024-09-19T04:16:42.017233+0000","last_clean":"2024-09-19T04:16:42.017233+0000","last_became_active":"2024-09-19T04:16:42.013362+0000","last_became_peered":"2024-09-19T04:16:42.013362+0000","last_unstale":"2024-09-19T04:16:42.017233+0000","last_undegraded":"2024-09-19T04:16:42.017233+0000","last_fullsized":"2024-09-19T04:16:42.017233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:28.800656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985724+0000","last_change":"2024-09-19T04:16:41.985724+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985724+0000","last_undegraded":"2024-09-19T04:16:41.985724+0000","last_fullsized":"2024-09-19T04:16:41.985724+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.019939+0000","last_change":"2024-09-19T04:16:42.018926+0000","last_active":"2024-09-19T04:16:42.019939+0000","last_peered":"2024-09-19T04:16:42.019939+0000","last_clean":"2024-09-19T04:16:42.019939+0000","last_became_active":"2024-09-19T04:16:42.018655+0000","last_became_peered":"2024-09-19T04:16:42.018655+0000","last_unstale":"2024-09-19T04:16:42.019939+0000","last_undegraded":"2024-09-19T04:16:42.019939+0000","last_fullsized":"2024-09-19T04:16:42.019939+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:44.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.988796+0000","last_change":"2024-09-19T04:16:41.988796+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.988796+0000","last_undegraded":"2024-09-19T04:16:41.988796+0000","last_fullsized":"2024-09-19T04:16:41.988796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.999950+0000","last_change":"2024-09-19T04:16:41.992501+0000","last_active":"2024-09-19T04:16:41.999950+0000","last_peered":"2024-09-19T04:16:41.999950+0000","last_clean":"2024-09-19T04:16:41.999950+0000","last_became_active":"2024-09-19T04:16:41.992135+0000","last_became_peered":"2024-09-19T04:16:41.992135+0000","last_unstale":"2024-09-19T04:16:41.999950+0000","last_undegraded":"2024-09-19T04:16:41.999950+0000","last_fullsized":"2024-09-19T04:16:41.999950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.537033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.991338+0000","last_change":"2024-09-19T04:16:41.991338+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.991338+0000","last_undegraded":"2024-09-19T04:16:41.991338+0000","last_fullsized":"2024-09-19T04:16:41.991338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005233+0000","last_change":"2024-09-19T04:16:42.000668+0000","last_active":"2024-09-19T04:16:42.005233+0000","last_peered":"2024-09-19T04:16:42.005233+0000","last_clean":"2024-09-19T04:16:42.005233+0000","last_became_active":"2024-09-19T04:16:42.000249+0000","last_became_peered":"2024-09-19T04:16:42.000249+0000","last_unstale":"2024-09-19T04:16:42.005233+0000","last_undegraded":"2024-09-19T04:16:42.005233+0000","last_fullsized":"2024-09-19T04:16:42.005233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.567498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984416+0000","last_change":"2024-09-19T04:16:41.984416+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984416+0000","last_undegraded":"2024-09-19T04:16:41.984416+0000","last_fullsized":"2024-09-19T04:16:41.984416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.990792+0000","last_change":"2024-09-19T04:16:41.989580+0000","last_active":"2024-09-19T04:16:41.990792+0000","last_peered":"2024-09-19T04:16:41.990792+0000","last_clean":"2024-09-19T04:16:41.990792+0000","last_became_active":"2024-09-19T04:16:41.989078+0000","last_became_peered":"2024-09-19T04:16:41.989078+0000","last_unstale":"2024-09-19T04:16:41.990792+0000","last_undegraded":"2024-09-19T04:16:41.990792+0000","last_fullsized":"2024-09-19T04:16:41.990792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.567498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978924+0000","last_change":"2024-09-19T04:16:41.978924+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978924+0000","last_undegraded":"2024-09-19T04:16:41.978924+0000","last_fullsized":"2024-09-19T04:16:41.978924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997003+0000","last_change":"2024-09-19T04:16:41.995699+0000","last_active":"2024-09-19T04:16:41.997003+0000","last_peered":"2024-09-19T04:16:41.997003+0000","last_clean":"2024-09-19T04:16:41.997003+0000","last_became_active":"2024-09-19T04:16:41.993440+0000","last_became_peered":"2024-09-19T04:16:41.993440+0000","last_unstale":"2024-09-19T04:16:41.997003+0000","last_undegraded":"2024-09-19T04:16:41.997003+0000","last_fullsized":"2024-09-19T04:16:41.997003+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:12.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.015296+0000","last_change":"2024-09-19T04:16:42.015296+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.015296+0000","last_undegraded":"2024-09-19T04:16:42.015296+0000","last_fullsized":"2024-09-19T04:16:42.015296+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024309+0000","last_change":"2024-09-19T04:16:42.022337+0000","last_active":"2024-09-19T04:16:42.024309+0000","last_peered":"2024-09-19T04:16:42.024309+0000","last_clean":"2024-09-19T04:16:42.024309+0000","last_became_active":"2024-09-19T04:16:42.021791+0000","last_became_peered":"2024-09-19T04:16:42.021791+0000","last_unstale":"2024-09-19T04:16:42.024309+0000","last_undegraded":"2024-09-19T04:16:42.024309+0000","last_fullsized":"2024-09-19T04:16:42.024309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:33.295425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980459+0000","last_change":"2024-09-19T04:16:41.980459+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980459+0000","last_undegraded":"2024-09-19T04:16:41.980459+0000","last_fullsized":"2024-09-19T04:16:41.980459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.021722+0000","last_change":"2024-09-19T04:16:42.019556+0000","last_active":"2024-09-19T04:16:42.021722+0000","last_peered":"2024-09-19T04:16:42.021722+0000","last_clean":"2024-09-19T04:16:42.021722+0000","last_became_active":"2024-09-19T04:16:42.018870+0000","last_became_peered":"2024-09-19T04:16:42.018870+0000","last_unstale":"2024-09-19T04:16:42.021722+0000","last_undegraded":"2024-09-19T04:16:42.021722+0000","last_fullsized":"2024-09-19T04:16:42.021722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:19.666213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979749+0000","last_change":"2024-09-19T04:16:41.979749+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979749+0000","last_undegraded":"2024-09-19T04:16:41.979749+0000","last_fullsized":"2024-09-19T04:16:41.979749+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028493+0000","last_change":"2024-09-19T04:16:42.026415+0000","last_active":"2024-09-19T04:16:42.028493+0000","last_peered":"2024-09-19T04:16:42.028493+0000","last_clean":"2024-09-19T04:16:42.028493+0000","last_became_active":"2024-09-19T04:16:42.025740+0000","last_became_peered":"2024-09-19T04:16:42.025740+0000","last_unstale":"2024-09-19T04:16:42.028493+0000","last_undegraded":"2024-09-19T04:16:42.028493+0000","last_fullsized":"2024-09-19T04:16:42.028493+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:36.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979412+0000","last_change":"2024-09-19T04:16:41.979412+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979412+0000","last_undegraded":"2024-09-19T04:16:41.979412+0000","last_fullsized":"2024-09-19T04:16:41.979412+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005588+0000","last_change":"2024-09-19T04:16:42.001820+0000","last_active":"2024-09-19T04:16:42.005588+0000","last_peered":"2024-09-19T04:16:42.005588+0000","last_clean":"2024-09-19T04:16:42.005588+0000","last_became_active":"2024-09-19T04:16:42.001141+0000","last_became_peered":"2024-09-19T04:16:42.001141+0000","last_unstale":"2024-09-19T04:16:42.005588+0000","last_undegraded":"2024-09-19T04:16:42.005588+0000","last_fullsized":"2024-09-19T04:16:42.005588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:28.101048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.994594+0000","last_change":"2024-09-19T04:16:41.994594+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.994594+0000","last_undegraded":"2024-09-19T04:16:41.994594+0000","last_fullsized":"2024-09-19T04:16:41.994594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.025625+0000","last_change":"2024-09-19T04:16:42.018953+0000","last_active":"2024-09-19T04:16:42.025625+0000","last_peered":"2024-09-19T04:16:42.025625+0000","last_clean":"2024-09-19T04:16:42.025625+0000","last_became_active":"2024-09-19T04:16:42.018215+0000","last_became_peered":"2024-09-19T04:16:42.018215+0000","last_unstale":"2024-09-19T04:16:42.025625+0000","last_undegraded":"2024-09-19T04:16:42.025625+0000","last_fullsized":"2024-09-19T04:16:42.025625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:09.582870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987703+0000","last_change":"2024-09-19T04:16:41.987703+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987703+0000","last_undegraded":"2024-09-19T04:16:41.987703+0000","last_fullsized":"2024-09-19T04:16:41.987703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.008487+0000","last_change":"2024-09-19T04:16:42.006482+0000","last_active":"2024-09-19T04:16:42.008487+0000","last_peered":"2024-09-19T04:16:42.008487+0000","last_clean":"2024-09-19T04:16:42.008487+0000","last_became_active":"2024-09-19T04:16:42.006111+0000","last_became_peered":"2024-09-19T04:16:42.006111+0000","last_unstale":"2024-09-19T04:16:42.008487+0000","last_undegraded":"2024-09-19T04:16:42.008487+0000","last_fullsized":"2024-09-19T04:16:42.008487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:11.291559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986477+0000","last_change":"2024-09-19T04:16:41.986477+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986477+0000","last_undegraded":"2024-09-19T04:16:41.986477+0000","last_fullsized":"2024-09-19T04:16:41.986477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028432+0000","last_change":"2024-09-19T04:16:42.024717+0000","last_active":"2024-09-19T04:16:42.028432+0000","last_peered":"2024-09-19T04:16:42.028432+0000","last_clean":"2024-09-19T04:16:42.028432+0000","last_became_active":"2024-09-19T04:16:42.023979+0000","last_became_peered":"2024-09-19T04:16:42.023979+0000","last_unstale":"2024-09-19T04:16:42.028432+0000","last_undegraded":"2024-09-19T04:16:42.028432+0000","last_fullsized":"2024-09-19T04:16:42.028432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:14.632041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978077+0000","last_change":"2024-09-19T04:16:41.978077+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978077+0000","last_undegraded":"2024-09-19T04:16:41.978077+0000","last_fullsized":"2024-09-19T04:16:41.978077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.006392+0000","last_change":"2024-09-19T04:16:42.002996+0000","last_active":"2024-09-19T04:16:42.006392+0000","last_peered":"2024-09-19T04:16:42.006392+0000","last_clean":"2024-09-19T04:16:42.006392+0000","last_became_active":"2024-09-19T04:16:42.001542+0000","last_became_peered":"2024-09-19T04:16:42.001542+0000","last_unstale":"2024-09-19T04:16:42.006392+0000","last_undegraded":"2024-09-19T04:16:42.006392+0000","last_fullsized":"2024-09-19T04:16:42.006392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:08.070747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.014200+0000","last_change":"2024-09-19T04:16:42.014200+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.014200+0000","last_undegraded":"2024-09-19T04:16:42.014200+0000","last_fullsized":"2024-09-19T04:16:42.014200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024770+0000","last_change":"2024-09-19T04:16:42.018769+0000","last_active":"2024-09-19T04:16:42.024770+0000","last_peered":"2024-09-19T04:16:42.024770+0000","last_clean":"2024-09-19T04:16:42.024770+0000","last_became_active":"2024-09-19T04:16:42.017955+0000","last_became_peered":"2024-09-19T04:16:42.017955+0000","last_unstale":"2024-09-19T04:16:42.024770+0000","last_undegraded":"2024-09-19T04:16:42.024770+0000","last_fullsized":"2024-09-19T04:16:42.024770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:19.225233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979070+0000","last_change":"2024-09-19T04:16:41.979070+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979070+0000","last_undegraded":"2024-09-19T04:16:41.979070+0000","last_fullsized":"2024-09-19T04:16:41.979070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.016442+0000","last_change":"2024-09-19T04:16:42.011251+0000","last_active":"2024-09-19T04:16:42.016442+0000","last_peered":"2024-09-19T04:16:42.016442+0000","last_clean":"2024-09-19T04:16:42.016442+0000","last_became_active":"2024-09-19T04:16:42.010096+0000","last_became_peered":"2024-09-19T04:16:42.010096+0000","last_unstale":"2024-09-19T04:16:42.016442+0000","last_undegraded":"2024-09-19T04:16:42.016442+0000","last_fullsized":"2024-09-19T04:16:42.016442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:09.087251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987287+0000","last_change":"2024-09-19T04:16:41.987287+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987287+0000","last_undegraded":"2024-09-19T04:16:41.987287+0000","last_fullsized":"2024-09-19T04:16:41.987287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.001783+0000","last_change":"2024-09-19T04:16:42.000089+0000","last_active":"2024-09-19T04:16:42.001783+0000","last_peered":"2024-09-19T04:16:42.001783+0000","last_clean":"2024-09-19T04:16:42.001783+0000","last_became_active":"2024-09-19T04:16:41.999376+0000","last_became_peered":"2024-09-19T04:16:41.999376+0000","last_unstale":"2024-09-19T04:16:42.001783+0000","last_undegraded":"2024-09-19T04:16:42.001783+0000","last_fullsized":"2024-09-19T04:16:42.001783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:34.395480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.997756+0000","last_change":"2024-09-19T04:16:41.997756+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.997756+0000","last_undegraded":"2024-09-19T04:16:41.997756+0000","last_fullsized":"2024-09-19T04:16:41.997756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.027534+0000","last_change":"2024-09-19T04:16:42.023483+0000","last_active":"2024-09-19T04:16:42.027534+0000","last_peered":"2024-09-19T04:16:42.027534+0000","last_clean":"2024-09-19T04:16:42.027534+0000","last_became_active":"2024-09-19T04:16:42.022787+0000","last_became_peered":"2024-09-19T04:16:42.022787+0000","last_unstale":"2024-09-19T04:16:42.027534+0000","last_undegraded":"2024-09-19T04:16:42.027534+0000","last_fullsized":"2024-09-19T04:16:42.027534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:54.902392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984262+0000","last_change":"2024-09-19T04:16:41.984262+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984262+0000","last_undegraded":"2024-09-19T04:16:41.984262+0000","last_fullsized":"2024-09-19T04:16:41.984262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028461+0000","last_change":"2024-09-19T04:16:42.026255+0000","last_active":"2024-09-19T04:16:42.028461+0000","last_peered":"2024-09-19T04:16:42.028461+0000","last_clean":"2024-09-19T04:16:42.028461+0000","last_became_active":"2024-09-19T04:16:42.025430+0000","last_became_peered":"2024-09-19T04:16:42.025430+0000","last_unstale":"2024-09-19T04:16:42.028461+0000","last_undegraded":"2024-09-19T04:16:42.028461+0000","last_fullsized":"2024-09-19T04:16:42.028461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:18.263858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999999+0000","last_change":"2024-09-19T04:16:41.999999+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999999+0000","last_undegraded":"2024-09-19T04:16:41.999999+0000","last_fullsized":"2024-09-19T04:16:41.999999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.999802+0000","last_change":"2024-09-19T04:16:41.998882+0000","last_active":"2024-09-19T04:16:41.999802+0000","last_peered":"2024-09-19T04:16:41.999802+0000","last_clean":"2024-09-19T04:16:41.999802+0000","last_became_active":"2024-09-19T04:16:41.998612+0000","last_became_peered":"2024-09-19T04:16:41.998612+0000","last_unstale":"2024-09-19T04:16:41.999802+0000","last_undegraded":"2024-09-19T04:16:41.999802+0000","last_fullsized":"2024-09-19T04:16:41.999802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:44.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987405+0000","last_change":"2024-09-19T04:16:41.987405+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987405+0000","last_undegraded":"2024-09-19T04:16:41.987405+0000","last_fullsized":"2024-09-19T04:16:41.987405+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.017443+0000","last_change":"2024-09-19T04:16:42.015741+0000","last_active":"2024-09-19T04:16:42.017443+0000","last_peered":"2024-09-19T04:16:42.017443+0000","last_clean":"2024-09-19T04:16:42.017443+0000","last_became_active":"2024-09-19T04:16:42.015285+0000","last_became_peered":"2024-09-19T04:16:42.015285+0000","last_unstale":"2024-09-19T04:16:42.017443+0000","last_undegraded":"2024-09-19T04:16:42.017443+0000","last_fullsized":"2024-09-19T04:16:42.017443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:12.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976440+0000","last_change":"2024-09-19T04:16:41.976440+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976440+0000","last_undegraded":"2024-09-19T04:16:41.976440+0000","last_fullsized":"2024-09-19T04:16:41.976440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.004913+0000","last_change":"2024-09-19T04:16:42.002264+0000","last_active":"2024-09-19T04:16:42.004913+0000","last_peered":"2024-09-19T04:16:42.004913+0000","last_clean":"2024-09-19T04:16:42.004913+0000","last_became_active":"2024-09-19T04:16:42.000593+0000","last_became_peered":"2024-09-19T04:16:42.000593+0000","last_unstale":"2024-09-19T04:16:42.004913+0000","last_undegraded":"2024-09-19T04:16:42.004913+0000","last_fullsized":"2024-09-19T04:16:42.004913+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:36.127882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999116+0000","last_change":"2024-09-19T04:16:41.999116+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999116+0000","last_undegraded":"2024-09-19T04:16:41.999116+0000","last_fullsized":"2024-09-19T04:16:41.999116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.011261+0000","last_change":"2024-09-19T04:16:42.010194+0000","last_active":"2024-09-19T04:16:42.011261+0000","last_peered":"2024-09-19T04:16:42.011261+0000","last_clean":"2024-09-19T04:16:42.011261+0000","last_became_active":"2024-09-19T04:16:42.009783+0000","last_became_peered":"2024-09-19T04:16:42.009783+0000","last_unstale":"2024-09-19T04:16:42.011261+0000","last_undegraded":"2024-09-19T04:16:42.011261+0000","last_fullsized":"2024-09-19T04:16:42.011261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:02.962114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987074+0000","last_change":"2024-09-19T04:16:41.987074+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987074+0000","last_undegraded":"2024-09-19T04:16:41.987074+0000","last_fullsized":"2024-09-19T04:16:41.987074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.012421+0000","last_change":"2024-09-19T04:16:42.010812+0000","last_active":"2024-09-19T04:16:42.012421+0000","last_peered":"2024-09-19T04:16:42.012421+0000","last_clean":"2024-09-19T04:16:42.012421+0000","last_became_active":"2024-09-19T04:16:42.010051+0000","last_became_peered":"2024-09-19T04:16:42.010051+0000","last_unstale":"2024-09-19T04:16:42.012421+0000","last_undegraded":"2024-09-19T04:16:42.012421+0000","last_fullsized":"2024-09-19T04:16:42.012421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:52:53.450248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986784+0000","last_change":"2024-09-19T04:16:41.986784+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986784+0000","last_undegraded":"2024-09-19T04:16:41.986784+0000","last_fullsized":"2024-09-19T04:16:41.986784+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997616+0000","last_change":"2024-09-19T04:16:41.995074+0000","last_active":"2024-09-19T04:16:41.997616+0000","last_peered":"2024-09-19T04:16:41.997616+0000","last_clean":"2024-09-19T04:16:41.997616+0000","last_became_active":"2024-09-19T04:16:41.994321+0000","last_became_peered":"2024-09-19T04:16:41.994321+0000","last_unstale":"2024-09-19T04:16:41.997616+0000","last_undegraded":"2024-09-19T04:16:41.997616+0000","last_fullsized":"2024-09-19T04:16:41.997616+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.017414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978779+0000","last_change":"2024-09-19T04:16:41.978779+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978779+0000","last_undegraded":"2024-09-19T04:16:41.978779+0000","last_fullsized":"2024-09-19T04:16:41.978779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997258+0000","last_change":"2024-09-19T04:16:41.991513+0000","last_active":"2024-09-19T04:16:41.997258+0000","last_peered":"2024-09-19T04:16:41.997258+0000","last_clean":"2024-09-19T04:16:41.997258+0000","last_became_active":"2024-09-19T04:16:41.990986+0000","last_became_peered":"2024-09-19T04:16:41.990986+0000","last_unstale":"2024-09-19T04:16:41.997258+0000","last_undegraded":"2024-09-19T04:16:41.997258+0000","last_fullsized":"2024-09-19T04:16:41.997258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:47.451247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.998881+0000","last_change":"2024-09-19T04:16:41.998881+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.998881+0000","last_undegraded":"2024-09-19T04:16:41.998881+0000","last_fullsized":"2024-09-19T04:16:41.998881+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005686+0000","last_change":"2024-09-19T04:16:42.003466+0000","last_active":"2024-09-19T04:16:42.005686+0000","last_peered":"2024-09-19T04:16:42.005686+0000","last_clean":"2024-09-19T04:16:42.005686+0000","last_became_active":"2024-09-19T04:16:42.003135+0000","last_became_peered":"2024-09-19T04:16:42.003135+0000","last_unstale":"2024-09-19T04:16:42.005686+0000","last_undegraded":"2024-09-19T04:16:42.005686+0000","last_fullsized":"2024-09-19T04:16:42.005686+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:23.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.014982+0000","last_change":"2024-09-19T04:16:42.014982+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.014982+0000","last_undegraded":"2024-09-19T04:16:42.014982+0000","last_fullsized":"2024-09-19T04:16:42.014982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.026933+0000","last_change":"2024-09-19T04:16:42.019204+0000","last_active":"2024-09-19T04:16:42.026933+0000","last_peered":"2024-09-19T04:16:42.026933+0000","last_clean":"2024-09-19T04:16:42.026933+0000","last_became_active":"2024-09-19T04:16:42.018134+0000","last_became_peered":"2024-09-19T04:16:42.018134+0000","last_unstale":"2024-09-19T04:16:42.026933+0000","last_undegraded":"2024-09-19T04:16:42.026933+0000","last_fullsized":"2024-09-19T04:16:42.026933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:41.461855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987022+0000","last_change":"2024-09-19T04:16:41.987022+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987022+0000","last_undegraded":"2024-09-19T04:16:41.987022+0000","last_fullsized":"2024-09-19T04:16:41.987022+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.996533+0000","last_change":"2024-09-19T04:16:41.994558+0000","last_active":"2024-09-19T04:16:41.996533+0000","last_peered":"2024-09-19T04:16:41.996533+0000","last_clean":"2024-09-19T04:16:41.996533+0000","last_became_active":"2024-09-19T04:16:41.993965+0000","last_became_peered":"2024-09-19T04:16:41.993965+0000","last_unstale":"2024-09-19T04:16:41.996533+0000","last_undegraded":"2024-09-19T04:16:41.996533+0000","last_fullsized":"2024-09-19T04:16:41.996533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:02.962114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980245+0000","last_change":"2024-09-19T04:16:41.980245+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980245+0000","last_undegraded":"2024-09-19T04:16:41.980245+0000","last_fullsized":"2024-09-19T04:16:41.980245+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024381+0000","last_change":"2024-09-19T04:16:42.022760+0000","last_active":"2024-09-19T04:16:42.024381+0000","last_peered":"2024-09-19T04:16:42.024381+0000","last_clean":"2024-09-19T04:16:42.024381+0000","last_became_active":"2024-09-19T04:16:42.022030+0000","last_became_peered":"2024-09-19T04:16:42.022030+0000","last_unstale":"2024-09-19T04:16:42.024381+0000","last_undegraded":"2024-09-19T04:16:42.024381+0000","last_fullsized":"2024-09-19T04:16:42.024381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:59.963896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986891+0000","last_change":"2024-09-19T04:16:41.986891+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986891+0000","last_undegraded":"2024-09-19T04:16:41.986891+0000","last_fullsized":"2024-09-19T04:16:41.986891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024272+0000","last_change":"2024-09-19T04:16:42.022541+0000","last_active":"2024-09-19T04:16:42.024272+0000","last_peered":"2024-09-19T04:16:42.024272+0000","last_clean":"2024-09-19T04:16:42.024272+0000","last_became_active":"2024-09-19T04:16:42.022021+0000","last_became_peered":"2024-09-19T04:16:42.022021+0000","last_unstale":"2024-09-19T04:16:42.024272+0000","last_undegraded":"2024-09-19T04:16:42.024272+0000","last_fullsized":"2024-09-19T04:16:42.024272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:20.649775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.994302+0000","last_change":"2024-09-19T04:16:41.994302+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.994302+0000","last_undegraded":"2024-09-19T04:16:41.994302+0000","last_fullsized":"2024-09-19T04:16:41.994302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.989613+0000","last_change":"2024-09-19T04:16:41.988994+0000","last_active":"2024-09-19T04:16:41.989613+0000","last_peered":"2024-09-19T04:16:41.989613+0000","last_clean":"2024-09-19T04:16:41.989613+0000","last_became_active":"2024-09-19T04:16:41.988755+0000","last_became_peered":"2024-09-19T04:16:41.988755+0000","last_unstale":"2024-09-19T04:16:41.989613+0000","last_undegraded":"2024-09-19T04:16:41.989613+0000","last_fullsized":"2024-09-19T04:16:41.989613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.537033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.009326+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:16:42.009326+0000","last_peered":"2024-09-19T04:16:42.009326+0000","last_clean":"2024-09-19T04:16:42.009326+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:16:42.009326+0000","last_undegraded":"2024-09-19T04:16:42.009326+0000","last_fullsized":"2024-09-19T04:16:42.009326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":945112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":945112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:16:47.399 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:16:47.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd908fd1640 1 Processor -- start 2024-09-19T04:16:47.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd908fd1640 1 -- start start 2024-09-19T04:16:47.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd908fd1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 0x7fd9040ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd908fd1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd90415c2a0 0x7fd9040aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd908fd1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9040b3290 con 0x7fd9040af2a0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd908fd1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd9040b3570 con 0x7fd90415c2a0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd908fd1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd9040b3850 con 0x7fd90415bec0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd902d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd9040af2a0 0x7fd90415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33306/0 (socket says 172.21.15.76:33306) 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd902d9d640 1 -- 172.21.15.76:0/735343192 learned_addr learned my addr 172.21.15.76:0/735343192 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd901d9b640 1 --2- 172.21.15.76:0/735343192 >> v2:172.21.15.116:3300/0 conn(0x7fd90415c2a0 0x7fd9040aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 608391750 0 0) 0x7fd9040b3290 con 0x7fd9040af2a0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd90259c640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 0x7fd9040ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.551+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8ec003530 con 0x7fd9040af2a0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 596829033 0 0) 0x7fd8ec003530 con 0x7fd9040af2a0 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 msgr2=0x7fd9040ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 0x7fd9040ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 >> v2:172.21.15.116:3300/0 conn(0x7fd90415c2a0 msgr2=0x7fd9040aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 --2- 172.21.15.76:0/735343192 >> v2:172.21.15.116:3300/0 conn(0x7fd90415c2a0 0x7fd9040aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9040b3b30 con 0x7fd9040af2a0 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fd8fc001a00 con 0x7fd9040af2a0 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 >> v1:172.21.15.76:6789/0 conn(0x7fd9040af2a0 legacy=0x7fd90415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8ec0039c0 0x7fd8ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 0x7fd8ec004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec004d50 0x7fd8ec0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9040b3570 con 0x7fd8ec004d50 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9040b3850 con 0x7fd8ec0039c0 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ec00b9d0 con 0x7fd90415bec0 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec004d50 0x7fd8ec0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd901d9b640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 0x7fd8ec004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90259c640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8ec0039c0 0x7fd8ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 -- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 msgr2=0x7fd8ec004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 0x7fd8ec004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 -- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8ec0039c0 msgr2=0x7fd8ec003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8ec0039c0 0x7fd8ec003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 -- 172.21.15.76:0/735343192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ec003530 con 0x7fd8ec004d50 2024-09-19T04:16:47.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd901d9b640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd90415bec0 0x7fd8ec004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec004d50 0x7fd8ec0051d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00a8b0 tx=0x7fd8fc01d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd42816dbd922e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc00a4a0 con 0x7fd8ec004d50 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc0ad630 con 0x7fd8ec004d50 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd90159a640 1 -- 172.21.15.76:0/735343192 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0ada60 con 0x7fd8ec004d50 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd908fd1640 1 -- 172.21.15.76:0/735343192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec004d50 msgr2=0x7fd8ec0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd908fd1640 1 --2- 172.21.15.76:0/735343192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8ec004d50 0x7fd8ec0051d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00a8b0 tx=0x7fd8fc01d3a0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd902d9d640 1 -- 172.21.15.76:0/735343192 reap_dead start 2024-09-19T04:16:47.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd908fd1640 1 -- 172.21.15.76:0/735343192 shutdown_connections 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd908fd1640 1 -- 172.21.15.76:0/735343192 >> 172.21.15.76:0/735343192 conn(0x7fd9040b1f90 msgr2=0x7fd9040b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd908fd1640 1 -- 172.21.15.76:0/735343192 shutdown_connections 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.555+0000 7fd908fd1640 1 -- 172.21.15.76:0/735343192 wait complete. 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 Processor -- start 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 -- start start 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90414f9d0 0x7fd90406eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd90406f0c0 0x7fd9041e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9041e54b0 0x7fd904071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9040af730 con 0x7fd90406f0c0 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9040b40f0 con 0x7fd90414f9d0 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd9040afc90 con 0x7fd9041e54b0 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd90406f0c0 0x7fd9041e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd90406f0c0 0x7fd9041e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44668/0 (socket says 172.21.15.76:44668) 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 -- 172.21.15.76:0/8318355 learned_addr learned my addr 172.21.15.76:0/8318355 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd90259c640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90414f9d0 0x7fd90406eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd902d9d640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9041e54b0 0x7fd904071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 -- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9041e54b0 msgr2=0x7fd904071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9041e54b0 0x7fd904071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 -- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90414f9d0 msgr2=0x7fd90406eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90414f9d0 0x7fd90406eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 -- 172.21.15.76:0/8318355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041537b0 con 0x7fd90406f0c0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd901d9b640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd90406f0c0 0x7fd9041e4f70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8010950 tx=0x7fd8f8010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd8f2ffd640 1 -- 172.21.15.76:0/8318355 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800f040 con 0x7fd90406f0c0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd8f2ffd640 1 -- 172.21.15.76:0/8318355 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f800ec20 con 0x7fd90406f0c0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd904153a90 con 0x7fd90406f0c0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd8f2ffd640 1 -- 172.21.15.76:0/8318355 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8004020 con 0x7fd90406f0c0 2024-09-19T04:16:47.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd904154050 con 0x7fd90406f0c0 2024-09-19T04:16:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9040aed60 con 0x7fd90406f0c0 2024-09-19T04:16:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd8f2ffd640 1 -- 172.21.15.76:0/8318355 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8f80041c0 con 0x7fd90406f0c0 2024-09-19T04:16:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd8f2ffd640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8e40678c0 0x7fd8e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd8f2ffd640 1 -- 172.21.15.76:0/8318355 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd8f8086270 con 0x7fd90406f0c0 2024-09-19T04:16:47.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.559+0000 7fd90259c640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8e40678c0 0x7fd8e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.563+0000 7fd90259c640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8e40678c0 0x7fd8e4069d80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c800 tx=0x7fd8f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.567+0000 7fd8f2ffd640 1 -- 172.21.15.76:0/8318355 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd8f8018030 con 0x7fd90406f0c0 2024-09-19T04:16:47.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.719+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd90406fcd0 con 0x7fd8e40678c0 2024-09-19T04:16:47.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.727+0000 7fd8f2ffd640 1 -- 172.21.15.76:0/8318355 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+246712 (secure 0 0 0) 0x7fd90406fcd0 con 0x7fd8e40678c0 2024-09-19T04:16:47.732 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:47.734 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:16:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8e40678c0 msgr2=0x7fd8e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8e40678c0 0x7fd8e4069d80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fd8f400c800 tx=0x7fd8f4008040 comp rx=0 tx=0).stop 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd90406f0c0 msgr2=0x7fd9041e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd90406f0c0 0x7fd9041e4f70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8010950 tx=0x7fd8f8010e10 comp rx=0 tx=0).stop 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 shutdown_connections 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8e40678c0 0x7fd8e4069d80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9041e54b0 0x7fd904071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd90406f0c0 0x7fd9041e4f70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 --2- 172.21.15.76:0/8318355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90414f9d0 0x7fd90406eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 >> 172.21.15.76:0/8318355 conn(0x7fd9040b1f90 msgr2=0x7fd9041e3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:47.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 shutdown_connections 2024-09-19T04:16:47.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.731+0000 7fd908fd1640 1 -- 172.21.15.76:0/8318355 wait complete. 2024-09-19T04:16:47.751 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-19T04:16:45.802592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235996,"kb_used_data":7272,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738724,"statfs":{"total":773094113280,"available":772852453376,"internally_reserved":0,"allocated":7446528,"data_stored":4018688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.838052"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.981392+0000","last_change":"2024-09-19T04:16:41.981392+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.981392+0000","last_undegraded":"2024-09-19T04:16:41.981392+0000","last_fullsized":"2024-09-19T04:16:41.981392+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980603+0000","last_change":"2024-09-19T04:16:41.980603+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980603+0000","last_undegraded":"2024-09-19T04:16:41.980603+0000","last_fullsized":"2024-09-19T04:16:41.980603+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982715+0000","last_change":"2024-09-19T04:16:41.982715+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982715+0000","last_undegraded":"2024-09-19T04:16:41.982715+0000","last_fullsized":"2024-09-19T04:16:41.982715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984499+0000","last_change":"2024-09-19T04:16:41.984499+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984499+0000","last_undegraded":"2024-09-19T04:16:41.984499+0000","last_fullsized":"2024-09-19T04:16:41.984499+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.998802+0000","last_change":"2024-09-19T04:16:41.998802+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.998802+0000","last_undegraded":"2024-09-19T04:16:41.998802+0000","last_fullsized":"2024-09-19T04:16:41.998802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.988501+0000","last_change":"2024-09-19T04:16:41.988501+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.988501+0000","last_undegraded":"2024-09-19T04:16:41.988501+0000","last_fullsized":"2024-09-19T04:16:41.988501+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984236+0000","last_change":"2024-09-19T04:16:41.984236+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984236+0000","last_undegraded":"2024-09-19T04:16:41.984236+0000","last_fullsized":"2024-09-19T04:16:41.984236+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976138+0000","last_change":"2024-09-19T04:16:41.976138+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976138+0000","last_undegraded":"2024-09-19T04:16:41.976138+0000","last_fullsized":"2024-09-19T04:16:41.976138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984883+0000","last_change":"2024-09-19T04:16:41.984883+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984883+0000","last_undegraded":"2024-09-19T04:16:41.984883+0000","last_fullsized":"2024-09-19T04:16:41.984883+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984199+0000","last_change":"2024-09-19T04:16:41.984199+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984199+0000","last_undegraded":"2024-09-19T04:16:41.984199+0000","last_fullsized":"2024-09-19T04:16:41.984199+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987719+0000","last_change":"2024-09-19T04:16:41.987719+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987719+0000","last_undegraded":"2024-09-19T04:16:41.987719+0000","last_fullsized":"2024-09-19T04:16:41.987719+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976359+0000","last_change":"2024-09-19T04:16:41.976359+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976359+0000","last_undegraded":"2024-09-19T04:16:41.976359+0000","last_fullsized":"2024-09-19T04:16:41.976359+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979859+0000","last_change":"2024-09-19T04:16:41.979859+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979859+0000","last_undegraded":"2024-09-19T04:16:41.979859+0000","last_fullsized":"2024-09-19T04:16:41.979859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.983462+0000","last_change":"2024-09-19T04:16:41.983462+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.983462+0000","last_undegraded":"2024-09-19T04:16:41.983462+0000","last_fullsized":"2024-09-19T04:16:41.983462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982729+0000","last_change":"2024-09-19T04:16:41.982729+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982729+0000","last_undegraded":"2024-09-19T04:16:41.982729+0000","last_fullsized":"2024-09-19T04:16:41.982729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985026+0000","last_change":"2024-09-19T04:16:41.985026+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985026+0000","last_undegraded":"2024-09-19T04:16:41.985026+0000","last_fullsized":"2024-09-19T04:16:41.985026+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980867+0000","last_change":"2024-09-19T04:16:41.980867+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980867+0000","last_undegraded":"2024-09-19T04:16:41.980867+0000","last_fullsized":"2024-09-19T04:16:41.980867+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985372+0000","last_change":"2024-09-19T04:16:41.985372+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985372+0000","last_undegraded":"2024-09-19T04:16:41.985372+0000","last_fullsized":"2024-09-19T04:16:41.985372+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999302+0000","last_change":"2024-09-19T04:16:41.999302+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999302+0000","last_undegraded":"2024-09-19T04:16:41.999302+0000","last_fullsized":"2024-09-19T04:16:41.999302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980025+0000","last_change":"2024-09-19T04:16:41.980025+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980025+0000","last_undegraded":"2024-09-19T04:16:41.980025+0000","last_fullsized":"2024-09-19T04:16:41.980025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.977096+0000","last_change":"2024-09-19T04:16:41.977096+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.977096+0000","last_undegraded":"2024-09-19T04:16:41.977096+0000","last_fullsized":"2024-09-19T04:16:41.977096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984660+0000","last_change":"2024-09-19T04:16:41.984660+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984660+0000","last_undegraded":"2024-09-19T04:16:41.984660+0000","last_fullsized":"2024-09-19T04:16:41.984660+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.983716+0000","last_change":"2024-09-19T04:16:41.983716+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.983716+0000","last_undegraded":"2024-09-19T04:16:41.983716+0000","last_fullsized":"2024-09-19T04:16:41.983716+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982552+0000","last_change":"2024-09-19T04:16:41.982552+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982552+0000","last_undegraded":"2024-09-19T04:16:41.982552+0000","last_fullsized":"2024-09-19T04:16:41.982552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.977837+0000","last_change":"2024-09-19T04:16:41.977837+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.977837+0000","last_undegraded":"2024-09-19T04:16:41.977837+0000","last_fullsized":"2024-09-19T04:16:41.977837+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979819+0000","last_change":"2024-09-19T04:16:41.979819+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979819+0000","last_undegraded":"2024-09-19T04:16:41.979819+0000","last_fullsized":"2024-09-19T04:16:41.979819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986619+0000","last_change":"2024-09-19T04:16:41.986619+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986619+0000","last_undegraded":"2024-09-19T04:16:41.986619+0000","last_fullsized":"2024-09-19T04:16:41.986619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980172+0000","last_change":"2024-09-19T04:16:41.980172+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980172+0000","last_undegraded":"2024-09-19T04:16:41.980172+0000","last_fullsized":"2024-09-19T04:16:41.980172+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982680+0000","last_change":"2024-09-19T04:16:41.982680+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982680+0000","last_undegraded":"2024-09-19T04:16:41.982680+0000","last_fullsized":"2024-09-19T04:16:41.982680+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978736+0000","last_change":"2024-09-19T04:16:41.978736+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978736+0000","last_undegraded":"2024-09-19T04:16:41.978736+0000","last_fullsized":"2024-09-19T04:16:41.978736+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.989513+0000","last_change":"2024-09-19T04:16:41.989513+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.989513+0000","last_undegraded":"2024-09-19T04:16:41.989513+0000","last_fullsized":"2024-09-19T04:16:41.989513+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.990810+0000","last_change":"2024-09-19T04:16:41.990810+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.990810+0000","last_undegraded":"2024-09-19T04:16:41.990810+0000","last_fullsized":"2024-09-19T04:16:41.990810+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.993046+0000","last_change":"2024-09-19T04:16:41.993046+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.993046+0000","last_undegraded":"2024-09-19T04:16:41.993046+0000","last_fullsized":"2024-09-19T04:16:41.993046+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.017233+0000","last_change":"2024-09-19T04:16:42.013739+0000","last_active":"2024-09-19T04:16:42.017233+0000","last_peered":"2024-09-19T04:16:42.017233+0000","last_clean":"2024-09-19T04:16:42.017233+0000","last_became_active":"2024-09-19T04:16:42.013362+0000","last_became_peered":"2024-09-19T04:16:42.013362+0000","last_unstale":"2024-09-19T04:16:42.017233+0000","last_undegraded":"2024-09-19T04:16:42.017233+0000","last_fullsized":"2024-09-19T04:16:42.017233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:45:28.800656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985724+0000","last_change":"2024-09-19T04:16:41.985724+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985724+0000","last_undegraded":"2024-09-19T04:16:41.985724+0000","last_fullsized":"2024-09-19T04:16:41.985724+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.019939+0000","last_change":"2024-09-19T04:16:42.018926+0000","last_active":"2024-09-19T04:16:42.019939+0000","last_peered":"2024-09-19T04:16:42.019939+0000","last_clean":"2024-09-19T04:16:42.019939+0000","last_became_active":"2024-09-19T04:16:42.018655+0000","last_became_peered":"2024-09-19T04:16:42.018655+0000","last_unstale":"2024-09-19T04:16:42.019939+0000","last_undegraded":"2024-09-19T04:16:42.019939+0000","last_fullsized":"2024-09-19T04:16:42.019939+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:44.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.988796+0000","last_change":"2024-09-19T04:16:41.988796+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.988796+0000","last_undegraded":"2024-09-19T04:16:41.988796+0000","last_fullsized":"2024-09-19T04:16:41.988796+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.999950+0000","last_change":"2024-09-19T04:16:41.992501+0000","last_active":"2024-09-19T04:16:41.999950+0000","last_peered":"2024-09-19T04:16:41.999950+0000","last_clean":"2024-09-19T04:16:41.999950+0000","last_became_active":"2024-09-19T04:16:41.992135+0000","last_became_peered":"2024-09-19T04:16:41.992135+0000","last_unstale":"2024-09-19T04:16:41.999950+0000","last_undegraded":"2024-09-19T04:16:41.999950+0000","last_fullsized":"2024-09-19T04:16:41.999950+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.537033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.991338+0000","last_change":"2024-09-19T04:16:41.991338+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.991338+0000","last_undegraded":"2024-09-19T04:16:41.991338+0000","last_fullsized":"2024-09-19T04:16:41.991338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005233+0000","last_change":"2024-09-19T04:16:42.000668+0000","last_active":"2024-09-19T04:16:42.005233+0000","last_peered":"2024-09-19T04:16:42.005233+0000","last_clean":"2024-09-19T04:16:42.005233+0000","last_became_active":"2024-09-19T04:16:42.000249+0000","last_became_peered":"2024-09-19T04:16:42.000249+0000","last_unstale":"2024-09-19T04:16:42.005233+0000","last_undegraded":"2024-09-19T04:16:42.005233+0000","last_fullsized":"2024-09-19T04:16:42.005233+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.567498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984416+0000","last_change":"2024-09-19T04:16:41.984416+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984416+0000","last_undegraded":"2024-09-19T04:16:41.984416+0000","last_fullsized":"2024-09-19T04:16:41.984416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.990792+0000","last_change":"2024-09-19T04:16:41.989580+0000","last_active":"2024-09-19T04:16:41.990792+0000","last_peered":"2024-09-19T04:16:41.990792+0000","last_clean":"2024-09-19T04:16:41.990792+0000","last_became_active":"2024-09-19T04:16:41.989078+0000","last_became_peered":"2024-09-19T04:16:41.989078+0000","last_unstale":"2024-09-19T04:16:41.990792+0000","last_undegraded":"2024-09-19T04:16:41.990792+0000","last_fullsized":"2024-09-19T04:16:41.990792+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.567498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978924+0000","last_change":"2024-09-19T04:16:41.978924+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978924+0000","last_undegraded":"2024-09-19T04:16:41.978924+0000","last_fullsized":"2024-09-19T04:16:41.978924+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997003+0000","last_change":"2024-09-19T04:16:41.995699+0000","last_active":"2024-09-19T04:16:41.997003+0000","last_peered":"2024-09-19T04:16:41.997003+0000","last_clean":"2024-09-19T04:16:41.997003+0000","last_became_active":"2024-09-19T04:16:41.993440+0000","last_became_peered":"2024-09-19T04:16:41.993440+0000","last_unstale":"2024-09-19T04:16:41.997003+0000","last_undegraded":"2024-09-19T04:16:41.997003+0000","last_fullsized":"2024-09-19T04:16:41.997003+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:12.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.015296+0000","last_change":"2024-09-19T04:16:42.015296+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.015296+0000","last_undegraded":"2024-09-19T04:16:42.015296+0000","last_fullsized":"2024-09-19T04:16:42.015296+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024309+0000","last_change":"2024-09-19T04:16:42.022337+0000","last_active":"2024-09-19T04:16:42.024309+0000","last_peered":"2024-09-19T04:16:42.024309+0000","last_clean":"2024-09-19T04:16:42.024309+0000","last_became_active":"2024-09-19T04:16:42.021791+0000","last_became_peered":"2024-09-19T04:16:42.021791+0000","last_unstale":"2024-09-19T04:16:42.024309+0000","last_undegraded":"2024-09-19T04:16:42.024309+0000","last_fullsized":"2024-09-19T04:16:42.024309+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:33.295425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980459+0000","last_change":"2024-09-19T04:16:41.980459+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980459+0000","last_undegraded":"2024-09-19T04:16:41.980459+0000","last_fullsized":"2024-09-19T04:16:41.980459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.021722+0000","last_change":"2024-09-19T04:16:42.019556+0000","last_active":"2024-09-19T04:16:42.021722+0000","last_peered":"2024-09-19T04:16:42.021722+0000","last_clean":"2024-09-19T04:16:42.021722+0000","last_became_active":"2024-09-19T04:16:42.018870+0000","last_became_peered":"2024-09-19T04:16:42.018870+0000","last_unstale":"2024-09-19T04:16:42.021722+0000","last_undegraded":"2024-09-19T04:16:42.021722+0000","last_fullsized":"2024-09-19T04:16:42.021722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:50:19.666213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979749+0000","last_change":"2024-09-19T04:16:41.979749+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979749+0000","last_undegraded":"2024-09-19T04:16:41.979749+0000","last_fullsized":"2024-09-19T04:16:41.979749+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028493+0000","last_change":"2024-09-19T04:16:42.026415+0000","last_active":"2024-09-19T04:16:42.028493+0000","last_peered":"2024-09-19T04:16:42.028493+0000","last_clean":"2024-09-19T04:16:42.028493+0000","last_became_active":"2024-09-19T04:16:42.025740+0000","last_became_peered":"2024-09-19T04:16:42.025740+0000","last_unstale":"2024-09-19T04:16:42.028493+0000","last_undegraded":"2024-09-19T04:16:42.028493+0000","last_fullsized":"2024-09-19T04:16:42.028493+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:36.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979412+0000","last_change":"2024-09-19T04:16:41.979412+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979412+0000","last_undegraded":"2024-09-19T04:16:41.979412+0000","last_fullsized":"2024-09-19T04:16:41.979412+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005588+0000","last_change":"2024-09-19T04:16:42.001820+0000","last_active":"2024-09-19T04:16:42.005588+0000","last_peered":"2024-09-19T04:16:42.005588+0000","last_clean":"2024-09-19T04:16:42.005588+0000","last_became_active":"2024-09-19T04:16:42.001141+0000","last_became_peered":"2024-09-19T04:16:42.001141+0000","last_unstale":"2024-09-19T04:16:42.005588+0000","last_undegraded":"2024-09-19T04:16:42.005588+0000","last_fullsized":"2024-09-19T04:16:42.005588+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:28.101048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.994594+0000","last_change":"2024-09-19T04:16:41.994594+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.994594+0000","last_undegraded":"2024-09-19T04:16:41.994594+0000","last_fullsized":"2024-09-19T04:16:41.994594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.025625+0000","last_change":"2024-09-19T04:16:42.018953+0000","last_active":"2024-09-19T04:16:42.025625+0000","last_peered":"2024-09-19T04:16:42.025625+0000","last_clean":"2024-09-19T04:16:42.025625+0000","last_became_active":"2024-09-19T04:16:42.018215+0000","last_became_peered":"2024-09-19T04:16:42.018215+0000","last_unstale":"2024-09-19T04:16:42.025625+0000","last_undegraded":"2024-09-19T04:16:42.025625+0000","last_fullsized":"2024-09-19T04:16:42.025625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:09.582870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987703+0000","last_change":"2024-09-19T04:16:41.987703+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987703+0000","last_undegraded":"2024-09-19T04:16:41.987703+0000","last_fullsized":"2024-09-19T04:16:41.987703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.008487+0000","last_change":"2024-09-19T04:16:42.006482+0000","last_active":"2024-09-19T04:16:42.008487+0000","last_peered":"2024-09-19T04:16:42.008487+0000","last_clean":"2024-09-19T04:16:42.008487+0000","last_became_active":"2024-09-19T04:16:42.006111+0000","last_became_peered":"2024-09-19T04:16:42.006111+0000","last_unstale":"2024-09-19T04:16:42.008487+0000","last_undegraded":"2024-09-19T04:16:42.008487+0000","last_fullsized":"2024-09-19T04:16:42.008487+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:46:11.291559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986477+0000","last_change":"2024-09-19T04:16:41.986477+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986477+0000","last_undegraded":"2024-09-19T04:16:41.986477+0000","last_fullsized":"2024-09-19T04:16:41.986477+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028432+0000","last_change":"2024-09-19T04:16:42.024717+0000","last_active":"2024-09-19T04:16:42.028432+0000","last_peered":"2024-09-19T04:16:42.028432+0000","last_clean":"2024-09-19T04:16:42.028432+0000","last_became_active":"2024-09-19T04:16:42.023979+0000","last_became_peered":"2024-09-19T04:16:42.023979+0000","last_unstale":"2024-09-19T04:16:42.028432+0000","last_undegraded":"2024-09-19T04:16:42.028432+0000","last_fullsized":"2024-09-19T04:16:42.028432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:14.632041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978077+0000","last_change":"2024-09-19T04:16:41.978077+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978077+0000","last_undegraded":"2024-09-19T04:16:41.978077+0000","last_fullsized":"2024-09-19T04:16:41.978077+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.006392+0000","last_change":"2024-09-19T04:16:42.002996+0000","last_active":"2024-09-19T04:16:42.006392+0000","last_peered":"2024-09-19T04:16:42.006392+0000","last_clean":"2024-09-19T04:16:42.006392+0000","last_became_active":"2024-09-19T04:16:42.001542+0000","last_became_peered":"2024-09-19T04:16:42.001542+0000","last_unstale":"2024-09-19T04:16:42.006392+0000","last_undegraded":"2024-09-19T04:16:42.006392+0000","last_fullsized":"2024-09-19T04:16:42.006392+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:08.070747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.014200+0000","last_change":"2024-09-19T04:16:42.014200+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.014200+0000","last_undegraded":"2024-09-19T04:16:42.014200+0000","last_fullsized":"2024-09-19T04:16:42.014200+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024770+0000","last_change":"2024-09-19T04:16:42.018769+0000","last_active":"2024-09-19T04:16:42.024770+0000","last_peered":"2024-09-19T04:16:42.024770+0000","last_clean":"2024-09-19T04:16:42.024770+0000","last_became_active":"2024-09-19T04:16:42.017955+0000","last_became_peered":"2024-09-19T04:16:42.017955+0000","last_unstale":"2024-09-19T04:16:42.024770+0000","last_undegraded":"2024-09-19T04:16:42.024770+0000","last_fullsized":"2024-09-19T04:16:42.024770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:19.225233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.979070+0000","last_change":"2024-09-19T04:16:41.979070+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.979070+0000","last_undegraded":"2024-09-19T04:16:41.979070+0000","last_fullsized":"2024-09-19T04:16:41.979070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.016442+0000","last_change":"2024-09-19T04:16:42.011251+0000","last_active":"2024-09-19T04:16:42.016442+0000","last_peered":"2024-09-19T04:16:42.016442+0000","last_clean":"2024-09-19T04:16:42.016442+0000","last_became_active":"2024-09-19T04:16:42.010096+0000","last_became_peered":"2024-09-19T04:16:42.010096+0000","last_unstale":"2024-09-19T04:16:42.016442+0000","last_undegraded":"2024-09-19T04:16:42.016442+0000","last_fullsized":"2024-09-19T04:16:42.016442+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:09.087251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987287+0000","last_change":"2024-09-19T04:16:41.987287+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987287+0000","last_undegraded":"2024-09-19T04:16:41.987287+0000","last_fullsized":"2024-09-19T04:16:41.987287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.001783+0000","last_change":"2024-09-19T04:16:42.000089+0000","last_active":"2024-09-19T04:16:42.001783+0000","last_peered":"2024-09-19T04:16:42.001783+0000","last_clean":"2024-09-19T04:16:42.001783+0000","last_became_active":"2024-09-19T04:16:41.999376+0000","last_became_peered":"2024-09-19T04:16:41.999376+0000","last_unstale":"2024-09-19T04:16:42.001783+0000","last_undegraded":"2024-09-19T04:16:42.001783+0000","last_fullsized":"2024-09-19T04:16:42.001783+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:34.395480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.997756+0000","last_change":"2024-09-19T04:16:41.997756+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.997756+0000","last_undegraded":"2024-09-19T04:16:41.997756+0000","last_fullsized":"2024-09-19T04:16:41.997756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.027534+0000","last_change":"2024-09-19T04:16:42.023483+0000","last_active":"2024-09-19T04:16:42.027534+0000","last_peered":"2024-09-19T04:16:42.027534+0000","last_clean":"2024-09-19T04:16:42.027534+0000","last_became_active":"2024-09-19T04:16:42.022787+0000","last_became_peered":"2024-09-19T04:16:42.022787+0000","last_unstale":"2024-09-19T04:16:42.027534+0000","last_undegraded":"2024-09-19T04:16:42.027534+0000","last_fullsized":"2024-09-19T04:16:42.027534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:54.902392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984262+0000","last_change":"2024-09-19T04:16:41.984262+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984262+0000","last_undegraded":"2024-09-19T04:16:41.984262+0000","last_fullsized":"2024-09-19T04:16:41.984262+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028461+0000","last_change":"2024-09-19T04:16:42.026255+0000","last_active":"2024-09-19T04:16:42.028461+0000","last_peered":"2024-09-19T04:16:42.028461+0000","last_clean":"2024-09-19T04:16:42.028461+0000","last_became_active":"2024-09-19T04:16:42.025430+0000","last_became_peered":"2024-09-19T04:16:42.025430+0000","last_unstale":"2024-09-19T04:16:42.028461+0000","last_undegraded":"2024-09-19T04:16:42.028461+0000","last_fullsized":"2024-09-19T04:16:42.028461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:18.263858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999999+0000","last_change":"2024-09-19T04:16:41.999999+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999999+0000","last_undegraded":"2024-09-19T04:16:41.999999+0000","last_fullsized":"2024-09-19T04:16:41.999999+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.999802+0000","last_change":"2024-09-19T04:16:41.998882+0000","last_active":"2024-09-19T04:16:41.999802+0000","last_peered":"2024-09-19T04:16:41.999802+0000","last_clean":"2024-09-19T04:16:41.999802+0000","last_became_active":"2024-09-19T04:16:41.998612+0000","last_became_peered":"2024-09-19T04:16:41.998612+0000","last_unstale":"2024-09-19T04:16:41.999802+0000","last_undegraded":"2024-09-19T04:16:41.999802+0000","last_fullsized":"2024-09-19T04:16:41.999802+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:44.382019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987405+0000","last_change":"2024-09-19T04:16:41.987405+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987405+0000","last_undegraded":"2024-09-19T04:16:41.987405+0000","last_fullsized":"2024-09-19T04:16:41.987405+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.017443+0000","last_change":"2024-09-19T04:16:42.015741+0000","last_active":"2024-09-19T04:16:42.017443+0000","last_peered":"2024-09-19T04:16:42.017443+0000","last_clean":"2024-09-19T04:16:42.017443+0000","last_became_active":"2024-09-19T04:16:42.015285+0000","last_became_peered":"2024-09-19T04:16:42.015285+0000","last_unstale":"2024-09-19T04:16:42.017443+0000","last_undegraded":"2024-09-19T04:16:42.017443+0000","last_fullsized":"2024-09-19T04:16:42.017443+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:12.121469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.976440+0000","last_change":"2024-09-19T04:16:41.976440+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.976440+0000","last_undegraded":"2024-09-19T04:16:41.976440+0000","last_fullsized":"2024-09-19T04:16:41.976440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.004913+0000","last_change":"2024-09-19T04:16:42.002264+0000","last_active":"2024-09-19T04:16:42.004913+0000","last_peered":"2024-09-19T04:16:42.004913+0000","last_clean":"2024-09-19T04:16:42.004913+0000","last_became_active":"2024-09-19T04:16:42.000593+0000","last_became_peered":"2024-09-19T04:16:42.000593+0000","last_unstale":"2024-09-19T04:16:42.004913+0000","last_undegraded":"2024-09-19T04:16:42.004913+0000","last_fullsized":"2024-09-19T04:16:42.004913+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:36.127882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.999116+0000","last_change":"2024-09-19T04:16:41.999116+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.999116+0000","last_undegraded":"2024-09-19T04:16:41.999116+0000","last_fullsized":"2024-09-19T04:16:41.999116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.011261+0000","last_change":"2024-09-19T04:16:42.010194+0000","last_active":"2024-09-19T04:16:42.011261+0000","last_peered":"2024-09-19T04:16:42.011261+0000","last_clean":"2024-09-19T04:16:42.011261+0000","last_became_active":"2024-09-19T04:16:42.009783+0000","last_became_peered":"2024-09-19T04:16:42.009783+0000","last_unstale":"2024-09-19T04:16:42.011261+0000","last_undegraded":"2024-09-19T04:16:42.011261+0000","last_fullsized":"2024-09-19T04:16:42.011261+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:02.962114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987074+0000","last_change":"2024-09-19T04:16:41.987074+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987074+0000","last_undegraded":"2024-09-19T04:16:41.987074+0000","last_fullsized":"2024-09-19T04:16:41.987074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.012421+0000","last_change":"2024-09-19T04:16:42.010812+0000","last_active":"2024-09-19T04:16:42.012421+0000","last_peered":"2024-09-19T04:16:42.012421+0000","last_clean":"2024-09-19T04:16:42.012421+0000","last_became_active":"2024-09-19T04:16:42.010051+0000","last_became_peered":"2024-09-19T04:16:42.010051+0000","last_unstale":"2024-09-19T04:16:42.012421+0000","last_undegraded":"2024-09-19T04:16:42.012421+0000","last_fullsized":"2024-09-19T04:16:42.012421+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:52:53.450248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986784+0000","last_change":"2024-09-19T04:16:41.986784+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986784+0000","last_undegraded":"2024-09-19T04:16:41.986784+0000","last_fullsized":"2024-09-19T04:16:41.986784+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997616+0000","last_change":"2024-09-19T04:16:41.995074+0000","last_active":"2024-09-19T04:16:41.997616+0000","last_peered":"2024-09-19T04:16:41.997616+0000","last_clean":"2024-09-19T04:16:41.997616+0000","last_became_active":"2024-09-19T04:16:41.994321+0000","last_became_peered":"2024-09-19T04:16:41.994321+0000","last_unstale":"2024-09-19T04:16:41.997616+0000","last_undegraded":"2024-09-19T04:16:41.997616+0000","last_fullsized":"2024-09-19T04:16:41.997616+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:34.017414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.978779+0000","last_change":"2024-09-19T04:16:41.978779+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.978779+0000","last_undegraded":"2024-09-19T04:16:41.978779+0000","last_fullsized":"2024-09-19T04:16:41.978779+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.997258+0000","last_change":"2024-09-19T04:16:41.991513+0000","last_active":"2024-09-19T04:16:41.997258+0000","last_peered":"2024-09-19T04:16:41.997258+0000","last_clean":"2024-09-19T04:16:41.997258+0000","last_became_active":"2024-09-19T04:16:41.990986+0000","last_became_peered":"2024-09-19T04:16:41.990986+0000","last_unstale":"2024-09-19T04:16:41.997258+0000","last_undegraded":"2024-09-19T04:16:41.997258+0000","last_fullsized":"2024-09-19T04:16:41.997258+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:52:47.451247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.998881+0000","last_change":"2024-09-19T04:16:41.998881+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.998881+0000","last_undegraded":"2024-09-19T04:16:41.998881+0000","last_fullsized":"2024-09-19T04:16:41.998881+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.005686+0000","last_change":"2024-09-19T04:16:42.003466+0000","last_active":"2024-09-19T04:16:42.005686+0000","last_peered":"2024-09-19T04:16:42.005686+0000","last_clean":"2024-09-19T04:16:42.005686+0000","last_became_active":"2024-09-19T04:16:42.003135+0000","last_became_peered":"2024-09-19T04:16:42.003135+0000","last_unstale":"2024-09-19T04:16:42.005686+0000","last_undegraded":"2024-09-19T04:16:42.005686+0000","last_fullsized":"2024-09-19T04:16:42.005686+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:23.772770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:42.014982+0000","last_change":"2024-09-19T04:16:42.014982+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:42.014982+0000","last_undegraded":"2024-09-19T04:16:42.014982+0000","last_fullsized":"2024-09-19T04:16:42.014982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.026933+0000","last_change":"2024-09-19T04:16:42.019204+0000","last_active":"2024-09-19T04:16:42.026933+0000","last_peered":"2024-09-19T04:16:42.026933+0000","last_clean":"2024-09-19T04:16:42.026933+0000","last_became_active":"2024-09-19T04:16:42.018134+0000","last_became_peered":"2024-09-19T04:16:42.018134+0000","last_unstale":"2024-09-19T04:16:42.026933+0000","last_undegraded":"2024-09-19T04:16:42.026933+0000","last_fullsized":"2024-09-19T04:16:42.026933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:41.461855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987022+0000","last_change":"2024-09-19T04:16:41.987022+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987022+0000","last_undegraded":"2024-09-19T04:16:41.987022+0000","last_fullsized":"2024-09-19T04:16:41.987022+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.996533+0000","last_change":"2024-09-19T04:16:41.994558+0000","last_active":"2024-09-19T04:16:41.996533+0000","last_peered":"2024-09-19T04:16:41.996533+0000","last_clean":"2024-09-19T04:16:41.996533+0000","last_became_active":"2024-09-19T04:16:41.993965+0000","last_became_peered":"2024-09-19T04:16:41.993965+0000","last_unstale":"2024-09-19T04:16:41.996533+0000","last_undegraded":"2024-09-19T04:16:41.996533+0000","last_fullsized":"2024-09-19T04:16:41.996533+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:42:02.962114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.980245+0000","last_change":"2024-09-19T04:16:41.980245+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.980245+0000","last_undegraded":"2024-09-19T04:16:41.980245+0000","last_fullsized":"2024-09-19T04:16:41.980245+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024381+0000","last_change":"2024-09-19T04:16:42.022760+0000","last_active":"2024-09-19T04:16:42.024381+0000","last_peered":"2024-09-19T04:16:42.024381+0000","last_clean":"2024-09-19T04:16:42.024381+0000","last_became_active":"2024-09-19T04:16:42.022030+0000","last_became_peered":"2024-09-19T04:16:42.022030+0000","last_unstale":"2024-09-19T04:16:42.024381+0000","last_undegraded":"2024-09-19T04:16:42.024381+0000","last_fullsized":"2024-09-19T04:16:42.024381+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:59.963896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986891+0000","last_change":"2024-09-19T04:16:41.986891+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986891+0000","last_undegraded":"2024-09-19T04:16:41.986891+0000","last_fullsized":"2024-09-19T04:16:41.986891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024272+0000","last_change":"2024-09-19T04:16:42.022541+0000","last_active":"2024-09-19T04:16:42.024272+0000","last_peered":"2024-09-19T04:16:42.024272+0000","last_clean":"2024-09-19T04:16:42.024272+0000","last_became_active":"2024-09-19T04:16:42.022021+0000","last_became_peered":"2024-09-19T04:16:42.022021+0000","last_unstale":"2024-09-19T04:16:42.024272+0000","last_undegraded":"2024-09-19T04:16:42.024272+0000","last_fullsized":"2024-09-19T04:16:42.024272+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:34:20.649775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.994302+0000","last_change":"2024-09-19T04:16:41.994302+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.994302+0000","last_undegraded":"2024-09-19T04:16:41.994302+0000","last_fullsized":"2024-09-19T04:16:41.994302+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:41.989613+0000","last_change":"2024-09-19T04:16:41.988994+0000","last_active":"2024-09-19T04:16:41.989613+0000","last_peered":"2024-09-19T04:16:41.989613+0000","last_clean":"2024-09-19T04:16:41.989613+0000","last_became_active":"2024-09-19T04:16:41.988755+0000","last_became_peered":"2024-09-19T04:16:41.988755+0000","last_unstale":"2024-09-19T04:16:41.989613+0000","last_undegraded":"2024-09-19T04:16:41.989613+0000","last_fullsized":"2024-09-19T04:16:41.989613+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.537033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.009326+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:16:42.009326+0000","last_peered":"2024-09-19T04:16:42.009326+0000","last_clean":"2024-09-19T04:16:42.009326+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:16:42.009326+0000","last_undegraded":"2024-09-19T04:16:42.009326+0000","last_fullsized":"2024-09-19T04:16:42.009326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":945112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542190,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542190,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542190,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":945112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:16:47.757 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3f is not active+clean 2024-09-19T04:16:47.757 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.981392+0000', 'last_change': '2024-09-19T04:16:41.981392+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.981392+0000', 'last_undegraded': '2024-09-19T04:16:41.981392+0000', 'last_fullsized': '2024-09-19T04:16:41.981392+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.757 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3e is not active+clean 2024-09-19T04:16:47.757 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.980603+0000', 'last_change': '2024-09-19T04:16:41.980603+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.980603+0000', 'last_undegraded': '2024-09-19T04:16:41.980603+0000', 'last_fullsized': '2024-09-19T04:16:41.980603+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.758 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3d is not active+clean 2024-09-19T04:16:47.758 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.982715+0000', 'last_change': '2024-09-19T04:16:41.982715+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.982715+0000', 'last_undegraded': '2024-09-19T04:16:41.982715+0000', 'last_fullsized': '2024-09-19T04:16:41.982715+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.758 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3c is not active+clean 2024-09-19T04:16:47.758 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.984499+0000', 'last_change': '2024-09-19T04:16:41.984499+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.984499+0000', 'last_undegraded': '2024-09-19T04:16:41.984499+0000', 'last_fullsized': '2024-09-19T04:16:41.984499+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.758 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3b is not active+clean 2024-09-19T04:16:47.758 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.998802+0000', 'last_change': '2024-09-19T04:16:41.998802+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.998802+0000', 'last_undegraded': '2024-09-19T04:16:41.998802+0000', 'last_fullsized': '2024-09-19T04:16:41.998802+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.758 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3a is not active+clean 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.988501+0000', 'last_change': '2024-09-19T04:16:41.988501+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.988501+0000', 'last_undegraded': '2024-09-19T04:16:41.988501+0000', 'last_fullsized': '2024-09-19T04:16:41.988501+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:PG 10.39 is not active+clean 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.984236+0000', 'last_change': '2024-09-19T04:16:41.984236+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.984236+0000', 'last_undegraded': '2024-09-19T04:16:41.984236+0000', 'last_fullsized': '2024-09-19T04:16:41.984236+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:PG 10.38 is not active+clean 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.976138+0000', 'last_change': '2024-09-19T04:16:41.976138+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.976138+0000', 'last_undegraded': '2024-09-19T04:16:41.976138+0000', 'last_fullsized': '2024-09-19T04:16:41.976138+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:PG 10.37 is not active+clean 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.984883+0000', 'last_change': '2024-09-19T04:16:41.984883+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.984883+0000', 'last_undegraded': '2024-09-19T04:16:41.984883+0000', 'last_fullsized': '2024-09-19T04:16:41.984883+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:PG 10.36 is not active+clean 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.984199+0000', 'last_change': '2024-09-19T04:16:41.984199+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.984199+0000', 'last_undegraded': '2024-09-19T04:16:41.984199+0000', 'last_fullsized': '2024-09-19T04:16:41.984199+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.759 INFO:tasks.ceph.ceph_manager.ceph:PG 10.35 is not active+clean 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.987719+0000', 'last_change': '2024-09-19T04:16:41.987719+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.987719+0000', 'last_undegraded': '2024-09-19T04:16:41.987719+0000', 'last_fullsized': '2024-09-19T04:16:41.987719+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:PG 10.34 is not active+clean 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.976359+0000', 'last_change': '2024-09-19T04:16:41.976359+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.976359+0000', 'last_undegraded': '2024-09-19T04:16:41.976359+0000', 'last_fullsized': '2024-09-19T04:16:41.976359+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:PG 10.33 is not active+clean 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.979859+0000', 'last_change': '2024-09-19T04:16:41.979859+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.979859+0000', 'last_undegraded': '2024-09-19T04:16:41.979859+0000', 'last_fullsized': '2024-09-19T04:16:41.979859+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:PG 10.32 is not active+clean 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.983462+0000', 'last_change': '2024-09-19T04:16:41.983462+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.983462+0000', 'last_undegraded': '2024-09-19T04:16:41.983462+0000', 'last_fullsized': '2024-09-19T04:16:41.983462+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:PG 10.31 is not active+clean 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.982729+0000', 'last_change': '2024-09-19T04:16:41.982729+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.982729+0000', 'last_undegraded': '2024-09-19T04:16:41.982729+0000', 'last_fullsized': '2024-09-19T04:16:41.982729+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.760 INFO:tasks.ceph.ceph_manager.ceph:PG 10.30 is not active+clean 2024-09-19T04:16:47.761 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.985026+0000', 'last_change': '2024-09-19T04:16:41.985026+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.985026+0000', 'last_undegraded': '2024-09-19T04:16:41.985026+0000', 'last_fullsized': '2024-09-19T04:16:41.985026+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:16:47.761 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2f is not active+clean 2024-09-19T04:16:47.761 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.980867+0000', 'last_change': '2024-09-19T04:16:41.980867+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.980867+0000', 'last_undegraded': '2024-09-19T04:16:41.980867+0000', 'last_fullsized': '2024-09-19T04:16:41.980867+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.761 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2e is not active+clean 2024-09-19T04:16:47.761 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.985372+0000', 'last_change': '2024-09-19T04:16:41.985372+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.985372+0000', 'last_undegraded': '2024-09-19T04:16:41.985372+0000', 'last_fullsized': '2024-09-19T04:16:41.985372+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.761 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2d is not active+clean 2024-09-19T04:16:47.761 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.999302+0000', 'last_change': '2024-09-19T04:16:41.999302+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.999302+0000', 'last_undegraded': '2024-09-19T04:16:41.999302+0000', 'last_fullsized': '2024-09-19T04:16:41.999302+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2c is not active+clean 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.980025+0000', 'last_change': '2024-09-19T04:16:41.980025+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.980025+0000', 'last_undegraded': '2024-09-19T04:16:41.980025+0000', 'last_fullsized': '2024-09-19T04:16:41.980025+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2b is not active+clean 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.977096+0000', 'last_change': '2024-09-19T04:16:41.977096+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.977096+0000', 'last_undegraded': '2024-09-19T04:16:41.977096+0000', 'last_fullsized': '2024-09-19T04:16:41.977096+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2a is not active+clean 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.984660+0000', 'last_change': '2024-09-19T04:16:41.984660+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.984660+0000', 'last_undegraded': '2024-09-19T04:16:41.984660+0000', 'last_fullsized': '2024-09-19T04:16:41.984660+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:PG 10.29 is not active+clean 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.983716+0000', 'last_change': '2024-09-19T04:16:41.983716+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.983716+0000', 'last_undegraded': '2024-09-19T04:16:41.983716+0000', 'last_fullsized': '2024-09-19T04:16:41.983716+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:PG 10.28 is not active+clean 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.982552+0000', 'last_change': '2024-09-19T04:16:41.982552+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.982552+0000', 'last_undegraded': '2024-09-19T04:16:41.982552+0000', 'last_fullsized': '2024-09-19T04:16:41.982552+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:PG 10.27 is not active+clean 2024-09-19T04:16:47.762 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.977837+0000', 'last_change': '2024-09-19T04:16:41.977837+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.977837+0000', 'last_undegraded': '2024-09-19T04:16:41.977837+0000', 'last_fullsized': '2024-09-19T04:16:41.977837+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:PG 10.26 is not active+clean 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.979819+0000', 'last_change': '2024-09-19T04:16:41.979819+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.979819+0000', 'last_undegraded': '2024-09-19T04:16:41.979819+0000', 'last_fullsized': '2024-09-19T04:16:41.979819+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:PG 10.25 is not active+clean 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.986619+0000', 'last_change': '2024-09-19T04:16:41.986619+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.986619+0000', 'last_undegraded': '2024-09-19T04:16:41.986619+0000', 'last_fullsized': '2024-09-19T04:16:41.986619+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:PG 10.24 is not active+clean 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.980172+0000', 'last_change': '2024-09-19T04:16:41.980172+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.980172+0000', 'last_undegraded': '2024-09-19T04:16:41.980172+0000', 'last_fullsized': '2024-09-19T04:16:41.980172+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:PG 10.23 is not active+clean 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.982680+0000', 'last_change': '2024-09-19T04:16:41.982680+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.982680+0000', 'last_undegraded': '2024-09-19T04:16:41.982680+0000', 'last_fullsized': '2024-09-19T04:16:41.982680+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:PG 10.22 is not active+clean 2024-09-19T04:16:47.763 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.978736+0000', 'last_change': '2024-09-19T04:16:41.978736+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.978736+0000', 'last_undegraded': '2024-09-19T04:16:41.978736+0000', 'last_fullsized': '2024-09-19T04:16:41.978736+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:PG 10.21 is not active+clean 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.989513+0000', 'last_change': '2024-09-19T04:16:41.989513+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.989513+0000', 'last_undegraded': '2024-09-19T04:16:41.989513+0000', 'last_fullsized': '2024-09-19T04:16:41.989513+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:PG 10.20 is not active+clean 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.990810+0000', 'last_change': '2024-09-19T04:16:41.990810+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.990810+0000', 'last_undegraded': '2024-09-19T04:16:41.990810+0000', 'last_fullsized': '2024-09-19T04:16:41.990810+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1c is not active+clean 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.993046+0000', 'last_change': '2024-09-19T04:16:41.993046+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.993046+0000', 'last_undegraded': '2024-09-19T04:16:41.993046+0000', 'last_fullsized': '2024-09-19T04:16:41.993046+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1d is not active+clean 2024-09-19T04:16:47.764 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.985724+0000', 'last_change': '2024-09-19T04:16:41.985724+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.985724+0000', 'last_undegraded': '2024-09-19T04:16:41.985724+0000', 'last_fullsized': '2024-09-19T04:16:41.985724+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1e is not active+clean 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.988796+0000', 'last_change': '2024-09-19T04:16:41.988796+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.988796+0000', 'last_undegraded': '2024-09-19T04:16:41.988796+0000', 'last_fullsized': '2024-09-19T04:16:41.988796+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1f is not active+clean 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.991338+0000', 'last_change': '2024-09-19T04:16:41.991338+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.991338+0000', 'last_undegraded': '2024-09-19T04:16:41.991338+0000', 'last_fullsized': '2024-09-19T04:16:41.991338+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:PG 10.18 is not active+clean 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.984416+0000', 'last_change': '2024-09-19T04:16:41.984416+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.984416+0000', 'last_undegraded': '2024-09-19T04:16:41.984416+0000', 'last_fullsized': '2024-09-19T04:16:41.984416+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:PG 10.19 is not active+clean 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.978924+0000', 'last_change': '2024-09-19T04:16:41.978924+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.978924+0000', 'last_undegraded': '2024-09-19T04:16:41.978924+0000', 'last_fullsized': '2024-09-19T04:16:41.978924+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1a is not active+clean 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:42.015296+0000', 'last_change': '2024-09-19T04:16:42.015296+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:42.015296+0000', 'last_undegraded': '2024-09-19T04:16:42.015296+0000', 'last_fullsized': '2024-09-19T04:16:42.015296+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1b is not active+clean 2024-09-19T04:16:47.765 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.980459+0000', 'last_change': '2024-09-19T04:16:41.980459+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.980459+0000', 'last_undegraded': '2024-09-19T04:16:41.980459+0000', 'last_fullsized': '2024-09-19T04:16:41.980459+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:PG 10.14 is not active+clean 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.979749+0000', 'last_change': '2024-09-19T04:16:41.979749+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.979749+0000', 'last_undegraded': '2024-09-19T04:16:41.979749+0000', 'last_fullsized': '2024-09-19T04:16:41.979749+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:PG 10.15 is not active+clean 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.979412+0000', 'last_change': '2024-09-19T04:16:41.979412+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.979412+0000', 'last_undegraded': '2024-09-19T04:16:41.979412+0000', 'last_fullsized': '2024-09-19T04:16:41.979412+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:PG 10.16 is not active+clean 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.994594+0000', 'last_change': '2024-09-19T04:16:41.994594+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.994594+0000', 'last_undegraded': '2024-09-19T04:16:41.994594+0000', 'last_fullsized': '2024-09-19T04:16:41.994594+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:PG 10.17 is not active+clean 2024-09-19T04:16:47.766 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.987703+0000', 'last_change': '2024-09-19T04:16:41.987703+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.987703+0000', 'last_undegraded': '2024-09-19T04:16:41.987703+0000', 'last_fullsized': '2024-09-19T04:16:41.987703+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.767 INFO:tasks.ceph.ceph_manager.ceph:PG 10.10 is not active+clean 2024-09-19T04:16:47.767 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.986477+0000', 'last_change': '2024-09-19T04:16:41.986477+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.986477+0000', 'last_undegraded': '2024-09-19T04:16:41.986477+0000', 'last_fullsized': '2024-09-19T04:16:41.986477+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.767 INFO:tasks.ceph.ceph_manager.ceph:PG 10.11 is not active+clean 2024-09-19T04:16:47.767 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.978077+0000', 'last_change': '2024-09-19T04:16:41.978077+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.978077+0000', 'last_undegraded': '2024-09-19T04:16:41.978077+0000', 'last_fullsized': '2024-09-19T04:16:41.978077+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.767 INFO:tasks.ceph.ceph_manager.ceph:PG 10.12 is not active+clean 2024-09-19T04:16:47.767 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:42.014200+0000', 'last_change': '2024-09-19T04:16:42.014200+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:42.014200+0000', 'last_undegraded': '2024-09-19T04:16:42.014200+0000', 'last_fullsized': '2024-09-19T04:16:42.014200+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.768 INFO:tasks.ceph.ceph_manager.ceph:PG 10.13 is not active+clean 2024-09-19T04:16:47.768 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.979070+0000', 'last_change': '2024-09-19T04:16:41.979070+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.979070+0000', 'last_undegraded': '2024-09-19T04:16:41.979070+0000', 'last_fullsized': '2024-09-19T04:16:41.979070+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.768 INFO:tasks.ceph.ceph_manager.ceph:PG 10.c is not active+clean 2024-09-19T04:16:47.768 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.987287+0000', 'last_change': '2024-09-19T04:16:41.987287+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.987287+0000', 'last_undegraded': '2024-09-19T04:16:41.987287+0000', 'last_fullsized': '2024-09-19T04:16:41.987287+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-19T04:16:47.768 INFO:tasks.ceph.ceph_manager.ceph:PG 10.d is not active+clean 2024-09-19T04:16:47.768 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.997756+0000', 'last_change': '2024-09-19T04:16:41.997756+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.997756+0000', 'last_undegraded': '2024-09-19T04:16:41.997756+0000', 'last_fullsized': '2024-09-19T04:16:41.997756+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:PG 10.e is not active+clean 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.984262+0000', 'last_change': '2024-09-19T04:16:41.984262+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.984262+0000', 'last_undegraded': '2024-09-19T04:16:41.984262+0000', 'last_fullsized': '2024-09-19T04:16:41.984262+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:PG 10.f is not active+clean 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.999999+0000', 'last_change': '2024-09-19T04:16:41.999999+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.999999+0000', 'last_undegraded': '2024-09-19T04:16:41.999999+0000', 'last_fullsized': '2024-09-19T04:16:41.999999+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:PG 10.8 is not active+clean 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.987405+0000', 'last_change': '2024-09-19T04:16:41.987405+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.987405+0000', 'last_undegraded': '2024-09-19T04:16:41.987405+0000', 'last_fullsized': '2024-09-19T04:16:41.987405+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:PG 10.9 is not active+clean 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.976440+0000', 'last_change': '2024-09-19T04:16:41.976440+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.976440+0000', 'last_undegraded': '2024-09-19T04:16:41.976440+0000', 'last_fullsized': '2024-09-19T04:16:41.976440+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:PG 10.a is not active+clean 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.999116+0000', 'last_change': '2024-09-19T04:16:41.999116+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.999116+0000', 'last_undegraded': '2024-09-19T04:16:41.999116+0000', 'last_fullsized': '2024-09-19T04:16:41.999116+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.769 INFO:tasks.ceph.ceph_manager.ceph:PG 10.4 is not active+clean 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.987074+0000', 'last_change': '2024-09-19T04:16:41.987074+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.987074+0000', 'last_undegraded': '2024-09-19T04:16:41.987074+0000', 'last_fullsized': '2024-09-19T04:16:41.987074+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:PG 10.5 is not active+clean 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.986784+0000', 'last_change': '2024-09-19T04:16:41.986784+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.986784+0000', 'last_undegraded': '2024-09-19T04:16:41.986784+0000', 'last_fullsized': '2024-09-19T04:16:41.986784+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:PG 10.6 is not active+clean 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.978779+0000', 'last_change': '2024-09-19T04:16:41.978779+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.978779+0000', 'last_undegraded': '2024-09-19T04:16:41.978779+0000', 'last_fullsized': '2024-09-19T04:16:41.978779+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:PG 10.7 is not active+clean 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.998881+0000', 'last_change': '2024-09-19T04:16:41.998881+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.998881+0000', 'last_undegraded': '2024-09-19T04:16:41.998881+0000', 'last_fullsized': '2024-09-19T04:16:41.998881+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:PG 10.0 is not active+clean 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:42.014982+0000', 'last_change': '2024-09-19T04:16:42.014982+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:42.014982+0000', 'last_undegraded': '2024-09-19T04:16:42.014982+0000', 'last_fullsized': '2024-09-19T04:16:42.014982+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:PG 10.1 is not active+clean 2024-09-19T04:16:47.770 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.987022+0000', 'last_change': '2024-09-19T04:16:41.987022+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.987022+0000', 'last_undegraded': '2024-09-19T04:16:41.987022+0000', 'last_fullsized': '2024-09-19T04:16:41.987022+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-19T04:16:47.771 INFO:tasks.ceph.ceph_manager.ceph:PG 10.2 is not active+clean 2024-09-19T04:16:47.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.980245+0000', 'last_change': '2024-09-19T04:16:41.980245+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.980245+0000', 'last_undegraded': '2024-09-19T04:16:41.980245+0000', 'last_fullsized': '2024-09-19T04:16:41.980245+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-19T04:16:47.771 INFO:tasks.ceph.ceph_manager.ceph:PG 10.3 is not active+clean 2024-09-19T04:16:47.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.986891+0000', 'last_change': '2024-09-19T04:16:41.986891+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.986891+0000', 'last_undegraded': '2024-09-19T04:16:41.986891+0000', 'last_fullsized': '2024-09-19T04:16:41.986891+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-19T04:16:47.771 INFO:tasks.ceph.ceph_manager.ceph:PG 10.b is not active+clean 2024-09-19T04:16:47.771 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 65, 'state': 'creating+peering', 'last_fresh': '2024-09-19T04:16:41.994302+0000', 'last_change': '2024-09-19T04:16:41.994302+0000', 'last_active': '2024-09-19T04:16:41.962650+0000', 'last_peered': '2024-09-19T04:16:41.962650+0000', 'last_clean': '2024-09-19T04:16:41.962650+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-19T04:16:41.994302+0000', 'last_undegraded': '2024-09-19T04:16:41.994302+0000', 'last_fullsized': '2024-09-19T04:16:41.994302+0000', 'mapping_epoch': 65, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 65, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'last_clean_scrub_stamp': '2024-09-19T04:16:41.962650+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-19T04:16:47.772 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:16:47.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.899+0000 7f7b44bf8640 1 Processor -- start 2024-09-19T04:16:47.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.899+0000 7f7b44bf8640 1 -- start start 2024-09-19T04:16:47.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.899+0000 7f7b44bf8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b401630c0 0x7f7b4006f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.899+0000 7f7b44bf8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b40163bb0 con 0x7f7b4006df70 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.899+0000 7f7b44bf8640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b4006e470 con 0x7f7b400b3290 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.899+0000 7f7b44bf8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7b4006e750 con 0x7f7b401630c0 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7b4006df70 0x7f7b401634a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33316/0 (socket says 172.21.15.76:33316) 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 -- 172.21.15.76:0/2077912922 learned_addr learned my addr 172.21.15.76:0/2077912922 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3e59c640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b401630c0 0x7f7b4006f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2733905985 0 0) 0x7f7b40163bb0 con 0x7f7b4006df70 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b20003530 con 0x7f7b4006df70 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3149314228 0 0) 0x7f7b20003530 con 0x7f7b4006df70 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b401630c0 msgr2=0x7f7b4006f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b401630c0 0x7f7b4006f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 >> v1:172.21.15.116:6789/0 conn(0x7f7b400b3290 legacy=0x7f7b400ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4006ea30 con 0x7f7b4006df70 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f7b30001680 con 0x7f7b4006df70 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 >> v1:172.21.15.76:6789/0 conn(0x7f7b4006df70 legacy=0x7f7b401634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b401630c0 0x7f7b200039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b20004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b20004d50 0x7f7b200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4006e750 con 0x7f7b20003f70 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7b2000bab0 con 0x7f7b20004d50 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7b2000bcb0 con 0x7f7b401630c0 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b20004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3ed9d640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b20004d50 0x7f7b200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 -- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b401630c0 msgr2=0x7f7b200039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b401630c0 0x7f7b200039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 -- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b20004d50 msgr2=0x7f7b200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b20004d50 0x7f7b200051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 -- 172.21.15.76:0/2077912922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b20003530 con 0x7f7b20003f70 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b35d9b640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b20004810 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7b4009a000 tx=0x7f7b3001f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67b1a901e311c2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30007d00 con 0x7f7b20003f70 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b30005b00 con 0x7f7b20003f70 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3dd9b640 1 -- 172.21.15.76:0/2077912922 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b30007070 con 0x7f7b20003f70 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b44bf8640 1 -- 172.21.15.76:0/2077912922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 msgr2=0x7f7b20004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b44bf8640 1 --2- 172.21.15.76:0/2077912922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b20004810 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f7b4009a000 tx=0x7f7b3001f370 comp rx=0 tx=0).stop 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b3ed9d640 1 -- 172.21.15.76:0/2077912922 reap_dead start 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.903+0000 7f7b44bf8640 1 -- 172.21.15.76:0/2077912922 shutdown_connections 2024-09-19T04:16:47.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- 172.21.15.76:0/2077912922 >> 172.21.15.76:0/2077912922 conn(0x7f7b400b1f90 msgr2=0x7f7b400b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- 172.21.15.76:0/2077912922 shutdown_connections 2024-09-19T04:16:47.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- 172.21.15.76:0/2077912922 wait complete. 2024-09-19T04:16:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 Processor -- start 2024-09-19T04:16:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- start start 2024-09-19T04:16:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b4015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b40158fb0 0x7f7b40157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b40155150 0x7f7b401555d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b400ad910 con 0x7f7b20003f70 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7b4006eff0 con 0x7f7b40155150 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7b400ada90 con 0x7f7b40158fb0 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b4015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b4015b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44690/0 (socket says 172.21.15.76:44690) 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 -- 172.21.15.76:0/1356770922 learned_addr learned my addr 172.21.15.76:0/1356770922 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b35d9b640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b40158fb0 0x7f7b40157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 -- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b40158fb0 msgr2=0x7f7b40157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b40158fb0 0x7f7b40157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 -- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b40155150 msgr2=0x7f7b401555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3ed9d640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b40155150 0x7f7b401555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b40155150 0x7f7b401555d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 -- 172.21.15.76:0/1356770922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b401576d0 con 0x7f7b20003f70 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b3e59c640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b4015b7d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e850 tx=0x7f7b2800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b377fe640 1 -- 172.21.15.76:0/1356770922 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28017070 con 0x7f7b20003f70 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b377fe640 1 -- 172.21.15.76:0/1356770922 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b2800c8e0 con 0x7f7b20003f70 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b40154690 con 0x7f7b20003f70 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b377fe640 1 -- 172.21.15.76:0/1356770922 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2800ccd0 con 0x7f7b20003f70 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b4015c170 con 0x7f7b20003f70 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.907+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b400b3720 con 0x7f7b20003f70 2024-09-19T04:16:47.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.911+0000 7f7b377fe640 1 -- 172.21.15.76:0/1356770922 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7b2801a070 con 0x7f7b20003f70 2024-09-19T04:16:47.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.911+0000 7f7b377fe640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b240678c0 0x7f7b24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:47.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.911+0000 7f7b377fe640 1 -- 172.21.15.76:0/1356770922 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f7b28085bd0 con 0x7f7b20003f70 2024-09-19T04:16:47.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.911+0000 7f7b35d9b640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b240678c0 0x7f7b24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:47.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.911+0000 7f7b35d9b640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b240678c0 0x7f7b24069d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b300ae000 tx=0x7f7b4006e640 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:47.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:47.915+0000 7f7b377fe640 1 -- 172.21.15.76:0/1356770922 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7b28056610 con 0x7f7b20003f70 2024-09-19T04:16:48.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.123+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7b401edf90 con 0x7f7b20003f70 2024-09-19T04:16:48.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.123+0000 7f7b377fe640 1 -- 172.21.15.76:0/1356770922 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1560 (secure 0 0 0) 0x7f7b28055f10 con 0x7f7b20003f70 2024-09-19T04:16:48.128 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:48.128 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":227,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":83},{"state_name":"creating+peering","count":14}],"num_pgs":97,"num_pools":3,"num_objects":18,"data_bytes":592584,"bytes_used":241860608,"bytes_avail":772852252672,"bytes_total":773094113280,"inactive_pgs_ratio":0.14432989060878754,"write_bytes_sec":853,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":47,"btime":"2024-09-19T04:16:45:985234+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"a","status":"up:active","gid":6165}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-09-19T04:16:31.799314+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:16:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b240678c0 msgr2=0x7f7b24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b240678c0 0x7f7b24069d80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b300ae000 tx=0x7f7b4006e640 comp rx=0 tx=0).stop 2024-09-19T04:16:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 msgr2=0x7f7b4015b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b4015b7d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f7b2800e850 tx=0x7f7b2800ed10 comp rx=0 tx=0).stop 2024-09-19T04:16:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 shutdown_connections 2024-09-19T04:16:48.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b240678c0 0x7f7b24069d80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b40155150 0x7f7b401555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b40158fb0 0x7f7b40157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 --2- 172.21.15.76:0/1356770922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b20003f70 0x7f7b4015b7d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 >> 172.21.15.76:0/1356770922 conn(0x7f7b400b1f90 msgr2=0x7f7b401646a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 shutdown_connections 2024-09-19T04:16:48.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.127+0000 7f7b44bf8640 1 -- 172.21.15.76:0/1356770922 wait complete. 2024-09-19T04:16:48.145 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-19T04:16:48.146 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:16:48.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86e2bd640 1 Processor -- start 2024-09-19T04:16:48.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86e2bd640 1 -- start start 2024-09-19T04:16:48.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86e2bd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff86815e450 0x7ff86809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86e2bd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8680b40c0 0x7ff86815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86e2bd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8680b5c90 con 0x7ff86806bcc0 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86e2bd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff8680b4860 con 0x7ff86815e450 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86e2bd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff8680b4580 con 0x7ff8680b40c0 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff867fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff86815e450 0x7ff86809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff867fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff86815e450 0x7ff86809aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44706/0 (socket says 172.21.15.76:44706) 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff867fff640 1 -- 172.21.15.76:0/908457468 learned_addr learned my addr 172.21.15.76:0/908457468 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff86c85a640 1 --1- 172.21.15.76:0/908457468 >> v1:172.21.15.76:6789/0 conn(0x7ff86806bcc0 0x7ff868162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33330/0 (socket says 172.21.15.76:33330) 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff8677fe640 1 --2- 172.21.15.76:0/908457468 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8680b40c0 0x7ff86815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff867fff640 1 -- 172.21.15.76:0/908457468 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8680b40c0 msgr2=0x7ff86815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:48.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff867fff640 1 --2- 172.21.15.76:0/908457468 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8680b40c0 0x7ff86815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff867fff640 1 -- 172.21.15.76:0/908457468 >> v1:172.21.15.76:6789/0 conn(0x7ff86806bcc0 legacy=0x7ff868162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff867fff640 1 -- 172.21.15.76:0/908457468 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8680b5f10 con 0x7ff86815e450 2024-09-19T04:16:48.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff866ffd640 1 -- 172.21.15.76:0/908457468 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081073660 0 0) 0x7ff8600014a0 con 0x7ff86806bcc0 2024-09-19T04:16:48.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.247+0000 7ff8677fe640 1 --2- 172.21.15.76:0/908457468 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8680b40c0 0x7ff86815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff867fff640 1 --2- 172.21.15.76:0/908457468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86815e450 0x7ff86809aa00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff858009870 tx=0x7ff85801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f56ca0fd87df09b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff866ffd640 1 -- 172.21.15.76:0/908457468 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580b1070 con 0x7ff86815e450 2024-09-19T04:16:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff866ffd640 1 -- 172.21.15.76:0/908457468 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8580a8050 con 0x7ff86815e450 2024-09-19T04:16:48.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff866ffd640 1 -- 172.21.15.76:0/908457468 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580ac860 con 0x7ff86815e450 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- 172.21.15.76:0/908457468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86815e450 msgr2=0x7ff86809aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/908457468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86815e450 0x7ff86809aa00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff858009870 tx=0x7ff85801c4d0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- 172.21.15.76:0/908457468 shutdown_connections 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/908457468 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff8680b40c0 0x7ff86815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/908457468 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff86815e450 0x7ff86809aa00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- 172.21.15.76:0/908457468 >> 172.21.15.76:0/908457468 conn(0x7ff8680b6800 msgr2=0x7ff8680b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- 172.21.15.76:0/908457468 shutdown_connections 2024-09-19T04:16:48.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- 172.21.15.76:0/908457468 wait complete. 2024-09-19T04:16:48.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 Processor -- start 2024-09-19T04:16:48.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- start start 2024-09-19T04:16:48.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff86806bcc0 0x7ff86814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:48.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8680b40c0 0x7ff868154c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:48.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff86815e450 0x7ff8681551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:48.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff86815f9c0 con 0x7ff8680b40c0 2024-09-19T04:16:48.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff86815f840 con 0x7ff86806bcc0 2024-09-19T04:16:48.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86e2bd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff868162e90 con 0x7ff86815e450 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8680b40c0 0x7ff868154c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8680b40c0 0x7ff868154c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44712/0 (socket says 172.21.15.76:44712) 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 -- 172.21.15.76:0/65707590 learned_addr learned my addr 172.21.15.76:0/65707590 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff86c85a640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff86815e450 0x7ff8681551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff867fff640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff86806bcc0 0x7ff86814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 -- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff86815e450 msgr2=0x7ff8681551c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff86815e450 0x7ff8681551c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 -- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff86806bcc0 msgr2=0x7ff86814fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff86806bcc0 0x7ff86814fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.251+0000 7ff8677fe640 1 -- 172.21.15.76:0/65707590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8681503d0 con 0x7ff8680b40c0 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff8677fe640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8680b40c0 0x7ff868154c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c7c0 tx=0x7ff85000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff864ff9640 1 -- 172.21.15.76:0/65707590 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff850007a30 con 0x7ff8680b40c0 2024-09-19T04:16:48.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff864ff9640 1 -- 172.21.15.76:0/65707590 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff850007bd0 con 0x7ff8680b40c0 2024-09-19T04:16:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681528d0 con 0x7ff8680b40c0 2024-09-19T04:16:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff864ff9640 1 -- 172.21.15.76:0/65707590 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8500113f0 con 0x7ff8680b40c0 2024-09-19T04:16:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff868153330 con 0x7ff8680b40c0 2024-09-19T04:16:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff868153530 con 0x7ff8680b40c0 2024-09-19T04:16:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff864ff9640 1 -- 172.21.15.76:0/65707590 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff850011590 con 0x7ff8680b40c0 2024-09-19T04:16:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff864ff9640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8480678c0 0x7ff848069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff864ff9640 1 -- 172.21.15.76:0/65707590 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff850085f50 con 0x7ff8680b40c0 2024-09-19T04:16:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff867fff640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8480678c0 0x7ff848069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.255+0000 7ff867fff640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8480678c0 0x7ff848069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ae000 tx=0x7ff858008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.259+0000 7ff864ff9640 1 -- 172.21.15.76:0/65707590 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff868153530 con 0x7ff8680b40c0 2024-09-19T04:16:48.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.415+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff868151e50 con 0x7ff8480678c0 2024-09-19T04:16:48.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.423+0000 7ff864ff9640 1 -- 172.21.15.76:0/65707590 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+250862 (secure 0 0 0) 0x7ff868151e50 con 0x7ff8480678c0 2024-09-19T04:16:48.430 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:48.431 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:16:48.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8480678c0 msgr2=0x7ff848069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8480678c0 0x7ff848069d80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ae000 tx=0x7ff858008040 comp rx=0 tx=0).stop 2024-09-19T04:16:48.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8680b40c0 msgr2=0x7ff868154c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8680b40c0 0x7ff868154c80 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff85000c7c0 tx=0x7ff85000cc80 comp rx=0 tx=0).stop 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 shutdown_connections 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8480678c0 0x7ff848069d80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff86815e450 0x7ff8681551c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff8680b40c0 0x7ff868154c80 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 --2- 172.21.15.76:0/65707590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff86806bcc0 0x7ff86814fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 >> 172.21.15.76:0/65707590 conn(0x7ff8680b6800 msgr2=0x7ff8681602c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 shutdown_connections 2024-09-19T04:16:48.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:48.427+0000 7ff86e2bd640 1 -- 172.21.15.76:0/65707590 wait complete. 2024-09-19T04:16:48.446 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-19T04:16:47.803099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592584,"num_objects":18,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":193,"num_write_kb":2421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236192,"kb_used_data":7464,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738528,"statfs":{"total":773094113280,"available":772852252672,"internally_reserved":0,"allocated":7643136,"data_stored":4105886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2216,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001424"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011967+0000","last_change":"2024-09-19T04:16:42.969949+0000","last_active":"2024-09-19T04:16:44.011967+0000","last_peered":"2024-09-19T04:16:44.011967+0000","last_clean":"2024-09-19T04:16:44.011967+0000","last_became_active":"2024-09-19T04:16:42.969397+0000","last_became_peered":"2024-09-19T04:16:42.969397+0000","last_unstale":"2024-09-19T04:16:44.011967+0000","last_undegraded":"2024-09-19T04:16:44.011967+0000","last_fullsized":"2024-09-19T04:16:44.011967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:17.590298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012509+0000","last_change":"2024-09-19T04:16:42.966076+0000","last_active":"2024-09-19T04:16:44.012509+0000","last_peered":"2024-09-19T04:16:44.012509+0000","last_clean":"2024-09-19T04:16:44.012509+0000","last_became_active":"2024-09-19T04:16:42.965438+0000","last_became_peered":"2024-09-19T04:16:42.965438+0000","last_unstale":"2024-09-19T04:16:44.012509+0000","last_undegraded":"2024-09-19T04:16:44.012509+0000","last_fullsized":"2024-09-19T04:16:44.012509+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:23.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982715+0000","last_change":"2024-09-19T04:16:41.982715+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982715+0000","last_undegraded":"2024-09-19T04:16:41.982715+0000","last_fullsized":"2024-09-19T04:16:41.982715+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926347+0000","last_change":"2024-09-19T04:16:42.933474+0000","last_active":"2024-09-19T04:16:43.926347+0000","last_peered":"2024-09-19T04:16:43.926347+0000","last_clean":"2024-09-19T04:16:43.926347+0000","last_became_active":"2024-09-19T04:16:42.932683+0000","last_became_peered":"2024-09-19T04:16:42.932683+0000","last_unstale":"2024-09-19T04:16:43.926347+0000","last_undegraded":"2024-09-19T04:16:43.926347+0000","last_fullsized":"2024-09-19T04:16:43.926347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:32.241411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014594+0000","last_change":"2024-09-19T04:16:42.961247+0000","last_active":"2024-09-19T04:16:44.014594+0000","last_peered":"2024-09-19T04:16:44.014594+0000","last_clean":"2024-09-19T04:16:44.014594+0000","last_became_active":"2024-09-19T04:16:42.960613+0000","last_became_peered":"2024-09-19T04:16:42.960613+0000","last_unstale":"2024-09-19T04:16:44.014594+0000","last_undegraded":"2024-09-19T04:16:44.014594+0000","last_fullsized":"2024-09-19T04:16:44.014594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:35.834614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921497+0000","last_change":"2024-09-19T04:16:42.978941+0000","last_active":"2024-09-19T04:16:43.921497+0000","last_peered":"2024-09-19T04:16:43.921497+0000","last_clean":"2024-09-19T04:16:43.921497+0000","last_became_active":"2024-09-19T04:16:42.978592+0000","last_became_peered":"2024-09-19T04:16:42.978592+0000","last_unstale":"2024-09-19T04:16:43.921497+0000","last_undegraded":"2024-09-19T04:16:43.921497+0000","last_fullsized":"2024-09-19T04:16:43.921497+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:42.286458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984236+0000","last_change":"2024-09-19T04:16:41.984236+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984236+0000","last_undegraded":"2024-09-19T04:16:41.984236+0000","last_fullsized":"2024-09-19T04:16:41.984236+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924508+0000","last_change":"2024-09-19T04:16:42.964398+0000","last_active":"2024-09-19T04:16:43.924508+0000","last_peered":"2024-09-19T04:16:43.924508+0000","last_clean":"2024-09-19T04:16:43.924508+0000","last_became_active":"2024-09-19T04:16:42.962066+0000","last_became_peered":"2024-09-19T04:16:42.962066+0000","last_unstale":"2024-09-19T04:16:43.924508+0000","last_undegraded":"2024-09-19T04:16:43.924508+0000","last_fullsized":"2024-09-19T04:16:43.924508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:34.299847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984883+0000","last_change":"2024-09-19T04:16:41.984883+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984883+0000","last_undegraded":"2024-09-19T04:16:41.984883+0000","last_fullsized":"2024-09-19T04:16:41.984883+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984199+0000","last_change":"2024-09-19T04:16:41.984199+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984199+0000","last_undegraded":"2024-09-19T04:16:41.984199+0000","last_fullsized":"2024-09-19T04:16:41.984199+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925889+0000","last_change":"2024-09-19T04:16:42.969013+0000","last_active":"2024-09-19T04:16:43.925889+0000","last_peered":"2024-09-19T04:16:43.925889+0000","last_clean":"2024-09-19T04:16:43.925889+0000","last_became_active":"2024-09-19T04:16:42.968258+0000","last_became_peered":"2024-09-19T04:16:42.968258+0000","last_unstale":"2024-09-19T04:16:43.925889+0000","last_undegraded":"2024-09-19T04:16:43.925889+0000","last_fullsized":"2024-09-19T04:16:43.925889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.850829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926610+0000","last_change":"2024-09-19T04:16:42.962680+0000","last_active":"2024-09-19T04:16:43.926610+0000","last_peered":"2024-09-19T04:16:43.926610+0000","last_clean":"2024-09-19T04:16:43.926610+0000","last_became_active":"2024-09-19T04:16:42.962011+0000","last_became_peered":"2024-09-19T04:16:42.962011+0000","last_unstale":"2024-09-19T04:16:43.926610+0000","last_undegraded":"2024-09-19T04:16:43.926610+0000","last_fullsized":"2024-09-19T04:16:43.926610+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:06.443119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926507+0000","last_change":"2024-09-19T04:16:42.940463+0000","last_active":"2024-09-19T04:16:43.926507+0000","last_peered":"2024-09-19T04:16:43.926507+0000","last_clean":"2024-09-19T04:16:43.926507+0000","last_became_active":"2024-09-19T04:16:42.938689+0000","last_became_peered":"2024-09-19T04:16:42.938689+0000","last_unstale":"2024-09-19T04:16:43.926507+0000","last_undegraded":"2024-09-19T04:16:43.926507+0000","last_fullsized":"2024-09-19T04:16:43.926507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:36.099771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.983462+0000","last_change":"2024-09-19T04:16:41.983462+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.983462+0000","last_undegraded":"2024-09-19T04:16:41.983462+0000","last_fullsized":"2024-09-19T04:16:41.983462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982729+0000","last_change":"2024-09-19T04:16:41.982729+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982729+0000","last_undegraded":"2024-09-19T04:16:41.982729+0000","last_fullsized":"2024-09-19T04:16:41.982729+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.985026+0000","last_change":"2024-09-19T04:16:41.985026+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.985026+0000","last_undegraded":"2024-09-19T04:16:41.985026+0000","last_fullsized":"2024-09-19T04:16:41.985026+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012127+0000","last_change":"2024-09-19T04:16:42.968957+0000","last_active":"2024-09-19T04:16:44.012127+0000","last_peered":"2024-09-19T04:16:44.012127+0000","last_clean":"2024-09-19T04:16:44.012127+0000","last_became_active":"2024-09-19T04:16:42.968321+0000","last_became_peered":"2024-09-19T04:16:42.968321+0000","last_unstale":"2024-09-19T04:16:44.012127+0000","last_undegraded":"2024-09-19T04:16:44.012127+0000","last_fullsized":"2024-09-19T04:16:44.012127+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.053674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925849+0000","last_change":"2024-09-19T04:16:42.933099+0000","last_active":"2024-09-19T04:16:43.925849+0000","last_peered":"2024-09-19T04:16:43.925849+0000","last_clean":"2024-09-19T04:16:43.925849+0000","last_became_active":"2024-09-19T04:16:42.932385+0000","last_became_peered":"2024-09-19T04:16:42.932385+0000","last_unstale":"2024-09-19T04:16:43.925849+0000","last_undegraded":"2024-09-19T04:16:43.925849+0000","last_fullsized":"2024-09-19T04:16:43.925849+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:17:23.327782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012455+0000","last_change":"2024-09-19T04:16:42.951344+0000","last_active":"2024-09-19T04:16:44.012455+0000","last_peered":"2024-09-19T04:16:44.012455+0000","last_clean":"2024-09-19T04:16:44.012455+0000","last_became_active":"2024-09-19T04:16:42.950151+0000","last_became_peered":"2024-09-19T04:16:42.950151+0000","last_unstale":"2024-09-19T04:16:44.012455+0000","last_undegraded":"2024-09-19T04:16:44.012455+0000","last_fullsized":"2024-09-19T04:16:44.012455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:37.132305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013530+0000","last_change":"2024-09-19T04:16:42.949344+0000","last_active":"2024-09-19T04:16:44.013530+0000","last_peered":"2024-09-19T04:16:44.013530+0000","last_clean":"2024-09-19T04:16:44.013530+0000","last_became_active":"2024-09-19T04:16:42.948576+0000","last_became_peered":"2024-09-19T04:16:42.948576+0000","last_unstale":"2024-09-19T04:16:44.013530+0000","last_undegraded":"2024-09-19T04:16:44.013530+0000","last_fullsized":"2024-09-19T04:16:44.013530+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:56:11.638219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926906+0000","last_change":"2024-09-19T04:16:42.936037+0000","last_active":"2024-09-19T04:16:43.926906+0000","last_peered":"2024-09-19T04:16:43.926906+0000","last_clean":"2024-09-19T04:16:43.926906+0000","last_became_active":"2024-09-19T04:16:42.935328+0000","last_became_peered":"2024-09-19T04:16:42.935328+0000","last_unstale":"2024-09-19T04:16:43.926906+0000","last_undegraded":"2024-09-19T04:16:43.926906+0000","last_fullsized":"2024-09-19T04:16:43.926906+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:47.147563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012192+0000","last_change":"2024-09-19T04:16:42.976023+0000","last_active":"2024-09-19T04:16:44.012192+0000","last_peered":"2024-09-19T04:16:44.012192+0000","last_clean":"2024-09-19T04:16:44.012192+0000","last_became_active":"2024-09-19T04:16:42.975285+0000","last_became_peered":"2024-09-19T04:16:42.975285+0000","last_unstale":"2024-09-19T04:16:44.012192+0000","last_undegraded":"2024-09-19T04:16:44.012192+0000","last_fullsized":"2024-09-19T04:16:44.012192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:46.642698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011872+0000","last_change":"2024-09-19T04:16:42.978717+0000","last_active":"2024-09-19T04:16:44.011872+0000","last_peered":"2024-09-19T04:16:44.011872+0000","last_clean":"2024-09-19T04:16:44.011872+0000","last_became_active":"2024-09-19T04:16:42.978388+0000","last_became_peered":"2024-09-19T04:16:42.978388+0000","last_unstale":"2024-09-19T04:16:44.011872+0000","last_undegraded":"2024-09-19T04:16:44.011872+0000","last_fullsized":"2024-09-19T04:16:44.011872+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:38.008584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982552+0000","last_change":"2024-09-19T04:16:41.982552+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982552+0000","last_undegraded":"2024-09-19T04:16:41.982552+0000","last_fullsized":"2024-09-19T04:16:41.982552+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927393+0000","last_change":"2024-09-19T04:16:42.948182+0000","last_active":"2024-09-19T04:16:43.927393+0000","last_peered":"2024-09-19T04:16:43.927393+0000","last_clean":"2024-09-19T04:16:43.927393+0000","last_became_active":"2024-09-19T04:16:42.947543+0000","last_became_peered":"2024-09-19T04:16:42.947543+0000","last_unstale":"2024-09-19T04:16:43.927393+0000","last_undegraded":"2024-09-19T04:16:43.927393+0000","last_fullsized":"2024-09-19T04:16:43.927393+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:46:40.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926671+0000","last_change":"2024-09-19T04:16:42.950205+0000","last_active":"2024-09-19T04:16:43.926671+0000","last_peered":"2024-09-19T04:16:43.926671+0000","last_clean":"2024-09-19T04:16:43.926671+0000","last_became_active":"2024-09-19T04:16:42.949551+0000","last_became_peered":"2024-09-19T04:16:42.949551+0000","last_unstale":"2024-09-19T04:16:43.926671+0000","last_undegraded":"2024-09-19T04:16:43.926671+0000","last_fullsized":"2024-09-19T04:16:43.926671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:50.434667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926298+0000","last_change":"2024-09-19T04:16:42.952280+0000","last_active":"2024-09-19T04:16:43.926298+0000","last_peered":"2024-09-19T04:16:43.926298+0000","last_clean":"2024-09-19T04:16:43.926298+0000","last_became_active":"2024-09-19T04:16:42.951553+0000","last_became_peered":"2024-09-19T04:16:42.951553+0000","last_unstale":"2024-09-19T04:16:43.926298+0000","last_undegraded":"2024-09-19T04:16:43.926298+0000","last_fullsized":"2024-09-19T04:16:43.926298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:53.356430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011745+0000","last_change":"2024-09-19T04:16:42.964343+0000","last_active":"2024-09-19T04:16:44.011745+0000","last_peered":"2024-09-19T04:16:44.011745+0000","last_clean":"2024-09-19T04:16:44.011745+0000","last_became_active":"2024-09-19T04:16:42.952426+0000","last_became_peered":"2024-09-19T04:16:42.952426+0000","last_unstale":"2024-09-19T04:16:44.011745+0000","last_undegraded":"2024-09-19T04:16:44.011745+0000","last_fullsized":"2024-09-19T04:16:44.011745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:35.834614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.982680+0000","last_change":"2024-09-19T04:16:41.982680+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.982680+0000","last_undegraded":"2024-09-19T04:16:41.982680+0000","last_fullsized":"2024-09-19T04:16:41.982680+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924661+0000","last_change":"2024-09-19T04:16:42.963672+0000","last_active":"2024-09-19T04:16:43.924661+0000","last_peered":"2024-09-19T04:16:43.924661+0000","last_clean":"2024-09-19T04:16:43.924661+0000","last_became_active":"2024-09-19T04:16:42.963006+0000","last_became_peered":"2024-09-19T04:16:42.963006+0000","last_unstale":"2024-09-19T04:16:43.924661+0000","last_undegraded":"2024-09-19T04:16:43.924661+0000","last_fullsized":"2024-09-19T04:16:43.924661+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:25.077988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925635+0000","last_change":"2024-09-19T04:16:42.952470+0000","last_active":"2024-09-19T04:16:43.925635+0000","last_peered":"2024-09-19T04:16:43.925635+0000","last_clean":"2024-09-19T04:16:43.925635+0000","last_became_active":"2024-09-19T04:16:42.951827+0000","last_became_peered":"2024-09-19T04:16:42.951827+0000","last_unstale":"2024-09-19T04:16:43.925635+0000","last_undegraded":"2024-09-19T04:16:43.925635+0000","last_fullsized":"2024-09-19T04:16:43.925635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:17.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921099+0000","last_change":"2024-09-19T04:16:42.951873+0000","last_active":"2024-09-19T04:16:43.921099+0000","last_peered":"2024-09-19T04:16:43.921099+0000","last_clean":"2024-09-19T04:16:43.921099+0000","last_became_active":"2024-09-19T04:16:42.948318+0000","last_became_peered":"2024-09-19T04:16:42.948318+0000","last_unstale":"2024-09-19T04:16:43.921099+0000","last_undegraded":"2024-09-19T04:16:43.921099+0000","last_fullsized":"2024-09-19T04:16:43.921099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:34.150659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012653+0000","last_change":"2024-09-19T04:16:42.969731+0000","last_active":"2024-09-19T04:16:44.012653+0000","last_peered":"2024-09-19T04:16:44.012653+0000","last_clean":"2024-09-19T04:16:44.012653+0000","last_became_active":"2024-09-19T04:16:42.969114+0000","last_became_peered":"2024-09-19T04:16:42.969114+0000","last_unstale":"2024-09-19T04:16:44.012653+0000","last_undegraded":"2024-09-19T04:16:44.012653+0000","last_fullsized":"2024-09-19T04:16:44.012653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:52:54.454670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012196+0000","last_change":"2024-09-19T04:16:42.013739+0000","last_active":"2024-09-19T04:16:44.012196+0000","last_peered":"2024-09-19T04:16:44.012196+0000","last_clean":"2024-09-19T04:16:44.012196+0000","last_became_active":"2024-09-19T04:16:42.013362+0000","last_became_peered":"2024-09-19T04:16:42.013362+0000","last_unstale":"2024-09-19T04:16:44.012196+0000","last_undegraded":"2024-09-19T04:16:44.012196+0000","last_fullsized":"2024-09-19T04:16:44.012196+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:52.006040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926709+0000","last_change":"2024-09-19T04:16:42.950145+0000","last_active":"2024-09-19T04:16:43.926709+0000","last_peered":"2024-09-19T04:16:43.926709+0000","last_clean":"2024-09-19T04:16:43.926709+0000","last_became_active":"2024-09-19T04:16:42.949460+0000","last_became_peered":"2024-09-19T04:16:42.949460+0000","last_unstale":"2024-09-19T04:16:43.926709+0000","last_undegraded":"2024-09-19T04:16:43.926709+0000","last_fullsized":"2024-09-19T04:16:43.926709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922760+0000","last_change":"2024-09-19T04:16:42.018926+0000","last_active":"2024-09-19T04:16:43.922760+0000","last_peered":"2024-09-19T04:16:43.922760+0000","last_clean":"2024-09-19T04:16:43.922760+0000","last_became_active":"2024-09-19T04:16:42.018655+0000","last_became_peered":"2024-09-19T04:16:42.018655+0000","last_unstale":"2024-09-19T04:16:43.922760+0000","last_undegraded":"2024-09-19T04:16:43.922760+0000","last_fullsized":"2024-09-19T04:16:43.922760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:59.963896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922698+0000","last_change":"2024-09-19T04:16:42.957511+0000","last_active":"2024-09-19T04:16:43.922698+0000","last_peered":"2024-09-19T04:16:43.922698+0000","last_clean":"2024-09-19T04:16:43.922698+0000","last_became_active":"2024-09-19T04:16:42.955571+0000","last_became_peered":"2024-09-19T04:16:42.955571+0000","last_unstale":"2024-09-19T04:16:43.922698+0000","last_undegraded":"2024-09-19T04:16:43.922698+0000","last_fullsized":"2024-09-19T04:16:43.922698+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:46.642698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922269+0000","last_change":"2024-09-19T04:16:41.992501+0000","last_active":"2024-09-19T04:16:43.922269+0000","last_peered":"2024-09-19T04:16:43.922269+0000","last_clean":"2024-09-19T04:16:43.922269+0000","last_became_active":"2024-09-19T04:16:41.992135+0000","last_became_peered":"2024-09-19T04:16:41.992135+0000","last_unstale":"2024-09-19T04:16:43.922269+0000","last_undegraded":"2024-09-19T04:16:43.922269+0000","last_fullsized":"2024-09-19T04:16:43.922269+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:57.542066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.991338+0000","last_change":"2024-09-19T04:16:41.991338+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.991338+0000","last_undegraded":"2024-09-19T04:16:41.991338+0000","last_fullsized":"2024-09-19T04:16:41.991338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921963+0000","last_change":"2024-09-19T04:16:42.000668+0000","last_active":"2024-09-19T04:16:43.921963+0000","last_peered":"2024-09-19T04:16:43.921963+0000","last_clean":"2024-09-19T04:16:43.921963+0000","last_became_active":"2024-09-19T04:16:42.000249+0000","last_became_peered":"2024-09-19T04:16:42.000249+0000","last_unstale":"2024-09-19T04:16:43.921963+0000","last_undegraded":"2024-09-19T04:16:43.921963+0000","last_fullsized":"2024-09-19T04:16:43.921963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:24.073566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.984416+0000","last_change":"2024-09-19T04:16:41.984416+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.984416+0000","last_undegraded":"2024-09-19T04:16:41.984416+0000","last_fullsized":"2024-09-19T04:16:41.984416+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012976+0000","last_change":"2024-09-19T04:16:41.989580+0000","last_active":"2024-09-19T04:16:44.012976+0000","last_peered":"2024-09-19T04:16:44.012976+0000","last_clean":"2024-09-19T04:16:44.012976+0000","last_became_active":"2024-09-19T04:16:41.989078+0000","last_became_peered":"2024-09-19T04:16:41.989078+0000","last_unstale":"2024-09-19T04:16:44.012976+0000","last_undegraded":"2024-09-19T04:16:44.012976+0000","last_fullsized":"2024-09-19T04:16:44.012976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:35.095349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011237+0000","last_change":"2024-09-19T04:16:42.965094+0000","last_active":"2024-09-19T04:16:44.011237+0000","last_peered":"2024-09-19T04:16:44.011237+0000","last_clean":"2024-09-19T04:16:44.011237+0000","last_became_active":"2024-09-19T04:16:42.964362+0000","last_became_peered":"2024-09-19T04:16:42.964362+0000","last_unstale":"2024-09-19T04:16:44.011237+0000","last_undegraded":"2024-09-19T04:16:44.011237+0000","last_fullsized":"2024-09-19T04:16:44.011237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:15.592286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012243+0000","last_change":"2024-09-19T04:16:41.995699+0000","last_active":"2024-09-19T04:16:44.012243+0000","last_peered":"2024-09-19T04:16:44.012243+0000","last_clean":"2024-09-19T04:16:44.012243+0000","last_became_active":"2024-09-19T04:16:41.993440+0000","last_became_peered":"2024-09-19T04:16:41.993440+0000","last_unstale":"2024-09-19T04:16:44.012243+0000","last_undegraded":"2024-09-19T04:16:44.012243+0000","last_fullsized":"2024-09-19T04:16:44.012243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:46.143141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011973+0000","last_change":"2024-09-19T04:16:42.948605+0000","last_active":"2024-09-19T04:16:44.011973+0000","last_peered":"2024-09-19T04:16:44.011973+0000","last_clean":"2024-09-19T04:16:44.011973+0000","last_became_active":"2024-09-19T04:16:42.947298+0000","last_became_peered":"2024-09-19T04:16:42.947298+0000","last_unstale":"2024-09-19T04:16:44.011973+0000","last_undegraded":"2024-09-19T04:16:44.011973+0000","last_fullsized":"2024-09-19T04:16:44.011973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:35.021836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011964+0000","last_change":"2024-09-19T04:16:42.022337+0000","last_active":"2024-09-19T04:16:44.011964+0000","last_peered":"2024-09-19T04:16:44.011964+0000","last_clean":"2024-09-19T04:16:44.011964+0000","last_became_active":"2024-09-19T04:16:42.021791+0000","last_became_peered":"2024-09-19T04:16:42.021791+0000","last_unstale":"2024-09-19T04:16:44.011964+0000","last_undegraded":"2024-09-19T04:16:44.011964+0000","last_fullsized":"2024-09-19T04:16:44.011964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:57.542066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012810+0000","last_change":"2024-09-19T04:16:42.961422+0000","last_active":"2024-09-19T04:16:44.012810+0000","last_peered":"2024-09-19T04:16:44.012810+0000","last_clean":"2024-09-19T04:16:44.012810+0000","last_became_active":"2024-09-19T04:16:42.960600+0000","last_became_peered":"2024-09-19T04:16:42.960600+0000","last_unstale":"2024-09-19T04:16:44.012810+0000","last_undegraded":"2024-09-19T04:16:44.012810+0000","last_fullsized":"2024-09-19T04:16:44.012810+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:18.223464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011820+0000","last_change":"2024-09-19T04:16:42.019556+0000","last_active":"2024-09-19T04:16:44.011820+0000","last_peered":"2024-09-19T04:16:44.011820+0000","last_clean":"2024-09-19T04:16:44.011820+0000","last_became_active":"2024-09-19T04:16:42.018870+0000","last_became_peered":"2024-09-19T04:16:42.018870+0000","last_unstale":"2024-09-19T04:16:44.011820+0000","last_undegraded":"2024-09-19T04:16:44.011820+0000","last_fullsized":"2024-09-19T04:16:44.011820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:56.469075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011719+0000","last_change":"2024-09-19T04:16:42.964622+0000","last_active":"2024-09-19T04:16:44.011719+0000","last_peered":"2024-09-19T04:16:44.011719+0000","last_clean":"2024-09-19T04:16:44.011719+0000","last_became_active":"2024-09-19T04:16:42.963938+0000","last_became_peered":"2024-09-19T04:16:42.963938+0000","last_unstale":"2024-09-19T04:16:44.011719+0000","last_undegraded":"2024-09-19T04:16:44.011719+0000","last_fullsized":"2024-09-19T04:16:44.011719+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:02.598595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028493+0000","last_change":"2024-09-19T04:16:42.026415+0000","last_active":"2024-09-19T04:16:42.028493+0000","last_peered":"2024-09-19T04:16:42.028493+0000","last_clean":"2024-09-19T04:16:42.028493+0000","last_became_active":"2024-09-19T04:16:42.025740+0000","last_became_peered":"2024-09-19T04:16:42.025740+0000","last_unstale":"2024-09-19T04:16:42.028493+0000","last_undegraded":"2024-09-19T04:16:42.028493+0000","last_fullsized":"2024-09-19T04:16:42.028493+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:36.325205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014048+0000","last_change":"2024-09-19T04:16:42.949543+0000","last_active":"2024-09-19T04:16:44.014048+0000","last_peered":"2024-09-19T04:16:44.014048+0000","last_clean":"2024-09-19T04:16:44.014048+0000","last_became_active":"2024-09-19T04:16:42.948796+0000","last_became_peered":"2024-09-19T04:16:42.948796+0000","last_unstale":"2024-09-19T04:16:44.014048+0000","last_undegraded":"2024-09-19T04:16:44.014048+0000","last_fullsized":"2024-09-19T04:16:44.014048+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:46.277558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927666+0000","last_change":"2024-09-19T04:16:42.001820+0000","last_active":"2024-09-19T04:16:43.927666+0000","last_peered":"2024-09-19T04:16:43.927666+0000","last_clean":"2024-09-19T04:16:43.927666+0000","last_became_active":"2024-09-19T04:16:42.001141+0000","last_became_peered":"2024-09-19T04:16:42.001141+0000","last_unstale":"2024-09-19T04:16:43.927666+0000","last_undegraded":"2024-09-19T04:16:43.927666+0000","last_fullsized":"2024-09-19T04:16:43.927666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:12.519867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012651+0000","last_change":"2024-09-19T04:16:42.954665+0000","last_active":"2024-09-19T04:16:44.012651+0000","last_peered":"2024-09-19T04:16:44.012651+0000","last_clean":"2024-09-19T04:16:44.012651+0000","last_became_active":"2024-09-19T04:16:42.953885+0000","last_became_peered":"2024-09-19T04:16:42.953885+0000","last_unstale":"2024-09-19T04:16:44.012651+0000","last_undegraded":"2024-09-19T04:16:44.012651+0000","last_fullsized":"2024-09-19T04:16:44.012651+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:09.075169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.025625+0000","last_change":"2024-09-19T04:16:42.018953+0000","last_active":"2024-09-19T04:16:42.025625+0000","last_peered":"2024-09-19T04:16:42.025625+0000","last_clean":"2024-09-19T04:16:42.025625+0000","last_became_active":"2024-09-19T04:16:42.018215+0000","last_became_peered":"2024-09-19T04:16:42.018215+0000","last_unstale":"2024-09-19T04:16:42.025625+0000","last_undegraded":"2024-09-19T04:16:42.025625+0000","last_fullsized":"2024-09-19T04:16:42.025625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:09.582870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.928208+0000","last_change":"2024-09-19T04:16:42.951952+0000","last_active":"2024-09-19T04:16:43.928208+0000","last_peered":"2024-09-19T04:16:43.928208+0000","last_clean":"2024-09-19T04:16:43.928208+0000","last_became_active":"2024-09-19T04:16:42.951251+0000","last_became_peered":"2024-09-19T04:16:42.951251+0000","last_unstale":"2024-09-19T04:16:43.928208+0000","last_undegraded":"2024-09-19T04:16:43.928208+0000","last_fullsized":"2024-09-19T04:16:43.928208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:35.399902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011835+0000","last_change":"2024-09-19T04:16:42.006482+0000","last_active":"2024-09-19T04:16:44.011835+0000","last_peered":"2024-09-19T04:16:44.011835+0000","last_clean":"2024-09-19T04:16:44.011835+0000","last_became_active":"2024-09-19T04:16:42.006111+0000","last_became_peered":"2024-09-19T04:16:42.006111+0000","last_unstale":"2024-09-19T04:16:44.011835+0000","last_undegraded":"2024-09-19T04:16:44.011835+0000","last_fullsized":"2024-09-19T04:16:44.011835+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:32.846407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924834+0000","last_change":"2024-09-19T04:16:42.974772+0000","last_active":"2024-09-19T04:16:43.924834+0000","last_peered":"2024-09-19T04:16:43.924834+0000","last_clean":"2024-09-19T04:16:43.924834+0000","last_became_active":"2024-09-19T04:16:42.974128+0000","last_became_peered":"2024-09-19T04:16:42.974128+0000","last_unstale":"2024-09-19T04:16:43.924834+0000","last_undegraded":"2024-09-19T04:16:43.924834+0000","last_fullsized":"2024-09-19T04:16:43.924834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:06.209292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028432+0000","last_change":"2024-09-19T04:16:42.024717+0000","last_active":"2024-09-19T04:16:42.028432+0000","last_peered":"2024-09-19T04:16:42.028432+0000","last_clean":"2024-09-19T04:16:42.028432+0000","last_became_active":"2024-09-19T04:16:42.023979+0000","last_became_peered":"2024-09-19T04:16:42.023979+0000","last_unstale":"2024-09-19T04:16:42.028432+0000","last_undegraded":"2024-09-19T04:16:42.028432+0000","last_fullsized":"2024-09-19T04:16:42.028432+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:14.632041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924607+0000","last_change":"2024-09-19T04:16:42.929172+0000","last_active":"2024-09-19T04:16:43.924607+0000","last_peered":"2024-09-19T04:16:43.924607+0000","last_clean":"2024-09-19T04:16:43.924607+0000","last_became_active":"2024-09-19T04:16:42.928397+0000","last_became_peered":"2024-09-19T04:16:42.928397+0000","last_unstale":"2024-09-19T04:16:43.924607+0000","last_undegraded":"2024-09-19T04:16:43.924607+0000","last_fullsized":"2024-09-19T04:16:43.924607+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:00.968318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924958+0000","last_change":"2024-09-19T04:16:42.002996+0000","last_active":"2024-09-19T04:16:43.924958+0000","last_peered":"2024-09-19T04:16:43.924958+0000","last_clean":"2024-09-19T04:16:43.924958+0000","last_became_active":"2024-09-19T04:16:42.001542+0000","last_became_peered":"2024-09-19T04:16:42.001542+0000","last_unstale":"2024-09-19T04:16:43.924958+0000","last_undegraded":"2024-09-19T04:16:43.924958+0000","last_fullsized":"2024-09-19T04:16:43.924958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:14.587864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014348+0000","last_change":"2024-09-19T04:16:42.952737+0000","last_active":"2024-09-19T04:16:44.014348+0000","last_peered":"2024-09-19T04:16:44.014348+0000","last_clean":"2024-09-19T04:16:44.014348+0000","last_became_active":"2024-09-19T04:16:42.951143+0000","last_became_peered":"2024-09-19T04:16:42.951143+0000","last_unstale":"2024-09-19T04:16:44.014348+0000","last_undegraded":"2024-09-19T04:16:44.014348+0000","last_fullsized":"2024-09-19T04:16:44.014348+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:02.598595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.024770+0000","last_change":"2024-09-19T04:16:42.018769+0000","last_active":"2024-09-19T04:16:42.024770+0000","last_peered":"2024-09-19T04:16:42.024770+0000","last_clean":"2024-09-19T04:16:42.024770+0000","last_became_active":"2024-09-19T04:16:42.017955+0000","last_became_peered":"2024-09-19T04:16:42.017955+0000","last_unstale":"2024-09-19T04:16:42.024770+0000","last_undegraded":"2024-09-19T04:16:42.024770+0000","last_fullsized":"2024-09-19T04:16:42.024770+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:40:19.225233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011455+0000","last_change":"2024-09-19T04:16:42.969447+0000","last_active":"2024-09-19T04:16:44.011455+0000","last_peered":"2024-09-19T04:16:44.011455+0000","last_clean":"2024-09-19T04:16:44.011455+0000","last_became_active":"2024-09-19T04:16:42.968597+0000","last_became_peered":"2024-09-19T04:16:42.968597+0000","last_unstale":"2024-09-19T04:16:44.011455+0000","last_undegraded":"2024-09-19T04:16:44.011455+0000","last_fullsized":"2024-09-19T04:16:44.011455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:46.028679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925185+0000","last_change":"2024-09-19T04:16:42.011251+0000","last_active":"2024-09-19T04:16:43.925185+0000","last_peered":"2024-09-19T04:16:43.925185+0000","last_clean":"2024-09-19T04:16:43.925185+0000","last_became_active":"2024-09-19T04:16:42.010096+0000","last_became_peered":"2024-09-19T04:16:42.010096+0000","last_unstale":"2024-09-19T04:16:43.925185+0000","last_undegraded":"2024-09-19T04:16:43.925185+0000","last_fullsized":"2024-09-19T04:16:43.925185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:35.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.987287+0000","last_change":"2024-09-19T04:16:41.987287+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.987287+0000","last_undegraded":"2024-09-19T04:16:41.987287+0000","last_fullsized":"2024-09-19T04:16:41.987287+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927569+0000","last_change":"2024-09-19T04:16:42.000089+0000","last_active":"2024-09-19T04:16:43.927569+0000","last_peered":"2024-09-19T04:16:43.927569+0000","last_clean":"2024-09-19T04:16:43.927569+0000","last_became_active":"2024-09-19T04:16:41.999376+0000","last_became_peered":"2024-09-19T04:16:41.999376+0000","last_unstale":"2024-09-19T04:16:43.927569+0000","last_undegraded":"2024-09-19T04:16:43.927569+0000","last_fullsized":"2024-09-19T04:16:43.927569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:45.024257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012859+0000","last_change":"2024-09-19T04:16:42.963194+0000","last_active":"2024-09-19T04:16:44.012859+0000","last_peered":"2024-09-19T04:16:44.012859+0000","last_clean":"2024-09-19T04:16:44.012859+0000","last_became_active":"2024-09-19T04:16:42.962439+0000","last_became_peered":"2024-09-19T04:16:42.962439+0000","last_unstale":"2024-09-19T04:16:44.012859+0000","last_undegraded":"2024-09-19T04:16:44.012859+0000","last_fullsized":"2024-09-19T04:16:44.012859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:15.592286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.027534+0000","last_change":"2024-09-19T04:16:42.023483+0000","last_active":"2024-09-19T04:16:42.027534+0000","last_peered":"2024-09-19T04:16:42.027534+0000","last_clean":"2024-09-19T04:16:42.027534+0000","last_became_active":"2024-09-19T04:16:42.022787+0000","last_became_peered":"2024-09-19T04:16:42.022787+0000","last_unstale":"2024-09-19T04:16:42.027534+0000","last_undegraded":"2024-09-19T04:16:42.027534+0000","last_fullsized":"2024-09-19T04:16:42.027534+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:54.902392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927434+0000","last_change":"2024-09-19T04:16:42.943810+0000","last_active":"2024-09-19T04:16:43.927434+0000","last_peered":"2024-09-19T04:16:43.927434+0000","last_clean":"2024-09-19T04:16:43.927434+0000","last_became_active":"2024-09-19T04:16:42.943076+0000","last_became_peered":"2024-09-19T04:16:42.943076+0000","last_unstale":"2024-09-19T04:16:43.927434+0000","last_undegraded":"2024-09-19T04:16:43.927434+0000","last_fullsized":"2024-09-19T04:16:43.927434+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:35.021836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.028461+0000","last_change":"2024-09-19T04:16:42.026255+0000","last_active":"2024-09-19T04:16:42.028461+0000","last_peered":"2024-09-19T04:16:42.028461+0000","last_clean":"2024-09-19T04:16:42.028461+0000","last_became_active":"2024-09-19T04:16:42.025430+0000","last_became_peered":"2024-09-19T04:16:42.025430+0000","last_unstale":"2024-09-19T04:16:42.028461+0000","last_undegraded":"2024-09-19T04:16:42.028461+0000","last_fullsized":"2024-09-19T04:16:42.028461+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:11:18.263858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012920+0000","last_change":"2024-09-19T04:16:42.958174+0000","last_active":"2024-09-19T04:16:44.012920+0000","last_peered":"2024-09-19T04:16:44.012920+0000","last_clean":"2024-09-19T04:16:44.012920+0000","last_became_active":"2024-09-19T04:16:42.957489+0000","last_became_peered":"2024-09-19T04:16:42.957489+0000","last_unstale":"2024-09-19T04:16:44.012920+0000","last_undegraded":"2024-09-19T04:16:44.012920+0000","last_fullsized":"2024-09-19T04:16:44.012920+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:34:17.899656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013703+0000","last_change":"2024-09-19T04:16:41.998882+0000","last_active":"2024-09-19T04:16:44.013703+0000","last_peered":"2024-09-19T04:16:44.013703+0000","last_clean":"2024-09-19T04:16:44.013703+0000","last_became_active":"2024-09-19T04:16:41.998612+0000","last_became_peered":"2024-09-19T04:16:41.998612+0000","last_unstale":"2024-09-19T04:16:44.013703+0000","last_undegraded":"2024-09-19T04:16:44.013703+0000","last_fullsized":"2024-09-19T04:16:44.013703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:49.430245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921049+0000","last_change":"2024-09-19T04:16:42.954340+0000","last_active":"2024-09-19T04:16:43.921049+0000","last_peered":"2024-09-19T04:16:43.921049+0000","last_clean":"2024-09-19T04:16:43.921049+0000","last_became_active":"2024-09-19T04:16:42.948861+0000","last_became_peered":"2024-09-19T04:16:42.948861+0000","last_unstale":"2024-09-19T04:16:43.921049+0000","last_undegraded":"2024-09-19T04:16:43.921049+0000","last_fullsized":"2024-09-19T04:16:43.921049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:09:10.896065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921725+0000","last_change":"2024-09-19T04:16:42.015741+0000","last_active":"2024-09-19T04:16:43.921725+0000","last_peered":"2024-09-19T04:16:43.921725+0000","last_clean":"2024-09-19T04:16:43.921725+0000","last_became_active":"2024-09-19T04:16:42.015285+0000","last_became_peered":"2024-09-19T04:16:42.015285+0000","last_unstale":"2024-09-19T04:16:43.921725+0000","last_undegraded":"2024-09-19T04:16:43.921725+0000","last_fullsized":"2024-09-19T04:16:43.921725+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:05.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924888+0000","last_change":"2024-09-19T04:16:42.943410+0000","last_active":"2024-09-19T04:16:43.924888+0000","last_peered":"2024-09-19T04:16:43.924888+0000","last_clean":"2024-09-19T04:16:43.924888+0000","last_became_active":"2024-09-19T04:16:42.942660+0000","last_became_peered":"2024-09-19T04:16:42.942660+0000","last_unstale":"2024-09-19T04:16:43.924888+0000","last_undegraded":"2024-09-19T04:16:43.924888+0000","last_fullsized":"2024-09-19T04:16:43.924888+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:58.546488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925054+0000","last_change":"2024-09-19T04:16:42.002264+0000","last_active":"2024-09-19T04:16:43.925054+0000","last_peered":"2024-09-19T04:16:43.925054+0000","last_clean":"2024-09-19T04:16:43.925054+0000","last_became_active":"2024-09-19T04:16:42.000593+0000","last_became_peered":"2024-09-19T04:16:42.000593+0000","last_unstale":"2024-09-19T04:16:43.925054+0000","last_undegraded":"2024-09-19T04:16:43.925054+0000","last_fullsized":"2024-09-19T04:16:43.925054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:40.593088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012399+0000","last_change":"2024-09-19T04:16:42.947997+0000","last_active":"2024-09-19T04:16:44.012399+0000","last_peered":"2024-09-19T04:16:44.012399+0000","last_clean":"2024-09-19T04:16:44.012399+0000","last_became_active":"2024-09-19T04:16:42.946350+0000","last_became_peered":"2024-09-19T04:16:42.946350+0000","last_unstale":"2024-09-19T04:16:44.012399+0000","last_undegraded":"2024-09-19T04:16:44.012399+0000","last_fullsized":"2024-09-19T04:16:44.012399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:35.399902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921195+0000","last_change":"2024-09-19T04:16:42.010194+0000","last_active":"2024-09-19T04:16:43.921195+0000","last_peered":"2024-09-19T04:16:43.921195+0000","last_clean":"2024-09-19T04:16:43.921195+0000","last_became_active":"2024-09-19T04:16:42.009783+0000","last_became_peered":"2024-09-19T04:16:42.009783+0000","last_unstale":"2024-09-19T04:16:43.921195+0000","last_undegraded":"2024-09-19T04:16:43.921195+0000","last_fullsized":"2024-09-19T04:16:43.921195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:33.295425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922031+0000","last_change":"2024-09-19T04:16:42.936445+0000","last_active":"2024-09-19T04:16:43.922031+0000","last_peered":"2024-09-19T04:16:43.922031+0000","last_clean":"2024-09-19T04:16:43.922031+0000","last_became_active":"2024-09-19T04:16:42.935756+0000","last_became_peered":"2024-09-19T04:16:42.935756+0000","last_unstale":"2024-09-19T04:16:43.922031+0000","last_undegraded":"2024-09-19T04:16:43.922031+0000","last_fullsized":"2024-09-19T04:16:43.922031+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:38.008584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927853+0000","last_change":"2024-09-19T04:16:42.010812+0000","last_active":"2024-09-19T04:16:43.927853+0000","last_peered":"2024-09-19T04:16:43.927853+0000","last_clean":"2024-09-19T04:16:43.927853+0000","last_became_active":"2024-09-19T04:16:42.010051+0000","last_became_peered":"2024-09-19T04:16:42.010051+0000","last_unstale":"2024-09-19T04:16:43.927853+0000","last_undegraded":"2024-09-19T04:16:43.927853+0000","last_fullsized":"2024-09-19T04:16:43.927853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:01.594173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986784+0000","last_change":"2024-09-19T04:16:41.986784+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986784+0000","last_undegraded":"2024-09-19T04:16:41.986784+0000","last_fullsized":"2024-09-19T04:16:41.986784+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927518+0000","last_change":"2024-09-19T04:16:41.995074+0000","last_active":"2024-09-19T04:16:43.927518+0000","last_peered":"2024-09-19T04:16:43.927518+0000","last_clean":"2024-09-19T04:16:43.927518+0000","last_became_active":"2024-09-19T04:16:41.994321+0000","last_became_peered":"2024-09-19T04:16:41.994321+0000","last_unstale":"2024-09-19T04:16:43.927518+0000","last_undegraded":"2024-09-19T04:16:43.927518+0000","last_fullsized":"2024-09-19T04:16:43.927518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:34:16.895234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927038+0000","last_change":"2024-09-19T04:16:42.929147+0000","last_active":"2024-09-19T04:16:43.927038+0000","last_peered":"2024-09-19T04:16:43.927038+0000","last_clean":"2024-09-19T04:16:43.927038+0000","last_became_active":"2024-09-19T04:16:42.928337+0000","last_became_peered":"2024-09-19T04:16:42.928337+0000","last_unstale":"2024-09-19T04:16:43.927038+0000","last_undegraded":"2024-09-19T04:16:43.927038+0000","last_fullsized":"2024-09-19T04:16:43.927038+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:17.836899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922384+0000","last_change":"2024-09-19T04:16:41.991513+0000","last_active":"2024-09-19T04:16:43.922384+0000","last_peered":"2024-09-19T04:16:43.922384+0000","last_clean":"2024-09-19T04:16:43.922384+0000","last_became_active":"2024-09-19T04:16:41.990986+0000","last_became_peered":"2024-09-19T04:16:41.990986+0000","last_unstale":"2024-09-19T04:16:43.922384+0000","last_undegraded":"2024-09-19T04:16:43.922384+0000","last_fullsized":"2024-09-19T04:16:43.922384+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:35.095349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013311+0000","last_change":"2024-09-19T04:16:42.958068+0000","last_active":"2024-09-19T04:16:44.013311+0000","last_peered":"2024-09-19T04:16:44.013311+0000","last_clean":"2024-09-19T04:16:44.013311+0000","last_became_active":"2024-09-19T04:16:42.957349+0000","last_became_peered":"2024-09-19T04:16:42.957349+0000","last_unstale":"2024-09-19T04:16:44.013311+0000","last_undegraded":"2024-09-19T04:16:44.013311+0000","last_fullsized":"2024-09-19T04:16:44.013311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:41.597510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011981+0000","last_change":"2024-09-19T04:16:42.003466+0000","last_active":"2024-09-19T04:16:44.011981+0000","last_peered":"2024-09-19T04:16:44.011981+0000","last_clean":"2024-09-19T04:16:44.011981+0000","last_became_active":"2024-09-19T04:16:42.003135+0000","last_became_peered":"2024-09-19T04:16:42.003135+0000","last_unstale":"2024-09-19T04:16:44.011981+0000","last_undegraded":"2024-09-19T04:16:44.011981+0000","last_fullsized":"2024-09-19T04:16:44.011981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:17:22.323360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011818+0000","last_change":"2024-09-19T04:16:42.971396+0000","last_active":"2024-09-19T04:16:44.011818+0000","last_peered":"2024-09-19T04:16:44.011818+0000","last_clean":"2024-09-19T04:16:44.011818+0000","last_became_active":"2024-09-19T04:16:42.970709+0000","last_became_peered":"2024-09-19T04:16:42.970709+0000","last_unstale":"2024-09-19T04:16:44.011818+0000","last_undegraded":"2024-09-19T04:16:44.011818+0000","last_fullsized":"2024-09-19T04:16:44.011818+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.026933+0000","last_change":"2024-09-19T04:16:42.019204+0000","last_active":"2024-09-19T04:16:42.026933+0000","last_peered":"2024-09-19T04:16:42.026933+0000","last_clean":"2024-09-19T04:16:42.026933+0000","last_became_active":"2024-09-19T04:16:42.018134+0000","last_became_peered":"2024-09-19T04:16:42.018134+0000","last_unstale":"2024-09-19T04:16:42.026933+0000","last_undegraded":"2024-09-19T04:16:42.026933+0000","last_fullsized":"2024-09-19T04:16:42.026933+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:22:41.461855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926029+0000","last_change":"2024-09-19T04:16:42.975446+0000","last_active":"2024-09-19T04:16:43.926029+0000","last_peered":"2024-09-19T04:16:43.926029+0000","last_clean":"2024-09-19T04:16:43.926029+0000","last_became_active":"2024-09-19T04:16:42.974747+0000","last_became_peered":"2024-09-19T04:16:42.974747+0000","last_unstale":"2024-09-19T04:16:43.926029+0000","last_undegraded":"2024-09-19T04:16:43.926029+0000","last_fullsized":"2024-09-19T04:16:43.926029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:53.010462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012390+0000","last_change":"2024-09-19T04:16:41.994558+0000","last_active":"2024-09-19T04:16:44.012390+0000","last_peered":"2024-09-19T04:16:44.012390+0000","last_clean":"2024-09-19T04:16:44.012390+0000","last_became_active":"2024-09-19T04:16:41.993965+0000","last_became_peered":"2024-09-19T04:16:41.993965+0000","last_unstale":"2024-09-19T04:16:44.012390+0000","last_undegraded":"2024-09-19T04:16:44.012390+0000","last_fullsized":"2024-09-19T04:16:44.012390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:16.832477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011286+0000","last_change":"2024-09-19T04:16:42.969221+0000","last_active":"2024-09-19T04:16:44.011286+0000","last_peered":"2024-09-19T04:16:44.011286+0000","last_clean":"2024-09-19T04:16:44.011286+0000","last_became_active":"2024-09-19T04:16:42.968453+0000","last_became_peered":"2024-09-19T04:16:42.968453+0000","last_unstale":"2024-09-19T04:16:44.011286+0000","last_undegraded":"2024-09-19T04:16:44.011286+0000","last_fullsized":"2024-09-19T04:16:44.011286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:41.597510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012001+0000","last_change":"2024-09-19T04:16:42.022760+0000","last_active":"2024-09-19T04:16:44.012001+0000","last_peered":"2024-09-19T04:16:44.012001+0000","last_clean":"2024-09-19T04:16:44.012001+0000","last_became_active":"2024-09-19T04:16:42.022030+0000","last_became_peered":"2024-09-19T04:16:42.022030+0000","last_unstale":"2024-09-19T04:16:44.012001+0000","last_undegraded":"2024-09-19T04:16:44.012001+0000","last_fullsized":"2024-09-19T04:16:44.012001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:05.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":65,"state":"creating+peering","last_fresh":"2024-09-19T04:16:41.986891+0000","last_change":"2024-09-19T04:16:41.986891+0000","last_active":"2024-09-19T04:16:41.962650+0000","last_peered":"2024-09-19T04:16:41.962650+0000","last_clean":"2024-09-19T04:16:41.962650+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-19T04:16:41.986891+0000","last_undegraded":"2024-09-19T04:16:41.986891+0000","last_fullsized":"2024-09-19T04:16:41.986891+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011086+0000","last_change":"2024-09-19T04:16:42.022541+0000","last_active":"2024-09-19T04:16:44.011086+0000","last_peered":"2024-09-19T04:16:44.011086+0000","last_clean":"2024-09-19T04:16:44.011086+0000","last_became_active":"2024-09-19T04:16:42.022021+0000","last_became_peered":"2024-09-19T04:16:42.022021+0000","last_unstale":"2024-09-19T04:16:44.011086+0000","last_undegraded":"2024-09-19T04:16:44.011086+0000","last_fullsized":"2024-09-19T04:16:44.011086+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:16.089701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013503+0000","last_change":"2024-09-19T04:16:42.970273+0000","last_active":"2024-09-19T04:16:44.013503+0000","last_peered":"2024-09-19T04:16:44.013503+0000","last_clean":"2024-09-19T04:16:44.013503+0000","last_became_active":"2024-09-19T04:16:42.969727+0000","last_became_peered":"2024-09-19T04:16:42.969727+0000","last_unstale":"2024-09-19T04:16:44.013503+0000","last_undegraded":"2024-09-19T04:16:44.013503+0000","last_fullsized":"2024-09-19T04:16:44.013503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:46.028679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012001+0000","last_change":"2024-09-19T04:16:41.988994+0000","last_active":"2024-09-19T04:16:44.012001+0000","last_peered":"2024-09-19T04:16:44.012001+0000","last_clean":"2024-09-19T04:16:44.012001+0000","last_became_active":"2024-09-19T04:16:41.988755+0000","last_became_peered":"2024-09-19T04:16:41.988755+0000","last_unstale":"2024-09-19T04:16:44.012001+0000","last_undegraded":"2024-09-19T04:16:44.012001+0000","last_fullsized":"2024-09-19T04:16:44.012001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:24.073566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-19T04:16:42.009326+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:16:42.009326+0000","last_peered":"2024-09-19T04:16:42.009326+0000","last_clean":"2024-09-19T04:16:42.009326+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:16:42.009326+0000","last_undegraded":"2024-09-19T04:16:42.009326+0000","last_fullsized":"2024-09-19T04:16:42.009326+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2216,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":31,"ondisk_log_size":31,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542189,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1376256,"data_stored":945112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":368637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":368625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":370215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":368603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542190,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31952,"kb_used_data":764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339888,"statfs":{"total":96636764160,"available":96604045312,"internally_reserved":0,"allocated":782336,"data_stored":354744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":835584,"data_stored":370979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1400832,"data_stored":958971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:16:48.870 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:16:48.867+0000 7f9c43701640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:16:51.451 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:16:51.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.575+0000 7fd0b34a6640 1 Processor -- start 2024-09-19T04:16:51.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.575+0000 7fd0b34a6640 1 -- start start 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b34a6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0ac15bec0 0x7fd0ac0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b34a6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac0af470 0x7fd0ac06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b34a6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0ac0b3290 con 0x7fd0ac15c2a0 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b34a6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd0ac0b3570 con 0x7fd0ac15bec0 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b34a6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ac0b3850 con 0x7fd0ac0af470 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1242640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0ac15bec0 0x7fd0ac0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b0a41640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd0ac15c2a0 0x7fd0ac0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33338/0 (socket says 172.21.15.76:33338) 2024-09-19T04:16:51.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1242640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd0ac15bec0 0x7fd0ac0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44726/0 (socket says 172.21.15.76:44726) 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b0a41640 1 -- 172.21.15.76:0/495368243 learned_addr learned my addr 172.21.15.76:0/495368243 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1a43640 1 --2- 172.21.15.76:0/495368243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac0af470 0x7fd0ac06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1242640 1 -- 172.21.15.76:0/495368243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac0af470 msgr2=0x7fd0ac06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1242640 1 --2- 172.21.15.76:0/495368243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac0af470 0x7fd0ac06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1242640 1 -- 172.21.15.76:0/495368243 >> v1:172.21.15.76:6789/0 conn(0x7fd0ac15c2a0 legacy=0x7fd0ac0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0abfff640 1 -- 172.21.15.76:0/495368243 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1961441957 0 0) 0x7fd0ac0b3290 con 0x7fd0ac15c2a0 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1242640 1 -- 172.21.15.76:0/495368243 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ac0b3b30 con 0x7fd0ac15bec0 2024-09-19T04:16:51.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.579+0000 7fd0b1a43640 1 --2- 172.21.15.76:0/495368243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac0af470 0x7fd0ac06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:51.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b1242640 1 --2- 172.21.15.76:0/495368243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac15bec0 0x7fd0ac0ad200 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd0980097c0 tx=0x7fd09801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f13f8e9947df9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0abfff640 1 -- 172.21.15.76:0/495368243 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980b1070 con 0x7fd0ac15bec0 2024-09-19T04:16:51.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0abfff640 1 -- 172.21.15.76:0/495368243 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0980a8050 con 0x7fd0ac15bec0 2024-09-19T04:16:51.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0abfff640 1 -- 172.21.15.76:0/495368243 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980ac860 con 0x7fd0ac15bec0 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- 172.21.15.76:0/495368243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac15bec0 msgr2=0x7fd0ac0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/495368243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac15bec0 0x7fd0ac0ad200 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd0980097c0 tx=0x7fd09801c3e0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- 172.21.15.76:0/495368243 shutdown_connections 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/495368243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac0af470 0x7fd0ac06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/495368243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac15bec0 0x7fd0ac0ad200 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- 172.21.15.76:0/495368243 >> 172.21.15.76:0/495368243 conn(0x7fd0ac0b1f90 msgr2=0x7fd0ac0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- 172.21.15.76:0/495368243 shutdown_connections 2024-09-19T04:16:51.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- 172.21.15.76:0/495368243 wait complete. 2024-09-19T04:16:51.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 Processor -- start 2024-09-19T04:16:51.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- start start 2024-09-19T04:16:51.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac0af470 0x7fd0ac14b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac15bec0 0x7fd0ac15c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0ac15c2a0 0x7fd0ac149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac06ed50 con 0x7fd0ac0af470 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ac0ad740 con 0x7fd0ac15c2a0 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b34a6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ac06eed0 con 0x7fd0ac15bec0 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b1242640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac0af470 0x7fd0ac14b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b1242640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac0af470 0x7fd0ac14b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44730/0 (socket says 172.21.15.76:44730) 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b1242640 1 -- 172.21.15.76:0/1859911559 learned_addr learned my addr 172.21.15.76:0/1859911559 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b1242640 1 -- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac15bec0 msgr2=0x7fd0ac15c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b0a41640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac15bec0 0x7fd0ac15c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b1a43640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0ac15c2a0 0x7fd0ac149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.583+0000 7fd0b1242640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac15bec0 0x7fd0ac15c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b1242640 1 -- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0ac15c2a0 msgr2=0x7fd0ac149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b1242640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0ac15c2a0 0x7fd0ac149df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b1242640 1 -- 172.21.15.76:0/1859911559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ac0987d0 con 0x7fd0ac0af470 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b1242640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac0af470 0x7fd0ac14b9b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd0980af040 tx=0x7fd098019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0a9ffb640 1 -- 172.21.15.76:0/1859911559 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980b9020 con 0x7fd0ac0af470 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0a9ffb640 1 -- 172.21.15.76:0/1859911559 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0980b5410 con 0x7fd0ac0af470 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0a9ffb640 1 -- 172.21.15.76:0/1859911559 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980b1040 con 0x7fd0ac0af470 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ac098ab0 con 0x7fd0ac0af470 2024-09-19T04:16:51.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ac099070 con 0x7fd0ac0af470 2024-09-19T04:16:51.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ac06e000 con 0x7fd0ac0af470 2024-09-19T04:16:51.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0a9ffb640 1 -- 172.21.15.76:0/1859911559 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd0980bc020 con 0x7fd0ac0af470 2024-09-19T04:16:51.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0a9ffb640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0900678c0 0x7fd090069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0a9ffb640 1 -- 172.21.15.76:0/1859911559 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7fd09811fbc0 con 0x7fd0ac0af470 2024-09-19T04:16:51.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b0a41640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0900678c0 0x7fd090069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.587+0000 7fd0b0a41640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0900678c0 0x7fd090069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0007fd0 tx=0x7fd0a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.591+0000 7fd0a9ffb640 1 -- 172.21.15.76:0/1859911559 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd0980c3030 con 0x7fd0ac0af470 2024-09-19T04:16:51.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.751+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0ac06fcd0 con 0x7fd0900678c0 2024-09-19T04:16:51.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.759+0000 7fd0a9ffb640 1 -- 172.21.15.76:0/1859911559 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252020 (secure 0 0 0) 0x7fd0ac06fcd0 con 0x7fd0900678c0 2024-09-19T04:16:51.767 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:51.769 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0900678c0 msgr2=0x7fd090069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0900678c0 0x7fd090069d80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd0a0007fd0 tx=0x7fd0a0009040 comp rx=0 tx=0).stop 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac0af470 msgr2=0x7fd0ac14b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac0af470 0x7fd0ac14b9b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd0980af040 tx=0x7fd098019180 comp rx=0 tx=0).stop 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 shutdown_connections 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0900678c0 0x7fd090069d80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0ac15c2a0 0x7fd0ac149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd0ac15bec0 0x7fd0ac15c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 --2- 172.21.15.76:0/1859911559 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd0ac0af470 0x7fd0ac14b9b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 >> 172.21.15.76:0/1859911559 conn(0x7fd0ac0b1f90 msgr2=0x7fd0ac075230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 shutdown_connections 2024-09-19T04:16:51.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.763+0000 7fd0b34a6640 1 -- 172.21.15.76:0/1859911559 wait complete. 2024-09-19T04:16:51.786 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-19T04:16:49.803664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236272,"kb_used_data":7528,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":754738448,"statfs":{"total":773094113280,"available":772852170752,"internally_reserved":0,"allocated":7708672,"data_stored":4134380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":234016305},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.831654"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011967+0000","last_change":"2024-09-19T04:16:42.969949+0000","last_active":"2024-09-19T04:16:44.011967+0000","last_peered":"2024-09-19T04:16:44.011967+0000","last_clean":"2024-09-19T04:16:44.011967+0000","last_became_active":"2024-09-19T04:16:42.969397+0000","last_became_peered":"2024-09-19T04:16:42.969397+0000","last_unstale":"2024-09-19T04:16:44.011967+0000","last_undegraded":"2024-09-19T04:16:44.011967+0000","last_fullsized":"2024-09-19T04:16:44.011967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:17.590298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012509+0000","last_change":"2024-09-19T04:16:42.966076+0000","last_active":"2024-09-19T04:16:44.012509+0000","last_peered":"2024-09-19T04:16:44.012509+0000","last_clean":"2024-09-19T04:16:44.012509+0000","last_became_active":"2024-09-19T04:16:42.965438+0000","last_became_peered":"2024-09-19T04:16:42.965438+0000","last_unstale":"2024-09-19T04:16:44.012509+0000","last_undegraded":"2024-09-19T04:16:44.012509+0000","last_fullsized":"2024-09-19T04:16:44.012509+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:42:23.863935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012903+0000","last_change":"2024-09-19T04:16:42.980403+0000","last_active":"2024-09-19T04:16:44.012903+0000","last_peered":"2024-09-19T04:16:44.012903+0000","last_clean":"2024-09-19T04:16:44.012903+0000","last_became_active":"2024-09-19T04:16:42.976221+0000","last_became_peered":"2024-09-19T04:16:42.976221+0000","last_unstale":"2024-09-19T04:16:44.012903+0000","last_undegraded":"2024-09-19T04:16:44.012903+0000","last_fullsized":"2024-09-19T04:16:44.012903+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:41.017244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926347+0000","last_change":"2024-09-19T04:16:42.933474+0000","last_active":"2024-09-19T04:16:43.926347+0000","last_peered":"2024-09-19T04:16:43.926347+0000","last_clean":"2024-09-19T04:16:43.926347+0000","last_became_active":"2024-09-19T04:16:42.932683+0000","last_became_peered":"2024-09-19T04:16:42.932683+0000","last_unstale":"2024-09-19T04:16:43.926347+0000","last_undegraded":"2024-09-19T04:16:43.926347+0000","last_fullsized":"2024-09-19T04:16:43.926347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:10:32.241411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014594+0000","last_change":"2024-09-19T04:16:42.961247+0000","last_active":"2024-09-19T04:16:44.014594+0000","last_peered":"2024-09-19T04:16:44.014594+0000","last_clean":"2024-09-19T04:16:44.014594+0000","last_became_active":"2024-09-19T04:16:42.960613+0000","last_became_peered":"2024-09-19T04:16:42.960613+0000","last_unstale":"2024-09-19T04:16:44.014594+0000","last_undegraded":"2024-09-19T04:16:44.014594+0000","last_fullsized":"2024-09-19T04:16:44.014594+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:35.834614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921497+0000","last_change":"2024-09-19T04:16:42.978941+0000","last_active":"2024-09-19T04:16:43.921497+0000","last_peered":"2024-09-19T04:16:43.921497+0000","last_clean":"2024-09-19T04:16:43.921497+0000","last_became_active":"2024-09-19T04:16:42.978592+0000","last_became_peered":"2024-09-19T04:16:42.978592+0000","last_unstale":"2024-09-19T04:16:43.921497+0000","last_undegraded":"2024-09-19T04:16:43.921497+0000","last_fullsized":"2024-09-19T04:16:43.921497+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:43:42.286458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012967+0000","last_change":"2024-09-19T04:16:42.975423+0000","last_active":"2024-09-19T04:16:44.012967+0000","last_peered":"2024-09-19T04:16:44.012967+0000","last_clean":"2024-09-19T04:16:44.012967+0000","last_became_active":"2024-09-19T04:16:42.974803+0000","last_became_peered":"2024-09-19T04:16:42.974803+0000","last_unstale":"2024-09-19T04:16:44.012967+0000","last_undegraded":"2024-09-19T04:16:44.012967+0000","last_fullsized":"2024-09-19T04:16:44.012967+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:17.639830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924508+0000","last_change":"2024-09-19T04:16:42.964398+0000","last_active":"2024-09-19T04:16:43.924508+0000","last_peered":"2024-09-19T04:16:43.924508+0000","last_clean":"2024-09-19T04:16:43.924508+0000","last_became_active":"2024-09-19T04:16:42.962066+0000","last_became_peered":"2024-09-19T04:16:42.962066+0000","last_unstale":"2024-09-19T04:16:43.924508+0000","last_undegraded":"2024-09-19T04:16:43.924508+0000","last_fullsized":"2024-09-19T04:16:43.924508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:34.299847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014901+0000","last_change":"2024-09-19T04:16:42.981313+0000","last_active":"2024-09-19T04:16:44.014901+0000","last_peered":"2024-09-19T04:16:44.014901+0000","last_clean":"2024-09-19T04:16:44.014901+0000","last_became_active":"2024-09-19T04:16:42.977690+0000","last_became_peered":"2024-09-19T04:16:42.977690+0000","last_unstale":"2024-09-19T04:16:44.014901+0000","last_undegraded":"2024-09-19T04:16:44.014901+0000","last_fullsized":"2024-09-19T04:16:44.014901+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:39.915761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014572+0000","last_change":"2024-09-19T04:16:42.977739+0000","last_active":"2024-09-19T04:16:44.014572+0000","last_peered":"2024-09-19T04:16:44.014572+0000","last_clean":"2024-09-19T04:16:44.014572+0000","last_became_active":"2024-09-19T04:16:42.976770+0000","last_became_peered":"2024-09-19T04:16:42.976770+0000","last_unstale":"2024-09-19T04:16:44.014572+0000","last_undegraded":"2024-09-19T04:16:44.014572+0000","last_fullsized":"2024-09-19T04:16:44.014572+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:26.112969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925889+0000","last_change":"2024-09-19T04:16:42.969013+0000","last_active":"2024-09-19T04:16:43.925889+0000","last_peered":"2024-09-19T04:16:43.925889+0000","last_clean":"2024-09-19T04:16:43.925889+0000","last_became_active":"2024-09-19T04:16:42.968258+0000","last_became_peered":"2024-09-19T04:16:42.968258+0000","last_unstale":"2024-09-19T04:16:43.925889+0000","last_undegraded":"2024-09-19T04:16:43.925889+0000","last_fullsized":"2024-09-19T04:16:43.925889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:33.850829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926610+0000","last_change":"2024-09-19T04:16:42.962680+0000","last_active":"2024-09-19T04:16:43.926610+0000","last_peered":"2024-09-19T04:16:43.926610+0000","last_clean":"2024-09-19T04:16:43.926610+0000","last_became_active":"2024-09-19T04:16:42.962011+0000","last_became_peered":"2024-09-19T04:16:42.962011+0000","last_unstale":"2024-09-19T04:16:43.926610+0000","last_undegraded":"2024-09-19T04:16:43.926610+0000","last_fullsized":"2024-09-19T04:16:43.926610+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:06.443119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926507+0000","last_change":"2024-09-19T04:16:42.940463+0000","last_active":"2024-09-19T04:16:43.926507+0000","last_peered":"2024-09-19T04:16:43.926507+0000","last_clean":"2024-09-19T04:16:43.926507+0000","last_became_active":"2024-09-19T04:16:42.938689+0000","last_became_peered":"2024-09-19T04:16:42.938689+0000","last_unstale":"2024-09-19T04:16:43.926507+0000","last_undegraded":"2024-09-19T04:16:43.926507+0000","last_fullsized":"2024-09-19T04:16:43.926507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:36.099771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014268+0000","last_change":"2024-09-19T04:16:42.977359+0000","last_active":"2024-09-19T04:16:44.014268+0000","last_peered":"2024-09-19T04:16:44.014268+0000","last_clean":"2024-09-19T04:16:44.014268+0000","last_became_active":"2024-09-19T04:16:42.976285+0000","last_became_peered":"2024-09-19T04:16:42.976285+0000","last_unstale":"2024-09-19T04:16:44.014268+0000","last_undegraded":"2024-09-19T04:16:44.014268+0000","last_fullsized":"2024-09-19T04:16:44.014268+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:47.845965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013719+0000","last_change":"2024-09-19T04:16:42.982391+0000","last_active":"2024-09-19T04:16:44.013719+0000","last_peered":"2024-09-19T04:16:44.013719+0000","last_clean":"2024-09-19T04:16:44.013719+0000","last_became_active":"2024-09-19T04:16:42.981762+0000","last_became_peered":"2024-09-19T04:16:42.981762+0000","last_unstale":"2024-09-19T04:16:44.013719+0000","last_undegraded":"2024-09-19T04:16:44.013719+0000","last_fullsized":"2024-09-19T04:16:44.013719+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:58:44.342511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012256+0000","last_change":"2024-09-19T04:16:42.943841+0000","last_active":"2024-09-19T04:16:44.012256+0000","last_peered":"2024-09-19T04:16:44.012256+0000","last_clean":"2024-09-19T04:16:44.012256+0000","last_became_active":"2024-09-19T04:16:42.943052+0000","last_became_peered":"2024-09-19T04:16:42.943052+0000","last_unstale":"2024-09-19T04:16:44.012256+0000","last_undegraded":"2024-09-19T04:16:44.012256+0000","last_fullsized":"2024-09-19T04:16:44.012256+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:50.532040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012127+0000","last_change":"2024-09-19T04:16:42.968957+0000","last_active":"2024-09-19T04:16:44.012127+0000","last_peered":"2024-09-19T04:16:44.012127+0000","last_clean":"2024-09-19T04:16:44.012127+0000","last_became_active":"2024-09-19T04:16:42.968321+0000","last_became_peered":"2024-09-19T04:16:42.968321+0000","last_unstale":"2024-09-19T04:16:44.012127+0000","last_undegraded":"2024-09-19T04:16:44.012127+0000","last_fullsized":"2024-09-19T04:16:44.012127+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:23.053674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925849+0000","last_change":"2024-09-19T04:16:42.933099+0000","last_active":"2024-09-19T04:16:43.925849+0000","last_peered":"2024-09-19T04:16:43.925849+0000","last_clean":"2024-09-19T04:16:43.925849+0000","last_became_active":"2024-09-19T04:16:42.932385+0000","last_became_peered":"2024-09-19T04:16:42.932385+0000","last_unstale":"2024-09-19T04:16:43.925849+0000","last_undegraded":"2024-09-19T04:16:43.925849+0000","last_fullsized":"2024-09-19T04:16:43.925849+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:17:23.327782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012455+0000","last_change":"2024-09-19T04:16:42.951344+0000","last_active":"2024-09-19T04:16:44.012455+0000","last_peered":"2024-09-19T04:16:44.012455+0000","last_clean":"2024-09-19T04:16:44.012455+0000","last_became_active":"2024-09-19T04:16:42.950151+0000","last_became_peered":"2024-09-19T04:16:42.950151+0000","last_unstale":"2024-09-19T04:16:44.012455+0000","last_undegraded":"2024-09-19T04:16:44.012455+0000","last_fullsized":"2024-09-19T04:16:44.012455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:29:37.132305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013530+0000","last_change":"2024-09-19T04:16:42.949344+0000","last_active":"2024-09-19T04:16:44.013530+0000","last_peered":"2024-09-19T04:16:44.013530+0000","last_clean":"2024-09-19T04:16:44.013530+0000","last_became_active":"2024-09-19T04:16:42.948576+0000","last_became_peered":"2024-09-19T04:16:42.948576+0000","last_unstale":"2024-09-19T04:16:44.013530+0000","last_undegraded":"2024-09-19T04:16:44.013530+0000","last_fullsized":"2024-09-19T04:16:44.013530+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:56:11.638219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926906+0000","last_change":"2024-09-19T04:16:42.936037+0000","last_active":"2024-09-19T04:16:43.926906+0000","last_peered":"2024-09-19T04:16:43.926906+0000","last_clean":"2024-09-19T04:16:43.926906+0000","last_became_active":"2024-09-19T04:16:42.935328+0000","last_became_peered":"2024-09-19T04:16:42.935328+0000","last_unstale":"2024-09-19T04:16:43.926906+0000","last_undegraded":"2024-09-19T04:16:43.926906+0000","last_fullsized":"2024-09-19T04:16:43.926906+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:47.147563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012192+0000","last_change":"2024-09-19T04:16:42.976023+0000","last_active":"2024-09-19T04:16:44.012192+0000","last_peered":"2024-09-19T04:16:44.012192+0000","last_clean":"2024-09-19T04:16:44.012192+0000","last_became_active":"2024-09-19T04:16:42.975285+0000","last_became_peered":"2024-09-19T04:16:42.975285+0000","last_unstale":"2024-09-19T04:16:44.012192+0000","last_undegraded":"2024-09-19T04:16:44.012192+0000","last_fullsized":"2024-09-19T04:16:44.012192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:46.642698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011872+0000","last_change":"2024-09-19T04:16:42.978717+0000","last_active":"2024-09-19T04:16:44.011872+0000","last_peered":"2024-09-19T04:16:44.011872+0000","last_clean":"2024-09-19T04:16:44.011872+0000","last_became_active":"2024-09-19T04:16:42.978388+0000","last_became_peered":"2024-09-19T04:16:42.978388+0000","last_unstale":"2024-09-19T04:16:44.011872+0000","last_undegraded":"2024-09-19T04:16:44.011872+0000","last_fullsized":"2024-09-19T04:16:44.011872+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:38.008584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013583+0000","last_change":"2024-09-19T04:16:42.982773+0000","last_active":"2024-09-19T04:16:44.013583+0000","last_peered":"2024-09-19T04:16:44.013583+0000","last_clean":"2024-09-19T04:16:44.013583+0000","last_became_active":"2024-09-19T04:16:42.980750+0000","last_became_peered":"2024-09-19T04:16:42.980750+0000","last_unstale":"2024-09-19T04:16:44.013583+0000","last_undegraded":"2024-09-19T04:16:44.013583+0000","last_fullsized":"2024-09-19T04:16:44.013583+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:41:33.009212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927393+0000","last_change":"2024-09-19T04:16:42.948182+0000","last_active":"2024-09-19T04:16:43.927393+0000","last_peered":"2024-09-19T04:16:43.927393+0000","last_clean":"2024-09-19T04:16:43.927393+0000","last_became_active":"2024-09-19T04:16:42.947543+0000","last_became_peered":"2024-09-19T04:16:42.947543+0000","last_unstale":"2024-09-19T04:16:43.927393+0000","last_undegraded":"2024-09-19T04:16:43.927393+0000","last_fullsized":"2024-09-19T04:16:43.927393+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:46:40.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926671+0000","last_change":"2024-09-19T04:16:42.950205+0000","last_active":"2024-09-19T04:16:43.926671+0000","last_peered":"2024-09-19T04:16:43.926671+0000","last_clean":"2024-09-19T04:16:43.926671+0000","last_became_active":"2024-09-19T04:16:42.949551+0000","last_became_peered":"2024-09-19T04:16:42.949551+0000","last_unstale":"2024-09-19T04:16:43.926671+0000","last_undegraded":"2024-09-19T04:16:43.926671+0000","last_fullsized":"2024-09-19T04:16:43.926671+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:50.434667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926298+0000","last_change":"2024-09-19T04:16:42.952280+0000","last_active":"2024-09-19T04:16:43.926298+0000","last_peered":"2024-09-19T04:16:43.926298+0000","last_clean":"2024-09-19T04:16:43.926298+0000","last_became_active":"2024-09-19T04:16:42.951553+0000","last_became_peered":"2024-09-19T04:16:42.951553+0000","last_unstale":"2024-09-19T04:16:43.926298+0000","last_undegraded":"2024-09-19T04:16:43.926298+0000","last_fullsized":"2024-09-19T04:16:43.926298+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:06:53.356430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011745+0000","last_change":"2024-09-19T04:16:42.964343+0000","last_active":"2024-09-19T04:16:44.011745+0000","last_peered":"2024-09-19T04:16:44.011745+0000","last_clean":"2024-09-19T04:16:44.011745+0000","last_became_active":"2024-09-19T04:16:42.952426+0000","last_became_peered":"2024-09-19T04:16:42.952426+0000","last_unstale":"2024-09-19T04:16:44.011745+0000","last_undegraded":"2024-09-19T04:16:44.011745+0000","last_fullsized":"2024-09-19T04:16:44.011745+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:19:35.834614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014385+0000","last_change":"2024-09-19T04:16:42.978018+0000","last_active":"2024-09-19T04:16:44.014385+0000","last_peered":"2024-09-19T04:16:44.014385+0000","last_clean":"2024-09-19T04:16:44.014385+0000","last_became_active":"2024-09-19T04:16:42.974132+0000","last_became_peered":"2024-09-19T04:16:42.974132+0000","last_unstale":"2024-09-19T04:16:44.014385+0000","last_undegraded":"2024-09-19T04:16:44.014385+0000","last_fullsized":"2024-09-19T04:16:44.014385+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:26:11.291119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924661+0000","last_change":"2024-09-19T04:16:42.963672+0000","last_active":"2024-09-19T04:16:43.924661+0000","last_peered":"2024-09-19T04:16:43.924661+0000","last_clean":"2024-09-19T04:16:43.924661+0000","last_became_active":"2024-09-19T04:16:42.963006+0000","last_became_peered":"2024-09-19T04:16:42.963006+0000","last_unstale":"2024-09-19T04:16:43.924661+0000","last_undegraded":"2024-09-19T04:16:43.924661+0000","last_fullsized":"2024-09-19T04:16:43.924661+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:25.077988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925635+0000","last_change":"2024-09-19T04:16:42.952470+0000","last_active":"2024-09-19T04:16:43.925635+0000","last_peered":"2024-09-19T04:16:43.925635+0000","last_clean":"2024-09-19T04:16:43.925635+0000","last_became_active":"2024-09-19T04:16:42.951827+0000","last_became_peered":"2024-09-19T04:16:42.951827+0000","last_unstale":"2024-09-19T04:16:43.925635+0000","last_undegraded":"2024-09-19T04:16:43.925635+0000","last_fullsized":"2024-09-19T04:16:43.925635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:17.094123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921099+0000","last_change":"2024-09-19T04:16:42.951873+0000","last_active":"2024-09-19T04:16:43.921099+0000","last_peered":"2024-09-19T04:16:43.921099+0000","last_clean":"2024-09-19T04:16:43.921099+0000","last_became_active":"2024-09-19T04:16:42.948318+0000","last_became_peered":"2024-09-19T04:16:42.948318+0000","last_unstale":"2024-09-19T04:16:43.921099+0000","last_undegraded":"2024-09-19T04:16:43.921099+0000","last_fullsized":"2024-09-19T04:16:43.921099+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:55:34.150659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012653+0000","last_change":"2024-09-19T04:16:42.969731+0000","last_active":"2024-09-19T04:16:44.012653+0000","last_peered":"2024-09-19T04:16:44.012653+0000","last_clean":"2024-09-19T04:16:44.012653+0000","last_became_active":"2024-09-19T04:16:42.969114+0000","last_became_peered":"2024-09-19T04:16:42.969114+0000","last_unstale":"2024-09-19T04:16:44.012653+0000","last_undegraded":"2024-09-19T04:16:44.012653+0000","last_fullsized":"2024-09-19T04:16:44.012653+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:52:54.454670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"66'4","reported_seq":18,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012196+0000","last_change":"2024-09-19T04:16:42.013739+0000","last_active":"2024-09-19T04:16:44.012196+0000","last_peered":"2024-09-19T04:16:44.012196+0000","last_clean":"2024-09-19T04:16:44.012196+0000","last_became_active":"2024-09-19T04:16:42.013362+0000","last_became_peered":"2024-09-19T04:16:42.013362+0000","last_unstale":"2024-09-19T04:16:44.012196+0000","last_undegraded":"2024-09-19T04:16:44.012196+0000","last_fullsized":"2024-09-19T04:16:44.012196+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:52.006040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926709+0000","last_change":"2024-09-19T04:16:42.950145+0000","last_active":"2024-09-19T04:16:43.926709+0000","last_peered":"2024-09-19T04:16:43.926709+0000","last_clean":"2024-09-19T04:16:43.926709+0000","last_became_active":"2024-09-19T04:16:42.949460+0000","last_became_peered":"2024-09-19T04:16:42.949460+0000","last_unstale":"2024-09-19T04:16:43.926709+0000","last_undegraded":"2024-09-19T04:16:43.926709+0000","last_fullsized":"2024-09-19T04:16:43.926709+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922760+0000","last_change":"2024-09-19T04:16:42.018926+0000","last_active":"2024-09-19T04:16:43.922760+0000","last_peered":"2024-09-19T04:16:43.922760+0000","last_clean":"2024-09-19T04:16:43.922760+0000","last_became_active":"2024-09-19T04:16:42.018655+0000","last_became_peered":"2024-09-19T04:16:42.018655+0000","last_unstale":"2024-09-19T04:16:43.922760+0000","last_undegraded":"2024-09-19T04:16:43.922760+0000","last_fullsized":"2024-09-19T04:16:43.922760+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:11:59.963896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922698+0000","last_change":"2024-09-19T04:16:42.957511+0000","last_active":"2024-09-19T04:16:43.922698+0000","last_peered":"2024-09-19T04:16:43.922698+0000","last_clean":"2024-09-19T04:16:43.922698+0000","last_became_active":"2024-09-19T04:16:42.955571+0000","last_became_peered":"2024-09-19T04:16:42.955571+0000","last_unstale":"2024-09-19T04:16:43.922698+0000","last_undegraded":"2024-09-19T04:16:43.922698+0000","last_fullsized":"2024-09-19T04:16:43.922698+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:46.642698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922269+0000","last_change":"2024-09-19T04:16:41.992501+0000","last_active":"2024-09-19T04:16:43.922269+0000","last_peered":"2024-09-19T04:16:43.922269+0000","last_clean":"2024-09-19T04:16:43.922269+0000","last_became_active":"2024-09-19T04:16:41.992135+0000","last_became_peered":"2024-09-19T04:16:41.992135+0000","last_unstale":"2024-09-19T04:16:43.922269+0000","last_undegraded":"2024-09-19T04:16:43.922269+0000","last_fullsized":"2024-09-19T04:16:43.922269+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:57.542066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014399+0000","last_change":"2024-09-19T04:16:42.977067+0000","last_active":"2024-09-19T04:16:44.014399+0000","last_peered":"2024-09-19T04:16:44.014399+0000","last_clean":"2024-09-19T04:16:44.014399+0000","last_became_active":"2024-09-19T04:16:42.974265+0000","last_became_peered":"2024-09-19T04:16:42.974265+0000","last_unstale":"2024-09-19T04:16:44.014399+0000","last_undegraded":"2024-09-19T04:16:44.014399+0000","last_fullsized":"2024-09-19T04:16:44.014399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:42:56.237147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921963+0000","last_change":"2024-09-19T04:16:42.000668+0000","last_active":"2024-09-19T04:16:43.921963+0000","last_peered":"2024-09-19T04:16:43.921963+0000","last_clean":"2024-09-19T04:16:43.921963+0000","last_became_active":"2024-09-19T04:16:42.000249+0000","last_became_peered":"2024-09-19T04:16:42.000249+0000","last_unstale":"2024-09-19T04:16:43.921963+0000","last_undegraded":"2024-09-19T04:16:43.921963+0000","last_fullsized":"2024-09-19T04:16:43.921963+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:24.073566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012340+0000","last_change":"2024-09-19T04:16:42.953355+0000","last_active":"2024-09-19T04:16:44.012340+0000","last_peered":"2024-09-19T04:16:44.012340+0000","last_clean":"2024-09-19T04:16:44.012340+0000","last_became_active":"2024-09-19T04:16:42.952559+0000","last_became_peered":"2024-09-19T04:16:42.952559+0000","last_unstale":"2024-09-19T04:16:44.012340+0000","last_undegraded":"2024-09-19T04:16:44.012340+0000","last_fullsized":"2024-09-19T04:16:44.012340+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:28:29.158553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012976+0000","last_change":"2024-09-19T04:16:41.989580+0000","last_active":"2024-09-19T04:16:44.012976+0000","last_peered":"2024-09-19T04:16:44.012976+0000","last_clean":"2024-09-19T04:16:44.012976+0000","last_became_active":"2024-09-19T04:16:41.989078+0000","last_became_peered":"2024-09-19T04:16:41.989078+0000","last_unstale":"2024-09-19T04:16:44.012976+0000","last_undegraded":"2024-09-19T04:16:44.012976+0000","last_fullsized":"2024-09-19T04:16:44.012976+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:35.095349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011237+0000","last_change":"2024-09-19T04:16:42.965094+0000","last_active":"2024-09-19T04:16:44.011237+0000","last_peered":"2024-09-19T04:16:44.011237+0000","last_clean":"2024-09-19T04:16:44.011237+0000","last_became_active":"2024-09-19T04:16:42.964362+0000","last_became_peered":"2024-09-19T04:16:42.964362+0000","last_unstale":"2024-09-19T04:16:44.011237+0000","last_undegraded":"2024-09-19T04:16:44.011237+0000","last_fullsized":"2024-09-19T04:16:44.011237+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:15.592286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012243+0000","last_change":"2024-09-19T04:16:41.995699+0000","last_active":"2024-09-19T04:16:44.012243+0000","last_peered":"2024-09-19T04:16:44.012243+0000","last_clean":"2024-09-19T04:16:44.012243+0000","last_became_active":"2024-09-19T04:16:41.993440+0000","last_became_peered":"2024-09-19T04:16:41.993440+0000","last_unstale":"2024-09-19T04:16:44.012243+0000","last_undegraded":"2024-09-19T04:16:44.012243+0000","last_fullsized":"2024-09-19T04:16:44.012243+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:44:46.143141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011973+0000","last_change":"2024-09-19T04:16:42.948605+0000","last_active":"2024-09-19T04:16:44.011973+0000","last_peered":"2024-09-19T04:16:44.011973+0000","last_clean":"2024-09-19T04:16:44.011973+0000","last_became_active":"2024-09-19T04:16:42.947298+0000","last_became_peered":"2024-09-19T04:16:42.947298+0000","last_unstale":"2024-09-19T04:16:44.011973+0000","last_undegraded":"2024-09-19T04:16:44.011973+0000","last_fullsized":"2024-09-19T04:16:44.011973+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:35.021836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011964+0000","last_change":"2024-09-19T04:16:42.022337+0000","last_active":"2024-09-19T04:16:44.011964+0000","last_peered":"2024-09-19T04:16:44.011964+0000","last_clean":"2024-09-19T04:16:44.011964+0000","last_became_active":"2024-09-19T04:16:42.021791+0000","last_became_peered":"2024-09-19T04:16:42.021791+0000","last_unstale":"2024-09-19T04:16:44.011964+0000","last_undegraded":"2024-09-19T04:16:44.011964+0000","last_fullsized":"2024-09-19T04:16:44.011964+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:57.542066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012810+0000","last_change":"2024-09-19T04:16:42.961422+0000","last_active":"2024-09-19T04:16:44.012810+0000","last_peered":"2024-09-19T04:16:44.012810+0000","last_clean":"2024-09-19T04:16:44.012810+0000","last_became_active":"2024-09-19T04:16:42.960600+0000","last_became_peered":"2024-09-19T04:16:42.960600+0000","last_unstale":"2024-09-19T04:16:44.012810+0000","last_undegraded":"2024-09-19T04:16:44.012810+0000","last_fullsized":"2024-09-19T04:16:44.012810+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:18.223464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011820+0000","last_change":"2024-09-19T04:16:42.019556+0000","last_active":"2024-09-19T04:16:44.011820+0000","last_peered":"2024-09-19T04:16:44.011820+0000","last_clean":"2024-09-19T04:16:44.011820+0000","last_became_active":"2024-09-19T04:16:42.018870+0000","last_became_peered":"2024-09-19T04:16:42.018870+0000","last_unstale":"2024-09-19T04:16:44.011820+0000","last_undegraded":"2024-09-19T04:16:44.011820+0000","last_fullsized":"2024-09-19T04:16:44.011820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:56.469075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011719+0000","last_change":"2024-09-19T04:16:42.964622+0000","last_active":"2024-09-19T04:16:44.011719+0000","last_peered":"2024-09-19T04:16:44.011719+0000","last_clean":"2024-09-19T04:16:44.011719+0000","last_became_active":"2024-09-19T04:16:42.963938+0000","last_became_peered":"2024-09-19T04:16:42.963938+0000","last_unstale":"2024-09-19T04:16:44.011719+0000","last_undegraded":"2024-09-19T04:16:44.011719+0000","last_fullsized":"2024-09-19T04:16:44.011719+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:02.598595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014918+0000","last_change":"2024-09-19T04:16:42.026415+0000","last_active":"2024-09-19T04:16:44.014918+0000","last_peered":"2024-09-19T04:16:44.014918+0000","last_clean":"2024-09-19T04:16:44.014918+0000","last_became_active":"2024-09-19T04:16:42.025740+0000","last_became_peered":"2024-09-19T04:16:42.025740+0000","last_unstale":"2024-09-19T04:16:44.014918+0000","last_undegraded":"2024-09-19T04:16:44.014918+0000","last_fullsized":"2024-09-19T04:16:44.014918+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:25:40.357908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014048+0000","last_change":"2024-09-19T04:16:42.949543+0000","last_active":"2024-09-19T04:16:44.014048+0000","last_peered":"2024-09-19T04:16:44.014048+0000","last_clean":"2024-09-19T04:16:44.014048+0000","last_became_active":"2024-09-19T04:16:42.948796+0000","last_became_peered":"2024-09-19T04:16:42.948796+0000","last_unstale":"2024-09-19T04:16:44.014048+0000","last_undegraded":"2024-09-19T04:16:44.014048+0000","last_fullsized":"2024-09-19T04:16:44.014048+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:46.277558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927666+0000","last_change":"2024-09-19T04:16:42.001820+0000","last_active":"2024-09-19T04:16:43.927666+0000","last_peered":"2024-09-19T04:16:43.927666+0000","last_clean":"2024-09-19T04:16:43.927666+0000","last_became_active":"2024-09-19T04:16:42.001141+0000","last_became_peered":"2024-09-19T04:16:42.001141+0000","last_unstale":"2024-09-19T04:16:43.927666+0000","last_undegraded":"2024-09-19T04:16:43.927666+0000","last_fullsized":"2024-09-19T04:16:43.927666+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:12.519867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012651+0000","last_change":"2024-09-19T04:16:42.954665+0000","last_active":"2024-09-19T04:16:44.012651+0000","last_peered":"2024-09-19T04:16:44.012651+0000","last_clean":"2024-09-19T04:16:44.012651+0000","last_became_active":"2024-09-19T04:16:42.953885+0000","last_became_peered":"2024-09-19T04:16:42.953885+0000","last_unstale":"2024-09-19T04:16:44.012651+0000","last_undegraded":"2024-09-19T04:16:44.012651+0000","last_fullsized":"2024-09-19T04:16:44.012651+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:09.075169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"66'3","reported_seq":17,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013176+0000","last_change":"2024-09-19T04:16:42.018953+0000","last_active":"2024-09-19T04:16:44.013176+0000","last_peered":"2024-09-19T04:16:44.013176+0000","last_clean":"2024-09-19T04:16:44.013176+0000","last_became_active":"2024-09-19T04:16:42.018215+0000","last_became_peered":"2024-09-19T04:16:42.018215+0000","last_unstale":"2024-09-19T04:16:44.013176+0000","last_undegraded":"2024-09-19T04:16:44.013176+0000","last_fullsized":"2024-09-19T04:16:44.013176+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:47:26.160525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.928208+0000","last_change":"2024-09-19T04:16:42.951952+0000","last_active":"2024-09-19T04:16:43.928208+0000","last_peered":"2024-09-19T04:16:43.928208+0000","last_clean":"2024-09-19T04:16:43.928208+0000","last_became_active":"2024-09-19T04:16:42.951251+0000","last_became_peered":"2024-09-19T04:16:42.951251+0000","last_unstale":"2024-09-19T04:16:43.928208+0000","last_undegraded":"2024-09-19T04:16:43.928208+0000","last_fullsized":"2024-09-19T04:16:43.928208+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:35.399902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"66'6","reported_seq":20,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011835+0000","last_change":"2024-09-19T04:16:42.006482+0000","last_active":"2024-09-19T04:16:44.011835+0000","last_peered":"2024-09-19T04:16:44.011835+0000","last_clean":"2024-09-19T04:16:44.011835+0000","last_became_active":"2024-09-19T04:16:42.006111+0000","last_became_peered":"2024-09-19T04:16:42.006111+0000","last_unstale":"2024-09-19T04:16:44.011835+0000","last_undegraded":"2024-09-19T04:16:44.011835+0000","last_fullsized":"2024-09-19T04:16:44.011835+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:39:32.846407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924834+0000","last_change":"2024-09-19T04:16:42.974772+0000","last_active":"2024-09-19T04:16:43.924834+0000","last_peered":"2024-09-19T04:16:43.924834+0000","last_clean":"2024-09-19T04:16:43.924834+0000","last_became_active":"2024-09-19T04:16:42.974128+0000","last_became_peered":"2024-09-19T04:16:42.974128+0000","last_unstale":"2024-09-19T04:16:43.924834+0000","last_undegraded":"2024-09-19T04:16:43.924834+0000","last_fullsized":"2024-09-19T04:16:43.924834+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:35:06.209292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014276+0000","last_change":"2024-09-19T04:16:42.024717+0000","last_active":"2024-09-19T04:16:44.014276+0000","last_peered":"2024-09-19T04:16:44.014276+0000","last_clean":"2024-09-19T04:16:44.014276+0000","last_became_active":"2024-09-19T04:16:42.023979+0000","last_became_peered":"2024-09-19T04:16:42.023979+0000","last_unstale":"2024-09-19T04:16:44.014276+0000","last_undegraded":"2024-09-19T04:16:44.014276+0000","last_fullsized":"2024-09-19T04:16:44.014276+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:35:28.029764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924607+0000","last_change":"2024-09-19T04:16:42.929172+0000","last_active":"2024-09-19T04:16:43.924607+0000","last_peered":"2024-09-19T04:16:43.924607+0000","last_clean":"2024-09-19T04:16:43.924607+0000","last_became_active":"2024-09-19T04:16:42.928397+0000","last_became_peered":"2024-09-19T04:16:42.928397+0000","last_unstale":"2024-09-19T04:16:43.924607+0000","last_undegraded":"2024-09-19T04:16:43.924607+0000","last_fullsized":"2024-09-19T04:16:43.924607+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:12:00.968318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924958+0000","last_change":"2024-09-19T04:16:42.002996+0000","last_active":"2024-09-19T04:16:43.924958+0000","last_peered":"2024-09-19T04:16:43.924958+0000","last_clean":"2024-09-19T04:16:43.924958+0000","last_became_active":"2024-09-19T04:16:42.001542+0000","last_became_peered":"2024-09-19T04:16:42.001542+0000","last_unstale":"2024-09-19T04:16:43.924958+0000","last_undegraded":"2024-09-19T04:16:43.924958+0000","last_fullsized":"2024-09-19T04:16:43.924958+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:14.587864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014348+0000","last_change":"2024-09-19T04:16:42.952737+0000","last_active":"2024-09-19T04:16:44.014348+0000","last_peered":"2024-09-19T04:16:44.014348+0000","last_clean":"2024-09-19T04:16:44.014348+0000","last_became_active":"2024-09-19T04:16:42.951143+0000","last_became_peered":"2024-09-19T04:16:42.951143+0000","last_unstale":"2024-09-19T04:16:44.014348+0000","last_undegraded":"2024-09-19T04:16:44.014348+0000","last_fullsized":"2024-09-19T04:16:44.014348+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:02.598595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013957+0000","last_change":"2024-09-19T04:16:42.018769+0000","last_active":"2024-09-19T04:16:44.013957+0000","last_peered":"2024-09-19T04:16:44.013957+0000","last_clean":"2024-09-19T04:16:44.013957+0000","last_became_active":"2024-09-19T04:16:42.017955+0000","last_became_peered":"2024-09-19T04:16:42.017955+0000","last_unstale":"2024-09-19T04:16:44.013957+0000","last_undegraded":"2024-09-19T04:16:44.013957+0000","last_fullsized":"2024-09-19T04:16:44.013957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:57.809458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011455+0000","last_change":"2024-09-19T04:16:42.969447+0000","last_active":"2024-09-19T04:16:44.011455+0000","last_peered":"2024-09-19T04:16:44.011455+0000","last_clean":"2024-09-19T04:16:44.011455+0000","last_became_active":"2024-09-19T04:16:42.968597+0000","last_became_peered":"2024-09-19T04:16:42.968597+0000","last_unstale":"2024-09-19T04:16:44.011455+0000","last_undegraded":"2024-09-19T04:16:44.011455+0000","last_fullsized":"2024-09-19T04:16:44.011455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:46.028679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"66'8","reported_seq":22,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925185+0000","last_change":"2024-09-19T04:16:42.011251+0000","last_active":"2024-09-19T04:16:43.925185+0000","last_peered":"2024-09-19T04:16:43.925185+0000","last_clean":"2024-09-19T04:16:43.925185+0000","last_became_active":"2024-09-19T04:16:42.010096+0000","last_became_peered":"2024-09-19T04:16:42.010096+0000","last_unstale":"2024-09-19T04:16:43.925185+0000","last_undegraded":"2024-09-19T04:16:43.925185+0000","last_fullsized":"2024-09-19T04:16:43.925185+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:13:35.195332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013116+0000","last_change":"2024-09-19T04:16:42.977614+0000","last_active":"2024-09-19T04:16:44.013116+0000","last_peered":"2024-09-19T04:16:44.013116+0000","last_clean":"2024-09-19T04:16:44.013116+0000","last_became_active":"2024-09-19T04:16:42.976929+0000","last_became_peered":"2024-09-19T04:16:42.976929+0000","last_unstale":"2024-09-19T04:16:44.013116+0000","last_undegraded":"2024-09-19T04:16:44.013116+0000","last_fullsized":"2024-09-19T04:16:44.013116+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:15:56.774883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927569+0000","last_change":"2024-09-19T04:16:42.000089+0000","last_active":"2024-09-19T04:16:43.927569+0000","last_peered":"2024-09-19T04:16:43.927569+0000","last_clean":"2024-09-19T04:16:43.927569+0000","last_became_active":"2024-09-19T04:16:41.999376+0000","last_became_peered":"2024-09-19T04:16:41.999376+0000","last_unstale":"2024-09-19T04:16:43.927569+0000","last_undegraded":"2024-09-19T04:16:43.927569+0000","last_fullsized":"2024-09-19T04:16:43.927569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:45.024257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012859+0000","last_change":"2024-09-19T04:16:42.963194+0000","last_active":"2024-09-19T04:16:44.012859+0000","last_peered":"2024-09-19T04:16:44.012859+0000","last_clean":"2024-09-19T04:16:44.012859+0000","last_became_active":"2024-09-19T04:16:42.962439+0000","last_became_peered":"2024-09-19T04:16:42.962439+0000","last_unstale":"2024-09-19T04:16:44.012859+0000","last_undegraded":"2024-09-19T04:16:44.012859+0000","last_fullsized":"2024-09-19T04:16:44.012859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:25:15.592286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014531+0000","last_change":"2024-09-19T04:16:42.023483+0000","last_active":"2024-09-19T04:16:44.014531+0000","last_peered":"2024-09-19T04:16:44.014531+0000","last_clean":"2024-09-19T04:16:44.014531+0000","last_became_active":"2024-09-19T04:16:42.022787+0000","last_became_peered":"2024-09-19T04:16:42.022787+0000","last_unstale":"2024-09-19T04:16:44.014531+0000","last_undegraded":"2024-09-19T04:16:44.014531+0000","last_fullsized":"2024-09-19T04:16:44.014531+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:51:09.821584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927434+0000","last_change":"2024-09-19T04:16:42.943810+0000","last_active":"2024-09-19T04:16:43.927434+0000","last_peered":"2024-09-19T04:16:43.927434+0000","last_clean":"2024-09-19T04:16:43.927434+0000","last_became_active":"2024-09-19T04:16:42.943076+0000","last_became_peered":"2024-09-19T04:16:42.943076+0000","last_unstale":"2024-09-19T04:16:43.927434+0000","last_undegraded":"2024-09-19T04:16:43.927434+0000","last_fullsized":"2024-09-19T04:16:43.927434+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:40:35.021836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"66'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013537+0000","last_change":"2024-09-19T04:16:42.026255+0000","last_active":"2024-09-19T04:16:44.013537+0000","last_peered":"2024-09-19T04:16:44.013537+0000","last_clean":"2024-09-19T04:16:44.013537+0000","last_became_active":"2024-09-19T04:16:42.025430+0000","last_became_peered":"2024-09-19T04:16:42.025430+0000","last_unstale":"2024-09-19T04:16:44.013537+0000","last_undegraded":"2024-09-19T04:16:44.013537+0000","last_fullsized":"2024-09-19T04:16:44.013537+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:39:04.630906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012920+0000","last_change":"2024-09-19T04:16:42.958174+0000","last_active":"2024-09-19T04:16:44.012920+0000","last_peered":"2024-09-19T04:16:44.012920+0000","last_clean":"2024-09-19T04:16:44.012920+0000","last_became_active":"2024-09-19T04:16:42.957489+0000","last_became_peered":"2024-09-19T04:16:42.957489+0000","last_unstale":"2024-09-19T04:16:44.012920+0000","last_undegraded":"2024-09-19T04:16:44.012920+0000","last_fullsized":"2024-09-19T04:16:44.012920+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:34:17.899656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013703+0000","last_change":"2024-09-19T04:16:41.998882+0000","last_active":"2024-09-19T04:16:44.013703+0000","last_peered":"2024-09-19T04:16:44.013703+0000","last_clean":"2024-09-19T04:16:44.013703+0000","last_became_active":"2024-09-19T04:16:41.998612+0000","last_became_peered":"2024-09-19T04:16:41.998612+0000","last_unstale":"2024-09-19T04:16:44.013703+0000","last_undegraded":"2024-09-19T04:16:44.013703+0000","last_fullsized":"2024-09-19T04:16:44.013703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:52:49.430245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921049+0000","last_change":"2024-09-19T04:16:42.954340+0000","last_active":"2024-09-19T04:16:43.921049+0000","last_peered":"2024-09-19T04:16:43.921049+0000","last_clean":"2024-09-19T04:16:43.921049+0000","last_became_active":"2024-09-19T04:16:42.948861+0000","last_became_peered":"2024-09-19T04:16:42.948861+0000","last_unstale":"2024-09-19T04:16:43.921049+0000","last_undegraded":"2024-09-19T04:16:43.921049+0000","last_fullsized":"2024-09-19T04:16:43.921049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:09:10.896065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921725+0000","last_change":"2024-09-19T04:16:42.015741+0000","last_active":"2024-09-19T04:16:43.921725+0000","last_peered":"2024-09-19T04:16:43.921725+0000","last_clean":"2024-09-19T04:16:43.921725+0000","last_became_active":"2024-09-19T04:16:42.015285+0000","last_became_peered":"2024-09-19T04:16:42.015285+0000","last_unstale":"2024-09-19T04:16:43.921725+0000","last_undegraded":"2024-09-19T04:16:43.921725+0000","last_fullsized":"2024-09-19T04:16:43.921725+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:05.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.924888+0000","last_change":"2024-09-19T04:16:42.943410+0000","last_active":"2024-09-19T04:16:43.924888+0000","last_peered":"2024-09-19T04:16:43.924888+0000","last_clean":"2024-09-19T04:16:43.924888+0000","last_became_active":"2024-09-19T04:16:42.942660+0000","last_became_peered":"2024-09-19T04:16:42.942660+0000","last_unstale":"2024-09-19T04:16:43.924888+0000","last_undegraded":"2024-09-19T04:16:43.924888+0000","last_fullsized":"2024-09-19T04:16:43.924888+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:29:58.546488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.925054+0000","last_change":"2024-09-19T04:16:42.002264+0000","last_active":"2024-09-19T04:16:43.925054+0000","last_peered":"2024-09-19T04:16:43.925054+0000","last_clean":"2024-09-19T04:16:43.925054+0000","last_became_active":"2024-09-19T04:16:42.000593+0000","last_became_peered":"2024-09-19T04:16:42.000593+0000","last_unstale":"2024-09-19T04:16:43.925054+0000","last_undegraded":"2024-09-19T04:16:43.925054+0000","last_fullsized":"2024-09-19T04:16:43.925054+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:40.593088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012399+0000","last_change":"2024-09-19T04:16:42.947997+0000","last_active":"2024-09-19T04:16:44.012399+0000","last_peered":"2024-09-19T04:16:44.012399+0000","last_clean":"2024-09-19T04:16:44.012399+0000","last_became_active":"2024-09-19T04:16:42.946350+0000","last_became_peered":"2024-09-19T04:16:42.946350+0000","last_unstale":"2024-09-19T04:16:44.012399+0000","last_undegraded":"2024-09-19T04:16:44.012399+0000","last_fullsized":"2024-09-19T04:16:44.012399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:25:35.399902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.921195+0000","last_change":"2024-09-19T04:16:42.010194+0000","last_active":"2024-09-19T04:16:43.921195+0000","last_peered":"2024-09-19T04:16:43.921195+0000","last_clean":"2024-09-19T04:16:43.921195+0000","last_became_active":"2024-09-19T04:16:42.009783+0000","last_became_peered":"2024-09-19T04:16:42.009783+0000","last_unstale":"2024-09-19T04:16:43.921195+0000","last_undegraded":"2024-09-19T04:16:43.921195+0000","last_fullsized":"2024-09-19T04:16:43.921195+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:33.295425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922031+0000","last_change":"2024-09-19T04:16:42.936445+0000","last_active":"2024-09-19T04:16:43.922031+0000","last_peered":"2024-09-19T04:16:43.922031+0000","last_clean":"2024-09-19T04:16:43.922031+0000","last_became_active":"2024-09-19T04:16:42.935756+0000","last_became_peered":"2024-09-19T04:16:42.935756+0000","last_unstale":"2024-09-19T04:16:43.922031+0000","last_undegraded":"2024-09-19T04:16:43.922031+0000","last_fullsized":"2024-09-19T04:16:43.922031+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:47:38.008584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"66'2","reported_seq":16,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927853+0000","last_change":"2024-09-19T04:16:42.010812+0000","last_active":"2024-09-19T04:16:43.927853+0000","last_peered":"2024-09-19T04:16:43.927853+0000","last_clean":"2024-09-19T04:16:43.927853+0000","last_became_active":"2024-09-19T04:16:42.010051+0000","last_became_peered":"2024-09-19T04:16:42.010051+0000","last_unstale":"2024-09-19T04:16:43.927853+0000","last_undegraded":"2024-09-19T04:16:43.927853+0000","last_fullsized":"2024-09-19T04:16:43.927853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:01.594173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012727+0000","last_change":"2024-09-19T04:16:42.950162+0000","last_active":"2024-09-19T04:16:44.012727+0000","last_peered":"2024-09-19T04:16:44.012727+0000","last_clean":"2024-09-19T04:16:44.012727+0000","last_became_active":"2024-09-19T04:16:42.947792+0000","last_became_peered":"2024-09-19T04:16:42.947792+0000","last_unstale":"2024-09-19T04:16:44.012727+0000","last_undegraded":"2024-09-19T04:16:44.012727+0000","last_fullsized":"2024-09-19T04:16:44.012727+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:30.571641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"67'1","reported_seq":15,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927518+0000","last_change":"2024-09-19T04:16:41.995074+0000","last_active":"2024-09-19T04:16:43.927518+0000","last_peered":"2024-09-19T04:16:43.927518+0000","last_clean":"2024-09-19T04:16:43.927518+0000","last_became_active":"2024-09-19T04:16:41.994321+0000","last_became_peered":"2024-09-19T04:16:41.994321+0000","last_unstale":"2024-09-19T04:16:43.927518+0000","last_undegraded":"2024-09-19T04:16:43.927518+0000","last_fullsized":"2024-09-19T04:16:43.927518+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:34:16.895234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.927038+0000","last_change":"2024-09-19T04:16:42.929147+0000","last_active":"2024-09-19T04:16:43.927038+0000","last_peered":"2024-09-19T04:16:43.927038+0000","last_clean":"2024-09-19T04:16:43.927038+0000","last_became_active":"2024-09-19T04:16:42.928337+0000","last_became_peered":"2024-09-19T04:16:42.928337+0000","last_unstale":"2024-09-19T04:16:43.927038+0000","last_undegraded":"2024-09-19T04:16:43.927038+0000","last_fullsized":"2024-09-19T04:16:43.927038+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:17.836899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.922384+0000","last_change":"2024-09-19T04:16:41.991513+0000","last_active":"2024-09-19T04:16:43.922384+0000","last_peered":"2024-09-19T04:16:43.922384+0000","last_clean":"2024-09-19T04:16:43.922384+0000","last_became_active":"2024-09-19T04:16:41.990986+0000","last_became_peered":"2024-09-19T04:16:41.990986+0000","last_unstale":"2024-09-19T04:16:43.922384+0000","last_undegraded":"2024-09-19T04:16:43.922384+0000","last_fullsized":"2024-09-19T04:16:43.922384+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:22:35.095349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013311+0000","last_change":"2024-09-19T04:16:42.958068+0000","last_active":"2024-09-19T04:16:44.013311+0000","last_peered":"2024-09-19T04:16:44.013311+0000","last_clean":"2024-09-19T04:16:44.013311+0000","last_became_active":"2024-09-19T04:16:42.957349+0000","last_became_peered":"2024-09-19T04:16:42.957349+0000","last_unstale":"2024-09-19T04:16:44.013311+0000","last_undegraded":"2024-09-19T04:16:44.013311+0000","last_fullsized":"2024-09-19T04:16:44.013311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:41.597510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011981+0000","last_change":"2024-09-19T04:16:42.003466+0000","last_active":"2024-09-19T04:16:44.011981+0000","last_peered":"2024-09-19T04:16:44.011981+0000","last_clean":"2024-09-19T04:16:44.011981+0000","last_became_active":"2024-09-19T04:16:42.003135+0000","last_became_peered":"2024-09-19T04:16:42.003135+0000","last_unstale":"2024-09-19T04:16:44.011981+0000","last_undegraded":"2024-09-19T04:16:44.011981+0000","last_fullsized":"2024-09-19T04:16:44.011981+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:17:22.323360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011818+0000","last_change":"2024-09-19T04:16:42.971396+0000","last_active":"2024-09-19T04:16:44.011818+0000","last_peered":"2024-09-19T04:16:44.011818+0000","last_clean":"2024-09-19T04:16:44.011818+0000","last_became_active":"2024-09-19T04:16:42.970709+0000","last_became_peered":"2024-09-19T04:16:42.970709+0000","last_unstale":"2024-09-19T04:16:44.011818+0000","last_undegraded":"2024-09-19T04:16:44.011818+0000","last_fullsized":"2024-09-19T04:16:44.011818+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:02:41.931892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.014634+0000","last_change":"2024-09-19T04:16:42.019204+0000","last_active":"2024-09-19T04:16:44.014634+0000","last_peered":"2024-09-19T04:16:44.014634+0000","last_clean":"2024-09-19T04:16:44.014634+0000","last_became_active":"2024-09-19T04:16:42.018134+0000","last_became_peered":"2024-09-19T04:16:42.018134+0000","last_unstale":"2024-09-19T04:16:44.014634+0000","last_undegraded":"2024-09-19T04:16:44.014634+0000","last_fullsized":"2024-09-19T04:16:44.014634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:24.530237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:43.926029+0000","last_change":"2024-09-19T04:16:42.975446+0000","last_active":"2024-09-19T04:16:43.926029+0000","last_peered":"2024-09-19T04:16:43.926029+0000","last_clean":"2024-09-19T04:16:43.926029+0000","last_became_active":"2024-09-19T04:16:42.974747+0000","last_became_peered":"2024-09-19T04:16:42.974747+0000","last_unstale":"2024-09-19T04:16:43.926029+0000","last_undegraded":"2024-09-19T04:16:43.926029+0000","last_fullsized":"2024-09-19T04:16:43.926029+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:38:53.010462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012390+0000","last_change":"2024-09-19T04:16:41.994558+0000","last_active":"2024-09-19T04:16:44.012390+0000","last_peered":"2024-09-19T04:16:44.012390+0000","last_clean":"2024-09-19T04:16:44.012390+0000","last_became_active":"2024-09-19T04:16:41.993965+0000","last_became_peered":"2024-09-19T04:16:41.993965+0000","last_unstale":"2024-09-19T04:16:44.012390+0000","last_undegraded":"2024-09-19T04:16:44.012390+0000","last_fullsized":"2024-09-19T04:16:44.012390+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:45:16.832477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011286+0000","last_change":"2024-09-19T04:16:42.969221+0000","last_active":"2024-09-19T04:16:44.011286+0000","last_peered":"2024-09-19T04:16:44.011286+0000","last_clean":"2024-09-19T04:16:44.011286+0000","last_became_active":"2024-09-19T04:16:42.968453+0000","last_became_peered":"2024-09-19T04:16:42.968453+0000","last_unstale":"2024-09-19T04:16:44.011286+0000","last_undegraded":"2024-09-19T04:16:44.011286+0000","last_fullsized":"2024-09-19T04:16:44.011286+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:22:41.597510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012001+0000","last_change":"2024-09-19T04:16:42.022760+0000","last_active":"2024-09-19T04:16:44.012001+0000","last_peered":"2024-09-19T04:16:44.012001+0000","last_clean":"2024-09-19T04:16:44.012001+0000","last_became_active":"2024-09-19T04:16:42.022030+0000","last_became_peered":"2024-09-19T04:16:42.022030+0000","last_unstale":"2024-09-19T04:16:44.012001+0000","last_undegraded":"2024-09-19T04:16:44.012001+0000","last_fullsized":"2024-09-19T04:16:44.012001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:55:05.438697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012509+0000","last_change":"2024-09-19T04:16:42.940579+0000","last_active":"2024-09-19T04:16:44.012509+0000","last_peered":"2024-09-19T04:16:44.012509+0000","last_clean":"2024-09-19T04:16:44.012509+0000","last_became_active":"2024-09-19T04:16:42.939737+0000","last_became_peered":"2024-09-19T04:16:42.939737+0000","last_unstale":"2024-09-19T04:16:44.012509+0000","last_undegraded":"2024-09-19T04:16:44.012509+0000","last_fullsized":"2024-09-19T04:16:44.012509+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:32:32.305386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.011086+0000","last_change":"2024-09-19T04:16:42.022541+0000","last_active":"2024-09-19T04:16:44.011086+0000","last_peered":"2024-09-19T04:16:44.011086+0000","last_clean":"2024-09-19T04:16:44.011086+0000","last_became_active":"2024-09-19T04:16:42.022021+0000","last_became_peered":"2024-09-19T04:16:42.022021+0000","last_unstale":"2024-09-19T04:16:44.011086+0000","last_undegraded":"2024-09-19T04:16:44.011086+0000","last_fullsized":"2024-09-19T04:16:44.011086+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:27:16.089701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013503+0000","last_change":"2024-09-19T04:16:42.970273+0000","last_active":"2024-09-19T04:16:44.013503+0000","last_peered":"2024-09-19T04:16:44.013503+0000","last_clean":"2024-09-19T04:16:44.013503+0000","last_became_active":"2024-09-19T04:16:42.969727+0000","last_became_peered":"2024-09-19T04:16:42.969727+0000","last_unstale":"2024-09-19T04:16:44.013503+0000","last_undegraded":"2024-09-19T04:16:44.013503+0000","last_fullsized":"2024-09-19T04:16:44.013503+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:41.962650+0000","last_clean_scrub_stamp":"2024-09-19T04:16:41.962650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:19:46.028679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.012001+0000","last_change":"2024-09-19T04:16:41.988994+0000","last_active":"2024-09-19T04:16:44.012001+0000","last_peered":"2024-09-19T04:16:44.012001+0000","last_clean":"2024-09-19T04:16:44.012001+0000","last_became_active":"2024-09-19T04:16:41.988755+0000","last_became_peered":"2024-09-19T04:16:41.988755+0000","last_unstale":"2024-09-19T04:16:44.012001+0000","last_undegraded":"2024-09-19T04:16:44.012001+0000","last_fullsized":"2024-09-19T04:16:44.012001+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:16:40.958228+0000","last_clean_scrub_stamp":"2024-09-19T04:16:40.958228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:54:24.073566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":256,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-19T04:16:44.013804+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:16:44.013804+0000","last_peered":"2024-09-19T04:16:44.013804+0000","last_clean":"2024-09-19T04:16:44.013804+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:16:44.013804+0000","last_undegraded":"2024-09-19T04:16:44.013804+0000","last_fullsized":"2024-09-19T04:16:44.013804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32580,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339260,"statfs":{"total":96636764160,"available":96603402240,"internally_reserved":0,"allocated":1417216,"data_stored":959747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":811008,"data_stored":368637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":811008,"data_stored":368625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542190,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":819200,"data_stored":370215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542190,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":806912,"data_stored":368603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":806912,"data_stored":368603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27852,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343988,"statfs":{"total":96636764160,"available":96608243712,"internally_reserved":0,"allocated":835584,"data_stored":370979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1400832,"data_stored":958971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:16:51.790 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:16:51.790 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T04:16:51.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.955+0000 7f615323b640 1 Processor -- start 2024-09-19T04:16:51.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.955+0000 7f615323b640 1 -- start start 2024-09-19T04:16:51.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f615323b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f614c162070 0x7f614c1624a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f615323b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c0cf070 con 0x7f614c15e4f0 2024-09-19T04:16:51.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f615323b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614c0d1c20 con 0x7f614c0d15d0 2024-09-19T04:16:51.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f615323b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f614c1629e0 con 0x7f614c162070 2024-09-19T04:16:51.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f61517d8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f614c162070 0x7f614c1624a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f6150fd7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f614c15e4f0 0x7f614c0d4e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:33352/0 (socket says 172.21.15.76:33352) 2024-09-19T04:16:51.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f6150fd7640 1 -- 172.21.15.76:0/2581093329 learned_addr learned my addr 172.21.15.76:0/2581093329 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:51.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f61517d8640 1 -- 172.21.15.76:0/2581093329 >> v1:172.21.15.116:6789/0 conn(0x7f614c0d15d0 legacy=0x7f614c0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f614b7fe640 1 -- 172.21.15.76:0/2581093329 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3826894924 0 0) 0x7f614c0cf070 con 0x7f614c15e4f0 2024-09-19T04:16:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f61517d8640 1 -- 172.21.15.76:0/2581093329 >> v1:172.21.15.76:6789/0 conn(0x7f614c15e4f0 legacy=0x7f614c0d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.959+0000 7f61517d8640 1 -- 172.21.15.76:0/2581093329 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0cf330 con 0x7f614c162070 2024-09-19T04:16:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f61517d8640 1 --2- 172.21.15.76:0/2581093329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c162070 0x7f614c1624a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6144009dc0 tx=0x7f614401f140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbed389d369261b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614b7fe640 1 -- 172.21.15.76:0/2581093329 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440b4070 con 0x7f614c162070 2024-09-19T04:16:51.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614b7fe640 1 -- 172.21.15.76:0/2581093329 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61440af600 con 0x7f614c162070 2024-09-19T04:16:51.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614b7fe640 1 -- 172.21.15.76:0/2581093329 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440af9d0 con 0x7f614c162070 2024-09-19T04:16:51.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- 172.21.15.76:0/2581093329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c162070 msgr2=0x7f614c1624a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 --2- 172.21.15.76:0/2581093329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c162070 0x7f614c1624a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f6144009dc0 tx=0x7f614401f140 comp rx=0 tx=0).stop 2024-09-19T04:16:51.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- 172.21.15.76:0/2581093329 shutdown_connections 2024-09-19T04:16:51.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 --2- 172.21.15.76:0/2581093329 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c162070 0x7f614c1624a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- 172.21.15.76:0/2581093329 >> 172.21.15.76:0/2581093329 conn(0x7f614c08bf90 msgr2=0x7f614c0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:51.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- 172.21.15.76:0/2581093329 shutdown_connections 2024-09-19T04:16:51.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- 172.21.15.76:0/2581093329 wait complete. 2024-09-19T04:16:51.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 Processor -- start 2024-09-19T04:16:51.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- start start 2024-09-19T04:16:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f614c0d15d0 0x7f614c0cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c15e4f0 0x7f614c15c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f614c162070 0x7f614c0ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f614c15e8d0 con 0x7f614c15e4f0 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f614c0cf8f0 con 0x7f614c162070 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f615323b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f614c0b7260 con 0x7f614c0d15d0 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c15e4f0 0x7f614c15c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c15e4f0 0x7f614c15c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44744/0 (socket says 172.21.15.76:44744) 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 -- 172.21.15.76:0/2068639335 learned_addr learned my addr 172.21.15.76:0/2068639335 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f6150fd7640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f614c0d15d0 0x7f614c0cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f61517d8640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f614c162070 0x7f614c0ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 -- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f614c0d15d0 msgr2=0x7f614c0cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f614c0d15d0 0x7f614c0cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 -- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f614c162070 msgr2=0x7f614c0ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f614c162070 0x7f614c0ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 -- 172.21.15.76:0/2068639335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c0ccd30 con 0x7f614c15e4f0 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f614bfff640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c15e4f0 0x7f614c15c660 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f613800c530 tx=0x7f613800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.963+0000 7f61497fa640 1 -- 172.21.15.76:0/2068639335 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138016020 con 0x7f614c15e4f0 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f61497fa640 1 -- 172.21.15.76:0/2068639335 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6138005220 con 0x7f614c15e4f0 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c0cd010 con 0x7f614c15e4f0 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f61497fa640 1 -- 172.21.15.76:0/2068639335 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138007030 con 0x7f614c15e4f0 2024-09-19T04:16:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c0cd5d0 con 0x7f614c15e4f0 2024-09-19T04:16:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c1624a0 con 0x7f614c15e4f0 2024-09-19T04:16:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f61497fa640 1 -- 172.21.15.76:0/2068639335 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f61380071d0 con 0x7f614c15e4f0 2024-09-19T04:16:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f61497fa640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61300678c0 0x7f6130069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f61497fa640 1 -- 172.21.15.76:0/2068639335 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f61380864b0 con 0x7f614c15e4f0 2024-09-19T04:16:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f6150fd7640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61300678c0 0x7f6130069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:51.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.967+0000 7f6150fd7640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61300678c0 0x7f6130069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6140007fd0 tx=0x7f6140007ba0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:51.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:51.971+0000 7f61497fa640 1 -- 172.21.15.76:0/2068639335 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6138020020 con 0x7f614c15e4f0 2024-09-19T04:16:52.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.187+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f614c161e50 con 0x7f614c15e4f0 2024-09-19T04:16:52.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f61497fa640 1 -- 172.21.15.76:0/2068639335 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f61380074c0 con 0x7f614c15e4f0 2024-09-19T04:16:52.195 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-19T04:16:52.195 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:16:52.195 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2024-09-19T04:16:52.196 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2024-09-19T04:16:52.196 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T04:16:52.196 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2024-09-19T04:16:52.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61300678c0 msgr2=0x7f6130069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61300678c0 0x7f6130069d80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6140007fd0 tx=0x7f6140007ba0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c15e4f0 msgr2=0x7f614c15c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c15e4f0 0x7f614c15c660 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f613800c530 tx=0x7f613800c9f0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 shutdown_connections 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61300678c0 0x7f6130069d80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f614c162070 0x7f614c0ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f614c15e4f0 0x7f614c15c660 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 --2- 172.21.15.76:0/2068639335 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f614c0d15d0 0x7f614c0cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 >> 172.21.15.76:0/2068639335 conn(0x7f614c08bf90 msgr2=0x7f614c06cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 shutdown_connections 2024-09-19T04:16:52.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.191+0000 7f615323b640 1 -- 172.21.15.76:0/2068639335 wait complete. 2024-09-19T04:16:52.213 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:52.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e3978f640 1 Processor -- start 2024-09-19T04:16:52.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e3978f640 1 -- start start 2024-09-19T04:16:52.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e3978f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6e34144ee0 0x7f6e340b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:52.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e3978f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e341452c0 0x7f6e3415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:52.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e3978f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e341457b0 con 0x7f6e34145c60 2024-09-19T04:16:52.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e3978f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6e340b3d00 con 0x7f6e34144ee0 2024-09-19T04:16:52.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e3978f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6e34145a90 con 0x7f6e341452c0 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e341452c0 0x7f6e3415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e32ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6e34144ee0 0x7f6e340b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e341452c0 0x7f6e3415fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:50418/0 (socket says 172.21.15.76:50418) 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e32ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6e34144ee0 0x7f6e340b37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:60570/0 (socket says 172.21.15.76:60570) 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e337fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6e34145c60 0x7f6e340b03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:58666/0 (socket says 172.21.15.76:58666) 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e327fc640 1 -- 172.21.15.76:0/1058154764 learned_addr learned my addr 172.21.15.76:0/1058154764 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e327fc640 1 -- 172.21.15.76:0/1058154764 >> v2:172.21.15.116:3300/0 conn(0x7f6e34144ee0 msgr2=0x7f6e340b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e327fc640 1 --2- 172.21.15.76:0/1058154764 >> v2:172.21.15.116:3300/0 conn(0x7f6e34144ee0 0x7f6e340b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e327fc640 1 -- 172.21.15.76:0/1058154764 >> v1:172.21.15.116:6789/0 conn(0x7f6e34145c60 legacy=0x7f6e340b03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.327+0000 7f6e327fc640 1 -- 172.21.15.76:0/1058154764 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e34160b30 con 0x7f6e341452c0 2024-09-19T04:16:52.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e327fc640 1 --2- 172.21.15.76:0/1058154764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e341452c0 0x7f6e3415fed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009870 tx=0x7f6e2801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca72a28d2e79715 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:52.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e31ffb640 1 -- 172.21.15.76:0/1058154764 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280b0070 con 0x7f6e341452c0 2024-09-19T04:16:52.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e31ffb640 1 -- 172.21.15.76:0/1058154764 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e280ab600 con 0x7f6e341452c0 2024-09-19T04:16:52.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e31ffb640 1 -- 172.21.15.76:0/1058154764 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280ab9d0 con 0x7f6e341452c0 2024-09-19T04:16:52.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 -- 172.21.15.76:0/1058154764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e341452c0 msgr2=0x7f6e3415fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 --2- 172.21.15.76:0/1058154764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e341452c0 0x7f6e3415fed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e28009870 tx=0x7f6e2801c440 comp rx=0 tx=0).stop 2024-09-19T04:16:52.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 -- 172.21.15.76:0/1058154764 shutdown_connections 2024-09-19T04:16:52.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 --2- 172.21.15.76:0/1058154764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e341452c0 0x7f6e3415fed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 --2- 172.21.15.76:0/1058154764 >> v2:172.21.15.116:3300/0 conn(0x7f6e34144ee0 0x7f6e340b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 -- 172.21.15.76:0/1058154764 >> 172.21.15.76:0/1058154764 conn(0x7f6e34073360 msgr2=0x7f6e340b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:52.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 -- 172.21.15.76:0/1058154764 shutdown_connections 2024-09-19T04:16:52.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 -- 172.21.15.76:0/1058154764 wait complete. 2024-09-19T04:16:52.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 Processor -- start 2024-09-19T04:16:52.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.331+0000 7f6e3978f640 1 -- start start 2024-09-19T04:16:52.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e34144ee0 0x7f6e34154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:52.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e341452c0 0x7f6e3415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:52.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e34145c60 0x7f6e3415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:52.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6e340b0c60 con 0x7f6e341452c0 2024-09-19T04:16:52.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6e340b0ae0 con 0x7f6e34144ee0 2024-09-19T04:16:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6e340b0de0 con 0x7f6e34145c60 2024-09-19T04:16:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e341452c0 0x7f6e3415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e341452c0 0x7f6e3415cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44758/0 (socket says 172.21.15.76:44758) 2024-09-19T04:16:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 -- 172.21.15.76:0/4022706533 learned_addr learned my addr 172.21.15.76:0/4022706533 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e337fe640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e34145c60 0x7f6e3415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e32ffd640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e34144ee0 0x7f6e34154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 -- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e34145c60 msgr2=0x7f6e3415b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e34145c60 0x7f6e3415b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 -- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e34144ee0 msgr2=0x7f6e34154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e34144ee0 0x7f6e34154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 -- 172.21.15.76:0/4022706533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3415d260 con 0x7f6e341452c0 2024-09-19T04:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e327fc640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e341452c0 0x7f6e3415cd20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6e2801aaf0 tx=0x7f6e280191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e1b7fe640 1 -- 172.21.15.76:0/4022706533 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280b0040 con 0x7f6e341452c0 2024-09-19T04:16:52.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e1b7fe640 1 -- 172.21.15.76:0/4022706533 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e280b4640 con 0x7f6e341452c0 2024-09-19T04:16:52.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e34154d20 con 0x7f6e341452c0 2024-09-19T04:16:52.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e1b7fe640 1 -- 172.21.15.76:0/4022706533 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e280b4a30 con 0x7f6e341452c0 2024-09-19T04:16:52.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3415b640 con 0x7f6e341452c0 2024-09-19T04:16:52.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e340d0ef0 con 0x7f6e341452c0 2024-09-19T04:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e1b7fe640 1 -- 172.21.15.76:0/4022706533 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6e280ab4c0 con 0x7f6e341452c0 2024-09-19T04:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e1b7fe640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e200678c0 0x7f6e20069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e1b7fe640 1 -- 172.21.15.76:0/4022706533 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f6e2811fa60 con 0x7f6e341452c0 2024-09-19T04:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.335+0000 7f6e32ffd640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e200678c0 0x7f6e20069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.339+0000 7f6e32ffd640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e200678c0 0x7f6e20069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6e240073f0 tx=0x7f6e2400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:52.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.343+0000 7f6e1b7fe640 1 -- 172.21.15.76:0/4022706533 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6e28016150 con 0x7f6e341452c0 2024-09-19T04:16:52.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.535+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e3415a460 con 0x7f6e341452c0 2024-09-19T04:16:52.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e1b7fe640 1 -- 172.21.15.76:0/4022706533 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5230 (secure 0 0 0) 0x7f6e280f0410 con 0x7f6e341452c0 2024-09-19T04:16:52.542 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:52.542 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":47,"btime":"2024-09-19T04:16:45:985234+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":6156,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:16:45.346923+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6165},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-09-19T04:16:52.542 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 47 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e200678c0 msgr2=0x7f6e20069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e200678c0 0x7f6e20069d80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6e240073f0 tx=0x7f6e2400b040 comp rx=0 tx=0).stop 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e341452c0 msgr2=0x7f6e3415cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e341452c0 0x7f6e3415cd20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f6e2801aaf0 tx=0x7f6e280191e0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 shutdown_connections 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e200678c0 0x7f6e20069d80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e34145c60 0x7f6e3415b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e341452c0 0x7f6e3415cd20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 --2- 172.21.15.76:0/4022706533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e34144ee0 0x7f6e34154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 >> 172.21.15.76:0/4022706533 conn(0x7f6e34073360 msgr2=0x7f6e3415d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 shutdown_connections 2024-09-19T04:16:52.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:52.539+0000 7f6e3978f640 1 -- 172.21.15.76:0/4022706533 wait complete. 2024-09-19T04:16:52.964 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:16:42.356083+0000', 'modified': '2024-09-19T04:16:45.346923+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6165}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6165': {'gid': 6165, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3052694229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3052694229}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3052694229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6165, 'qdb_cluster': [6165]} 2024-09-19T04:16:52.964 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:16:52.964 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:16:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5484489640 1 Processor -- start 2024-09-19T04:16:53.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5484489640 1 -- start start 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5484489640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f547c1457a0 0x7f547c0d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5484489640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0cf800 0x7f547c0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5484489640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f547c0d27a0 con 0x7f547c0b4340 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5484489640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f547c0b47b0 con 0x7f547c1457a0 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5484489640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f547c0d1720 con 0x7f547c0cf800 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5482225640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f547c1457a0 0x7f547c0d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5482225640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f547c1457a0 0x7f547c0d1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35222/0 (socket says 172.21.15.76:35222) 2024-09-19T04:16:53.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5482225640 1 -- 172.21.15.76:0/1130039998 learned_addr learned my addr 172.21.15.76:0/1130039998 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5481a24640 1 --2- 172.21.15.76:0/1130039998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0cf800 0x7f547c0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5482225640 1 -- 172.21.15.76:0/1130039998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0cf800 msgr2=0x7f547c0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5482225640 1 --2- 172.21.15.76:0/1130039998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0cf800 0x7f547c0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5482225640 1 -- 172.21.15.76:0/1130039998 >> v1:172.21.15.76:6789/0 conn(0x7f547c0b4340 legacy=0x7f547c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5482225640 1 -- 172.21.15.76:0/1130039998 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c0d2a80 con 0x7f547c1457a0 2024-09-19T04:16:53.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5481223640 1 -- 172.21.15.76:0/1130039998 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279941135 0 0) 0x7f547c0d27a0 con 0x7f547c0b4340 2024-09-19T04:16:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.091+0000 7f5481a24640 1 --2- 172.21.15.76:0/1130039998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0cf800 0x7f547c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:53.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482225640 1 --2- 172.21.15.76:0/1130039998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c0d1ea0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5478009870 tx=0x7f547801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=103734286fa61f7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5481223640 1 -- 172.21.15.76:0/1130039998 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780b0070 con 0x7f547c1457a0 2024-09-19T04:16:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5481223640 1 -- 172.21.15.76:0/1130039998 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54780ab600 con 0x7f547c1457a0 2024-09-19T04:16:53.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5481223640 1 -- 172.21.15.76:0/1130039998 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780ab9d0 con 0x7f547c1457a0 2024-09-19T04:16:53.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- 172.21.15.76:0/1130039998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 msgr2=0x7f547c0d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 --2- 172.21.15.76:0/1130039998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c0d1ea0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5478009870 tx=0x7f547801c4d0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- 172.21.15.76:0/1130039998 shutdown_connections 2024-09-19T04:16:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 --2- 172.21.15.76:0/1130039998 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0cf800 0x7f547c0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 --2- 172.21.15.76:0/1130039998 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c0d1ea0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- 172.21.15.76:0/1130039998 >> 172.21.15.76:0/1130039998 conn(0x7f547c06bcc0 msgr2=0x7f547c06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- 172.21.15.76:0/1130039998 shutdown_connections 2024-09-19T04:16:53.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- 172.21.15.76:0/1130039998 wait complete. 2024-09-19T04:16:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 Processor -- start 2024-09-19T04:16:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- start start 2024-09-19T04:16:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0b4340 0x7f547c0b6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f547c0cf800 0x7f547c15bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c15e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0b52f0 con 0x7f547c1457a0 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0b5170 con 0x7f547c0cf800 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5484489640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f547c1690b0 con 0x7f547c0b4340 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c15e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c15e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35234/0 (socket says 172.21.15.76:35234) 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 -- 172.21.15.76:0/485355963 learned_addr learned my addr 172.21.15.76:0/485355963 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482225640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0b4340 0x7f547c0b6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5481a24640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f547c0cf800 0x7f547c15bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 -- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0b4340 msgr2=0x7f547c0b6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0b4340 0x7f547c0b6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 -- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f547c0cf800 msgr2=0x7f547c15bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f547c0cf800 0x7f547c15bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 -- 172.21.15.76:0/485355963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c0b73b0 con 0x7f547c1457a0 2024-09-19T04:16:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.095+0000 7f5482a26640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c15e460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f547400c780 tx=0x7f547400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5472ffd640 1 -- 172.21.15.76:0/485355963 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474003db0 con 0x7f547c1457a0 2024-09-19T04:16:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c0b7690 con 0x7f547c1457a0 2024-09-19T04:16:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5472ffd640 1 -- 172.21.15.76:0/485355963 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5474005070 con 0x7f547c1457a0 2024-09-19T04:16:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5472ffd640 1 -- 172.21.15.76:0/485355963 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474005460 con 0x7f547c1457a0 2024-09-19T04:16:53.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c15c090 con 0x7f547c1457a0 2024-09-19T04:16:53.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5472ffd640 1 -- 172.21.15.76:0/485355963 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5474005600 con 0x7f547c1457a0 2024-09-19T04:16:53.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5472ffd640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54600678c0 0x7f5460069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5472ffd640 1 -- 172.21.15.76:0/485355963 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f5474085dd0 con 0x7f547c1457a0 2024-09-19T04:16:53.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5482225640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54600678c0 0x7f5460069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5482225640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54600678c0 0x7f5460069d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54780ad000 tx=0x7f54780a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 Processor -- start 2024-09-19T04:16:53.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 -- start start 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d386f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d33cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d33d290 0x56464d33d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56464d224b60 con 0x56464d33d290 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56464d18bc70 con 0x56464d386b20 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56464d131ed0 con 0x56464d387440 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.099+0000 7f5482a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d33d290 0x56464d33d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d33d290 0x56464d33d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35236/0 (socket says 172.21.15.76:35236) 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 -- 172.21.15.76:0/3088553217 learned_addr learned my addr 172.21.15.76:0/3088553217 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d33cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482225640 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d386f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 -- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 msgr2=0x56464d33cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d33cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 -- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 msgr2=0x56464d386f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d386f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 -- 172.21.15.76:0/3088553217 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56464d143e50 con 0x56464d33d290 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d33cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d33d290 0x56464d33d670 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f547405e740 tx=0x7f5474055f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18cb8a5d52a4329d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545ffff640 1 -- 172.21.15.76:0/3088553217 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547406e0d0 con 0x56464d33d290 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545ffff640 1 -- 172.21.15.76:0/3088553217 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547405a440 con 0x56464d33d290 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545ffff640 1 -- 172.21.15.76:0/3088553217 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547405ec70 con 0x56464d33d290 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- 172.21.15.76:0/3088553217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d33d290 msgr2=0x56464d33d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d33d290 0x56464d33d670 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f547405e740 tx=0x7f5474055f90 comp rx=0 tx=0).stop 2024-09-19T04:16:53.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- 172.21.15.76:0/3088553217 shutdown_connections 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d33d290 0x56464d33d670 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d33cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 --2- 172.21.15.76:0/3088553217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d386f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- 172.21.15.76:0/3088553217 >> 172.21.15.76:0/3088553217 conn(0x56464d343600 msgr2=0x56464d33b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- 172.21.15.76:0/3088553217 shutdown_connections 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- 172.21.15.76:0/3088553217 wait complete. 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- start start 2024-09-19T04:16:53.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 Processor -- start 2024-09-19T04:16:53.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d341e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d347cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d348230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56464d1edb00 con 0x56464d387440 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56464d166360 con 0x56464d386b20 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56464d1632d0 con 0x56464d342390 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 10 client.0 _ll_register_callbacks cb 0x56464d328820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d347cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d347cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35244/0 (socket says 172.21.15.76:35244) 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 -- 172.21.15.76:0/3056419616 learned_addr learned my addr 172.21.15.76:0/3056419616 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482a26640 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d348230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5482225640 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d341e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 -- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 msgr2=0x56464d348230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d348230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 -- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 msgr2=0x56464d341e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d341e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 -- 172.21.15.76:0/3056419616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56464d132120 con 0x56464d387440 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5481a24640 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d347cf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e560 tx=0x7f546c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26f53fa0c2743127 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545d7fa640 1 -- 172.21.15.76:0/3056419616 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c012840 con 0x56464d387440 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545d7fa640 1 -- 172.21.15.76:0/3056419616 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f546c0129e0 con 0x56464d387440 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f547405a440 con 0x56464d387440 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545d7fa640 1 -- 172.21.15.76:0/3056419616 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f546c022050 con 0x56464d387440 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 10 client.6312 fetch_fsmap learned FSMap version 47 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x56464d12b450 con 0x56464d387440 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.103+0000 7f545d7fa640 1 -- 172.21.15.76:0/3056419616 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547405a440 con 0x56464d387440 2024-09-19T04:16:53.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f545d7fa640 1 -- 172.21.15.76:0/3056419616 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f546c025020 con 0x56464d387440 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 10 client.6312 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 10 client.6312 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 20 client.6312 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d38cf50 0x56464d38d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 4 client.6312 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56464d125c00 con 0x56464d38cf50 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5482225640 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d38cf50 0x56464d38d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f545d7fa640 10 client.6312 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5482225640 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d38cf50 0x56464d38d330 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb666eabb6412133 server_cookie=ec4c3326cc374125 in_seq=0 out_seq=0 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f545d7fa640 1 -- 172.21.15.76:0/3056419616 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56464d125c00 con 0x56464d38cf50 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f545d7fa640 10 client.6312 handle_command_reply: tid=0 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 client.6312 shutdown 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 msgr2=0x56464d347cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d347cf0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e560 tx=0x7f546c00ea20 comp rx=0 tx=0).stop 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 shutdown_connections 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d38cf50 0x56464d38d330 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d348230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d347cf0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 --2- 172.21.15.76:0/3056419616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d341e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f545d7fa640 0 client.6312 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 >> 172.21.15.76:0/3056419616 conn(0x56464d33d290 msgr2=0x56464d33bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 shutdown_connections 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 1 -- 172.21.15.76:0/3056419616 wait complete. 2024-09-19T04:16:53.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.107+0000 7f5487545000 20 client.6312 trim_cache size 0 max 16384 2024-09-19T04:16:53.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 Processor -- start 2024-09-19T04:16:53.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- start start 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d342390 0x56464d385f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d386440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d39c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f547c200b50 con 0x56464d342390 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f547c13e020 con 0x56464d386b20 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f547c200500 con 0x56464d387440 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d342390 0x56464d385f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d342390 0x56464d385f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35252/0 (socket says 172.21.15.76:35252) 2024-09-19T04:16:53.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 -- 172.21.15.76:0/2480014009 learned_addr learned my addr 172.21.15.76:0/2480014009 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 -- 172.21.15.76:0/2480014009 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 msgr2=0x56464d39c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 --2- 172.21.15.76:0/2480014009 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d39c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 -- 172.21.15.76:0/2480014009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 msgr2=0x56464d386440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 --2- 172.21.15.76:0/2480014009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d386440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 -- 172.21.15.76:0/2480014009 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56464d387270 con 0x56464d342390 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482225640 1 --2- 172.21.15.76:0/2480014009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d342390 0x56464d385f00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5478097cc0 tx=0x7f5478004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f5138eb7da8df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f545f7fe640 1 -- 172.21.15.76:0/2480014009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780d0070 con 0x56464d342390 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f545f7fe640 1 -- 172.21.15.76:0/2480014009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54780b7460 con 0x56464d342390 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f545f7fe640 1 -- 172.21.15.76:0/2480014009 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54780b0040 con 0x56464d342390 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- 172.21.15.76:0/2480014009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d342390 msgr2=0x56464d385f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- 172.21.15.76:0/2480014009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d342390 0x56464d385f00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5478097cc0 tx=0x7f5478004740 comp rx=0 tx=0).stop 2024-09-19T04:16:53.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- 172.21.15.76:0/2480014009 shutdown_connections 2024-09-19T04:16:53.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- 172.21.15.76:0/2480014009 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d387440 0x56464d39c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- 172.21.15.76:0/2480014009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d386440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- 172.21.15.76:0/2480014009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d342390 0x56464d385f00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- 172.21.15.76:0/2480014009 >> 172.21.15.76:0/2480014009 conn(0x56464d33d290 msgr2=0x56464d3935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- 172.21.15.76:0/2480014009 shutdown_connections 2024-09-19T04:16:53.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- 172.21.15.76:0/2480014009 wait complete. 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- start start 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 Processor -- start 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d3fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d3facd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d390390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56464d18af80 con 0x56464d387440 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56464d143ca0 con 0x56464d386b20 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56464d1902a0 con 0x56464d342390 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5487545000 10 client.0 _ll_register_callbacks cb 0x7f546c00c250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:53.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d390390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482a26640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d390390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35264/0 (socket says 172.21.15.76:35264) 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.127+0000 7f5482a26640 1 -- 172.21.15.76:0/14994792 learned_addr learned my addr 172.21.15.76:0/14994792 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482225640 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d3fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482a26640 1 -- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 msgr2=0x56464d3fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482a26640 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d3fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482a26640 1 -- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 msgr2=0x56464d3facd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5481a24640 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d3facd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482a26640 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d3facd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482a26640 1 -- 172.21.15.76:0/14994792 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56464d112fd0 con 0x56464d387440 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482a26640 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d390390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f547401a000 tx=0x7f54740840d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e79ff9cfd13045c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:53.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 1 -- 172.21.15.76:0/14994792 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547406e0d0 con 0x56464d387440 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 1 -- 172.21.15.76:0/14994792 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f547405e7e0 con 0x56464d387440 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54780b7460 con 0x56464d387440 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 1 -- 172.21.15.76:0/14994792 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f547405ec70 con 0x56464d387440 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 10 client.6318 fetch_fsmap learned FSMap version 47 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) -- 0x56464d20e750 con 0x56464d387440 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 1 -- 172.21.15.76:0/14994792 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54780b7460 con 0x56464d387440 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 1 -- 172.21.15.76:0/14994792 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 47) ==== 3502+0+0 (secure 0 0 0) 0x7f547406f030 con 0x56464d387440 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 10 client.6318 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 10 client.6318 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:16:53.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 20 client.6318 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d3908d0 0x56464d390cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 4 client.6318 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "status"}) -- 0x56464d1993a0 con 0x56464d3908d0 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482225640 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d3908d0 0x56464d390cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 10 client.6318 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5482225640 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d3908d0 0x56464d390cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d621a0311b4da4 server_cookie=85f92ab23ae1c26b in_seq=0 out_seq=0 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 1 -- 172.21.15.76:0/14994792 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56464d1993a0 con 0x56464d3908d0 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 10 client.6318 handle_command_reply: tid=0 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 client.6318 shutdown 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 msgr2=0x56464d390390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d390390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f547401a000 tx=0x7f54740840d0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 shutdown_connections 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x56464d3908d0 0x56464d390cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56464d387440 0x56464d390390 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56464d386b20 0x56464d3facd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f545d7fa640 0 client.6318 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 --2- 172.21.15.76:0/14994792 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56464d342390 0x56464d3fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 >> 172.21.15.76:0/14994792 conn(0x56464d33d290 msgr2=0x56464d341860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 shutdown_connections 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 1 -- 172.21.15.76:0/14994792 wait complete. 2024-09-19T04:16:53.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5487545000 20 client.6318 trim_cache size 0 max 16384 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6165, 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 10.158658014, 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 47, 2024-09-19T04:16:53.145 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 67, 2024-09-19T04:16:53.146 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 67, 2024-09-19T04:16:53.146 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 15.034399885999999 2024-09-19T04:16:53.146 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:53.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54600678c0 msgr2=0x7f5460069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54600678c0 0x7f5460069d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54780ad000 tx=0x7f54780a6000 comp rx=0 tx=0).stop 2024-09-19T04:16:53.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 msgr2=0x7f547c15e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c15e460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f547400c780 tx=0x7f547400cc40 comp rx=0 tx=0).stop 2024-09-19T04:16:53.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 shutdown_connections 2024-09-19T04:16:53.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54600678c0 0x7f5460069d80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f547c1457a0 0x7f547c15e460 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f547c0cf800 0x7f547c15bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 --2- 172.21.15.76:0/485355963 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f547c0b4340 0x7f547c0b6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 >> 172.21.15.76:0/485355963 conn(0x7f547c06bcc0 msgr2=0x7f547c0b6290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 shutdown_connections 2024-09-19T04:16:53.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.131+0000 7f5484489640 1 -- 172.21.15.76:0/485355963 wait complete. 2024-09-19T04:16:53.152 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:16:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5edbda9640 1 Processor -- start 2024-09-19T04:16:53.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5edbda9640 1 -- start start 2024-09-19T04:16:53.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5edbda9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5ed4073e60 0x7f5ed406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5edbda9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 0x7f5ed40b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5edbda9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ed40b4670 con 0x7f5ed406fef0 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5edbda9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5ed406ee50 con 0x7f5ed4073e60 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5edbda9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed406f130 con 0x7f5ed40b3d50 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5ed9344640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5ed406fef0 0x7f5ed415b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52052/0 (socket says 172.21.15.76:52052) 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.251+0000 7f5ed9344640 1 -- 172.21.15.76:0/475389484 learned_addr learned my addr 172.21.15.76:0/475389484 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5eda346640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 0x7f5ed40b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9b45640 1 --2- 172.21.15.76:0/475389484 >> v2:172.21.15.116:3300/0 conn(0x7f5ed4073e60 0x7f5ed406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1918387282 0 0) 0x7f5ed40b4670 con 0x7f5ed406fef0 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ec0004640 con 0x7f5ed406fef0 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1499471436 0 0) 0x7f5ec0004640 con 0x7f5ed406fef0 2024-09-19T04:16:53.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 msgr2=0x7f5ed40b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 0x7f5ed40b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 >> v2:172.21.15.116:3300/0 conn(0x7f5ed4073e60 msgr2=0x7f5ed406e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 --2- 172.21.15.76:0/475389484 >> v2:172.21.15.116:3300/0 conn(0x7f5ed4073e60 0x7f5ed406e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed406f410 con 0x7f5ed406fef0 2024-09-19T04:16:53.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5ec4001680 con 0x7f5ed406fef0 2024-09-19T04:16:53.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 >> v1:172.21.15.76:6789/0 conn(0x7f5ed406fef0 legacy=0x7f5ed415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ec0004ad0 0x7f5ec0004ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ec0005920 0x7f5ec0005da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 0x7f5ec00062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed406ee50 con 0x7f5ec0005920 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed406f130 con 0x7f5ec0004ad0 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec0003480 con 0x7f5ed40b3d50 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9344640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ec0005920 0x7f5ec0005da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9b45640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ec0004ad0 0x7f5ec0004ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5eda346640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 0x7f5ec00062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9344640 1 -- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 msgr2=0x7f5ec00062e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9344640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 0x7f5ec00062e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9344640 1 -- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ec0004ad0 msgr2=0x7f5ec0004ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9344640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ec0004ad0 0x7f5ec0004ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9344640 1 -- 172.21.15.76:0/475389484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec0004640 con 0x7f5ec0005920 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5eda346640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ed40b3d50 0x7f5ec00062e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed9344640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ec0005920 0x7f5ec0005da0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c290 tx=0x7f5ec401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16822b561dcde199 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4007d60 con 0x7f5ec0005920 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec40a9020 con 0x7f5ec0005920 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5ed8b43640 1 -- 172.21.15.76:0/475389484 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4005b20 con 0x7f5ec0005920 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5edbda9640 1 -- 172.21.15.76:0/475389484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ec0005920 msgr2=0x7f5ec0005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5edbda9640 1 --2- 172.21.15.76:0/475389484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ec0005920 0x7f5ec0005da0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c290 tx=0x7f5ec401ccc0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5eda346640 1 -- 172.21.15.76:0/475389484 reap_dead start 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5edbda9640 1 -- 172.21.15.76:0/475389484 shutdown_connections 2024-09-19T04:16:53.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5edbda9640 1 -- 172.21.15.76:0/475389484 >> 172.21.15.76:0/475389484 conn(0x7f5ed40b1f90 msgr2=0x7f5ed40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5edbda9640 1 -- 172.21.15.76:0/475389484 shutdown_connections 2024-09-19T04:16:53.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.255+0000 7f5edbda9640 1 -- 172.21.15.76:0/475389484 wait complete. 2024-09-19T04:16:53.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 Processor -- start 2024-09-19T04:16:53.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 -- start start 2024-09-19T04:16:53.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ec0005920 0x7f5ed40999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ed415b2c0 0x7f5ed414f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ed4159840 0x7f5ed4159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed40729a0 con 0x7f5ed415b2c0 2024-09-19T04:16:53.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4072820 con 0x7f5ed4159840 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ed4072b20 con 0x7f5ec0005920 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ed415b2c0 0x7f5ed414f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ed415b2c0 0x7f5ed414f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35276/0 (socket says 172.21.15.76:35276) 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 -- 172.21.15.76:0/1486743006 learned_addr learned my addr 172.21.15.76:0/1486743006 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9b45640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ec0005920 0x7f5ed40999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5eda346640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ed4159840 0x7f5ed4159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 -- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ec0005920 msgr2=0x7f5ed40999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ec0005920 0x7f5ed40999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 -- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ed4159840 msgr2=0x7f5ed4159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ed4159840 0x7f5ed4159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 -- 172.21.15.76:0/1486743006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed4152bb0 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5ed9344640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ed415b2c0 0x7f5ed414f1e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c290 tx=0x7f5ec0006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5eca7fc640 1 -- 172.21.15.76:0/1486743006 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4001640 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5eca7fc640 1 -- 172.21.15.76:0/1486743006 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec40017e0 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed4152e90 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5eca7fc640 1 -- 172.21.15.76:0/1486743006 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40049f0 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed4152860 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed40b4650 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.259+0000 7f5eca7fc640 1 -- 172.21.15.76:0/1486743006 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ec4004b90 con 0x7f5ed415b2c0 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.263+0000 7f5eca7fc640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5eb80678f0 0x7f5eb8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:53.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.263+0000 7f5ed9b45640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5eb80678f0 0x7f5eb8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:53.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.263+0000 7f5eca7fc640 1 -- 172.21.15.76:0/1486743006 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f5ec411f390 con 0x7f5ed415b2c0 2024-09-19T04:16:53.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.263+0000 7f5ed9b45640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5eb80678f0 0x7f5eb8069db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009fc0 tx=0x7f5ecc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:53.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.267+0000 7f5eca7fc640 1 -- 172.21.15.76:0/1486743006 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5ec4016a30 con 0x7f5ed415b2c0 2024-09-19T04:16:53.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.463+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5ed40723f0 con 0x7f5ed415b2c0 2024-09-19T04:16:53.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.463+0000 7f5eca7fc640 1 -- 172.21.15.76:0/1486743006 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) ==== 53+0+73 (secure 0 0 0) 0x7f5ec40efdd0 con 0x7f5ed415b2c0 2024-09-19T04:16:53.468 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5eb80678f0 msgr2=0x7f5eb8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5eb80678f0 0x7f5eb8069db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc009fc0 tx=0x7f5ecc00b040 comp rx=0 tx=0).stop 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ed415b2c0 msgr2=0x7f5ed414f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ed415b2c0 0x7f5ed414f1e0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400c290 tx=0x7f5ec0006950 comp rx=0 tx=0).stop 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 shutdown_connections 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5eb80678f0 0x7f5eb8069db0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ed4159840 0x7f5ed4159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ed415b2c0 0x7f5ed414f1e0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 --2- 172.21.15.76:0/1486743006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ec0005920 0x7f5ed40999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 >> 172.21.15.76:0/1486743006 conn(0x7f5ed40b1f90 msgr2=0x7f5ed4146620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 shutdown_connections 2024-09-19T04:16:53.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53.467+0000 7f5edbda9640 1 -- 172.21.15.76:0/1486743006 wait complete. 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:16:53.486 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:16:53.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:53.487 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:16:53.492 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:16:53.492 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:16:53.492 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:16:53.492 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.492 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:16:53.493 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42307sec preferred_lft 42307sec 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:16:53.494 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T04:16:53.495 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:16:53.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:53.496 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:16:53.543 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:16:53.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:53.545 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:16:53.581 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:16:53.592 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:16:53.598 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:16:53.598 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:16:53.598 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:16:53.599 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:16:53.599 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:16:53.599 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:16:53.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:16:53.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:16:53.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:16:53.600 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:16:53.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:16:53.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:53.715 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:53.716 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:16:53.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T04:16:53.762 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:16:53.762 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:16:53.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:53.763 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:16:53.818 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T04:16:53.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:16:53.818 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:53.866 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:16:53.866 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:16:53.866 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:53.914 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:16:53.971 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:16:54.023 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:16:54.023 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:16:54.023 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:16:54.023 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:16:54.024 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:16:54.025 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:16:54.026 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:16:54.026 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:16:54.026 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:16:54.026 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:16:54.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:54.027 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:16:54.075 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:16:54.075 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T04:16:54.119 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:16:54.134 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:16:54.182 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:16:54.183 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:16:54.184 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:16:54.185 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:16:54.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:54.186 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:16:54.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-09-19T04:16:54.199+0000 7f88def0a580 -1 init, newargv = 0x5579c1fd4590 newargc=15 2024-09-19T04:16:54.203 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[42588]: starting ceph client 2024-09-19T04:16:54.208 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[42588]: starting fuse 2024-09-19T04:16:55.307 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:16:55.324 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:16:55.375 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:16:55.375 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:16:55.375 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:16:55.375 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:16:55.376 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:16:55.377 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:16:55.378 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:16:55.379 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:16:55.379 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:55.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:55.381 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:16:55.430 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:16:55.430 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:16:55.430 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:16:55.431 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-09-19T04:16:55.432 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:16:55.552 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.42588.asok 2024-09-19T04:16:55.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:16:55.565 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42588.asok 2024-09-19T04:16:55.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:55.565 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42588.asok status 2024-09-19T04:16:55.719 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:16:55.719 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "42588", 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6330, 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:16:55.720 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6330 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2201784578 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2201784578 2024-09-19T04:16:55.721 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.6330 192.168.0.1:0/2201784578", 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/2201784578", 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 47, 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 67, 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:16:55.722 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:55.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:55.733 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:16:55.733 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:55.739 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:16:55.739 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:55.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:16:55.740 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:16:55.795 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-19T04:16:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d37663640 1 Processor -- start 2024-09-19T04:16:55.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d37663640 1 -- start start 2024-09-19T04:16:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d37663640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d301520a0 0x7f4d3006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d37663640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d30070250 con 0x7f4d3006fe50 2024-09-19T04:16:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d37663640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d300aede0 con 0x7f4d3015f3a0 2024-09-19T04:16:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d37663640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4d30152480 con 0x7f4d301520a0 2024-09-19T04:16:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d353ff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4d301520a0 0x7f4d3006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d34bfe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4d3006fe50 0x7f4d3015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52058/0 (socket says 172.21.15.76:52058) 2024-09-19T04:16:55.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d34bfe640 1 -- 172.21.15.76:0/3276650462 learned_addr learned my addr 172.21.15.76:0/3276650462 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d353ff640 1 -- 172.21.15.76:0/3276650462 >> v1:172.21.15.116:6789/0 conn(0x7f4d3015f3a0 legacy=0x7f4d300adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d353ff640 1 -- 172.21.15.76:0/3276650462 >> v1:172.21.15.76:6789/0 conn(0x7f4d3006fe50 legacy=0x7f4d3015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:55.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.963+0000 7f4d353ff640 1 -- 172.21.15.76:0/3276650462 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3015fa10 con 0x7f4d301520a0 2024-09-19T04:16:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 --2- 172.21.15.76:0/3276650462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d301520a0 0x7f4d3006f910 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0097c0 tx=0x7f4d2c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5874c53752e3627 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:55.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d27fff640 1 -- 172.21.15.76:0/3276650462 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0b0070 con 0x7f4d301520a0 2024-09-19T04:16:55.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d27fff640 1 -- 172.21.15.76:0/3276650462 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2c004230 con 0x7f4d301520a0 2024-09-19T04:16:55.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d27fff640 1 -- 172.21.15.76:0/3276650462 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d300aeba0 con 0x7f4d301520a0 2024-09-19T04:16:55.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- 172.21.15.76:0/3276650462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d301520a0 msgr2=0x7f4d3006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:55.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 --2- 172.21.15.76:0/3276650462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d301520a0 0x7f4d3006f910 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c0097c0 tx=0x7f4d2c01c460 comp rx=0 tx=0).stop 2024-09-19T04:16:55.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- 172.21.15.76:0/3276650462 shutdown_connections 2024-09-19T04:16:55.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 --2- 172.21.15.76:0/3276650462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d301520a0 0x7f4d3006f910 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:55.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- 172.21.15.76:0/3276650462 >> 172.21.15.76:0/3276650462 conn(0x7f4d300b1f90 msgr2=0x7f4d300b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:55.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- 172.21.15.76:0/3276650462 shutdown_connections 2024-09-19T04:16:55.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- 172.21.15.76:0/3276650462 wait complete. 2024-09-19T04:16:55.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 Processor -- start 2024-09-19T04:16:55.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- start start 2024-09-19T04:16:55.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3006fe50 0x7f4d3014e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d301520a0 0x7f4d3015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d3015f3a0 0x7f4d3015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30073d30 con 0x7f4d3006fe50 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d30073a50 con 0x7f4d3015f3a0 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d37663640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4d30168f20 con 0x7f4d301520a0 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3006fe50 0x7f4d3014e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3006fe50 0x7f4d3014e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35294/0 (socket says 172.21.15.76:35294) 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 -- 172.21.15.76:0/3706416429 learned_addr learned my addr 172.21.15.76:0/3706416429 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d34bfe640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d301520a0 0x7f4d3015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d35c00640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d3015f3a0 0x7f4d3015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 -- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d301520a0 msgr2=0x7f4d3015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d301520a0 0x7f4d3015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:55.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 -- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d3015f3a0 msgr2=0x7f4d3015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d3015f3a0 0x7f4d3015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 -- 172.21.15.76:0/3706416429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d300b1db0 con 0x7f4d3006fe50 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d35c00640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d3015f3a0 0x7f4d3015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.967+0000 7f4d353ff640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3006fe50 0x7f4d3014e930 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009710 tx=0x7f4d2c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d25ffb640 1 -- 172.21.15.76:0/3706416429 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0b0040 con 0x7f4d3006fe50 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d25ffb640 1 -- 172.21.15.76:0/3706416429 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d2c0bb070 con 0x7f4d3006fe50 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d300b0d10 con 0x7f4d3006fe50 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d25ffb640 1 -- 172.21.15.76:0/3706416429 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0b5480 con 0x7f4d3006fe50 2024-09-19T04:16:55.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d300b12d0 con 0x7f4d3006fe50 2024-09-19T04:16:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d3006f910 con 0x7f4d3006fe50 2024-09-19T04:16:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d25ffb640 1 -- 172.21.15.76:0/3706416429 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d2c0b3070 con 0x7f4d3006fe50 2024-09-19T04:16:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d25ffb640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d140678c0 0x7f4d14069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d25ffb640 1 -- 172.21.15.76:0/3706416429 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f4d2c11fd50 con 0x7f4d3006fe50 2024-09-19T04:16:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d34bfe640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d140678c0 0x7f4d14069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:55.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.971+0000 7f4d34bfe640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d140678c0 0x7f4d14069d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4d3015bcf0 tx=0x7f4d20009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:55.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:55.975+0000 7f4d25ffb640 1 -- 172.21.15.76:0/3706416429 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4d2c123030 con 0x7f4d3006fe50 2024-09-19T04:16:56.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:56.175+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f4d3006e540 con 0x7f4d3006fe50 2024-09-19T04:16:57.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.027+0000 7f4d25ffb640 1 -- 172.21.15.76:0/3706416429 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v48) ==== 105+0+0 (secure 0 0 0) 0x7f4d300b1db0 con 0x7f4d3006fe50 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d140678c0 msgr2=0x7f4d14069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d140678c0 0x7f4d14069d80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4d3015bcf0 tx=0x7f4d20009040 comp rx=0 tx=0).stop 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3006fe50 msgr2=0x7f4d3014e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3006fe50 0x7f4d3014e930 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009710 tx=0x7f4d2c019200 comp rx=0 tx=0).stop 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 shutdown_connections 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d140678c0 0x7f4d14069d80 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d3015f3a0 0x7f4d3015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d301520a0 0x7f4d3015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 --2- 172.21.15.76:0/3706416429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d3006fe50 0x7f4d3014e930 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 >> 172.21.15.76:0/3706416429 conn(0x7f4d300b1f90 msgr2=0x7f4d300b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:57.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 shutdown_connections 2024-09-19T04:16:57.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.035+0000 7f4d37663640 1 -- 172.21.15.76:0/3706416429 wait complete. 2024-09-19T04:16:57.066 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:57.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e3140640 1 Processor -- start 2024-09-19T04:16:57.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e3140640 1 -- start start 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e3140640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f31dc070020 0x7f31dc15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e3140640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31dc15f1d0 0x7f31dc0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e3140640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31dc0aede0 con 0x7f31dc1520a0 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e3140640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f31dc152480 con 0x7f31dc15f1d0 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e3140640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f31dc15fa10 con 0x7f31dc070020 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e16dd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f31dc15f1d0 0x7f31dc0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e0edc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f31dc1520a0 0x7f31dc06f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52066/0 (socket says 172.21.15.76:52066) 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e0edc640 1 -- 172.21.15.76:0/2596107795 learned_addr learned my addr 172.21.15.76:0/2596107795 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:57.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e16dd640 1 -- 172.21.15.76:0/2596107795 >> v2:172.21.15.116:3300/0 conn(0x7f31dc070020 msgr2=0x7f31dc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e16dd640 1 --2- 172.21.15.76:0/2596107795 >> v2:172.21.15.116:3300/0 conn(0x7f31dc070020 0x7f31dc15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e16dd640 1 -- 172.21.15.76:0/2596107795 >> v1:172.21.15.76:6789/0 conn(0x7f31dc1520a0 legacy=0x7f31dc06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e16dd640 1 -- 172.21.15.76:0/2596107795 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc074250 con 0x7f31dc15f1d0 2024-09-19T04:16:57.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31db7fe640 1 -- 172.21.15.76:0/2596107795 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546415527 0 0) 0x7f31dc0aede0 con 0x7f31dc1520a0 2024-09-19T04:16:57.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31e16dd640 1 --2- 172.21.15.76:0/2596107795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc15f1d0 0x7f31dc0adc80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f31dc06e510 tx=0x7f31d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12536146f6e1c7f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:57.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.179+0000 7f31db7fe640 1 -- 172.21.15.76:0/2596107795 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40ab650 con 0x7f31dc15f1d0 2024-09-19T04:16:57.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31db7fe640 1 -- 172.21.15.76:0/2596107795 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d40b0070 con 0x7f31dc15f1d0 2024-09-19T04:16:57.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31db7fe640 1 -- 172.21.15.76:0/2596107795 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d40aba10 con 0x7f31dc15f1d0 2024-09-19T04:16:57.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- 172.21.15.76:0/2596107795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc15f1d0 msgr2=0x7f31dc0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 --2- 172.21.15.76:0/2596107795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc15f1d0 0x7f31dc0adc80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f31dc06e510 tx=0x7f31d401c450 comp rx=0 tx=0).stop 2024-09-19T04:16:57.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- 172.21.15.76:0/2596107795 shutdown_connections 2024-09-19T04:16:57.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 --2- 172.21.15.76:0/2596107795 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc15f1d0 0x7f31dc0adc80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 --2- 172.21.15.76:0/2596107795 >> v2:172.21.15.116:3300/0 conn(0x7f31dc070020 0x7f31dc15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- 172.21.15.76:0/2596107795 >> 172.21.15.76:0/2596107795 conn(0x7f31dc0b1f90 msgr2=0x7f31dc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:57.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- 172.21.15.76:0/2596107795 shutdown_connections 2024-09-19T04:16:57.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- 172.21.15.76:0/2596107795 wait complete. 2024-09-19T04:16:57.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 Processor -- start 2024-09-19T04:16:57.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- start start 2024-09-19T04:16:57.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31dc070020 0x7f31dc15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:57.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc1520a0 0x7f31dc15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:57.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31dc15f1d0 0x7f31dc14a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:57.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc15f680 con 0x7f31dc1520a0 2024-09-19T04:16:57.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc073d30 con 0x7f31dc15f1d0 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f31dc167f10 con 0x7f31dc070020 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc1520a0 0x7f31dc15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc1520a0 0x7f31dc15dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35310/0 (socket says 172.21.15.76:35310) 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 -- 172.21.15.76:0/4240935719 learned_addr learned my addr 172.21.15.76:0/4240935719 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e0edc640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31dc070020 0x7f31dc15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e16dd640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31dc15f1d0 0x7f31dc14a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 -- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31dc070020 msgr2=0x7f31dc15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31dc070020 0x7f31dc15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 -- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31dc15f1d0 msgr2=0x7f31dc14a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31dc15f1d0 0x7f31dc14a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 -- 172.21.15.76:0/4240935719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc14bc50 con 0x7f31dc1520a0 2024-09-19T04:16:57.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e16dd640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31dc15f1d0 0x7f31dc14a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:57.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31dbfff640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc1520a0 0x7f31dc15dc40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b3f0 tx=0x7f31d000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:57.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31d97fa640 1 -- 172.21.15.76:0/4240935719 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0015020 con 0x7f31dc1520a0 2024-09-19T04:16:57.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31d97fa640 1 -- 172.21.15.76:0/4240935719 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31d0011850 con 0x7f31dc1520a0 2024-09-19T04:16:57.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc14f370 con 0x7f31dc1520a0 2024-09-19T04:16:57.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31d97fa640 1 -- 172.21.15.76:0/4240935719 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31d0011c40 con 0x7f31dc1520a0 2024-09-19T04:16:57.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.183+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc14f090 con 0x7f31dc1520a0 2024-09-19T04:16:57.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.187+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc15ec90 con 0x7f31dc1520a0 2024-09-19T04:16:57.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.187+0000 7f31d97fa640 1 -- 172.21.15.76:0/4240935719 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f31d0018020 con 0x7f31dc1520a0 2024-09-19T04:16:57.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.187+0000 7f31d97fa640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31c80678f0 0x7f31c8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:57.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.187+0000 7f31e0edc640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31c80678f0 0x7f31c8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:57.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.187+0000 7f31d97fa640 1 -- 172.21.15.76:0/4240935719 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f31d0085cb0 con 0x7f31dc1520a0 2024-09-19T04:16:57.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.187+0000 7f31e0edc640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31c80678f0 0x7f31c8069db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007fd0 tx=0x7f31cc007f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:57.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.191+0000 7f31d97fa640 1 -- 172.21.15.76:0/4240935719 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f31d001f020 con 0x7f31dc1520a0 2024-09-19T04:16:57.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.379+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31dc06e540 con 0x7f31dc1520a0 2024-09-19T04:16:57.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.379+0000 7f31d97fa640 1 -- 172.21.15.76:0/4240935719 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5246 (secure 0 0 0) 0x7f31d00172c0 con 0x7f31dc1520a0 2024-09-19T04:16:57.385 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:57.385 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":49,"btime":"2024-09-19T04:16:57:039241+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5272,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:16:57.039211+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6165,"mds_1":6156},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6156":{"gid":6156,"name":"c","rank":1,"incarnation":49,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[6165]},"id":4}]} 2024-09-19T04:16:57.385 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 49 2024-09-19T04:16:57.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31c80678f0 msgr2=0x7f31c8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31c80678f0 0x7f31c8069db0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f31cc007fd0 tx=0x7f31cc007f00 comp rx=0 tx=0).stop 2024-09-19T04:16:57.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc1520a0 msgr2=0x7f31dc15dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:57.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc1520a0 0x7f31dc15dc40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f31d000b3f0 tx=0x7f31d000b8b0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 shutdown_connections 2024-09-19T04:16:57.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31c80678f0 0x7f31c8069db0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f31dc15f1d0 0x7f31dc14a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f31dc1520a0 0x7f31dc15dc40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 --2- 172.21.15.76:0/4240935719 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31dc070020 0x7f31dc15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:57.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 >> 172.21.15.76:0/4240935719 conn(0x7f31dc0b1f90 msgr2=0x7f31dc151370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:57.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 shutdown_connections 2024-09-19T04:16:57.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:57.383+0000 7f31e3140640 1 -- 172.21.15.76:0/4240935719 wait complete. 2024-09-19T04:16:57.804 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:16:42.356083+0000', 'modified': '2024-09-19T04:16:57.039211+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6165, 'mds_1': 6156}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6156': {'gid': 6156, 'name': 'c', 'rank': 1, 'incarnation': 49, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.76:6835/2740411101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 2740411101}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 2740411101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6165': {'gid': 6165, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3052694229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3052694229}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3052694229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6165, 'qdb_cluster': [6165]} 2024-09-19T04:16:57.805 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6156:up:creating 2024-09-19T04:16:58.806 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:16:58.871 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:16:58.867+0000 7f9c43701640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T04:16:58.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efcec18c640 1 Processor -- start 2024-09-19T04:16:58.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efcec18c640 1 -- start start 2024-09-19T04:16:58.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efcec18c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efce415bec0 0x7efce40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:58.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efcec18c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efce415c2a0 0x7efce40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efcec18c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efce40b3290 con 0x7efce40af2a0 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efcec18c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7efce40b3570 con 0x7efce415bec0 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efcec18c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efce40b3850 con 0x7efce415c2a0 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9f28640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efce415bec0 0x7efce40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9f28640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efce415bec0 0x7efce40ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35326/0 (socket says 172.21.15.76:35326) 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9f28640 1 -- 172.21.15.76:0/4224829612 learned_addr learned my addr 172.21.15.76:0/4224829612 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9727640 1 --2- 172.21.15.76:0/4224829612 >> v2:172.21.15.116:3300/0 conn(0x7efce415c2a0 0x7efce40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9f28640 1 -- 172.21.15.76:0/4224829612 >> v2:172.21.15.116:3300/0 conn(0x7efce415c2a0 msgr2=0x7efce40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9f28640 1 --2- 172.21.15.76:0/4224829612 >> v2:172.21.15.116:3300/0 conn(0x7efce415c2a0 0x7efce40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9f28640 1 -- 172.21.15.76:0/4224829612 >> v1:172.21.15.116:6789/0 conn(0x7efce40af2a0 legacy=0x7efce415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:58.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9f28640 1 -- 172.21.15.76:0/4224829612 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce40b3b30 con 0x7efce415bec0 2024-09-19T04:16:58.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.935+0000 7efce9727640 1 --2- 172.21.15.76:0/4224829612 >> v2:172.21.15.116:3300/0 conn(0x7efce415c2a0 0x7efce40aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:16:58.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efce9f28640 1 --2- 172.21.15.76:0/4224829612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce40ad200 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efce40ae510 tx=0x7efcd801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aea3480a0939abd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:58.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efce8f26640 1 -- 172.21.15.76:0/4224829612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd80aa650 con 0x7efce415bec0 2024-09-19T04:16:58.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efce8f26640 1 -- 172.21.15.76:0/4224829612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcd80af070 con 0x7efce415bec0 2024-09-19T04:16:58.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efce8f26640 1 -- 172.21.15.76:0/4224829612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd80aaa10 con 0x7efce415bec0 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 -- 172.21.15.76:0/4224829612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 msgr2=0x7efce40ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 --2- 172.21.15.76:0/4224829612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce40ad200 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efce40ae510 tx=0x7efcd801c4f0 comp rx=0 tx=0).stop 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 -- 172.21.15.76:0/4224829612 shutdown_connections 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 --2- 172.21.15.76:0/4224829612 >> v2:172.21.15.116:3300/0 conn(0x7efce415c2a0 0x7efce40aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 --2- 172.21.15.76:0/4224829612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce40ad200 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 -- 172.21.15.76:0/4224829612 >> 172.21.15.76:0/4224829612 conn(0x7efce40b1f90 msgr2=0x7efce40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 -- 172.21.15.76:0/4224829612 shutdown_connections 2024-09-19T04:16:58.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 -- 172.21.15.76:0/4224829612 wait complete. 2024-09-19T04:16:58.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 Processor -- start 2024-09-19T04:16:58.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.939+0000 7efcec18c640 1 -- start start 2024-09-19T04:16:58.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efce40af2a0 0x7efce415d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:58.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce409b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:58.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efce415c2a0 0x7efce409a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efce40af6b0 con 0x7efce415bec0 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efce40ad740 con 0x7efce415c2a0 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efce4169000 con 0x7efce40af2a0 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce409b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce409b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35330/0 (socket says 172.21.15.76:35330) 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 -- 172.21.15.76:0/3565314358 learned_addr learned my addr 172.21.15.76:0/3565314358 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcea729640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efce415c2a0 0x7efce409a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9f28640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efce40af2a0 0x7efce415d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 -- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efce40af2a0 msgr2=0x7efce415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efce40af2a0 0x7efce415d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 -- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efce415c2a0 msgr2=0x7efce409a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efce415c2a0 0x7efce409a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:58.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 -- 172.21.15.76:0/3565314358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efce409ad20 con 0x7efce415bec0 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcea729640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efce415c2a0 0x7efce409a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9727640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce409b7c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efce000d830 tx=0x7efce000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcd67fc640 1 -- 172.21.15.76:0/3565314358 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce000b6c0 con 0x7efce415bec0 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcd67fc640 1 -- 172.21.15.76:0/3565314358 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce0019070 con 0x7efce415bec0 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efce415c810 con 0x7efce415bec0 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcd67fc640 1 -- 172.21.15.76:0/3565314358 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce000ba80 con 0x7efce415bec0 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efce415cdd0 con 0x7efce415bec0 2024-09-19T04:16:58.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efce40ad200 con 0x7efce415bec0 2024-09-19T04:16:58.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcd67fc640 1 -- 172.21.15.76:0/3565314358 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efce000bc20 con 0x7efce415bec0 2024-09-19T04:16:58.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcd67fc640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efcc80678c0 0x7efcc8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:58.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efce9f28640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efcc80678c0 0x7efcc8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:58.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.943+0000 7efcd67fc640 1 -- 172.21.15.76:0/3565314358 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7efce00861b0 con 0x7efce415bec0 2024-09-19T04:16:58.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.947+0000 7efce9f28640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efcc80678c0 0x7efcc8069d80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efcd80ae000 tx=0x7efcd80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:58.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:58.951+0000 7efcd67fc640 1 -- 172.21.15.76:0/3565314358 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efce0056bf0 con 0x7efce415bec0 2024-09-19T04:16:59.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.151+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efce406fcd0 con 0x7efce415bec0 2024-09-19T04:16:59.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.151+0000 7efcd67fc640 1 -- 172.21.15.76:0/3565314358 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5268 (secure 0 0 0) 0x7efce00564f0 con 0x7efce415bec0 2024-09-19T04:16:59.156 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:16:59.156 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":52,"btime":"2024-09-19T04:16:59:055490+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:16:59.055484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6165,"mds_1":6156,"mds_2":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"b","rank":2,"incarnation":51,"state":"up:active","state_seq":8,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6156":{"gid":6156,"name":"c","rank":1,"incarnation":49,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[5272,6156,6165]},"id":4}]} 2024-09-19T04:16:59.156 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 52 2024-09-19T04:16:59.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efcc80678c0 msgr2=0x7efcc8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efcc80678c0 0x7efcc8069d80 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7efcd80ae000 tx=0x7efcd80a7000 comp rx=0 tx=0).stop 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 msgr2=0x7efce409b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce409b7c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7efce000d830 tx=0x7efce000dcf0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 shutdown_connections 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efcc80678c0 0x7efcc8069d80 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efce415c2a0 0x7efce409a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efce415bec0 0x7efce409b7c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 --2- 172.21.15.76:0/3565314358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efce40af2a0 0x7efce415d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 >> 172.21.15.76:0/3565314358 conn(0x7efce40b1f90 msgr2=0x7efce40cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 shutdown_connections 2024-09-19T04:16:59.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.155+0000 7efcec18c640 1 -- 172.21.15.76:0/3565314358 wait complete. 2024-09-19T04:16:59.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:16:42.356083+0000', 'modified': '2024-09-19T04:16:59.055484+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6165, 'mds_1': 6156, 'mds_2': 5272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5272': {'gid': 5272, 'name': 'b', 'rank': 2, 'incarnation': 51, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.116:6833/1054899754', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 1054899754}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 1054899754}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6156': {'gid': 6156, 'name': 'c', 'rank': 1, 'incarnation': 49, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.76:6835/2740411101', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 2740411101}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 2740411101}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6165': {'gid': 6165, 'name': 'a', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3052694229', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3052694229}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3052694229}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6165, 'qdb_cluster': [5272, 6156, 6165]} 2024-09-19T04:16:59.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:16:59.579 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-19T04:16:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10fd1f640 1 Processor -- start 2024-09-19T04:16:59.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10fd1f640 1 -- start start 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10fd1f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10fd1f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff10815f3a0 0x7ff1080adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10fd1f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff108070250 con 0x7ff10806fe50 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10fd1f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff108152480 con 0x7ff10815f3a0 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10fd1f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1080aede0 con 0x7ff1081520a0 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10e2bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff10815f3a0 0x7ff1080adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10d2ba640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff10806fe50 0x7ff10815ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52082/0 (socket says 172.21.15.76:52082) 2024-09-19T04:16:59.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10d2ba640 1 -- 172.21.15.76:0/3909314171 learned_addr learned my addr 172.21.15.76:0/3909314171 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:16:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10cab9640 1 -- 172.21.15.76:0/3909314171 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2374205035 0 0) 0x7ff108070250 con 0x7ff10806fe50 2024-09-19T04:16:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10dabb640 1 --2- 172.21.15.76:0/3909314171 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.707+0000 7ff10cab9640 1 -- 172.21.15.76:0/3909314171 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0f8003530 con 0x7ff10806fe50 2024-09-19T04:16:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10e2bc640 1 -- 172.21.15.76:0/3909314171 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 msgr2=0x7ff10806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/3909314171 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10e2bc640 1 -- 172.21.15.76:0/3909314171 >> v1:172.21.15.76:6789/0 conn(0x7ff10806fe50 legacy=0x7ff10815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10e2bc640 1 -- 172.21.15.76:0/3909314171 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff10815fa10 con 0x7ff10815f3a0 2024-09-19T04:16:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/3909314171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff1080adc80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff10001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbf1542b51f39a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10cab9640 1 -- 172.21.15.76:0/3909314171 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000aa600 con 0x7ff10815f3a0 2024-09-19T04:16:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10cab9640 1 -- 172.21.15.76:0/3909314171 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1000af070 con 0x7ff10815f3a0 2024-09-19T04:16:59.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10cab9640 1 -- 172.21.15.76:0/3909314171 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000aa9c0 con 0x7ff10815f3a0 2024-09-19T04:16:59.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 -- 172.21.15.76:0/3909314171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 msgr2=0x7ff1080adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/3909314171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff1080adc80 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff10001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 -- 172.21.15.76:0/3909314171 shutdown_connections 2024-09-19T04:16:59.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/3909314171 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff1080adc80 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/3909314171 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 -- 172.21.15.76:0/3909314171 >> 172.21.15.76:0/3909314171 conn(0x7ff1080b1f90 msgr2=0x7ff1080b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 -- 172.21.15.76:0/3909314171 shutdown_connections 2024-09-19T04:16:59.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.711+0000 7ff10fd1f640 1 -- 172.21.15.76:0/3909314171 wait complete. 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 Processor -- start 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 -- start start 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff10806fe50 0x7ff10814ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10813c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff10814c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff108073d30 con 0x7ff10815f3a0 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff108073a50 con 0x7ff10806fe50 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff108167f10 con 0x7ff1081520a0 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff10814c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff10814c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35352/0 (socket says 172.21.15.76:35352) 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 -- 172.21.15.76:0/871844844 learned_addr learned my addr 172.21.15.76:0/871844844 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10dabb640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff10806fe50 0x7ff10814ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10813c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 -- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 msgr2=0x7ff10813c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10813c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 -- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff10806fe50 msgr2=0x7ff10814ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff10806fe50 0x7ff10814ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 -- 172.21.15.76:0/871844844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1080b1c10 con 0x7ff10815f3a0 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10dabb640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff10806fe50 0x7ff10814ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff10814c160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff100019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff1067fc640 1 -- 172.21.15.76:0/871844844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000aaba0 con 0x7ff10815f3a0 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff1067fc640 1 -- 172.21.15.76:0/871844844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1000aad40 con 0x7ff10815f3a0 2024-09-19T04:16:59.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff108164640 con 0x7ff10815f3a0 2024-09-19T04:16:59.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff1067fc640 1 -- 172.21.15.76:0/871844844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000be690 con 0x7ff10815f3a0 2024-09-19T04:16:59.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.715+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1080b18a0 con 0x7ff10815f3a0 2024-09-19T04:16:59.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff1067fc640 1 -- 172.21.15.76:0/871844844 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff1000a9070 con 0x7ff10815f3a0 2024-09-19T04:16:59.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff1067fc640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0ec0678c0 0x7ff0ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff1067fc640 1 -- 172.21.15.76:0/871844844 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff1001202f0 con 0x7ff10815f3a0 2024-09-19T04:16:59.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10dabb640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0ec0678c0 0x7ff0ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10dabb640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0ec0678c0 0x7ff0ec069d80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc007330 tx=0x7ff0fc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 Processor -- start 2024-09-19T04:16:59.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- start start 2024-09-19T04:16:59.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893cb2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893c68d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c69290 0x55a893c69670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a893b50b60 con 0x55a893cb3440 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a893ab7c70 con 0x55a893cb2b20 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a893a5ded0 con 0x55a893c69290 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893c68d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893c68d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35360/0 (socket says 172.21.15.76:35360) 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 -- 172.21.15.76:0/1865831355 learned_addr learned my addr 172.21.15.76:0/1865831355 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 -- 172.21.15.76:0/1865831355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c69290 msgr2=0x55a893c69670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c69290 0x55a893c69670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 -- 172.21.15.76:0/1865831355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 msgr2=0x55a893cb2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10dabb640 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893cb2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893cb2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 -- 172.21.15.76:0/1865831355 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a893a6fe50 con 0x55a893cb3440 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893c68d50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff10806e510 tx=0x7ff0f4001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e441e8cdb914a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff0eb7fe640 1 -- 172.21.15.76:0/1865831355 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4018070 con 0x55a893cb3440 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff0eb7fe640 1 -- 172.21.15.76:0/1865831355 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f4005aa0 con 0x55a893cb3440 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff0eb7fe640 1 -- 172.21.15.76:0/1865831355 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40070d0 con 0x55a893cb3440 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- 172.21.15.76:0/1865831355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 msgr2=0x55a893c68d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893c68d50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff10806e510 tx=0x7ff0f4001a40 comp rx=0 tx=0).stop 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- 172.21.15.76:0/1865831355 shutdown_connections 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c69290 0x55a893c69670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893c68d50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1865831355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893cb2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- 172.21.15.76:0/1865831355 >> 172.21.15.76:0/1865831355 conn(0x55a893c6f600 msgr2=0x55a893c67630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- 172.21.15.76:0/1865831355 shutdown_connections 2024-09-19T04:16:59.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.719+0000 7ff112ddb000 1 -- 172.21.15.76:0/1865831355 wait complete. 2024-09-19T04:16:59.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 -- start start 2024-09-19T04:16:59.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 Processor -- start 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb2b20 0x55a893c6de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb3440 0x55a893c73cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893c74230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a893b19b00 con 0x55a893c6e390 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a893a92360 con 0x55a893cb3440 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a893a8f2d0 con 0x55a893cb2b20 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 10 client.0 _ll_register_callbacks cb 0x55a893c54820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:59.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893c74230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893c74230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35366/0 (socket says 172.21.15.76:35366) 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 -- 172.21.15.76:0/1289171368 learned_addr learned my addr 172.21.15.76:0/1289171368 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb3440 0x55a893c73cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10dabb640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb2b20 0x55a893c6de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 -- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb2b20 msgr2=0x55a893c6de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb2b20 0x55a893c6de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 -- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb3440 msgr2=0x55a893c73cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb3440 0x55a893c73cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 -- 172.21.15.76:0/1289171368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a893a5e120 con 0x55a893c6e390 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb3440 0x55a893c73cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893c74230 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff10001ca10 tx=0x7ff1000f0de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608c98885ab35a97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff0e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:59.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/1289171368 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000f9180 con 0x55a893c6e390 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/1289171368 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff1000ef9f0 con 0x55a893c6e390 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f4005aa0 con 0x55a893c6e390 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/1289171368 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff1000efc10 con 0x55a893c6e390 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 10 client.6369 fetch_fsmap learned FSMap version 52 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55a893a57450 con 0x55a893c6e390 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/1289171368 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0f4005aa0 con 0x55a893c6e390 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/1289171368 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 52) ==== 3542+0+0 (secure 0 0 0) 0x7ff1000c6030 con 0x55a893c6e390 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 10 client.6369 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 10 client.6369 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 20 client.6369 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893cc1280 0x55a893cc3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 4 client.6369 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 --> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a893a51c00 con 0x55a893cc1280 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10dabb640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893cc1280 0x55a893cc3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff0e8ff9640 10 client.6369 ms_handle_connect on v2:172.21.15.116:6832/1054899754 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.723+0000 7ff10dabb640 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893cc1280 0x55a893cc3670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3511ca0f718ee98 server_cookie=d6906f88eed6c52e in_seq=0 out_seq=0 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/1289171368 <== mds.2 v2:172.21.15.116:6832/1054899754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a893a51c00 con 0x55a893cc1280 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff0e8ff9640 10 client.6369 handle_command_reply: tid=0 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 client.6369 shutdown 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 msgr2=0x55a893c74230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893c74230 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff10001ca10 tx=0x7ff1000f0de0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 shutdown_connections 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893cc1280 0x55a893cc3670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893c74230 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb3440 0x55a893c73cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff0e8ff9640 0 client.6369 ms_handle_reset on v2:172.21.15.116:6832/1054899754 2024-09-19T04:16:59.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 --2- 172.21.15.76:0/1289171368 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb2b20 0x55a893c6de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 >> 172.21.15.76:0/1289171368 conn(0x55a893c69290 msgr2=0x55a893c67d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 shutdown_connections 2024-09-19T04:16:59.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 1 -- 172.21.15.76:0/1289171368 wait complete. 2024-09-19T04:16:59.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.727+0000 7ff112ddb000 20 client.6369 trim_cache size 0 max 16384 2024-09-19T04:16:59.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 Processor -- start 2024-09-19T04:16:59.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- start start 2024-09-19T04:16:59.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c6e390 0x55a893cb1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893cb2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893cc3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff108202f50 con 0x55a893cb3440 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff108203620 con 0x55a893cb2b20 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff10806f450 con 0x55a893c6e390 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893cc3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893cc3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35376/0 (socket says 172.21.15.76:35376) 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 -- 172.21.15.76:0/2419454786 learned_addr learned my addr 172.21.15.76:0/2419454786 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 -- 172.21.15.76:0/2419454786 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c6e390 msgr2=0x55a893cb1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/2419454786 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c6e390 0x55a893cb1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 -- 172.21.15.76:0/2419454786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 msgr2=0x55a893cb2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/2419454786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893cb2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 -- 172.21.15.76:0/2419454786 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a893a3efd0 con 0x55a893cb3440 2024-09-19T04:16:59.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff10e2bc640 1 --2- 172.21.15.76:0/2419454786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893cc3670 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff1000f0e50 tx=0x7ff100105000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94bf1736c85b7bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff0eaffd640 1 -- 172.21.15.76:0/2419454786 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1000f9180 con 0x55a893cb3440 2024-09-19T04:16:59.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff0eaffd640 1 -- 172.21.15.76:0/2419454786 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1000af020 con 0x55a893cb3440 2024-09-19T04:16:59.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff0eaffd640 1 -- 172.21.15.76:0/2419454786 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff10011e2f0 con 0x55a893cb3440 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- 172.21.15.76:0/2419454786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 msgr2=0x55a893cc3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 --2- 172.21.15.76:0/2419454786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893cc3670 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff1000f0e50 tx=0x7ff100105000 comp rx=0 tx=0).stop 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- 172.21.15.76:0/2419454786 shutdown_connections 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 --2- 172.21.15.76:0/2419454786 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893cb3440 0x55a893cc3670 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 --2- 172.21.15.76:0/2419454786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893cb2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 --2- 172.21.15.76:0/2419454786 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893c6e390 0x55a893cb1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- 172.21.15.76:0/2419454786 >> 172.21.15.76:0/2419454786 conn(0x55a893c69290 msgr2=0x55a893c6d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- 172.21.15.76:0/2419454786 shutdown_connections 2024-09-19T04:16:59.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- 172.21.15.76:0/2419454786 wait complete. 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 -- start start 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.747+0000 7ff112ddb000 1 Processor -- start 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893cc6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893ce1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb3440 0x55a893ce1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a893ab6f80 con 0x55a893c6e390 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a893a6fca0 con 0x55a893cb2b20 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a893abc2a0 con 0x55a893cb3440 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 10 client.0 _ll_register_callbacks cb 0x7ff100009420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893cc6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893cc6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35392/0 (socket says 172.21.15.76:35392) 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 -- 172.21.15.76:0/3341037762 learned_addr learned my addr 172.21.15.76:0/3341037762 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 -- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb3440 msgr2=0x55a893ce1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb3440 0x55a893ce1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 -- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 msgr2=0x55a893ce1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893ce1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893ce1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 -- 172.21.15.76:0/3341037762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a893b3a750 con 0x55a893c6e390 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10dabb640 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893cc6590 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55a893a7c0b0 tx=0x7ff0fc004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4042d0e5c4371b5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/3341037762 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc0268f0 con 0x55a893c6e390 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/3341037762 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff0fc026a90 con 0x55a893c6e390 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1000af020 con 0x55a893c6e390 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/3341037762 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7ff0fc035050 con 0x55a893c6e390 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 10 client.6375 fetch_fsmap learned FSMap version 52 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55a893a45a30 con 0x55a893c6e390 2024-09-19T04:16:59.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/3341037762 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1000af020 con 0x55a893c6e390 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/3341037762 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 52) ==== 3542+0+0 (secure 0 0 0) 0x7ff0fc038020 con 0x55a893c6e390 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 10 client.6375 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 10 client.6375 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 20 client.6375 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893ce23e0 0x55a893d25ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 4 client.6375 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 --> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] -- command(tid 0: {"prefix": "status"}) -- 0x55a893ac53a0 con 0x55a893ce23e0 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893ce23e0 0x55a893d25ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10d2ba640 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893ce23e0 0x55a893d25ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7fa176aa046757f3 server_cookie=f9953a4001f568a5 in_seq=0 out_seq=0 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 10 client.6375 ms_handle_connect on v2:172.21.15.116:6832/1054899754 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 1 -- 172.21.15.76:0/3341037762 <== mds.2 v2:172.21.15.116:6832/1054899754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x55a893ac53a0 con 0x55a893ce23e0 2024-09-19T04:16:59.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 10 client.6375 handle_command_reply: tid=0 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 client.6375 shutdown 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 msgr2=0x55a893cc6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893cc6590 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x55a893a7c0b0 tx=0x7ff0fc004a50 comp rx=0 tx=0).stop 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 shutdown_connections 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a893ce23e0 0x55a893d25ce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a893cb3440 0x55a893ce1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff0e8ff9640 0 client.6375 ms_handle_reset on v2:172.21.15.116:6832/1054899754 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a893cb2b20 0x55a893ce1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 --2- 172.21.15.76:0/3341037762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a893c6e390 0x55a893cc6590 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 >> 172.21.15.76:0/3341037762 conn(0x55a893c69290 msgr2=0x55a893cb0cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 shutdown_connections 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 1 -- 172.21.15.76:0/3341037762 wait complete. 2024-09-19T04:16:59.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff112ddb000 20 client.6375 trim_cache size 0 max 16384 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5272, 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 1.69932813, 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 52, 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 67, 2024-09-19T04:16:59.762 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 67, 2024-09-19T04:16:59.763 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 27.620673242999999 2024-09-19T04:16:59.763 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:59.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0ec0678c0 msgr2=0x7ff0ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0ec0678c0 0x7ff0ec069d80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc007330 tx=0x7ff0fc00b040 comp rx=0 tx=0).stop 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 msgr2=0x7ff10814c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff10814c160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff1000097c0 tx=0x7ff100019240 comp rx=0 tx=0).stop 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 shutdown_connections 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff0ec0678c0 0x7ff0ec069d80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff10815f3a0 0x7ff10814c160 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1081520a0 0x7ff10813c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 --2- 172.21.15.76:0/871844844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff10806fe50 0x7ff10814ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.751+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 >> 172.21.15.76:0/871844844 conn(0x7ff1080b1f90 msgr2=0x7ff10814f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.755+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 shutdown_connections 2024-09-19T04:16:59.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.755+0000 7ff10fd1f640 1 -- 172.21.15.76:0/871844844 wait complete. 2024-09-19T04:16:59.773 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:16:59.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4202df1640 1 Processor -- start 2024-09-19T04:16:59.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4202df1640 1 -- start start 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4202df1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f41fc073e60 0x7f41fc06e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4202df1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc0b3d50 0x7f41fc0b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4202df1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41fc0b4670 con 0x7f41fc06fef0 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4202df1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f41fc06ee50 con 0x7f41fc073e60 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4202df1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc06f130 con 0x7f41fc0b3d50 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4200b8d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f41fc073e60 0x7f41fc06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4200b8d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f41fc073e60 0x7f41fc06e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35406/0 (socket says 172.21.15.76:35406) 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4200b8d640 1 -- 172.21.15.76:0/3259634382 learned_addr learned my addr 172.21.15.76:0/3259634382 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f420138e640 1 --2- 172.21.15.76:0/3259634382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc0b3d50 0x7f41fc0b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4200b8d640 1 -- 172.21.15.76:0/3259634382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc0b3d50 msgr2=0x7f41fc0b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3259634382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc0b3d50 0x7f41fc0b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4200b8d640 1 -- 172.21.15.76:0/3259634382 >> v1:172.21.15.116:6789/0 conn(0x7f41fc06fef0 legacy=0x7f41fc15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.883+0000 7f4200b8d640 1 -- 172.21.15.76:0/3259634382 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc06f410 con 0x7f41fc073e60 2024-09-19T04:16:59.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3259634382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc073e60 0x7f41fc06e800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00aa10 tx=0x7f41ec01d860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8dd46b08b4e82b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f41fb7fe640 1 -- 172.21.15.76:0/3259634382 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec00b040 con 0x7f41fc073e60 2024-09-19T04:16:59.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f41fb7fe640 1 -- 172.21.15.76:0/3259634382 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ec0a7710 con 0x7f41fc073e60 2024-09-19T04:16:59.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f41fb7fe640 1 -- 172.21.15.76:0/3259634382 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0a7ae0 con 0x7f41fc073e60 2024-09-19T04:16:59.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 -- 172.21.15.76:0/3259634382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc073e60 msgr2=0x7f41fc06e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 --2- 172.21.15.76:0/3259634382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc073e60 0x7f41fc06e800 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f41ec00aa10 tx=0x7f41ec01d860 comp rx=0 tx=0).stop 2024-09-19T04:16:59.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 -- 172.21.15.76:0/3259634382 shutdown_connections 2024-09-19T04:16:59.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 --2- 172.21.15.76:0/3259634382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc0b3d50 0x7f41fc0b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 --2- 172.21.15.76:0/3259634382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc073e60 0x7f41fc06e800 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 -- 172.21.15.76:0/3259634382 >> 172.21.15.76:0/3259634382 conn(0x7f41fc0b1f90 msgr2=0x7f41fc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 -- 172.21.15.76:0/3259634382 shutdown_connections 2024-09-19T04:16:59.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.887+0000 7f4202df1640 1 -- 172.21.15.76:0/3259634382 wait complete. 2024-09-19T04:16:59.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 Processor -- start 2024-09-19T04:16:59.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 -- start start 2024-09-19T04:16:59.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc06fef0 0x7f41fc156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41fc073e60 0x7f41fc154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc0b3d50 0x7f41fc14de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0729a0 con 0x7f41fc0b3d50 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc072820 con 0x7f41fc073e60 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc072b20 con 0x7f41fc06fef0 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc0b3d50 0x7f41fc14de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc0b3d50 0x7f41fc14de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35416/0 (socket says 172.21.15.76:35416) 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 -- 172.21.15.76:0/2490824982 learned_addr learned my addr 172.21.15.76:0/2490824982 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4200b8d640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc06fef0 0x7f41fc156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f41fbfff640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41fc073e60 0x7f41fc154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 -- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc06fef0 msgr2=0x7f41fc156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc06fef0 0x7f41fc156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 -- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41fc073e60 msgr2=0x7f41fc154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41fc073e60 0x7f41fc154190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 -- 172.21.15.76:0/2490824982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc152640 con 0x7f41fc0b3d50 2024-09-19T04:16:59.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f420138e640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc0b3d50 0x7f41fc14de70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f41f4007e40 tx=0x7f41f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f41f97fa640 1 -- 172.21.15.76:0/2490824982 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4014620 con 0x7f41fc0b3d50 2024-09-19T04:16:59.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f41f97fa640 1 -- 172.21.15.76:0/2490824982 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4019070 con 0x7f41fc0b3d50 2024-09-19T04:16:59.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc152920 con 0x7f41fc0b3d50 2024-09-19T04:16:59.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f41f97fa640 1 -- 172.21.15.76:0/2490824982 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4014a20 con 0x7f41fc0b3d50 2024-09-19T04:16:59.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc14e3b0 con 0x7f41fc0b3d50 2024-09-19T04:16:59.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.891+0000 7f41f97fa640 1 -- 172.21.15.76:0/2490824982 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f41f4014bc0 con 0x7f41fc0b3d50 2024-09-19T04:16:59.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f41f97fa640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f41e80678c0 0x7f41e8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f41f97fa640 1 -- 172.21.15.76:0/2490824982 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f41f4086650 con 0x7f41fc0b3d50 2024-09-19T04:16:59.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4200b8d640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f41e80678c0 0x7f41e8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4200b8d640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f41e80678c0 0x7f41e8069d80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0b0000 tx=0x7f41ec0a8000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 Processor -- start 2024-09-19T04:16:59.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- start start 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a0af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c849c0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c1290 0x563c849c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563c848a8b60 con 0x563c849c1290 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563c8480fc70 con 0x563c84a0ab20 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x563c847b5ed0 con 0x563c84a0b440 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c1290 0x563c849c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c1290 0x563c849c1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35426/0 (socket says 172.21.15.76:35426) 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 -- 172.21.15.76:0/2323235228 learned_addr learned my addr 172.21.15.76:0/2323235228 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4200b8d640 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a0af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f41fbfff640 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c849c0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 -- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 msgr2=0x563c849c0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c849c0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 -- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 msgr2=0x563c84a0af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a0af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 -- 172.21.15.76:0/2323235228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c847c7e50 con 0x563c849c1290 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f420138e640 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c1290 0x563c849c1670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f41f405efb0 tx=0x7f41f40570c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76ee43d9cd99ed41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f41e27fc640 1 -- 172.21.15.76:0/2323235228 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f405f4e0 con 0x563c849c1290 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f41e27fc640 1 -- 172.21.15.76:0/2323235228 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4054c00 con 0x563c849c1290 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f41e27fc640 1 -- 172.21.15.76:0/2323235228 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f405d070 con 0x563c849c1290 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- 172.21.15.76:0/2323235228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c1290 msgr2=0x563c849c1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c1290 0x563c849c1670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f41f405efb0 tx=0x7f41f40570c0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- 172.21.15.76:0/2323235228 shutdown_connections 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c1290 0x563c849c1670 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c849c0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 --2- 172.21.15.76:0/2323235228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a0af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- 172.21.15.76:0/2323235228 >> 172.21.15.76:0/2323235228 conn(0x563c849c7600 msgr2=0x563c849bf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- 172.21.15.76:0/2323235228 shutdown_connections 2024-09-19T04:16:59.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- 172.21.15.76:0/2323235228 wait complete. 2024-09-19T04:16:59.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.895+0000 7f4205ead000 1 -- start start 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 Processor -- start 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c849c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0b440 0x563c849cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c849cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563c84871b00 con 0x563c84a0b440 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563c847ea360 con 0x563c84a0ab20 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x563c847e72d0 con 0x563c849c6390 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 10 client.0 _ll_register_callbacks cb 0x563c849ac820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0b440 0x563c849cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0b440 0x563c849cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35434/0 (socket says 172.21.15.76:35434) 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 -- 172.21.15.76:0/3599468084 learned_addr learned my addr 172.21.15.76:0/3599468084 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c849c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f420138e640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c849cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 -- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 msgr2=0x563c849cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c849cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 -- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 msgr2=0x563c849c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c849c5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 -- 172.21.15.76:0/3599468084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c847b6120 con 0x563c84a0b440 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f420138e640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c849cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41fbfff640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0b440 0x563c849cbcf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f41f000df10 tx=0x7f41f000f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41c485169321688 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 1 -- 172.21.15.76:0/3599468084 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0018030 con 0x563c84a0b440 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 1 -- 172.21.15.76:0/3599468084 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f41f0004020 con 0x563c84a0b440 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f4054c00 con 0x563c84a0b440 2024-09-19T04:16:59.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 1 -- 172.21.15.76:0/3599468084 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f41f0021050 con 0x563c84a0b440 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 10 client.6387 fetch_fsmap learned FSMap version 52 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x563c847af450 con 0x563c84a0b440 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 1 -- 172.21.15.76:0/3599468084 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4054c00 con 0x563c84a0b440 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 1 -- 172.21.15.76:0/3599468084 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 52) ==== 3542+0+0 (secure 0 0 0) 0x7f41f0024020 con 0x563c84a0b440 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 10 client.6387 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 10 client.6387 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 20 client.6387 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a19280 0x563c84a1b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 4 client.6387 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 --> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c847a9c00 con 0x563c84a19280 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a19280 0x563c84a1b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a19280 0x563c84a1b670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2a86dc2fcda8a326 server_cookie=a6e16986962b0cbf in_seq=0 out_seq=0 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 10 client.6387 ms_handle_connect on v2:172.21.15.76:6834/2740411101 2024-09-19T04:16:59.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 1 -- 172.21.15.76:0/3599468084 <== mds.1 v2:172.21.15.76:6834/2740411101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c847a9c00 con 0x563c84a19280 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 10 client.6387 handle_command_reply: tid=0 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 client.6387 shutdown 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0b440 msgr2=0x563c849cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0b440 0x563c849cbcf0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f41f000df10 tx=0x7f41f000f5b0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 shutdown_connections 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a19280 0x563c84a1b670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c849cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0b440 0x563c849cbcf0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f41db7fe640 0 client.6387 ms_handle_reset on v2:172.21.15.76:6834/2740411101 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 --2- 172.21.15.76:0/3599468084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c849c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 >> 172.21.15.76:0/3599468084 conn(0x563c849c1290 msgr2=0x563c849bfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 shutdown_connections 2024-09-19T04:16:59.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 1 -- 172.21.15.76:0/3599468084 wait complete. 2024-09-19T04:16:59.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.899+0000 7f4205ead000 20 client.6387 trim_cache size 0 max 16384 2024-09-19T04:16:59.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 Processor -- start 2024-09-19T04:16:59.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 -- start start 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c84a09cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0ab20 0x563c84a0a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0b440 0x563c84a1b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc200ec0 con 0x563c84a0ab20 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc201590 con 0x563c84a0b440 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f41fc09be60 con 0x563c849c6390 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0ab20 0x563c84a0a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0ab20 0x563c84a0a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35444/0 (socket says 172.21.15.76:35444) 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 -- 172.21.15.76:0/1638711315 learned_addr learned my addr 172.21.15.76:0/1638711315 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 -- 172.21.15.76:0/1638711315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 msgr2=0x563c84a09cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 --2- 172.21.15.76:0/1638711315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c84a09cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 -- 172.21.15.76:0/1638711315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0b440 msgr2=0x563c84a1b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 --2- 172.21.15.76:0/1638711315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0b440 0x563c84a1b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 -- 172.21.15.76:0/1638711315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c84796fd0 con 0x563c84a0ab20 2024-09-19T04:16:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41fbfff640 1 --2- 172.21.15.76:0/1638711315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0ab20 0x563c84a0a200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006cd0 tx=0x7f41f0009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ab939a30232fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41e1ffb640 1 -- 172.21.15.76:0/1638711315 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f002a070 con 0x563c84a0ab20 2024-09-19T04:16:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41e1ffb640 1 -- 172.21.15.76:0/1638711315 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f000fab0 con 0x563c84a0ab20 2024-09-19T04:16:59.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.919+0000 7f41e1ffb640 1 -- 172.21.15.76:0/1638711315 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0018070 con 0x563c84a0ab20 2024-09-19T04:16:59.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/1638711315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0ab20 msgr2=0x563c84a0a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- 172.21.15.76:0/1638711315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0ab20 0x563c84a0a200 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006cd0 tx=0x7f41f0009000 comp rx=0 tx=0).stop 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/1638711315 shutdown_connections 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- 172.21.15.76:0/1638711315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0b440 0x563c84a1b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- 172.21.15.76:0/1638711315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c84a0ab20 0x563c84a0a200 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- 172.21.15.76:0/1638711315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c849c6390 0x563c84a09cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/1638711315 >> 172.21.15.76:0/1638711315 conn(0x563c849c1290 msgr2=0x563c849c5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/1638711315 shutdown_connections 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/1638711315 wait complete. 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- start start 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 Processor -- start 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c6390 0x563c84a1e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a3dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c84a3e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563c8480ef80 con 0x563c849c6390 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563c847c7ca0 con 0x563c84a0ab20 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x563c848142a0 con 0x563c84a0b440 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 10 client.0 _ll_register_callbacks cb 0x7f41f000d220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c6390 0x563c84a1e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c6390 0x563c84a1e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35446/0 (socket says 172.21.15.76:35446) 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 -- 172.21.15.76:0/3910872149 learned_addr learned my addr 172.21.15.76:0/3910872149 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41fbfff640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a3dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 -- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 msgr2=0x563c84a3e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f420138e640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c84a3e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c84a3e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 -- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 msgr2=0x563c84a3dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a3dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 -- 172.21.15.76:0/3910872149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c84892750 con 0x563c849c6390 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41fbfff640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a3dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4200b8d640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c6390 0x563c84a1e590 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f41ec01d2d0 tx=0x7f41ec0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecd6122e0155b24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 1 -- 172.21.15.76:0/3910872149 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0b8af0 con 0x563c849c6390 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 1 -- 172.21.15.76:0/3910872149 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f41ec0b8c90 con 0x563c849c6390 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f000fab0 con 0x563c849c6390 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 1 -- 172.21.15.76:0/3910872149 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f41ec0d6050 con 0x563c849c6390 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 10 client.6393 fetch_fsmap learned FSMap version 52 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x563c8479da30 con 0x563c849c6390 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 1 -- 172.21.15.76:0/3910872149 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f000fab0 con 0x563c849c6390 2024-09-19T04:16:59.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 1 -- 172.21.15.76:0/3910872149 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 52) ==== 3542+0+0 (secure 0 0 0) 0x7f41ec0d9020 con 0x563c849c6390 2024-09-19T04:16:59.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 10 client.6393 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:16:59.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 10 client.6393 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:16:59.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 20 client.6393 populate_metadata read hostname 'smithi076' 2024-09-19T04:16:59.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a3e7f0 0x563c84a81ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:16:59.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 4 client.6393 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:16:59.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 --> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] -- command(tid 0: {"prefix": "status"}) -- 0x563c8481d3a0 con 0x563c84a3e7f0 2024-09-19T04:16:59.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41fbfff640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a3e7f0 0x563c84a81ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 10 client.6393 ms_handle_connect on v2:172.21.15.76:6834/2740411101 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41fbfff640 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a3e7f0 0x563c84a81ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f8beb2a27c8c9bc2 server_cookie=88158b29dc72cda0 in_seq=0 out_seq=0 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 1 -- 172.21.15.76:0/3910872149 <== mds.1 v2:172.21.15.76:6834/2740411101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x563c8481d3a0 con 0x563c84a3e7f0 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f41db7fe640 10 client.6393 handle_command_reply: tid=0 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 client.6393 shutdown 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c6390 msgr2=0x563c84a1e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.923+0000 7f4205ead000 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c6390 0x563c84a1e590 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f41ec01d2d0 tx=0x7f41ec0046d0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 shutdown_connections 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x563c84a3e7f0 0x563c84a81ee0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563c84a0b440 0x563c84a3e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563c84a0ab20 0x563c84a3dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f41db7fe640 0 client.6393 ms_handle_reset on v2:172.21.15.76:6834/2740411101 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 --2- 172.21.15.76:0/3910872149 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563c849c6390 0x563c84a1e590 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 >> 172.21.15.76:0/3910872149 conn(0x563c849c1290 msgr2=0x563c84a08cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 shutdown_connections 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 1 -- 172.21.15.76:0/3910872149 wait complete. 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4205ead000 20 client.6393 trim_cache size 0 max 16384 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-09-19T04:16:59.938 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6156, 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 2.8882246559999998, 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 52, 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 67, 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 67, 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 27.791015040000001 2024-09-19T04:16:59.939 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:16:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f41e80678c0 msgr2=0x7f41e8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f41e80678c0 0x7f41e8069d80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f41ec0b0000 tx=0x7f41ec0a8000 comp rx=0 tx=0).stop 2024-09-19T04:16:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc0b3d50 msgr2=0x7f41fc14de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:16:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc0b3d50 0x7f41fc14de70 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f41f4007e40 tx=0x7f41f400e4b0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 shutdown_connections 2024-09-19T04:16:59.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f41e80678c0 0x7f41e8069d80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f41fc0b3d50 0x7f41fc14de70 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f41fc073e60 0x7f41fc154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 --2- 172.21.15.76:0/2490824982 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f41fc06fef0 0x7f41fc156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:16:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 >> 172.21.15.76:0/2490824982 conn(0x7f41fc0b1f90 msgr2=0x7f41fc0d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:16:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 shutdown_connections 2024-09-19T04:16:59.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:16:59.927+0000 7f4202df1640 1 -- 172.21.15.76:0/2490824982 wait complete. 2024-09-19T04:16:59.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:17:00.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4c381640 1 Processor -- start 2024-09-19T04:17:00.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4c381640 1 -- start start 2024-09-19T04:17:00.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4c381640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a4406d3e0 0x7f6a440ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4c381640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a4415ccb0 con 0x7f6a440b5880 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4c381640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a440cd5b0 con 0x7f6a4415c1c0 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4c381640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6a440cd890 con 0x7f6a4406d3e0 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4a91e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a4406d3e0 0x7f6a440ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4a91e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a4406d3e0 0x7f6a440ce650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35450/0 (socket says 172.21.15.76:35450) 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4991c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6a440b5880 0x7f6a4406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52098/0 (socket says 172.21.15.76:52098) 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.051+0000 7f6a4a91e640 1 -- 172.21.15.76:0/2062124658 learned_addr learned my addr 172.21.15.76:0/2062124658 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4a91e640 1 -- 172.21.15.76:0/2062124658 >> v1:172.21.15.116:6789/0 conn(0x7f6a4415c1c0 legacy=0x7f6a4415c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4911b640 1 -- 172.21.15.76:0/2062124658 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3949623851 0 0) 0x7f6a4415ccb0 con 0x7f6a440b5880 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4a91e640 1 -- 172.21.15.76:0/2062124658 >> v1:172.21.15.76:6789/0 conn(0x7f6a440b5880 legacy=0x7f6a4406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4a91e640 1 -- 172.21.15.76:0/2062124658 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a440cdb70 con 0x7f6a4406d3e0 2024-09-19T04:17:00.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/2062124658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a440ce650 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009e50 tx=0x7f6a3c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a793db4e19f907a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4911b640 1 -- 172.21.15.76:0/2062124658 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0a5830 con 0x7f6a4406d3e0 2024-09-19T04:17:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4911b640 1 -- 172.21.15.76:0/2062124658 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a3c0a59d0 con 0x7f6a4406d3e0 2024-09-19T04:17:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4911b640 1 -- 172.21.15.76:0/2062124658 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0a5da0 con 0x7f6a4406d3e0 2024-09-19T04:17:00.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4c381640 1 -- 172.21.15.76:0/2062124658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 msgr2=0x7f6a440ce650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2062124658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a440ce650 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c009e50 tx=0x7f6a3c01c970 comp rx=0 tx=0).stop 2024-09-19T04:17:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4c381640 1 -- 172.21.15.76:0/2062124658 shutdown_connections 2024-09-19T04:17:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2062124658 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a440ce650 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4c381640 1 -- 172.21.15.76:0/2062124658 >> 172.21.15.76:0/2062124658 conn(0x7f6a44072670 msgr2=0x7f6a44072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4c381640 1 -- 172.21.15.76:0/2062124658 shutdown_connections 2024-09-19T04:17:00.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.055+0000 7f6a4c381640 1 -- 172.21.15.76:0/2062124658 wait complete. 2024-09-19T04:17:00.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 Processor -- start 2024-09-19T04:17:00.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 -- start start 2024-09-19T04:17:00.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a4414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a440b5880 0x7f6a4414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a4415c1c0 0x7f6a44157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440b4c20 con 0x7f6a4406d3e0 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a440b4aa0 con 0x7f6a4415c1c0 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4c381640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4416a610 con 0x7f6a440b5880 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a4414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a4414f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35462/0 (socket says 172.21.15.76:35462) 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 -- 172.21.15.76:0/2685784137 learned_addr learned my addr 172.21.15.76:0/2685784137 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4991c640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a440b5880 0x7f6a4414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a4415c1c0 0x7f6a44157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 -- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a440b5880 msgr2=0x7f6a4414fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a440b5880 0x7f6a4414fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 -- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a4415c1c0 msgr2=0x7f6a44157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a4415c1c0 0x7f6a44157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 -- 172.21.15.76:0/2685784137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a441509e0 con 0x7f6a4406d3e0 2024-09-19T04:17:00.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a4415c1c0 0x7f6a44157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:00.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.059+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a4414f6b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c500 tx=0x7f6a3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a36ffd640 1 -- 172.21.15.76:0/2685784137 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38016020 con 0x7f6a4406d3e0 2024-09-19T04:17:00.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a36ffd640 1 -- 172.21.15.76:0/2685784137 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38005220 con 0x7f6a4406d3e0 2024-09-19T04:17:00.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4415b0f0 con 0x7f6a4406d3e0 2024-09-19T04:17:00.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a36ffd640 1 -- 172.21.15.76:0/2685784137 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38007030 con 0x7f6a4406d3e0 2024-09-19T04:17:00.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a44150c50 con 0x7f6a4406d3e0 2024-09-19T04:17:00.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a36ffd640 1 -- 172.21.15.76:0/2685784137 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6a38007210 con 0x7f6a4406d3e0 2024-09-19T04:17:00.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a36ffd640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a280678c0 0x7f6a28069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a36ffd640 1 -- 172.21.15.76:0/2685784137 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f6a38085820 con 0x7f6a4406d3e0 2024-09-19T04:17:00.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a4991c640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a280678c0 0x7f6a28069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a4991c640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a280678c0 0x7f6a28069d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6a440af8c0 tx=0x7f6a40007d10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a4f43d000 1 Processor -- start 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.063+0000 7f6a4f43d000 1 -- start start 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e9131150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e90e7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e90e76a0 0x5590e90e7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5590e8fcab60 con 0x5590e9131690 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5590e8f31c70 con 0x5590e9130d70 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5590e8ed7ed0 con 0x5590e90e76a0 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e90e7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e90e7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35470/0 (socket says 172.21.15.76:35470) 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 -- 172.21.15.76:0/2815223602 learned_addr learned my addr 172.21.15.76:0/2815223602 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:00.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e9131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e90e76a0 0x5590e90e7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 -- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e90e76a0 msgr2=0x5590e90e7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e90e76a0 0x5590e90e7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 -- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 msgr2=0x5590e9131150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e9131150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 -- 172.21.15.76:0/2815223602 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e8ee9e50 con 0x5590e9131690 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e90e76a0 0x5590e90e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e90e7160 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6a440b5cc0 tx=0x7f6a40012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd5014155394dd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a27fff640 1 -- 172.21.15.76:0/2815223602 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a400076a0 con 0x5590e9131690 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a27fff640 1 -- 172.21.15.76:0/2815223602 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a40015110 con 0x5590e9131690 2024-09-19T04:17:00.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a27fff640 1 -- 172.21.15.76:0/2815223602 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40015b90 con 0x5590e9131690 2024-09-19T04:17:00.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- 172.21.15.76:0/2815223602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 msgr2=0x5590e90e7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e90e7160 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f6a440b5cc0 tx=0x7f6a40012000 comp rx=0 tx=0).stop 2024-09-19T04:17:00.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- 172.21.15.76:0/2815223602 shutdown_connections 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e90e76a0 0x5590e90e7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e90e7160 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/2815223602 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e9131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- 172.21.15.76:0/2815223602 >> 172.21.15.76:0/2815223602 conn(0x5590e90e9600 msgr2=0x5590e90e1630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- 172.21.15.76:0/2815223602 shutdown_connections 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- 172.21.15.76:0/2815223602 wait complete. 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- start start 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 Processor -- start 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9136eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e91373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911d490 0x5590e911d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5590e8f93b00 con 0x5590e9131690 2024-09-19T04:17:00.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5590e8f0c360 con 0x5590e911d490 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5590e8f092d0 con 0x5590e9130d70 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4f43d000 10 client.0 _ll_register_callbacks cb 0x5590e90ce820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e91373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e91373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35476/0 (socket says 172.21.15.76:35476) 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4991c640 1 -- 172.21.15.76:0/1231487431 learned_addr learned my addr 172.21.15.76:0/1231487431 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.067+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9136eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4991c640 1 -- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 msgr2=0x5590e9136eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911d490 0x5590e911d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4991c640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9136eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4991c640 1 -- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911d490 msgr2=0x5590e911d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4991c640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911d490 0x5590e911d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4991c640 1 -- 172.21.15.76:0/1231487431 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e8ed8120 con 0x5590e9131690 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4991c640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e91373f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6a40008fd0 tx=0x7f6a40007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f72be384e834e56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 1 -- 172.21.15.76:0/1231487431 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a40026070 con 0x5590e9131690 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a40015110 con 0x5590e9131690 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 1 -- 172.21.15.76:0/1231487431 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f6a4002b460 con 0x5590e9131690 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 1 -- 172.21.15.76:0/1231487431 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6a40033050 con 0x5590e9131690 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 10 client.6408 fetch_fsmap learned FSMap version 52 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5590e8ed1450 con 0x5590e9131690 2024-09-19T04:17:00.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 1 -- 172.21.15.76:0/1231487431 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a40015110 con 0x5590e9131690 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 1 -- 172.21.15.76:0/1231487431 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 52) ==== 3542+0+0 (secure 0 0 0) 0x7f6a40036020 con 0x5590e9131690 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 10 client.6408 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 10 client.6408 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 20 client.6408 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e911e280 0x5590e911e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 4 client.6408 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590e8ecbc00 con 0x5590e911e280 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e911e280 0x5590e911e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 10 client.6408 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e911e280 0x5590e911e660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30a1d490de8e5a19 server_cookie=5a1ba163af9435ef in_seq=0 out_seq=0 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 1 -- 172.21.15.76:0/1231487431 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5590e8ecbc00 con 0x5590e911e280 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 10 client.6408 handle_command_reply: tid=0 2024-09-19T04:17:00.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 client.6408 shutdown 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 msgr2=0x5590e91373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e91373f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f6a40008fd0 tx=0x7f6a40007b50 comp rx=0 tx=0).stop 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 shutdown_connections 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e911e280 0x5590e911e660 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a257fa640 0 client.6408 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911d490 0x5590e911d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e9131690 0x5590e91373f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/1231487431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9136eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 >> 172.21.15.76:0/1231487431 conn(0x5590e90e76a0 msgr2=0x5590e90e89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 shutdown_connections 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 1 -- 172.21.15.76:0/1231487431 wait complete. 2024-09-19T04:17:00.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.071+0000 7f6a4f43d000 20 client.6408 trim_cache size 0 max 16384 2024-09-19T04:17:00.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 Processor -- start 2024-09-19T04:17:00.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- start start 2024-09-19T04:17:00.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e9147260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911e280 0x5590e91477a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9135ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6a44070690 con 0x5590e911d490 2024-09-19T04:17:00.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a441ffa00 con 0x5590e911e280 2024-09-19T04:17:00.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6a44152360 con 0x5590e9130d70 2024-09-19T04:17:00.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e9147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e9147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35480/0 (socket says 172.21.15.76:35480) 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 -- 172.21.15.76:0/979563688 learned_addr learned my addr 172.21.15.76:0/979563688 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4991c640 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911e280 0x5590e91477a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9135ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 -- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 msgr2=0x5590e9135ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9135ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 -- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911e280 msgr2=0x5590e91477a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911e280 0x5590e91477a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 -- 172.21.15.76:0/979563688 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590e8eb8fd0 con 0x5590e911d490 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4991c640 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911e280 0x5590e91477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e9147260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6a38055a70 tx=0x7f6a38074040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb84673f151e7675 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a277fe640 1 -- 172.21.15.76:0/979563688 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38053de0 con 0x5590e911d490 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a277fe640 1 -- 172.21.15.76:0/979563688 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38072070 con 0x5590e911d490 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a277fe640 1 -- 172.21.15.76:0/979563688 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38010d70 con 0x5590e911d490 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- 172.21.15.76:0/979563688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 msgr2=0x5590e9147260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e9147260 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6a38055a70 tx=0x7f6a38074040 comp rx=0 tx=0).stop 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- 172.21.15.76:0/979563688 shutdown_connections 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e9130d70 0x5590e9135ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e911e280 0x5590e91477a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/979563688 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e9147260 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- 172.21.15.76:0/979563688 >> 172.21.15.76:0/979563688 conn(0x5590e90e76a0 msgr2=0x5590e90e1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- 172.21.15.76:0/979563688 shutdown_connections 2024-09-19T04:17:00.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- 172.21.15.76:0/979563688 wait complete. 2024-09-19T04:17:00.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 -- start start 2024-09-19T04:17:00.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.091+0000 7f6a4f43d000 1 Processor -- start 2024-09-19T04:17:00.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e91a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e911e280 0x5590e91a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e91a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5590e8f30f80 con 0x5590e911d490 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5590e8ee9ca0 con 0x5590e9130d70 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5590e8f362a0 con 0x5590e911e280 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 10 client.0 _ll_register_callbacks cb 0x7f6a400074e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e91a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e91a4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35492/0 (socket says 172.21.15.76:35492) 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 -- 172.21.15.76:0/3897637817 learned_addr learned my addr 172.21.15.76:0/3897637817 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a91e640 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e91a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4991c640 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e911e280 0x5590e91a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 -- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e911e280 msgr2=0x5590e91a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e911e280 0x5590e91a4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 -- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 msgr2=0x5590e91a54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e91a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 -- 172.21.15.76:0/3897637817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590e8fb4750 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4a11d640 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e91a4a40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6a380557e0 tx=0x7f6a38083b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56fcf99c18dc5aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 1 -- 172.21.15.76:0/3897637817 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a38078020 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 1 -- 172.21.15.76:0/3897637817 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7f6a38074040 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a38072070 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 1 -- 172.21.15.76:0/3897637817 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f6a38052150 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 10 client.6414 fetch_fsmap learned FSMap version 52 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x5590e8ebfa30 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 1 -- 172.21.15.76:0/3897637817 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a38072070 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 1 -- 172.21.15.76:0/3897637817 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 52) ==== 3542+0+0 (secure 0 0 0) 0x7f6a38007500 con 0x5590e911d490 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 10 client.6414 fetch_fsmap finished waiting for FSMap version 52 2024-09-19T04:17:00.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 10 client.6414 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 20 client.6414 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e9131690 0x5590e913bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 4 client.6414 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "status"}) -- 0x5590e8f3f3a0 con 0x5590e9131690 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4991c640 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e9131690 0x5590e913bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 10 client.6414 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4991c640 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e9131690 0x5590e913bcc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7403132f9bb69979 server_cookie=d5dc5dab15a50049 in_seq=0 out_seq=0 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 1 -- 172.21.15.76:0/3897637817 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5590e8f3f3a0 con 0x5590e9131690 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 10 client.6414 handle_command_reply: tid=0 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 client.6414 shutdown 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 msgr2=0x5590e91a4a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e91a4a40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6a380557e0 tx=0x7f6a38083b20 comp rx=0 tx=0).stop 2024-09-19T04:17:00.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 shutdown_connections 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5590e9131690 0x5590e913bcc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5590e9130d70 0x5590e91a54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a257fa640 0 client.6414 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5590e911e280 0x5590e91a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 --2- 172.21.15.76:0/3897637817 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5590e911d490 0x5590e91a4a40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 >> 172.21.15.76:0/3897637817 conn(0x5590e90e76a0 msgr2=0x5590e9146250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 shutdown_connections 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 1 -- 172.21.15.76:0/3897637817 wait complete. 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4f43d000 20 client.6414 trim_cache size 0 max 16384 2024-09-19T04:17:00.106 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6165, 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 17.123095434, 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 52, 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 67, 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 67, 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 21.998837118000001 2024-09-19T04:17:00.107 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:00.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a280678c0 msgr2=0x7f6a28069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a280678c0 0x7f6a28069d80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6a440af8c0 tx=0x7f6a40007d10 comp rx=0 tx=0).stop 2024-09-19T04:17:00.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 msgr2=0x7f6a4414f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a4414f6b0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f6a3800c500 tx=0x7f6a3800c9c0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 shutdown_connections 2024-09-19T04:17:00.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a280678c0 0x7f6a28069d80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a4415c1c0 0x7f6a44157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a440b5880 0x7f6a4414fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 --2- 172.21.15.76:0/2685784137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a4406d3e0 0x7f6a4414f6b0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 >> 172.21.15.76:0/2685784137 conn(0x7f6a44072670 msgr2=0x7f6a440b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 shutdown_connections 2024-09-19T04:17:00.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.095+0000 7f6a4c381640 1 -- 172.21.15.76:0/2685784137 wait complete. 2024-09-19T04:17:00.116 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.219+0000 7ff1c7b82640 1 Processor -- start 2024-09-19T04:17:00.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.219+0000 7ff1c7b82640 1 -- start start 2024-09-19T04:17:00.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c7b82640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1c015e450 0x7ff1c009aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c7b82640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 0x7ff1c015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c7b82640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1c00b5c90 con 0x7ff1c006bcc0 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c7b82640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff1c00b4860 con 0x7ff1c015e450 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c7b82640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c00b4580 con 0x7ff1c00b40c0 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c611f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff1c006bcc0 0x7ff1c0162370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52106/0 (socket says 172.21.15.76:52106) 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c611f640 1 -- 172.21.15.76:0/4246435894 learned_addr learned my addr 172.21.15.76:0/4246435894 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c511d640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 0x7ff1c015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c591e640 1 --2- 172.21.15.76:0/4246435894 >> v2:172.21.15.116:3300/0 conn(0x7ff1c015e450 0x7ff1c009aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 111293111 0 0) 0x7ff1b80014a0 con 0x7ff1c006bcc0 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1ac003530 con 0x7ff1c006bcc0 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 409291 0 0) 0x7ff1ac003530 con 0x7ff1c006bcc0 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 msgr2=0x7ff1c015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 0x7ff1c015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 >> v2:172.21.15.116:3300/0 conn(0x7ff1c015e450 msgr2=0x7ff1c009aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 --2- 172.21.15.76:0/4246435894 >> v2:172.21.15.116:3300/0 conn(0x7ff1c015e450 0x7ff1c009aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00b5f10 con 0x7ff1c006bcc0 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7ff1b8001bd0 con 0x7ff1c006bcc0 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 >> v1:172.21.15.76:6789/0 conn(0x7ff1c006bcc0 legacy=0x7ff1c0162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 0x7ff1ac0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ac003f90 0x7ff1ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac004d50 0x7ff1ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00b4860 con 0x7ff1ac003f90 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00b4580 con 0x7ff1ac004d50 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1ac00ba80 con 0x7ff1c00b40c0 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c511d640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ac003f90 0x7ff1ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c591e640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 0x7ff1ac0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c511d640 1 -- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 msgr2=0x7ff1ac0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c511d640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c00b40c0 0x7ff1ac0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c511d640 1 -- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac004d50 msgr2=0x7ff1ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:00.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c611f640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac004d50 0x7ff1ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c511d640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac004d50 0x7ff1ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c511d640 1 -- 172.21.15.76:0/4246435894 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1ac003530 con 0x7ff1ac003f90 2024-09-19T04:17:00.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.223+0000 7ff1c611f640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac004d50 0x7ff1ac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c511d640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ac003f90 0x7ff1ac004810 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009ae0 tx=0x7ff1b001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327903da83f6f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00b2070 con 0x7ff1ac003f90 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b00a5510 con 0x7ff1ac003f90 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c491c640 1 -- 172.21.15.76:0/4246435894 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b00096b0 con 0x7ff1ac003f90 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- 172.21.15.76:0/4246435894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ac003f90 msgr2=0x7ff1ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 --2- 172.21.15.76:0/4246435894 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1ac003f90 0x7ff1ac004810 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff1b0009ae0 tx=0x7ff1b001ca70 comp rx=0 tx=0).stop 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 -- 172.21.15.76:0/4246435894 reap_dead start 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- 172.21.15.76:0/4246435894 shutdown_connections 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- 172.21.15.76:0/4246435894 >> 172.21.15.76:0/4246435894 conn(0x7ff1c00b6800 msgr2=0x7ff1c00b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- 172.21.15.76:0/4246435894 shutdown_connections 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- 172.21.15.76:0/4246435894 wait complete. 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 Processor -- start 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- start start 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac003f90 0x7ff1c0164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c01571c0 0x7ff1c0157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1c00afea0 0x7ff1c00b0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0073720 con 0x7ff1c00afea0 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c00735a0 con 0x7ff1ac003f90 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1c015f840 con 0x7ff1c01571c0 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1c00afea0 0x7ff1c00b0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1c00afea0 0x7ff1c00b0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:35498/0 (socket says 172.21.15.76:35498) 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 -- 172.21.15.76:0/1668033898 learned_addr learned my addr 172.21.15.76:0/1668033898 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:00.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c591e640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac003f90 0x7ff1c0164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c511d640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c01571c0 0x7ff1c0157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 -- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c01571c0 msgr2=0x7ff1c0157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c01571c0 0x7ff1c0157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 -- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac003f90 msgr2=0x7ff1c0164440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac003f90 0x7ff1c0164440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 -- 172.21.15.76:0/1668033898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c00b0cd0 con 0x7ff1c00afea0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c611f640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1c00afea0 0x7ff1c00b0320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0072fd0 tx=0x7ff1b800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1b67fc640 1 -- 172.21.15.76:0/1668033898 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800de90 con 0x7ff1c00afea0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1b67fc640 1 -- 172.21.15.76:0/1668033898 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1b800bb80 con 0x7ff1c00afea0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1b67fc640 1 -- 172.21.15.76:0/1668033898 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8007110 con 0x7ff1c00afea0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.227+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c00b0fb0 con 0x7ff1c00afea0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.231+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c0164980 con 0x7ff1c00afea0 2024-09-19T04:17:00.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.231+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1c009aa00 con 0x7ff1c00afea0 2024-09-19T04:17:00.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.231+0000 7ff1b67fc640 1 -- 172.21.15.76:0/1668033898 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff1b801b020 con 0x7ff1c00afea0 2024-09-19T04:17:00.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.231+0000 7ff1b67fc640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1a4067a90 0x7ff1a4069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:00.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.231+0000 7ff1b67fc640 1 -- 172.21.15.76:0/1668033898 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6101+0+0 (secure 0 0 0) 0x7ff1b8086760 con 0x7ff1c00afea0 2024-09-19T04:17:00.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.231+0000 7ff1c591e640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1a4067a90 0x7ff1a4069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:00.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.231+0000 7ff1c591e640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1a4067a90 0x7ff1a4069f50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ab0 tx=0x7ff1bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:00.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.235+0000 7ff1b67fc640 1 -- 172.21.15.76:0/1668033898 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff1b8022070 con 0x7ff1c00afea0 2024-09-19T04:17:00.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.427+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1c015dc10 con 0x7ff1c00afea0 2024-09-19T04:17:00.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.427+0000 7ff1b67fc640 1 -- 172.21.15.76:0/1668033898 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5268 (secure 0 0 0) 0x7ff1c00b5c90 con 0x7ff1c00afea0 2024-09-19T04:17:00.431 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:00.431 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":52,"btime":"2024-09-19T04:16:59:055490+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:16:59.055484+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6165,"mds_1":6156,"mds_2":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"b","rank":2,"incarnation":51,"state":"up:active","state_seq":8,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6156":{"gid":6156,"name":"c","rank":1,"incarnation":49,"state":"up:active","state_seq":8,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[5272,6156,6165]},"id":4}]} 2024-09-19T04:17:00.432 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 52 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1a4067a90 msgr2=0x7ff1a4069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1a4067a90 0x7ff1a4069f50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009ab0 tx=0x7ff1bc008040 comp rx=0 tx=0).stop 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1c00afea0 msgr2=0x7ff1c00b0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1c00afea0 0x7ff1c00b0320 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0072fd0 tx=0x7ff1b800b4a0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 shutdown_connections 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1a4067a90 0x7ff1a4069f50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1c00afea0 0x7ff1c00b0320 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1c01571c0 0x7ff1c0157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 --2- 172.21.15.76:0/1668033898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1ac003f90 0x7ff1c0164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 >> 172.21.15.76:0/1668033898 conn(0x7ff1c00b6800 msgr2=0x7ff1c006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 shutdown_connections 2024-09-19T04:17:00.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:00.431+0000 7ff1c7b82640 1 -- 172.21.15.76:0/1668033898 wait complete. 2024-09-19T04:17:00.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:00.449 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:17:00.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:00.471 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:17:00.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:00.528 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-19T04:17:00.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:00.575 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-19T04:17:00.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:00.624 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-19T04:17:00.673 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:17:00.849 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:17:00.849 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:00.850 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:17:02.851 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:17:02.998 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:17:02.998 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:02.998 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:17:03.871 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:17:03.867+0000 7f9c43701640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:17:04.999 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:17:05.149 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:17:05.150 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:05.150 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:17:07.151 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:17:07.297 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:17:07.298 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:07.298 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:17:09.299 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:17:09.447 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:17:09.447 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:09.448 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:17:09.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:17:09.586 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:17:09.587 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:09.587 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:17:11.588 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:17:11.735 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:17:11.736 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:11.736 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:17:13.737 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:17:13.888 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-19T04:17:13.889 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:17:13.889 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:17:13.890 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-19T04:17:14.039 INFO:teuthology.orchestra.run.smithi116.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:17:14.040 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:17:14.040 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:17:14.040 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 flush journal 2024-09-19T04:17:14.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.167+0000 7fa4c13a8640 1 Processor -- start 2024-09-19T04:17:14.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.167+0000 7fa4c13a8640 1 -- start start 2024-09-19T04:17:14.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4c13a8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4bc1520a0 0x7fa4bc06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4c13a8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc06fe50 0x7fa4bc15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4c13a8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4bc070250 con 0x7fa4bc15f1d0 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4c13a8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa4bc152480 con 0x7fa4bc1520a0 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4c13a8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa4bc0aede0 con 0x7fa4bc06fe50 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4baffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa4bc1520a0 0x7fa4bc06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4bb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa4bc15f1d0 0x7fa4bc071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47724/0 (socket says 172.21.15.76:47724) 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4bb7fe640 1 -- 172.21.15.76:0/1869087721 learned_addr learned my addr 172.21.15.76:0/1869087721 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/1869087721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc06fe50 0x7fa4bc15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4baffd640 1 -- 172.21.15.76:0/1869087721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc06fe50 msgr2=0x7fa4bc15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4baffd640 1 --2- 172.21.15.76:0/1869087721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc06fe50 0x7fa4bc15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4baffd640 1 -- 172.21.15.76:0/1869087721 >> v1:172.21.15.76:6789/0 conn(0x7fa4bc15f1d0 legacy=0x7fa4bc071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1869087721 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1099286148 0 0) 0x7fa4bc070250 con 0x7fa4bc15f1d0 2024-09-19T04:17:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4baffd640 1 -- 172.21.15.76:0/1869087721 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4bc15fa10 con 0x7fa4bc1520a0 2024-09-19T04:17:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4baffd640 1 --2- 172.21.15.76:0/1869087721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc1520a0 0x7fa4bc06f910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009d90 tx=0x7fa4ac01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483e7018d56a4148 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1869087721 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b2070 con 0x7fa4bc1520a0 2024-09-19T04:17:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1869087721 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ac0a5930 con 0x7fa4bc1520a0 2024-09-19T04:17:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.171+0000 7fa4b9ffb640 1 -- 172.21.15.76:0/1869087721 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0a5d00 con 0x7fa4bc1520a0 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- 172.21.15.76:0/1869087721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc1520a0 msgr2=0x7fa4bc06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/1869087721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc1520a0 0x7fa4bc06f910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009d90 tx=0x7fa4ac01c950 comp rx=0 tx=0).stop 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- 172.21.15.76:0/1869087721 shutdown_connections 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/1869087721 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc06fe50 0x7fa4bc15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/1869087721 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc1520a0 0x7fa4bc06f910 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- 172.21.15.76:0/1869087721 >> 172.21.15.76:0/1869087721 conn(0x7fa4bc0b1f90 msgr2=0x7fa4bc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- 172.21.15.76:0/1869087721 shutdown_connections 2024-09-19T04:17:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- 172.21.15.76:0/1869087721 wait complete. 2024-09-19T04:17:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 Processor -- start 2024-09-19T04:17:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- start start 2024-09-19T04:17:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc06fe50 0x7fa4bc09b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc1520a0 0x7fa4bc09db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4bc15f1d0 0x7fa4bc14a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc073d30 con 0x7fa4bc06fe50 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc073a50 con 0x7fa4bc15f1d0 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4c13a8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa4bc15f640 con 0x7fa4bc1520a0 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc06fe50 0x7fa4bc09b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc06fe50 0x7fa4bc09b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54318/0 (socket says 172.21.15.76:54318) 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 -- 172.21.15.76:0/2728388572 learned_addr learned my addr 172.21.15.76:0/2728388572 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 -- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc1520a0 msgr2=0x7fa4bc09db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc1520a0 0x7fa4bc09db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc1520a0 0x7fa4bc09db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4bb7fe640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4bc15f1d0 0x7fa4bc14a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 -- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4bc15f1d0 msgr2=0x7fa4bc14a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4bc15f1d0 0x7fa4bc14a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 -- 172.21.15.76:0/2728388572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4bc09aa80 con 0x7fa4bc06fe50 2024-09-19T04:17:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc1520a0 0x7fa4bc09db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4bb7fe640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4bc15f1d0 0x7fa4bc14a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4baffd640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc06fe50 0x7fa4bc09b800 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac01a580 tx=0x7fa4ac0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4b37fe640 1 -- 172.21.15.76:0/2728388572 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0b2050 con 0x7fa4bc06fe50 2024-09-19T04:17:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4b37fe640 1 -- 172.21.15.76:0/2728388572 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4ac0be070 con 0x7fa4bc06fe50 2024-09-19T04:17:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.175+0000 7fa4b37fe640 1 -- 172.21.15.76:0/2728388572 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0adaf0 con 0x7fa4bc06fe50 2024-09-19T04:17:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4bc09ad20 con 0x7fa4bc06fe50 2024-09-19T04:17:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4bc09b2e0 con 0x7fa4bc06fe50 2024-09-19T04:17:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4b37fe640 1 -- 172.21.15.76:0/2728388572 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4ac0adc90 con 0x7fa4bc06fe50 2024-09-19T04:17:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4b37fe640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa49c0678c0 0x7fa49c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4b37fe640 1 -- 172.21.15.76:0/2728388572 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fa4ac121190 con 0x7fa4bc06fe50 2024-09-19T04:17:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa49c0678c0 0x7fa49c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa49c0678c0 0x7fa49c069d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc15ad90 tx=0x7fa4a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 Processor -- start 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 -- start start 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f529d70 0x55718f52a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f52a690 0x55718f4dd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f4dddb0 0x55718f4de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55718f3c7960 con 0x55718f52a690 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55718f30b900 con 0x55718f4dddb0 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55718f32ed40 con 0x55718f529d70 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4ba7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f52a690 0x55718f4dd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4ba7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f52a690 0x55718f4dd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54320/0 (socket says 172.21.15.76:54320) 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.179+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/3642849405 learned_addr learned my addr 172.21.15.76:0/3642849405 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f529d70 0x55718f52a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f529d70 msgr2=0x55718f52a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4bb7fe640 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f4dddb0 0x55718f4de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f529d70 0x55718f52a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f4dddb0 msgr2=0x55718f4de190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f4dddb0 0x55718f4de190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/3642849405 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718f2d5580 con 0x55718f52a690 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f52a690 0x55718f4dd870 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400e8c0 tx=0x7fa4a4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41760bbc5421e216 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4b0ff9640 1 -- 172.21.15.76:0/3642849405 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4025070 con 0x55718f52a690 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4b0ff9640 1 -- 172.21.15.76:0/3642849405 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4015640 con 0x55718f52a690 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4b0ff9640 1 -- 172.21.15.76:0/3642849405 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a400f040 con 0x55718f52a690 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- 172.21.15.76:0/3642849405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f52a690 msgr2=0x55718f4dd870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f52a690 0x55718f4dd870 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400e8c0 tx=0x7fa4a4011000 comp rx=0 tx=0).stop 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- 172.21.15.76:0/3642849405 shutdown_connections 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f4dddb0 0x55718f4de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f52a690 0x55718f4dd870 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3642849405 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f529d70 0x55718f52a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- 172.21.15.76:0/3642849405 >> 172.21.15.76:0/3642849405 conn(0x55718f50b040 msgr2=0x55718f50d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- 172.21.15.76:0/3642849405 shutdown_connections 2024-09-19T04:17:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- 172.21.15.76:0/3642849405 wait complete. 2024-09-19T04:17:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- start start 2024-09-19T04:17:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 Processor -- start 2024-09-19T04:17:14.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f4e55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f4e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f50b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55718f32ad00 con 0x55718f4dddb0 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55718f2e9aa0 con 0x55718f529d70 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55718f2c8340 con 0x55718f52a690 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 10 client.0 _ll_register_callbacks cb 0x55718f4cb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f4e55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f4e55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54322/0 (socket says 172.21.15.76:54322) 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 -- 172.21.15.76:0/228050135 learned_addr learned my addr 172.21.15.76:0/228050135 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4bb7fe640 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f50b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f4e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 -- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 msgr2=0x55718f50b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f50b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 -- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 msgr2=0x55718f4e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f4e5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 -- 172.21.15.76:0/228050135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718f2d1dc0 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4baffd640 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f4e55a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac098450 tx=0x7fa4ac0f0d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881df37d2c43b04c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa499ffb640 1 -- 172.21.15.76:0/228050135 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0f1ac0 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa499ffb640 1 -- 172.21.15.76:0/228050135 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fa4ac10e1b0 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a4015640 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa499ffb640 1 -- 172.21.15.76:0/228050135 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac11d050 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 10 client.6435 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55718f2ce7f0 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa499ffb640 1 -- 172.21.15.76:0/228050135 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4015640 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.183+0000 7fa499ffb640 1 -- 172.21.15.76:0/228050135 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7fa4ac0f9b10 con 0x55718f4dddb0 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 10 client.6435 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 10 client.6435 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 20 client.6435 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f50be30 0x55718f53b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 4 client.6435 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55718f2c8e10 con 0x55718f50be30 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f50be30 0x55718f53b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa499ffb640 10 client.6435 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f50be30 0x55718f53b020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633c14046d9d8330 server_cookie=f125d2468f8ca55f in_seq=0 out_seq=0 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa499ffb640 1 -- 172.21.15.76:0/228050135 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55718f2c8e10 con 0x55718f50be30 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa499ffb640 10 client.6435 handle_command_reply: tid=0 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 client.6435 shutdown 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 msgr2=0x55718f4e55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f4e55a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac098450 tx=0x7fa4ac0f0d20 comp rx=0 tx=0).stop 2024-09-19T04:17:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 shutdown_connections 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f50be30 0x55718f53b020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f50b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f4e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa499ffb640 0 client.6435 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 --2- 172.21.15.76:0/228050135 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f4e55a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 >> 172.21.15.76:0/228050135 conn(0x55718f50b040 msgr2=0x55718f4dcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 shutdown_connections 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 1 -- 172.21.15.76:0/228050135 wait complete. 2024-09-19T04:17:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.187+0000 7fa4c4464000 20 client.6435 trim_cache size 0 max 16384 2024-09-19T04:17:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 Processor -- start 2024-09-19T04:17:14.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- start start 2024-09-19T04:17:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f4dddb0 0x55718f4e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f529d70 0x55718f528b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f52a690 0x55718f529050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc201d90 con 0x55718f529d70 2024-09-19T04:17:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc200f60 con 0x55718f52a690 2024-09-19T04:17:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa4bc201660 con 0x55718f4dddb0 2024-09-19T04:17:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f529d70 0x55718f528b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f529d70 0x55718f528b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54332/0 (socket says 172.21.15.76:54332) 2024-09-19T04:17:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/4247058022 learned_addr learned my addr 172.21.15.76:0/4247058022 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4baffd640 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f4dddb0 0x55718f4e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4bb7fe640 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f52a690 0x55718f529050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f4dddb0 msgr2=0x55718f4e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f4dddb0 0x55718f4e5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f52a690 msgr2=0x55718f529050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f52a690 0x55718f529050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 -- 172.21.15.76:0/4247058022 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55718f2ec470 con 0x55718f529d70 2024-09-19T04:17:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f529d70 0x55718f528b10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4023040 tx=0x7fa4a401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae7e879daffb8c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa49bfff640 1 -- 172.21.15.76:0/4247058022 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4025040 con 0x55718f529d70 2024-09-19T04:17:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa49bfff640 1 -- 172.21.15.76:0/4247058022 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4030020 con 0x55718f529d70 2024-09-19T04:17:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa49bfff640 1 -- 172.21.15.76:0/4247058022 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a4015cd0 con 0x55718f529d70 2024-09-19T04:17:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- 172.21.15.76:0/4247058022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f529d70 msgr2=0x55718f528b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f529d70 0x55718f528b10 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa4a4023040 tx=0x7fa4a401d000 comp rx=0 tx=0).stop 2024-09-19T04:17:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- 172.21.15.76:0/4247058022 shutdown_connections 2024-09-19T04:17:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f52a690 0x55718f529050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f529d70 0x55718f528b10 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 --2- 172.21.15.76:0/4247058022 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f4dddb0 0x55718f4e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- 172.21.15.76:0/4247058022 >> 172.21.15.76:0/4247058022 conn(0x55718f50b040 msgr2=0x55718f4e2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- 172.21.15.76:0/4247058022 shutdown_connections 2024-09-19T04:17:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- 172.21.15.76:0/4247058022 wait complete. 2024-09-19T04:17:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 -- start start 2024-09-19T04:17:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.203+0000 7fa4c4464000 1 Processor -- start 2024-09-19T04:17:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f59d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f59de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f53c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa4bc0cd970 con 0x55718f4dddb0 2024-09-19T04:17:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55718f32ad00 con 0x55718f529d70 2024-09-19T04:17:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55718f33cfd0 con 0x55718f52a690 2024-09-19T04:17:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 10 client.0 _ll_register_callbacks cb 0x7fa4ac009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f59d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f59d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54342/0 (socket says 172.21.15.76:54342) 2024-09-19T04:17:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 -- 172.21.15.76:0/3828215960 learned_addr learned my addr 172.21.15.76:0/3828215960 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4bb7fe640 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f53c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f59de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 -- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 msgr2=0x55718f53c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f53c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 -- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 msgr2=0x55718f59de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f59de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 -- 172.21.15.76:0/3828215960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55718f2c2640 con 0x55718f4dddb0 2024-09-19T04:17:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4baffd640 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f59d920 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac019560 tx=0x7fa4ac11f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799280b39c3714fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa499ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa499ffb640 1 -- 172.21.15.76:0/3828215960 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0f1190 con 0x55718f4dddb0 2024-09-19T04:17:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa499ffb640 1 -- 172.21.15.76:0/3828215960 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fa4ac0f0a10 con 0x55718f4dddb0 2024-09-19T04:17:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a4030020 con 0x55718f4dddb0 2024-09-19T04:17:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa499ffb640 1 -- 172.21.15.76:0/3828215960 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fa4ac118050 con 0x55718f4dddb0 2024-09-19T04:17:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 10 client.6441 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55718f2c63b0 con 0x55718f4dddb0 2024-09-19T04:17:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa499ffb640 1 -- 172.21.15.76:0/3828215960 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4a4030020 con 0x55718f4dddb0 2024-09-19T04:17:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa499ffb640 1 -- 172.21.15.76:0/3828215960 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7fa4ac0fd020 con 0x55718f4dddb0 2024-09-19T04:17:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 10 client.6441 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 10 client.6441 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 20 client.6441 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f53c5b0 0x55718f53c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 4 client.6441 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:17:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55718f33c470 con 0x55718f53c5b0 2024-09-19T04:17:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f53c5b0 0x55718f53c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa4ba7fc640 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f53c5b0 0x55718f53c990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8815d1270057c168 server_cookie=1ce4b9dbe539ace2 in_seq=0 out_seq=0 2024-09-19T04:17:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.207+0000 7fa499ffb640 10 client.6441 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:14.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.223+0000 7fa499ffb640 1 -- 172.21.15.76:0/3828215960 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55718f33c470 con 0x55718f53c5b0 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.223+0000 7fa499ffb640 10 client.6441 handle_command_reply: tid=0 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.223+0000 7fa4c4464000 1 client.6441 shutdown 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.223+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 msgr2=0x55718f59d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.223+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f59d920 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac019560 tx=0x7fa4ac11f400 comp rx=0 tx=0).stop 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 shutdown_connections 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55718f53c5b0 0x55718f53c990 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55718f52a690 0x55718f53c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55718f529d70 0x55718f59de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 --2- 172.21.15.76:0/3828215960 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55718f4dddb0 0x55718f59d920 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 >> 172.21.15.76:0/3828215960 conn(0x55718f50b040 msgr2=0x55718f528b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa499ffb640 0 client.6441 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:14.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 shutdown_connections 2024-09-19T04:17:14.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 1 -- 172.21.15.76:0/3828215960 wait complete. 2024-09-19T04:17:14.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c4464000 20 client.6441 trim_cache size 0 max 16384 2024-09-19T04:17:14.231 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:14.231 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:17:14.231 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:17:14.231 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:14.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa49c0678c0 msgr2=0x7fa49c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa49c0678c0 0x7fa49c069d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc15ad90 tx=0x7fa4a4008040 comp rx=0 tx=0).stop 2024-09-19T04:17:14.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc06fe50 msgr2=0x7fa4bc09b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc06fe50 0x7fa4bc09b800 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac01a580 tx=0x7fa4ac0196f0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 shutdown_connections 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa49c0678c0 0x7fa49c069d80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4bc15f1d0 0x7fa4bc14a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa4bc1520a0 0x7fa4bc09db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 --2- 172.21.15.76:0/2728388572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4bc06fe50 0x7fa4bc09b800 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 >> 172.21.15.76:0/2728388572 conn(0x7fa4bc0b1f90 msgr2=0x7fa4bc0adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 shutdown_connections 2024-09-19T04:17:14.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.227+0000 7fa4c13a8640 1 -- 172.21.15.76:0/2728388572 wait complete. 2024-09-19T04:17:14.247 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 flush journal 2024-09-19T04:17:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f453a7e3640 1 Processor -- start 2024-09-19T04:17:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f453a7e3640 1 -- start start 2024-09-19T04:17:14.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f453a7e3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 0x7f45340cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f453a7e3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45340d0300 con 0x7f45340cf450 2024-09-19T04:17:14.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f453a7e3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45340b0f60 con 0x7f4534161a60 2024-09-19T04:17:14.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f453a7e3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f45340cfd40 con 0x7f45340d3060 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f4538d80640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f45340cf450 0x7f453406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47732/0 (socket says 172.21.15.76:47732) 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f4538d80640 1 -- 172.21.15.76:0/2512555659 learned_addr learned my addr 172.21.15.76:0/2512555659 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f45337fe640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 0x7f45340cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.355+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 853226747 0 0) 0x7f452c001120 con 0x7f45340cf450 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f451c003530 con 0x7f45340cf450 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4109349386 0 0) 0x7f451c003530 con 0x7f45340cf450 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 msgr2=0x7f45340cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 0x7f45340cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 >> v1:172.21.15.116:6789/0 conn(0x7f4534161a60 legacy=0x7f45340d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45340d0020 con 0x7f45340cf450 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f452c001850 con 0x7f45340cf450 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 >> v1:172.21.15.76:6789/0 conn(0x7f45340cf450 legacy=0x7f453406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 0x7f451c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f451c003f70 0x7f451c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f451c004d50 0x7f451c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f45340cfd40 con 0x7f451c004d50 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f451c00bb60 con 0x7f451c003f70 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f451c00bd60 con 0x7f45340d3060 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f451c004d50 0x7f451c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f45337fe640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f451c003f70 0x7f451c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4533fff640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 0x7f451c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 -- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 msgr2=0x7f451c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 0x7f451c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 -- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f451c003f70 msgr2=0x7f451c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f451c003f70 0x7f451c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 -- 172.21.15.76:0/2512555659 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451c003530 con 0x7f451c004d50 2024-09-19T04:17:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f45337fe640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f451c003f70 0x7f451c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4533fff640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340d3060 0x7f451c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f451c004d50 0x7f451c0051d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c4c0 tx=0x7f452c01f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a93391684c76e1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c00c0e0 con 0x7f451c004d50 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452c005b00 con 0x7f451c004d50 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4532ffd640 1 -- 172.21.15.76:0/2512555659 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452c007070 con 0x7f451c004d50 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f453a7e3640 1 -- 172.21.15.76:0/2512555659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f451c004d50 msgr2=0x7f451c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f453a7e3640 1 --2- 172.21.15.76:0/2512555659 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f451c004d50 0x7f451c0051d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c4c0 tx=0x7f452c01f3e0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f4538d80640 1 -- 172.21.15.76:0/2512555659 reap_dead start 2024-09-19T04:17:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f453a7e3640 1 -- 172.21.15.76:0/2512555659 shutdown_connections 2024-09-19T04:17:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f453a7e3640 1 -- 172.21.15.76:0/2512555659 >> 172.21.15.76:0/2512555659 conn(0x7f453408bf90 msgr2=0x7f453409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.359+0000 7f453a7e3640 1 -- 172.21.15.76:0/2512555659 shutdown_connections 2024-09-19T04:17:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 -- 172.21.15.76:0/2512555659 wait complete. 2024-09-19T04:17:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 Processor -- start 2024-09-19T04:17:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 -- start start 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4534158bd0 0x7f45340cc230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45340cc770 0x7f45340b9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340b9670 0x7f45340d0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f45340cf8e0 con 0x7f45340cc770 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f453406d190 con 0x7f4534158bd0 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f45340cfa60 con 0x7f45340b9670 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45340cc770 0x7f45340b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45340cc770 0x7f45340b9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54360/0 (socket says 172.21.15.76:54360) 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 -- 172.21.15.76:0/2018614226 learned_addr learned my addr 172.21.15.76:0/2018614226 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f4538d80640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340b9670 0x7f45340d0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f4533fff640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4534158bd0 0x7f45340cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 -- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340b9670 msgr2=0x7f45340d0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340b9670 0x7f45340d0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 -- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4534158bd0 msgr2=0x7f45340cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4534158bd0 0x7f45340cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 -- 172.21.15.76:0/2018614226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45340cb4e0 con 0x7f45340cc770 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f4533fff640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4534158bd0 0x7f45340cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f45337fe640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45340cc770 0x7f45340b9130 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4528011620 tx=0x7f4528011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f4530ff9640 1 -- 172.21.15.76:0/2018614226 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528004220 con 0x7f45340cc770 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f4530ff9640 1 -- 172.21.15.76:0/2018614226 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45280043c0 con 0x7f45340cc770 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45340cb750 con 0x7f45340cc770 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f4530ff9640 1 -- 172.21.15.76:0/2018614226 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452800f8c0 con 0x7f45340cc770 2024-09-19T04:17:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.363+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45340cbd10 con 0x7f45340cc770 2024-09-19T04:17:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4530ff9640 1 -- 172.21.15.76:0/2018614226 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f452800fa60 con 0x7f45340cc770 2024-09-19T04:17:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4530ff9640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45200678c0 0x7f4520069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4533fff640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45200678c0 0x7f4520069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4530ff9640 1 -- 172.21.15.76:0/2018614226 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f4528086170 con 0x7f45340cc770 2024-09-19T04:17:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4533fff640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45200678c0 0x7f4520069d80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4524001e10 tx=0x7f4524009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 Processor -- start 2024-09-19T04:17:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 -- start start 2024-09-19T04:17:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c6f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6f690 0x55ce97c22870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c22db0 0x55ce97c23190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ce97b0c960 con 0x55ce97c6f690 2024-09-19T04:17:14.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce97a50900 con 0x55ce97c22db0 2024-09-19T04:17:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ce97a73d40 con 0x55ce97c6ed70 2024-09-19T04:17:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6f690 0x55ce97c22870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6f690 0x55ce97c22870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54370/0 (socket says 172.21.15.76:54370) 2024-09-19T04:17:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 -- 172.21.15.76:0/3881409534 learned_addr learned my addr 172.21.15.76:0/3881409534 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4538d80640 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c22db0 0x55ce97c23190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4533fff640 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c6f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 -- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 msgr2=0x55ce97c6f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c6f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 -- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c22db0 msgr2=0x55ce97c23190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c22db0 0x55ce97c23190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 -- 172.21.15.76:0/3881409534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce97a1a580 con 0x55ce97c6f690 2024-09-19T04:17:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4538d80640 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c22db0 0x55ce97c23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f45337fe640 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6f690 0x55ce97c22870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f45280843b0 tx=0x7f4528056440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20e89fbe8d3a145 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4519ffb640 1 -- 172.21.15.76:0/3881409534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4528022070 con 0x55ce97c6f690 2024-09-19T04:17:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4519ffb640 1 -- 172.21.15.76:0/3881409534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4528054680 con 0x55ce97c6f690 2024-09-19T04:17:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f4519ffb640 1 -- 172.21.15.76:0/3881409534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45280529f0 con 0x55ce97c6f690 2024-09-19T04:17:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 -- 172.21.15.76:0/3881409534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6f690 msgr2=0x55ce97c22870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6f690 0x55ce97c22870 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f45280843b0 tx=0x7f4528056440 comp rx=0 tx=0).stop 2024-09-19T04:17:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 -- 172.21.15.76:0/3881409534 shutdown_connections 2024-09-19T04:17:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c22db0 0x55ce97c23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6f690 0x55ce97c22870 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 --2- 172.21.15.76:0/3881409534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c6f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.367+0000 7f453d89f000 1 -- 172.21.15.76:0/3881409534 >> 172.21.15.76:0/3881409534 conn(0x55ce97c50040 msgr2=0x55ce97c52430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- 172.21.15.76:0/3881409534 shutdown_connections 2024-09-19T04:17:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- 172.21.15.76:0/3881409534 wait complete. 2024-09-19T04:17:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- start start 2024-09-19T04:17:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 Processor -- start 2024-09-19T04:17:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97c2a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c2aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97c50420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ce97a6fd00 con 0x55ce97c22db0 2024-09-19T04:17:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce97a2eaa0 con 0x55ce97c6f690 2024-09-19T04:17:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ce97a0d340 con 0x55ce97c6ed70 2024-09-19T04:17:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 10 client.0 _ll_register_callbacks cb 0x55ce97c10760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97c2a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97c2a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54384/0 (socket says 172.21.15.76:54384) 2024-09-19T04:17:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 -- 172.21.15.76:0/2667449160 learned_addr learned my addr 172.21.15.76:0/2667449160 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f45337fe640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c2aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4538d80640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97c50420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 -- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 msgr2=0x55ce97c2aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c2aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 -- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 msgr2=0x55ce97c50420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97c50420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 -- 172.21.15.76:0/2667449160 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce97a16dc0 con 0x55ce97c22db0 2024-09-19T04:17:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f45337fe640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4533fff640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97c2a520 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f45240139f0 tx=0x7f4524001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a70c6a27a47f96 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 1 -- 172.21.15.76:0/2667449160 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524014800 con 0x55ce97c22db0 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 1 -- 172.21.15.76:0/2667449160 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f45240149a0 con 0x55ce97c22db0 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4528054680 con 0x55ce97c22db0 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 1 -- 172.21.15.76:0/2667449160 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f452402d050 con 0x55ce97c22db0 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 10 client.6453 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55ce97a137f0 con 0x55ce97c22db0 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 1 -- 172.21.15.76:0/2667449160 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4528054680 con 0x55ce97c22db0 2024-09-19T04:17:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 1 -- 172.21.15.76:0/2667449160 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7f4524030020 con 0x55ce97c22db0 2024-09-19T04:17:14.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 10 client.6453 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:14.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 10 client.6453 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-09-19T04:17:14.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 20 client.6453 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:14.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97c50e30 0x55ce97c80020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 4 client.6453 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:14.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 --> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce97a0de10 con 0x55ce97c50e30 2024-09-19T04:17:14.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f45337fe640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97c50e30 0x55ce97c80020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f45337fe640 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97c50e30 0x55ce97c80020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2104bc6e77cb66f6 server_cookie=40e7a6514d12cf96 in_seq=0 out_seq=0 2024-09-19T04:17:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 10 client.6453 ms_handle_connect on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 1 -- 172.21.15.76:0/2667449160 <== mds.1 v2:172.21.15.76:6834/2740411101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ce97a0de10 con 0x55ce97c50e30 2024-09-19T04:17:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f4512ffd640 10 client.6453 handle_command_reply: tid=0 2024-09-19T04:17:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 client.6453 shutdown 2024-09-19T04:17:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 msgr2=0x55ce97c2a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.371+0000 7f453d89f000 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97c2a520 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f45240139f0 tx=0x7f4524001520 comp rx=0 tx=0).stop 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 shutdown_connections 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97c50e30 0x55ce97c80020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97c50420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97c2aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 --2- 172.21.15.76:0/2667449160 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97c2a520 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f4512ffd640 0 client.6453 ms_handle_reset on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 >> 172.21.15.76:0/2667449160 conn(0x55ce97c50040 msgr2=0x55ce97c21f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 shutdown_connections 2024-09-19T04:17:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 1 -- 172.21.15.76:0/2667449160 wait complete. 2024-09-19T04:17:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.375+0000 7f453d89f000 20 client.6453 trim_cache size 0 max 16384 2024-09-19T04:17:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.387+0000 7f453d89f000 1 Processor -- start 2024-09-19T04:17:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.387+0000 7f453d89f000 1 -- start start 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.387+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c22db0 0x55ce97c4ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6ed70 0x55ce97c4f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97c290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4534202070 con 0x55ce97c6ed70 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4534201240 con 0x55ce97c6f690 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4534201940 con 0x55ce97c22db0 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6ed70 0x55ce97c4f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6ed70 0x55ce97c4f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54386/0 (socket says 172.21.15.76:54386) 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 -- 172.21.15.76:0/670679870 learned_addr learned my addr 172.21.15.76:0/670679870 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 -- 172.21.15.76:0/670679870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c22db0 msgr2=0x55ce97c4ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 --2- 172.21.15.76:0/670679870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c22db0 0x55ce97c4ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 -- 172.21.15.76:0/670679870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 msgr2=0x55ce97c290e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 --2- 172.21.15.76:0/670679870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97c290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 -- 172.21.15.76:0/670679870 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce97a31470 con 0x55ce97c6ed70 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45337fe640 1 --2- 172.21.15.76:0/670679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6ed70 0x55ce97c4f480 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f452805f120 tx=0x7f4528085000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29fe8a97648044ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45197fa640 1 -- 172.21.15.76:0/670679870 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45280544a0 con 0x55ce97c6ed70 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45197fa640 1 -- 172.21.15.76:0/670679870 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452800fd50 con 0x55ce97c6ed70 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f45197fa640 1 -- 172.21.15.76:0/670679870 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f452806e170 con 0x55ce97c6ed70 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- 172.21.15.76:0/670679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6ed70 msgr2=0x55ce97c4f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- 172.21.15.76:0/670679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6ed70 0x55ce97c4f480 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f452805f120 tx=0x7f4528085000 comp rx=0 tx=0).stop 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- 172.21.15.76:0/670679870 shutdown_connections 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- 172.21.15.76:0/670679870 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97c290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- 172.21.15.76:0/670679870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c6ed70 0x55ce97c4f480 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- 172.21.15.76:0/670679870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c22db0 0x55ce97c4ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- 172.21.15.76:0/670679870 >> 172.21.15.76:0/670679870 conn(0x55ce97c50040 msgr2=0x55ce97c6d2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- 172.21.15.76:0/670679870 shutdown_connections 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- 172.21.15.76:0/670679870 wait complete. 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- start start 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 Processor -- start 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97ceac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97ceb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97ce7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ce97a6fd00 con 0x55ce97c22db0 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ce97a78370 con 0x55ce97c6f690 2024-09-19T04:17:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ce97a73050 con 0x55ce97c6ed70 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 10 client.0 _ll_register_callbacks cb 0x7f4524013670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97ceac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97ceac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54392/0 (socket says 172.21.15.76:54392) 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 -- 172.21.15.76:0/1562167800 learned_addr learned my addr 172.21.15.76:0/1562167800 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 -- 172.21.15.76:0/1562167800 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 msgr2=0x55ce97ceb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97ceb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 -- 172.21.15.76:0/1562167800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 msgr2=0x55ce97ce7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97ce7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 -- 172.21.15.76:0/1562167800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce97a07640 con 0x55ce97c22db0 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4533fff640 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97ceac90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4524001560 tx=0x7f45240014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0d6e2d2746f448 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4512ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4512ffd640 1 -- 172.21.15.76:0/1562167800 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524035030 con 0x55ce97c22db0 2024-09-19T04:17:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4512ffd640 1 -- 172.21.15.76:0/1562167800 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f4524022910 con 0x55ce97c22db0 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f452800fd50 con 0x55ce97c22db0 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4512ffd640 1 -- 172.21.15.76:0/1562167800 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f452403f050 con 0x55ce97c22db0 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 10 client.6459 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55ce97a0b3b0 con 0x55ce97c22db0 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4512ffd640 1 -- 172.21.15.76:0/1562167800 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f452800fd50 con 0x55ce97c22db0 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f4512ffd640 1 -- 172.21.15.76:0/1562167800 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7f4524042020 con 0x55ce97c22db0 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 10 client.6459 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 10 client.6459 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.391+0000 7f453d89f000 20 client.6459 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.395+0000 7f453d89f000 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97ce7990 0x55ce97ce7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.395+0000 7f453d89f000 4 client.6459 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.395+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 --> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ce97a81470 con 0x55ce97ce7990 2024-09-19T04:17:14.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.395+0000 7f45337fe640 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97ce7990 0x55ce97ce7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.395+0000 7f4512ffd640 10 client.6459 ms_handle_connect on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:14.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.395+0000 7f45337fe640 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97ce7990 0x55ce97ce7da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80a09bd25e2bd02f server_cookie=5e7b9feb929365bc in_seq=0 out_seq=0 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f4512ffd640 1 -- 172.21.15.76:0/1562167800 <== mds.1 v2:172.21.15.76:6834/2740411101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ce97a81470 con 0x55ce97ce7990 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f4512ffd640 10 client.6459 handle_command_reply: tid=0 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 client.6459 shutdown 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 msgr2=0x55ce97ceac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97ceac90 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f4524001560 tx=0x7f45240014b0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 shutdown_connections 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x55ce97ce7990 0x55ce97ce7da0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ce97c6f690 0x55ce97ce7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ce97c6ed70 0x55ce97ceb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f4512ffd640 0 client.6459 ms_handle_reset on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:14.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 --2- 172.21.15.76:0/1562167800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ce97c22db0 0x55ce97ceac90 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 >> 172.21.15.76:0/1562167800 conn(0x55ce97c50040 msgr2=0x55ce97c6db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 shutdown_connections 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 1 -- 172.21.15.76:0/1562167800 wait complete. 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453d89f000 20 client.6459 trim_cache size 0 max 16384 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:17:14.413 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45200678c0 msgr2=0x7f4520069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45200678c0 0x7f4520069d80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4524001e10 tx=0x7f4524009040 comp rx=0 tx=0).stop 2024-09-19T04:17:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45340cc770 msgr2=0x7f45340b9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45340cc770 0x7f45340b9130 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4528011620 tx=0x7f4528011ae0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 shutdown_connections 2024-09-19T04:17:14.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45200678c0 0x7f4520069d80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45340b9670 0x7f45340d0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f45340cc770 0x7f45340b9130 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 --2- 172.21.15.76:0/2018614226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4534158bd0 0x7f45340cc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 >> 172.21.15.76:0/2018614226 conn(0x7f453408bf90 msgr2=0x7f453415eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 shutdown_connections 2024-09-19T04:17:14.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.407+0000 7f453a7e3640 1 -- 172.21.15.76:0/2018614226 wait complete. 2024-09-19T04:17:14.429 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 flush journal 2024-09-19T04:17:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.539+0000 7fba54bd3640 1 Processor -- start 2024-09-19T04:17:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.539+0000 7fba54bd3640 1 -- start start 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.539+0000 7fba54bd3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 0x7fba500d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.539+0000 7fba54bd3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba500d32b0 con 0x7fba500cf620 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.539+0000 7fba54bd3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba500cfad0 con 0x7fba500d2e90 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.539+0000 7fba54bd3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fba500d0300 con 0x7fba50161a60 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4ed9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fba500cf620 0x7fba5006d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47744/0 (socket says 172.21.15.76:47744) 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4ed9d640 1 -- 172.21.15.76:0/817210881 learned_addr learned my addr 172.21.15.76:0/817210881 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 0x7fba500d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2697503902 0 0) 0x7fba500d32b0 con 0x7fba500cf620 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba38003530 con 0x7fba500cf620 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1434770909 0 0) 0x7fba38003530 con 0x7fba500cf620 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 msgr2=0x7fba500d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 0x7fba500d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 >> v1:172.21.15.116:6789/0 conn(0x7fba500d2e90 legacy=0x7fba500cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba500b0f60 con 0x7fba500cf620 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fba48001a00 con 0x7fba500cf620 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 >> v1:172.21.15.76:6789/0 conn(0x7fba500cf620 legacy=0x7fba5006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba38003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba38004810 0x7fba38004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 0x7fba3800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fba500d0300 con 0x7fba380039f0 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fba38005530 con 0x7fba38004810 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fba3800bf10 con 0x7fba50161a60 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba38003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4ed9d640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 0x7fba3800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba38004810 0x7fba38004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 -- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 msgr2=0x7fba3800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 0x7fba3800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 -- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba38004810 msgr2=0x7fba38004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba38004810 0x7fba38004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 -- 172.21.15.76:0/817210881 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba38003530 con 0x7fba380039f0 2024-09-19T04:17:14.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba38004810 0x7fba38004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4ed9d640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba50161a60 0x7fba3800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4e59c640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba38003e10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fba4000ba30 tx=0x7fba4001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f770f5525433f14c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4000b600 con 0x7fba380039f0 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba400b1070 con 0x7fba380039f0 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4d59a640 1 -- 172.21.15.76:0/817210881 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400ac850 con 0x7fba380039f0 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba54bd3640 1 -- 172.21.15.76:0/817210881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 msgr2=0x7fba38003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba54bd3640 1 --2- 172.21.15.76:0/817210881 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba38003e10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fba4000ba30 tx=0x7fba4001c670 comp rx=0 tx=0).stop 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.543+0000 7fba4ed9d640 1 -- 172.21.15.76:0/817210881 reap_dead start 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- 172.21.15.76:0/817210881 shutdown_connections 2024-09-19T04:17:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- 172.21.15.76:0/817210881 >> 172.21.15.76:0/817210881 conn(0x7fba5008bf90 msgr2=0x7fba5009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- 172.21.15.76:0/817210881 shutdown_connections 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- 172.21.15.76:0/817210881 wait complete. 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 Processor -- start 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- start start 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba5015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba5015fd30 0x7fba500b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba500b4400 0x7fba500b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fba50169f20 con 0x7fba380039f0 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fba5006d190 con 0x7fba500b4400 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fba5016a0a0 con 0x7fba5015fd30 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba5015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba5015cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54408/0 (socket says 172.21.15.76:54408) 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 -- 172.21.15.76:0/1658879568 learned_addr learned my addr 172.21.15.76:0/1658879568 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba5015fd30 0x7fba500b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4ed9d640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba500b4400 0x7fba500b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 -- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba5015fd30 msgr2=0x7fba500b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba5015fd30 0x7fba500b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 -- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba500b4400 msgr2=0x7fba500b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba500b4400 0x7fba500b4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 -- 172.21.15.76:0/1658879568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5015d0c0 con 0x7fba380039f0 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba5015fd30 0x7fba500b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4ed9d640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba500b4400 0x7fba500b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba4e59c640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba5015cb80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fba400ae040 tx=0x7fba38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba3effd640 1 -- 172.21.15.76:0/1658879568 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400bb070 con 0x7fba380039f0 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba3effd640 1 -- 172.21.15.76:0/1658879568 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba40004220 con 0x7fba380039f0 2024-09-19T04:17:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba3effd640 1 -- 172.21.15.76:0/1658879568 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba40003620 con 0x7fba380039f0 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba500b8e30 con 0x7fba380039f0 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.547+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba500b93f0 con 0x7fba380039f0 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba3effd640 1 -- 172.21.15.76:0/1658879568 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fba5006e5d0 con 0x7fba380039f0 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba3effd640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba300678c0 0x7fba30069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba3effd640 1 -- 172.21.15.76:0/1658879568 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fba40120000 con 0x7fba380039f0 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba300678c0 0x7fba30069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba300678c0 0x7fba30069d80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fba44000c10 tx=0x7fba44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 Processor -- start 2024-09-19T04:17:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- start start 2024-09-19T04:17:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500afb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 0x558500aae870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 0x558500aaf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558500998960 con 0x558500afad70 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5585008dc900 con 0x558500afb690 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5585008ffd40 con 0x558500aaedb0 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500afb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500afb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54416/0 (socket says 172.21.15.76:54416) 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 -- 172.21.15.76:0/2309349403 learned_addr learned my addr 172.21.15.76:0/2309349403 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 -- 172.21.15.76:0/2309349403 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 msgr2=0x558500aaf190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 --2- 172.21.15.76:0/2309349403 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 0x558500aaf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 -- 172.21.15.76:0/2309349403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 msgr2=0x558500aae870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 --2- 172.21.15.76:0/2309349403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 0x558500aae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 -- 172.21.15.76:0/2309349403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585008a6580 con 0x558500afad70 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba4e59c640 1 --2- 172.21.15.76:0/2309349403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500afb150 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fba400f88d0 tx=0x7fba400f09e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da395ea87009068a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba2ffff640 1 -- 172.21.15.76:0/2309349403 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400f8e00 con 0x558500afad70 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba2ffff640 1 -- 172.21.15.76:0/2309349403 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba400b1040 con 0x558500afad70 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba2ffff640 1 -- 172.21.15.76:0/2309349403 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400ec890 con 0x558500afad70 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- 172.21.15.76:0/2309349403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 msgr2=0x558500afb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2309349403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500afb150 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fba400f88d0 tx=0x7fba400f09e0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- 172.21.15.76:0/2309349403 shutdown_connections 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2309349403 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 0x558500aaf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2309349403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 0x558500aae870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2309349403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500afb150 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- 172.21.15.76:0/2309349403 >> 172.21.15.76:0/2309349403 conn(0x558500adc040 msgr2=0x558500ade430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- 172.21.15.76:0/2309349403 shutdown_connections 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.551+0000 7fba57c8f000 1 -- 172.21.15.76:0/2309349403 wait complete. 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 -- start start 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 Processor -- start 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 0x558500b01ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500b02010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 0x558500b02550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5585008fbd00 con 0x558500afad70 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5585008baaa0 con 0x558500afb690 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x558500899340 con 0x558500aaedb0 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 10 client.0 _ll_register_callbacks cb 0x558500a9c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500b02010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500b02010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54418/0 (socket says 172.21.15.76:54418) 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 -- 172.21.15.76:0/2536372974 learned_addr learned my addr 172.21.15.76:0/2536372974 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 -- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 msgr2=0x558500b01ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4e59c640 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 0x558500b01ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 0x558500b01ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 -- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 msgr2=0x558500b02550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4ed9d640 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 0x558500b02550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 0x558500b02550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 -- 172.21.15.76:0/2536372974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585008a2dc0 con 0x558500afad70 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500b02010 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fba440136f0 tx=0x7fba44013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd26d9260e60624 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba2d7fa640 1 -- 172.21.15.76:0/2536372974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba44024060 con 0x558500afad70 2024-09-19T04:17:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba2d7fa640 1 -- 172.21.15.76:0/2536372974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fba4401f600 con 0x558500afad70 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba400b1040 con 0x558500afad70 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba2d7fa640 1 -- 172.21.15.76:0/2536372974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fba4402a050 con 0x558500afad70 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 10 client.6471 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55850089f7f0 con 0x558500afad70 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba2d7fa640 1 -- 172.21.15.76:0/2536372974 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba400b1040 con 0x558500afad70 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba2d7fa640 1 -- 172.21.15.76:0/2536372974 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7fba4402d020 con 0x558500afad70 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 10 client.6471 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 10 client.6471 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 20 client.6471 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500b09ea0 0x558500b0c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 4 client.6471 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 --> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558500899e10 con 0x558500b09ea0 2024-09-19T04:17:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4e59c640 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500b09ea0 0x558500b0c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba4e59c640 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500b09ea0 0x558500b0c290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29287ec81e95ab54 server_cookie=c3b69e0c85ad6e5b in_seq=0 out_seq=0 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.555+0000 7fba2d7fa640 10 client.6471 ms_handle_connect on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba2d7fa640 1 -- 172.21.15.76:0/2536372974 <== mds.2 v2:172.21.15.116:6832/1054899754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558500899e10 con 0x558500b09ea0 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba2d7fa640 10 client.6471 handle_command_reply: tid=0 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 client.6471 shutdown 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 msgr2=0x558500b02010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500b02010 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fba440136f0 tx=0x7fba44013720 comp rx=0 tx=0).stop 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 shutdown_connections 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500b09ea0 0x558500b0c290 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afb690 0x558500b02550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500afad70 0x558500b02010 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba2d7fa640 0 client.6471 ms_handle_reset on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:14.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 --2- 172.21.15.76:0/2536372974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500aaedb0 0x558500b01ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 >> 172.21.15.76:0/2536372974 conn(0x558500adc040 msgr2=0x558500aadf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 shutdown_connections 2024-09-19T04:17:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 1 -- 172.21.15.76:0/2536372974 wait complete. 2024-09-19T04:17:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.559+0000 7fba57c8f000 20 client.6471 trim_cache size 0 max 16384 2024-09-19T04:17:14.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 Processor -- start 2024-09-19T04:17:14.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- start start 2024-09-19T04:17:14.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500adc040 0x558500ab6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500aaedb0 0x558500ab7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500afad70 0x558500b0c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fba50205260 con 0x558500adc040 2024-09-19T04:17:14.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fba50204430 con 0x558500aaedb0 2024-09-19T04:17:14.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fba50204b30 con 0x558500afad70 2024-09-19T04:17:14.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500adc040 0x558500ab6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500adc040 0x558500ab6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54422/0 (socket says 172.21.15.76:54422) 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4111112827 learned_addr learned my addr 172.21.15.76:0/4111112827 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4111112827 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500afad70 msgr2=0x558500b0c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- 172.21.15.76:0/4111112827 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500afad70 0x558500b0c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4111112827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500aaedb0 msgr2=0x558500ab7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- 172.21.15.76:0/4111112827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500aaedb0 0x558500ab7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4111112827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585008bd470 con 0x558500adc040 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- 172.21.15.76:0/4111112827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500adc040 0x558500ab6c00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fba400f0160 tx=0x7fba400f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e60b74225be1caa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba2f7fe640 1 -- 172.21.15.76:0/4111112827 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400f8e00 con 0x558500adc040 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba2f7fe640 1 -- 172.21.15.76:0/4111112827 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba400b1040 con 0x558500adc040 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba2f7fe640 1 -- 172.21.15.76:0/4111112827 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba400ec890 con 0x558500adc040 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- 172.21.15.76:0/4111112827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500adc040 msgr2=0x558500ab6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4111112827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500adc040 0x558500ab6c00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fba400f0160 tx=0x7fba400f2000 comp rx=0 tx=0).stop 2024-09-19T04:17:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- 172.21.15.76:0/4111112827 shutdown_connections 2024-09-19T04:17:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4111112827 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500afad70 0x558500b0c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4111112827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500aaedb0 0x558500ab7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4111112827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500adc040 0x558500ab6c00 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- 172.21.15.76:0/4111112827 >> 172.21.15.76:0/4111112827 conn(0x558500ab3890 msgr2=0x558500b07ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- 172.21.15.76:0/4111112827 shutdown_connections 2024-09-19T04:17:14.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- 172.21.15.76:0/4111112827 wait complete. 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- start start 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 Processor -- start 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500aaedb0 0x558500b0f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500adc040 0x558500b32d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afad70 0x558500b332d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5585008fbd00 con 0x558500aaedb0 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fba50203500 con 0x558500afad70 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5585008a5f20 con 0x558500adc040 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 10 client.0 _ll_register_callbacks cb 0x7fba440015c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500aaedb0 0x558500b0f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500aaedb0 0x558500b0f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54428/0 (socket says 172.21.15.76:54428) 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4235884938 learned_addr learned my addr 172.21.15.76:0/4235884938 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500adc040 msgr2=0x558500b32d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500adc040 0x558500b32d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afad70 msgr2=0x558500b332d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afad70 0x558500b332d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 -- 172.21.15.76:0/4235884938 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558500893640 con 0x558500aaedb0 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba4e59c640 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500aaedb0 0x558500b0f120 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fba4000ba30 tx=0x7fba4001c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6e254614ea38b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba2d7fa640 1 -- 172.21.15.76:0/4235884938 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4010e020 con 0x558500aaedb0 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba2d7fa640 1 -- 172.21.15.76:0/4235884938 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fba400ec890 con 0x558500aaedb0 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.575+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba400b1040 con 0x558500aaedb0 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba2d7fa640 1 -- 172.21.15.76:0/4235884938 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7fba40114050 con 0x558500aaedb0 2024-09-19T04:17:14.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 10 client.6477 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5585008973b0 con 0x558500aaedb0 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba2d7fa640 1 -- 172.21.15.76:0/4235884938 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba400b1040 con 0x558500aaedb0 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba2d7fa640 1 -- 172.21.15.76:0/4235884938 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7fba401080d0 con 0x558500aaedb0 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 10 client.6477 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 10 client.6477 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 20 client.6477 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500afb690 0x558500b33810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 4 client.6477 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 --> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55850090d470 con 0x558500afb690 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500afb690 0x558500b33810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba2d7fa640 10 client.6477 ms_handle_connect on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:14.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.579+0000 7fba4dd9b640 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500afb690 0x558500b33810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e71e2193b9e7ee57 server_cookie=fa640cbb937373ac in_seq=0 out_seq=0 2024-09-19T04:17:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba2d7fa640 1 -- 172.21.15.76:0/4235884938 <== mds.2 v2:172.21.15.116:6832/1054899754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55850090d470 con 0x558500afb690 2024-09-19T04:17:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba2d7fa640 10 client.6477 handle_command_reply: tid=0 2024-09-19T04:17:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 client.6477 shutdown 2024-09-19T04:17:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500aaedb0 msgr2=0x558500b0f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500aaedb0 0x558500b0f120 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fba4000ba30 tx=0x7fba4001c670 comp rx=0 tx=0).stop 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 shutdown_connections 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x558500afb690 0x558500b33810 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558500afad70 0x558500b332d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba2d7fa640 0 client.6477 ms_handle_reset on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558500adc040 0x558500b32d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 --2- 172.21.15.76:0/4235884938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558500aaedb0 0x558500b0f120 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 >> 172.21.15.76:0/4235884938 conn(0x558500ab3890 msgr2=0x558500b07ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 shutdown_connections 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 1 -- 172.21.15.76:0/4235884938 wait complete. 2024-09-19T04:17:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba57c8f000 20 client.6477 trim_cache size 0 max 16384 2024-09-19T04:17:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:14.601 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:17:14.601 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:17:14.601 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba300678c0 msgr2=0x7fba30069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba300678c0 0x7fba30069d80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fba44000c10 tx=0x7fba44009040 comp rx=0 tx=0).stop 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 msgr2=0x7fba5015cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba5015cb80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fba400ae040 tx=0x7fba38005840 comp rx=0 tx=0).stop 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 shutdown_connections 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fba300678c0 0x7fba30069d80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba500b4400 0x7fba500b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fba5015fd30 0x7fba500b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 --2- 172.21.15.76:0/1658879568 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fba380039f0 0x7fba5015cb80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:14.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 >> 172.21.15.76:0/1658879568 conn(0x7fba5008bf90 msgr2=0x7fba500cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:14.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 shutdown_connections 2024-09-19T04:17:14.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14.595+0000 7fba54bd3640 1 -- 172.21.15.76:0/1658879568 wait complete. 2024-09-19T04:17:14.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:14.615 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-19T04:17:14.641 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/ 2024-09-19T04:17:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4 2024-09-19T04:17:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5 2024-09-19T04:17:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6 2024-09-19T04:17:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-19T04:17:14.642 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-19T04:17:14.642 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:17:14.642 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:17:14.643 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:17:14.643 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:17:14.643 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-19T04:17:14.643 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:17:14.738 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627778 2024-09-19T04:17:14.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:17:14.746 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:17:14.747 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:17:14.747 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:17:14.747 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:17:14.747 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-19T04:17:14.747 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:17:14.842 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627779 2024-09-19T04:17:14.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:17:14.851 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:17:14.851 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:17:14.851 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:17:14.851 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:17:14.851 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-19T04:17:14.851 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:17:14.960 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627780 2024-09-19T04:17:14.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:17:14.969 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:17:14.969 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:17:14.970 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:17:14.970 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:17:14.970 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-19T04:17:14.970 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:17:15.078 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627781 2024-09-19T04:17:15.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:17:15.088 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:17:15.088 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:17:15.088 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:17:15.088 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:17:15.088 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-19T04:17:15.088 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:17:15.193 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627782 2024-09-19T04:17:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:17:15.202 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:17:15.203 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:17:15.203 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:17:15.203 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:17:15.203 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-19T04:17:15.203 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:17:15.309 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627783 2024-09-19T04:17:15.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:17:15.318 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-19T04:17:15.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 Processor -- start 2024-09-19T04:17:15.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- start start 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bbd3fa0 0x55953bbb9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc603d0 0x55953bc64240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bc607b0 0x55953bc5f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55953bb8b040 con 0x55953bbd3fa0 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55953bb8aec0 con 0x55953bc603d0 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55953bb8b1c0 con 0x55953bc607b0 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bbd3fa0 0x55953bbb9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bbd3fa0 0x55953bbb9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54456/0 (socket says 172.21.15.76:54456) 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 -- 172.21.15.76:0/264373940 learned_addr learned my addr 172.21.15.76:0/264373940 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabf2ba640 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bc607b0 0x55953bc5f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabe2b8640 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc603d0 0x55953bc64240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 -- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bc607b0 msgr2=0x55953bc5f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bc607b0 0x55953bc5f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 -- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc603d0 msgr2=0x55953bc64240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc603d0 0x55953bc64240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 -- 172.21.15.76:0/264373940 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953bc63da0 con 0x55953bbd3fa0 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabeab9640 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bbd3fa0 0x55953bbb9310 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7faab00098d0 tx=0x7faab0005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3a9f93bc9d4aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabd2b6640 1 -- 172.21.15.76:0/264373940 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0019070 con 0x55953bbd3fa0 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabd2b6640 1 -- 172.21.15.76:0/264373940 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faab0008d40 con 0x55953bbd3fa0 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faabd2b6640 1 -- 172.21.15.76:0/264373940 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab00146a0 con 0x55953bbd3fa0 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- 172.21.15.76:0/264373940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bbd3fa0 msgr2=0x55953bbb9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bbd3fa0 0x55953bbb9310 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7faab00098d0 tx=0x7faab0005a10 comp rx=0 tx=0).stop 2024-09-19T04:17:15.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- 172.21.15.76:0/264373940 shutdown_connections 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bc607b0 0x55953bc5f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc603d0 0x55953bc64240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 --2- 172.21.15.76:0/264373940 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bbd3fa0 0x55953bbb9310 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- 172.21.15.76:0/264373940 >> 172.21.15.76:0/264373940 conn(0x55953bbb6060 msgr2=0x55953bc633d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- 172.21.15.76:0/264373940 shutdown_connections 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.395+0000 7faac0fa6080 1 -- 172.21.15.76:0/264373940 wait complete. 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 Processor -- start 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 -- start start 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bbd3fa0 0x55953bc64420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bc603d0 0x55953bc44390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc607b0 0x55953bc448d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55953bbb99b0 con 0x55953bc603d0 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55953bbb9c50 con 0x55953bc607b0 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55953bcf2ec0 con 0x55953bbd3fa0 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bc603d0 0x55953bc44390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bc603d0 0x55953bc44390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54464/0 (socket says 172.21.15.76:54464) 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 -- 172.21.15.76:0/2771974467 learned_addr learned my addr 172.21.15.76:0/2771974467 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabeab9640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bbd3fa0 0x55953bc64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabf2ba640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc607b0 0x55953bc448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 -- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bbd3fa0 msgr2=0x55953bc64420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55953bbd3fa0 0x55953bc64420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 -- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc607b0 msgr2=0x55953bc448d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc607b0 0x55953bc448d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 -- 172.21.15.76:0/2771974467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55953bc44f80 con 0x55953bc603d0 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabf2ba640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55953bc607b0 0x55953bc448d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabe2b8640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bc603d0 0x55953bc44390 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faaa801b980 tx=0x7faaa8017e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faab7fff640 1 -- 172.21.15.76:0/2771974467 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80b0040 con 0x55953bc603d0 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faab7fff640 1 -- 172.21.15.76:0/2771974467 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faaa80ba070 con 0x55953bc603d0 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55953bc45220 con 0x55953bc603d0 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faab7fff640 1 -- 172.21.15.76:0/2771974467 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80b5610 con 0x55953bc603d0 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55953bc45760 con 0x55953bc603d0 2024-09-19T04:17:15.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faab7fff640 1 -- 172.21.15.76:0/2771974467 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faaa80a3d10 con 0x55953bc603d0 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faab7fff640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faaa00662d0 0x7faaa0068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faab7fff640 1 -- 172.21.15.76:0/2771974467 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7faaa811f3b0 con 0x55953bc603d0 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.399+0000 7faabeab9640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faaa00662d0 0x7faaa0068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55953bc64960 0x55953bc43bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 --> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] -- osd_op(unknown.0.0:1 9.12 9:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x55953bcf3590 con 0x55953bc64960 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faabeab9640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faaa00662d0 0x7faaa0068790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faab0017040 tx=0x7faab0011000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faabf2ba640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55953bc64960 0x55953bc43bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faabf2ba640 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55953bc64960 0x55953bc43bc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faabf2ba640 1 -- 172.21.15.76:0/2771974467 <== osd.5 v2:172.21.15.116:6808/3134608166 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v69'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faab8009620 con 0x55953bc64960 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55953bc64960 msgr2=0x55953bc43bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55953bc64960 0x55953bc43bc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faaa00662d0 msgr2=0x7faaa0068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faaa00662d0 0x7faaa0068790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7faab0017040 tx=0x7faab0011000 comp rx=0 tx=0).stop 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bc603d0 msgr2=0x55953bc44390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 --2- 172.21.15.76:0/2771974467 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55953bc603d0 0x55953bc44390 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7faaa801b980 tx=0x7faaa8017e80 comp rx=0 tx=0).stop 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faabf2ba640 1 -- 172.21.15.76:0/2771974467 reap_dead start 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 shutdown_connections 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 >> 172.21.15.76:0/2771974467 conn(0x55953bbb6060 msgr2=0x55953bc63330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 shutdown_connections 2024-09-19T04:17:15.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.403+0000 7faac0fa6080 1 -- 172.21.15.76:0/2771974467 wait complete. 2024-09-19T04:17:15.580 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-19T04:17:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 Processor -- start 2024-09-19T04:17:15.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 -- start start 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38223a0 0x55bec38510c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec3847bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38dfbb0 0x55bec38dcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bec37731b0 con 0x55bec386f990 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bec37cc7b0 con 0x55bec38dfbb0 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5d061080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bec3855310 con 0x55bec38223a0 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec3847bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec3847bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54478/0 (socket says 172.21.15.76:54478) 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 -- 172.21.15.76:0/3438397049 learned_addr learned my addr 172.21.15.76:0/3438397049 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5ab74640 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38223a0 0x55bec38510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5b375640 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38dfbb0 0x55bec38dcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 -- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38223a0 msgr2=0x55bec38510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38223a0 0x55bec38510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 -- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38dfbb0 msgr2=0x55bec38dcaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38dfbb0 0x55bec38dcaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5a373640 1 -- 172.21.15.76:0/3438397049 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec38defa0 con 0x55bec386f990 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.655+0000 7faa5b375640 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38dfbb0 0x55bec38dcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec3847bb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b330 tx=0x7faa5000b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c5ee6734395c99 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa59371640 1 -- 172.21.15.76:0/3438397049 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5000d5d0 con 0x55bec386f990 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa59371640 1 -- 172.21.15.76:0/3438397049 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa50008070 con 0x55bec386f990 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa59371640 1 -- 172.21.15.76:0/3438397049 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5000d9b0 con 0x55bec386f990 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- 172.21.15.76:0/3438397049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 msgr2=0x55bec3847bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec3847bb0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b330 tx=0x7faa5000b7f0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- 172.21.15.76:0/3438397049 shutdown_connections 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38dfbb0 0x55bec38dcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec3847bb0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 --2- 172.21.15.76:0/3438397049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38223a0 0x55bec38510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- 172.21.15.76:0/3438397049 >> 172.21.15.76:0/3438397049 conn(0x55bec3822dc0 msgr2=0x55bec38509b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- 172.21.15.76:0/3438397049 shutdown_connections 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- 172.21.15.76:0/3438397049 wait complete. 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 Processor -- start 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- start start 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38223a0 0x55bec38f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec384e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38dfbb0 0x55bec384ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bec3850270 con 0x55bec386f990 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bec38df240 con 0x55bec38223a0 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bec3850510 con 0x55bec38dfbb0 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec384e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec384e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54490/0 (socket says 172.21.15.76:54490) 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 -- 172.21.15.76:0/3672232727 learned_addr learned my addr 172.21.15.76:0/3672232727 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5ab74640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38223a0 0x55bec38f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5b375640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38dfbb0 0x55bec384ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 -- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38dfbb0 msgr2=0x55bec384ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bec38dfbb0 0x55bec384ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 -- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38223a0 msgr2=0x55bec38f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38223a0 0x55bec38f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 -- 172.21.15.76:0/3672232727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bec3992c60 con 0x55bec386f990 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5ab74640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bec38223a0 0x55bec38f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5a373640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec384e4e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7faa5000bcf0 tx=0x7faa50000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa43fff640 1 -- 172.21.15.76:0/3672232727 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5001c850 con 0x55bec386f990 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa43fff640 1 -- 172.21.15.76:0/3672232727 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa50020030 con 0x55bec386f990 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bec3992f00 con 0x55bec386f990 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa43fff640 1 -- 172.21.15.76:0/3672232727 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa5001cc30 con 0x55bec386f990 2024-09-19T04:17:15.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.659+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bec3993440 con 0x55bec386f990 2024-09-19T04:17:15.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa43fff640 1 -- 172.21.15.76:0/3672232727 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa50017040 con 0x55bec386f990 2024-09-19T04:17:15.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa43fff640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa480662d0 0x7faa48068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa43fff640 1 -- 172.21.15.76:0/3672232727 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7faa50086cb0 con 0x55bec386f990 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa5ab74640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa480662d0 0x7faa48068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa5d061080 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55bec38f6c30 0x55bec3996a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 --> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] -- osd_op(unknown.0.0:1 9.12 9:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x55bec39938e0 con 0x55bec38f6c30 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa5ab74640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa480662d0 0x7faa48068790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faa4c000c10 tx=0x7faa4c002580 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa5b375640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55bec38f6c30 0x55bec3996a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.663+0000 7faa5b375640 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55bec38f6c30 0x55bec3996a10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5b375640 1 -- 172.21.15.76:0/3672232727 <== osd.5 v2:172.21.15.116:6808/3134608166 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v69'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7faa540ab7b0 con 0x55bec38f6c30 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55bec38f6c30 msgr2=0x55bec3996a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55bec38f6c30 0x55bec3996a10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa480662d0 msgr2=0x7faa48068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa480662d0 0x7faa48068790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7faa4c000c10 tx=0x7faa4c002580 comp rx=0 tx=0).stop 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 msgr2=0x55bec384e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 --2- 172.21.15.76:0/3672232727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bec386f990 0x55bec384e4e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7faa5000bcf0 tx=0x7faa50000ee0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5b375640 1 -- 172.21.15.76:0/3672232727 reap_dead start 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 shutdown_connections 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 >> 172.21.15.76:0/3672232727 conn(0x55bec3822dc0 msgr2=0x55bec38217b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 shutdown_connections 2024-09-19T04:17:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.667+0000 7faa5d061080 1 -- 172.21.15.76:0/3672232727 wait complete. 2024-09-19T04:17:15.674 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-19T04:17:15.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 Processor -- start 2024-09-19T04:17:15.757 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- start start 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd9557d60 0x560dd95ffb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd96936c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd95fe580 0x560dd9693c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560dd94781b0 con 0x560dd9571280 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560dd95555b0 con 0x560dd95fe580 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560dd9600040 con 0x560dd9557d60 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd96936c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd96936c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54506/0 (socket says 172.21.15.76:54506) 2024-09-19T04:17:15.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/4236751653 learned_addr learned my addr 172.21.15.76:0/4236751653 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efa4ea640 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd9557d60 0x560dd95ffb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efaceb640 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd95fe580 0x560dd9693c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd9557d60 msgr2=0x560dd95ffb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd9557d60 0x560dd95ffb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd95fe580 msgr2=0x560dd9693c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd95fe580 0x560dd9693c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/4236751653 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd9600280 con 0x560dd9571280 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efaceb640 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd95fe580 0x560dd9693c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef9ce9640 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd96936c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00a4b0 tx=0x7f9eec0033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d2ef2dd595580 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef8ce7640 1 -- 172.21.15.76:0/4236751653 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0b9070 con 0x560dd9571280 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef8ce7640 1 -- 172.21.15.76:0/4236751653 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec0ab410 con 0x560dd9571280 2024-09-19T04:17:15.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9ef8ce7640 1 -- 172.21.15.76:0/4236751653 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0ab800 con 0x560dd9571280 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- 172.21.15.76:0/4236751653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 msgr2=0x560dd96936c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd96936c0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f9eec00a4b0 tx=0x7f9eec0033b0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- 172.21.15.76:0/4236751653 shutdown_connections 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd95fe580 0x560dd9693c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd96936c0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/4236751653 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd9557d60 0x560dd95ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- 172.21.15.76:0/4236751653 >> 172.21.15.76:0/4236751653 conn(0x560dd9552220 msgr2=0x560dd9570210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.763 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- 172.21.15.76:0/4236751653 shutdown_connections 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- 172.21.15.76:0/4236751653 wait complete. 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 Processor -- start 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- start start 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd9557d60 0x560dd95de270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd95f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd95fe580 0x560dd95fae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560dd968fff0 con 0x560dd9571280 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560dd9600520 con 0x560dd9557d60 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.755+0000 7f9efc9d7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560dd9690290 con 0x560dd95fe580 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd95f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd95f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54512/0 (socket says 172.21.15.76:54512) 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/1724393565 learned_addr learned my addr 172.21.15.76:0/1724393565 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efaceb640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd95fe580 0x560dd95fae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efa4ea640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd9557d60 0x560dd95de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/1724393565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd95fe580 msgr2=0x560dd95fae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560dd95fe580 0x560dd95fae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/1724393565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd9557d60 msgr2=0x560dd95de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560dd9557d60 0x560dd95de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 -- 172.21.15.76:0/1724393565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd95de9f0 con 0x560dd9571280 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef9ce9640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd95f5820 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9eec01b710 tx=0x7f9eec017f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef37fe640 1 -- 172.21.15.76:0/1724393565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0b9040 con 0x560dd9571280 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef37fe640 1 -- 172.21.15.76:0/1724393565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec0c4070 con 0x560dd9571280 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef37fe640 1 -- 172.21.15.76:0/1724393565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0abe10 con 0x560dd9571280 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560dd95dec90 con 0x560dd9571280 2024-09-19T04:17:15.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd95fa970 con 0x560dd9571280 2024-09-19T04:17:15.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef37fe640 1 -- 172.21.15.76:0/1724393565 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9eec0b3070 con 0x560dd9571280 2024-09-19T04:17:15.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef37fe640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9edc0662d0 0x7f9edc068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9ef37fe640 1 -- 172.21.15.76:0/1724393565 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f9eec1200d0 con 0x560dd9571280 2024-09-19T04:17:15.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efa4ea640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9edc0662d0 0x7f9edc068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560dd95f5d60 0x560dd9692c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 9.5 9:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x560dd95fb350 con 0x560dd95f5d60 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efaceb640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560dd95f5d60 0x560dd9692c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efa4ea640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9edc0662d0 0x7f9edc068790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005640 tx=0x7f9ee40054d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.759+0000 7f9efaceb640 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560dd95f5d60 0x560dd9692c30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efaceb640 1 -- 172.21.15.76:0/1724393565 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v69'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f9ef4010410 con 0x560dd95f5d60 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560dd95f5d60 msgr2=0x560dd9692c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560dd95f5d60 0x560dd9692c30 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9edc0662d0 msgr2=0x7f9edc068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9edc0662d0 0x7f9edc068790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f9ee4005640 tx=0x7f9ee40054d0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 msgr2=0x560dd95f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 --2- 172.21.15.76:0/1724393565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560dd9571280 0x560dd95f5820 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f9eec01b710 tx=0x7f9eec017f40 comp rx=0 tx=0).stop 2024-09-19T04:17:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efaceb640 1 -- 172.21.15.76:0/1724393565 reap_dead start 2024-09-19T04:17:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 shutdown_connections 2024-09-19T04:17:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 >> 172.21.15.76:0/1724393565 conn(0x560dd9552220 msgr2=0x560dd95fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 shutdown_connections 2024-09-19T04:17:15.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.763+0000 7f9efc9d7080 1 -- 172.21.15.76:0/1724393565 wait complete. 2024-09-19T04:17:15.771 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-19T04:17:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 Processor -- start 2024-09-19T04:17:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 -- start start 2024-09-19T04:17:15.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb272ad40 0x560fb27b6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb270df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb27b60c0 0x560fb27b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560fb2710db0 con 0x560fb27b60c0 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560fb26331b0 con 0x560fb272ad40 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f52719f7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560fb27b66b0 con 0x560fb272c810 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb27b60c0 0x560fb27b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb27b60c0 0x560fb27b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54526/0 (socket says 172.21.15.76:54526) 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 -- 172.21.15.76:0/2084408640 learned_addr learned my addr 172.21.15.76:0/2084408640 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526ed09640 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb270df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526f50a640 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb272ad40 0x560fb27b6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 -- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 msgr2=0x560fb270df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb270df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 -- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb272ad40 msgr2=0x560fb27b6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb272ad40 0x560fb27b6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 -- 172.21.15.76:0/2084408640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb27b90c0 con 0x560fb27b60c0 2024-09-19T04:17:15.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526f50a640 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb272ad40 0x560fb27b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.847+0000 7f526fd0b640 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb27b60c0 0x560fb27b40f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5260001640 tx=0x7f5260001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fafcaa13da6d8d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526dd07640 1 -- 172.21.15.76:0/2084408640 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260013620 con 0x560fb27b60c0 2024-09-19T04:17:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526dd07640 1 -- 172.21.15.76:0/2084408640 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52600137c0 con 0x560fb27b60c0 2024-09-19T04:17:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526dd07640 1 -- 172.21.15.76:0/2084408640 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5260013b90 con 0x560fb27b60c0 2024-09-19T04:17:15.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- 172.21.15.76:0/2084408640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb27b60c0 msgr2=0x560fb27b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb27b60c0 0x560fb27b40f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5260001640 tx=0x7f5260001b00 comp rx=0 tx=0).stop 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- 172.21.15.76:0/2084408640 shutdown_connections 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb27b60c0 0x560fb27b40f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb270df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 --2- 172.21.15.76:0/2084408640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb272ad40 0x560fb27b6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- 172.21.15.76:0/2084408640 >> 172.21.15.76:0/2084408640 conn(0x560fb270d220 msgr2=0x560fb270d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- 172.21.15.76:0/2084408640 shutdown_connections 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- 172.21.15.76:0/2084408640 wait complete. 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 Processor -- start 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- start start 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb272ad40 0x560fb27918a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb27b7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb27b60c0 0x560fb27ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x560fb27b3710 con 0x560fb272ad40 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x560fb27b9360 con 0x560fb27b60c0 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x560fb27b39b0 con 0x560fb272c810 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb272ad40 0x560fb27918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb272ad40 0x560fb27918a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54542/0 (socket says 172.21.15.76:54542) 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 -- 172.21.15.76:0/3129437306 learned_addr learned my addr 172.21.15.76:0/3129437306 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526ed09640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb27b7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526fd0b640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb27b60c0 0x560fb27ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 -- 172.21.15.76:0/3129437306 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 msgr2=0x560fb27b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb27b7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 -- 172.21.15.76:0/3129437306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb27b60c0 msgr2=0x560fb27ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x560fb27b60c0 0x560fb27ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 -- 172.21.15.76:0/3129437306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fb2790980 con 0x560fb272ad40 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526ed09640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x560fb272c810 0x560fb27b7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526f50a640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb272ad40 0x560fb27918a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f526801b8c0 tx=0x7f5268018180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526cd05640 1 -- 172.21.15.76:0/3129437306 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52680af040 con 0x560fb272ad40 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526cd05640 1 -- 172.21.15.76:0/3129437306 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52680b5440 con 0x560fb272ad40 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f526cd05640 1 -- 172.21.15.76:0/3129437306 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52680057d0 con 0x560fb272ad40 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560fb2790c20 con 0x560fb272ad40 2024-09-19T04:17:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.851+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fb2791160 con 0x560fb272ad40 2024-09-19T04:17:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f526cd05640 1 -- 172.21.15.76:0/3129437306 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f52680b57f0 con 0x560fb272ad40 2024-09-19T04:17:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f526cd05640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52500662d0 0x7f5250068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f526cd05640 1 -- 172.21.15.76:0/3129437306 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f526811fb50 con 0x560fb272ad40 2024-09-19T04:17:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f526ed09640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52500662d0 0x7f5250068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f52719f7080 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560fb27b83c0 0x560fb27aebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 9.9 9:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x560fb27af0f0 con 0x560fb27b83c0 2024-09-19T04:17:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f526fd0b640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560fb27b83c0 0x560fb27aebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f526ed09640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52500662d0 0x7f5250068790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5258005d00 tx=0x7f525800c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.855+0000 7f526fd0b640 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560fb27b83c0 0x560fb27aebb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f526fd0b640 1 -- 172.21.15.76:0/3129437306 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v69'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f526000f030 con 0x560fb27b83c0 2024-09-19T04:17:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560fb27b83c0 msgr2=0x560fb27aebb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x560fb27b83c0 0x560fb27aebb0 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52500662d0 msgr2=0x7f5250068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f52500662d0 0x7f5250068790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5258005d00 tx=0x7f525800c040 comp rx=0 tx=0).stop 2024-09-19T04:17:15.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb272ad40 msgr2=0x560fb27918a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 --2- 172.21.15.76:0/3129437306 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x560fb272ad40 0x560fb27918a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f526801b8c0 tx=0x7f5268018180 comp rx=0 tx=0).stop 2024-09-19T04:17:15.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f526fd0b640 1 -- 172.21.15.76:0/3129437306 reap_dead start 2024-09-19T04:17:15.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 shutdown_connections 2024-09-19T04:17:15.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 >> 172.21.15.76:0/3129437306 conn(0x560fb270d220 msgr2=0x560fb27b4ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 shutdown_connections 2024-09-19T04:17:15.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.859+0000 7f52719f7080 1 -- 172.21.15.76:0/3129437306 wait complete. 2024-09-19T04:17:15.866 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-19T04:17:15.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.943+0000 7f6162b33080 1 Processor -- start 2024-09-19T04:17:15.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.943+0000 7f6162b33080 1 -- start start 2024-09-19T04:17:15.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8ad9fa0 0x5623b8b60720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8b663d0 0x5623b8b6a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8bfd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5623b8b67c30 con 0x5623b8b667b0 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5623b89e21b0 con 0x5623b8ad9fa0 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5623b8b67db0 con 0x5623b8b663d0 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8bfd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8bfd1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54562/0 (socket says 172.21.15.76:54562) 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/272278889 learned_addr learned my addr 172.21.15.76:0/272278889 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160646640 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8ad9fa0 0x5623b8b60720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/272278889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8b663d0 msgr2=0x5623b8b6a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8b663d0 0x5623b8b6a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/272278889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8ad9fa0 msgr2=0x5623b8b60720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8ad9fa0 0x5623b8b60720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/272278889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b8b61f60 con 0x5623b8b667b0 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8bfd1d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6150018f30 tx=0x7f6150003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89db2868ff338f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f615ee43640 1 -- 172.21.15.76:0/272278889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61500af040 con 0x5623b8b667b0 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f615ee43640 1 -- 172.21.15.76:0/272278889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61500b9070 con 0x5623b8b667b0 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f615ee43640 1 -- 172.21.15.76:0/272278889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61500aae00 con 0x5623b8b667b0 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- 172.21.15.76:0/272278889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 msgr2=0x5623b8bfd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8bfd1d0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6150018f30 tx=0x7f6150003030 comp rx=0 tx=0).stop 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- 172.21.15.76:0/272278889 shutdown_connections 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8bfd1d0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8b663d0 0x5623b8b6a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- 172.21.15.76:0/272278889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8ad9fa0 0x5623b8b60720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- 172.21.15.76:0/272278889 >> 172.21.15.76:0/272278889 conn(0x5623b8abc060 msgr2=0x5623b8a907b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- 172.21.15.76:0/272278889 shutdown_connections 2024-09-19T04:17:15.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- 172.21.15.76:0/272278889 wait complete. 2024-09-19T04:17:15.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 Processor -- start 2024-09-19T04:17:15.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- start start 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8ad9fa0 0x5623b8b60ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8b663d0 0x5623b8b616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8a90990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5623b8b64330 con 0x5623b8b667b0 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5623b8b640d0 con 0x5623b8b663d0 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6162b33080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5623b8b645d0 con 0x5623b8ad9fa0 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8a90990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8a90990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54572/0 (socket says 172.21.15.76:54572) 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/1775714509 learned_addr learned my addr 172.21.15.76:0/1775714509 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f615fe45640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8b663d0 0x5623b8b616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160646640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8ad9fa0 0x5623b8b60ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/1775714509 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8ad9fa0 msgr2=0x5623b8b60ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5623b8ad9fa0 0x5623b8b60ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/1775714509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8b663d0 msgr2=0x5623b8b616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5623b8b663d0 0x5623b8b616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.947+0000 7f6160e47640 1 -- 172.21.15.76:0/1775714509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623b8b5fbc0 con 0x5623b8b667b0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f6160e47640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8a90990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f61500aa5d0 tx=0x7f61500180a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615de41640 1 -- 172.21.15.76:0/1775714509 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61500af040 con 0x5623b8b667b0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615de41640 1 -- 172.21.15.76:0/1775714509 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61500b4bf0 con 0x5623b8b667b0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5623b8b5fe60 con 0x5623b8b667b0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615de41640 1 -- 172.21.15.76:0/1775714509 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61500c6400 con 0x5623b8b667b0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623b8b603a0 con 0x5623b8b667b0 2024-09-19T04:17:15.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615de41640 1 -- 172.21.15.76:0/1775714509 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f61500cf430 con 0x5623b8b667b0 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615de41640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61400662a0 0x7f6140068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615de41640 1 -- 172.21.15.76:0/1775714509 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f6150121aa0 con 0x5623b8b667b0 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f6160646640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61400662a0 0x7f6140068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f6162b33080 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x5623b8a90ed0 0x5623b8ad27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 9.5 9:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x5623b8ad2ce0 con 0x5623b8a90ed0 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615fe45640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x5623b8a90ed0 0x5623b8ad27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f6160646640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61400662a0 0x7f6140068760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6158000f90 tx=0x7f6158008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.951+0000 7f615fe45640 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x5623b8a90ed0 0x5623b8ad27a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:15.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f615fe45640 1 -- 172.21.15.76:0/1775714509 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v69'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f61480076c0 con 0x5623b8a90ed0 2024-09-19T04:17:15.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x5623b8a90ed0 msgr2=0x5623b8ad27a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x5623b8a90ed0 0x5623b8ad27a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61400662a0 msgr2=0x7f6140068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f61400662a0 0x7f6140068760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f6158000f90 tx=0x7f6158008040 comp rx=0 tx=0).stop 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 msgr2=0x5623b8a90990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 --2- 172.21.15.76:0/1775714509 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5623b8b667b0 0x5623b8a90990 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f61500aa5d0 tx=0x7f61500180a0 comp rx=0 tx=0).stop 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6160e47640 1 -- 172.21.15.76:0/1775714509 reap_dead start 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 shutdown_connections 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 >> 172.21.15.76:0/1775714509 conn(0x5623b8abc060 msgr2=0x5623b8b6a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 shutdown_connections 2024-09-19T04:17:15.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:15.955+0000 7f6162b33080 1 -- 172.21.15.76:0/1775714509 wait complete. 2024-09-19T04:17:15.962 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-19T04:17:16.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 Processor -- start 2024-09-19T04:17:16.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 -- start start 2024-09-19T04:17:16.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741b901d0 0x564741c50510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741bc2dd0 0x564741bbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741c4c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564741bc3550 con 0x564741c4dbb0 2024-09-19T04:17:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564741ae11b0 con 0x564741b901d0 2024-09-19T04:17:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7796ca080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x564741bc37f0 con 0x564741bc2dd0 2024-09-19T04:17:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741c4c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741c4c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54582/0 (socket says 172.21.15.76:54582) 2024-09-19T04:17:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 -- 172.21.15.76:0/447478010 learned_addr learned my addr 172.21.15.76:0/447478010 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7769dc640 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741bc2dd0 0x564741bbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7771dd640 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741b901d0 0x564741c50510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 -- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741bc2dd0 msgr2=0x564741bbd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741bc2dd0 0x564741bbd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 -- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741b901d0 msgr2=0x564741c50510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741b901d0 0x564741c50510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 -- 172.21.15.76:0/447478010 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564741c65f50 con 0x564741c4dbb0 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7779de640 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741c4c0f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b220 tx=0x7fb76800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb686b789b594de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7759da640 1 -- 172.21.15.76:0/447478010 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800a040 con 0x564741c4dbb0 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7759da640 1 -- 172.21.15.76:0/447478010 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb76800f620 con 0x564741c4dbb0 2024-09-19T04:17:16.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.035+0000 7fb7759da640 1 -- 172.21.15.76:0/447478010 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800fa10 con 0x564741c4dbb0 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- 172.21.15.76:0/447478010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 msgr2=0x564741c4c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741c4c0f0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fb76800b220 tx=0x7fb76800b6e0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- 172.21.15.76:0/447478010 shutdown_connections 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741c4c0f0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741bc2dd0 0x564741bbd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 --2- 172.21.15.76:0/447478010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741b901d0 0x564741c50510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- 172.21.15.76:0/447478010 >> 172.21.15.76:0/447478010 conn(0x564741b90dc0 msgr2=0x564741c4fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- 172.21.15.76:0/447478010 shutdown_connections 2024-09-19T04:17:16.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- 172.21.15.76:0/447478010 wait complete. 2024-09-19T04:17:16.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 Processor -- start 2024-09-19T04:17:16.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- start start 2024-09-19T04:17:16.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741b901d0 0x564741d00520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741bc2dd0 0x564741c66850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741bbaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564741c66490 con 0x564741c4dbb0 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564741c661f0 con 0x564741bc2dd0 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x564741c50f60 con 0x564741b901d0 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741bbaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741bbaa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54598/0 (socket says 172.21.15.76:54598) 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 -- 172.21.15.76:0/1357220505 learned_addr learned my addr 172.21.15.76:0/1357220505 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7771dd640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741b901d0 0x564741d00520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7769dc640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741bc2dd0 0x564741c66850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 -- 172.21.15.76:0/1357220505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741b901d0 msgr2=0x564741d00520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741b901d0 0x564741d00520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 -- 172.21.15.76:0/1357220505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741bc2dd0 msgr2=0x564741c66850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564741bc2dd0 0x564741c66850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 -- 172.21.15.76:0/1357220505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564741bb8c00 con 0x564741c4dbb0 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7771dd640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564741b901d0 0x564741d00520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7779de640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741bbaa20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb768003040 tx=0x7fb768000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7749d8640 1 -- 172.21.15.76:0/1357220505 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800e8b0 con 0x564741c4dbb0 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7749d8640 1 -- 172.21.15.76:0/1357220505 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb768021020 con 0x564741c4dbb0 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564741bb8ea0 con 0x564741c4dbb0 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7749d8640 1 -- 172.21.15.76:0/1357220505 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76800ea90 con 0x564741c4dbb0 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.039+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564741bb93e0 con 0x564741c4dbb0 2024-09-19T04:17:16.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7749d8640 1 -- 172.21.15.76:0/1357220505 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb768017070 con 0x564741c4dbb0 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7749d8640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7580662a0 0x7fb758068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7749d8640 1 -- 172.21.15.76:0/1357220505 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fb768086710 con 0x564741c4dbb0 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7771dd640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7580662a0 0x7fb758068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x564741d00a60 0x564741bb84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 9.1d 9:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e69) -- 0x564741c66d90 con 0x564741d00a60 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7769dc640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x564741d00a60 0x564741bb84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7771dd640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7580662a0 0x7fb758068760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7700ac000 tx=0x7fb7700a5000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7769dc640 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x564741d00a60 0x564741bb84b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7769dc640 1 -- 172.21.15.76:0/1357220505 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v69'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fb760011040 con 0x564741d00a60 2024-09-19T04:17:16.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x564741d00a60 msgr2=0x564741bb84b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x564741d00a60 0x564741bb84b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7580662a0 msgr2=0x7fb758068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7580662a0 0x7fb758068760 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb7700ac000 tx=0x7fb7700a5000 comp rx=0 tx=0).stop 2024-09-19T04:17:16.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 msgr2=0x564741bbaa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 --2- 172.21.15.76:0/1357220505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564741c4dbb0 0x564741bbaa20 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb768003040 tx=0x7fb768000ee0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7779de640 1 -- 172.21.15.76:0/1357220505 reap_dead start 2024-09-19T04:17:16.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 shutdown_connections 2024-09-19T04:17:16.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.043+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 >> 172.21.15.76:0/1357220505 conn(0x564741b90dc0 msgr2=0x564741c63150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.047+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 shutdown_connections 2024-09-19T04:17:16.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.047+0000 7fb7796ca080 1 -- 172.21.15.76:0/1357220505 wait complete. 2024-09-19T04:17:16.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:16.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start /d1/d2/d3 recursive,force 2024-09-19T04:17:16.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59bcfe3640 1 Processor -- start 2024-09-19T04:17:16.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59bcfe3640 1 -- start start 2024-09-19T04:17:16.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59bcfe3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b80cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59bcfe3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59b80b3e00 0x7f59b80b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59bcfe3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59b8159070 con 0x7f59b815f840 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59bcfe3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f59b806d070 con 0x7f59b80b3e00 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59bcfe3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f59b806d2c0 con 0x7f59b8158c90 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59b80b3e00 0x7f59b80b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59b80b3e00 0x7f59b80b42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54602/0 (socket says 172.21.15.76:54602) 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3922515099 learned_addr learned my addr 172.21.15.76:0/3922515099 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b659c640 1 --2- 172.21.15.76:0/3922515099 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b80cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3922515099 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 msgr2=0x7f59b80cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/3922515099 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b80cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3922515099 >> v1:172.21.15.116:6789/0 conn(0x7f59b815f840 legacy=0x7f59b806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.223+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3922515099 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b806d5a0 con 0x7f59b80b3e00 2024-09-19T04:17:16.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/3922515099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b80b42b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009870 tx=0x7f59a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7318d86d7bf05675 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59b559a640 1 -- 172.21.15.76:0/3922515099 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80b0070 con 0x7f59b80b3e00 2024-09-19T04:17:16.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59b559a640 1 -- 172.21.15.76:0/3922515099 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a80ab600 con 0x7f59b80b3e00 2024-09-19T04:17:16.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59b559a640 1 -- 172.21.15.76:0/3922515099 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80ab9d0 con 0x7f59b80b3e00 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 -- 172.21.15.76:0/3922515099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 msgr2=0x7f59b80b42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/3922515099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b80b42b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009870 tx=0x7f59a801c520 comp rx=0 tx=0).stop 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 -- 172.21.15.76:0/3922515099 shutdown_connections 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/3922515099 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b80b42b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/3922515099 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b80cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 -- 172.21.15.76:0/3922515099 >> 172.21.15.76:0/3922515099 conn(0x7f59b806bcc0 msgr2=0x7f59b806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 -- 172.21.15.76:0/3922515099 shutdown_connections 2024-09-19T04:17:16.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 -- 172.21.15.76:0/3922515099 wait complete. 2024-09-19T04:17:16.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 Processor -- start 2024-09-19T04:17:16.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.227+0000 7f59bcfe3640 1 -- start start 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b8159850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b815fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59b815f840 0x7f59b8153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59b80cd180 con 0x7f59b80b3e00 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59b80cd000 con 0x7f59b815f840 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f59b80cd300 con 0x7f59b8158c90 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b8159850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b8159850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54606/0 (socket says 172.21.15.76:54606) 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 -- 172.21.15.76:0/2421317429 learned_addr learned my addr 172.21.15.76:0/2421317429 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59b815f840 0x7f59b8153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b815fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 -- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 msgr2=0x7f59b815fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b815fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 -- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59b815f840 msgr2=0x7f59b8153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59b815f840 0x7f59b8153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 -- 172.21.15.76:0/2421317429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b8159f30 con 0x7f59b80b3e00 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59b815f840 0x7f59b8153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b815fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b659c640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b8159850 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f59b0007ef0 tx=0x7f59b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59aeffd640 1 -- 172.21.15.76:0/2421317429 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0014620 con 0x7f59b80b3e00 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59aeffd640 1 -- 172.21.15.76:0/2421317429 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b0019070 con 0x7f59b80b3e00 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b815a210 con 0x7f59b80b3e00 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59aeffd640 1 -- 172.21.15.76:0/2421317429 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0014a00 con 0x7f59b80b3e00 2024-09-19T04:17:16.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b815a4f0 con 0x7f59b80b3e00 2024-09-19T04:17:16.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59aeffd640 1 -- 172.21.15.76:0/2421317429 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f59b0014ba0 con 0x7f59b80b3e00 2024-09-19T04:17:16.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59aeffd640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59980678f0 0x7f5998069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59aeffd640 1 -- 172.21.15.76:0/2421317429 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f59b00867e0 con 0x7f59b80b3e00 2024-09-19T04:17:16.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.231+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59980678f0 0x7f5998069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59980678f0 0x7f5998069db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f59b81682d0 tx=0x7f59a0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 Processor -- start 2024-09-19T04:17:16.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- start start 2024-09-19T04:17:16.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f9460 0x55f6f03af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f03afc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f6f01fdbd0 con 0x55f6f03af830 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f6f01a32f0 con 0x55f6f03f8b40 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f6f01f9b90 con 0x55f6f03f9460 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f03afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f03afc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54618/0 (socket says 172.21.15.76:54618) 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b5d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f9460 0x55f6f03af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3647574532 learned_addr learned my addr 172.21.15.76:0/3647574532 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b659c640 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3647574532 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f9460 msgr2=0x55f6f03af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f9460 0x55f6f03af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3647574532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 msgr2=0x55f6f03f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 -- 172.21.15.76:0/3647574532 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6f01c9b10 con 0x55f6f03af830 2024-09-19T04:17:16.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f03afc10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f59a801cdc0 tx=0x7f59a80a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9553fa2d23d4e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f5997fff640 1 -- 172.21.15.76:0/3647574532 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80b0040 con 0x55f6f03af830 2024-09-19T04:17:16.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f5997fff640 1 -- 172.21.15.76:0/3647574532 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a80bb070 con 0x55f6f03af830 2024-09-19T04:17:16.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f5997fff640 1 -- 172.21.15.76:0/3647574532 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80aba50 con 0x55f6f03af830 2024-09-19T04:17:16.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- 172.21.15.76:0/3647574532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 msgr2=0x55f6f03afc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f03afc10 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f59a801cdc0 tx=0x7f59a80a6000 comp rx=0 tx=0).stop 2024-09-19T04:17:16.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- 172.21.15.76:0/3647574532 shutdown_connections 2024-09-19T04:17:16.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f03afc10 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f9460 0x55f6f03af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 --2- 172.21.15.76:0/3647574532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- 172.21.15.76:0/3647574532 >> 172.21.15.76:0/3647574532 conn(0x55f6f03ab7e0 msgr2=0x55f6f03adbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- 172.21.15.76:0/3647574532 shutdown_connections 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- 172.21.15.76:0/3647574532 wait complete. 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 -- start start 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.235+0000 7f59c009f000 1 Processor -- start 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 0x55f6f03b98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03b9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f9460 0x55f6f03ba370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f6f027f700 con 0x55f6f03f9460 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f6f01975c0 con 0x55f6f03f8b40 2024-09-19T04:17:16.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f6f01a7500 con 0x55f6f03af830 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 10 client.0 _ll_register_callbacks cb 0x55f6f02b15a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f9460 0x55f6f03ba370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f9460 0x55f6f03ba370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54620/0 (socket says 172.21.15.76:54620) 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 -- 172.21.15.76:0/2513842209 learned_addr learned my addr 172.21.15.76:0/2513842209 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b659c640 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 0x55f6f03b98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03b9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 -- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 msgr2=0x55f6f03b98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 0x55f6f03b98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 -- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 msgr2=0x55f6f03b9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03b9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 -- 172.21.15.76:0/2513842209 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6f01a3500 con 0x55f6f03f9460 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f9460 0x55f6f03ba370 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f59a8000c10 tx=0x7f59a801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70518c4eb26075a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:16.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 1 -- 172.21.15.76:0/2513842209 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a80c6020 con 0x55f6f03f9460 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 1 -- 172.21.15.76:0/2513842209 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f59a80ab850 con 0x55f6f03f9460 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59a80bb070 con 0x55f6f03f9460 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 1 -- 172.21.15.76:0/2513842209 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f59a80d0050 con 0x55f6f03f9460 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 10 client.6549 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55f6f019d550 con 0x55f6f03f9460 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 1 -- 172.21.15.76:0/2513842209 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59a80bb070 con 0x55f6f03f9460 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 1 -- 172.21.15.76:0/2513842209 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7f59a80d3020 con 0x55f6f03f9460 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 10 client.6549 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 10 client.6549 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 20 client.6549 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0407bd0 0x55f6f0409fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 4 client.6549 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:16.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6f0197d00 con 0x55f6f0407bd0 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b659c640 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0407bd0 0x55f6f0409fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59b659c640 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0407bd0 0x55f6f0409fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56bde5563f40ee14 server_cookie=cd36334d8deedfe3 in_seq=0 out_seq=0 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 10 client.6549 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 1 -- 172.21.15.76:0/2513842209 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6f0197d00 con 0x55f6f0407bd0 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 10 client.6549 handle_command_reply: tid=0 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 client.6549 shutdown 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f9460 msgr2=0x55f6f03ba370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f9460 0x55f6f03ba370 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f59a8000c10 tx=0x7f59a801ad60 comp rx=0 tx=0).stop 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 shutdown_connections 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0407bd0 0x55f6f0409fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f9460 0x55f6f03ba370 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f8b40 0x55f6f03b9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 --2- 172.21.15.76:0/2513842209 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 0x55f6f03b98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59957fa640 0 client.6549 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 >> 172.21.15.76:0/2513842209 conn(0x55f6f03ab7e0 msgr2=0x55f6f03ae3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 shutdown_connections 2024-09-19T04:17:16.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 1 -- 172.21.15.76:0/2513842209 wait complete. 2024-09-19T04:17:16.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.239+0000 7f59c009f000 20 client.6549 trim_cache size 0 max 16384 2024-09-19T04:17:16.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 Processor -- start 2024-09-19T04:17:16.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- start start 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f040d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f8b40 0x55f6f03b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 0x55f6f03b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59b8204020 con 0x55f6f03af830 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59b82031f0 con 0x55f6f03f9460 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f59b82038f0 con 0x55f6f03f8b40 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f040d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f040d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54632/0 (socket says 172.21.15.76:54632) 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 -- 172.21.15.76:0/2553860367 learned_addr learned my addr 172.21.15.76:0/2553860367 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 -- 172.21.15.76:0/2553860367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f8b40 msgr2=0x55f6f03b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 --2- 172.21.15.76:0/2553860367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f8b40 0x55f6f03b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 -- 172.21.15.76:0/2553860367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 msgr2=0x55f6f03b4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 --2- 172.21.15.76:0/2553860367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 0x55f6f03b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 -- 172.21.15.76:0/2553860367 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6f027fb90 con 0x55f6f03af830 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59b659c640 1 --2- 172.21.15.76:0/2553860367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f040d620 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f59b0056990 tx=0x7f59b0016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b9ff7c0180f5ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59977fe640 1 -- 172.21.15.76:0/2553860367 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b006e020 con 0x55f6f03af830 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59977fe640 1 -- 172.21.15.76:0/2553860367 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b005f5e0 con 0x55f6f03af830 2024-09-19T04:17:16.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59977fe640 1 -- 172.21.15.76:0/2553860367 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0054d00 con 0x55f6f03af830 2024-09-19T04:17:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- 172.21.15.76:0/2553860367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 msgr2=0x55f6f040d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 --2- 172.21.15.76:0/2553860367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f040d620 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f59b0056990 tx=0x7f59b0016040 comp rx=0 tx=0).stop 2024-09-19T04:17:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- 172.21.15.76:0/2553860367 shutdown_connections 2024-09-19T04:17:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 --2- 172.21.15.76:0/2553860367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 0x55f6f03b4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 --2- 172.21.15.76:0/2553860367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03f8b40 0x55f6f03b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 --2- 172.21.15.76:0/2553860367 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03af830 0x55f6f040d620 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- 172.21.15.76:0/2553860367 >> 172.21.15.76:0/2553860367 conn(0x55f6f03ab7e0 msgr2=0x55f6f03aedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- 172.21.15.76:0/2553860367 shutdown_connections 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.259+0000 7f59c009f000 1 -- 172.21.15.76:0/2553860367 wait complete. 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- start start 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 Processor -- start 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 0x55f6f040c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f8b40 0x55f6f04282b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 0x55f6f04287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f6f01fcee0 con 0x55f6f03f8b40 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59b80b6730 con 0x55f6f03f9460 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f6f027f700 con 0x55f6f03af830 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 10 client.0 _ll_register_callbacks cb 0x7f59a000ada0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f8b40 0x55f6f04282b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f8b40 0x55f6f04282b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54648/0 (socket says 172.21.15.76:54648) 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 -- 172.21.15.76:0/2440620031 learned_addr learned my addr 172.21.15.76:0/2440620031 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 -- 172.21.15.76:0/2440620031 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 msgr2=0x55f6f040c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 0x55f6f040c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b6d9d640 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 0x55f6f04287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 -- 172.21.15.76:0/2440620031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 msgr2=0x55f6f04287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 0x55f6f04287f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 -- 172.21.15.76:0/2440620031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6f0256ca0 con 0x55f6f03f8b40 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b5d9b640 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f8b40 0x55f6f04282b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f59b806cc40 tx=0x7f59a0004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=716b40172ace083b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 1 -- 172.21.15.76:0/2440620031 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0024020 con 0x55f6f03f8b40 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 1 -- 172.21.15.76:0/2440620031 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f59a0013d40 con 0x55f6f03f8b40 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b005f5e0 con 0x55f6f03f8b40 2024-09-19T04:17:16.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 1 -- 172.21.15.76:0/2440620031 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f59a002e050 con 0x55f6f03f8b40 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 10 client.6555 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55f6efeb84c0 con 0x55f6f03f8b40 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 1 -- 172.21.15.76:0/2440620031 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59b005f5e0 con 0x55f6f03f8b40 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 1 -- 172.21.15.76:0/2440620031 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7f59a0031020 con 0x55f6f03f8b40 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 10 client.6555 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 10 client.6555 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 20 client.6555 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0428d30 0x55f6f046c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 4 client.6555 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55f6f020b300 con 0x55f6f0428d30 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b659c640 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0428d30 0x55f6f046c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 10 client.6555 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59b659c640 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0428d30 0x55f6f046c040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddab50b5bdd5ea7a server_cookie=98758e02b9a8fdcc in_seq=0 out_seq=0 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 1 -- 172.21.15.76:0/2440620031 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f6f020b300 con 0x55f6f0428d30 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 10 client.6555 handle_command_reply: tid=0 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 client.6555 shutdown 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f8b40 msgr2=0x55f6f04282b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f8b40 0x55f6f04282b0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f59b806cc40 tx=0x7f59a0004700 comp rx=0 tx=0).stop 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 shutdown_connections 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55f6f0428d30 0x55f6f046c040 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59957fa640 0 client.6555 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f6f03f9460 0x55f6f04287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f6f03f8b40 0x55f6f04282b0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 --2- 172.21.15.76:0/2440620031 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f6f03af830 0x55f6f040c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 >> 172.21.15.76:0/2440620031 conn(0x55f6f03ab7e0 msgr2=0x55f6f03b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 shutdown_connections 2024-09-19T04:17:16.274 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f9c3f6f9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:17:00.464621+0000 1=0+1) n(v3 rc2024-09-19T04:17:00.671707+0000 6=0+6) (inest mix dirty) (ifile excl) caps={6330=pAsLsXsFsx/AsLsXsFsx@10},l=6330 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560dd6095600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.464621+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.671707+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.464621+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:17:00.671707+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:17:16.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.263+0000 7f59c009f000 1 -- 172.21.15.76:0/2440620031 wait complete. 2024-09-19T04:17:16.274 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f0ebd98a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:17:00.467434+0000 1=0+1) n(v2 rc2024-09-19T04:17:00.671707+0000 4=0+4) (inest lock) caps={6330=pAsLsXsFs/-@7},l=6330 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x564dc4818b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.467434+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.671707+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.467434+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:17:00.671707+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:17:16.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59c009f000 20 client.6555 trim_cache size 0 max 16384 2024-09-19T04:17:16.275 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:16.275 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-09-19T04:17:16.275 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "083d6f2f-f483-4e72-8f57-e4d2bfd14b4e", 2024-09-19T04:17:16.275 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-09-19T04:17:16.275 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59980678f0 msgr2=0x7f5998069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59980678f0 0x7f5998069db0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f59b81682d0 tx=0x7f59a0008040 comp rx=0 tx=0).stop 2024-09-19T04:17:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 msgr2=0x7f59b8159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b8159850 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f59b0007ef0 tx=0x7f59b000e4b0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 shutdown_connections 2024-09-19T04:17:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59980678f0 0x7f5998069db0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59b815f840 0x7f59b8153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59b8158c90 0x7f59b815fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 --2- 172.21.15.76:0/2421317429 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59b80b3e00 0x7f59b8159850 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 >> 172.21.15.76:0/2421317429 conn(0x7f59b806bcc0 msgr2=0x7f59b806c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 shutdown_connections 2024-09-19T04:17:16.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f59bcfe3640 1 -- 172.21.15.76:0/2421317429 wait complete. 2024-09-19T04:17:16.277 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f9c45f06640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:17:00.465988+0000 1=0+1) n(v4 rc2024-09-19T04:17:00.671707+0000 5=0+5) (inest sync->mix g=1,2) caps={6330=pAsLsXsFs/-@11} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x560dd6095b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.465988+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.671707+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.465988+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:17:00.671707+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:17:16.278 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:17:16.267+0000 7f0ec4197640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:17:00.468809+0000 1=0+1) n(v1 rc2024-09-19T04:17:00.671707+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={6330=pAsLsXsFs/-@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x564dc4819080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.468809+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.470271+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.468809+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:17:00.671707+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:17:16.278 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-09-19T04:17:16.267+0000 7f89f9c5a640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:17:00.470271+0000 1=0+1) n(v0 rc2024-09-19T04:17:00.470271+0000 2=0+2) (inest lock) (ifile excl) caps={6330=pAsLsXsFsx/-@6},l=6330 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5639178a1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.470271+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.470271+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.470271+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:17:00.470271+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:17:16.278 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-09-19T04:17:16.267+0000 7f89f9c5a640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={6330=pAsLsXsFs/-@3},l=6330 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5639188e0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:17:16.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:16.287 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-19T04:17:16.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e823b2640 1 Processor -- start 2024-09-19T04:17:16.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e823b2640 1 -- start start 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e823b2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c1520a0 0x7f3e7c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e823b2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3e7c15f3a0 0x7f3e7c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e823b2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3e7c070250 con 0x7f3e7c06fe50 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e823b2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3e7c152480 con 0x7f3e7c15f3a0 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e823b2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c0aede0 con 0x7f3e7c1520a0 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c1520a0 0x7f3e7c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e8094f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3e7c15f3a0 0x7f3e7c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c1520a0 0x7f3e7c06f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:52086/0 (socket says 172.21.15.76:52086) 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3304895539 learned_addr learned my addr 172.21.15.76:0/3304895539 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7b7fe640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3e7c06fe50 0x7f3e7c15ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:48160/0 (socket says 172.21.15.76:48160) 2024-09-19T04:17:16.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3304895539 >> v2:172.21.15.116:3300/0 conn(0x7f3e7c15f3a0 msgr2=0x7f3e7c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3304895539 >> v2:172.21.15.116:3300/0 conn(0x7f3e7c15f3a0 0x7f3e7c0adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3304895539 >> v1:172.21.15.116:6789/0 conn(0x7f3e7c06fe50 legacy=0x7f3e7c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3304895539 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c15fa10 con 0x7f3e7c1520a0 2024-09-19T04:17:16.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.399+0000 7f3e8094f640 1 --2- 172.21.15.76:0/3304895539 >> v2:172.21.15.116:3300/0 conn(0x7f3e7c15f3a0 0x7f3e7c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:16.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3304895539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c1520a0 0x7f3e7c06f910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009e50 tx=0x7f3e7401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9562462c7884afb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e7affd640 1 -- 172.21.15.76:0/3304895539 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740a5900 con 0x7f3e7c1520a0 2024-09-19T04:17:16.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e7affd640 1 -- 172.21.15.76:0/3304895539 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e740a5aa0 con 0x7f3e7c1520a0 2024-09-19T04:17:16.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e7affd640 1 -- 172.21.15.76:0/3304895539 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740ad3f0 con 0x7f3e7c1520a0 2024-09-19T04:17:16.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 -- 172.21.15.76:0/3304895539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c1520a0 msgr2=0x7f3e7c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 --2- 172.21.15.76:0/3304895539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c1520a0 0x7f3e7c06f910 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e74009e50 tx=0x7f3e7401ca30 comp rx=0 tx=0).stop 2024-09-19T04:17:16.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 -- 172.21.15.76:0/3304895539 shutdown_connections 2024-09-19T04:17:16.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 --2- 172.21.15.76:0/3304895539 >> v2:172.21.15.116:3300/0 conn(0x7f3e7c15f3a0 0x7f3e7c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 --2- 172.21.15.76:0/3304895539 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c1520a0 0x7f3e7c06f910 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 -- 172.21.15.76:0/3304895539 >> 172.21.15.76:0/3304895539 conn(0x7f3e7c0b1f90 msgr2=0x7f3e7c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 -- 172.21.15.76:0/3304895539 shutdown_connections 2024-09-19T04:17:16.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.403+0000 7f3e823b2640 1 -- 172.21.15.76:0/3304895539 wait complete. 2024-09-19T04:17:16.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 Processor -- start 2024-09-19T04:17:16.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 -- start start 2024-09-19T04:17:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c06fe50 0x7f3e7c148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e7c1520a0 0x7f3e7c098740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e7c15f3a0 0x7f3e7c098c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c073d30 con 0x7f3e7c1520a0 2024-09-19T04:17:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c073a50 con 0x7f3e7c15f3a0 2024-09-19T04:17:16.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c165f00 con 0x7f3e7c06fe50 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e7c1520a0 0x7f3e7c098740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e7c1520a0 0x7f3e7c098740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54658/0 (socket says 172.21.15.76:54658) 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 -- 172.21.15.76:0/2267339784 learned_addr learned my addr 172.21.15.76:0/2267339784 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c06fe50 0x7f3e7c148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e8094f640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e7c15f3a0 0x7f3e7c098c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 -- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c06fe50 msgr2=0x7f3e7c148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c06fe50 0x7f3e7c148c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 -- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e7c15f3a0 msgr2=0x7f3e7c098c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e7c15f3a0 0x7f3e7c098c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 -- 172.21.15.76:0/2267339784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c151e90 con 0x7f3e7c1520a0 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e8094f640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e7c15f3a0 0x7f3e7c098c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e7b7fe640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e7c1520a0 0x7f3e7c098740 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c370 tx=0x7f3e6400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e78ff9640 1 -- 172.21.15.76:0/2267339784 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e640074e0 con 0x7f3e7c1520a0 2024-09-19T04:17:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e78ff9640 1 -- 172.21.15.76:0/2267339784 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e64007680 con 0x7f3e7c1520a0 2024-09-19T04:17:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7c14a580 con 0x7f3e7c1520a0 2024-09-19T04:17:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e78ff9640 1 -- 172.21.15.76:0/2267339784 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e64004a30 con 0x7f3e7c1520a0 2024-09-19T04:17:16.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7c149190 con 0x7f3e7c1520a0 2024-09-19T04:17:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.407+0000 7f3e78ff9640 1 -- 172.21.15.76:0/2267339784 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3e6401a020 con 0x7f3e7c1520a0 2024-09-19T04:17:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e78ff9640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3e600678c0 0x7f3e60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e78ff9640 1 -- 172.21.15.76:0/2267339784 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f3e640854e0 con 0x7f3e7c1520a0 2024-09-19T04:17:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3e600678c0 0x7f3e60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3e600678c0 0x7f3e60069d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e740ae000 tx=0x7f3e740a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 Processor -- start 2024-09-19T04:17:16.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- start start 2024-09-19T04:17:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa8d70 0x55fcc2aa9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa9690 0x55fcc2a5c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2a5cdb0 0x55fcc2a5d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcc2946960 con 0x55fcc2aa8d70 2024-09-19T04:17:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fcc288a900 con 0x55fcc2aa9690 2024-09-19T04:17:16.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55fcc28add40 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa8d70 0x55fcc2aa9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa8d70 0x55fcc2aa9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54660/0 (socket says 172.21.15.76:54660) 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3296732168 learned_addr learned my addr 172.21.15.76:0/3296732168 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7b7fe640 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa9690 0x55fcc2a5c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8094f640 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2a5cdb0 0x55fcc2a5d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2a5cdb0 msgr2=0x55fcc2a5d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2a5cdb0 0x55fcc2a5d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa9690 msgr2=0x55fcc2a5c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa9690 0x55fcc2a5c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 -- 172.21.15.76:0/3296732168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc2854580 con 0x55fcc2aa8d70 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa8d70 0x55fcc2aa9150 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3e740af040 tx=0x7f3e74004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3d6be62bfb9895 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e71ffb640 1 -- 172.21.15.76:0/3296732168 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740c6070 con 0x55fcc2aa8d70 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e71ffb640 1 -- 172.21.15.76:0/3296732168 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e740b6940 con 0x55fcc2aa8d70 2024-09-19T04:17:16.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e71ffb640 1 -- 172.21.15.76:0/3296732168 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740b6cf0 con 0x55fcc2aa8d70 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- 172.21.15.76:0/3296732168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa8d70 msgr2=0x55fcc2aa9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa8d70 0x55fcc2aa9150 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f3e740af040 tx=0x7f3e74004ac0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- 172.21.15.76:0/3296732168 shutdown_connections 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2a5cdb0 0x55fcc2a5d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa9690 0x55fcc2a5c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3296732168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa8d70 0x55fcc2aa9150 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- 172.21.15.76:0/3296732168 >> 172.21.15.76:0/3296732168 conn(0x55fcc2a8a040 msgr2=0x55fcc2a8c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- 172.21.15.76:0/3296732168 shutdown_connections 2024-09-19T04:17:16.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- 172.21.15.76:0/3296732168 wait complete. 2024-09-19T04:17:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.411+0000 7f3e8546e000 1 -- start start 2024-09-19T04:17:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 Processor -- start 2024-09-19T04:17:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2aafce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ab0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ab0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcc28a9d00 con 0x55fcc2aa9690 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fcc2868aa0 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55fcc2847340 con 0x55fcc2aa8d70 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 10 client.0 _ll_register_callbacks cb 0x55fcc2a4a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ab0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ab0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54674/0 (socket says 172.21.15.76:54674) 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 -- 172.21.15.76:0/1413668264 learned_addr learned my addr 172.21.15.76:0/1413668264 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e7b7fe640 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ab0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2aafce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 -- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 msgr2=0x55fcc2ab0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ab0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 -- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 msgr2=0x55fcc2aafce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2aafce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 -- 172.21.15.76:0/1413668264 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc2850dc0 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8094f640 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ab0760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c006d50 tx=0x7f3e6c0129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcdbb64585cc1618 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 1 -- 172.21.15.76:0/1413668264 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c014880 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 1 -- 172.21.15.76:0/1413668264 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f3e6c014a20 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e740b6940 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 1 -- 172.21.15.76:0/1413668264 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3e6c026050 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 10 client.6564 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55fcc284d7f0 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 1 -- 172.21.15.76:0/1413668264 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e740b6940 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 1 -- 172.21.15.76:0/1413668264 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7f3e6c029020 con 0x55fcc2aa9690 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 10 client.6564 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 10 client.6564 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 20 client.6564 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:16.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ab80b0 0x55fcc2aba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 4 client.6564 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fcc2847e10 con 0x55fcc2ab80b0 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ab80b0 0x55fcc2aba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 10 client.6564 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ab80b0 0x55fcc2aba4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3d82d76bbc42121 server_cookie=a0ea31f4db45406c in_seq=0 out_seq=0 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 1 -- 172.21.15.76:0/1413668264 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fcc2847e10 con 0x55fcc2ab80b0 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 10 client.6564 handle_command_reply: tid=0 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 client.6564 shutdown 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 msgr2=0x55fcc2ab0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ab0760 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c006d50 tx=0x7f3e6c0129e0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 shutdown_connections 2024-09-19T04:17:16.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ab80b0 0x55fcc2aba4a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ab0760 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ab0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e6affd640 0 client.6564 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1413668264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2aafce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 >> 172.21.15.76:0/1413668264 conn(0x55fcc2a8a040 msgr2=0x55fcc2a5bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 shutdown_connections 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 1 -- 172.21.15.76:0/1413668264 wait complete. 2024-09-19T04:17:16.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.415+0000 7f3e8546e000 20 client.6564 trim_cache size 0 max 16384 2024-09-19T04:17:16.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 Processor -- start 2024-09-19T04:17:16.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- start start 2024-09-19T04:17:16.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa8d70 0x55fcc2a64bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa9690 0x55fcc2a65110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c202380 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c201550 con 0x55fcc2aa8d70 2024-09-19T04:17:16.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3e7c201c50 con 0x55fcc2aa9690 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54684/0 (socket says 172.21.15.76:54684) 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 -- 172.21.15.76:0/1637528136 learned_addr learned my addr 172.21.15.76:0/1637528136 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 -- 172.21.15.76:0/1637528136 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa9690 msgr2=0x55fcc2a65110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/1637528136 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa9690 0x55fcc2a65110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 -- 172.21.15.76:0/1637528136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa8d70 msgr2=0x55fcc2a64bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/1637528136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa8d70 0x55fcc2a64bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 -- 172.21.15.76:0/1637528136 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fcc286b470 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/1637528136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab6360 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3e7401ca70 tx=0x7f3e740b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5657a491cec1f7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e717fa640 1 -- 172.21.15.76:0/1637528136 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740c6040 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e717fa640 1 -- 172.21.15.76:0/1637528136 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e740b6c00 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e717fa640 1 -- 172.21.15.76:0/1637528136 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e740ba040 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- 172.21.15.76:0/1637528136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2a5cdb0 msgr2=0x55fcc2ab6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1637528136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab6360 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3e7401ca70 tx=0x7f3e740b7000 comp rx=0 tx=0).stop 2024-09-19T04:17:16.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- 172.21.15.76:0/1637528136 shutdown_connections 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1637528136 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa9690 0x55fcc2a65110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1637528136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2aa8d70 0x55fcc2a64bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 --2- 172.21.15.76:0/1637528136 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab6360 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- 172.21.15.76:0/1637528136 >> 172.21.15.76:0/1637528136 conn(0x55fcc2a8a040 msgr2=0x55fcc2aafaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- 172.21.15.76:0/1637528136 shutdown_connections 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.435+0000 7f3e8546e000 1 -- 172.21.15.76:0/1637528136 wait complete. 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- start start 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 Processor -- start 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ae0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ae14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55fcc286c7c0 con 0x55fcc2aa9690 2024-09-19T04:17:16.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55fcc2a63a00 con 0x55fcc2a5cdb0 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55fcc2875400 con 0x55fcc2aa8d70 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 10 client.0 _ll_register_callbacks cb 0x7f3e6c009650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ae14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ae14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:54694/0 (socket says 172.21.15.76:54694) 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 -- 172.21.15.76:0/3304754475 learned_addr learned my addr 172.21.15.76:0/3304754475 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e7b7fe640 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ae0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 -- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 msgr2=0x55fcc2ae0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ae0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 -- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 msgr2=0x55fcc2ab5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 -- 172.21.15.76:0/3304754475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fcc2841640 con 0x55fcc2aa9690 2024-09-19T04:17:16.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8094f640 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ae14e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c012a20 tx=0x7f3e6c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5601e2f3d3afd55a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 1 -- 172.21.15.76:0/3304754475 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c005ba0 con 0x55fcc2aa9690 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 1 -- 172.21.15.76:0/3304754475 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f3e6c005d40 con 0x55fcc2aa9690 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e740b6c00 con 0x55fcc2aa9690 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 1 -- 172.21.15.76:0/3304754475 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f3e6c037050 con 0x55fcc2aa9690 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 10 client.6570 fetch_fsmap learned FSMap version 54 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x55fcc28453b0 con 0x55fcc2aa9690 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 1 -- 172.21.15.76:0/3304754475 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e740b6c00 con 0x55fcc2aa9690 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 1 -- 172.21.15.76:0/3304754475 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 54) ==== 3554+0+0 (secure 0 0 0) 0x7f3e6c03a020 con 0x55fcc2aa9690 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 10 client.6570 fetch_fsmap finished waiting for FSMap version 54 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 10 client.6570 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 20 client.6570 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:16.448 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ae1a20 0x55fcc2b253e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 4 client.6570 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fcc28bb470 con 0x55fcc2ae1a20 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ae1a20 0x55fcc2b253e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 10 client.6570 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e7bfff640 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ae1a20 0x55fcc2b253e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0316713960977a5 server_cookie=e2515f378f42a6d3 in_seq=0 out_seq=0 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 1 -- 172.21.15.76:0/3304754475 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55fcc28bb470 con 0x55fcc2ae1a20 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 10 client.6570 handle_command_reply: tid=0 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 client.6570 shutdown 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 msgr2=0x55fcc2ae14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ae14e0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c012a20 tx=0x7f3e6c004740 comp rx=0 tx=0).stop 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 shutdown_connections 2024-09-19T04:17:16.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55fcc2ae1a20 0x55fcc2b253e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55fcc2aa9690 0x55fcc2ae14e0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55fcc2aa8d70 0x55fcc2ae0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e6affd640 0 client.6570 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 --2- 172.21.15.76:0/3304754475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55fcc2a5cdb0 0x55fcc2ab5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 >> 172.21.15.76:0/3304754475 conn(0x55fcc2a8a040 msgr2=0x55fcc2ab02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 shutdown_connections 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 1 -- 172.21.15.76:0/3304754475 wait complete. 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.439+0000 7f3e8546e000 20 client.6570 trim_cache size 0 max 16384 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:16.450 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout: "083d6f2f-f483-4e72-8f57-e4d2bfd14b4e": { 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "083d6f2f-f483-4e72-8f57-e4d2bfd14b4e", 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:16.451 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3e600678c0 msgr2=0x7f3e60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3e600678c0 0x7f3e60069d80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3e740ae000 tx=0x7f3e740a7070 comp rx=0 tx=0).stop 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e7c1520a0 msgr2=0x7f3e7c098740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e7c1520a0 0x7f3e7c098740 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f3e6400c370 tx=0x7f3e6400c830 comp rx=0 tx=0).stop 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 shutdown_connections 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3e600678c0 0x7f3e60069d80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3e7c15f3a0 0x7f3e7c098c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3e7c1520a0 0x7f3e7c098740 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 --2- 172.21.15.76:0/2267339784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3e7c06fe50 0x7f3e7c148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:16.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 >> 172.21.15.76:0/2267339784 conn(0x7f3e7c0b1f90 msgr2=0x7f3e7c166290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:16.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 shutdown_connections 2024-09-19T04:17:16.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:16.443+0000 7f3e823b2640 1 -- 172.21.15.76:0/2267339784 wait complete. 2024-09-19T04:17:16.461 INFO:tasks.cephfs.filesystem:scrub status for tag:083d6f2f-f483-4e72-8f57-e4d2bfd14b4e - {'path': '/d1/d2/d3', 'tag': '083d6f2f-f483-4e72-8f57-e4d2bfd14b4e', 'options': 'recursive,force'} 2024-09-19T04:17:18.898 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:17:18.895+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:17:23.645 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:17:23.648 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:17:25.760 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:17:25.759+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:17:46.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:46.462 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-19T04:17:46.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 Processor -- start 2024-09-19T04:17:46.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 -- start start 2024-09-19T04:17:46.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f55bc1457a0 0x7f55bc0d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f55bc0cf800 0x7f55bc0b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f55bc0b4790 con 0x7f55bc1457a0 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f55bc0d27a0 con 0x7f55bc0cf800 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c26e0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc0d2a80 con 0x7f55bc0b4340 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f55bc1457a0 0x7f55bc0d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f55bc1457a0 0x7f55bc0d1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39162/0 (socket says 172.21.15.76:39162) 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 -- 172.21.15.76:0/3568828402 learned_addr learned my addr 172.21.15.76:0/3568828402 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c0c7d640 1 --2- 172.21.15.76:0/3568828402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 -- 172.21.15.76:0/3568828402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 msgr2=0x7f55bc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 --2- 172.21.15.76:0/3568828402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/3568828402 >> v2:172.21.15.116:3300/0 conn(0x7f55bc0cf800 0x7f55bc0b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 -- 172.21.15.76:0/3568828402 >> v2:172.21.15.116:3300/0 conn(0x7f55bc0cf800 msgr2=0x7f55bc0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 --2- 172.21.15.76:0/3568828402 >> v2:172.21.15.116:3300/0 conn(0x7f55bc0cf800 0x7f55bc0b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bbfff640 1 -- 172.21.15.76:0/3568828402 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc0d2d60 con 0x7f55bc1457a0 2024-09-19T04:17:46.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55c0c7d640 1 --2- 172.21.15.76:0/3568828402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.594+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/3568828402 >> v2:172.21.15.116:3300/0 conn(0x7f55bc0cf800 0x7f55bc0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bbfff640 1 --2- 172.21.15.76:0/3568828402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc1457a0 0x7f55bc0d1ea0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0cfc70 tx=0x7f55ac01c1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357059bff06b82c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55baffd640 1 -- 172.21.15.76:0/3568828402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0b0070 con 0x7f55bc1457a0 2024-09-19T04:17:46.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55baffd640 1 -- 172.21.15.76:0/3568828402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac0ab600 con 0x7f55bc1457a0 2024-09-19T04:17:46.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55baffd640 1 -- 172.21.15.76:0/3568828402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0ab9d0 con 0x7f55bc1457a0 2024-09-19T04:17:46.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- 172.21.15.76:0/3568828402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc1457a0 msgr2=0x7f55bc0d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 --2- 172.21.15.76:0/3568828402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc1457a0 0x7f55bc0d1ea0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f55bc0cfc70 tx=0x7f55ac01c1d0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- 172.21.15.76:0/3568828402 shutdown_connections 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 --2- 172.21.15.76:0/3568828402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 --2- 172.21.15.76:0/3568828402 >> v2:172.21.15.116:3300/0 conn(0x7f55bc0cf800 0x7f55bc0b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 --2- 172.21.15.76:0/3568828402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc1457a0 0x7f55bc0d1ea0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- 172.21.15.76:0/3568828402 >> 172.21.15.76:0/3568828402 conn(0x7f55bc06bcc0 msgr2=0x7f55bc06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- 172.21.15.76:0/3568828402 shutdown_connections 2024-09-19T04:17:46.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- 172.21.15.76:0/3568828402 wait complete. 2024-09-19T04:17:46.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 Processor -- start 2024-09-19T04:17:46.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- start start 2024-09-19T04:17:46.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc0cf800 0x7f55bc1657e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55bc1457a0 0x7f55bc13ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc0d0890 con 0x7f55bc0cf800 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc0d0710 con 0x7f55bc1457a0 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c26e0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc0d0a10 con 0x7f55bc0b4340 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc0cf800 0x7f55bc1657e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc0cf800 0x7f55bc1657e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39166/0 (socket says 172.21.15.76:39166) 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1040953396 learned_addr learned my addr 172.21.15.76:0/1040953396 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c0c7d640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55bc1457a0 0x7f55bc13ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bbfff640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 msgr2=0x7f55bc140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc140c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55bc1457a0 msgr2=0x7f55bc13ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55bc1457a0 0x7f55bc13ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1040953396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc15aed0 con 0x7f55bc0cf800 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55c0c7d640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55bc1457a0 0x7f55bc13ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bbfff640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.598+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc0cf800 0x7f55bc1657e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c7c0 tx=0x7f55b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55b8ff9640 1 -- 172.21.15.76:0/1040953396 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0007a30 con 0x7f55bc0cf800 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55b8ff9640 1 -- 172.21.15.76:0/1040953396 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b0007bd0 con 0x7f55bc0cf800 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc15b1b0 con 0x7f55bc0cf800 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55b8ff9640 1 -- 172.21.15.76:0/1040953396 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b00113f0 con 0x7f55bc0cf800 2024-09-19T04:17:46.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc15b770 con 0x7f55bc0cf800 2024-09-19T04:17:46.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55b8ff9640 1 -- 172.21.15.76:0/1040953396 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f55b0011590 con 0x7f55bc0cf800 2024-09-19T04:17:46.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55b8ff9640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55a80678c0 0x7f55a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55b8ff9640 1 -- 172.21.15.76:0/1040953396 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f55b0086160 con 0x7f55bc0cf800 2024-09-19T04:17:46.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55bbfff640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55a80678c0 0x7f55a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55bbfff640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55a80678c0 0x7f55a8069d80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0ad000 tx=0x7f55ac0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 Processor -- start 2024-09-19T04:17:46.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 -- start start 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2ef1d0 0x5622ef2ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2efaf0 0x5622ef2a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2a31c0 0x5622ef2a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5622ef188960 con 0x5622ef2efaf0 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5622ef0cc900 con 0x5622ef2ef1d0 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.602+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5622ef0efd40 con 0x5622ef2a31c0 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2efaf0 0x5622ef2a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2efaf0 0x5622ef2a2c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39180/0 (socket says 172.21.15.76:39180) 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1815133156 learned_addr learned my addr 172.21.15.76:0/1815133156 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1815133156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2a31c0 msgr2=0x5622ef2a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/1815133156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2a31c0 0x5622ef2a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1815133156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2ef1d0 msgr2=0x5622ef2ef5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/1815133156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2ef1d0 0x5622ef2ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/1815133156 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ef096580 con 0x5622ef2efaf0 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/1815133156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2efaf0 0x5622ef2a2c80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f55b000afd0 tx=0x7f55b0055d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16752b40732b6c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55a1ffb640 1 -- 172.21.15.76:0/1815133156 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b006e070 con 0x5622ef2efaf0 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55a1ffb640 1 -- 172.21.15.76:0/1815133156 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b005eec0 con 0x5622ef2efaf0 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55a1ffb640 1 -- 172.21.15.76:0/1815133156 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b001e040 con 0x5622ef2efaf0 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- 172.21.15.76:0/1815133156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2efaf0 msgr2=0x5622ef2a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 --2- 172.21.15.76:0/1815133156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2efaf0 0x5622ef2a2c80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f55b000afd0 tx=0x7f55b0055d70 comp rx=0 tx=0).stop 2024-09-19T04:17:46.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- 172.21.15.76:0/1815133156 shutdown_connections 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 --2- 172.21.15.76:0/1815133156 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2a31c0 0x5622ef2a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 --2- 172.21.15.76:0/1815133156 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2efaf0 0x5622ef2a2c80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 --2- 172.21.15.76:0/1815133156 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2ef1d0 0x5622ef2ef5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- 172.21.15.76:0/1815133156 >> 172.21.15.76:0/1815133156 conn(0x5622ef2cc040 msgr2=0x5622ef2ce430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- 172.21.15.76:0/1815133156 shutdown_connections 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- 172.21.15.76:0/1815133156 wait complete. 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- start start 2024-09-19T04:17:46.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 Processor -- start 2024-09-19T04:17:46.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 0x5622ef2cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef2cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef2b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5622ef0ebd00 con 0x5622ef2ef1d0 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5622ef0aaaa0 con 0x5622ef2a31c0 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5622ef089340 con 0x5622ef2efaf0 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 10 client.0 _ll_register_callbacks cb 0x5622ef28c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef2cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef2cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39194/0 (socket says 172.21.15.76:39194) 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/4079041129 learned_addr learned my addr 172.21.15.76:0/4079041129 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c0c7d640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef2b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bbfff640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 0x5622ef2cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 msgr2=0x5622ef2b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef2b02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 msgr2=0x5622ef2cb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 0x5622ef2cb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 -- 172.21.15.76:0/4079041129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ef092dc0 con 0x5622ef2ef1d0 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c0c7d640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef2b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef2cb710 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f55b000afd0 tx=0x7f55b0000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d08e8c453d3059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f559affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f559affd640 1 -- 172.21.15.76:0/4079041129 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0082070 con 0x5622ef2ef1d0 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f559affd640 1 -- 172.21.15.76:0/4079041129 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f55b006fe10 con 0x5622ef2ef1d0 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55b005eec0 con 0x5622ef2ef1d0 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f559affd640 1 -- 172.21.15.76:0/4079041129 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f55b001e040 con 0x5622ef2ef1d0 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 10 client.6582 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:46.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.606+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x5622ef08f7f0 con 0x5622ef2ef1d0 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f559affd640 1 -- 172.21.15.76:0/4079041129 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55b005eec0 con 0x5622ef2ef1d0 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f559affd640 1 -- 172.21.15.76:0/4079041129 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f55b0055730 con 0x5622ef2ef1d0 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 10 client.6582 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 10 client.6582 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 20 client.6582 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef2b0d00 0x5622ef301280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 4 client.6582 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622ef089e10 con 0x5622ef2b0d00 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55bbfff640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef2b0d00 0x5622ef301280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f559affd640 10 client.6582 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55bbfff640 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef2b0d00 0x5622ef301280 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ee76c815b7f80a server_cookie=22938015f92eef20 in_seq=0 out_seq=0 2024-09-19T04:17:46.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f559affd640 1 -- 172.21.15.76:0/4079041129 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5622ef089e10 con 0x5622ef2b0d00 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f559affd640 10 client.6582 handle_command_reply: tid=0 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 client.6582 shutdown 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 msgr2=0x5622ef2cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef2cb710 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f55b000afd0 tx=0x7f55b0000bd0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 shutdown_connections 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef2b0d00 0x5622ef301280 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef2b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef2cb710 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f559affd640 0 client.6582 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 --2- 172.21.15.76:0/4079041129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 0x5622ef2cb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 >> 172.21.15.76:0/4079041129 conn(0x5622ef2cc040 msgr2=0x5622ef29df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 shutdown_connections 2024-09-19T04:17:46.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 1 -- 172.21.15.76:0/4079041129 wait complete. 2024-09-19T04:17:46.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.610+0000 7f55c579c000 20 client.6582 trim_cache size 0 max 16384 2024-09-19T04:17:46.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.626+0000 7f55c579c000 1 Processor -- start 2024-09-19T04:17:46.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.626+0000 7f55c579c000 1 -- start start 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2a31c0 0x5622ef3046d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2ef1d0 0x5622ef2aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2efaf0 0x5622ef2af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc200020 con 0x5622ef2a31c0 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc06ec50 con 0x5622ef2efaf0 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc1ff8f0 con 0x5622ef2ef1d0 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2a31c0 0x5622ef3046d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2a31c0 0x5622ef3046d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39206/0 (socket says 172.21.15.76:39206) 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 -- 172.21.15.76:0/2701579803 learned_addr learned my addr 172.21.15.76:0/2701579803 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 -- 172.21.15.76:0/2701579803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2ef1d0 msgr2=0x5622ef2aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 --2- 172.21.15.76:0/2701579803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2ef1d0 0x5622ef2aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 -- 172.21.15.76:0/2701579803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2efaf0 msgr2=0x5622ef2af3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 --2- 172.21.15.76:0/2701579803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2efaf0 0x5622ef2af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 -- 172.21.15.76:0/2701579803 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ef0ad470 con 0x5622ef2a31c0 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bbfff640 1 --2- 172.21.15.76:0/2701579803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2a31c0 0x5622ef3046d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f55ac006c90 tx=0x7f55ac0a8000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b9d10ef3b12863e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55a17fa640 1 -- 172.21.15.76:0/2701579803 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac0abd30 con 0x5622ef2a31c0 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55a17fa640 1 -- 172.21.15.76:0/2701579803 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac0b1040 con 0x5622ef2a31c0 2024-09-19T04:17:46.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55a17fa640 1 -- 172.21.15.76:0/2701579803 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55ac003820 con 0x5622ef2a31c0 2024-09-19T04:17:46.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- 172.21.15.76:0/2701579803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2a31c0 msgr2=0x5622ef3046d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- 172.21.15.76:0/2701579803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2a31c0 0x5622ef3046d0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f55ac006c90 tx=0x7f55ac0a8000 comp rx=0 tx=0).stop 2024-09-19T04:17:46.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- 172.21.15.76:0/2701579803 shutdown_connections 2024-09-19T04:17:46.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- 172.21.15.76:0/2701579803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2efaf0 0x5622ef2af3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- 172.21.15.76:0/2701579803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2ef1d0 0x5622ef2aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- 172.21.15.76:0/2701579803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2a31c0 0x5622ef3046d0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- 172.21.15.76:0/2701579803 >> 172.21.15.76:0/2701579803 conn(0x5622ef2cc040 msgr2=0x5622ef29d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- 172.21.15.76:0/2701579803 shutdown_connections 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- 172.21.15.76:0/2701579803 wait complete. 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- start start 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 Processor -- start 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 0x5622ef303c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef3630f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef363630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5622ef0b7400 con 0x5622ef2ef1d0 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5622ef0ef050 con 0x5622ef2a31c0 2024-09-19T04:17:46.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5622ef2aa3f0 con 0x5622ef2efaf0 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 10 client.0 _ll_register_callbacks cb 0x7f55b0055c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef3630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef3630f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39218/0 (socket says 172.21.15.76:39218) 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 -- 172.21.15.76:0/2035336599 learned_addr learned my addr 172.21.15.76:0/2035336599 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c0c7d640 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef363630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 -- 172.21.15.76:0/2035336599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 msgr2=0x5622ef363630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef363630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 -- 172.21.15.76:0/2035336599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 msgr2=0x5622ef303c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 0x5622ef303c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 -- 172.21.15.76:0/2035336599 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622ef083640 con 0x5622ef2ef1d0 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55bb7fe640 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef3630f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f55b0056270 tx=0x7f55b00842a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a785ea07f41b83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f559affd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f559affd640 1 -- 172.21.15.76:0/2035336599 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0082070 con 0x5622ef2ef1d0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f559affd640 1 -- 172.21.15.76:0/2035336599 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f55b00551a0 con 0x5622ef2ef1d0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55ac0b1040 con 0x5622ef2ef1d0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f559affd640 1 -- 172.21.15.76:0/2035336599 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f55b006e040 con 0x5622ef2ef1d0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 10 client.6588 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x5622ef0873b0 con 0x5622ef2ef1d0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.630+0000 7f559affd640 1 -- 172.21.15.76:0/2035336599 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55ac0b1040 con 0x5622ef2ef1d0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f559affd640 1 -- 172.21.15.76:0/2035336599 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f55b0011880 con 0x5622ef2ef1d0 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 10 client.6588 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 10 client.6588 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 20 client.6588 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef363b70 0x5622ef35fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 4 client.6588 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622ef0fd470 con 0x5622ef363b70 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55bbfff640 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef363b70 0x5622ef35fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f559affd640 10 client.6588 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55bbfff640 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef363b70 0x5622ef35fb50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14ee95a7238c0e65 server_cookie=f24a46bc0bae7102 in_seq=0 out_seq=0 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f559affd640 1 -- 172.21.15.76:0/2035336599 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622ef0fd470 con 0x5622ef363b70 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f559affd640 10 client.6588 handle_command_reply: tid=0 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 client.6588 shutdown 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 msgr2=0x5622ef3630f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef3630f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f55b0056270 tx=0x7f55b00842a0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 shutdown_connections 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x5622ef363b70 0x5622ef35fb50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5622ef2efaf0 0x5622ef363630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5622ef2ef1d0 0x5622ef3630f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 --2- 172.21.15.76:0/2035336599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5622ef2a31c0 0x5622ef303c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f559affd640 0 client.6588 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 >> 172.21.15.76:0/2035336599 conn(0x5622ef2cc040 msgr2=0x5622ef29d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 shutdown_connections 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 1 -- 172.21.15.76:0/2035336599 wait complete. 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c579c000 20 client.6588 trim_cache size 0 max 16384 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:46.644 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "no active scrubs running", 2024-09-19T04:17:46.645 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": {} 2024-09-19T04:17:46.645 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55a80678c0 msgr2=0x7f55a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55a80678c0 0x7f55a8069d80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0ad000 tx=0x7f55ac0a6000 comp rx=0 tx=0).stop 2024-09-19T04:17:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc0cf800 msgr2=0x7f55bc1657e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc0cf800 0x7f55bc1657e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c7c0 tx=0x7f55b000cc80 comp rx=0 tx=0).stop 2024-09-19T04:17:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 shutdown_connections 2024-09-19T04:17:46.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f55a80678c0 0x7f55a8069d80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55bc1457a0 0x7f55bc13ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f55bc0cf800 0x7f55bc1657e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 --2- 172.21.15.76:0/1040953396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f55bc0b4340 0x7f55bc140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 >> 172.21.15.76:0/1040953396 conn(0x7f55bc06bcc0 msgr2=0x7f55bc0b7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 shutdown_connections 2024-09-19T04:17:46.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.634+0000 7f55c26e0640 1 -- 172.21.15.76:0/1040953396 wait complete. 2024-09-19T04:17:46.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:17:46.655 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 damage ls 2024-09-19T04:17:46.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.762+0000 7f01f8584640 1 Processor -- start 2024-09-19T04:17:46.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.762+0000 7f01f8584640 1 -- start start 2024-09-19T04:17:46.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.762+0000 7f01f8584640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01f015bec0 0x7f01f00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.762+0000 7f01f8584640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f00b3290 con 0x7f01f00af470 2024-09-19T04:17:46.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.762+0000 7f01f8584640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01f00b3570 con 0x7f01f015c2a0 2024-09-19T04:17:46.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.762+0000 7f01f8584640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f01f00b3850 con 0x7f01f015bec0 2024-09-19T04:17:46.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f6320640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f01f015bec0 0x7f01f00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f6b21640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f01f00af470 0x7f01f006e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57914/0 (socket says 172.21.15.76:57914) 2024-09-19T04:17:46.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f6b21640 1 -- 172.21.15.76:0/2814672680 learned_addr learned my addr 172.21.15.76:0/2814672680 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:46.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f6320640 1 -- 172.21.15.76:0/2814672680 >> v1:172.21.15.116:6789/0 conn(0x7f01f015c2a0 legacy=0x7f01f00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f6320640 1 -- 172.21.15.76:0/2814672680 >> v1:172.21.15.76:6789/0 conn(0x7f01f00af470 legacy=0x7f01f006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f6320640 1 -- 172.21.15.76:0/2814672680 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f00b3b30 con 0x7f01f015bec0 2024-09-19T04:17:46.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f531e640 1 -- 172.21.15.76:0/2814672680 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2776199449 0 0) 0x7f01f00b3290 con 0x7f01f00af470 2024-09-19T04:17:46.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f6320640 1 --2- 172.21.15.76:0/2814672680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f00ad200 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097c0 tx=0x7f01e801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c86b1422baddf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f531e640 1 -- 172.21.15.76:0/2814672680 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80ab430 con 0x7f01f015bec0 2024-09-19T04:17:46.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f531e640 1 -- 172.21.15.76:0/2814672680 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e80b0070 con 0x7f01f015bec0 2024-09-19T04:17:46.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f531e640 1 -- 172.21.15.76:0/2814672680 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e80ab7f0 con 0x7f01f015bec0 2024-09-19T04:17:46.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f8584640 1 -- 172.21.15.76:0/2814672680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 msgr2=0x7f01f00ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f8584640 1 --2- 172.21.15.76:0/2814672680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f00ad200 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097c0 tx=0x7f01e801c430 comp rx=0 tx=0).stop 2024-09-19T04:17:46.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f8584640 1 -- 172.21.15.76:0/2814672680 shutdown_connections 2024-09-19T04:17:46.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f8584640 1 --2- 172.21.15.76:0/2814672680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f00ad200 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f8584640 1 -- 172.21.15.76:0/2814672680 >> 172.21.15.76:0/2814672680 conn(0x7f01f00b1f90 msgr2=0x7f01f00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f8584640 1 -- 172.21.15.76:0/2814672680 shutdown_connections 2024-09-19T04:17:46.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.766+0000 7f01f8584640 1 -- 172.21.15.76:0/2814672680 wait complete. 2024-09-19T04:17:46.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 Processor -- start 2024-09-19T04:17:46.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 -- start start 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f00af470 0x7f01f0156bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01f015c2a0 0x7f01f015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01f006ef20 con 0x7f01f015bec0 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01f00ad740 con 0x7f01f00af470 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f01f006f0a0 con 0x7f01f015c2a0 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f014daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39234/0 (socket says 172.21.15.76:39234) 2024-09-19T04:17:46.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1431261667 learned_addr learned my addr 172.21.15.76:0/1431261667 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f6320640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f00af470 0x7f01f0156bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f6b21640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01f015c2a0 0x7f01f015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01f015c2a0 msgr2=0x7f01f015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01f015c2a0 0x7f01f015a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f00af470 msgr2=0x7f01f0156bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f00af470 0x7f01f0156bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1431261667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f01572d0 con 0x7f01f015bec0 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f6b21640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01f015c2a0 0x7f01f015a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f6320640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f00af470 0x7f01f0156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f014daf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b9c0 tx=0x7f01ec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01e6ffd640 1 -- 172.21.15.76:0/1431261667 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec00cba0 con 0x7f01f015bec0 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01e6ffd640 1 -- 172.21.15.76:0/1431261667 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec00cd40 con 0x7f01f015bec0 2024-09-19T04:17:46.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01e6ffd640 1 -- 172.21.15.76:0/1431261667 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec003bc0 con 0x7f01f015bec0 2024-09-19T04:17:46.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f01575b0 con 0x7f01f015bec0 2024-09-19T04:17:46.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.770+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f0157b70 con 0x7f01f015bec0 2024-09-19T04:17:46.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01e6ffd640 1 -- 172.21.15.76:0/1431261667 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f01ec006d10 con 0x7f01f015bec0 2024-09-19T04:17:46.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01e6ffd640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01d40678c0 0x7f01d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01e6ffd640 1 -- 172.21.15.76:0/1431261667 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f01ec085a30 con 0x7f01f015bec0 2024-09-19T04:17:46.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f6320640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01d40678c0 0x7f01d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f6320640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01d40678c0 0x7f01d4069d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01e80af000 tx=0x7f01e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 Processor -- start 2024-09-19T04:17:46.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 -- start start 2024-09-19T04:17:46.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957d2380 0x5591957d2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x559195788290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957887d0 0x559195788bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55919566fc40 con 0x5591957d2ca0 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5591955d6d50 con 0x5591957887d0 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559195591ab0 con 0x5591957d2380 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x559195788290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x559195788290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39242/0 (socket says 172.21.15.76:39242) 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1618411424 learned_addr learned my addr 172.21.15.76:0/1618411424 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1618411424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957d2380 msgr2=0x5591957d2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/1618411424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957d2380 0x5591957d2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1618411424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957887d0 msgr2=0x559195788bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/1618411424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957887d0 0x559195788bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 -- 172.21.15.76:0/1618411424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919557d590 con 0x5591957d2ca0 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/1618411424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x559195788290 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f01ec052140 tx=0x7f01ec055da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc43fbac3573da43 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01d3fff640 1 -- 172.21.15.76:0/1618411424 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec06b070 con 0x5591957d2ca0 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01d3fff640 1 -- 172.21.15.76:0/1618411424 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec053fe0 con 0x5591957d2ca0 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01d3fff640 1 -- 172.21.15.76:0/1618411424 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec05e430 con 0x5591957d2ca0 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 -- 172.21.15.76:0/1618411424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 msgr2=0x559195788290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 --2- 172.21.15.76:0/1618411424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x559195788290 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f01ec052140 tx=0x7f01ec055da0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 -- 172.21.15.76:0/1618411424 shutdown_connections 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 --2- 172.21.15.76:0/1618411424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957887d0 0x559195788bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 --2- 172.21.15.76:0/1618411424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x559195788290 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 --2- 172.21.15.76:0/1618411424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957d2380 0x5591957d2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.774+0000 7f01fb640000 1 -- 172.21.15.76:0/1618411424 >> 172.21.15.76:0/1618411424 conn(0x55919577fbc0 msgr2=0x559195786b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- 172.21.15.76:0/1618411424 shutdown_connections 2024-09-19T04:17:46.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- 172.21.15.76:0/1618411424 wait complete. 2024-09-19T04:17:46.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- start start 2024-09-19T04:17:46.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 Processor -- start 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x55919578d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2380 0x5591957bc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x5591957bcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559195595400 con 0x5591957d2ca0 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5591956933a0 con 0x5591957d2380 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559195616260 con 0x5591957887d0 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 10 client.0 _ll_register_callbacks cb 0x559195638020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x5591957bcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x5591957bcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39246/0 (socket says 172.21.15.76:39246) 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 -- 172.21.15.76:0/1449969430 learned_addr learned my addr 172.21.15.76:0/1449969430 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2380 0x5591957bc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6320640 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x55919578d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 -- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 msgr2=0x55919578d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x55919578d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 -- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2380 msgr2=0x5591957bc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2380 0x5591957bc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 -- 172.21.15.76:0/1449969430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559195579dd0 con 0x5591957d2ca0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6b21640 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x5591957bcb10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f01e0001e90 tx=0x7f01e000d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b859823c97876546 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 1 -- 172.21.15.76:0/1449969430 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e00072d0 con 0x5591957d2ca0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 1 -- 172.21.15.76:0/1449969430 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f01e0013ca0 con 0x5591957d2ca0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec053fe0 con 0x5591957d2ca0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 1 -- 172.21.15.76:0/1449969430 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f01e0007470 con 0x5591957d2ca0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 10 client.6603 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x559195576800 con 0x5591957d2ca0 2024-09-19T04:17:46.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 1 -- 172.21.15.76:0/1449969430 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec053fe0 con 0x5591957d2ca0 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 1 -- 172.21.15.76:0/1449969430 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f01e00047f0 con 0x5591957d2ca0 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 10 client.6603 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 10 client.6603 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 20 client.6603 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919578dda0 0x5591957d9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 4 client.6603 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559195570e20 con 0x55919578dda0 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6320640 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919578dda0 0x5591957d9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 10 client.6603 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01f6320640 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919578dda0 0x5591957d9150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e6a260f1c2a566d server_cookie=23e5de76814999e0 in_seq=0 out_seq=0 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 1 -- 172.21.15.76:0/1449969430 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559195570e20 con 0x55919578dda0 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01d17fa640 10 client.6603 handle_command_reply: tid=0 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.778+0000 7f01fb640000 1 client.6603 shutdown 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 msgr2=0x5591957bcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x5591957bcb10 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f01e0001e90 tx=0x7f01e000d5e0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 shutdown_connections 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919578dda0 0x5591957d9150 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2ca0 0x5591957bcb10 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2380 0x5591957bc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01d17fa640 0 client.6603 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 --2- 172.21.15.76:0/1449969430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x55919578d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 >> 172.21.15.76:0/1449969430 conn(0x55919577fbc0 msgr2=0x559195787280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 shutdown_connections 2024-09-19T04:17:46.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 1 -- 172.21.15.76:0/1449969430 wait complete. 2024-09-19T04:17:46.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.782+0000 7f01fb640000 20 client.6603 trim_cache size 0 max 16384 2024-09-19T04:17:46.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 Processor -- start 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 -- start start 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x559195849d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x55919584a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 0x55919584a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0200ef0 con 0x5591957d2380 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0201620 con 0x5591957d2ca0 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f01f014f1e0 con 0x5591957887d0 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x55919584a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x55919584a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39248/0 (socket says 172.21.15.76:39248) 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 -- 172.21.15.76:0/3376903776 learned_addr learned my addr 172.21.15.76:0/3376903776 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 -- 172.21.15.76:0/3376903776 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 msgr2=0x559195849d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/3376903776 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x559195849d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 -- 172.21.15.76:0/3376903776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 msgr2=0x55919584a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/3376903776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 0x55919584a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 -- 172.21.15.76:0/3376903776 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559195594480 con 0x5591957d2380 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/3376903776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x55919584a260 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f01ec05a250 tx=0x7f01ec085000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceda9876a7381a6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01d37fe640 1 -- 172.21.15.76:0/3376903776 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec06b040 con 0x5591957d2380 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01d37fe640 1 -- 172.21.15.76:0/3376903776 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec05e8c0 con 0x5591957d2380 2024-09-19T04:17:46.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.794+0000 7f01d37fe640 1 -- 172.21.15.76:0/3376903776 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec052350 con 0x5591957d2380 2024-09-19T04:17:46.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/3376903776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 msgr2=0x55919584a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/3376903776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x55919584a260 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f01ec05a250 tx=0x7f01ec085000 comp rx=0 tx=0).stop 2024-09-19T04:17:46.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/3376903776 shutdown_connections 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/3376903776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 0x55919584a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/3376903776 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x55919584a260 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/3376903776 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x559195849d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/3376903776 >> 172.21.15.76:0/3376903776 conn(0x55919577fbc0 msgr2=0x5591957bb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/3376903776 shutdown_connections 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/3376903776 wait complete. 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- start start 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 Processor -- start 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x55919584a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x5591957e7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 0x5591957e84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55919557cf30 con 0x5591957d2380 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5591955957d0 con 0x5591957d2ca0 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5591955d6060 con 0x5591957887d0 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 10 client.0 _ll_register_callbacks cb 0x7f01e0007c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x5591957e7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x5591957e7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39262/0 (socket says 172.21.15.76:39262) 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 -- 172.21.15.76:0/804116177 learned_addr learned my addr 172.21.15.76:0/804116177 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 -- 172.21.15.76:0/804116177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 msgr2=0x55919584a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x55919584a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 -- 172.21.15.76:0/804116177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 msgr2=0x5591957e84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 0x5591957e84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 -- 172.21.15.76:0/804116177 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55919556a650 con 0x5591957d2380 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f5b1f640 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x5591957e7f80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b990 tx=0x7f01ec00c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2623b0a74d5679 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 1 -- 172.21.15.76:0/804116177 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec078020 con 0x5591957d2380 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 1 -- 172.21.15.76:0/804116177 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f01ec05e430 con 0x5591957d2380 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01ec05e8c0 con 0x5591957d2380 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 1 -- 172.21.15.76:0/804116177 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f01ec071040 con 0x5591957d2380 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 10 client.6609 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55919556e3c0 con 0x5591957d2380 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 1 -- 172.21.15.76:0/804116177 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01ec05e8c0 con 0x5591957d2380 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 1 -- 172.21.15.76:0/804116177 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f01ec055130 con 0x5591957d2380 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 10 client.6609 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 10 client.6609 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 20 client.6609 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919584af00 0x5591957e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 4 client.6609 mds_command: new command op to 6165 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 --> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591955e4480 con 0x55919584af00 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f6320640 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919584af00 0x5591957e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01f6320640 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919584af00 0x5591957e8a00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7679e9eea21efde server_cookie=51de474068b7f4b8 in_seq=0 out_seq=0 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 10 client.6609 ms_handle_connect on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 1 -- 172.21.15.76:0/804116177 <== mds.0 v2:172.21.15.76:6836/3052694229 1 ==== command_reply(tid 0: 0 ) ==== 8+0+272 (crc 0 0 0) 0x5591955e4480 con 0x55919584af00 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 10 client.6609 handle_command_reply: tid=0 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 client.6609 shutdown 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 msgr2=0x5591957e7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x5591957e7f80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b990 tx=0x7f01ec00c610 comp rx=0 tx=0).stop 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 shutdown_connections 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:6836/3052694229,v1:172.21.15.76:6837/3052694229] conn(0x55919584af00 0x5591957e8a00 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5591957d2ca0 0x5591957e84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01d17fa640 0 client.6609 ms_handle_reset on v2:172.21.15.76:6836/3052694229 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5591957d2380 0x5591957e7f80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 --2- 172.21.15.76:0/804116177 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5591957887d0 0x55919584a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 >> 172.21.15.76:0/804116177 conn(0x55919577fbc0 msgr2=0x5591957bba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 shutdown_connections 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.798+0000 7f01fb640000 1 -- 172.21.15.76:0/804116177 wait complete. 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01fb640000 20 client.6609 trim_cache size 0 max 16384 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:46.811 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 2822133692, 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511627779, 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3/d4" 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 4293077121, 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511627778, 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3" 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:46.812 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:17:46.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01d40678c0 msgr2=0x7f01d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01d40678c0 0x7f01d4069d80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f01e80af000 tx=0x7f01e8008040 comp rx=0 tx=0).stop 2024-09-19T04:17:46.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 msgr2=0x7f01f014daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f014daf0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f01ec00b9c0 tx=0x7f01ec00be80 comp rx=0 tx=0).stop 2024-09-19T04:17:46.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 shutdown_connections 2024-09-19T04:17:46.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01d40678c0 0x7f01d4069d80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f01f015c2a0 0x7f01f015a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f01f015bec0 0x7f01f014daf0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 --2- 172.21.15.76:0/1431261667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01f00af470 0x7f01f0156bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 >> 172.21.15.76:0/1431261667 conn(0x7f01f00b1f90 msgr2=0x7f01f01597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 shutdown_connections 2024-09-19T04:17:46.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.802+0000 7f01f8584640 1 -- 172.21.15.76:0/1431261667 wait complete. 2024-09-19T04:17:46.822 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:1 damage ls 2024-09-19T04:17:46.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 Processor -- start 2024-09-19T04:17:46.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 -- start start 2024-09-19T04:17:46.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f7415bec0 0x7f8f740ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8f7415c2a0 0x7f8f740aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f740af2a0 0x7f8f7415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8f740b3290 con 0x7f8f7415bec0 2024-09-19T04:17:46.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8f740b3570 con 0x7f8f7415c2a0 2024-09-19T04:17:46.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7ba7e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8f740b3850 con 0x7f8f740af2a0 2024-09-19T04:17:46.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7981a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f7415bec0 0x7f8f740ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7981a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8f7415bec0 0x7f8f740ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39270/0 (socket says 172.21.15.76:39270) 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7981a640 1 -- 172.21.15.76:0/1768442011 learned_addr learned my addr 172.21.15.76:0/1768442011 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/1768442011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f740af2a0 0x7f8f7415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.934+0000 7f8f79019640 1 --2- 172.21.15.76:0/1768442011 >> v2:172.21.15.116:3300/0 conn(0x7f8f7415c2a0 0x7f8f740aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7981a640 1 -- 172.21.15.76:0/1768442011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f740af2a0 msgr2=0x7f8f7415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1768442011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f740af2a0 0x7f8f7415fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7981a640 1 -- 172.21.15.76:0/1768442011 >> v2:172.21.15.116:3300/0 conn(0x7f8f7415c2a0 msgr2=0x7f8f740aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1768442011 >> v2:172.21.15.116:3300/0 conn(0x7f8f7415c2a0 0x7f8f740aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7981a640 1 -- 172.21.15.76:0/1768442011 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f740b3b30 con 0x7f8f7415bec0 2024-09-19T04:17:46.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1768442011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f740ad200 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8f740ae510 tx=0x7f8f6401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ad2906b80d89c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f78818640 1 -- 172.21.15.76:0/1768442011 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640a5580 con 0x7f8f7415bec0 2024-09-19T04:17:46.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f78818640 1 -- 172.21.15.76:0/1768442011 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f640a5720 con 0x7f8f7415bec0 2024-09-19T04:17:46.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f78818640 1 -- 172.21.15.76:0/1768442011 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640a5af0 con 0x7f8f7415bec0 2024-09-19T04:17:46.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1768442011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 msgr2=0x7f8f740ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1768442011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f740ad200 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8f740ae510 tx=0x7f8f6401c700 comp rx=0 tx=0).stop 2024-09-19T04:17:46.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1768442011 shutdown_connections 2024-09-19T04:17:46.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1768442011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f740af2a0 0x7f8f7415fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1768442011 >> v2:172.21.15.116:3300/0 conn(0x7f8f7415c2a0 0x7f8f740aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1768442011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f740ad200 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1768442011 >> 172.21.15.76:0/1768442011 conn(0x7f8f740b1f90 msgr2=0x7f8f740b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1768442011 shutdown_connections 2024-09-19T04:17:46.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.938+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1768442011 wait complete. 2024-09-19T04:17:46.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 Processor -- start 2024-09-19T04:17:46.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 -- start start 2024-09-19T04:17:46.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f740af2a0 0x7f8f74149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f74150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f7415c2a0 0x7f8f7414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f740af6b0 con 0x7f8f7415bec0 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f740ad740 con 0x7f8f740af2a0 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7406e000 con 0x7f8f7415c2a0 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f74150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f74150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39280/0 (socket says 172.21.15.76:39280) 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 -- 172.21.15.76:0/1989456739 learned_addr learned my addr 172.21.15.76:0/1989456739 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f7415c2a0 0x7f8f7414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f740af2a0 0x7f8f74149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 -- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f7415c2a0 msgr2=0x7f8f7414dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f7415c2a0 0x7f8f7414dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 -- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f740af2a0 msgr2=0x7f8f74149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f740af2a0 0x7f8f74149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 -- 172.21.15.76:0/1989456739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f74152a70 con 0x7f8f7415bec0 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f7415c2a0 0x7f8f7414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f740af2a0 0x7f8f74149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f79019640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f74150fc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d800 tx=0x7f8f7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f6a7fc640 1 -- 172.21.15.76:0/1989456739 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000b6c0 con 0x7f8f7415bec0 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f6a7fc640 1 -- 172.21.15.76:0/1989456739 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f70019070 con 0x7f8f7415bec0 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f6a7fc640 1 -- 172.21.15.76:0/1989456739 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f7000ba80 con 0x7f8f7415bec0 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f74154e00 con 0x7f8f7415bec0 2024-09-19T04:17:46.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f741553c0 con 0x7f8f7415bec0 2024-09-19T04:17:46.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.942+0000 7f8f6a7fc640 1 -- 172.21.15.76:0/1989456739 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8f7000bc20 con 0x7f8f7415bec0 2024-09-19T04:17:46.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f6a7fc640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8f600678c0 0x7f8f60069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f6a7fc640 1 -- 172.21.15.76:0/1989456739 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f8f70086330 con 0x7f8f7415bec0 2024-09-19T04:17:46.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8f600678c0 0x7f8f60069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8f600678c0 0x7f8f60069d80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f640ae000 tx=0x7f8f640a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 Processor -- start 2024-09-19T04:17:46.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- start start 2024-09-19T04:17:46.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3855760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855ca0 0x561ac380b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac380bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ac36f2c40 con 0x561ac3855380 2024-09-19T04:17:46.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ac3659d50 con 0x561ac3855ca0 2024-09-19T04:17:46.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561ac3614ab0 con 0x561ac380b7d0 2024-09-19T04:17:46.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3855760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3855760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39286/0 (socket says 172.21.15.76:39286) 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 -- 172.21.15.76:0/874133184 learned_addr learned my addr 172.21.15.76:0/874133184 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac380bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f79019640 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855ca0 0x561ac380b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 -- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 msgr2=0x561ac380bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac380bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 -- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855ca0 msgr2=0x561ac380b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855ca0 0x561ac380b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 -- 172.21.15.76:0/874133184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac3600590 con 0x561ac3855380 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac380bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f79019640 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855ca0 0x561ac380b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7981a640 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3855760 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8f640af040 tx=0x7f8f64004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf031e496f69c294 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f5b7fe640 1 -- 172.21.15.76:0/874133184 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640c8070 con 0x561ac3855380 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f5b7fe640 1 -- 172.21.15.76:0/874133184 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f640a5b50 con 0x561ac3855380 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f5b7fe640 1 -- 172.21.15.76:0/874133184 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640b9450 con 0x561ac3855380 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/874133184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 msgr2=0x561ac3855760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3855760 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f8f640af040 tx=0x7f8f64004ac0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/874133184 shutdown_connections 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac380bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855ca0 0x561ac380b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/874133184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3855760 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/874133184 >> 172.21.15.76:0/874133184 conn(0x561ac3802bc0 msgr2=0x561ac3809b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/874133184 shutdown_connections 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.946+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/874133184 wait complete. 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- start start 2024-09-19T04:17:46.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 Processor -- start 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac3810860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855380 0x561ac383f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855ca0 0x561ac383fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ac3618400 con 0x561ac3855ca0 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ac37163a0 con 0x561ac3855380 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561ac3699260 con 0x561ac380b7d0 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 10 client.0 _ll_register_callbacks cb 0x561ac36bb020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855ca0 0x561ac383fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855ca0 0x561ac383fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39294/0 (socket says 172.21.15.76:39294) 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 -- 172.21.15.76:0/1249056042 learned_addr learned my addr 172.21.15.76:0/1249056042 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f79019640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855380 0x561ac383f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac3810860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 -- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 msgr2=0x561ac3810860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac3810860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 -- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855380 msgr2=0x561ac383f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855380 0x561ac383f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 -- 172.21.15.76:0/1249056042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac35fcdd0 con 0x561ac3855ca0 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f79019640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855380 0x561ac383f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855ca0 0x561ac383fb10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00df40 tx=0x7f8f6c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af074956d2a578e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 1 -- 172.21.15.76:0/1249056042 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c018070 con 0x561ac3855ca0 2024-09-19T04:17:46.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f640a5b50 con 0x561ac3855ca0 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 1 -- 172.21.15.76:0/1249056042 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f8f6c014690 con 0x561ac3855ca0 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 1 -- 172.21.15.76:0/1249056042 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8f6c023050 con 0x561ac3855ca0 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 10 client.6621 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x561ac35f9800 con 0x561ac3855ca0 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 1 -- 172.21.15.76:0/1249056042 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f640a5b50 con 0x561ac3855ca0 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 1 -- 172.21.15.76:0/1249056042 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f8f6c026020 con 0x561ac3855ca0 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 10 client.6621 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 10 client.6621 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 20 client.6621 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac385c0e0 0x561ac385c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 4 client.6621 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 --> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ac35f3e20 con 0x561ac385c0e0 2024-09-19T04:17:46.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac385c0e0 0x561ac385c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 10 client.6621 ms_handle_connect on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7981a640 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac385c0e0 0x561ac385c4f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c88a5340c0997056 server_cookie=40c4c503e9b3068b in_seq=0 out_seq=0 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 1 -- 172.21.15.76:0/1249056042 <== mds.1 v2:172.21.15.76:6834/2740411101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ac35f3e20 con 0x561ac385c0e0 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 10 client.6621 handle_command_reply: tid=0 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 client.6621 shutdown 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855ca0 msgr2=0x561ac383fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855ca0 0x561ac383fb10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00df40 tx=0x7f8f6c00f670 comp rx=0 tx=0).stop 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 shutdown_connections 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac385c0e0 0x561ac385c4f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855ca0 0x561ac383fb10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac3855380 0x561ac383f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/1249056042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac380b7d0 0x561ac3810860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f58ff9640 0 client.6621 ms_handle_reset on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:46.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.950+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 >> 172.21.15.76:0/1249056042 conn(0x561ac3802bc0 msgr2=0x561ac380a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.954+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 shutdown_connections 2024-09-19T04:17:46.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.954+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/1249056042 wait complete. 2024-09-19T04:17:46.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.954+0000 7f8f7eb3a000 20 client.6621 trim_cache size 0 max 16384 2024-09-19T04:17:46.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 Processor -- start 2024-09-19T04:17:46.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 -- start start 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3802bc0 0x561ac386c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac386cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3858090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8f742012c0 con 0x561ac3855380 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8f742019f0 con 0x561ac380b7d0 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8f74152240 con 0x561ac3802bc0 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3858090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3858090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39310/0 (socket says 172.21.15.76:39310) 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 -- 172.21.15.76:0/3267248250 learned_addr learned my addr 172.21.15.76:0/3267248250 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f79019640 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac386cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7981a640 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3802bc0 0x561ac386c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 -- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3802bc0 msgr2=0x561ac386c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3802bc0 0x561ac386c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 -- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 msgr2=0x561ac386cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac386cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f7a01b640 1 -- 172.21.15.76:0/3267248250 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac3617480 con 0x561ac3855380 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.966+0000 7f8f79019640 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac386cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3858090 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c014840 tx=0x7f8f6c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49dc49f113b8d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f5affd640 1 -- 172.21.15.76:0/3267248250 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c02c070 con 0x561ac3855380 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f5affd640 1 -- 172.21.15.76:0/3267248250 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c000e50 con 0x561ac3855380 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f5affd640 1 -- 172.21.15.76:0/3267248250 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c018070 con 0x561ac3855380 2024-09-19T04:17:46.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/3267248250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 msgr2=0x561ac3858090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3858090 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c014840 tx=0x7f8f6c014000 comp rx=0 tx=0).stop 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/3267248250 shutdown_connections 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3855380 0x561ac3858090 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac386cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/3267248250 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3802bc0 0x561ac386c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/3267248250 >> 172.21.15.76:0/3267248250 conn(0x561ac3854e30 msgr2=0x561ac383ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/3267248250 shutdown_connections 2024-09-19T04:17:46.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/3267248250 wait complete. 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- start start 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 Processor -- start 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3802bc0 0x561ac385a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac385ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3855380 0x561ac38c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ac35fff30 con 0x561ac3802bc0 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ac36187d0 con 0x561ac380b7d0 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561ac3659060 con 0x561ac3855380 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 10 client.0 _ll_register_callbacks cb 0x7f8f6c00d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3802bc0 0x561ac385a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3802bc0 0x561ac385a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39324/0 (socket says 172.21.15.76:39324) 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 -- 172.21.15.76:0/688805234 learned_addr learned my addr 172.21.15.76:0/688805234 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 -- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3855380 msgr2=0x561ac38c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7a01b640 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3855380 0x561ac38c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f79019640 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac385ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3855380 0x561ac38c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 -- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 msgr2=0x561ac385ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac385ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 -- 172.21.15.76:0/688805234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac35ed650 con 0x561ac3802bc0 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7981a640 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3802bc0 0x561ac385a830 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8f6401cbe0 tx=0x7f8f64004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30843cc331520e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f58ff9640 1 -- 172.21.15.76:0/688805234 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f640d2020 con 0x561ac3802bc0 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f58ff9640 1 -- 172.21.15.76:0/688805234 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f8f640b9450 con 0x561ac3802bc0 2024-09-19T04:17:46.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f6c000e50 con 0x561ac3802bc0 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f58ff9640 1 -- 172.21.15.76:0/688805234 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f8f640dc050 con 0x561ac3802bc0 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 10 client.6627 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x561ac35f13c0 con 0x561ac3802bc0 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f58ff9640 1 -- 172.21.15.76:0/688805234 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c000e50 con 0x561ac3802bc0 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f58ff9640 1 -- 172.21.15.76:0/688805234 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f8f640df020 con 0x561ac3802bc0 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.970+0000 7f8f7eb3a000 10 client.6627 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 10 client.6627 resolve_mds: resolved 4:1 to role '4:1' aka daemon mds.c 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 20 client.6627 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac3855ca0 0x561ac38ca570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 4 client.6627 mds_command: new command op to 6156 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 --> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ac3667480 con 0x561ac3855ca0 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f79019640 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac3855ca0 0x561ac38ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:46.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f58ff9640 10 client.6627 ms_handle_connect on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f79019640 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac3855ca0 0x561ac38ca570 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0016989196d64ec server_cookie=b309b8303a12d45c in_seq=0 out_seq=0 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f58ff9640 1 -- 172.21.15.76:0/688805234 <== mds.1 v2:172.21.15.76:6834/2740411101 1 ==== command_reply(tid 0: 0 ) ==== 8+0+282 (crc 0 0 0) 0x561ac3667480 con 0x561ac3855ca0 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f58ff9640 10 client.6627 handle_command_reply: tid=0 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 client.6627 shutdown 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3802bc0 msgr2=0x561ac385a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3802bc0 0x561ac385a830 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8f6401cbe0 tx=0x7f8f64004a50 comp rx=0 tx=0).stop 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 shutdown_connections 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:6834/2740411101,v1:172.21.15.76:6835/2740411101] conn(0x561ac3855ca0 0x561ac38ca570 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ac3855380 0x561ac38c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ac380b7d0 0x561ac385ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 --2- 172.21.15.76:0/688805234 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ac3802bc0 0x561ac385a830 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 >> 172.21.15.76:0/688805234 conn(0x561ac3854e30 msgr2=0x561ac386c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f58ff9640 0 client.6627 ms_handle_reset on v2:172.21.15.76:6834/2740411101 2024-09-19T04:17:46.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 shutdown_connections 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 1 -- 172.21.15.76:0/688805234 wait complete. 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7eb3a000 20 client.6627 trim_cache size 0 max 16384 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 10052381, 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511627781, 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3/d4/d5/d6" 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:46.984 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:46.985 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-09-19T04:17:46.985 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 1703659257, 2024-09-19T04:17:46.985 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511627780, 2024-09-19T04:17:46.985 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3/d4/d5" 2024-09-19T04:17:46.985 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:46.985 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:17:46.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8f600678c0 msgr2=0x7f8f60069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8f600678c0 0x7f8f60069d80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8f640ae000 tx=0x7f8f640a7070 comp rx=0 tx=0).stop 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 msgr2=0x7f8f74150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f74150fc0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000d800 tx=0x7f8f7000dcc0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 shutdown_connections 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8f600678c0 0x7f8f60069d80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8f7415c2a0 0x7f8f7414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8f7415bec0 0x7f8f74150fc0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 --2- 172.21.15.76:0/1989456739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8f740af2a0 0x7f8f74149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 >> 172.21.15.76:0/1989456739 conn(0x7f8f740b1f90 msgr2=0x7f8f740aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 shutdown_connections 2024-09-19T04:17:46.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:46.974+0000 7f8f7ba7e640 1 -- 172.21.15.76:0/1989456739 wait complete. 2024-09-19T04:17:46.995 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:2 damage ls 2024-09-19T04:17:47.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.102+0000 7f3090037640 1 Processor -- start 2024-09-19T04:17:47.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.102+0000 7f3090037640 1 -- start start 2024-09-19T04:17:47.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f3090037640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30881520a0 0x7f308806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f3090037640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f3090037640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3088070250 con 0x7f308806fe50 2024-09-19T04:17:47.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f3090037640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3088152480 con 0x7f30881520a0 2024-09-19T04:17:47.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f3090037640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f30880aede0 con 0x7f308815f3a0 2024-09-19T04:17:47.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308ddd3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30881520a0 0x7f308806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308e5d4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308d5d2640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f308806fe50 0x7f308815ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:48544/0 (socket says 172.21.15.76:48544) 2024-09-19T04:17:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308d5d2640 1 -- 172.21.15.76:0/838687726 learned_addr learned my addr 172.21.15.76:0/838687726 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308e5d4640 1 -- 172.21.15.76:0/838687726 >> v2:172.21.15.116:3300/0 conn(0x7f30881520a0 msgr2=0x7f308806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308e5d4640 1 --2- 172.21.15.76:0/838687726 >> v2:172.21.15.116:3300/0 conn(0x7f30881520a0 0x7f308806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308e5d4640 1 -- 172.21.15.76:0/838687726 >> v1:172.21.15.116:6789/0 conn(0x7f308806fe50 legacy=0x7f308815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308e5d4640 1 -- 172.21.15.76:0/838687726 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f308815fa10 con 0x7f308815f3a0 2024-09-19T04:17:47.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308e5d4640 1 --2- 172.21.15.76:0/838687726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880adc80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3080009d90 tx=0x7f308001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadd04a97d48d1f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308cdd1640 1 -- 172.21.15.76:0/838687726 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30800b2070 con 0x7f308815f3a0 2024-09-19T04:17:47.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308cdd1640 1 -- 172.21.15.76:0/838687726 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30800a5920 con 0x7f308815f3a0 2024-09-19T04:17:47.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.106+0000 7f308cdd1640 1 -- 172.21.15.76:0/838687726 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30800a5cf0 con 0x7f308815f3a0 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- 172.21.15.76:0/838687726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 msgr2=0x7f30880adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 --2- 172.21.15.76:0/838687726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880adc80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3080009d90 tx=0x7f308001c970 comp rx=0 tx=0).stop 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- 172.21.15.76:0/838687726 shutdown_connections 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 --2- 172.21.15.76:0/838687726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880adc80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 --2- 172.21.15.76:0/838687726 >> v2:172.21.15.116:3300/0 conn(0x7f30881520a0 0x7f308806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- 172.21.15.76:0/838687726 >> 172.21.15.76:0/838687726 conn(0x7f30880b1f90 msgr2=0x7f30880b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- 172.21.15.76:0/838687726 shutdown_connections 2024-09-19T04:17:47.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- 172.21.15.76:0/838687726 wait complete. 2024-09-19T04:17:47.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 Processor -- start 2024-09-19T04:17:47.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- start start 2024-09-19T04:17:47.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f308806fe50 0x7f30880cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30881520a0 0x7f308814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3088073d30 con 0x7f308806fe50 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3088073a50 con 0x7f30881520a0 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3088167f10 con 0x7f308815f3a0 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f308806fe50 0x7f30880cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f308806fe50 0x7f30880cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39328/0 (socket says 172.21.15.76:39328) 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 -- 172.21.15.76:0/3441086059 learned_addr learned my addr 172.21.15.76:0/3441086059 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308e5d4640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308d5d2640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30881520a0 0x7f308814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 -- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 msgr2=0x7f30880ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880ce490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 -- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30881520a0 msgr2=0x7f308814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30881520a0 0x7f308814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 -- 172.21.15.76:0/3441086059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30880cfd80 con 0x7f308806fe50 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f308ddd3640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f308806fe50 0x7f30880cd6c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f308400e9b0 tx=0x7f308400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f307e7fc640 1 -- 172.21.15.76:0/3441086059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084017070 con 0x7f308806fe50 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f307e7fc640 1 -- 172.21.15.76:0/3441086059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f308400c9a0 con 0x7f308806fe50 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30880cece0 con 0x7f308806fe50 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f307e7fc640 1 -- 172.21.15.76:0/3441086059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308400cd70 con 0x7f308806fe50 2024-09-19T04:17:47.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.110+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30880cf2a0 con 0x7f308806fe50 2024-09-19T04:17:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f307e7fc640 1 -- 172.21.15.76:0/3441086059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f308401a070 con 0x7f308806fe50 2024-09-19T04:17:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f307e7fc640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678f0 0x7f306c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f307e7fc640 1 -- 172.21.15.76:0/3441086059 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f3084086240 con 0x7f308806fe50 2024-09-19T04:17:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308d5d2640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678f0 0x7f306c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308d5d2640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678f0 0x7f306c069db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30880b2f60 tx=0x7f3078009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 Processor -- start 2024-09-19T04:17:47.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 -- start start 2024-09-19T04:17:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b134760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b0ea290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0eabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a94afd1c40 con 0x55a94b0ea7d0 2024-09-19T04:17:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a94af38d50 con 0x55a94b134ca0 2024-09-19T04:17:47.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a94aef3ab0 con 0x55a94b134380 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0eabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0eabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39342/0 (socket says 172.21.15.76:39342) 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 -- 172.21.15.76:0/1582923257 learned_addr learned my addr 172.21.15.76:0/1582923257 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308ddd3640 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b134760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308d5d2640 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b0ea290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 -- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 msgr2=0x55a94b134760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b134760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 -- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 msgr2=0x55a94b0ea290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b0ea290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 -- 172.21.15.76:0/1582923257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94aedf590 con 0x55a94b0ea7d0 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308d5d2640 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b0ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.114+0000 7f308e5d4640 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0eabb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f30800b0040 tx=0x7f3080004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf1907890c3c76c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f306b7fe640 1 -- 172.21.15.76:0/1582923257 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30800be070 con 0x55a94b0ea7d0 2024-09-19T04:17:47.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f306b7fe640 1 -- 172.21.15.76:0/1582923257 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30800b3040 con 0x55a94b0ea7d0 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f306b7fe640 1 -- 172.21.15.76:0/1582923257 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30800b2040 con 0x55a94b0ea7d0 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/1582923257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 msgr2=0x55a94b0eabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0eabb0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f30800b0040 tx=0x7f3080004ac0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/1582923257 shutdown_connections 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0eabb0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b0ea290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- 172.21.15.76:0/1582923257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b134760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/1582923257 >> 172.21.15.76:0/1582923257 conn(0x55a94b0e1bc0 msgr2=0x55a94b0e8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/1582923257 shutdown_connections 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/1582923257 wait complete. 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- start start 2024-09-19T04:17:47.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 Processor -- start 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0ef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134380 0x55a94b11e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134ca0 0x55a94b11eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a94aef7400 con 0x55a94b0ea7d0 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a94aff53a0 con 0x55a94b134380 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a94af78260 con 0x55a94b134ca0 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 10 client.0 _ll_register_callbacks cb 0x55a94af9a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0ef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0ef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39356/0 (socket says 172.21.15.76:39356) 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 -- 172.21.15.76:0/604337453 learned_addr learned my addr 172.21.15.76:0/604337453 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308e5d4640 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134ca0 0x55a94b11eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308d5d2640 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134380 0x55a94b11e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 -- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134ca0 msgr2=0x55a94b11eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134ca0 0x55a94b11eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 -- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134380 msgr2=0x55a94b11e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134380 0x55a94b11e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 -- 172.21.15.76:0/604337453 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94aedbdd0 con 0x55a94b0ea7d0 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308ddd3640 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0ef860 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f30840564a0 tx=0x7f3084055f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3d028517eef2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f3068ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f3068ff9640 1 -- 172.21.15.76:0/604337453 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308406e020 con 0x55a94b0ea7d0 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f3068ff9640 1 -- 172.21.15.76:0/604337453 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f308406e3c0 con 0x55a94b0ea7d0 2024-09-19T04:17:47.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30800b3040 con 0x55a94b0ea7d0 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f3068ff9640 1 -- 172.21.15.76:0/604337453 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3084021040 con 0x55a94b0ea7d0 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 10 client.6636 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55a94aed8800 con 0x55a94b0ea7d0 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f3068ff9640 1 -- 172.21.15.76:0/604337453 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30800b3040 con 0x55a94b0ea7d0 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f3068ff9640 1 -- 172.21.15.76:0/604337453 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f30840840d0 con 0x55a94b0ea7d0 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 10 client.6636 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 10 client.6636 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 20 client.6636 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b13b1c0 0x55a94b13b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 4 client.6636 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 --> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a94aed2e20 con 0x55a94b13b1c0 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.118+0000 7f308d5d2640 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b13b1c0 0x55a94b13b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f3068ff9640 10 client.6636 ms_handle_connect on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f308d5d2640 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b13b1c0 0x55a94b13b5a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=362df4c9ad8cdb44 server_cookie=d50a60c06485e429 in_seq=0 out_seq=0 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f3068ff9640 1 -- 172.21.15.76:0/604337453 <== mds.2 v2:172.21.15.116:6832/1054899754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a94aed2e20 con 0x55a94b13b1c0 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f3068ff9640 10 client.6636 handle_command_reply: tid=0 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 client.6636 shutdown 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 msgr2=0x55a94b0ef860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0ef860 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f30840564a0 tx=0x7f3084055f30 comp rx=0 tx=0).stop 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 shutdown_connections 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b13b1c0 0x55a94b13b5a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134ca0 0x55a94b11eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134380 0x55a94b11e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f3068ff9640 0 client.6636 ms_handle_reset on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 --2- 172.21.15.76:0/604337453 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b0ea7d0 0x55a94b0ef860 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 >> 172.21.15.76:0/604337453 conn(0x55a94b0e1bc0 msgr2=0x55a94b0e9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 shutdown_connections 2024-09-19T04:17:47.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 1 -- 172.21.15.76:0/604337453 wait complete. 2024-09-19T04:17:47.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.122+0000 7f30930f3000 20 client.6636 trim_cache size 0 max 16384 2024-09-19T04:17:47.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 Processor -- start 2024-09-19T04:17:47.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- start start 2024-09-19T04:17:47.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b0ea7d0 0x55a94b14a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b14acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134ca0 0x55a94b14b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3088201360 con 0x55a94b134ca0 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3088201a90 con 0x55a94b0ea7d0 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f308809b040 con 0x55a94b134380 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134ca0 0x55a94b14b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134ca0 0x55a94b14b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39362/0 (socket says 172.21.15.76:39362) 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 -- 172.21.15.76:0/189285330 learned_addr learned my addr 172.21.15.76:0/189285330 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308d5d2640 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b14acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 -- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 msgr2=0x55a94b14acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b14acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 -- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b0ea7d0 msgr2=0x55a94b14a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308ddd3640 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b0ea7d0 0x55a94b14a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b0ea7d0 0x55a94b14a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 -- 172.21.15.76:0/189285330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a94aef6480 con 0x55a94b134ca0 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308d5d2640 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308e5d4640 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134ca0 0x55a94b14b230 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f308001ce50 tx=0x7f30800a7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4ecefb38b4f20d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f306affd640 1 -- 172.21.15.76:0/189285330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30800bc070 con 0x55a94b134ca0 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f306affd640 1 -- 172.21.15.76:0/189285330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30800c8070 con 0x55a94b134ca0 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f306affd640 1 -- 172.21.15.76:0/189285330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30800b2040 con 0x55a94b134ca0 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- 172.21.15.76:0/189285330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134ca0 msgr2=0x55a94b14b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134ca0 0x55a94b14b230 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f308001ce50 tx=0x7f30800a7000 comp rx=0 tx=0).stop 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- 172.21.15.76:0/189285330 shutdown_connections 2024-09-19T04:17:47.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134ca0 0x55a94b14b230 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b134380 0x55a94b14acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- 172.21.15.76:0/189285330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b0ea7d0 0x55a94b14a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- 172.21.15.76:0/189285330 >> 172.21.15.76:0/189285330 conn(0x55a94b0e1bc0 msgr2=0x55a94b11d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- 172.21.15.76:0/189285330 shutdown_connections 2024-09-19T04:17:47.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- 172.21.15.76:0/189285330 wait complete. 2024-09-19T04:17:47.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- start start 2024-09-19T04:17:47.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 Processor -- start 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b0ea7d0 0x55a94b1ac390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134380 0x55a94b1393d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b139910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a94aedef30 con 0x55a94b134380 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a94aef77d0 con 0x55a94b134ca0 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a94af38060 con 0x55a94b0ea7d0 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f30930f3000 10 client.0 _ll_register_callbacks cb 0x7f30840090c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308d5d2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134380 0x55a94b1393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308d5d2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134380 0x55a94b1393d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39372/0 (socket says 172.21.15.76:39372) 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.138+0000 7f308d5d2640 1 -- 172.21.15.76:0/1748016393 learned_addr learned my addr 172.21.15.76:0/1748016393 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:47.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308d5d2640 1 -- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b0ea7d0 msgr2=0x55a94b1ac390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308ddd3640 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b0ea7d0 0x55a94b1ac390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308e5d4640 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308d5d2640 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b0ea7d0 0x55a94b1ac390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308d5d2640 1 -- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 msgr2=0x55a94b139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308d5d2640 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b139910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308d5d2640 1 -- 172.21.15.76:0/1748016393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a94aecc650 con 0x55a94b134380 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308d5d2640 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134380 0x55a94b1393d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3078005f60 tx=0x7f3078000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32fb10e536390a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 1 -- 172.21.15.76:0/1748016393 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30780248b0 con 0x55a94b134380 2024-09-19T04:17:47.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 1 -- 172.21.15.76:0/1748016393 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f307802dcf0 con 0x55a94b134380 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f30800c8070 con 0x55a94b134380 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 1 -- 172.21.15.76:0/1748016393 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f3078024a50 con 0x55a94b134380 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 10 client.6642 fetch_fsmap learned FSMap version 58 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) -- 0x55a94aed03c0 con 0x55a94b134380 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 1 -- 172.21.15.76:0/1748016393 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30800c8070 con 0x55a94b134380 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 1 -- 172.21.15.76:0/1748016393 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 58) ==== 3558+0+0 (secure 0 0 0) 0x7f3078034050 con 0x55a94b134380 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 10 client.6642 fetch_fsmap finished waiting for FSMap version 58 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 10 client.6642 resolve_mds: resolved 4:2 to role '4:2' aka daemon mds.b 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 20 client.6642 populate_metadata read hostname 'smithi076' 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b139e50 0x55a94b13a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 4 client.6642 mds_command: new command op to 5272 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T04:17:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 --> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a94af46480 con 0x55a94b139e50 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308ddd3640 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b139e50 0x55a94b13a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f308ddd3640 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b139e50 0x55a94b13a260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd8ef09a1073bddb server_cookie=2cf8b683454a7c4b in_seq=0 out_seq=0 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 10 client.6642 ms_handle_connect on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 1 -- 172.21.15.76:0/1748016393 <== mds.2 v2:172.21.15.116:6832/1054899754 1 ==== command_reply(tid 0: 0 ) ==== 8+0+296 (crc 0 0 0) 0x55a94af46480 con 0x55a94b139e50 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 10 client.6642 handle_command_reply: tid=0 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 client.6642 shutdown 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134380 msgr2=0x55a94b1393d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134380 0x55a94b1393d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f3078005f60 tx=0x7f3078000bd0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 shutdown_connections 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:6832/1054899754,v1:172.21.15.116:6833/1054899754] conn(0x55a94b139e50 0x55a94b13a260 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a94b134ca0 0x55a94b139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3068ff9640 0 client.6642 ms_handle_reset on v2:172.21.15.116:6832/1054899754 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a94b134380 0x55a94b1393d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 --2- 172.21.15.76:0/1748016393 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a94b0ea7d0 0x55a94b1ac390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 >> 172.21.15.76:0/1748016393 conn(0x55a94b0e1bc0 msgr2=0x55a94b0ef800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 shutdown_connections 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 1 -- 172.21.15.76:0/1748016393 wait complete. 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f30930f3000 20 client.6642 trim_cache size 0 max 16384 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:47.154 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 2166872207, 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511627782, 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7" 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: "damage_type": "backtrace", 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 3262255239, 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: "ino": 1099511627783, 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3/d4/d5/d6/d7/d8" 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:47.155 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678f0 msgr2=0x7f306c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678f0 0x7f306c069db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30880b2f60 tx=0x7f3078009040 comp rx=0 tx=0).stop 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f308806fe50 msgr2=0x7f30880cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f308806fe50 0x7f30880cd6c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f308400e9b0 tx=0x7f308400ee70 comp rx=0 tx=0).stop 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 shutdown_connections 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678f0 0x7f306c069db0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815f3a0 0x7f30880ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30881520a0 0x7f308814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 --2- 172.21.15.76:0/3441086059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f308806fe50 0x7f30880cd6c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 >> 172.21.15.76:0/3441086059 conn(0x7f30880b1f90 msgr2=0x7f30881510f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 shutdown_connections 2024-09-19T04:17:47.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.142+0000 7f3090037640 1 -- 172.21.15.76:0/3441086059 wait complete. 2024-09-19T04:17:47.164 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:17:47.164 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:17:47.238 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:17:47.323 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:17:47.323 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:17:47.324 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:17:47.324 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:47.330 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:17:47.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa861e68640 1 Processor -- start 2024-09-19T04:17:47.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa861e68640 1 -- start start 2024-09-19T04:17:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa861e68640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa85c1520a0 0x7fa85c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa861e68640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c06fe50 0x7fa85c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa861e68640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa85c070250 con 0x7fa85c15f1d0 2024-09-19T04:17:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa861e68640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa85c152480 con 0x7fa85c1520a0 2024-09-19T04:17:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa861e68640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c0aede0 con 0x7fa85c06fe50 2024-09-19T04:17:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa85affd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c06fe50 0x7fa85c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa85b7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa85c1520a0 0x7fa85c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa85bfff640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa85c15f1d0 0x7fa85c071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:48552/0 (socket says 172.21.15.76:48552) 2024-09-19T04:17:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.490+0000 7fa85bfff640 1 -- 172.21.15.76:0/760444155 learned_addr learned my addr 172.21.15.76:0/760444155 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85affd640 1 -- 172.21.15.76:0/760444155 >> v2:172.21.15.116:3300/0 conn(0x7fa85c1520a0 msgr2=0x7fa85c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85affd640 1 --2- 172.21.15.76:0/760444155 >> v2:172.21.15.116:3300/0 conn(0x7fa85c1520a0 0x7fa85c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85a7fc640 1 -- 172.21.15.76:0/760444155 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 153039465 0 0) 0x7fa85c070250 con 0x7fa85c15f1d0 2024-09-19T04:17:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85affd640 1 -- 172.21.15.76:0/760444155 >> v1:172.21.15.116:6789/0 conn(0x7fa85c15f1d0 legacy=0x7fa85c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85affd640 1 -- 172.21.15.76:0/760444155 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c15fa10 con 0x7fa85c06fe50 2024-09-19T04:17:47.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85affd640 1 --2- 172.21.15.76:0/760444155 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c06fe50 0x7fa85c15ec90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8440097c0 tx=0x7fa84401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87b53badf7efa10b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85a7fc640 1 -- 172.21.15.76:0/760444155 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440b1070 con 0x7fa85c06fe50 2024-09-19T04:17:47.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85a7fc640 1 -- 172.21.15.76:0/760444155 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8440ac600 con 0x7fa85c06fe50 2024-09-19T04:17:47.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.494+0000 7fa85a7fc640 1 -- 172.21.15.76:0/760444155 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8440ac9d0 con 0x7fa85c06fe50 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/760444155 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c06fe50 msgr2=0x7fa85c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 --2- 172.21.15.76:0/760444155 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c06fe50 0x7fa85c15ec90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa8440097c0 tx=0x7fa84401c480 comp rx=0 tx=0).stop 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/760444155 shutdown_connections 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 --2- 172.21.15.76:0/760444155 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c06fe50 0x7fa85c15ec90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 --2- 172.21.15.76:0/760444155 >> v2:172.21.15.116:3300/0 conn(0x7fa85c1520a0 0x7fa85c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/760444155 >> 172.21.15.76:0/760444155 conn(0x7fa85c0b1f90 msgr2=0x7fa85c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/760444155 shutdown_connections 2024-09-19T04:17:47.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/760444155 wait complete. 2024-09-19T04:17:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 Processor -- start 2024-09-19T04:17:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- start start 2024-09-19T04:17:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa85c06fe50 0x7fa85c0b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa85c1520a0 0x7fa85c09dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c15f1d0 0x7fa85c15cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c073d30 con 0x7fa85c06fe50 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa85c073a50 con 0x7fa85c1520a0 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa85c15f640 con 0x7fa85c15f1d0 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa85c06fe50 0x7fa85c0b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa85c06fe50 0x7fa85c0b0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39386/0 (socket says 172.21.15.76:39386) 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 -- 172.21.15.76:0/4186856291 learned_addr learned my addr 172.21.15.76:0/4186856291 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85bfff640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c15f1d0 0x7fa85c15cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85affd640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa85c1520a0 0x7fa85c09dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 -- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c15f1d0 msgr2=0x7fa85c15cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c15f1d0 0x7fa85c15cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 -- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa85c1520a0 msgr2=0x7fa85c09dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa85c1520a0 0x7fa85c09dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 -- 172.21.15.76:0/4186856291 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa85c150530 con 0x7fa85c06fe50 2024-09-19T04:17:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85bfff640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c15f1d0 0x7fa85c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa85b7fe640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa85c06fe50 0x7fa85c0b0f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e9b0 tx=0x7fa84c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa853fff640 1 -- 172.21.15.76:0/4186856291 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c017070 con 0x7fa85c06fe50 2024-09-19T04:17:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa853fff640 1 -- 172.21.15.76:0/4186856291 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84c00c9a0 con 0x7fa85c06fe50 2024-09-19T04:17:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa85c150810 con 0x7fa85c06fe50 2024-09-19T04:17:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa853fff640 1 -- 172.21.15.76:0/4186856291 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c00cd70 con 0x7fa85c06fe50 2024-09-19T04:17:47.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa85c150dd0 con 0x7fa85c06fe50 2024-09-19T04:17:47.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.498+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa85c15ec90 con 0x7fa85c06fe50 2024-09-19T04:17:47.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.502+0000 7fa853fff640 1 -- 172.21.15.76:0/4186856291 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa84c01a070 con 0x7fa85c06fe50 2024-09-19T04:17:47.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.502+0000 7fa853fff640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa83c0678c0 0x7fa83c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:47.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.502+0000 7fa85affd640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa83c0678c0 0x7fa83c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:47.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.502+0000 7fa853fff640 1 -- 172.21.15.76:0/4186856291 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fa84c086340 con 0x7fa85c06fe50 2024-09-19T04:17:47.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.502+0000 7fa85affd640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa83c0678c0 0x7fa83c069d80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa8440ae000 tx=0x7fa844008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:47.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.506+0000 7fa853fff640 1 -- 172.21.15.76:0/4186856291 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa84c056c60 con 0x7fa85c06fe50 2024-09-19T04:17:47.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa85c0b5420 con 0x7fa85c06fe50 2024-09-19T04:17:47.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa853fff640 1 -- 172.21.15.76:0/4186856291 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v69) ==== 80+0+0 (secure 0 0 0) 0x7fa84c056560 con 0x7fa85c06fe50 2024-09-19T04:17:47.670 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa83c0678c0 msgr2=0x7fa83c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa83c0678c0 0x7fa83c069d80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa8440ae000 tx=0x7fa844008040 comp rx=0 tx=0).stop 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa85c06fe50 msgr2=0x7fa85c0b0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa85c06fe50 0x7fa85c0b0f70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00e9b0 tx=0x7fa84c00ee70 comp rx=0 tx=0).stop 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 shutdown_connections 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa83c0678c0 0x7fa83c069d80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa85c15f1d0 0x7fa85c15cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa85c1520a0 0x7fa85c09dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 --2- 172.21.15.76:0/4186856291 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa85c06fe50 0x7fa85c0b0f70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:47.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 >> 172.21.15.76:0/4186856291 conn(0x7fa85c0b1f90 msgr2=0x7fa85c06ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:47.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 shutdown_connections 2024-09-19T04:17:47.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:47.666+0000 7fa861e68640 1 -- 172.21.15.76:0/4186856291 wait complete. 2024-09-19T04:17:47.688 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:17:47.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:47.688 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T04:17:47.697 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[42588]: fuse finished with error 0 and tester_r 0 2024-09-19T04:17:47.698 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:17:47.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:47.699 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:17:47.863 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:17:47.863 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:17:47.863 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:47.870 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:17:47.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:17:47.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:47.870 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T04:17:47.918 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T04:17:47.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:17:47.920 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:17:47.920 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:17:47.920 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:47.926 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:17:47.926 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:17:47.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:47.926 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T04:17:47.973 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T04:17:47.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:17:47.975 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:17:47.975 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:17:47.975 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:47.981 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:17:47.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:17:47.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:17:47.981 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T04:17:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T04:17:48.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:17:48.028 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:17:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa25c10e640 1 Processor -- start 2024-09-19T04:17:48.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa25c10e640 1 -- start start 2024-09-19T04:17:48.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa25c10e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa2540b3e00 0x7fa2540b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa25c10e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa25406d950 0x7fa2540cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa25c10e640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2540b4720 con 0x7fa25406dde0 2024-09-19T04:17:48.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa25c10e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa25406e3a0 con 0x7fa25406d950 2024-09-19T04:17:48.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa25c10e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa25406e680 con 0x7fa2540b3e00 2024-09-19T04:17:48.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa25406d950 0x7fa2540cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa25406d950 0x7fa2540cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39390/0 (socket says 172.21.15.76:39390) 2024-09-19T04:17:48.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 -- 172.21.15.76:0/2214354854 learned_addr learned my addr 172.21.15.76:0/2214354854 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:48.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa259eaa640 1 --2- 172.21.15.76:0/2214354854 >> v2:172.21.15.116:3300/0 conn(0x7fa2540b3e00 0x7fa2540b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 -- 172.21.15.76:0/2214354854 >> v2:172.21.15.116:3300/0 conn(0x7fa2540b3e00 msgr2=0x7fa2540b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:48.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 --2- 172.21.15.76:0/2214354854 >> v2:172.21.15.116:3300/0 conn(0x7fa2540b3e00 0x7fa2540b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 -- 172.21.15.76:0/2214354854 >> v1:172.21.15.116:6789/0 conn(0x7fa25406dde0 legacy=0x7fa2540cba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:48.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 -- 172.21.15.76:0/2214354854 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25415fe00 con 0x7fa25406d950 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.154+0000 7fa2596a9640 1 --2- 172.21.15.76:0/2214354854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa2540cb4f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa254068500 tx=0x7fa25001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa02c1c00df3fd14 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa258ea8640 1 -- 172.21.15.76:0/2214354854 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2500b0070 con 0x7fa25406d950 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa258ea8640 1 -- 172.21.15.76:0/2214354854 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2500ab600 con 0x7fa25406d950 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa258ea8640 1 -- 172.21.15.76:0/2214354854 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2500ab9d0 con 0x7fa25406d950 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- 172.21.15.76:0/2214354854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 msgr2=0x7fa2540cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 --2- 172.21.15.76:0/2214354854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa2540cb4f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fa254068500 tx=0x7fa25001c4e0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- 172.21.15.76:0/2214354854 shutdown_connections 2024-09-19T04:17:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 --2- 172.21.15.76:0/2214354854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa2540cb4f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 --2- 172.21.15.76:0/2214354854 >> v2:172.21.15.116:3300/0 conn(0x7fa2540b3e00 0x7fa2540b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- 172.21.15.76:0/2214354854 >> 172.21.15.76:0/2214354854 conn(0x7fa25406bcc0 msgr2=0x7fa25406c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:48.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- 172.21.15.76:0/2214354854 shutdown_connections 2024-09-19T04:17:48.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- 172.21.15.76:0/2214354854 wait complete. 2024-09-19T04:17:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 Processor -- start 2024-09-19T04:17:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- start start 2024-09-19T04:17:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa25413c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25406dde0 0x7fa25413b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2540b3e00 0x7fa254160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa2540cc2c0 con 0x7fa25406d950 2024-09-19T04:17:48.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa2540cc140 con 0x7fa25406dde0 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa25416b8f0 con 0x7fa2540b3e00 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa25413c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa25413c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39394/0 (socket says 172.21.15.76:39394) 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 -- 172.21.15.76:0/1334808212 learned_addr learned my addr 172.21.15.76:0/1334808212 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa2596a9640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25406dde0 0x7fa25413b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25a6ab640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2540b3e00 0x7fa254160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 -- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2540b3e00 msgr2=0x7fa254160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2540b3e00 0x7fa254160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 -- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25406dde0 msgr2=0x7fa25413b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25406dde0 0x7fa25413b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 -- 172.21.15.76:0/1334808212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25413d0d0 con 0x7fa25406d950 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25a6ab640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2540b3e00 0x7fa254160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa259eaa640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa25413c7d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa24800d830 tx=0x7fa24800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa2467fc640 1 -- 172.21.15.76:0/1334808212 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800b6c0 con 0x7fa25406d950 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa2467fc640 1 -- 172.21.15.76:0/1334808212 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa248019070 con 0x7fa25406d950 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa25413d3b0 con 0x7fa25406d950 2024-09-19T04:17:48.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa2467fc640 1 -- 172.21.15.76:0/1334808212 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24800ba60 con 0x7fa25406d950 2024-09-19T04:17:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.158+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2540ccbe0 con 0x7fa25406d950 2024-09-19T04:17:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.162+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2540b41e0 con 0x7fa25406d950 2024-09-19T04:17:48.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.162+0000 7fa2467fc640 1 -- 172.21.15.76:0/1334808212 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa24801e460 con 0x7fa25406d950 2024-09-19T04:17:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.162+0000 7fa2467fc640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2380678c0 0x7fa238069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.162+0000 7fa2596a9640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2380678c0 0x7fa238069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.162+0000 7fa2467fc640 1 -- 172.21.15.76:0/1334808212 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fa2480865b0 con 0x7fa25406d950 2024-09-19T04:17:48.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.162+0000 7fa2596a9640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2380678c0 0x7fa238069d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa2500ad000 tx=0x7fa250008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.166+0000 7fa2467fc640 1 -- 172.21.15.76:0/1334808212 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa248056f60 con 0x7fa25406d950 2024-09-19T04:17:48.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa2540b1ac0 con 0x7fa25406d950 2024-09-19T04:17:48.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa2467fc640 1 -- 172.21.15.76:0/1334808212 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5277 (secure 0 0 0) 0x7fa248056860 con 0x7fa25406d950 2024-09-19T04:17:48.354 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:48.354 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"btime":"2024-09-19T04:17:21:921326+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:17:21.062516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6165,"mds_1":6156,"mds_2":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"b","rank":2,"incarnation":51,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6156":{"gid":6156,"name":"c","rank":1,"incarnation":49,"state":"up:active","state_seq":14,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[5272,6156,6165]},"id":4}]} 2024-09-19T04:17:48.354 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 58 2024-09-19T04:17:48.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2380678c0 msgr2=0x7fa238069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2380678c0 0x7fa238069d80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa2500ad000 tx=0x7fa250008040 comp rx=0 tx=0).stop 2024-09-19T04:17:48.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 msgr2=0x7fa25413c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa25413c7d0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fa24800d830 tx=0x7fa24800dcf0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 shutdown_connections 2024-09-19T04:17:48.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa2380678c0 0x7fa238069d80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa2540b3e00 0x7fa254160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25406dde0 0x7fa25413b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 --2- 172.21.15.76:0/1334808212 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa25406d950 0x7fa25413c7d0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 >> 172.21.15.76:0/1334808212 conn(0x7fa25406bcc0 msgr2=0x7fa25406c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:48.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 shutdown_connections 2024-09-19T04:17:48.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.350+0000 7fa25c10e640 1 -- 172.21.15.76:0/1334808212 wait complete. 2024-09-19T04:17:48.372 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:17:48.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33aa2de640 1 Processor -- start 2024-09-19T04:17:48.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33aa2de640 1 -- start start 2024-09-19T04:17:48.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33aa2de640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a4071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33aa2de640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33a4071c70 0x7f33a4141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33aa2de640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33a40720b0 con 0x7f33a4141890 2024-09-19T04:17:48.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33aa2de640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f33a4072660 con 0x7f33a4071c70 2024-09-19T04:17:48.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33aa2de640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f33a40d15e0 con 0x7f33a4140f70 2024-09-19T04:17:48.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33a4071c70 0x7f33a4141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a887b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f33a4141890 0x7f33a415fbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57920/0 (socket says 172.21.15.76:57920) 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a887b640 1 -- 172.21.15.76:0/3084656254 learned_addr learned my addr 172.21.15.76:0/3084656254 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33a4071c70 0x7f33a4141350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39404/0 (socket says 172.21.15.76:39404) 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a3fff640 1 --2- 172.21.15.76:0/3084656254 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a4071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a37fe640 1 -- 172.21.15.76:0/3084656254 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 msgr2=0x7f33a4071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a37fe640 1 --2- 172.21.15.76:0/3084656254 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a4071730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a37fe640 1 -- 172.21.15.76:0/3084656254 >> v1:172.21.15.76:6789/0 conn(0x7f33a4141890 legacy=0x7f33a415fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a37fe640 1 -- 172.21.15.76:0/3084656254 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a4072380 con 0x7f33a4071c70 2024-09-19T04:17:48.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.486+0000 7f33a2ffd640 1 -- 172.21.15.76:0/3084656254 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194731149 0 0) 0x7f33a40720b0 con 0x7f33a4141890 2024-09-19T04:17:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a37fe640 1 --2- 172.21.15.76:0/3084656254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a4141350 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f338c0097c0 tx=0x7f338c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a580196ee40d9c64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a2ffd640 1 -- 172.21.15.76:0/3084656254 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c0a7070 con 0x7f33a4071c70 2024-09-19T04:17:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a2ffd640 1 -- 172.21.15.76:0/3084656254 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f338c0b0070 con 0x7f33a4071c70 2024-09-19T04:17:48.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a2ffd640 1 -- 172.21.15.76:0/3084656254 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c0ab890 con 0x7f33a4071c70 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- 172.21.15.76:0/3084656254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 msgr2=0x7f33a4141350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 --2- 172.21.15.76:0/3084656254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a4141350 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f338c0097c0 tx=0x7f338c01c470 comp rx=0 tx=0).stop 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- 172.21.15.76:0/3084656254 shutdown_connections 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 --2- 172.21.15.76:0/3084656254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a4141350 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 --2- 172.21.15.76:0/3084656254 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a4071730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- 172.21.15.76:0/3084656254 >> 172.21.15.76:0/3084656254 conn(0x7f33a40b3e70 msgr2=0x7f33a40b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- 172.21.15.76:0/3084656254 shutdown_connections 2024-09-19T04:17:48.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- 172.21.15.76:0/3084656254 wait complete. 2024-09-19T04:17:48.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 Processor -- start 2024-09-19T04:17:48.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- start start 2024-09-19T04:17:48.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a41576d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a41f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a4141890 0x7f33a4145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40d2c00 con 0x7f33a4071c70 2024-09-19T04:17:48.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33a40d2a80 con 0x7f33a4141890 2024-09-19T04:17:48.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33aa2de640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f33a4168d80 con 0x7f33a4140f70 2024-09-19T04:17:48.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a41576d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a41576d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39418/0 (socket says 172.21.15.76:39418) 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a3fff640 1 -- 172.21.15.76:0/1598191891 learned_addr learned my addr 172.21.15.76:0/1598191891 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.490+0000 7f33a37fe640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a41f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a887b640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a4141890 0x7f33a4145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a3fff640 1 -- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 msgr2=0x7f33a41f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a3fff640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a41f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a3fff640 1 -- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a4141890 msgr2=0x7f33a4145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a3fff640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a4141890 0x7f33a4145290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a3fff640 1 -- 172.21.15.76:0/1598191891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33a4157cb0 con 0x7f33a4071c70 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a887b640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a4141890 0x7f33a4145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a3fff640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a41576d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f339400d830 tx=0x7f339400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a0ff9640 1 -- 172.21.15.76:0/1598191891 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339400b6c0 con 0x7f33a4071c70 2024-09-19T04:17:48.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a0ff9640 1 -- 172.21.15.76:0/1598191891 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3394019070 con 0x7f33a4071c70 2024-09-19T04:17:48.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33a4157f90 con 0x7f33a4071c70 2024-09-19T04:17:48.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a0ff9640 1 -- 172.21.15.76:0/1598191891 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339400ba60 con 0x7f33a4071c70 2024-09-19T04:17:48.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33a4158550 con 0x7f33a4071c70 2024-09-19T04:17:48.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33a4141350 con 0x7f33a4071c70 2024-09-19T04:17:48.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a0ff9640 1 -- 172.21.15.76:0/1598191891 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f339401e460 con 0x7f33a4071c70 2024-09-19T04:17:48.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a0ff9640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33840678c0 0x7f3384069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a0ff9640 1 -- 172.21.15.76:0/1598191891 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f3394086fd0 con 0x7f33a4071c70 2024-09-19T04:17:48.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a37fe640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33840678c0 0x7f3384069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.494+0000 7f33a37fe640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33840678c0 0x7f3384069d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f338c0af000 tx=0x7f338c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.498+0000 7f33a0ff9640 1 -- 172.21.15.76:0/1598191891 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f33940579d0 con 0x7f33a4071c70 2024-09-19T04:17:48.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.694+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33a40cf850 con 0x7f33a4071c70 2024-09-19T04:17:48.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.694+0000 7f33a0ff9640 1 -- 172.21.15.76:0/1598191891 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5277 (secure 0 0 0) 0x7f33940572d0 con 0x7f33a4071c70 2024-09-19T04:17:48.700 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:48.701 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"btime":"2024-09-19T04:17:21:921326+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:17:21.062516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6165,"mds_1":6156,"mds_2":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"b","rank":2,"incarnation":51,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6156":{"gid":6156,"name":"c","rank":1,"incarnation":49,"state":"up:active","state_seq":14,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[5272,6156,6165]},"id":4}]} 2024-09-19T04:17:48.701 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 58 2024-09-19T04:17:48.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33840678c0 msgr2=0x7f3384069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33840678c0 0x7f3384069d80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f338c0af000 tx=0x7f338c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 msgr2=0x7f33a41576d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a41576d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f339400d830 tx=0x7f339400dcf0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 shutdown_connections 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33840678c0 0x7f3384069d80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33a4141890 0x7f33a4145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33a4140f70 0x7f33a41f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 --2- 172.21.15.76:0/1598191891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33a4071c70 0x7f33a41576d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 >> 172.21.15.76:0/1598191891 conn(0x7f33a40b3e70 msgr2=0x7f33a406ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:48.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 shutdown_connections 2024-09-19T04:17:48.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.698+0000 7f33aa2de640 1 -- 172.21.15.76:0/1598191891 wait complete. 2024-09-19T04:17:48.720 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:17:48.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.830+0000 7f739fe45640 1 Processor -- start 2024-09-19T04:17:48.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.830+0000 7f739fe45640 1 -- start start 2024-09-19T04:17:48.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739fe45640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 0x7f73980d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739fe45640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73980d1f70 con 0x7f739815e450 2024-09-19T04:17:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739fe45640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73980b58d0 con 0x7f73980d2a10 2024-09-19T04:17:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739fe45640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f73980b55f0 con 0x7f73980d4130 2024-09-19T04:17:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f739815e450 0x7f73980b7130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57930/0 (socket says 172.21.15.76:57930) 2024-09-19T04:17:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 -- 172.21.15.76:0/1501065280 learned_addr learned my addr 172.21.15.76:0/1501065280 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739d3e0640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 0x7f73980d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413933785 0 0) 0x7f73980d1f70 con 0x7f739815e450 2024-09-19T04:17:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7384003530 con 0x7f739815e450 2024-09-19T04:17:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2617677678 0 0) 0x7f7384003530 con 0x7f739815e450 2024-09-19T04:17:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 msgr2=0x7f73980d4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 0x7f73980d4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 >> v1:172.21.15.116:6789/0 conn(0x7f73980d2a10 legacy=0x7f73980d2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73980d2210 con 0x7f739815e450 2024-09-19T04:17:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f73940016a0 con 0x7f739815e450 2024-09-19T04:17:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 >> v1:172.21.15.76:6789/0 conn(0x7f739815e450 legacy=0x7f73980b7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f7384003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 0x7f7384004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7384004d50 0x7f73840051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f73980b55f0 con 0x7f73840039f0 2024-09-19T04:17:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f738400ba80 con 0x7f7384004d50 2024-09-19T04:17:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f738400bc80 con 0x7f73980d4130 2024-09-19T04:17:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f7384003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739e3e2640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7384004d50 0x7f73840051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 -- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 msgr2=0x7f7384004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739d3e0640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 0x7f7384004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 0x7f7384004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 -- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7384004d50 msgr2=0x7f73840051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7384004d50 0x7f73840051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739dbe1640 1 -- 172.21.15.76:0/1501065280 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7384003530 con 0x7f73840039f0 2024-09-19T04:17:48.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739e3e2640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7384004d50 0x7f73840051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.834+0000 7f739d3e0640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980d4130 0x7f7384004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f7384003e10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f73980d2140 tx=0x7f739401ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3178bbe1950b69ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394007d60 con 0x7f73840039f0 2024-09-19T04:17:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73940a9070 con 0x7f73840039f0 2024-09-19T04:17:48.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739cbdf640 1 -- 172.21.15.76:0/1501065280 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940a5d60 con 0x7f73840039f0 2024-09-19T04:17:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- 172.21.15.76:0/1501065280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 msgr2=0x7f7384003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 --2- 172.21.15.76:0/1501065280 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f7384003e10 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f73980d2140 tx=0x7f739401ccb0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739e3e2640 1 -- 172.21.15.76:0/1501065280 reap_dead start 2024-09-19T04:17:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- 172.21.15.76:0/1501065280 shutdown_connections 2024-09-19T04:17:48.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- 172.21.15.76:0/1501065280 >> 172.21.15.76:0/1501065280 conn(0x7f739808bf90 msgr2=0x7f739809d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- 172.21.15.76:0/1501065280 shutdown_connections 2024-09-19T04:17:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- 172.21.15.76:0/1501065280 wait complete. 2024-09-19T04:17:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 Processor -- start 2024-09-19T04:17:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- start start 2024-09-19T04:17:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f739815f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7398160f20 0x7f739815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980b29a0 0x7f73980b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f73980d4920 con 0x7f73840039f0 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f739815ea40 con 0x7f7398160f20 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f73980d4aa0 con 0x7f73980b29a0 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f739815f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f739815f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39438/0 (socket says 172.21.15.76:39438) 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 -- 172.21.15.76:0/469266460 learned_addr learned my addr 172.21.15.76:0/469266460 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739d3e0640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7398160f20 0x7f739815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739e3e2640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980b29a0 0x7f73980b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 -- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980b29a0 msgr2=0x7f73980b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980b29a0 0x7f73980b2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 -- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7398160f20 msgr2=0x7f739815b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7398160f20 0x7f739815b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 -- 172.21.15.76:0/469266460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73981613a0 con 0x7f73840039f0 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739d3e0640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7398160f20 0x7f739815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739dbe1640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f739815f580 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f739401cae0 tx=0x7f7384005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f738e7fc640 1 -- 172.21.15.76:0/469266460 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7394000ee0 con 0x7f73840039f0 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f738e7fc640 1 -- 172.21.15.76:0/469266460 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73940a5900 con 0x7f73840039f0 2024-09-19T04:17:48.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f739815fcb0 con 0x7f73840039f0 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f738e7fc640 1 -- 172.21.15.76:0/469266460 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73940047f0 con 0x7f73840039f0 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.838+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7398160270 con 0x7f73840039f0 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.842+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73980b5030 con 0x7f73840039f0 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.842+0000 7f738e7fc640 1 -- 172.21.15.76:0/469266460 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7394004990 con 0x7f73840039f0 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.842+0000 7f738e7fc640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f737c067a90 0x7f737c069f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.842+0000 7f738e7fc640 1 -- 172.21.15.76:0/469266460 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f739411faf0 con 0x7f73840039f0 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.842+0000 7f739d3e0640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f737c067a90 0x7f737c069f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:48.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.842+0000 7f739d3e0640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f737c067a90 0x7f737c069f50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7388009a80 tx=0x7f7388008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:48.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:48.846+0000 7f738e7fc640 1 -- 172.21.15.76:0/469266460 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f73940bf060 con 0x7f73840039f0 2024-09-19T04:17:49.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.010+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f73980d0dc0 con 0x7f73840039f0 2024-09-19T04:17:49.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f738e7fc640 1 -- 172.21.15.76:0/469266460 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17207 (secure 0 0 0) 0x7f73940169f0 con 0x7f73840039f0 2024-09-19T04:17:49.018 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:49.018 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":69,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:17:08.086666+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:16:40.908065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T04:16:41.340553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.13","mappings":[{"from":1,"to":3}]},{"pgid":"10.22","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:17:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f737c067a90 msgr2=0x7f737c069f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f737c067a90 0x7f737c069f50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7388009a80 tx=0x7f7388008040 comp rx=0 tx=0).stop 2024-09-19T04:17:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 msgr2=0x7f739815f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f739815f580 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f739401cae0 tx=0x7f7384005840 comp rx=0 tx=0).stop 2024-09-19T04:17:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 shutdown_connections 2024-09-19T04:17:49.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f737c067a90 0x7f737c069f50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f73980b29a0 0x7f73980b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7398160f20 0x7f739815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 --2- 172.21.15.76:0/469266460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73840039f0 0x7f739815f580 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 >> 172.21.15.76:0/469266460 conn(0x7f739808bf90 msgr2=0x7f73980d15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 shutdown_connections 2024-09-19T04:17:49.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.014+0000 7f739fe45640 1 -- 172.21.15.76:0/469266460 wait complete. 2024-09-19T04:17:49.037 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:17:49.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:17:49.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 Processor -- start 2024-09-19T04:17:49.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 -- start start 2024-09-19T04:17:49.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33c015c2a0 0x7f33c00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f33c00af2a0 0x7f33c015fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00b3290 con 0x7f33c015c2a0 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f33c00b3570 con 0x7f33c00af2a0 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c74da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f33c00b3850 con 0x7f33c015bec0 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33c015c2a0 0x7f33c00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f33c015c2a0 0x7f33c00aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39454/0 (socket says 172.21.15.76:39454) 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 -- 172.21.15.76:0/878207095 learned_addr learned my addr 172.21.15.76:0/878207095 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c5276640 1 --2- 172.21.15.76:0/878207095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c5a77640 1 --2- 172.21.15.76:0/878207095 >> v2:172.21.15.116:3300/0 conn(0x7f33c00af2a0 0x7f33c015fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 -- 172.21.15.76:0/878207095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 msgr2=0x7f33c00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 --2- 172.21.15.76:0/878207095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c00ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 -- 172.21.15.76:0/878207095 >> v2:172.21.15.116:3300/0 conn(0x7f33c00af2a0 msgr2=0x7f33c015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 --2- 172.21.15.76:0/878207095 >> v2:172.21.15.116:3300/0 conn(0x7f33c00af2a0 0x7f33c015fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c4a75640 1 -- 172.21.15.76:0/878207095 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c00b3b30 con 0x7f33c015c2a0 2024-09-19T04:17:49.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.154+0000 7f33c5276640 1 --2- 172.21.15.76:0/878207095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c00ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:49.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c4a75640 1 --2- 172.21.15.76:0/878207095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c015c2a0 0x7f33c00aed60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ada40 tx=0x7f33bc01c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1d7e0a0605bd382 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33b7fff640 1 -- 172.21.15.76:0/878207095 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0b0070 con 0x7f33c015c2a0 2024-09-19T04:17:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33b7fff640 1 -- 172.21.15.76:0/878207095 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33bc0ab600 con 0x7f33c015c2a0 2024-09-19T04:17:49.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33b7fff640 1 -- 172.21.15.76:0/878207095 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0ab9d0 con 0x7f33c015c2a0 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- 172.21.15.76:0/878207095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c015c2a0 msgr2=0x7f33c00aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 --2- 172.21.15.76:0/878207095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c015c2a0 0x7f33c00aed60 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f33c00ada40 tx=0x7f33bc01c230 comp rx=0 tx=0).stop 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- 172.21.15.76:0/878207095 shutdown_connections 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 --2- 172.21.15.76:0/878207095 >> v2:172.21.15.116:3300/0 conn(0x7f33c00af2a0 0x7f33c015fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 --2- 172.21.15.76:0/878207095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c015c2a0 0x7f33c00aed60 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 --2- 172.21.15.76:0/878207095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c00ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- 172.21.15.76:0/878207095 >> 172.21.15.76:0/878207095 conn(0x7f33c00b1f90 msgr2=0x7f33c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- 172.21.15.76:0/878207095 shutdown_connections 2024-09-19T04:17:49.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- 172.21.15.76:0/878207095 wait complete. 2024-09-19T04:17:49.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 Processor -- start 2024-09-19T04:17:49.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- start start 2024-09-19T04:17:49.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c00af2a0 0x7f33c0160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c009ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c015c2a0 0x7f33c015eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00af6b0 con 0x7f33c00af2a0 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f33c00ad740 con 0x7f33c015c2a0 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c74da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f33c006e000 con 0x7f33c015bec0 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c00af2a0 0x7f33c0160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c00af2a0 0x7f33c0160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39470/0 (socket says 172.21.15.76:39470) 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 -- 172.21.15.76:0/940183861 learned_addr learned my addr 172.21.15.76:0/940183861 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c4a75640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c009ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 -- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 msgr2=0x7f33c009ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c009ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 -- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c015c2a0 msgr2=0x7f33c015eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5a77640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c015c2a0 0x7f33c015eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c015c2a0 0x7f33c015eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 -- 172.21.15.76:0/940183861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33c0155670 con 0x7f33c00af2a0 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.158+0000 7f33c5276640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c00af2a0 0x7f33c0160c70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e810 tx=0x7f33b000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33b5ffb640 1 -- 172.21.15.76:0/940183861 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b0017070 con 0x7f33c00af2a0 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33b5ffb640 1 -- 172.21.15.76:0/940183861 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33b000c8d0 con 0x7f33c00af2a0 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33c0154dd0 con 0x7f33c00af2a0 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33b5ffb640 1 -- 172.21.15.76:0/940183861 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b000f040 con 0x7f33c00af2a0 2024-09-19T04:17:49.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33c0155390 con 0x7f33c00af2a0 2024-09-19T04:17:49.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33c00ad200 con 0x7f33c00af2a0 2024-09-19T04:17:49.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33b5ffb640 1 -- 172.21.15.76:0/940183861 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f33b001a070 con 0x7f33c00af2a0 2024-09-19T04:17:49.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33b5ffb640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33a40678c0 0x7f33a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33c4a75640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33a40678c0 0x7f33a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33b5ffb640 1 -- 172.21.15.76:0/940183861 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f33b00859b0 con 0x7f33c00af2a0 2024-09-19T04:17:49.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.162+0000 7f33c4a75640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33a40678c0 0x7f33a4069d80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0ad000 tx=0x7f33bc0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.166+0000 7f33b5ffb640 1 -- 172.21.15.76:0/940183861 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f33b0056360 con 0x7f33c00af2a0 2024-09-19T04:17:49.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.366+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33c006fcd0 con 0x7f33c00af2a0 2024-09-19T04:17:49.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.366+0000 7f33b5ffb640 1 -- 172.21.15.76:0/940183861 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5277 (secure 0 0 0) 0x7f33b0055c60 con 0x7f33c00af2a0 2024-09-19T04:17:49.373 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:49.373 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"btime":"2024-09-19T04:17:21:921326+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:17:21.062516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6165,"mds_1":6156,"mds_2":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"b","rank":2,"incarnation":51,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6156":{"gid":6156,"name":"c","rank":1,"incarnation":49,"state":"up:active","state_seq":14,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[5272,6156,6165]},"id":4}]} 2024-09-19T04:17:49.373 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 58 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33a40678c0 msgr2=0x7f33a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33a40678c0 0x7f33a4069d80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0ad000 tx=0x7f33bc0a6000 comp rx=0 tx=0).stop 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c00af2a0 msgr2=0x7f33c0160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c00af2a0 0x7f33c0160c70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f33b000e810 tx=0x7f33b000ecd0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 shutdown_connections 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f33a40678c0 0x7f33a4069d80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33c015c2a0 0x7f33c015eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f33c015bec0 0x7f33c009ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 --2- 172.21.15.76:0/940183861 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f33c00af2a0 0x7f33c0160c70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 >> 172.21.15.76:0/940183861 conn(0x7f33c00b1f90 msgr2=0x7f33c00cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:49.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 shutdown_connections 2024-09-19T04:17:49.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.370+0000 7f33c74da640 1 -- 172.21.15.76:0/940183861 wait complete. 2024-09-19T04:17:49.392 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:17:49.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6952fa640 1 Processor -- start 2024-09-19T04:17:49.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6952fa640 1 -- start start 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6952fa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6901520a0 0x7fe69006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6952fa640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6952fa640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe690070250 con 0x7fe69015f1d0 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6952fa640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe690152480 con 0x7fe6901520a0 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6952fa640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe6900aede0 con 0x7fe69006fe50 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6901520a0 0x7fe69006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe6901520a0 0x7fe69006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39484/0 (socket says 172.21.15.76:39484) 2024-09-19T04:17:49.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 -- 172.21.15.76:0/1641884068 learned_addr learned my addr 172.21.15.76:0/1641884068 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:49.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6867fc640 1 --2- 172.21.15.76:0/1641884068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 -- 172.21.15.76:0/1641884068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 msgr2=0x7fe69015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:49.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 --2- 172.21.15.76:0/1641884068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 -- 172.21.15.76:0/1641884068 >> v1:172.21.15.116:6789/0 conn(0x7fe69015f1d0 legacy=0x7fe690071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:49.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 -- 172.21.15.76:0/1641884068 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69015fa10 con 0x7fe6901520a0 2024-09-19T04:17:49.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe6867fc640 1 --2- 172.21.15.76:0/1641884068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:49.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.498+0000 7fe68effd640 1 --2- 172.21.15.76:0/1641884068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6901520a0 0x7fe69006f910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe680009870 tx=0x7fe68001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e6f27b6cda0297 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68e7fc640 1 -- 172.21.15.76:0/1641884068 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800b1070 con 0x7fe6901520a0 2024-09-19T04:17:49.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68e7fc640 1 -- 172.21.15.76:0/1641884068 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6800a8050 con 0x7fe6901520a0 2024-09-19T04:17:49.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68e7fc640 1 -- 172.21.15.76:0/1641884068 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6800ac860 con 0x7fe6901520a0 2024-09-19T04:17:49.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- 172.21.15.76:0/1641884068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6901520a0 msgr2=0x7fe69006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 --2- 172.21.15.76:0/1641884068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6901520a0 0x7fe69006f910 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe680009870 tx=0x7fe68001c500 comp rx=0 tx=0).stop 2024-09-19T04:17:49.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- 172.21.15.76:0/1641884068 shutdown_connections 2024-09-19T04:17:49.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 --2- 172.21.15.76:0/1641884068 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 --2- 172.21.15.76:0/1641884068 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe6901520a0 0x7fe69006f910 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- 172.21.15.76:0/1641884068 >> 172.21.15.76:0/1641884068 conn(0x7fe6900b1f90 msgr2=0x7fe6900b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:49.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- 172.21.15.76:0/1641884068 shutdown_connections 2024-09-19T04:17:49.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- 172.21.15.76:0/1641884068 wait complete. 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 Processor -- start 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- start start 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6901520a0 0x7fe69009dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe69015f1d0 0x7fe69015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe690073d30 con 0x7fe69015f1d0 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe690073a50 con 0x7fe6901520a0 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe69015f680 con 0x7fe69006fe50 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe69015f1d0 0x7fe69015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe69015f1d0 0x7fe69015dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39488/0 (socket says 172.21.15.76:39488) 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 -- 172.21.15.76:0/2434044108 learned_addr learned my addr 172.21.15.76:0/2434044108 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6867fc640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6901520a0 0x7fe69009dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68effd640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 -- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 msgr2=0x7fe69015c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 -- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6901520a0 msgr2=0x7fe69009dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6901520a0 0x7fe69009dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 -- 172.21.15.76:0/2434044108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69015ce20 con 0x7fe69015f1d0 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe68f7fe640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe69015f1d0 0x7fe69015dec0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c780 tx=0x7fe68800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe687fff640 1 -- 172.21.15.76:0/2434044108 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688004b70 con 0x7fe69015f1d0 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe687fff640 1 -- 172.21.15.76:0/2434044108 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe688004d10 con 0x7fe69015f1d0 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69015d100 con 0x7fe69015f1d0 2024-09-19T04:17:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe687fff640 1 -- 172.21.15.76:0/2434044108 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68800aa90 con 0x7fe69015f1d0 2024-09-19T04:17:49.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.502+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe69015d6c0 con 0x7fe69015f1d0 2024-09-19T04:17:49.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.506+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69015ec90 con 0x7fe69015f1d0 2024-09-19T04:17:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.506+0000 7fe687fff640 1 -- 172.21.15.76:0/2434044108 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe68800ac30 con 0x7fe69015f1d0 2024-09-19T04:17:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.506+0000 7fe687fff640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe6700678c0 0x7fe670069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.506+0000 7fe687fff640 1 -- 172.21.15.76:0/2434044108 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fe6880861b0 con 0x7fe69015f1d0 2024-09-19T04:17:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.506+0000 7fe68effd640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe6700678c0 0x7fe670069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.506+0000 7fe68effd640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe6700678c0 0x7fe670069d80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe6800ae000 tx=0x7fe680008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.510+0000 7fe687fff640 1 -- 172.21.15.76:0/2434044108 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe688020070 con 0x7fe69015f1d0 2024-09-19T04:17:49.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.690+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe69006e540 con 0x7fe69015f1d0 2024-09-19T04:17:49.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.690+0000 7fe687fff640 1 -- 172.21.15.76:0/2434044108 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) ==== 94+0+5277 (secure 0 0 0) 0x7fe688056ad0 con 0x7fe69015f1d0 2024-09-19T04:17:49.696 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:49.696 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":58,"btime":"2024-09-19T04:17:21:921326+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:16:42.356083+0000","modified":"2024-09-19T04:17:21.062516+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6165,"mds_1":6156,"mds_2":5272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5272":{"gid":5272,"name":"b","rank":2,"incarnation":51,"state":"up:active","state_seq":13,"addr":"172.21.15.116:6833/1054899754","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1054899754},{"type":"v1","addr":"172.21.15.116:6833","nonce":1054899754}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6156":{"gid":6156,"name":"c","rank":1,"incarnation":49,"state":"up:active","state_seq":14,"addr":"172.21.15.76:6835/2740411101","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2740411101},{"type":"v1","addr":"172.21.15.76:6835","nonce":2740411101}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6165":{"gid":6165,"name":"a","rank":0,"incarnation":44,"state":"up:active","state_seq":12,"addr":"172.21.15.76:6837/3052694229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3052694229},{"type":"v1","addr":"172.21.15.76:6837","nonce":3052694229}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6165,"qdb_cluster":[5272,6156,6165]},"id":4}]} 2024-09-19T04:17:49.697 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 58 2024-09-19T04:17:49.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe6700678c0 msgr2=0x7fe670069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe6700678c0 0x7fe670069d80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe6800ae000 tx=0x7fe680008040 comp rx=0 tx=0).stop 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe69015f1d0 msgr2=0x7fe69015dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe69015f1d0 0x7fe69015dec0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe68800c780 tx=0x7fe68800cc40 comp rx=0 tx=0).stop 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 shutdown_connections 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe6700678c0 0x7fe670069d80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe69015f1d0 0x7fe69015dec0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe6901520a0 0x7fe69009dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 --2- 172.21.15.76:0/2434044108 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe69006fe50 0x7fe69015c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 >> 172.21.15.76:0/2434044108 conn(0x7fe6900b1f90 msgr2=0x7fe6900b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 shutdown_connections 2024-09-19T04:17:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.694+0000 7fe6952fa640 1 -- 172.21.15.76:0/2434044108 wait complete. 2024-09-19T04:17:49.714 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:17:49.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.822+0000 7fa4607fc640 1 Processor -- start 2024-09-19T04:17:49.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.822+0000 7fa4607fc640 1 -- start start 2024-09-19T04:17:49.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa4607fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa458070020 0x7fa45815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa4607fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa45815f1d0 0x7fa4580adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa4607fc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4580aede0 con 0x7fa4581520a0 2024-09-19T04:17:49.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa4607fc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa458152480 con 0x7fa45815f1d0 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa4607fc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa45815fa10 con 0x7fa458070020 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa45815f1d0 0x7fa4580adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa45815f1d0 0x7fa4580adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39494/0 (socket says 172.21.15.76:39494) 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 -- 172.21.15.76:0/979755316 learned_addr learned my addr 172.21.15.76:0/979755316 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45dd97640 1 --2- 172.21.15.76:0/979755316 >> v2:172.21.15.116:3300/0 conn(0x7fa458070020 0x7fa45815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 -- 172.21.15.76:0/979755316 >> v2:172.21.15.116:3300/0 conn(0x7fa458070020 msgr2=0x7fa45815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 --2- 172.21.15.76:0/979755316 >> v2:172.21.15.116:3300/0 conn(0x7fa458070020 0x7fa45815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 -- 172.21.15.76:0/979755316 >> v1:172.21.15.116:6789/0 conn(0x7fa4581520a0 legacy=0x7fa45806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 -- 172.21.15.76:0/979755316 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa458074250 con 0x7fa45815f1d0 2024-09-19T04:17:49.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45ed99640 1 --2- 172.21.15.76:0/979755316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa45815f1d0 0x7fa4580adc80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa45806e510 tx=0x7fa45001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5573395929a87cf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45d596640 1 -- 172.21.15.76:0/979755316 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4500a5880 con 0x7fa45815f1d0 2024-09-19T04:17:49.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45d596640 1 -- 172.21.15.76:0/979755316 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4500a5a20 con 0x7fa45815f1d0 2024-09-19T04:17:49.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.826+0000 7fa45d596640 1 -- 172.21.15.76:0/979755316 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4500a5df0 con 0x7fa45815f1d0 2024-09-19T04:17:49.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/979755316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa45815f1d0 msgr2=0x7fa4580adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 --2- 172.21.15.76:0/979755316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa45815f1d0 0x7fa4580adc80 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa45806e510 tx=0x7fa45001c9c0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/979755316 shutdown_connections 2024-09-19T04:17:49.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 --2- 172.21.15.76:0/979755316 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa45815f1d0 0x7fa4580adc80 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 --2- 172.21.15.76:0/979755316 >> v2:172.21.15.116:3300/0 conn(0x7fa458070020 0x7fa45815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/979755316 >> 172.21.15.76:0/979755316 conn(0x7fa4580b1f90 msgr2=0x7fa4580b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:49.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/979755316 shutdown_connections 2024-09-19T04:17:49.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/979755316 wait complete. 2024-09-19T04:17:49.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 Processor -- start 2024-09-19T04:17:49.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- start start 2024-09-19T04:17:49.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa458070020 0x7fa458146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4581520a0 0x7fa45814ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa45815f1d0 0x7fa4580b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa45815f680 con 0x7fa4581520a0 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa458073d30 con 0x7fa458070020 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa458167f10 con 0x7fa45815f1d0 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4581520a0 0x7fa45814ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4581520a0 0x7fa45814ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39498/0 (socket says 172.21.15.76:39498) 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 -- 172.21.15.76:0/3570027006 learned_addr learned my addr 172.21.15.76:0/3570027006 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45e598640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa458070020 0x7fa458146720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45ed99640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa45815f1d0 0x7fa4580b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 -- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa45815f1d0 msgr2=0x7fa4580b1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa45815f1d0 0x7fa4580b1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 -- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa458070020 msgr2=0x7fa458146720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa458070020 0x7fa458146720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 -- 172.21.15.76:0/3570027006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa458146c60 con 0x7fa4581520a0 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa45dd97640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4581520a0 0x7fa45814ba60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa44800d5d0 tx=0x7fa44800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa44effd640 1 -- 172.21.15.76:0/3570027006 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448004240 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45814bfa0 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa44effd640 1 -- 172.21.15.76:0/3570027006 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4480043e0 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa44effd640 1 -- 172.21.15.76:0/3570027006 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448003840 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa458147590 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.830+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa45815ec90 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.834+0000 7fa44effd640 1 -- 172.21.15.76:0/3570027006 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa4480039e0 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.834+0000 7fa44effd640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa43c0678f0 0x7fa43c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.834+0000 7fa44effd640 1 -- 172.21.15.76:0/3570027006 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7fa448086120 con 0x7fa4581520a0 2024-09-19T04:17:49.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.834+0000 7fa45e598640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa43c0678f0 0x7fa43c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:49.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.834+0000 7fa45e598640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa43c0678f0 0x7fa43c069db0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa4580cc470 tx=0x7fa454009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:49.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.838+0000 7fa44effd640 1 -- 172.21.15.76:0/3570027006 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa44801c360 con 0x7fa4581520a0 2024-09-19T04:17:49.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.986+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa45806e540 con 0x7fa4581520a0 2024-09-19T04:17:49.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.986+0000 7fa44effd640 1 -- 172.21.15.76:0/3570027006 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+17207 (secure 0 0 0) 0x7fa44807de20 con 0x7fa4581520a0 2024-09-19T04:17:49.993 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:49.993 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":69,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:17:08.086666+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:16:40.908065+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-09-19T04:16:41.340553+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.10","mappings":[{"from":7,"to":2}]},{"pgid":"10.2","mappings":[{"from":4,"to":5}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.13","mappings":[{"from":1,"to":3}]},{"pgid":"10.22","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa43c0678f0 msgr2=0x7fa43c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa43c0678f0 0x7fa43c069db0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa4580cc470 tx=0x7fa454009040 comp rx=0 tx=0).stop 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4581520a0 msgr2=0x7fa45814ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4581520a0 0x7fa45814ba60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fa44800d5d0 tx=0x7fa44800da90 comp rx=0 tx=0).stop 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 shutdown_connections 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa43c0678f0 0x7fa43c069db0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa45815f1d0 0x7fa4580b1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa4581520a0 0x7fa45814ba60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 --2- 172.21.15.76:0/3570027006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa458070020 0x7fa458146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 >> 172.21.15.76:0/3570027006 conn(0x7fa4580b1f90 msgr2=0x7fa458071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:49.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 shutdown_connections 2024-09-19T04:17:49.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:49.990+0000 7fa4607fc640 1 -- 172.21.15.76:0/3570027006 wait complete. 2024-09-19T04:17:50.011 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T04:17:50.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1918e7d640 1 Processor -- start 2024-09-19T04:17:50.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1918e7d640 1 -- start start 2024-09-19T04:17:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1918e7d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1914156720 0x7f19140b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1918e7d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19141412e0 0x7f19140b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1918e7d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f191413fb30 con 0x7f19140b5b50 2024-09-19T04:17:50.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1918e7d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1914140740 con 0x7f19141412e0 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1918e7d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f191413fe10 con 0x7f1914156720 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1912d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19141412e0 0x7f19140b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1912d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f19141412e0 0x7f19140b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39514/0 (socket says 172.21.15.76:39514) 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1912d9d640 1 -- 172.21.15.76:0/1650155422 learned_addr learned my addr 172.21.15.76:0/1650155422 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1912d9d640 1 -- 172.21.15.76:0/1650155422 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1914156720 msgr2=0x7f19140b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1912d9d640 1 --2- 172.21.15.76:0/1650155422 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1914156720 0x7f19140b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1912d9d640 1 -- 172.21.15.76:0/1650155422 >> v1:172.21.15.116:6789/0 conn(0x7f19140b5b50 legacy=0x7f1914140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:50.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.118+0000 7f1912d9d640 1 -- 172.21.15.76:0/1650155422 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19141400f0 con 0x7f19141412e0 2024-09-19T04:17:50.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1912d9d640 1 --2- 172.21.15.76:0/1650155422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19141412e0 0x7f19140b4410 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f190c009e50 tx=0x7f190c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a650e2d177757cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:50.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1911d9b640 1 -- 172.21.15.76:0/1650155422 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c0a5670 con 0x7f19141412e0 2024-09-19T04:17:50.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1911d9b640 1 -- 172.21.15.76:0/1650155422 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c0a5810 con 0x7f19141412e0 2024-09-19T04:17:50.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1911d9b640 1 -- 172.21.15.76:0/1650155422 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c0a5be0 con 0x7f19141412e0 2024-09-19T04:17:50.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- 172.21.15.76:0/1650155422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19141412e0 msgr2=0x7f19140b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 --2- 172.21.15.76:0/1650155422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19141412e0 0x7f19140b4410 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f190c009e50 tx=0x7f190c01c9f0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- 172.21.15.76:0/1650155422 shutdown_connections 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 --2- 172.21.15.76:0/1650155422 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19141412e0 0x7f19140b4410 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 --2- 172.21.15.76:0/1650155422 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1914156720 0x7f19140b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- 172.21.15.76:0/1650155422 >> 172.21.15.76:0/1650155422 conn(0x7f191406c0c0 msgr2=0x7f191406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- 172.21.15.76:0/1650155422 shutdown_connections 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- 172.21.15.76:0/1650155422 wait complete. 2024-09-19T04:17:50.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 Processor -- start 2024-09-19T04:17:50.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- start start 2024-09-19T04:17:50.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19140b5b50 0x7f1914159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f19141412e0 0x7f191406e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1914156720 0x7f19141f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f191415ca20 con 0x7f19140b5b50 2024-09-19T04:17:50.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f191413f8f0 con 0x7f1914156720 2024-09-19T04:17:50.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f191415cba0 con 0x7f19141412e0 2024-09-19T04:17:50.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19140b5b50 0x7f1914159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19140b5b50 0x7f1914159860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39516/0 (socket says 172.21.15.76:39516) 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 -- 172.21.15.76:0/3723127866 learned_addr learned my addr 172.21.15.76:0/3723127866 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 -- 172.21.15.76:0/3723127866 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f19141412e0 msgr2=0x7f191406e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f19141412e0 0x7f191406e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 -- 172.21.15.76:0/3723127866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1914156720 msgr2=0x7f19141f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1914156720 0x7f19141f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 -- 172.21.15.76:0/3723127866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191406e920 con 0x7f19140b5b50 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f191259c640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19140b5b50 0x7f1914159860 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f190400b410 tx=0x7f190400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f190affd640 1 -- 172.21.15.76:0/3723127866 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904015020 con 0x7f19140b5b50 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f190affd640 1 -- 172.21.15.76:0/3723127866 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1904011830 con 0x7f19140b5b50 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191406ec00 con 0x7f19140b5b50 2024-09-19T04:17:50.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f190affd640 1 -- 172.21.15.76:0/3723127866 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904011c20 con 0x7f19140b5b50 2024-09-19T04:17:50.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.122+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1914143d10 con 0x7f19140b5b50 2024-09-19T04:17:50.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.126+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1914069170 con 0x7f19140b5b50 2024-09-19T04:17:50.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.126+0000 7f190affd640 1 -- 172.21.15.76:0/3723127866 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1904018020 con 0x7f19140b5b50 2024-09-19T04:17:50.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.126+0000 7f190affd640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f18f40678c0 0x7f18f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.126+0000 7f190affd640 1 -- 172.21.15.76:0/3723127866 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6246+0+0 (secure 0 0 0) 0x7f1904085e50 con 0x7f19140b5b50 2024-09-19T04:17:50.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.126+0000 7f190bfff640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f18f40678c0 0x7f18f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.126+0000 7f190bfff640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f18f40678c0 0x7f18f4069d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007fd0 tx=0x7f18fc0079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:50.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.130+0000 7f190affd640 1 -- 172.21.15.76:0/3723127866 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f190401f020 con 0x7f19140b5b50 2024-09-19T04:17:50.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.318+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f191415b280 con 0x7f19140b5b50 2024-09-19T04:17:50.326 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:17:50.322+0000 7fa674384640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:17:50.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.326+0000 7f190affd640 1 -- 172.21.15.76:0/3723127866 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) ==== 161+0+0 (secure 0 0 0) 0x7f1904056770 con 0x7f19140b5b50 2024-09-19T04:17:50.329 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:17:50.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.326+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f18f40678c0 msgr2=0x7f18f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.326+0000 7f1918e7d640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f18f40678c0 0x7f18f4069d80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f18fc007fd0 tx=0x7f18fc0079a0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.326+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19140b5b50 msgr2=0x7f1914159860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.326+0000 7f1918e7d640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19140b5b50 0x7f1914159860 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f190400b410 tx=0x7f190400b8d0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.330+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 shutdown_connections 2024-09-19T04:17:50.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.330+0000 7f1918e7d640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f18f40678c0 0x7f18f4069d80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.330+0000 7f1918e7d640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1914156720 0x7f19141f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.330+0000 7f1918e7d640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f19141412e0 0x7f191406e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.330+0000 7f1918e7d640 1 --2- 172.21.15.76:0/3723127866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f19140b5b50 0x7f1914159860 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.330+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 >> 172.21.15.76:0/3723127866 conn(0x7f191406c0c0 msgr2=0x7f19140d18e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:50.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.330+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 shutdown_connections 2024-09-19T04:17:50.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.334+0000 7f1918e7d640 1 -- 172.21.15.76:0/3723127866 wait complete. 2024-09-19T04:17:50.344 INFO:tasks.ceph.mds.a.smithi076.stderr: -8377> 2024-09-19T04:17:03.867+0000 7f9c43701640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:17:50.346 INFO:tasks.ceph.mds.c.smithi076.stderr: -1546> 2024-09-19T04:17:16.267+0000 7f0ebd98a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:17:00.467434+0000 1=0+1) n(v2 rc2024-09-19T04:17:00.671707+0000 4=0+4) (inest lock) caps={6330=pAsLsXsFs/-@7},l=6330 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x564dc4818b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.467434+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.671707+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.467434+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:17:00.671707+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:17:50.346 INFO:tasks.ceph.mds.c.smithi076.stderr: -1408> 2024-09-19T04:17:16.267+0000 7f0ec4197640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:17:00.468809+0000 1=0+1) n(v1 rc2024-09-19T04:17:00.671707+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={6330=pAsLsXsFs/-@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x564dc4819080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.468809+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.470271+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.468809+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:17:00.671707+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:17:50.351 INFO:tasks.ceph.mds.b.smithi116.stderr: -1438> 2024-09-19T04:17:16.267+0000 7f89f9c5a640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:17:00.470271+0000 1=0+1) n(v0 rc2024-09-19T04:17:00.470271+0000 2=0+2) (inest lock) (ifile excl) caps={6330=pAsLsXsFsx/-@6},l=6330 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x5639178a1080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.470271+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.470271+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.470271+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:17:00.470271+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:17:50.351 INFO:tasks.ceph.mds.b.smithi116.stderr: -1418> 2024-09-19T04:17:16.267+0000 7f89f9c5a640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={6330=pAsLsXsFs/-@3},l=6330 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x5639188e0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:17:50.354 INFO:tasks.ceph.mds.a.smithi076.stderr: -3368> 2024-09-19T04:17:16.263+0000 7f9c3f6f9640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:17:00.464621+0000 1=0+1) n(v3 rc2024-09-19T04:17:00.671707+0000 6=0+6) (inest mix dirty) (ifile excl) caps={6330=pAsLsXsFsx/AsLsXsFsx@10},l=6330 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x560dd6095600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.464621+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.671707+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.464621+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:17:00.671707+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:17:50.354 INFO:tasks.ceph.mds.a.smithi076.stderr: -3270> 2024-09-19T04:17:16.267+0000 7f9c45f06640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:17:00.465988+0000 1=0+1) n(v4 rc2024-09-19T04:17:00.671707+0000 5=0+5) (inest sync->mix g=1,2) caps={6330=pAsLsXsFs/-@11} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x560dd6095b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:17:00.465988+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:17:00.671707+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:17:00.465988+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:17:00.671707+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:17:50.355 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:17:50.373 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:17:50.390 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:17:50.407 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:17:50.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.466+0000 7fc9e8a20640 1 Processor -- start 2024-09-19T04:17:50.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.466+0000 7fc9e8a20640 1 -- start start 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e8a20640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 0x7fc9e40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e8a20640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e40b3290 con 0x7fc9e40af470 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e8a20640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9e40b3570 con 0x7fc9e415c2a0 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e8a20640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40b3850 con 0x7fc9e415bec0 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e2d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc9e40af470 0x7fc9e406e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57958/0 (socket says 172.21.15.76:57958) 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e2d9d640 1 -- 172.21.15.76:0/2337636708 learned_addr learned my addr 172.21.15.76:0/2337636708 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e259c640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 0x7fc9e40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2984485307 0 0) 0x7fc9e40b3290 con 0x7fc9e40af470 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9c8003530 con 0x7fc9e40af470 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2398215396 0 0) 0x7fc9c8003530 con 0x7fc9e40af470 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 msgr2=0x7fc9e40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:50.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 0x7fc9e40ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 >> v1:172.21.15.116:6789/0 conn(0x7fc9e415c2a0 legacy=0x7fc9e40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e40b3b30 con 0x7fc9e40af470 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fc9dc001a00 con 0x7fc9e40af470 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 >> v1:172.21.15.76:6789/0 conn(0x7fc9e40af470 legacy=0x7fc9e406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9c80039f0 0x7fc9c8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9c8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 0x7fc9c80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40b3850 con 0x7fc9c8004810 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c800ba80 con 0x7fc9c80039f0 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc9c800bc80 con 0x7fc9e415bec0 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e1d9b640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9c8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e2d9d640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 0x7fc9c80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e259c640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9c80039f0 0x7fc9c8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e1d9b640 1 -- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 msgr2=0x7fc9c80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e1d9b640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e415bec0 0x7fc9c80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e1d9b640 1 -- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9c80039f0 msgr2=0x7fc9c8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e1d9b640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9c80039f0 0x7fc9c8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e1d9b640 1 -- 172.21.15.76:0/2337636708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c8003530 con 0x7fc9c8004810 2024-09-19T04:17:50.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e1d9b640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9c8004c90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40ae510 tx=0x7fc9d001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a71424c99abf4a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:50.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00013a0 con 0x7fc9c8004810 2024-09-19T04:17:50.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d00052f0 con 0x7fc9c8004810 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.470+0000 7fc9e159a640 1 -- 172.21.15.76:0/2337636708 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0003840 con 0x7fc9c8004810 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- 172.21.15.76:0/2337636708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 msgr2=0x7fc9c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 --2- 172.21.15.76:0/2337636708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9c8004c90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fc9e40ae510 tx=0x7fc9d001c360 comp rx=0 tx=0).stop 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e2d9d640 1 -- 172.21.15.76:0/2337636708 reap_dead start 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- 172.21.15.76:0/2337636708 shutdown_connections 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- 172.21.15.76:0/2337636708 >> 172.21.15.76:0/2337636708 conn(0x7fc9e40b1f90 msgr2=0x7fc9e40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- 172.21.15.76:0/2337636708 shutdown_connections 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- 172.21.15.76:0/2337636708 wait complete. 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 Processor -- start 2024-09-19T04:17:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- start start 2024-09-19T04:17:50.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9e4156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9e409cb50 0x7fc9e409cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e40c6280 0x7fc9e40c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40afc90 con 0x7fc9c8004810 2024-09-19T04:17:50.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e40b40f0 con 0x7fc9e409cb50 2024-09-19T04:17:50.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e40ad200 con 0x7fc9e40c6280 2024-09-19T04:17:50.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9e4156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9e4156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39570/0 (socket says 172.21.15.76:39570) 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 -- 172.21.15.76:0/1713336057 learned_addr learned my addr 172.21.15.76:0/1713336057 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e2d9d640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e40c6280 0x7fc9e40c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e1d9b640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9e409cb50 0x7fc9e409cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 -- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e40c6280 msgr2=0x7fc9e40c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e40c6280 0x7fc9e40c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 -- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9e409cb50 msgr2=0x7fc9e409cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9e409cb50 0x7fc9e409cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 -- 172.21.15.76:0/1713336057 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e4156e60 con 0x7fc9c8004810 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e259c640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9e4156920 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40115e0 tx=0x7fc9d4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9daffd640 1 -- 172.21.15.76:0/1713336057 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4015870 con 0x7fc9c8004810 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9daffd640 1 -- 172.21.15.76:0/1713336057 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9d4015a10 con 0x7fc9c8004810 2024-09-19T04:17:50.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9daffd640 1 -- 172.21.15.76:0/1713336057 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4015dc0 con 0x7fc9c8004810 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e409ae30 con 0x7fc9c8004810 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.474+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e409b3b0 con 0x7fc9c8004810 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.478+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e40ad630 con 0x7fc9c8004810 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.478+0000 7fc9daffd640 1 -- 172.21.15.76:0/1713336057 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9d400f480 con 0x7fc9c8004810 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.478+0000 7fc9daffd640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9cc0678c0 0x7fc9cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.478+0000 7fc9e1d9b640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9cc0678c0 0x7fc9cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.478+0000 7fc9daffd640 1 -- 172.21.15.76:0/1713336057 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6504+0+0 (secure 0 0 0) 0x7fc9d4086500 con 0x7fc9c8004810 2024-09-19T04:17:50.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.478+0000 7fc9e1d9b640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9cc0678c0 0x7fc9cc069d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80057f0 tx=0x7fc9d00011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:50.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.482+0000 7fc9daffd640 1 -- 172.21.15.76:0/1713336057 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc9d4019030 con 0x7fc9c8004810 2024-09-19T04:17:50.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:50.666+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc9e406fcd0 con 0x7fc9c8004810 2024-09-19T04:17:51.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9daffd640 1 -- 172.21.15.76:0/1713336057 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) ==== 104+0+0 (secure 0 0 0) 0x7fc9d4056d00 con 0x7fc9c8004810 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9cc0678c0 msgr2=0x7fc9cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9cc0678c0 0x7fc9cc069d80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80057f0 tx=0x7fc9d00011a0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 msgr2=0x7fc9e4156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9e4156920 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fc9d40115e0 tx=0x7fc9d4011aa0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 shutdown_connections 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9cc0678c0 0x7fc9cc069d80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9e40c6280 0x7fc9e40c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9e409cb50 0x7fc9e409cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 --2- 172.21.15.76:0/1713336057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9c8004810 0x7fc9e4156920 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 >> 172.21.15.76:0/1713336057 conn(0x7fc9e40b1f90 msgr2=0x7fc9e406df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:51.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 shutdown_connections 2024-09-19T04:17:51.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.330+0000 7fc9e8a20640 1 -- 172.21.15.76:0/1713336057 wait complete. 2024-09-19T04:17:51.357 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:17:51.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdc025b640 1 Processor -- start 2024-09-19T04:17:51.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdc025b640 1 -- start start 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdc025b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbdb815c2a0 0x7fbdb80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdc025b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdb80af2a0 0x7fbdb806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdc025b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdb80b3290 con 0x7fbdb815bec0 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdc025b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb80b3570 con 0x7fbdb80af2a0 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdc025b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbdb80b3850 con 0x7fbdb815c2a0 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbe7f8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdb80af2a0 0x7fbdb806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbe7f8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbdb80af2a0 0x7fbdb806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39584/0 (socket says 172.21.15.76:39584) 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbe7f8640 1 -- 172.21.15.76:0/4207683528 learned_addr learned my addr 172.21.15.76:0/4207683528 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbd7f6640 1 --2- 172.21.15.76:0/4207683528 >> v2:172.21.15.116:3300/0 conn(0x7fbdb815c2a0 0x7fbdb80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbe7f8640 1 -- 172.21.15.76:0/4207683528 >> v2:172.21.15.116:3300/0 conn(0x7fbdb815c2a0 msgr2=0x7fbdb80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbe7f8640 1 --2- 172.21.15.76:0/4207683528 >> v2:172.21.15.116:3300/0 conn(0x7fbdb815c2a0 0x7fbdb80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbe7f8640 1 -- 172.21.15.76:0/4207683528 >> v1:172.21.15.116:6789/0 conn(0x7fbdb815bec0 legacy=0x7fbdb80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:51.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.526+0000 7fbdbe7f8640 1 -- 172.21.15.76:0/4207683528 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb80b3b30 con 0x7fbdb80af2a0 2024-09-19T04:17:51.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdbe7f8640 1 --2- 172.21.15.76:0/4207683528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb80af2a0 0x7fbdb806e000 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ae510 tx=0x7fbdb001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60761dd65396a6f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:51.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdbcff5640 1 -- 172.21.15.76:0/4207683528 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00a5840 con 0x7fbdb80af2a0 2024-09-19T04:17:51.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdbcff5640 1 -- 172.21.15.76:0/4207683528 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdb00a59e0 con 0x7fbdb80af2a0 2024-09-19T04:17:51.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdbcff5640 1 -- 172.21.15.76:0/4207683528 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb00a5db0 con 0x7fbdb80af2a0 2024-09-19T04:17:51.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 -- 172.21.15.76:0/4207683528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb80af2a0 msgr2=0x7fbdb806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 --2- 172.21.15.76:0/4207683528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb80af2a0 0x7fbdb806e000 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80ae510 tx=0x7fbdb001c980 comp rx=0 tx=0).stop 2024-09-19T04:17:51.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 -- 172.21.15.76:0/4207683528 shutdown_connections 2024-09-19T04:17:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 --2- 172.21.15.76:0/4207683528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb80af2a0 0x7fbdb806e000 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 --2- 172.21.15.76:0/4207683528 >> v2:172.21.15.116:3300/0 conn(0x7fbdb815c2a0 0x7fbdb80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 -- 172.21.15.76:0/4207683528 >> 172.21.15.76:0/4207683528 conn(0x7fbdb80b1f90 msgr2=0x7fbdb80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 -- 172.21.15.76:0/4207683528 shutdown_connections 2024-09-19T04:17:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 -- 172.21.15.76:0/4207683528 wait complete. 2024-09-19T04:17:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 Processor -- start 2024-09-19T04:17:51.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.530+0000 7fbdc025b640 1 -- start start 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdb80af2a0 0x7fbdb815d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb815bec0 0x7fbdb8155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbdb815c2a0 0x7fbdb814a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb806ed50 con 0x7fbdb815bec0 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbdb80af710 con 0x7fbdb80af2a0 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbdb806eed0 con 0x7fbdb815c2a0 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb815bec0 0x7fbdb8155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb815bec0 0x7fbdb8155f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39594/0 (socket says 172.21.15.76:39594) 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 -- 172.21.15.76:0/1197255577 learned_addr learned my addr 172.21.15.76:0/1197255577 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:51.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbdff7640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdb80af2a0 0x7fbdb815d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbe7f8640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbdb815c2a0 0x7fbdb814a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 -- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbdb815c2a0 msgr2=0x7fbdb814a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbdb815c2a0 0x7fbdb814a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 -- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdb80af2a0 msgr2=0x7fbdb815d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdb80af2a0 0x7fbdb815d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 -- 172.21.15.76:0/1197255577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdb8156790 con 0x7fbdb815bec0 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbd7f6640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb815bec0 0x7fbdb8155f40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c5e0 tx=0x7fbdac00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdaa7fc640 1 -- 172.21.15.76:0/1197255577 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac015070 con 0x7fbdb815bec0 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdaa7fc640 1 -- 172.21.15.76:0/1197255577 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdac004220 con 0x7fbdb815bec0 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdb8156a70 con 0x7fbdb815bec0 2024-09-19T04:17:51.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdaa7fc640 1 -- 172.21.15.76:0/1197255577 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdac003620 con 0x7fbdb815bec0 2024-09-19T04:17:51.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdb8157030 con 0x7fbdb815bec0 2024-09-19T04:17:51.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdb806f260 con 0x7fbdb815bec0 2024-09-19T04:17:51.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdaa7fc640 1 -- 172.21.15.76:0/1197255577 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbdac0037e0 con 0x7fbdb815bec0 2024-09-19T04:17:51.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdaa7fc640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd9c0678c0 0x7fbd9c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:51.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbdff7640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd9c0678c0 0x7fbd9c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:51.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdaa7fc640 1 -- 172.21.15.76:0/1197255577 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6504+0+0 (secure 0 0 0) 0x7fbdac085e50 con 0x7fbdb815bec0 2024-09-19T04:17:51.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.534+0000 7fbdbdff7640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd9c0678c0 0x7fbd9c069d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4001600 tx=0x7fbdb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:51.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.538+0000 7fbdaa7fc640 1 -- 172.21.15.76:0/1197255577 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbdac0565f0 con 0x7fbdb815bec0 2024-09-19T04:17:51.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:51.702+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbdb806fcd0 con 0x7fbdb815bec0 2024-09-19T04:17:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.338+0000 7fbdaa7fc640 1 -- 172.21.15.76:0/1197255577 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v71) ==== 181+0+0 (secure 0 0 0) 0x7fbdac055ef0 con 0x7fbdb815bec0 2024-09-19T04:17:52.344 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:17:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd9c0678c0 msgr2=0x7fbd9c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd9c0678c0 0x7fbd9c069d80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4001600 tx=0x7fbdb4009040 comp rx=0 tx=0).stop 2024-09-19T04:17:52.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb815bec0 msgr2=0x7fbdb8155f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:52.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb815bec0 0x7fbdb8155f40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c5e0 tx=0x7fbdac00caa0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 shutdown_connections 2024-09-19T04:17:52.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd9c0678c0 0x7fbd9c069d80 secure :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4001600 tx=0x7fbdb4009040 comp rx=0 tx=0).stop 2024-09-19T04:17:52.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbdb815c2a0 0x7fbdb814a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbdb815bec0 0x7fbdb8155f40 secure :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbdac00c5e0 tx=0x7fbdac00caa0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 --2- 172.21.15.76:0/1197255577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdb80af2a0 0x7fbdb815d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.342+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 >> 172.21.15.76:0/1197255577 conn(0x7fbdb80b1f90 msgr2=0x7fbdb80b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:52.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.346+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 shutdown_connections 2024-09-19T04:17:52.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.346+0000 7fbdc025b640 1 -- 172.21.15.76:0/1197255577 wait complete. 2024-09-19T04:17:52.370 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:17:52.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77ed316640 1 Processor -- start 2024-09-19T04:17:52.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77ed316640 1 -- start start 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77ed316640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f77e815bec0 0x7f77e80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77ed316640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77ed316640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77e80b3290 con 0x7f77e815c2a0 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77ed316640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f77e80b3570 con 0x7f77e815bec0 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77ed316640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80b3850 con 0x7f77e80af470 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f77e815bec0 0x7f77e80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e67fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f77e815c2a0 0x7f77e80aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:57968/0 (socket says 172.21.15.76:57968) 2024-09-19T04:17:52.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e67fc640 1 -- 172.21.15.76:0/2253568961 learned_addr learned my addr 172.21.15.76:0/2253568961 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:52.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e77fe640 1 --2- 172.21.15.76:0/2253568961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:52.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e6ffd640 1 -- 172.21.15.76:0/2253568961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 msgr2=0x7f77e806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:52.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e6ffd640 1 --2- 172.21.15.76:0/2253568961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e6ffd640 1 -- 172.21.15.76:0/2253568961 >> v1:172.21.15.76:6789/0 conn(0x7f77e815c2a0 legacy=0x7f77e80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:52.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e6ffd640 1 -- 172.21.15.76:0/2253568961 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e80b3b30 con 0x7f77e815bec0 2024-09-19T04:17:52.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.498+0000 7f77e5ffb640 1 -- 172.21.15.76:0/2253568961 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 504253811 0 0) 0x7f77e80b3290 con 0x7f77e815c2a0 2024-09-19T04:17:52.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77e6ffd640 1 --2- 172.21.15.76:0/2253568961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e80ad200 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097c0 tx=0x7f77d001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d28c2f76997f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:52.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77e5ffb640 1 -- 172.21.15.76:0/2253568961 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00b1070 con 0x7f77e815bec0 2024-09-19T04:17:52.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77e5ffb640 1 -- 172.21.15.76:0/2253568961 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d00a8050 con 0x7f77e815bec0 2024-09-19T04:17:52.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77e5ffb640 1 -- 172.21.15.76:0/2253568961 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d00ac860 con 0x7f77e815bec0 2024-09-19T04:17:52.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 -- 172.21.15.76:0/2253568961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 msgr2=0x7f77e80ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:52.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 --2- 172.21.15.76:0/2253568961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e80ad200 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f77d00097c0 tx=0x7f77d001c3e0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 -- 172.21.15.76:0/2253568961 shutdown_connections 2024-09-19T04:17:52.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 --2- 172.21.15.76:0/2253568961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 --2- 172.21.15.76:0/2253568961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e80ad200 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 -- 172.21.15.76:0/2253568961 >> 172.21.15.76:0/2253568961 conn(0x7f77e80b1f90 msgr2=0x7f77e80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:52.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 -- 172.21.15.76:0/2253568961 shutdown_connections 2024-09-19T04:17:52.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 -- 172.21.15.76:0/2253568961 wait complete. 2024-09-19T04:17:52.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 Processor -- start 2024-09-19T04:17:52.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.502+0000 7f77ed316640 1 -- start start 2024-09-19T04:17:52.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e814b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e8155360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77e815c2a0 0x7f77e81571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f77e806ed50 con 0x7f77e815bec0 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80ad740 con 0x7f77e815c2a0 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f77e806eed0 con 0x7f77e80af470 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e8155360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e8155360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:39622/0 (socket says 172.21.15.76:39622) 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 -- 172.21.15.76:0/3336057283 learned_addr learned my addr 172.21.15.76:0/3336057283 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e6ffd640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e814b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e77fe640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77e815c2a0 0x7f77e81571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 -- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 msgr2=0x7f77e814b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:52.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e814b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 -- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77e815c2a0 msgr2=0x7f77e81571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77e815c2a0 0x7f77e81571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 -- 172.21.15.76:0/3336057283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e81558a0 con 0x7f77e815bec0 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e6ffd640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e814b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e77fe640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77e815c2a0 0x7f77e81571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e67fc640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e8155360 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f77d800c450 tx=0x7f77d800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77df7fe640 1 -- 172.21.15.76:0/3336057283 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d8018070 con 0x7f77e815bec0 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77df7fe640 1 -- 172.21.15.76:0/3336057283 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77d80074e0 con 0x7f77e815bec0 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77df7fe640 1 -- 172.21.15.76:0/3336057283 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d80049f0 con 0x7f77e815bec0 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e8155b60 con 0x7f77e815bec0 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e814cf10 con 0x7f77e815bec0 2024-09-19T04:17:52.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e806e000 con 0x7f77e815bec0 2024-09-19T04:17:52.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77df7fe640 1 -- 172.21.15.76:0/3336057283 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f77d801b070 con 0x7f77e815bec0 2024-09-19T04:17:52.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77df7fe640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77c80678c0 0x7f77c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:52.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77df7fe640 1 -- 172.21.15.76:0/3336057283 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 6037+0+0 (secure 0 0 0) 0x7f77d8085180 con 0x7f77e815bec0 2024-09-19T04:17:52.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e6ffd640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77c80678c0 0x7f77c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:52.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.506+0000 7f77e6ffd640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77c80678c0 0x7f77c8069d80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77d00ae000 tx=0x7f77d00a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:52.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.510+0000 7f77df7fe640 1 -- 172.21.15.76:0/3336057283 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f77d8055b70 con 0x7f77e815bec0 2024-09-19T04:17:52.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:52.674+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f77e806fcd0 con 0x7f77e815bec0 2024-09-19T04:17:53.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.342+0000 7f77df7fe640 1 -- 172.21.15.76:0/3336057283 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v72) ==== 169+0+0 (secure 0 0 0) 0x7f77d8055470 con 0x7f77e815bec0 2024-09-19T04:17:53.347 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77c80678c0 msgr2=0x7f77c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77c80678c0 0x7f77c8069d80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77d00ae000 tx=0x7f77d00a7000 comp rx=0 tx=0).stop 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 msgr2=0x7f77e8155360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e8155360 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f77d800c450 tx=0x7f77d800c910 comp rx=0 tx=0).stop 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 shutdown_connections 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f77c80678c0 0x7f77c8069d80 secure :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f77d00ae000 tx=0x7f77d00a7000 comp rx=0 tx=0).stop 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77e815c2a0 0x7f77e81571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77e815bec0 0x7f77e8155360 secure :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f77d800c450 tx=0x7f77d800c910 comp rx=0 tx=0).stop 2024-09-19T04:17:53.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 --2- 172.21.15.76:0/3336057283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77e80af470 0x7f77e814b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 >> 172.21.15.76:0/3336057283 conn(0x7f77e80b1f90 msgr2=0x7f77e8159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:53.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 shutdown_connections 2024-09-19T04:17:53.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.346+0000 7f77ed316640 1 -- 172.21.15.76:0/3336057283 wait complete. 2024-09-19T04:17:53.376 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T04:17:53.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38dc452640 1 Processor -- start 2024-09-19T04:17:53.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38dc452640 1 -- start start 2024-09-19T04:17:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38dc452640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f38d4161a60 0x7f38d40d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38dc452640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d40d32b0 con 0x7f38d40cf620 2024-09-19T04:17:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38dc452640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d40d0300 con 0x7f38d40d2e90 2024-09-19T04:17:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38dc452640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f38d40cfa90 con 0x7f38d4161a60 2024-09-19T04:17:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38da9ef640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f38d40cf620 0x7f38d406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38392/0 (socket says 172.21.15.76:38392) 2024-09-19T04:17:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38da9ef640 1 -- 172.21.15.76:0/2872116330 learned_addr learned my addr 172.21.15.76:0/2872116330 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:53.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.498+0000 7f38da1ee640 1 --2- 172.21.15.76:0/2872116330 >> v2:172.21.15.116:3300/0 conn(0x7f38d4161a60 0x7f38d40d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2436421684 0 0) 0x7f38d40d32b0 con 0x7f38d40cf620 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38c0003530 con 0x7f38d40cf620 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 860734360 0 0) 0x7f38c0003530 con 0x7f38d40cf620 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 >> v2:172.21.15.116:3300/0 conn(0x7f38d4161a60 msgr2=0x7f38d40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 --2- 172.21.15.76:0/2872116330 >> v2:172.21.15.116:3300/0 conn(0x7f38d4161a60 0x7f38d40d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 >> v1:172.21.15.116:6789/0 conn(0x7f38d40d2e90 legacy=0x7f38d40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d40b0f60 con 0x7f38d40cf620 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f38cc001680 con 0x7f38d40cf620 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 >> v1:172.21.15.76:6789/0 conn(0x7f38d40cf620 legacy=0x7f38d406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38c0003a10 0x7f38c0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38c0004810 0x7f38c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38c000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f38d40cfa90 con 0x7f38c0005220 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38c000c170 con 0x7f38c0004810 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f38c000c370 con 0x7f38c0003a10 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38c000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d99ed640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38c0004810 0x7f38c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da1ee640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38c0003a10 0x7f38c0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 -- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38c0003a10 msgr2=0x7f38c0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38c0003a10 0x7f38c0003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 -- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38c0004810 msgr2=0x7f38c0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38c0004810 0x7f38c0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 -- 172.21.15.76:0/2872116330 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c0003530 con 0x7f38c0005220 2024-09-19T04:17:53.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d99ed640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38c0004810 0x7f38c0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 -- 172.21.15.76:0/2872116330 reap_dead start 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38da9ef640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38c000b9d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c160 tx=0x7f38cc01f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3371fb493289f7d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc007c70 con 0x7f38c0005220 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38cc005ae0 con 0x7f38c0005220 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38d91ec640 1 -- 172.21.15.76:0/2872116330 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc007070 con 0x7f38c0005220 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38dc452640 1 -- 172.21.15.76:0/2872116330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 msgr2=0x7f38c000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38dc452640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38c000b9d0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c160 tx=0x7f38cc01f310 comp rx=0 tx=0).stop 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38dc452640 1 -- 172.21.15.76:0/2872116330 shutdown_connections 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38dc452640 1 --2- 172.21.15.76:0/2872116330 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38c000b9d0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38dc452640 1 -- 172.21.15.76:0/2872116330 >> 172.21.15.76:0/2872116330 conn(0x7f38d408bf90 msgr2=0x7f38d409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:53.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38dc452640 1 -- 172.21.15.76:0/2872116330 shutdown_connections 2024-09-19T04:17:53.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.502+0000 7f38dc452640 1 -- 172.21.15.76:0/2872116330 wait complete. 2024-09-19T04:17:53.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 Processor -- start 2024-09-19T04:17:53.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 -- start start 2024-09-19T04:17:53.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38d4151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d4151b50 0x7f38d40b8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d40b9110 0x7f38d415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f38d406d190 con 0x7f38c0005220 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d406ceb0 con 0x7f38d40b9110 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f38d40cfc70 con 0x7f38d4151b50 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38d4151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38d4151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51914/0 (socket says 172.21.15.76:51914) 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 -- 172.21.15.76:0/2958447450 learned_addr learned my addr 172.21.15.76:0/2958447450 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da9ef640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d40b9110 0x7f38d415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38d99ed640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d4151b50 0x7f38d40b8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 -- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d4151b50 msgr2=0x7f38d40b8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d4151b50 0x7f38d40b8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 -- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d40b9110 msgr2=0x7f38d415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d40b9110 0x7f38d415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 -- 172.21.15.76:0/2958447450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d40cb740 con 0x7f38c0005220 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38d99ed640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d4151b50 0x7f38d40b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38da1ee640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38d4151610 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f38c8010890 tx=0x7f38c8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38c6ffd640 1 -- 172.21.15.76:0/2958447450 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c800f040 con 0x7f38c0005220 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38c6ffd640 1 -- 172.21.15.76:0/2958447450 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c800eb60 con 0x7f38c0005220 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38c6ffd640 1 -- 172.21.15.76:0/2958447450 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c801c400 con 0x7f38c0005220 2024-09-19T04:17:53.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d40b97e0 con 0x7f38c0005220 2024-09-19T04:17:53.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.506+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d40b9da0 con 0x7f38c0005220 2024-09-19T04:17:53.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.510+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38d40b1260 con 0x7f38c0005220 2024-09-19T04:17:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.510+0000 7f38c6ffd640 1 -- 172.21.15.76:0/2958447450 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38c801c5a0 con 0x7f38c0005220 2024-09-19T04:17:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.510+0000 7f38c6ffd640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b80678f0 0x7f38b8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.510+0000 7f38c6ffd640 1 -- 172.21.15.76:0/2958447450 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f38c80862e0 con 0x7f38c0005220 2024-09-19T04:17:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.510+0000 7f38d99ed640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b80678f0 0x7f38b8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.510+0000 7f38d99ed640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b80678f0 0x7f38b8069db0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38d0007fd0 tx=0x7f38d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:53.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.514+0000 7f38c6ffd640 1 -- 172.21.15.76:0/2958447450 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f38d40cb740 con 0x7f38c0005220 2024-09-19T04:17:53.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.678+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f38d40d1790 con 0x7f38c0005220 2024-09-19T04:17:53.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38c6ffd640 1 -- 172.21.15.76:0/2958447450 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f38c8018030 con 0x7f38c0005220 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b80678f0 msgr2=0x7f38b8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b80678f0 0x7f38b8069db0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f38d0007fd0 tx=0x7f38d0009040 comp rx=0 tx=0).stop 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 msgr2=0x7f38d4151610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38d4151610 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f38c8010890 tx=0x7f38c8010d50 comp rx=0 tx=0).stop 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 shutdown_connections 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b80678f0 0x7f38b8069db0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d40b9110 0x7f38d415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d4151b50 0x7f38d40b8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 --2- 172.21.15.76:0/2958447450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38c0005220 0x7f38d4151610 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 >> 172.21.15.76:0/2958447450 conn(0x7f38d408bf90 msgr2=0x7f38d40d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 shutdown_connections 2024-09-19T04:17:53.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.682+0000 7f38dc452640 1 -- 172.21.15.76:0/2958447450 wait complete. 2024-09-19T04:17:53.697 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:17:53.699 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:17:53.706 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace' 2024-09-19T04:17:53.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 Processor -- start 2024-09-19T04:17:53.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 -- start start 2024-09-19T04:17:53.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f706415e5f0 0x7f706406d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70640ce9d0 0x7f70640cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f70640cf920 0x7f70640d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f706406d9c0 con 0x7f70640ce9d0 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f706406dca0 con 0x7f70640cf920 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f706a7a5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f706406df80 con 0x7f706415e5f0 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70640ce9d0 0x7f70640cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f70640ce9d0 0x7f70640cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51920/0 (socket says 172.21.15.76:51920) 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 -- 172.21.15.76:0/1259016892 learned_addr learned my addr 172.21.15.76:0/1259016892 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f7068d42640 1 --2- 172.21.15.76:0/1259016892 >> v2:172.21.15.116:3300/0 conn(0x7f70640cf920 0x7f70640d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f7063fff640 1 --2- 172.21.15.76:0/1259016892 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f706415e5f0 0x7f706406d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 -- 172.21.15.76:0/1259016892 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f706415e5f0 msgr2=0x7f706406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 --2- 172.21.15.76:0/1259016892 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f706415e5f0 0x7f706406d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 -- 172.21.15.76:0/1259016892 >> v2:172.21.15.116:3300/0 conn(0x7f70640cf920 msgr2=0x7f70640d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:53.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 --2- 172.21.15.76:0/1259016892 >> v2:172.21.15.116:3300/0 conn(0x7f70640cf920 0x7f70640d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f705bfff640 1 -- 172.21.15.76:0/1259016892 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f706406e260 con 0x7f70640ce9d0 2024-09-19T04:17:53.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.870+0000 7f7068d42640 1 --2- 172.21.15.76:0/1259016892 >> v2:172.21.15.116:3300/0 conn(0x7f70640cf920 0x7f70640d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:53.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f705bfff640 1 --2- 172.21.15.76:0/1259016892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70640ce9d0 0x7f70640cf3e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f706408d860 tx=0x7f704c01c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4534a593f5afe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:53.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f70637fe640 1 -- 172.21.15.76:0/1259016892 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0b0070 con 0x7f70640ce9d0 2024-09-19T04:17:53.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f70637fe640 1 -- 172.21.15.76:0/1259016892 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f704c0ab600 con 0x7f70640ce9d0 2024-09-19T04:17:53.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f70637fe640 1 -- 172.21.15.76:0/1259016892 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f704c0ab9d0 con 0x7f70640ce9d0 2024-09-19T04:17:53.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 -- 172.21.15.76:0/1259016892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70640ce9d0 msgr2=0x7f70640cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 --2- 172.21.15.76:0/1259016892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70640ce9d0 0x7f70640cf3e0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f706408d860 tx=0x7f704c01c260 comp rx=0 tx=0).stop 2024-09-19T04:17:53.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 -- 172.21.15.76:0/1259016892 shutdown_connections 2024-09-19T04:17:53.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 --2- 172.21.15.76:0/1259016892 >> v2:172.21.15.116:3300/0 conn(0x7f70640cf920 0x7f70640d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 --2- 172.21.15.76:0/1259016892 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70640ce9d0 0x7f70640cf3e0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 --2- 172.21.15.76:0/1259016892 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f706415e5f0 0x7f706406d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 -- 172.21.15.76:0/1259016892 >> 172.21.15.76:0/1259016892 conn(0x7f7064074030 msgr2=0x7f706409d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:53.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 -- 172.21.15.76:0/1259016892 shutdown_connections 2024-09-19T04:17:53.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 -- 172.21.15.76:0/1259016892 wait complete. 2024-09-19T04:17:53.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 Processor -- start 2024-09-19T04:17:53.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.874+0000 7f706a7a5640 1 -- start start 2024-09-19T04:17:53.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70640cf920 0x7f706415afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70640b9a80 0x7f706414e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7064156550 0x7f70641569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70640d3700 con 0x7f7064156550 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70640d3580 con 0x7f70640cf920 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f70640d3880 con 0x7f70640b9a80 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7064156550 0x7f70641569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7064156550 0x7f70641569d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51924/0 (socket says 172.21.15.76:51924) 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 -- 172.21.15.76:0/3806621213 learned_addr learned my addr 172.21.15.76:0/3806621213 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7063fff640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70640cf920 0x7f706415afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f705bfff640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70640b9a80 0x7f706414e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 -- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70640b9a80 msgr2=0x7f706414e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70640b9a80 0x7f706414e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 -- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70640cf920 msgr2=0x7f706415afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70640cf920 0x7f706415afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 -- 172.21.15.76:0/3806621213 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70640b8ea0 con 0x7f7064156550 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f705bfff640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70640b9a80 0x7f706414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7068d42640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7064156550 0x7f70641569d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f705c007d60 tx=0x7f705c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f70617fa640 1 -- 172.21.15.76:0/3806621213 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c01a070 con 0x7f7064156550 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f70617fa640 1 -- 172.21.15.76:0/3806621213 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f705c0039a0 con 0x7f7064156550 2024-09-19T04:17:53.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70640b9180 con 0x7f7064156550 2024-09-19T04:17:53.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f70617fa640 1 -- 172.21.15.76:0/3806621213 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c003d90 con 0x7f7064156550 2024-09-19T04:17:53.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f706414ea30 con 0x7f7064156550 2024-09-19T04:17:53.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70640d3c90 con 0x7f7064156550 2024-09-19T04:17:53.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f70617fa640 1 -- 172.21.15.76:0/3806621213 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f705c01d070 con 0x7f7064156550 2024-09-19T04:17:53.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f70617fa640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f70440678f0 0x7f7044069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:53.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f70617fa640 1 -- 172.21.15.76:0/3806621213 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f705c085d40 con 0x7f7064156550 2024-09-19T04:17:53.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7063fff640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f70440678f0 0x7f7044069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:53.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.878+0000 7f7063fff640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f70440678f0 0x7f7044069db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f705400b930 tx=0x7f7054008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:53.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:53.882+0000 7f70617fa640 1 -- 172.21.15.76:0/3806621213 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f705c010070 con 0x7f7064156550 2024-09-19T04:17:54.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.106+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]} v 0) -- 0x7f7064158840 con 0x7f7064156550 2024-09-19T04:17:54.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.346+0000 7f70617fa640 1 -- 172.21.15.76:0/3806621213 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_backtrace"]}]=0 v229) ==== 142+0+0 (secure 0 0 0) 0x7f705c056940 con 0x7f7064156550 2024-09-19T04:17:54.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f70440678f0 msgr2=0x7f7044069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f70440678f0 0x7f7044069db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f705400b930 tx=0x7f7054008040 comp rx=0 tx=0).stop 2024-09-19T04:17:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7064156550 msgr2=0x7f70641569d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7064156550 0x7f70641569d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f705c007d60 tx=0x7f705c00d4a0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 shutdown_connections 2024-09-19T04:17:54.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f70440678f0 0x7f7044069db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7064156550 0x7f70641569d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70640b9a80 0x7f706414e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 --2- 172.21.15.76:0/3806621213 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70640cf920 0x7f706415afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 >> 172.21.15.76:0/3806621213 conn(0x7f70640ce9d0 msgr2=0x7f70640d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 shutdown_connections 2024-09-19T04:17:54.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.350+0000 7f706a7a5640 1 -- 172.21.15.76:0/3806621213 wait complete. 2024-09-19T04:17:54.378 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-19T04:17:54.379 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-19T04:17:54.379 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:17:54.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5551f46640 1 Processor -- start 2024-09-19T04:17:54.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5551f46640 1 -- start start 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5551f46640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f554c15bec0 0x7f554c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5551f46640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5551f46640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f554c0b3290 con 0x7f554c0af2a0 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5551f46640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f554c0b3570 con 0x7f554c15bec0 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5551f46640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f554c0b3850 con 0x7f554c15c2a0 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f554c15bec0 0x7f554c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f554c15bec0 0x7f554c0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51934/0 (socket says 172.21.15.76:51934) 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 -- 172.21.15.76:0/2036952565 learned_addr learned my addr 172.21.15.76:0/2036952565 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f5543fff640 1 --2- 172.21.15.76:0/2036952565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 -- 172.21.15.76:0/2036952565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 msgr2=0x7f554c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:54.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 --2- 172.21.15.76:0/2036952565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 -- 172.21.15.76:0/2036952565 >> v1:172.21.15.116:6789/0 conn(0x7f554c0af2a0 legacy=0x7f554c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:54.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 -- 172.21.15.76:0/2036952565 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c0b3b30 con 0x7f554c15bec0 2024-09-19T04:17:54.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.498+0000 7f554b7fe640 1 --2- 172.21.15.76:0/2036952565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c15bec0 0x7f554c0ad200 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f553c009870 tx=0x7f553c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9ab55c59898181 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:54.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554affd640 1 -- 172.21.15.76:0/2036952565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0aa610 con 0x7f554c15bec0 2024-09-19T04:17:54.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554affd640 1 -- 172.21.15.76:0/2036952565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c0af070 con 0x7f554c15bec0 2024-09-19T04:17:54.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554affd640 1 -- 172.21.15.76:0/2036952565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0aa9d0 con 0x7f554c15bec0 2024-09-19T04:17:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- 172.21.15.76:0/2036952565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c15bec0 msgr2=0x7f554c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 --2- 172.21.15.76:0/2036952565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c15bec0 0x7f554c0ad200 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f553c009870 tx=0x7f553c01c4f0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- 172.21.15.76:0/2036952565 shutdown_connections 2024-09-19T04:17:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 --2- 172.21.15.76:0/2036952565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 --2- 172.21.15.76:0/2036952565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c15bec0 0x7f554c0ad200 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- 172.21.15.76:0/2036952565 >> 172.21.15.76:0/2036952565 conn(0x7f554c0b1f90 msgr2=0x7f554c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:54.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- 172.21.15.76:0/2036952565 shutdown_connections 2024-09-19T04:17:54.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- 172.21.15.76:0/2036952565 wait complete. 2024-09-19T04:17:54.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 Processor -- start 2024-09-19T04:17:54.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- start start 2024-09-19T04:17:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0af2a0 0x7f554c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f554c15bec0 0x7f554c09cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c09d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0ad740 con 0x7f554c0af2a0 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f554c0afc90 con 0x7f554c15bec0 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5551f46640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f554c169000 con 0x7f554c15c2a0 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0af2a0 0x7f554c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0af2a0 0x7f554c1521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51942/0 (socket says 172.21.15.76:51942) 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 -- 172.21.15.76:0/3948673811 learned_addr learned my addr 172.21.15.76:0/3948673811 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5543fff640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f554c15bec0 0x7f554c09cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554bfff640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c09d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 -- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 msgr2=0x7f554c09d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c09d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 -- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f554c15bec0 msgr2=0x7f554c09cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f554c15bec0 0x7f554c09cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 -- 172.21.15.76:0/3948673811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f554c09d720 con 0x7f554c0af2a0 2024-09-19T04:17:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f5543fff640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f554c15bec0 0x7f554c09cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.502+0000 7f554b7fe640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0af2a0 0x7f554c1521e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f553c0ac040 tx=0x7f553c01ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5548ff9640 1 -- 172.21.15.76:0/3948673811 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0bd5f0 con 0x7f554c0af2a0 2024-09-19T04:17:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5548ff9640 1 -- 172.21.15.76:0/3948673811 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f553c0bd7b0 con 0x7f554c0af2a0 2024-09-19T04:17:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f554c09da00 con 0x7f554c0af2a0 2024-09-19T04:17:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5548ff9640 1 -- 172.21.15.76:0/3948673811 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553c0bdba0 con 0x7f554c0af2a0 2024-09-19T04:17:54.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f554c09dfc0 con 0x7f554c0af2a0 2024-09-19T04:17:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f554c0ad200 con 0x7f554c0af2a0 2024-09-19T04:17:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5548ff9640 1 -- 172.21.15.76:0/3948673811 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f553c0b33f0 con 0x7f554c0af2a0 2024-09-19T04:17:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5548ff9640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f552c0678c0 0x7f552c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5543fff640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f552c0678c0 0x7f552c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5548ff9640 1 -- 172.21.15.76:0/3948673811 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f553c120920 con 0x7f554c0af2a0 2024-09-19T04:17:54.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.506+0000 7f5543fff640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f552c0678c0 0x7f552c069d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f554c0cf820 tx=0x7f5534008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:54.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.510+0000 7f5548ff9640 1 -- 172.21.15.76:0/3948673811 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f553c0c3060 con 0x7f554c0af2a0 2024-09-19T04:17:54.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.670+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f554c0ad200 con 0x7f554c0af2a0 2024-09-19T04:17:54.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5548ff9640 1 -- 172.21.15.76:0/3948673811 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+13592 (secure 0 0 0) 0x7f553c0a7070 con 0x7f554c0af2a0 2024-09-19T04:17:54.678 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:54.678 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":72,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:17:53.344616+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6835/2740411101":"2024-09-20T04:17:50.326625+0000","172.21.15.76:6834/2740411101":"2024-09-20T04:17:50.326625+0000","172.21.15.76:6837/3052694229":"2024-09-20T04:17:50.326641+0000","172.21.15.76:6836/3052694229":"2024-09-20T04:17:50.326641+0000","172.21.15.116:6833/1054899754":"2024-09-20T04:17:50.326597+0000","172.21.15.116:6832/1054899754":"2024-09-20T04:17:50.326597+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:17:54.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f552c0678c0 msgr2=0x7f552c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f552c0678c0 0x7f552c069d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f554c0cf820 tx=0x7f5534008040 comp rx=0 tx=0).stop 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0af2a0 msgr2=0x7f554c1521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0af2a0 0x7f554c1521e0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f553c0ac040 tx=0x7f553c01ad30 comp rx=0 tx=0).stop 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 shutdown_connections 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f552c0678c0 0x7f552c069d80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f554c15c2a0 0x7f554c09d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f554c15bec0 0x7f554c09cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 --2- 172.21.15.76:0/3948673811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f554c0af2a0 0x7f554c1521e0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 >> 172.21.15.76:0/3948673811 conn(0x7f554c0b1f90 msgr2=0x7f554c159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 shutdown_connections 2024-09-19T04:17:54.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.674+0000 7f5551f46640 1 -- 172.21.15.76:0/3948673811 wait complete. 2024-09-19T04:17:54.698 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:13.812655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:17:54.698 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:17:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701ba02640 1 Processor -- start 2024-09-19T04:17:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701ba02640 1 -- start start 2024-09-19T04:17:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701ba02640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f701415c2a0 0x7f70140aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701ba02640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 0x7f701406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701ba02640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f70140b3290 con 0x7f701415bec0 2024-09-19T04:17:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701ba02640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f70140b3570 con 0x7f701415c2a0 2024-09-19T04:17:54.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701ba02640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f70140b3850 con 0x7f70140af2a0 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701979e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f701415bec0 0x7f70140ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38402/0 (socket says 172.21.15.76:38402) 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701979e640 1 -- 172.21.15.76:0/3306286703 learned_addr learned my addr 172.21.15.76:0/3306286703 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4274174091 0 0) 0x7f70140b3290 con 0x7f701415bec0 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7000003530 con 0x7f701415bec0 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7019f9f640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 0x7f701406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7018f9d640 1 --2- 172.21.15.76:0/3306286703 >> v2:172.21.15.116:3300/0 conn(0x7f701415c2a0 0x7f70140aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 869995625 0 0) 0x7f7000003530 con 0x7f701415bec0 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 msgr2=0x7f701406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 0x7f701406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 >> v2:172.21.15.116:3300/0 conn(0x7f701415c2a0 msgr2=0x7f70140aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 --2- 172.21.15.76:0/3306286703 >> v2:172.21.15.116:3300/0 conn(0x7f701415c2a0 0x7f70140aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70140b3b30 con 0x7f701415bec0 2024-09-19T04:17:54.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f7010001680 con 0x7f701415bec0 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 >> v1:172.21.15.76:6789/0 conn(0x7f701415bec0 legacy=0x7f70140ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 0x7f70000039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7000003ff0 0x7f7000004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7000004d50 0x7f70000051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b3570 con 0x7f7000004d50 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b3850 con 0x7f7000003ff0 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f700000bad0 con 0x7f70140af2a0 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7019f9f640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7000004d50 0x7f70000051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701979e640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 0x7f70000039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7018f9d640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7000003ff0 0x7f7000004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7019f9f640 1 -- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 msgr2=0x7f70000039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7019f9f640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 0x7f70000039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7019f9f640 1 -- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7000003ff0 msgr2=0x7f7000004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:54.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7019f9f640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7000003ff0 0x7f7000004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f7019f9f640 1 -- 172.21.15.76:0/3306286703 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7000003530 con 0x7f7000004d50 2024-09-19T04:17:54.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.806+0000 7f701979e640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f70140af2a0 0x7f70000039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:54.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f7019f9f640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7000004d50 0x7f70000051d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f700c007360 tx=0x7f700c01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79226feda368c33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:54.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0b2070 con 0x7f7000004d50 2024-09-19T04:17:54.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c0a5d40 con 0x7f7000004d50 2024-09-19T04:17:54.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f700bfff640 1 -- 172.21.15.76:0/3306286703 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c009be0 con 0x7f7000004d50 2024-09-19T04:17:54.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- 172.21.15.76:0/3306286703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7000004d50 msgr2=0x7f70000051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 --2- 172.21.15.76:0/3306286703 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7000004d50 0x7f70000051d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f700c007360 tx=0x7f700c01ce40 comp rx=0 tx=0).stop 2024-09-19T04:17:54.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f7019f9f640 1 -- 172.21.15.76:0/3306286703 reap_dead start 2024-09-19T04:17:54.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- 172.21.15.76:0/3306286703 shutdown_connections 2024-09-19T04:17:54.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- 172.21.15.76:0/3306286703 >> 172.21.15.76:0/3306286703 conn(0x7f70140b1f90 msgr2=0x7f70140b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:54.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- 172.21.15.76:0/3306286703 shutdown_connections 2024-09-19T04:17:54.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- 172.21.15.76:0/3306286703 wait complete. 2024-09-19T04:17:54.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 Processor -- start 2024-09-19T04:17:54.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- start start 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70141f7390 0x7f70140728f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7014072e30 0x7f70141e6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70141e6eb0 0x7f701415caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f70140af730 con 0x7f70141e6eb0 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f70140b40f0 con 0x7f70141f7390 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.810+0000 7f701ba02640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f70140afc90 con 0x7f7014072e30 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70141e6eb0 0x7f701415caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70141e6eb0 0x7f701415caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51968/0 (socket says 172.21.15.76:51968) 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 -- 172.21.15.76:0/2481179627 learned_addr learned my addr 172.21.15.76:0/2481179627 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:54.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f701979e640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70141f7390 0x7f70140728f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7018f9d640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7014072e30 0x7f70141e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 -- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7014072e30 msgr2=0x7f70141e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7014072e30 0x7f70141e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 -- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70141f7390 msgr2=0x7f70140728f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70141f7390 0x7f70140728f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 -- 172.21.15.76:0/2481179627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7014073370 con 0x7f70141e6eb0 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7018f9d640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7014072e30 0x7f70141e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f701979e640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70141f7390 0x7f70140728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7019f9f640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70141e6eb0 0x7f701415caa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f700c0b0040 tx=0x7f7000005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7009ffb640 1 -- 172.21.15.76:0/2481179627 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0bb070 con 0x7f70141e6eb0 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7009ffb640 1 -- 172.21.15.76:0/2481179627 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f700c0a5b40 con 0x7f70141e6eb0 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7014073650 con 0x7f70141e6eb0 2024-09-19T04:17:54.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7009ffb640 1 -- 172.21.15.76:0/2481179627 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f700c0b2040 con 0x7f70141e6eb0 2024-09-19T04:17:54.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7014073c10 con 0x7f70141e6eb0 2024-09-19T04:17:54.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70140aed60 con 0x7f70141e6eb0 2024-09-19T04:17:54.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7009ffb640 1 -- 172.21.15.76:0/2481179627 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f700c0be070 con 0x7f70141e6eb0 2024-09-19T04:17:54.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7009ffb640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ff80678f0 0x7f6ff8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:54.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f7009ffb640 1 -- 172.21.15.76:0/2481179627 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f700c11f6c0 con 0x7f70141e6eb0 2024-09-19T04:17:54.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f701979e640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ff80678f0 0x7f6ff8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:54.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.814+0000 7f701979e640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ff80678f0 0x7f6ff8069db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7010000c10 tx=0x7f7010001b40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:54.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.818+0000 7f7009ffb640 1 -- 172.21.15.76:0/2481179627 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f700c016b50 con 0x7f70141e6eb0 2024-09-19T04:17:54.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.970+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7014075780 con 0x7f70141e6eb0 2024-09-19T04:17:54.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.970+0000 7f7009ffb640 1 -- 172.21.15.76:0/2481179627 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f700c0efd00 con 0x7f70141e6eb0 2024-09-19T04:17:54.975 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-09-19T04:17:54.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.970+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ff80678f0 msgr2=0x7f6ff8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.970+0000 7f701ba02640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ff80678f0 0x7f6ff8069db0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7010000c10 tx=0x7f7010001b40 comp rx=0 tx=0).stop 2024-09-19T04:17:54.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.970+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70141e6eb0 msgr2=0x7f701415caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:54.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.970+0000 7f701ba02640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70141e6eb0 0x7f701415caa0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f700c0b0040 tx=0x7f7000005840 comp rx=0 tx=0).stop 2024-09-19T04:17:54.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 shutdown_connections 2024-09-19T04:17:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6ff80678f0 0x7f6ff8069db0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f70141e6eb0 0x7f701415caa0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7014072e30 0x7f70141e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 --2- 172.21.15.76:0/2481179627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f70141f7390 0x7f70140728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 >> 172.21.15.76:0/2481179627 conn(0x7f70140b1f90 msgr2=0x7f70140cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 shutdown_connections 2024-09-19T04:17:54.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:54.974+0000 7f701ba02640 1 -- 172.21.15.76:0/2481179627 wait complete. 2024-09-19T04:17:54.993 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-09-19T04:17:55.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3509434640 1 Processor -- start 2024-09-19T04:17:55.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3509434640 1 -- start start 2024-09-19T04:17:55.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3509434640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f35041520a0 0x7f350406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3509434640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f3a0 0x7f35040adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3509434640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3504070250 con 0x7f350406fe50 2024-09-19T04:17:55.115 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3509434640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3504152480 con 0x7f35041520a0 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3509434640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f35040aede0 con 0x7f350415f3a0 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3502ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f35041520a0 0x7f350406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f35027fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f350406fe50 0x7f350415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38412/0 (socket says 172.21.15.76:38412) 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f35027fc640 1 -- 172.21.15.76:0/2920878479 learned_addr learned my addr 172.21.15.76:0/2920878479 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3502ffd640 1 -- 172.21.15.76:0/2920878479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f3a0 msgr2=0x7f35040adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3502ffd640 1 --2- 172.21.15.76:0/2920878479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f3a0 0x7f35040adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3502ffd640 1 -- 172.21.15.76:0/2920878479 >> v1:172.21.15.76:6789/0 conn(0x7f350406fe50 legacy=0x7f350415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.110+0000 7f3502ffd640 1 -- 172.21.15.76:0/2920878479 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350415fa10 con 0x7f35041520a0 2024-09-19T04:17:55.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3502ffd640 1 --2- 172.21.15.76:0/2920878479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f350406f910 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00ba60 tx=0x7f34fc01eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16946b70f9e23a7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:55.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3501ffb640 1 -- 172.21.15.76:0/2920878479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0b4070 con 0x7f35041520a0 2024-09-19T04:17:55.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3501ffb640 1 -- 172.21.15.76:0/2920878479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc0a7b30 con 0x7f35041520a0 2024-09-19T04:17:55.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3501ffb640 1 -- 172.21.15.76:0/2920878479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc0af440 con 0x7f35041520a0 2024-09-19T04:17:55.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- 172.21.15.76:0/2920878479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 msgr2=0x7f350406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 --2- 172.21.15.76:0/2920878479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f350406f910 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00ba60 tx=0x7f34fc01eb90 comp rx=0 tx=0).stop 2024-09-19T04:17:55.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- 172.21.15.76:0/2920878479 shutdown_connections 2024-09-19T04:17:55.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 --2- 172.21.15.76:0/2920878479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f3a0 0x7f35040adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 --2- 172.21.15.76:0/2920878479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f350406f910 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- 172.21.15.76:0/2920878479 >> 172.21.15.76:0/2920878479 conn(0x7f35040b1f90 msgr2=0x7f35040b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:55.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- 172.21.15.76:0/2920878479 shutdown_connections 2024-09-19T04:17:55.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- 172.21.15.76:0/2920878479 wait complete. 2024-09-19T04:17:55.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 Processor -- start 2024-09-19T04:17:55.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- start start 2024-09-19T04:17:55.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350406fe50 0x7f350414dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f3504141e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350415f3a0 0x7f35040b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3504073d30 con 0x7f35041520a0 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3504073a50 con 0x7f350415f3a0 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.114+0000 7f3509434640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f350406e800 con 0x7f350406fe50 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f3504141e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f3504141e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51986/0 (socket says 172.21.15.76:51986) 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 -- 172.21.15.76:0/1993071716 learned_addr learned my addr 172.21.15.76:0/1993071716 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f3502ffd640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350406fe50 0x7f350414dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35037fe640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350415f3a0 0x7f35040b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 -- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350406fe50 msgr2=0x7f350414dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350406fe50 0x7f350414dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 -- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350415f3a0 msgr2=0x7f35040b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350415f3a0 0x7f35040b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 -- 172.21.15.76:0/1993071716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35040aff10 con 0x7f35041520a0 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f35027fc640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f3504141e70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c3a0 tx=0x7f34ec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f34fb7fe640 1 -- 172.21.15.76:0/1993071716 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec017070 con 0x7f35041520a0 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f34fb7fe640 1 -- 172.21.15.76:0/1993071716 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34ec005cf0 con 0x7f35041520a0 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35040b01f0 con 0x7f35041520a0 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f34fb7fe640 1 -- 172.21.15.76:0/1993071716 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34ec0072f0 con 0x7f35041520a0 2024-09-19T04:17:55.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35040b07b0 con 0x7f35041520a0 2024-09-19T04:17:55.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35040adc80 con 0x7f35041520a0 2024-09-19T04:17:55.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f34fb7fe640 1 -- 172.21.15.76:0/1993071716 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34ec01a070 con 0x7f35041520a0 2024-09-19T04:17:55.125 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f34fb7fe640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34e40678f0 0x7f34e4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:55.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f3502ffd640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34e40678f0 0x7f34e4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:55.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f34fb7fe640 1 -- 172.21.15.76:0/1993071716 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f34ec084ee0 con 0x7f35041520a0 2024-09-19T04:17:55.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.118+0000 7f3502ffd640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34e40678f0 0x7f34e4069db0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0b1000 tx=0x7f34fc0a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:55.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.122+0000 7f34fb7fe640 1 -- 172.21.15.76:0/1993071716 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f34ec055b70 con 0x7f35041520a0 2024-09-19T04:17:55.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:55.346+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7f350406e540 con 0x7f35041520a0 2024-09-19T04:17:56.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.358+0000 7f34fb7fe640 1 -- 172.21.15.76:0/1993071716 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v231) ==== 144+0+0 (secure 0 0 0) 0x7f34ec055470 con 0x7f35041520a0 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34e40678f0 msgr2=0x7f34e4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34e40678f0 0x7f34e4069db0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0b1000 tx=0x7f34fc0a9070 comp rx=0 tx=0).stop 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 msgr2=0x7f3504141e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f3504141e70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f34ec00c3a0 tx=0x7f34ec00c860 comp rx=0 tx=0).stop 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 shutdown_connections 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34e40678f0 0x7f34e4069db0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f350415f3a0 0x7f35040b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f35041520a0 0x7f3504141e70 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 --2- 172.21.15.76:0/1993071716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350406fe50 0x7f350414dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 >> 172.21.15.76:0/1993071716 conn(0x7f35040b1f90 msgr2=0x7f35040b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:56.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 shutdown_connections 2024-09-19T04:17:56.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.362+0000 7f3509434640 1 -- 172.21.15.76:0/1993071716 wait complete. 2024-09-19T04:17:56.383 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T04:17:56.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c681e2640 1 Processor -- start 2024-09-19T04:17:56.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c681e2640 1 -- start start 2024-09-19T04:17:56.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c681e2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c60070020 0x7f2c6015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:56.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c681e2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2c6015f1d0 0x7f2c600adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:56.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c681e2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c600aede0 con 0x7f2c601520a0 2024-09-19T04:17:56.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c681e2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2c60152480 con 0x7f2c6015f1d0 2024-09-19T04:17:56.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c681e2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2c6015fa10 con 0x7f2c60070020 2024-09-19T04:17:56.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c65f7e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2c601520a0 0x7f2c6006f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38418/0 (socket says 172.21.15.76:38418) 2024-09-19T04:17:56.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c65f7e640 1 -- 172.21.15.76:0/2869451991 learned_addr learned my addr 172.21.15.76:0/2869451991 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c6677f640 1 --2- 172.21.15.76:0/2869451991 >> v2:172.21.15.76:3300/0 conn(0x7f2c6015f1d0 0x7f2c600adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c6577d640 1 --2- 172.21.15.76:0/2869451991 >> v2:172.21.15.116:3300/0 conn(0x7f2c60070020 0x7f2c6015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c64f7c640 1 -- 172.21.15.76:0/2869451991 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006263192 0 0) 0x7f2c600aede0 con 0x7f2c601520a0 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c64f7c640 1 -- 172.21.15.76:0/2869451991 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2c48003530 con 0x7f2c601520a0 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c6677f640 1 -- 172.21.15.76:0/2869451991 >> v2:172.21.15.116:3300/0 conn(0x7f2c60070020 msgr2=0x7f2c6015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c6677f640 1 --2- 172.21.15.76:0/2869451991 >> v2:172.21.15.116:3300/0 conn(0x7f2c60070020 0x7f2c6015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c6677f640 1 -- 172.21.15.76:0/2869451991 >> v1:172.21.15.76:6789/0 conn(0x7f2c601520a0 legacy=0x7f2c6006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.510+0000 7f2c6677f640 1 -- 172.21.15.76:0/2869451991 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c60074250 con 0x7f2c6015f1d0 2024-09-19T04:17:56.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 --2- 172.21.15.76:0/2869451991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c600adc80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006e510 tx=0x7f2c5801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70415287e05a6b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:56.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c64f7c640 1 -- 172.21.15.76:0/2869451991 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580ab650 con 0x7f2c6015f1d0 2024-09-19T04:17:56.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c64f7c640 1 -- 172.21.15.76:0/2869451991 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c580b0070 con 0x7f2c6015f1d0 2024-09-19T04:17:56.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c64f7c640 1 -- 172.21.15.76:0/2869451991 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580aba10 con 0x7f2c6015f1d0 2024-09-19T04:17:56.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- 172.21.15.76:0/2869451991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 msgr2=0x7f2c600adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 --2- 172.21.15.76:0/2869451991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c600adc80 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f2c6006e510 tx=0x7f2c5801c450 comp rx=0 tx=0).stop 2024-09-19T04:17:56.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- 172.21.15.76:0/2869451991 shutdown_connections 2024-09-19T04:17:56.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 --2- 172.21.15.76:0/2869451991 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c600adc80 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 --2- 172.21.15.76:0/2869451991 >> v2:172.21.15.116:3300/0 conn(0x7f2c60070020 0x7f2c6015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- 172.21.15.76:0/2869451991 >> 172.21.15.76:0/2869451991 conn(0x7f2c600b1f90 msgr2=0x7f2c600b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:56.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- 172.21.15.76:0/2869451991 shutdown_connections 2024-09-19T04:17:56.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- 172.21.15.76:0/2869451991 wait complete. 2024-09-19T04:17:56.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 Processor -- start 2024-09-19T04:17:56.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- start start 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c60070020 0x7f2c60072460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c601520a0 0x7f2c60099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c60148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2c6015f680 con 0x7f2c6015f1d0 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2c60073d30 con 0x7f2c601520a0 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2c60167f10 con 0x7f2c60070020 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c60148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:56.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c60148790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52012/0 (socket says 172.21.15.76:52012) 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 -- 172.21.15.76:0/1918770913 learned_addr learned my addr 172.21.15.76:0/1918770913 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6577d640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c601520a0 0x7f2c60099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c65f7e640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c60070020 0x7f2c60072460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 -- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c60070020 msgr2=0x7f2c60072460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c60070020 0x7f2c60072460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 -- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c601520a0 msgr2=0x7f2c60099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c601520a0 0x7f2c60099940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 -- 172.21.15.76:0/1918770913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c60099e80 con 0x7f2c6015f1d0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c6677f640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c60148790 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2c58019060 tx=0x7f2c5801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c5e7fc640 1 -- 172.21.15.76:0/1918770913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580bf410 con 0x7f2c6015f1d0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c5e7fc640 1 -- 172.21.15.76:0/1918770913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c580bf5b0 con 0x7f2c6015f1d0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c6014a290 con 0x7f2c6015f1d0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c5e7fc640 1 -- 172.21.15.76:0/1918770913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c580bf980 con 0x7f2c6015f1d0 2024-09-19T04:17:56.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.514+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c6014c540 con 0x7f2c6015f1d0 2024-09-19T04:17:56.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.518+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c6015ec90 con 0x7f2c6015f1d0 2024-09-19T04:17:56.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.518+0000 7f2c5e7fc640 1 -- 172.21.15.76:0/1918770913 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2c580bd020 con 0x7f2c6015f1d0 2024-09-19T04:17:56.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.518+0000 7f2c5e7fc640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2c4c0678c0 0x7f2c4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.518+0000 7f2c5e7fc640 1 -- 172.21.15.76:0/1918770913 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f2c58120480 con 0x7f2c6015f1d0 2024-09-19T04:17:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.518+0000 7f2c65f7e640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2c4c0678c0 0x7f2c4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:56.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.518+0000 7f2c65f7e640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2c4c0678c0 0x7f2c4c069d80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005890 tx=0x7f2c54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:56.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.522+0000 7f2c5e7fc640 1 -- 172.21.15.76:0/1918770913 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2c58016070 con 0x7f2c6015f1d0 2024-09-19T04:17:56.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.678+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2c6006e540 con 0x7f2c6015f1d0 2024-09-19T04:17:56.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c5e7fc640 1 -- 172.21.15.76:0/1918770913 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7f2c580f1090 con 0x7f2c6015f1d0 2024-09-19T04:17:56.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2c4c0678c0 msgr2=0x7f2c4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2c4c0678c0 0x7f2c4c069d80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2c54005890 tx=0x7f2c54009040 comp rx=0 tx=0).stop 2024-09-19T04:17:56.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 msgr2=0x7f2c60148790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:56.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c60148790 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2c58019060 tx=0x7f2c5801ac90 comp rx=0 tx=0).stop 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 shutdown_connections 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2c4c0678c0 0x7f2c4c069d80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2c6015f1d0 0x7f2c60148790 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c601520a0 0x7f2c60099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 --2- 172.21.15.76:0/1918770913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2c60070020 0x7f2c60072460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 >> 172.21.15.76:0/1918770913 conn(0x7f2c600b1f90 msgr2=0x7f2c6006f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 shutdown_connections 2024-09-19T04:17:56.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:56.686+0000 7f2c681e2640 1 -- 172.21.15.76:0/1918770913 wait complete. 2024-09-19T04:17:56.703 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:17:56.703 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:17:56.779 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:17:56.835 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:17:56.835 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:17:56.835 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:56.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:17:56.841 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:17:56.841 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:56.882 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:17:56.882 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:17:56.882 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:56.889 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:17:56.890 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:17:56.890 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:17:56.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:17:56.931 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T04:17:56.931 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:17:57.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.050+0000 7f238c1ed640 1 Processor -- start 2024-09-19T04:17:57.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.050+0000 7f238c1ed640 1 -- start start 2024-09-19T04:17:57.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f238c1ed640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f23840ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f238c1ed640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f238415c2a0 0x7f23840aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f238c1ed640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23840b3290 con 0x7f23840af2a0 2024-09-19T04:17:57.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f238c1ed640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f23840b3570 con 0x7f238415c2a0 2024-09-19T04:17:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f238c1ed640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f23840b3850 con 0x7f238415bec0 2024-09-19T04:17:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f2389f89640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f23840ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f2389788640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f238415c2a0 0x7f23840aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f238a78a640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f23840af2a0 0x7f238415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:35984/0 (socket says 172.21.15.76:35984) 2024-09-19T04:17:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f238a78a640 1 -- 172.21.15.76:0/3863950047 learned_addr learned my addr 172.21.15.76:0/3863950047 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:57.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f2389f89640 1 -- 172.21.15.76:0/3863950047 >> v2:172.21.15.116:3300/0 conn(0x7f238415c2a0 msgr2=0x7f23840aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:57.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f2389f89640 1 --2- 172.21.15.76:0/3863950047 >> v2:172.21.15.116:3300/0 conn(0x7f238415c2a0 0x7f23840aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f2389f89640 1 -- 172.21.15.76:0/3863950047 >> v1:172.21.15.116:6789/0 conn(0x7f23840af2a0 legacy=0x7f238415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f2389f89640 1 -- 172.21.15.76:0/3863950047 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23840b3b30 con 0x7f238415bec0 2024-09-19T04:17:57.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.054+0000 7f2389788640 1 --2- 172.21.15.76:0/3863950047 >> v2:172.21.15.116:3300/0 conn(0x7f238415c2a0 0x7f23840aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:17:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 --2- 172.21.15.76:0/3863950047 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f23840ad200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23740097c0 tx=0x7f237401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f19d90357873fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2388f87640 1 -- 172.21.15.76:0/3863950047 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740a7070 con 0x7f238415bec0 2024-09-19T04:17:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2388f87640 1 -- 172.21.15.76:0/3863950047 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23740b0070 con 0x7f238415bec0 2024-09-19T04:17:57.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2388f87640 1 -- 172.21.15.76:0/3863950047 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740ab890 con 0x7f238415bec0 2024-09-19T04:17:57.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- 172.21.15.76:0/3863950047 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 msgr2=0x7f23840ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 --2- 172.21.15.76:0/3863950047 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f23840ad200 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23740097c0 tx=0x7f237401c470 comp rx=0 tx=0).stop 2024-09-19T04:17:57.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- 172.21.15.76:0/3863950047 shutdown_connections 2024-09-19T04:17:57.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 --2- 172.21.15.76:0/3863950047 >> v2:172.21.15.116:3300/0 conn(0x7f238415c2a0 0x7f23840aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 --2- 172.21.15.76:0/3863950047 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f23840ad200 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- 172.21.15.76:0/3863950047 >> 172.21.15.76:0/3863950047 conn(0x7f23840b1f90 msgr2=0x7f23840b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:57.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- 172.21.15.76:0/3863950047 shutdown_connections 2024-09-19T04:17:57.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- 172.21.15.76:0/3863950047 wait complete. 2024-09-19T04:17:57.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 Processor -- start 2024-09-19T04:17:57.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- start start 2024-09-19T04:17:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840af2a0 0x7f23841528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f2384158280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c2a0 0x7f23841587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f23840af6b0 con 0x7f23840af2a0 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f23840ad740 con 0x7f238415c2a0 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238c1ed640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2384169000 con 0x7f238415bec0 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840af2a0 0x7f23841528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840af2a0 0x7f23841528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52020/0 (socket says 172.21.15.76:52020) 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 -- 172.21.15.76:0/304266051 learned_addr learned my addr 172.21.15.76:0/304266051 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f238a78a640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c2a0 0x7f23841587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389788640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f2384158280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 -- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 msgr2=0x7f2384158280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f2384158280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 -- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c2a0 msgr2=0x7f23841587c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c2a0 0x7f23841587c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389f89640 1 -- 172.21.15.76:0/304266051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238415f960 con 0x7f23840af2a0 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.058+0000 7f2389788640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f2384158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f2389f89640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840af2a0 0x7f23841528e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f237401a0a0 tx=0x7f237401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f237a7fc640 1 -- 172.21.15.76:0/304266051 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740bf610 con 0x7f23840af2a0 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f237a7fc640 1 -- 172.21.15.76:0/304266051 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23740bf7d0 con 0x7f23840af2a0 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238415d120 con 0x7f23840af2a0 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f237a7fc640 1 -- 172.21.15.76:0/304266051 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740bfbc0 con 0x7f23840af2a0 2024-09-19T04:17:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2384072040 con 0x7f23840af2a0 2024-09-19T04:17:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23840ad200 con 0x7f23840af2a0 2024-09-19T04:17:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f237a7fc640 1 -- 172.21.15.76:0/304266051 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f23740bd020 con 0x7f23840af2a0 2024-09-19T04:17:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f237a7fc640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23680678c0 0x7f2368069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f237a7fc640 1 -- 172.21.15.76:0/304266051 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7f2374120760 con 0x7f23840af2a0 2024-09-19T04:17:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f2389788640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23680678c0 0x7f2368069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.062+0000 7f2389788640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23680678c0 0x7f2368069d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f23840b1f60 tx=0x7f2380008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.066+0000 7f237a7fc640 1 -- 172.21.15.76:0/304266051 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f23740c4060 con 0x7f23840af2a0 2024-09-19T04:17:57.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.254+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f238406fcd0 con 0x7f23840af2a0 2024-09-19T04:17:57.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f237a7fc640 1 -- 172.21.15.76:0/304266051 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) ==== 94+0+3911 (secure 0 0 0) 0x7f2374016090 con 0x7f23840af2a0 2024-09-19T04:17:57.257 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:57.258 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":60,"btime":"2024-09-19T04:17:51:330587+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3079277178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3079277178},{"type":"v1","addr":"172.21.15.116:6835","nonce":3079277178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":42},{"gid":5500,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/1148251531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":1148251531},{"type":"v1","addr":"172.21.15.76:6835","nonce":1148251531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":5509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/510289688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":510289688},{"type":"v1","addr":"172.21.15.116:6833","nonce":510289688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/904588834","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":904588834},{"type":"v1","addr":"172.21.15.76:6837","nonce":904588834}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":60}],"filesystems":[]} 2024-09-19T04:17:57.258 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 60 2024-09-19T04:17:57.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23680678c0 msgr2=0x7f2368069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23680678c0 0x7f2368069d80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f23840b1f60 tx=0x7f2380008040 comp rx=0 tx=0).stop 2024-09-19T04:17:57.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840af2a0 msgr2=0x7f23841528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840af2a0 0x7f23841528e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f237401a0a0 tx=0x7f237401acb0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 shutdown_connections 2024-09-19T04:17:57.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23680678c0 0x7f2368069d80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f238415c2a0 0x7f23841587c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f238415bec0 0x7f2384158280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 --2- 172.21.15.76:0/304266051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23840af2a0 0x7f23841528e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 >> 172.21.15.76:0/304266051 conn(0x7f23840b1f90 msgr2=0x7f23840b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:57.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 shutdown_connections 2024-09-19T04:17:57.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.258+0000 7f238c1ed640 1 -- 172.21.15.76:0/304266051 wait complete. 2024-09-19T04:17:57.276 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T04:17:57.276 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T04:17:57.276 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T04:17:57.277 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:17:57.277 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T04:17:57.277 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T04:17:57.277 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T04:17:57.277 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:17:57.277 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T04:17:57.277 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T04:17:57.280 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T04:17:57.280 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:17:57.280 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T04:17:57.280 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T04:17:57.281 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T04:17:57.281 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:17:57.287 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T04:17:57.287 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T04:17:57.288 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T04:17:57.288 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T04:17:57.293 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T04:17:57.294 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T04:17:57.297 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T04:17:57.297 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T04:17:57.334 INFO:tasks.ceph.mds.c:Started 2024-09-19T04:17:57.335 DEBUG:teuthology.parallel:result is None 2024-09-19T04:17:57.335 INFO:tasks.ceph.mds.a:Started 2024-09-19T04:17:57.336 DEBUG:teuthology.parallel:result is None 2024-09-19T04:17:57.338 INFO:tasks.ceph.mds.b:Started 2024-09-19T04:17:57.339 DEBUG:teuthology.parallel:result is None 2024-09-19T04:17:57.340 INFO:tasks.ceph.mds.d:Started 2024-09-19T04:17:57.340 DEBUG:teuthology.parallel:result is None 2024-09-19T04:17:57.340 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T04:17:57.395 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:17:57.398 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:17:57.410 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:17:57.411 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:17:57.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe75961c640 1 Processor -- start 2024-09-19T04:17:57.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe75961c640 1 -- start start 2024-09-19T04:17:57.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe75961c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe75406d3e0 0x7fe7540ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe75961c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7540cd520 con 0x7fe75415c0c0 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe75961c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7540cd800 con 0x7fe7540b5880 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe75961c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe7540cdae0 con 0x7fe75406d3e0 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe752ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe75415c0c0 0x7fe75415c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38440/0 (socket says 172.21.15.76:38440) 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe752ffd640 1 -- 172.21.15.76:0/3138382109 learned_addr learned my addr 172.21.15.76:0/3138382109 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe7537fe640 1 --2- 172.21.15.76:0/3138382109 >> v2:172.21.15.76:3300/0 conn(0x7fe75406d3e0 0x7fe7540ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe751ffb640 1 -- 172.21.15.76:0/3138382109 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2278145257 0 0) 0x7fe7540cd520 con 0x7fe75415c0c0 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe751ffb640 1 -- 172.21.15.76:0/3138382109 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe73c003530 con 0x7fe75415c0c0 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe7537fe640 1 -- 172.21.15.76:0/3138382109 >> v1:172.21.15.116:6789/0 conn(0x7fe7540b5880 legacy=0x7fe75406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe7537fe640 1 -- 172.21.15.76:0/3138382109 >> v1:172.21.15.76:6789/0 conn(0x7fe75415c0c0 legacy=0x7fe75415c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.458+0000 7fe7537fe640 1 -- 172.21.15.76:0/3138382109 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7540cddc0 con 0x7fe75406d3e0 2024-09-19T04:17:57.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 --2- 172.21.15.76:0/3138382109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75406d3e0 0x7fe7540ce5c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe7440097c0 tx=0x7fe74401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58f7ab8c749a070 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:57.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe751ffb640 1 -- 172.21.15.76:0/3138382109 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440ab650 con 0x7fe75406d3e0 2024-09-19T04:17:57.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe751ffb640 1 -- 172.21.15.76:0/3138382109 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7440b0070 con 0x7fe75406d3e0 2024-09-19T04:17:57.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe751ffb640 1 -- 172.21.15.76:0/3138382109 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440aba10 con 0x7fe75406d3e0 2024-09-19T04:17:57.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- 172.21.15.76:0/3138382109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75406d3e0 msgr2=0x7fe7540ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 --2- 172.21.15.76:0/3138382109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75406d3e0 0x7fe7540ce5c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fe7440097c0 tx=0x7fe74401c430 comp rx=0 tx=0).stop 2024-09-19T04:17:57.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- 172.21.15.76:0/3138382109 shutdown_connections 2024-09-19T04:17:57.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 --2- 172.21.15.76:0/3138382109 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75406d3e0 0x7fe7540ce5c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- 172.21.15.76:0/3138382109 >> 172.21.15.76:0/3138382109 conn(0x7fe754072670 msgr2=0x7fe754072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:57.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- 172.21.15.76:0/3138382109 shutdown_connections 2024-09-19T04:17:57.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- 172.21.15.76:0/3138382109 wait complete. 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 Processor -- start 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- start start 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe75406d3e0 0x7fe7540aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7540b5880 0x7fe75414a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75415c0c0 0x7fe754149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe754169f80 con 0x7fe75415c0c0 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe754169e00 con 0x7fe7540b5880 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe75961c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe75416a100 con 0x7fe75406d3e0 2024-09-19T04:17:57.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75415c0c0 0x7fe754149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75415c0c0 0x7fe754149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52070/0 (socket says 172.21.15.76:52070) 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 -- 172.21.15.76:0/915551053 learned_addr learned my addr 172.21.15.76:0/915551053 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7527fc640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7540b5880 0x7fe75414a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe752ffd640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe75406d3e0 0x7fe7540aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 -- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe75406d3e0 msgr2=0x7fe7540aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe75406d3e0 0x7fe7540aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 -- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7540b5880 msgr2=0x7fe75414a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7540b5880 0x7fe75414a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7537fe640 1 -- 172.21.15.76:0/915551053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe75406fd60 con 0x7fe75415c0c0 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe7527fc640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7540b5880 0x7fe75414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.462+0000 7fe752ffd640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe75406d3e0 0x7fe7540aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:57.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe7537fe640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75415c0c0 0x7fe754149e80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe744019040 tx=0x7fe74401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:57.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe73b7fe640 1 -- 172.21.15.76:0/915551053 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440bf410 con 0x7fe75415c0c0 2024-09-19T04:17:57.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe73b7fe640 1 -- 172.21.15.76:0/915551053 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7440bf5b0 con 0x7fe75415c0c0 2024-09-19T04:17:57.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7540ae450 con 0x7fe75415c0c0 2024-09-19T04:17:57.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe73b7fe640 1 -- 172.21.15.76:0/915551053 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7440bf980 con 0x7fe75415c0c0 2024-09-19T04:17:57.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7540aea10 con 0x7fe75415c0c0 2024-09-19T04:17:57.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7540ce5c0 con 0x7fe75415c0c0 2024-09-19T04:17:57.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe73b7fe640 1 -- 172.21.15.76:0/915551053 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7440bd020 con 0x7fe75415c0c0 2024-09-19T04:17:57.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe73b7fe640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7400678c0 0x7fe740069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:57.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe73b7fe640 1 -- 172.21.15.76:0/915551053 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5507+0+0 (secure 0 0 0) 0x7fe744120570 con 0x7fe75415c0c0 2024-09-19T04:17:57.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe752ffd640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7400678c0 0x7fe740069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:57.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.466+0000 7fe752ffd640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7400678c0 0x7fe740069d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe74c005c10 tx=0x7fe74c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:57.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.470+0000 7fe73b7fe640 1 -- 172.21.15.76:0/915551053 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe7440c4060 con 0x7fe75415c0c0 2024-09-19T04:17:57.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:57.642+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe75414a3c0 con 0x7fe75415c0c0 2024-09-19T04:17:58.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.374+0000 7fe73b7fe640 1 -- 172.21.15.76:0/915551053 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v74) ==== 97+0+0 (secure 0 0 0) 0x7fe744016050 con 0x7fe75415c0c0 2024-09-19T04:17:58.377 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2024-09-19T04:17:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.378+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7400678c0 msgr2=0x7fe740069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.378+0000 7fe75961c640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7400678c0 0x7fe740069d80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe74c005c10 tx=0x7fe74c009040 comp rx=0 tx=0).stop 2024-09-19T04:17:58.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.378+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75415c0c0 msgr2=0x7fe754149e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.378+0000 7fe75961c640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75415c0c0 0x7fe754149e80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fe744019040 tx=0x7fe74401ac70 comp rx=0 tx=0).stop 2024-09-19T04:17:58.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 shutdown_connections 2024-09-19T04:17:58.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7400678c0 0x7fe740069d80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe75415c0c0 0x7fe754149e80 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7540b5880 0x7fe75414a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 --2- 172.21.15.76:0/915551053 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe75406d3e0 0x7fe7540aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 >> 172.21.15.76:0/915551053 conn(0x7fe754072670 msgr2=0x7fe75415c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:58.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 shutdown_connections 2024-09-19T04:17:58.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.382+0000 7fe75961c640 1 -- 172.21.15.76:0/915551053 wait complete. 2024-09-19T04:17:58.403 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T04:17:58.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.522+0000 7fb79292d640 1 Processor -- start 2024-09-19T04:17:58.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb79292d640 1 -- start start 2024-09-19T04:17:58.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb79292d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 0x7fb78c0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb79292d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb78c0d2e90 0x7fb78c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb79292d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78c0d32b0 con 0x7fb78c0cf450 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb79292d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb78c0d0300 con 0x7fb78c0d2e90 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb79292d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c06e6c0 con 0x7fb78c161a60 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb78c0d2e90 0x7fb78c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb78c0d2e90 0x7fb78c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52082/0 (socket says 172.21.15.76:52082) 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 -- 172.21.15.76:0/3923074924 learned_addr learned my addr 172.21.15.76:0/3923074924 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 -- 172.21.15.76:0/3923074924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 msgr2=0x7fb78c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 --2- 172.21.15.76:0/3923074924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 0x7fb78c0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 -- 172.21.15.76:0/3923074924 >> v1:172.21.15.116:6789/0 conn(0x7fb78c0cf450 legacy=0x7fb78c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:58.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 -- 172.21.15.76:0/3923074924 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c0b0f60 con 0x7fb78c0d2e90 2024-09-19T04:17:58.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb7837fe640 1 --2- 172.21.15.76:0/3923074924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c0d2e90 0x7fb78c0cef10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb774009870 tx=0x7fb77401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc2935f1faa9577 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:58.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb78b7fe640 1 -- 172.21.15.76:0/3923074924 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0d1930 con 0x7fb78c0d2e90 2024-09-19T04:17:58.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb78b7fe640 1 -- 172.21.15.76:0/3923074924 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7740b0070 con 0x7fb78c0d2e90 2024-09-19T04:17:58.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.526+0000 7fb78b7fe640 1 -- 172.21.15.76:0/3923074924 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740ab840 con 0x7fb78c0d2e90 2024-09-19T04:17:58.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- 172.21.15.76:0/3923074924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c0d2e90 msgr2=0x7fb78c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 --2- 172.21.15.76:0/3923074924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c0d2e90 0x7fb78c0cef10 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fb774009870 tx=0x7fb77401c460 comp rx=0 tx=0).stop 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- 172.21.15.76:0/3923074924 shutdown_connections 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 --2- 172.21.15.76:0/3923074924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c0d2e90 0x7fb78c0cef10 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 --2- 172.21.15.76:0/3923074924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 0x7fb78c0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- 172.21.15.76:0/3923074924 >> 172.21.15.76:0/3923074924 conn(0x7fb78c08bf90 msgr2=0x7fb78c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- 172.21.15.76:0/3923074924 shutdown_connections 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- 172.21.15.76:0/3923074924 wait complete. 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 Processor -- start 2024-09-19T04:17:58.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- start start 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb78c0cf450 0x7fb78c1528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 0x7fb78c0b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c152390 0x7fb78c1584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c0cf900 con 0x7fb78c152390 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c06d190 con 0x7fb78c0cf450 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb78c0cfa80 con 0x7fb78c161a60 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c152390 0x7fb78c1584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c152390 0x7fb78c1584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52098/0 (socket says 172.21.15.76:52098) 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 -- 172.21.15.76:0/1979666827 learned_addr learned my addr 172.21.15.76:0/1979666827 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 -- 172.21.15.76:0/1979666827 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 msgr2=0x7fb78c0b4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 0x7fb78c0b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 -- 172.21.15.76:0/1979666827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb78c0cf450 msgr2=0x7fb78c1528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb78bfff640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb78c0cf450 0x7fb78c1528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb78c0cf450 0x7fb78c1528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 -- 172.21.15.76:0/1979666827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c153010 con 0x7fb78c152390 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb790eca640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c152390 0x7fb78c1584c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c780 tx=0x7fb78400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb7897fa640 1 -- 172.21.15.76:0/1979666827 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784017070 con 0x7fb78c152390 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb7897fa640 1 -- 172.21.15.76:0/1979666827 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb784003db0 con 0x7fb78c152390 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c1532f0 con 0x7fb78c152390 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb7897fa640 1 -- 172.21.15.76:0/1979666827 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7840052b0 con 0x7fb78c152390 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.530+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c1538b0 con 0x7fb78c152390 2024-09-19T04:17:58.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.534+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb78c0cef10 con 0x7fb78c152390 2024-09-19T04:17:58.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.534+0000 7fb7897fa640 1 -- 172.21.15.76:0/1979666827 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb78401a070 con 0x7fb78c152390 2024-09-19T04:17:58.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.534+0000 7fb7897fa640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb76c0678f0 0x7fb76c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.534+0000 7fb7897fa640 1 -- 172.21.15.76:0/1979666827 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7fb784085220 con 0x7fb78c152390 2024-09-19T04:17:58.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.534+0000 7fb78bfff640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb76c0678f0 0x7fb76c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.534+0000 7fb78bfff640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb76c0678f0 0x7fb76c069db0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00a790 tx=0x7fb77c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:58.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.538+0000 7fb7897fa640 1 -- 172.21.15.76:0/1979666827 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb784055e80 con 0x7fb78c152390 2024-09-19T04:17:58.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.758+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb78c0d4950 con 0x7fb78c152390 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.758+0000 7fb7897fa640 1 -- 172.21.15.76:0/1979666827 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7fb784055780 con 0x7fb78c152390 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w==", 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.761 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA==", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:17:58.762 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw==", 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.763 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA==", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ==", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.764 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w==", 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.765 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ==", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw==", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.766 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA==", 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw==", 2024-09-19T04:17:58.767 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg==", 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.768 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA==", 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.769 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg==", 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.770 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg==", 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ==", 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.771 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA==", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:17:58.772 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCopOtmGhUPLBAA9AkjLm1xF81Mjf1IYlN6Fg==", 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A==", 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg==", 2024-09-19T04:17:58.774 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:17:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:17:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:17:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:17:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:58.775 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-19T04:17:58.775 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb76c0678f0 msgr2=0x7fb76c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb76c0678f0 0x7fb76c069db0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00a790 tx=0x7fb77c008040 comp rx=0 tx=0).stop 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c152390 msgr2=0x7fb78c1584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c152390 0x7fb78c1584c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fb78400c780 tx=0x7fb78400cc40 comp rx=0 tx=0).stop 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 shutdown_connections 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb76c0678f0 0x7fb76c069db0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb78c152390 0x7fb78c1584c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb78c161a60 0x7fb78c0b4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 --2- 172.21.15.76:0/1979666827 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb78c0cf450 0x7fb78c1528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 >> 172.21.15.76:0/1979666827 conn(0x7fb78c08bf90 msgr2=0x7fb78c0b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 shutdown_connections 2024-09-19T04:17:58.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.762+0000 7fb79292d640 1 -- 172.21.15.76:0/1979666827 wait complete. 2024-09-19T04:17:58.781 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T04:17:58.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f313a534640 1 Processor -- start 2024-09-19T04:17:58.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f313a534640 1 -- start start 2024-09-19T04:17:58.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f313a534640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f313406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f313a534640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f313406fe50 0x7f313415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f313a534640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3134070250 con 0x7f313415f1d0 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f313a534640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3134152480 con 0x7f313406fe50 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f313a534640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f31340aede0 con 0x7f31341520a0 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f3138ad1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f313415f1d0 0x7f3134071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38446/0 (socket says 172.21.15.76:38446) 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f3138ad1640 1 -- 172.21.15.76:0/3921856476 learned_addr learned my addr 172.21.15.76:0/3921856476 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f31337fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f313406fe50 0x7f313415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f3133fff640 1 --2- 172.21.15.76:0/3921856476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f313406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f3132ffd640 1 -- 172.21.15.76:0/3921856476 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2641438849 0 0) 0x7f3134070250 con 0x7f313415f1d0 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f3132ffd640 1 -- 172.21.15.76:0/3921856476 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f311c003530 con 0x7f313415f1d0 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f31337fe640 1 -- 172.21.15.76:0/3921856476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 msgr2=0x7f313406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:58.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f31337fe640 1 --2- 172.21.15.76:0/3921856476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f313406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f31337fe640 1 -- 172.21.15.76:0/3921856476 >> v1:172.21.15.76:6789/0 conn(0x7f313415f1d0 legacy=0x7f3134071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.902+0000 7f31337fe640 1 -- 172.21.15.76:0/3921856476 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313415fa10 con 0x7f313406fe50 2024-09-19T04:17:58.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 --2- 172.21.15.76:0/3921856476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313406fe50 0x7f313415ec90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f31280097c0 tx=0x7f312801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0162ebd8057e136 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f3132ffd640 1 -- 172.21.15.76:0/3921856476 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31280b1070 con 0x7f313406fe50 2024-09-19T04:17:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f3132ffd640 1 -- 172.21.15.76:0/3921856476 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31280ac600 con 0x7f313406fe50 2024-09-19T04:17:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f3132ffd640 1 -- 172.21.15.76:0/3921856476 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31280ac9d0 con 0x7f313406fe50 2024-09-19T04:17:58.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- 172.21.15.76:0/3921856476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313406fe50 msgr2=0x7f313415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 --2- 172.21.15.76:0/3921856476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313406fe50 0x7f313415ec90 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f31280097c0 tx=0x7f312801c480 comp rx=0 tx=0).stop 2024-09-19T04:17:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- 172.21.15.76:0/3921856476 shutdown_connections 2024-09-19T04:17:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 --2- 172.21.15.76:0/3921856476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313406fe50 0x7f313415ec90 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 --2- 172.21.15.76:0/3921856476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f313406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- 172.21.15.76:0/3921856476 >> 172.21.15.76:0/3921856476 conn(0x7f31340b1f90 msgr2=0x7f31340b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- 172.21.15.76:0/3921856476 shutdown_connections 2024-09-19T04:17:58.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- 172.21.15.76:0/3921856476 wait complete. 2024-09-19T04:17:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 Processor -- start 2024-09-19T04:17:58.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- start start 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f3134148200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313415f1d0 0x7f3134148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313415c7c0 0x7f31340cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3134073d30 con 0x7f313415f1d0 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3134073a50 con 0x7f313415c7c0 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f313a534640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f313415f600 con 0x7f31341520a0 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313415f1d0 0x7f3134148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313415f1d0 0x7f3134148740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52104/0 (socket says 172.21.15.76:52104) 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 -- 172.21.15.76:0/1801736749 learned_addr learned my addr 172.21.15.76:0/1801736749 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:58.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f3133fff640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f3134148200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f3138ad1640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313415c7c0 0x7f31340cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 -- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 msgr2=0x7f3134148200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f3134148200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 -- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313415c7c0 msgr2=0x7f31340cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313415c7c0 0x7f31340cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f31337fe640 1 -- 172.21.15.76:0/1801736749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3134074980 con 0x7f313415f1d0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.906+0000 7f3133fff640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f3134148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f31337fe640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313415f1d0 0x7f3134148740 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3128019090 tx=0x7f312801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3130ff9640 1 -- 172.21.15.76:0/1801736749 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31280bc070 con 0x7f313415f1d0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3130ff9640 1 -- 172.21.15.76:0/1801736749 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31280b7430 con 0x7f313415f1d0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3134147f20 con 0x7f313415f1d0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3130ff9640 1 -- 172.21.15.76:0/1801736749 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31280b1040 con 0x7f313415f1d0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31340cf3f0 con 0x7f313415f1d0 2024-09-19T04:17:58.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313415ec90 con 0x7f313415f1d0 2024-09-19T04:17:58.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3130ff9640 1 -- 172.21.15.76:0/1801736749 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f31280bf070 con 0x7f313415f1d0 2024-09-19T04:17:58.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3130ff9640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31200678c0 0x7f3120069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:58.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3133fff640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31200678c0 0x7f3120069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:58.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3130ff9640 1 -- 172.21.15.76:0/1801736749 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f312811f070 con 0x7f313415f1d0 2024-09-19T04:17:58.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.910+0000 7f3133fff640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31200678c0 0x7f3120069d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3124009ae0 tx=0x7f3124008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:58.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:58.914+0000 7f3130ff9640 1 -- 172.21.15.76:0/1801736749 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3128016070 con 0x7f313415f1d0 2024-09-19T04:17:59.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.082+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f31340aea70 con 0x7f313415f1d0 2024-09-19T04:17:59.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.082+0000 7f3130ff9640 1 -- 172.21.15.76:0/1801736749 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f31280efe00 con 0x7f313415f1d0 2024-09-19T04:17:59.085 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:17:59.085 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:17:59.085 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:17:59.085 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-09-19T04:17:59.085 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-09-19T04:17:59.086 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:17:59.086 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:17:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31200678c0 msgr2=0x7f3120069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31200678c0 0x7f3120069d80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3124009ae0 tx=0x7f3124008040 comp rx=0 tx=0).stop 2024-09-19T04:17:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313415f1d0 msgr2=0x7f3134148740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313415f1d0 0x7f3134148740 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f3128019090 tx=0x7f312801acc0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 shutdown_connections 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31200678c0 0x7f3120069d80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313415c7c0 0x7f31340cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f313415f1d0 0x7f3134148740 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 --2- 172.21.15.76:0/1801736749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f31341520a0 0x7f3134148200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 >> 172.21.15.76:0/1801736749 conn(0x7f31340b1f90 msgr2=0x7f313415d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 shutdown_connections 2024-09-19T04:17:59.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.086+0000 7f313a534640 1 -- 172.21.15.76:0/1801736749 wait complete. 2024-09-19T04:17:59.104 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:17:59.105 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:17:59.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 Processor -- start 2024-09-19T04:17:59.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 -- start start 2024-09-19T04:17:59.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c15bec0 0x7f009c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f009c15c2a0 0x7f009c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f009c0af2a0 0x7f009c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f009c0b3290 con 0x7f009c15c2a0 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f009c0b3570 con 0x7f009c0af2a0 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a3344640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f009c0b3850 con 0x7f009c15bec0 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f009c15c2a0 0x7f009c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f009c15c2a0 0x7f009c0aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52110/0 (socket says 172.21.15.76:52110) 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 -- 172.21.15.76:0/1260743017 learned_addr learned my addr 172.21.15.76:0/1260743017 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a18e1640 1 --2- 172.21.15.76:0/1260743017 >> v2:172.21.15.116:3300/0 conn(0x7f009c0af2a0 0x7f009c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a10e0640 1 --2- 172.21.15.76:0/1260743017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c15bec0 0x7f009c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 -- 172.21.15.76:0/1260743017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c15bec0 msgr2=0x7f009c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 --2- 172.21.15.76:0/1260743017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c15bec0 0x7f009c0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 -- 172.21.15.76:0/1260743017 >> v2:172.21.15.116:3300/0 conn(0x7f009c0af2a0 msgr2=0x7f009c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:17:59.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 --2- 172.21.15.76:0/1260743017 >> v2:172.21.15.116:3300/0 conn(0x7f009c0af2a0 0x7f009c15fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a08df640 1 -- 172.21.15.76:0/1260743017 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c0b3b30 con 0x7f009c15c2a0 2024-09-19T04:17:59.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.234+0000 7f00a18e1640 1 --2- 172.21.15.76:0/1260743017 >> v2:172.21.15.116:3300/0 conn(0x7f009c0af2a0 0x7f009c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:17:59.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a08df640 1 --2- 172.21.15.76:0/1260743017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c0aed60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f009c0ada40 tx=0x7f009001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841bf7e5fff2ad8a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f009bfff640 1 -- 172.21.15.76:0/1260743017 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00900a5580 con 0x7f009c15c2a0 2024-09-19T04:17:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f009bfff640 1 -- 172.21.15.76:0/1260743017 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00900a5720 con 0x7f009c15c2a0 2024-09-19T04:17:59.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f009bfff640 1 -- 172.21.15.76:0/1260743017 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00900a5af0 con 0x7f009c15c2a0 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- 172.21.15.76:0/1260743017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 msgr2=0x7f009c0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 --2- 172.21.15.76:0/1260743017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c0aed60 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f009c0ada40 tx=0x7f009001c700 comp rx=0 tx=0).stop 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- 172.21.15.76:0/1260743017 shutdown_connections 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 --2- 172.21.15.76:0/1260743017 >> v2:172.21.15.116:3300/0 conn(0x7f009c0af2a0 0x7f009c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 --2- 172.21.15.76:0/1260743017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c0aed60 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 --2- 172.21.15.76:0/1260743017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c15bec0 0x7f009c0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- 172.21.15.76:0/1260743017 >> 172.21.15.76:0/1260743017 conn(0x7f009c0b1f90 msgr2=0x7f009c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- 172.21.15.76:0/1260743017 shutdown_connections 2024-09-19T04:17:59.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- 172.21.15.76:0/1260743017 wait complete. 2024-09-19T04:17:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 Processor -- start 2024-09-19T04:17:59.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- start start 2024-09-19T04:17:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c0af2a0 0x7f009c1502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f009c15bec0 0x7f009c154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c06e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f009c0af6b0 con 0x7f009c15c2a0 2024-09-19T04:17:59.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f009c0ad740 con 0x7f009c15bec0 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a3344640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f009c06e000 con 0x7f009c0af2a0 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a18e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c06e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a18e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c06e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52116/0 (socket says 172.21.15.76:52116) 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a18e1640 1 -- 172.21.15.76:0/4100431472 learned_addr learned my addr 172.21.15.76:0/4100431472 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a10e0640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c0af2a0 0x7f009c1502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.238+0000 7f00a08df640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f009c15bec0 0x7f009c154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a18e1640 1 -- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c0af2a0 msgr2=0x7f009c1502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a18e1640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c0af2a0 0x7f009c1502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a18e1640 1 -- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f009c15bec0 msgr2=0x7f009c154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a18e1640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f009c15bec0 0x7f009c154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a18e1640 1 -- 172.21.15.76:0/4100431472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f009c14f520 con 0x7f009c15c2a0 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a18e1640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c06e8f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f008c00c7f0 tx=0x7f008c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:59.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f0099ffb640 1 -- 172.21.15.76:0/4100431472 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c015070 con 0x7f009c15c2a0 2024-09-19T04:17:59.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f0099ffb640 1 -- 172.21.15.76:0/4100431472 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f008c007860 con 0x7f009c15c2a0 2024-09-19T04:17:59.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f009c14f7c0 con 0x7f009c15c2a0 2024-09-19T04:17:59.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f0099ffb640 1 -- 172.21.15.76:0/4100431472 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f008c007c50 con 0x7f009c15c2a0 2024-09-19T04:17:59.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f009c1552c0 con 0x7f009c15c2a0 2024-09-19T04:17:59.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f009c0ad200 con 0x7f009c15c2a0 2024-09-19T04:17:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f0099ffb640 1 -- 172.21.15.76:0/4100431472 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f008c018070 con 0x7f009c15c2a0 2024-09-19T04:17:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f0099ffb640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00880678c0 0x7f0088069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:17:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a10e0640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00880678c0 0x7f0088069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:17:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f0099ffb640 1 -- 172.21.15.76:0/4100431472 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5249+0+0 (secure 0 0 0) 0x7f008c085c70 con 0x7f009c15c2a0 2024-09-19T04:17:59.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.242+0000 7f00a10e0640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00880678c0 0x7f0088069d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f009c06f5b0 tx=0x7f0094008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:17:59.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.246+0000 7f0099ffb640 1 -- 172.21.15.76:0/4100431472 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f008c056970 con 0x7f009c15c2a0 2024-09-19T04:17:59.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:17:59.414+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f009c06fcd0 con 0x7f009c15c2a0 2024-09-19T04:18:00.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.382+0000 7f0099ffb640 1 -- 172.21.15.76:0/4100431472 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7f008c056270 con 0x7f009c15c2a0 2024-09-19T04:18:00.383 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-09-19T04:18:00.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00880678c0 msgr2=0x7f0088069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00880678c0 0x7f0088069d80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f009c06f5b0 tx=0x7f0094008040 comp rx=0 tx=0).stop 2024-09-19T04:18:00.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 msgr2=0x7f009c06e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c06e8f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f008c00c7f0 tx=0x7f008c00ccb0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 shutdown_connections 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f00880678c0 0x7f0088069d80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f009c15c2a0 0x7f009c06e8f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f009c15bec0 0x7f009c154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 --2- 172.21.15.76:0/4100431472 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f009c0af2a0 0x7f009c1502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 >> 172.21.15.76:0/4100431472 conn(0x7f009c0b1f90 msgr2=0x7f009c0d0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 shutdown_connections 2024-09-19T04:18:00.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.386+0000 7f00a3344640 1 -- 172.21.15.76:0/4100431472 wait complete. 2024-09-19T04:18:00.415 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:18:00.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10b571d640 1 Processor -- start 2024-09-19T04:18:00.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10b571d640 1 -- start start 2024-09-19T04:18:00.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10b571d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f10b0159d30 0x7f10b00b5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10b571d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10b0160aa0 con 0x7f10b00b5c10 2024-09-19T04:18:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10b571d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f10b009ab80 con 0x7f10b00b6060 2024-09-19T04:18:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10b571d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f10b015a110 con 0x7f10b0159d30 2024-09-19T04:18:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10aeffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f10b0159d30 0x7f10b00b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10aeffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f10b0159d30 0x7f10b00b5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52122/0 (socket says 172.21.15.76:52122) 2024-09-19T04:18:00.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10aeffd640 1 -- 172.21.15.76:0/2494560105 learned_addr learned my addr 172.21.15.76:0/2494560105 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:00.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10ae7fc640 1 --1- 172.21.15.76:0/2494560105 >> v1:172.21.15.76:6789/0 conn(0x7f10b00b5c10 0x7f10b015fc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38452/0 (socket says 172.21.15.76:38452) 2024-09-19T04:18:00.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10aeffd640 1 -- 172.21.15.76:0/2494560105 >> v1:172.21.15.116:6789/0 conn(0x7f10b00b6060 legacy=0x7f10b0160390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:00.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10aeffd640 1 -- 172.21.15.76:0/2494560105 >> v1:172.21.15.76:6789/0 conn(0x7f10b00b5c10 legacy=0x7f10b015fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10aeffd640 1 -- 172.21.15.76:0/2494560105 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b009ae60 con 0x7f10b0159d30 2024-09-19T04:18:00.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.542+0000 7f10adffb640 1 -- 172.21.15.76:0/2494560105 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2773181359 0 0) 0x7f10b0160aa0 con 0x7f10b00b5c10 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10aeffd640 1 --2- 172.21.15.76:0/2494560105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00b5190 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009870 tx=0x7f10a801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61b450c803775e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10adffb640 1 -- 172.21.15.76:0/2494560105 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a80b0070 con 0x7f10b0159d30 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10adffb640 1 -- 172.21.15.76:0/2494560105 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a80ab600 con 0x7f10b0159d30 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10adffb640 1 -- 172.21.15.76:0/2494560105 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a80ab9d0 con 0x7f10b0159d30 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- 172.21.15.76:0/2494560105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 msgr2=0x7f10b00b5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 --2- 172.21.15.76:0/2494560105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00b5190 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f10a8009870 tx=0x7f10a801c520 comp rx=0 tx=0).stop 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- 172.21.15.76:0/2494560105 shutdown_connections 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 --2- 172.21.15.76:0/2494560105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00b5190 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- 172.21.15.76:0/2494560105 >> 172.21.15.76:0/2494560105 conn(0x7f10b00b3e90 msgr2=0x7f10b00b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- 172.21.15.76:0/2494560105 shutdown_connections 2024-09-19T04:18:00.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- 172.21.15.76:0/2494560105 wait complete. 2024-09-19T04:18:00.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 Processor -- start 2024-09-19T04:18:00.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- start start 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10b00b5c10 0x7f10b0148580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f10b00b6060 0x7f10b00ceeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f10b009bc20 con 0x7f10b0159d30 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f10b00b6470 con 0x7f10b00b5c10 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10b571d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f10b009bda0 con 0x7f10b00b6060 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10af7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52130/0 (socket says 172.21.15.76:52130) 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10af7fe640 1 -- 172.21.15.76:0/1254481915 learned_addr learned my addr 172.21.15.76:0/1254481915 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:00.551 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.546+0000 7f10aeffd640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10b00b5c10 0x7f10b0148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:00.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10ae7fc640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f10b00b6060 0x7f10b00ceeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:00.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10af7fe640 1 -- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f10b00b6060 msgr2=0x7f10b00ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10af7fe640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f10b00b6060 0x7f10b00ceeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10af7fe640 1 -- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10b00b5c10 msgr2=0x7f10b0148580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:00.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10af7fe640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10b00b5c10 0x7f10b0148580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:00.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10af7fe640 1 -- 172.21.15.76:0/1254481915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b006e660 con 0x7f10b0159d30 2024-09-19T04:18:00.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10aeffd640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10b00b5c10 0x7f10b0148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:00.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10ae7fc640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f10b00b6060 0x7f10b00ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:00.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10af7fe640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00cf3f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c780 tx=0x7f10a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:00.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10a77fe640 1 -- 172.21.15.76:0/1254481915 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0017070 con 0x7f10b0159d30 2024-09-19T04:18:00.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10a77fe640 1 -- 172.21.15.76:0/1254481915 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0003db0 con 0x7f10b0159d30 2024-09-19T04:18:00.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b0156ab0 con 0x7f10b0159d30 2024-09-19T04:18:00.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10a77fe640 1 -- 172.21.15.76:0/1254481915 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a00052b0 con 0x7f10b0159d30 2024-09-19T04:18:00.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b0157070 con 0x7f10b0159d30 2024-09-19T04:18:00.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b00b5190 con 0x7f10b0159d30 2024-09-19T04:18:00.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10a77fe640 1 -- 172.21.15.76:0/1254481915 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f10a001a070 con 0x7f10b0159d30 2024-09-19T04:18:00.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10a77fe640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f10900678c0 0x7f1090069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:00.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10a77fe640 1 -- 172.21.15.76:0/1254481915 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5619+0+0 (secure 0 0 0) 0x7f10a0085360 con 0x7f10b0159d30 2024-09-19T04:18:00.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10aeffd640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f10900678c0 0x7f1090069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:00.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.550+0000 7f10aeffd640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f10900678c0 0x7f1090069d80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10a80ad000 tx=0x7f10a80a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:00.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.554+0000 7f10a77fe640 1 -- 172.21.15.76:0/1254481915 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f10a0055e60 con 0x7f10b0159d30 2024-09-19T04:18:00.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:00.718+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f10b01522e0 con 0x7f10b0159d30 2024-09-19T04:18:01.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.394+0000 7f10a77fe640 1 -- 172.21.15.76:0/1254481915 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v78) ==== 170+0+0 (secure 0 0 0) 0x7f10a0055760 con 0x7f10b0159d30 2024-09-19T04:18:01.397 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-09-19T04:18:01.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f10900678c0 msgr2=0x7f1090069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:01.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f10900678c0 0x7f1090069d80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f10a80ad000 tx=0x7f10a80a6000 comp rx=0 tx=0).stop 2024-09-19T04:18:01.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 msgr2=0x7f10b00cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:01.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00cf3f0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f10a000c780 tx=0x7f10a000cc40 comp rx=0 tx=0).stop 2024-09-19T04:18:01.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 shutdown_connections 2024-09-19T04:18:01.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f10900678c0 0x7f1090069d80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f10b0159d30 0x7f10b00cf3f0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f10b00b6060 0x7f10b00ceeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 --2- 172.21.15.76:0/1254481915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10b00b5c10 0x7f10b0148580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 >> 172.21.15.76:0/1254481915 conn(0x7f10b00b3e90 msgr2=0x7f10b0151440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:01.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 shutdown_connections 2024-09-19T04:18:01.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.398+0000 7f10b571d640 1 -- 172.21.15.76:0/1254481915 wait complete. 2024-09-19T04:18:01.431 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:18:01.549 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd78073640 1 Processor -- start 2024-09-19T04:18:01.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd78073640 1 -- start start 2024-09-19T04:18:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd78073640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd7006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:01.552 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd78073640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbd7015f3a0 0x7fbd700adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:01.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd78073640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd70070250 con 0x7fbd7006fe50 2024-09-19T04:18:01.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd78073640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbd70152480 con 0x7fbd7015f3a0 2024-09-19T04:18:01.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd78073640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbd700aede0 con 0x7fbd701520a0 2024-09-19T04:18:01.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd7560e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbd7006fe50 0x7fbd7015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38466/0 (socket says 172.21.15.76:38466) 2024-09-19T04:18:01.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd7560e640 1 -- 172.21.15.76:0/482076369 learned_addr learned my addr 172.21.15.76:0/482076369 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:01.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd76610640 1 --2- 172.21.15.76:0/482076369 >> v2:172.21.15.76:3300/0 conn(0x7fbd7015f3a0 0x7fbd700adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:01.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd75e0f640 1 --2- 172.21.15.76:0/482076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd7006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:01.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd74e0d640 1 -- 172.21.15.76:0/482076369 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3047039455 0 0) 0x7fbd70070250 con 0x7fbd7006fe50 2024-09-19T04:18:01.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd74e0d640 1 -- 172.21.15.76:0/482076369 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbd5c003530 con 0x7fbd7006fe50 2024-09-19T04:18:01.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd76610640 1 -- 172.21.15.76:0/482076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 msgr2=0x7fbd7006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:01.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd76610640 1 --2- 172.21.15.76:0/482076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd7006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd76610640 1 -- 172.21.15.76:0/482076369 >> v1:172.21.15.76:6789/0 conn(0x7fbd7006fe50 legacy=0x7fbd7015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:01.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd76610640 1 -- 172.21.15.76:0/482076369 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7015fa10 con 0x7fbd7015f3a0 2024-09-19T04:18:01.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.550+0000 7fbd76610640 1 --2- 172.21.15.76:0/482076369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd700adc80 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009d90 tx=0x7fbd6801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60f9952fbfa8285 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:01.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd74e0d640 1 -- 172.21.15.76:0/482076369 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680b2070 con 0x7fbd7015f3a0 2024-09-19T04:18:01.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd74e0d640 1 -- 172.21.15.76:0/482076369 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd680a5a30 con 0x7fbd7015f3a0 2024-09-19T04:18:01.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd74e0d640 1 -- 172.21.15.76:0/482076369 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680a5de0 con 0x7fbd7015f3a0 2024-09-19T04:18:01.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- 172.21.15.76:0/482076369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 msgr2=0x7fbd700adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:01.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 --2- 172.21.15.76:0/482076369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd700adc80 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbd68009d90 tx=0x7fbd6801c970 comp rx=0 tx=0).stop 2024-09-19T04:18:01.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- 172.21.15.76:0/482076369 shutdown_connections 2024-09-19T04:18:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 --2- 172.21.15.76:0/482076369 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd700adc80 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 --2- 172.21.15.76:0/482076369 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd7006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- 172.21.15.76:0/482076369 >> 172.21.15.76:0/482076369 conn(0x7fbd700b1f90 msgr2=0x7fbd700b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- 172.21.15.76:0/482076369 shutdown_connections 2024-09-19T04:18:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- 172.21.15.76:0/482076369 wait complete. 2024-09-19T04:18:01.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 Processor -- start 2024-09-19T04:18:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- start start 2024-09-19T04:18:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7006fe50 0x7fbd7014c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd70146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd701662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:01.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70073d30 con 0x7fbd7015f3a0 2024-09-19T04:18:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbd70073a50 con 0x7fbd7006fe50 2024-09-19T04:18:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbd70165f00 con 0x7fbd701520a0 2024-09-19T04:18:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd701662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd701662e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52152/0 (socket says 172.21.15.76:52152) 2024-09-19T04:18:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 -- 172.21.15.76:0/3456688657 learned_addr learned my addr 172.21.15.76:0/3456688657 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:01.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd75e0f640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7006fe50 0x7fbd7014c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd7560e640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd70146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 -- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 msgr2=0x7fbd70146820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd70146820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 -- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7006fe50 msgr2=0x7fbd7014c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7006fe50 0x7fbd7014c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:01.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 -- 172.21.15.76:0/3456688657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd7014cbc0 con 0x7fbd7015f3a0 2024-09-19T04:18:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd76610640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd701662e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbd680b0040 tx=0x7fbd68019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd667fc640 1 -- 172.21.15.76:0/3456688657 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd680b2050 con 0x7fbd7015f3a0 2024-09-19T04:18:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd667fc640 1 -- 172.21.15.76:0/3456688657 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd680be070 con 0x7fbd7015f3a0 2024-09-19T04:18:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd667fc640 1 -- 172.21.15.76:0/3456688657 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd68004040 con 0x7fbd7015f3a0 2024-09-19T04:18:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd70168460 con 0x7fbd7015f3a0 2024-09-19T04:18:01.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.554+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd70168a20 con 0x7fbd7015f3a0 2024-09-19T04:18:01.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.558+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd700adc80 con 0x7fbd7015f3a0 2024-09-19T04:18:01.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.558+0000 7fbd667fc640 1 -- 172.21.15.76:0/3456688657 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbd680041e0 con 0x7fbd7015f3a0 2024-09-19T04:18:01.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.558+0000 7fbd667fc640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd540678c0 0x7fbd54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:01.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.558+0000 7fbd667fc640 1 -- 172.21.15.76:0/3456688657 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 6001+0+0 (secure 0 0 0) 0x7fbd68120f40 con 0x7fbd7015f3a0 2024-09-19T04:18:01.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.558+0000 7fbd75e0f640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd540678c0 0x7fbd54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:01.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.558+0000 7fbd75e0f640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd540678c0 0x7fbd54069d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b9d0 tx=0x7fbd6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:01.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.562+0000 7fbd667fc640 1 -- 172.21.15.76:0/3456688657 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbd68016560 con 0x7fbd7015f3a0 2024-09-19T04:18:01.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:01.758+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbd7006e540 con 0x7fbd7015f3a0 2024-09-19T04:18:02.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.430+0000 7fbd667fc640 1 -- 172.21.15.76:0/3456688657 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v63) ==== 174+0+0 (secure 0 0 0) 0x7fbd680f19e0 con 0x7fbd7015f3a0 2024-09-19T04:18:02.436 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 11 and data pool 12 2024-09-19T04:18:02.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd540678c0 msgr2=0x7fbd54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd540678c0 0x7fbd54069d80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b9d0 tx=0x7fbd6c008040 comp rx=0 tx=0).stop 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 msgr2=0x7fbd701662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd701662e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbd680b0040 tx=0x7fbd68019710 comp rx=0 tx=0).stop 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 shutdown_connections 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbd540678c0 0x7fbd54069d80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbd7015f3a0 0x7fbd701662e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbd701520a0 0x7fbd70146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 --2- 172.21.15.76:0/3456688657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbd7006fe50 0x7fbd7014c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 >> 172.21.15.76:0/3456688657 conn(0x7fbd700b1f90 msgr2=0x7fbd700b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:02.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 shutdown_connections 2024-09-19T04:18:02.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.450+0000 7fbd78073640 1 -- 172.21.15.76:0/3456688657 wait complete. 2024-09-19T04:18:02.478 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.590+0000 7f9128106640 1 Processor -- start 2024-09-19T04:18:02.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.590+0000 7f9128106640 1 -- start start 2024-09-19T04:18:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f9128106640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015bec0 0x7f91200ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f9128106640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f91200af470 0x7f912006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f9128106640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91200b3290 con 0x7f912015c2a0 2024-09-19T04:18:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f9128106640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f91200b3570 con 0x7f91200af470 2024-09-19T04:18:02.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f9128106640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f91200b3850 con 0x7f912015bec0 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f91266a3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f91200af470 0x7f912006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f91256a1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f912015c2a0 0x7f91200aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:38474/0 (socket says 172.21.15.76:38474) 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f91256a1640 1 -- 172.21.15.76:0/1982484038 learned_addr learned my addr 172.21.15.76:0/1982484038 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f91266a3640 1 -- 172.21.15.76:0/1982484038 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015bec0 msgr2=0x7f91200ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f91266a3640 1 --2- 172.21.15.76:0/1982484038 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015bec0 0x7f91200ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f91266a3640 1 -- 172.21.15.76:0/1982484038 >> v1:172.21.15.76:6789/0 conn(0x7f912015c2a0 legacy=0x7f91200aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f91266a3640 1 -- 172.21.15.76:0/1982484038 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91200b3b30 con 0x7f91200af470 2024-09-19T04:18:02.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.594+0000 7f9124ea0640 1 -- 172.21.15.76:0/1982484038 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2371105303 0 0) 0x7f91200b3290 con 0x7f912015c2a0 2024-09-19T04:18:02.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f91266a3640 1 --2- 172.21.15.76:0/1982484038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912006e000 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9118009d90 tx=0x7f911801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66c515380ed57bd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9124ea0640 1 -- 172.21.15.76:0/1982484038 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91180b2070 con 0x7f91200af470 2024-09-19T04:18:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9124ea0640 1 -- 172.21.15.76:0/1982484038 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91180a5a00 con 0x7f91200af470 2024-09-19T04:18:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9124ea0640 1 -- 172.21.15.76:0/1982484038 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91180a5dd0 con 0x7f91200af470 2024-09-19T04:18:02.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 -- 172.21.15.76:0/1982484038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 msgr2=0x7f912006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 --2- 172.21.15.76:0/1982484038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912006e000 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9118009d90 tx=0x7f911801c940 comp rx=0 tx=0).stop 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 -- 172.21.15.76:0/1982484038 shutdown_connections 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 --2- 172.21.15.76:0/1982484038 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912006e000 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 --2- 172.21.15.76:0/1982484038 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015bec0 0x7f91200ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 -- 172.21.15.76:0/1982484038 >> 172.21.15.76:0/1982484038 conn(0x7f91200b1f90 msgr2=0x7f91200b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 -- 172.21.15.76:0/1982484038 shutdown_connections 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 -- 172.21.15.76:0/1982484038 wait complete. 2024-09-19T04:18:02.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 Processor -- start 2024-09-19T04:18:02.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.598+0000 7f9128106640 1 -- start start 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912009ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f912015bec0 0x7f912014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015c2a0 0x7f91200ce870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f912006ed50 con 0x7f91200af470 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f91200ad740 con 0x7f912015bec0 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f912006eed0 con 0x7f912015c2a0 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912009ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912009ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52168/0 (socket says 172.21.15.76:52168) 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 -- 172.21.15.76:0/3749224192 learned_addr learned my addr 172.21.15.76:0/3749224192 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91256a1640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f912015bec0 0x7f912014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91266a3640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015c2a0 0x7f91200ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 -- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015c2a0 msgr2=0x7f91200ce870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015c2a0 0x7f91200ce870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 -- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f912015bec0 msgr2=0x7f912014c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f912015bec0 0x7f912014c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 -- 172.21.15.76:0/3749224192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912009d5a0 con 0x7f91200af470 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9125ea2640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912009ce10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f911000b550 tx=0x7f911000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91167fc640 1 -- 172.21.15.76:0/3749224192 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110014070 con 0x7f91200af470 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91167fc640 1 -- 172.21.15.76:0/3749224192 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9110010890 con 0x7f91200af470 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912009d880 con 0x7f91200af470 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91167fc640 1 -- 172.21.15.76:0/3749224192 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110010c80 con 0x7f91200af470 2024-09-19T04:18:02.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912009de40 con 0x7f91200af470 2024-09-19T04:18:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912006f250 con 0x7f91200af470 2024-09-19T04:18:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91167fc640 1 -- 172.21.15.76:0/3749224192 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9110017070 con 0x7f91200af470 2024-09-19T04:18:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91167fc640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f91040678f0 0x7f9104069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91167fc640 1 -- 172.21.15.76:0/3749224192 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7f9110085b60 con 0x7f91200af470 2024-09-19T04:18:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.602+0000 7f91256a1640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f91040678f0 0x7f9104069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.606+0000 7f91256a1640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f91040678f0 0x7f9104069db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91201556d0 tx=0x7f911c007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:02.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.610+0000 7f91167fc640 1 -- 172.21.15.76:0/3749224192 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f91100565a0 con 0x7f91200af470 2024-09-19T04:18:02.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.774+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f912006fcd0 con 0x7f91200af470 2024-09-19T04:18:02.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.774+0000 7f91167fc640 1 -- 172.21.15.76:0/3749224192 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+17042 (secure 0 0 0) 0x7f9110010e20 con 0x7f91200af470 2024-09-19T04:18:02.775 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:02.775 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":79,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:01.761940+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:17:59.452038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:00.756649+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:02.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f91040678f0 msgr2=0x7f9104069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f91040678f0 0x7f9104069db0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91201556d0 tx=0x7f911c007c10 comp rx=0 tx=0).stop 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 msgr2=0x7f912009ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912009ce10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f911000b550 tx=0x7f911000ba10 comp rx=0 tx=0).stop 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 shutdown_connections 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f91040678f0 0x7f9104069db0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f912015c2a0 0x7f91200ce870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f912015bec0 0x7f912014c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 --2- 172.21.15.76:0/3749224192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f91200af470 0x7f912009ce10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 >> 172.21.15.76:0/3749224192 conn(0x7f91200b1f90 msgr2=0x7f91200cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 shutdown_connections 2024-09-19T04:18:02.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.778+0000 7f9128106640 1 -- 172.21.15.76:0/3749224192 wait complete. 2024-09-19T04:18:02.796 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:02.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9b6372640 1 Processor -- start 2024-09-19T04:18:02.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9b6372640 1 -- start start 2024-09-19T04:18:02.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9b6372640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9b015bec0 0x7fc9b00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9b6372640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b00af470 0x7fc9b006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9b6372640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9b00b3290 con 0x7fc9b015c2a0 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9b6372640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc9b00b3570 con 0x7fc9b015bec0 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9b6372640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b00b3850 con 0x7fc9b00af470 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9b015bec0 0x7fc9b00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc9b015bec0 0x7fc9b00ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41376/0 (socket says 172.21.15.76:41376) 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9a77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc9b015c2a0 0x7fc9b00aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:44968/0 (socket says 172.21.15.76:44968) 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 -- 172.21.15.76:0/2706313501 learned_addr learned my addr 172.21.15.76:0/2706313501 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 -- 172.21.15.76:0/2706313501 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b00af470 msgr2=0x7fc9b006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 --2- 172.21.15.76:0/2706313501 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b00af470 0x7fc9b006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 -- 172.21.15.76:0/2706313501 >> v1:172.21.15.76:6789/0 conn(0x7fc9b015c2a0 legacy=0x7fc9b00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 -- 172.21.15.76:0/2706313501 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b00b3b30 con 0x7fc9b015bec0 2024-09-19T04:18:02.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.918+0000 7fc9affff640 1 --2- 172.21.15.76:0/2706313501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b015bec0 0x7fc9b00ad200 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009870 tx=0x7fc9a001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f628b0fbfa57eb7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:02.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9af7fe640 1 -- 172.21.15.76:0/2706313501 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00aa600 con 0x7fc9b015bec0 2024-09-19T04:18:02.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9af7fe640 1 -- 172.21.15.76:0/2706313501 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a00af070 con 0x7fc9b015bec0 2024-09-19T04:18:02.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9af7fe640 1 -- 172.21.15.76:0/2706313501 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00aa9c0 con 0x7fc9b015bec0 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- 172.21.15.76:0/2706313501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b015bec0 msgr2=0x7fc9b00ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 --2- 172.21.15.76:0/2706313501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b015bec0 0x7fc9b00ad200 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009870 tx=0x7fc9a001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- 172.21.15.76:0/2706313501 shutdown_connections 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 --2- 172.21.15.76:0/2706313501 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b00af470 0x7fc9b006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 --2- 172.21.15.76:0/2706313501 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b015bec0 0x7fc9b00ad200 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- 172.21.15.76:0/2706313501 >> 172.21.15.76:0/2706313501 conn(0x7fc9b00b1f90 msgr2=0x7fc9b00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- 172.21.15.76:0/2706313501 shutdown_connections 2024-09-19T04:18:02.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- 172.21.15.76:0/2706313501 wait complete. 2024-09-19T04:18:02.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 Processor -- start 2024-09-19T04:18:02.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- start start 2024-09-19T04:18:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b00af470 0x7fc9b0149270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b015bec0 0x7fc9b00aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b015c2a0 0x7fc9b00b0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b006ed50 con 0x7fc9b00af470 2024-09-19T04:18:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b00ad740 con 0x7fc9b015c2a0 2024-09-19T04:18:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b6372640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc9b006eed0 con 0x7fc9b015bec0 2024-09-19T04:18:02.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b00af470 0x7fc9b0149270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b00af470 0x7fc9b0149270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41392/0 (socket says 172.21.15.76:41392) 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 -- 172.21.15.76:0/649447245 learned_addr learned my addr 172.21.15.76:0/649447245 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9a77fe640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b015bec0 0x7fc9b00aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9b490f640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b015c2a0 0x7fc9b00b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 -- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b015bec0 msgr2=0x7fc9b00aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b015bec0 0x7fc9b00aff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 -- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b015c2a0 msgr2=0x7fc9b00b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b015c2a0 0x7fc9b00b0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:02.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 -- 172.21.15.76:0/649447245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b01497b0 con 0x7fc9b00af470 2024-09-19T04:18:02.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.922+0000 7fc9affff640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b00af470 0x7fc9b0149270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00ac040 tx=0x7fc9a001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:02.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9ad7fa640 1 -- 172.21.15.76:0/649447245 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00bd5f0 con 0x7fc9b00af470 2024-09-19T04:18:02.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9ad7fa640 1 -- 172.21.15.76:0/649447245 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9a00bd7b0 con 0x7fc9b00af470 2024-09-19T04:18:02.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b0154570 con 0x7fc9b00af470 2024-09-19T04:18:02.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9ad7fa640 1 -- 172.21.15.76:0/649447245 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a00bdba0 con 0x7fc9b00af470 2024-09-19T04:18:02.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b0154b30 con 0x7fc9b00af470 2024-09-19T04:18:02.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9b006e000 con 0x7fc9b00af470 2024-09-19T04:18:02.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9ad7fa640 1 -- 172.21.15.76:0/649447245 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9a00a9070 con 0x7fc9b00af470 2024-09-19T04:18:02.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9ad7fa640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9900678c0 0x7fc990069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:02.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9ad7fa640 1 -- 172.21.15.76:0/649447245 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fc9a0120940 con 0x7fc9b00af470 2024-09-19T04:18:02.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9a77fe640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9900678c0 0x7fc990069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:02.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.926+0000 7fc9a77fe640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9900678c0 0x7fc990069d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0151a60 tx=0x7fc998007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:02.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:02.930+0000 7fc9ad7fa640 1 -- 172.21.15.76:0/649447245 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc9a00c3060 con 0x7fc9b00af470 2024-09-19T04:18:03.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.098+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9b006e000 con 0x7fc9b00af470 2024-09-19T04:18:03.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9ad7fa640 1 -- 172.21.15.76:0/649447245 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+17042 (secure 0 0 0) 0x7fc9a0016150 con 0x7fc9b00af470 2024-09-19T04:18:03.102 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:03.102 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":79,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:01.761940+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:17:59.452038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:00.756649+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:03.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9900678c0 msgr2=0x7fc990069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:03.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9900678c0 0x7fc990069d80 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0151a60 tx=0x7fc998007cf0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b00af470 msgr2=0x7fc9b0149270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:03.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b00af470 0x7fc9b0149270 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00ac040 tx=0x7fc9a001ace0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 shutdown_connections 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9900678c0 0x7fc990069d80 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9b015c2a0 0x7fc9b00b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc9b015bec0 0x7fc9b00aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 --2- 172.21.15.76:0/649447245 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc9b00af470 0x7fc9b0149270 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 >> 172.21.15.76:0/649447245 conn(0x7fc9b00b1f90 msgr2=0x7fc9b01597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 shutdown_connections 2024-09-19T04:18:03.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.102+0000 7fc9b6372640 1 -- 172.21.15.76:0/649447245 wait complete. 2024-09-19T04:18:03.121 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:18:03.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b2b49640 1 Processor -- start 2024-09-19T04:18:03.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b2b49640 1 -- start start 2024-09-19T04:18:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b2b49640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3ac161a60 0x7fd3ac0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b2b49640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3ac0d32b0 con 0x7fd3ac0d2e90 2024-09-19T04:18:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b2b49640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3ac0d0300 con 0x7fd3ac0cf620 2024-09-19T04:18:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b2b49640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd3ac0cfa90 con 0x7fd3ac161a60 2024-09-19T04:18:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b08e5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3ac161a60 0x7fd3ac0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b08e5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd3ac161a60 0x7fd3ac0d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41394/0 (socket says 172.21.15.76:41394) 2024-09-19T04:18:03.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b08e5640 1 -- 172.21.15.76:0/185451349 learned_addr learned my addr 172.21.15.76:0/185451349 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b08e5640 1 -- 172.21.15.76:0/185451349 >> v1:172.21.15.116:6789/0 conn(0x7fd3ac0cf620 legacy=0x7fd3ac06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b08e5640 1 -- 172.21.15.76:0/185451349 >> v1:172.21.15.76:6789/0 conn(0x7fd3ac0d2e90 legacy=0x7fd3ac0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3ab7fe640 1 -- 172.21.15.76:0/185451349 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3006629427 0 0) 0x7fd3ac0d32b0 con 0x7fd3ac0d2e90 2024-09-19T04:18:03.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.250+0000 7fd3b08e5640 1 -- 172.21.15.76:0/185451349 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac0b0f60 con 0x7fd3ac161a60 2024-09-19T04:18:03.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b08e5640 1 --2- 172.21.15.76:0/185451349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac0d2950 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009870 tx=0x7fd39c01c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1df256dfaaee97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3ab7fe640 1 -- 172.21.15.76:0/185451349 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0b0070 con 0x7fd3ac161a60 2024-09-19T04:18:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3ab7fe640 1 -- 172.21.15.76:0/185451349 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd39c0ab600 con 0x7fd3ac161a60 2024-09-19T04:18:03.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3ab7fe640 1 -- 172.21.15.76:0/185451349 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd39c0ab9d0 con 0x7fd3ac161a60 2024-09-19T04:18:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 -- 172.21.15.76:0/185451349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 msgr2=0x7fd3ac0d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/185451349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac0d2950 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009870 tx=0x7fd39c01c280 comp rx=0 tx=0).stop 2024-09-19T04:18:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 -- 172.21.15.76:0/185451349 shutdown_connections 2024-09-19T04:18:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/185451349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac0d2950 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 -- 172.21.15.76:0/185451349 >> 172.21.15.76:0/185451349 conn(0x7fd3ac08bf90 msgr2=0x7fd3ac09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 -- 172.21.15.76:0/185451349 shutdown_connections 2024-09-19T04:18:03.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 -- 172.21.15.76:0/185451349 wait complete. 2024-09-19T04:18:03.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 Processor -- start 2024-09-19T04:18:03.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.254+0000 7fd3b2b49640 1 -- start start 2024-09-19T04:18:03.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3ac0cf620 0x7fd3ac14eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:03.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd3ac0d2e90 0x7fd3ac1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:03.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:03.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac16bf30 con 0x7fd3ac161a60 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac06d190 con 0x7fd3ac0cf620 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd3ac16c0b0 con 0x7fd3ac0d2e90 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac158be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41406/0 (socket says 172.21.15.76:41406) 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 -- 172.21.15.76:0/1099942541 learned_addr learned my addr 172.21.15.76:0/1099942541 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b08e5640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3ac0cf620 0x7fd3ac14eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 -- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd3ac0d2e90 msgr2=0x7fd3ac1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3abfff640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd3ac0d2e90 0x7fd3ac1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd3ac0d2e90 0x7fd3ac1586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 -- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3ac0cf620 msgr2=0x7fd3ac14eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3ac0cf620 0x7fd3ac14eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 -- 172.21.15.76:0/1099942541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac0ba050 con 0x7fd3ac161a60 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b08e5640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3ac0cf620 0x7fd3ac14eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b10e6640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac158be0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400c780 tx=0x7fd3a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3a97fa640 1 -- 172.21.15.76:0/1099942541 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004b70 con 0x7fd3ac161a60 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3a97fa640 1 -- 172.21.15.76:0/1099942541 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a4004d10 con 0x7fd3ac161a60 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3a97fa640 1 -- 172.21.15.76:0/1099942541 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a400aa90 con 0x7fd3ac161a60 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac1542e0 con 0x7fd3ac161a60 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac14f000 con 0x7fd3ac161a60 2024-09-19T04:18:03.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac1540d0 con 0x7fd3ac161a60 2024-09-19T04:18:03.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3a97fa640 1 -- 172.21.15.76:0/1099942541 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd3a400ac30 con 0x7fd3ac161a60 2024-09-19T04:18:03.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3a97fa640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd3980678c0 0x7fd398069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:03.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3a97fa640 1 -- 172.21.15.76:0/1099942541 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6101+0+0 (secure 0 0 0) 0x7fd3a4086050 con 0x7fd3ac161a60 2024-09-19T04:18:03.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b08e5640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd3980678c0 0x7fd398069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:03.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.258+0000 7fd3b08e5640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd3980678c0 0x7fd398069d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0ad000 tx=0x7fd39c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:03.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.262+0000 7fd3a97fa640 1 -- 172.21.15.76:0/1099942541 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd3a4020070 con 0x7fd3ac161a60 2024-09-19T04:18:03.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:03.466+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd3ac0d11d0 con 0x7fd3ac161a60 2024-09-19T04:18:04.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3a97fa640 1 -- 172.21.15.76:0/1099942541 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7fd3a4056a90 con 0x7fd3ac161a60 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd3980678c0 msgr2=0x7fd398069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd3980678c0 0x7fd398069d80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd39c0ad000 tx=0x7fd39c0a6000 comp rx=0 tx=0).stop 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 msgr2=0x7fd3ac158be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac158be0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fd3a400c780 tx=0x7fd3a400cc40 comp rx=0 tx=0).stop 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 shutdown_connections 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd3980678c0 0x7fd398069d80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd3ac161a60 0x7fd3ac158be0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd3ac0d2e90 0x7fd3ac1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 --2- 172.21.15.76:0/1099942541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3ac0cf620 0x7fd3ac14eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 >> 172.21.15.76:0/1099942541 conn(0x7fd3ac08bf90 msgr2=0x7fd3ac0b1180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 shutdown_connections 2024-09-19T04:18:04.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.442+0000 7fd3b2b49640 1 -- 172.21.15.76:0/1099942541 wait complete. 2024-09-19T04:18:04.468 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T04:18:04.468 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T04:18:04.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3caf5b9640 1 Processor -- start 2024-09-19T04:18:04.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3caf5b9640 1 -- start start 2024-09-19T04:18:04.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3caf5b9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 0x7f3ca80d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3caf5b9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca80d32b0 con 0x7f3ca80cf620 2024-09-19T04:18:04.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3caf5b9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ca80cfad0 con 0x7f3ca80d2e90 2024-09-19T04:18:04.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3caf5b9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca80d0300 con 0x7f3ca8161a60 2024-09-19T04:18:04.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3ca80cf620 0x7f3ca806d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:44988/0 (socket says 172.21.15.76:44988) 2024-09-19T04:18:04.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 -- 172.21.15.76:0/2362117363 learned_addr learned my addr 172.21.15.76:0/2362117363 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:04.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cad355640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 0x7f3ca80d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1261482402 0 0) 0x7f3ca80d32b0 con 0x7f3ca80cf620 2024-09-19T04:18:04.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c94003530 con 0x7f3ca80cf620 2024-09-19T04:18:04.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1636311775 0 0) 0x7f3c94003530 con 0x7f3ca80cf620 2024-09-19T04:18:04.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 msgr2=0x7f3ca80d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:04.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 0x7f3ca80d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 >> v1:172.21.15.116:6789/0 conn(0x7f3ca80d2e90 legacy=0x7f3ca80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca80b0f60 con 0x7f3ca80cf620 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f3ca0001a00 con 0x7f3ca80cf620 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 >> v1:172.21.15.76:6789/0 conn(0x7f3ca80cf620 legacy=0x7f3ca806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 0x7f3c940039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c94003f70 0x7f3c94004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c94004d50 0x7f3c940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca80d0300 con 0x7f3c94004d50 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9400bb60 con 0x7f3c94003f70 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3c9400bd60 con 0x7f3ca8161a60 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c94004d50 0x7f3c940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.602 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cad355640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 0x7f3c940039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cacb54640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c94003f70 0x7f3c94004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 -- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 msgr2=0x7f3c940039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:04.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca8161a60 0x7f3c940039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 -- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c94003f70 msgr2=0x7f3c94004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:04.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c94003f70 0x7f3c94004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.598+0000 7f3cadb56640 1 -- 172.21.15.76:0/2362117363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c94003530 con 0x7f3c94004d50 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3cadb56640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c94004d50 0x7f3c940051d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000a8d0 tx=0x7f3ca001fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39e3b51e41ae123 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000a4a0 con 0x7f3c94004d50 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca00aa720 con 0x7f3c94004d50 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3c9ffff640 1 -- 172.21.15.76:0/2362117363 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00aab10 con 0x7f3c94004d50 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 -- 172.21.15.76:0/2362117363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c94004d50 msgr2=0x7f3c940051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 --2- 172.21.15.76:0/2362117363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c94004d50 0x7f3c940051d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000a8d0 tx=0x7f3ca001fb60 comp rx=0 tx=0).stop 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3cadb56640 1 -- 172.21.15.76:0/2362117363 reap_dead start 2024-09-19T04:18:04.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 -- 172.21.15.76:0/2362117363 shutdown_connections 2024-09-19T04:18:04.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 -- 172.21.15.76:0/2362117363 >> 172.21.15.76:0/2362117363 conn(0x7f3ca808bf90 msgr2=0x7f3ca809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:04.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 -- 172.21.15.76:0/2362117363 shutdown_connections 2024-09-19T04:18:04.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 -- 172.21.15.76:0/2362117363 wait complete. 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 Processor -- start 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.602+0000 7f3caf5b9640 1 -- start start 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ca80b8fd0 0x7f3ca80b7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca80b7280 0x7f3ca80b8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ca80b93b0 0x7f3ca80b9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca806ceb0 con 0x7f3ca80b93b0 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3ca80d0020 con 0x7f3ca80b8fd0 2024-09-19T04:18:04.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3ca806d190 con 0x7f3ca80b7280 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ca80b93b0 0x7f3ca80b9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ca80b93b0 0x7f3ca80b9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41428/0 (socket says 172.21.15.76:41428) 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 -- 172.21.15.76:0/1572440534 learned_addr learned my addr 172.21.15.76:0/1572440534 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cad355640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ca80b8fd0 0x7f3ca80b7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cacb54640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca80b7280 0x7f3ca80b8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 -- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca80b7280 msgr2=0x7f3ca80b8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca80b7280 0x7f3ca80b8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 -- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ca80b8fd0 msgr2=0x7f3ca80b7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ca80b8fd0 0x7f3ca80b7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 -- 172.21.15.76:0/1572440534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ca80cb440 con 0x7f3ca80b93b0 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cadb56640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ca80b93b0 0x7f3ca80b9840 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3ca001fba0 tx=0x7f3c94005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3c9dffb640 1 -- 172.21.15.76:0/1572440534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000a4a0 con 0x7f3ca80b93b0 2024-09-19T04:18:04.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3c9dffb640 1 -- 172.21.15.76:0/1572440534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ca00aa8d0 con 0x7f3ca80b93b0 2024-09-19T04:18:04.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3c9dffb640 1 -- 172.21.15.76:0/1572440534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0002010 con 0x7f3ca80b93b0 2024-09-19T04:18:04.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ca80b8a30 con 0x7f3ca80b93b0 2024-09-19T04:18:04.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ca80cb9c0 con 0x7f3ca80b93b0 2024-09-19T04:18:04.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ca806e3d0 con 0x7f3ca80b93b0 2024-09-19T04:18:04.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3c9dffb640 1 -- 172.21.15.76:0/1572440534 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3ca00aac80 con 0x7f3ca80b93b0 2024-09-19T04:18:04.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3c9dffb640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c8c0678c0 0x7f3c8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:04.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3c9dffb640 1 -- 172.21.15.76:0/1572440534 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f3ca0122fc0 con 0x7f3ca80b93b0 2024-09-19T04:18:04.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.606+0000 7f3cad355640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c8c0678c0 0x7f3c8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:04.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.610+0000 7f3cad355640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c8c0678c0 0x7f3c8c069d80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009b20 tx=0x7f3ca4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:04.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.614+0000 7f3c9dffb640 1 -- 172.21.15.76:0/1572440534 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3ca00ae030 con 0x7f3ca80b93b0 2024-09-19T04:18:04.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:04.810+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3ca806e0f0 con 0x7f3ca80b93b0 2024-09-19T04:18:05.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.450+0000 7f3c9dffb640 1 -- 172.21.15.76:0/1572440534 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7f3ca0019870 con 0x7f3ca80b93b0 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.450+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c8c0678c0 msgr2=0x7f3c8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c8c0678c0 0x7f3c8c069d80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4009b20 tx=0x7f3ca4008040 comp rx=0 tx=0).stop 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ca80b93b0 msgr2=0x7f3ca80b9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ca80b93b0 0x7f3ca80b9840 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f3ca001fba0 tx=0x7f3c94005840 comp rx=0 tx=0).stop 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 shutdown_connections 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c8c0678c0 0x7f3c8c069d80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ca80b93b0 0x7f3ca80b9840 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ca80b7280 0x7f3ca80b8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 --2- 172.21.15.76:0/1572440534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ca80b8fd0 0x7f3ca80b7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 >> 172.21.15.76:0/1572440534 conn(0x7f3ca808bf90 msgr2=0x7f3ca80b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:05.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 shutdown_connections 2024-09-19T04:18:05.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.454+0000 7f3caf5b9640 1 -- 172.21.15.76:0/1572440534 wait complete. 2024-09-19T04:18:05.475 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T04:18:05.475 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5f1839640 1 Processor -- start 2024-09-19T04:18:05.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5f1839640 1 -- start start 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5f1839640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5ec0b39a0 0x7fe5ec06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5f1839640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe5ec06e4b0 0x7fe5ec072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5f1839640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5ec06ec60 con 0x7fe5ec1587a0 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5f1839640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe5ec06f870 con 0x7fe5ec0b39a0 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5f1839640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe5ec06ef40 con 0x7fe5ec06e4b0 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5ea7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5ec0b39a0 0x7fe5ec06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5ea7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5ec0b39a0 0x7fe5ec06df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41440/0 (socket says 172.21.15.76:41440) 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5ea7fc640 1 -- 172.21.15.76:0/2561657868 learned_addr learned my addr 172.21.15.76:0/2561657868 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5ea7fc640 1 -- 172.21.15.76:0/2561657868 >> v2:172.21.15.116:3300/0 conn(0x7fe5ec06e4b0 msgr2=0x7fe5ec072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5ea7fc640 1 --2- 172.21.15.76:0/2561657868 >> v2:172.21.15.116:3300/0 conn(0x7fe5ec06e4b0 0x7fe5ec072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5ea7fc640 1 -- 172.21.15.76:0/2561657868 >> v1:172.21.15.116:6789/0 conn(0x7fe5ec1587a0 legacy=0x7fe5ec0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:05.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.594+0000 7fe5ea7fc640 1 -- 172.21.15.76:0/2561657868 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec06f220 con 0x7fe5ec0b39a0 2024-09-19T04:18:05.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5ea7fc640 1 --2- 172.21.15.76:0/2561657868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec0b39a0 0x7fe5ec06df70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec15e860 tx=0x7fe5d401c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8df9dc0fe60bb53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:05.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5e9ffb640 1 -- 172.21.15.76:0/2561657868 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40a5860 con 0x7fe5ec0b39a0 2024-09-19T04:18:05.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5e9ffb640 1 -- 172.21.15.76:0/2561657868 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5d40a5a00 con 0x7fe5ec0b39a0 2024-09-19T04:18:05.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5e9ffb640 1 -- 172.21.15.76:0/2561657868 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40a5dd0 con 0x7fe5ec0b39a0 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 -- 172.21.15.76:0/2561657868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec0b39a0 msgr2=0x7fe5ec06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 --2- 172.21.15.76:0/2561657868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec0b39a0 0x7fe5ec06df70 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec15e860 tx=0x7fe5d401c980 comp rx=0 tx=0).stop 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 -- 172.21.15.76:0/2561657868 shutdown_connections 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 --2- 172.21.15.76:0/2561657868 >> v2:172.21.15.116:3300/0 conn(0x7fe5ec06e4b0 0x7fe5ec072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 --2- 172.21.15.76:0/2561657868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec0b39a0 0x7fe5ec06df70 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 -- 172.21.15.76:0/2561657868 >> 172.21.15.76:0/2561657868 conn(0x7fe5ec0b1f90 msgr2=0x7fe5ec0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 -- 172.21.15.76:0/2561657868 shutdown_connections 2024-09-19T04:18:05.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 -- 172.21.15.76:0/2561657868 wait complete. 2024-09-19T04:18:05.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.598+0000 7fe5f1839640 1 Processor -- start 2024-09-19T04:18:05.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 -- start start 2024-09-19T04:18:05.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5ec06e4b0 0x7fe5ec156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5ec0b39a0 0x7fe5ec1554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.603 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec1587a0 0x7fe5ec14e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec073030 con 0x7fe5ec1587a0 2024-09-19T04:18:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5ec072eb0 con 0x7fe5ec0b39a0 2024-09-19T04:18:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe5ec0731b0 con 0x7fe5ec06e4b0 2024-09-19T04:18:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec1587a0 0x7fe5ec14e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec1587a0 0x7fe5ec14e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41448/0 (socket says 172.21.15.76:41448) 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 -- 172.21.15.76:0/3610264930 learned_addr learned my addr 172.21.15.76:0/3610264930 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eaffd640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5ec06e4b0 0x7fe5ec156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5ea7fc640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5ec0b39a0 0x7fe5ec1554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 -- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5ec06e4b0 msgr2=0x7fe5ec156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5ec06e4b0 0x7fe5ec156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 -- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5ec0b39a0 msgr2=0x7fe5ec1554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5ec0b39a0 0x7fe5ec1554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 -- 172.21.15.76:0/3610264930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5ec147210 con 0x7fe5ec1587a0 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eaffd640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5ec06e4b0 0x7fe5ec156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eb7fe640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec1587a0 0x7fe5ec14e6f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00b430 tx=0x7fe5dc00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5e3fff640 1 -- 172.21.15.76:0/3610264930 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc015020 con 0x7fe5ec1587a0 2024-09-19T04:18:05.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5e3fff640 1 -- 172.21.15.76:0/3610264930 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5dc011830 con 0x7fe5ec1587a0 2024-09-19T04:18:05.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5ec0ad480 con 0x7fe5ec1587a0 2024-09-19T04:18:05.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5e3fff640 1 -- 172.21.15.76:0/3610264930 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc011c20 con 0x7fe5ec1587a0 2024-09-19T04:18:05.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5ec14c700 con 0x7fe5ec1587a0 2024-09-19T04:18:05.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5ec072160 con 0x7fe5ec1587a0 2024-09-19T04:18:05.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5e3fff640 1 -- 172.21.15.76:0/3610264930 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe5dc018020 con 0x7fe5ec1587a0 2024-09-19T04:18:05.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5e3fff640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5cc0678c0 0x7fe5cc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5eaffd640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5cc0678c0 0x7fe5cc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.602+0000 7fe5e3fff640 1 -- 172.21.15.76:0/3610264930 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fe5dc085d10 con 0x7fe5ec1587a0 2024-09-19T04:18:05.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.606+0000 7fe5eaffd640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5cc0678c0 0x7fe5cc069d80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400a830 tx=0x7fe5e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:05.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.610+0000 7fe5e3fff640 1 -- 172.21.15.76:0/3610264930 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe5dc01f020 con 0x7fe5ec1587a0 2024-09-19T04:18:05.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.798+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5ec070080 con 0x7fe5ec1587a0 2024-09-19T04:18:05.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5e3fff640 1 -- 172.21.15.76:0/3610264930 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5225 (secure 0 0 0) 0x7fe5dc0172c0 con 0x7fe5ec1587a0 2024-09-19T04:18:05.802 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:05.802 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":67,"btime":"2024-09-19T04:18:05:447857+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5336,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:04.811666+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[6783]},"id":5}]} 2024-09-19T04:18:05.802 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 67 2024-09-19T04:18:05.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5cc0678c0 msgr2=0x7fe5cc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5cc0678c0 0x7fe5cc069d80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400a830 tx=0x7fe5e4008040 comp rx=0 tx=0).stop 2024-09-19T04:18:05.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec1587a0 msgr2=0x7fe5ec14e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec1587a0 0x7fe5ec14e6f0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00b430 tx=0x7fe5dc00b8f0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 shutdown_connections 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5cc0678c0 0x7fe5cc069d80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5ec1587a0 0x7fe5ec14e6f0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5ec0b39a0 0x7fe5ec1554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 --2- 172.21.15.76:0/3610264930 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5ec06e4b0 0x7fe5ec156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 >> 172.21.15.76:0/3610264930 conn(0x7fe5ec0b1f90 msgr2=0x7fe5ec160a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 shutdown_connections 2024-09-19T04:18:05.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.802+0000 7fe5f1839640 1 -- 172.21.15.76:0/3610264930 wait complete. 2024-09-19T04:18:05.819 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:05.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.926+0000 7f3c9b458640 1 Processor -- start 2024-09-19T04:18:05.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.926+0000 7f3c9b458640 1 -- start start 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c9b458640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 0x7f3c940ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c9b458640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c940b3290 con 0x7f3c9415c2a0 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c9b458640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c940b3570 con 0x7f3c940af470 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c9b458640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3c940b3850 con 0x7f3c9415bec0 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c989f3640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3c9415c2a0 0x7f3c940aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:44994/0 (socket says 172.21.15.76:44994) 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c989f3640 1 -- 172.21.15.76:0/2993389082 learned_addr learned my addr 172.21.15.76:0/2993389082 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c991f4640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 0x7f3c940ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3675663910 0 0) 0x7f3c940b3290 con 0x7f3c9415c2a0 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3c80003530 con 0x7f3c9415c2a0 2024-09-19T04:18:05.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4158169986 0 0) 0x7f3c80003530 con 0x7f3c9415c2a0 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 msgr2=0x7f3c940ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 0x7f3c940ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 >> v1:172.21.15.116:6789/0 conn(0x7f3c940af470 legacy=0x7f3c9406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c940b3b30 con 0x7f3c9415c2a0 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f3c84001680 con 0x7f3c9415c2a0 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 >> v1:172.21.15.76:6789/0 conn(0x7f3c9415c2a0 legacy=0x7f3c940aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800039f0 0x7f3c80003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 0x7f3c80004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c800051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3c940b3850 con 0x7f3c80004d50 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8000ba80 con 0x7f3c800039f0 2024-09-19T04:18:05.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3c8000bc80 con 0x7f3c9415bec0 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c999f5640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c800051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c991f4640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800039f0 0x7f3c80003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c989f3640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 0x7f3c80004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c999f5640 1 -- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 msgr2=0x7f3c80004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c999f5640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 0x7f3c80004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c999f5640 1 -- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800039f0 msgr2=0x7f3c80003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c999f5640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c800039f0 0x7f3c80003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c999f5640 1 -- 172.21.15.76:0/2993389082 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c80003530 con 0x7f3c80004d50 2024-09-19T04:18:05.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c989f3640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c9415bec0 0x7f3c80004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:05.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.930+0000 7f3c999f5640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c800051d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c001e40 tx=0x7f3c8c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57f213e24c5b0fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:05.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c001a90 con 0x7f3c80004d50 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c8c003b80 con 0x7f3c80004d50 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c8bfff640 1 -- 172.21.15.76:0/2993389082 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c005080 con 0x7f3c80004d50 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- 172.21.15.76:0/2993389082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 msgr2=0x7f3c800051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 --2- 172.21.15.76:0/2993389082 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c800051d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c001e40 tx=0x7f3c8c01c760 comp rx=0 tx=0).stop 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c999f5640 1 -- 172.21.15.76:0/2993389082 reap_dead start 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- 172.21.15.76:0/2993389082 shutdown_connections 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- 172.21.15.76:0/2993389082 >> 172.21.15.76:0/2993389082 conn(0x7f3c940b1f90 msgr2=0x7f3c940b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- 172.21.15.76:0/2993389082 shutdown_connections 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- 172.21.15.76:0/2993389082 wait complete. 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 Processor -- start 2024-09-19T04:18:05.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- start start 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c9414d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c941e5260 0x7f3c9414ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c941e1d70 0x7f3c94150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3c940afc90 con 0x7f3c80004d50 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3c940b40f0 con 0x7f3c941e1d70 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3c940ad200 con 0x7f3c941e5260 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c9414d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c9414d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41480/0 (socket says 172.21.15.76:41480) 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 -- 172.21.15.76:0/2632971308 learned_addr learned my addr 172.21.15.76:0/2632971308 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 -- 172.21.15.76:0/2632971308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c941e5260 msgr2=0x7f3c9414ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c941e5260 0x7f3c9414ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 -- 172.21.15.76:0/2632971308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c941e1d70 msgr2=0x7f3c94150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c941e1d70 0x7f3c94150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 -- 172.21.15.76:0/2632971308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c941e57d0 con 0x7f3c80004d50 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c991f4640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c9414d140 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3c90011610 tx=0x7f3c90011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c89ffb640 1 -- 172.21.15.76:0/2632971308 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90015870 con 0x7f3c80004d50 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c89ffb640 1 -- 172.21.15.76:0/2632971308 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c90015a10 con 0x7f3c80004d50 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c941e5ab0 con 0x7f3c80004d50 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c89ffb640 1 -- 172.21.15.76:0/2632971308 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c90015dc0 con 0x7f3c80004d50 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.934+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c941e6070 con 0x7f3c80004d50 2024-09-19T04:18:05.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.938+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c940ad5e0 con 0x7f3c80004d50 2024-09-19T04:18:05.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.938+0000 7f3c89ffb640 1 -- 172.21.15.76:0/2632971308 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3c9000f460 con 0x7f3c80004d50 2024-09-19T04:18:05.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.938+0000 7f3c89ffb640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c780678c0 0x7f3c78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:05.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.938+0000 7f3c89ffb640 1 -- 172.21.15.76:0/2632971308 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f3c900866d0 con 0x7f3c80004d50 2024-09-19T04:18:05.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.938+0000 7f3c989f3640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c780678c0 0x7f3c78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:05.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.938+0000 7f3c989f3640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c780678c0 0x7f3c78069d80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3c94157f30 tx=0x7f3c84009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:05.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:05.942+0000 7f3c89ffb640 1 -- 172.21.15.76:0/2632971308 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3c90019030 con 0x7f3c80004d50 2024-09-19T04:18:06.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.102+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c9406fcd0 con 0x7f3c80004d50 2024-09-19T04:18:06.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c89ffb640 1 -- 172.21.15.76:0/2632971308 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17029 (secure 0 0 0) 0x7f3c90057110 con 0x7f3c80004d50 2024-09-19T04:18:06.107 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:06.107 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":80,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:03.399539+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:17:59.452038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:00.756649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c780678c0 msgr2=0x7f3c78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c780678c0 0x7f3c78069d80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3c94157f30 tx=0x7f3c84009040 comp rx=0 tx=0).stop 2024-09-19T04:18:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 msgr2=0x7f3c9414d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c9414d140 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f3c90011610 tx=0x7f3c90011ad0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 shutdown_connections 2024-09-19T04:18:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3c780678c0 0x7f3c78069d80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c941e1d70 0x7f3c94150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3c941e5260 0x7f3c9414ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 --2- 172.21.15.76:0/2632971308 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3c80004d50 0x7f3c9414d140 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 >> 172.21.15.76:0/2632971308 conn(0x7f3c940b1f90 msgr2=0x7f3c9415a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 shutdown_connections 2024-09-19T04:18:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.106+0000 7f3c9b458640 1 -- 172.21.15.76:0/2632971308 wait complete. 2024-09-19T04:18:06.125 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:18:06.125 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:18:06.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.242+0000 7fb5cee85640 1 Processor -- start 2024-09-19T04:18:06.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.242+0000 7fb5cee85640 1 -- start start 2024-09-19T04:18:06.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cee85640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c80b39a0 0x7fb5c806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cee85640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5c806e4b0 0x7fb5c8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cee85640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c806ec60 con 0x7fb5c81587a0 2024-09-19T04:18:06.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cee85640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb5c806f870 con 0x7fb5c806e4b0 2024-09-19T04:18:06.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cee85640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c806ef40 con 0x7fb5c80b39a0 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cd422640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5c806e4b0 0x7fb5c8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5c7fff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c80b39a0 0x7fb5c806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cd422640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5c806e4b0 0x7fb5c8072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:39432/0 (socket says 172.21.15.76:39432) 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cd422640 1 -- 172.21.15.76:0/1486413020 learned_addr learned my addr 172.21.15.76:0/1486413020 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5ccc21640 1 --1- 172.21.15.76:0/1486413020 >> v1:172.21.15.116:6789/0 conn(0x7fb5c81587a0 0x7fb5c80b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:46786/0 (socket says 172.21.15.76:46786) 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cd422640 1 -- 172.21.15.76:0/1486413020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c80b39a0 msgr2=0x7fb5c806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cd422640 1 --2- 172.21.15.76:0/1486413020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c80b39a0 0x7fb5c806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cd422640 1 -- 172.21.15.76:0/1486413020 >> v1:172.21.15.116:6789/0 conn(0x7fb5c81587a0 legacy=0x7fb5c80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.246+0000 7fb5cd422640 1 -- 172.21.15.76:0/1486413020 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c806f220 con 0x7fb5c806e4b0 2024-09-19T04:18:06.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 --2- 172.21.15.76:0/1486413020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c806e4b0 0x7fb5c8072160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009e50 tx=0x7fb5c001d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd2452bc02d991ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5c77fe640 1 -- 172.21.15.76:0/1486413020 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a58f0 con 0x7fb5c806e4b0 2024-09-19T04:18:06.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5c77fe640 1 -- 172.21.15.76:0/1486413020 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c00a5a90 con 0x7fb5c806e4b0 2024-09-19T04:18:06.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5c77fe640 1 -- 172.21.15.76:0/1486413020 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a5e60 con 0x7fb5c806e4b0 2024-09-19T04:18:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- 172.21.15.76:0/1486413020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c806e4b0 msgr2=0x7fb5c8072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 --2- 172.21.15.76:0/1486413020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c806e4b0 0x7fb5c8072160 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009e50 tx=0x7fb5c001d460 comp rx=0 tx=0).stop 2024-09-19T04:18:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- 172.21.15.76:0/1486413020 shutdown_connections 2024-09-19T04:18:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 --2- 172.21.15.76:0/1486413020 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c806e4b0 0x7fb5c8072160 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 --2- 172.21.15.76:0/1486413020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c80b39a0 0x7fb5c806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- 172.21.15.76:0/1486413020 >> 172.21.15.76:0/1486413020 conn(0x7fb5c80b1f90 msgr2=0x7fb5c80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:06.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- 172.21.15.76:0/1486413020 shutdown_connections 2024-09-19T04:18:06.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- 172.21.15.76:0/1486413020 wait complete. 2024-09-19T04:18:06.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 Processor -- start 2024-09-19T04:18:06.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- start start 2024-09-19T04:18:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c806e4b0 0x7fb5c81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c80b39a0 0x7fb5c809d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5c81587a0 0x7fb5c809a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c8073030 con 0x7fb5c81587a0 2024-09-19T04:18:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c8072eb0 con 0x7fb5c80b39a0 2024-09-19T04:18:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cee85640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c80731b0 con 0x7fb5c806e4b0 2024-09-19T04:18:06.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5c81587a0 0x7fb5c809a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5c81587a0 0x7fb5c809a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41484/0 (socket says 172.21.15.76:41484) 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 -- 172.21.15.76:0/769573183 learned_addr learned my addr 172.21.15.76:0/769573183 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5c7fff640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c80b39a0 0x7fb5c809d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5ccc21640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c806e4b0 0x7fb5c81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 -- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c806e4b0 msgr2=0x7fb5c81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c806e4b0 0x7fb5c81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 -- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c80b39a0 msgr2=0x7fb5c809d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c80b39a0 0x7fb5c809d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 -- 172.21.15.76:0/769573183 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c809a380 con 0x7fb5c81587a0 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.250+0000 7fb5cd422640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5c81587a0 0x7fb5c809a880 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb5c001a630 tx=0x7fb5c00197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5c57fa640 1 -- 172.21.15.76:0/769573183 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a58f0 con 0x7fb5c81587a0 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5c57fa640 1 -- 172.21.15.76:0/769573183 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c0007c70 con 0x7fb5c81587a0 2024-09-19T04:18:06.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c81454b0 con 0x7fb5c81587a0 2024-09-19T04:18:06.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5c57fa640 1 -- 172.21.15.76:0/769573183 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0009510 con 0x7fb5c81587a0 2024-09-19T04:18:06.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c8145a70 con 0x7fb5c81587a0 2024-09-19T04:18:06.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5c8072160 con 0x7fb5c81587a0 2024-09-19T04:18:06.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5c57fa640 1 -- 172.21.15.76:0/769573183 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5c00c0020 con 0x7fb5c81587a0 2024-09-19T04:18:06.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5c57fa640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5ccc21640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5c57fa640 1 -- 172.21.15.76:0/769573183 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb5c0120c10 con 0x7fb5c81587a0 2024-09-19T04:18:06.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.254+0000 7fb5ccc21640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8158540 tx=0x7fb5b8007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.258+0000 7fb5c57fa640 1 -- 172.21.15.76:0/769573183 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb5c00c7020 con 0x7fb5c81587a0 2024-09-19T04:18:06.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.418+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5c8070080 con 0x7fb5a80678c0 2024-09-19T04:18:06.435 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.434+0000 7fb5c57fa640 1 -- 172.21.15.76:0/769573183 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252005 (secure 0 0 0) 0x7fb5c8070080 con 0x7fb5a80678c0 2024-09-19T04:18:06.438 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:06.439 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 msgr2=0x7fb5a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8158540 tx=0x7fb5b8007cf0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5c81587a0 msgr2=0x7fb5c809a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5c81587a0 0x7fb5c809a880 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb5c001a630 tx=0x7fb5c00197a0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 shutdown_connections 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5c81587a0 0x7fb5c809a880 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c80b39a0 0x7fb5c809d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 --2- 172.21.15.76:0/769573183 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c806e4b0 0x7fb5c81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 >> 172.21.15.76:0/769573183 conn(0x7fb5c80b1f90 msgr2=0x7fb5c815eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:06.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 shutdown_connections 2024-09-19T04:18:06.442 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.438+0000 7fb5cee85640 1 -- 172.21.15.76:0/769573183 wait complete. 2024-09-19T04:18:06.457 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-19T04:18:05.828929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245704,"kb_used_data":8652,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729016,"statfs":{"total":773094113280,"available":772842512384,"internally_reserved":0,"allocated":8859648,"data_stored":4704130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.442864"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445222+0000","last_change":"2024-09-19T04:18:02.441306+0000","last_active":"2024-09-19T04:18:02.445222+0000","last_peered":"2024-09-19T04:18:02.445222+0000","last_clean":"2024-09-19T04:18:02.445222+0000","last_became_active":"2024-09-19T04:18:02.440575+0000","last_became_peered":"2024-09-19T04:18:02.440575+0000","last_unstale":"2024-09-19T04:18:02.445222+0000","last_undegraded":"2024-09-19T04:18:02.445222+0000","last_fullsized":"2024-09-19T04:18:02.445222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:34.152269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459175+0000","last_change":"2024-09-19T04:18:02.452772+0000","last_active":"2024-09-19T04:18:02.459175+0000","last_peered":"2024-09-19T04:18:02.459175+0000","last_clean":"2024-09-19T04:18:02.459175+0000","last_became_active":"2024-09-19T04:18:02.445274+0000","last_became_peered":"2024-09-19T04:18:02.445274+0000","last_unstale":"2024-09-19T04:18:02.459175+0000","last_undegraded":"2024-09-19T04:18:02.459175+0000","last_fullsized":"2024-09-19T04:18:02.459175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:29.325204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448950+0000","last_change":"2024-09-19T04:18:02.444475+0000","last_active":"2024-09-19T04:18:02.448950+0000","last_peered":"2024-09-19T04:18:02.448950+0000","last_clean":"2024-09-19T04:18:02.448950+0000","last_became_active":"2024-09-19T04:18:02.444096+0000","last_became_peered":"2024-09-19T04:18:02.444096+0000","last_unstale":"2024-09-19T04:18:02.448950+0000","last_undegraded":"2024-09-19T04:18:02.448950+0000","last_fullsized":"2024-09-19T04:18:02.448950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:23.831213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.455437+0000","last_change":"2024-09-19T04:18:02.450165+0000","last_active":"2024-09-19T04:18:02.455437+0000","last_peered":"2024-09-19T04:18:02.455437+0000","last_clean":"2024-09-19T04:18:02.455437+0000","last_became_active":"2024-09-19T04:18:02.448643+0000","last_became_peered":"2024-09-19T04:18:02.448643+0000","last_unstale":"2024-09-19T04:18:02.455437+0000","last_undegraded":"2024-09-19T04:18:02.455437+0000","last_fullsized":"2024-09-19T04:18:02.455437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:56.594682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.422394+0000","last_change":"2024-09-19T04:18:02.415450+0000","last_active":"2024-09-19T04:18:02.422394+0000","last_peered":"2024-09-19T04:18:02.422394+0000","last_clean":"2024-09-19T04:18:02.422394+0000","last_became_active":"2024-09-19T04:18:02.413682+0000","last_became_peered":"2024-09-19T04:18:02.413682+0000","last_unstale":"2024-09-19T04:18:02.422394+0000","last_undegraded":"2024-09-19T04:18:02.422394+0000","last_fullsized":"2024-09-19T04:18:02.422394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.955712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.461127+0000","last_change":"2024-09-19T04:18:02.453536+0000","last_active":"2024-09-19T04:18:02.461127+0000","last_peered":"2024-09-19T04:18:02.461127+0000","last_clean":"2024-09-19T04:18:02.461127+0000","last_became_active":"2024-09-19T04:18:02.453270+0000","last_became_peered":"2024-09-19T04:18:02.453270+0000","last_unstale":"2024-09-19T04:18:02.461127+0000","last_undegraded":"2024-09-19T04:18:02.461127+0000","last_fullsized":"2024-09-19T04:18:02.461127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:29.935876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437323+0000","last_change":"2024-09-19T04:18:02.430489+0000","last_active":"2024-09-19T04:18:02.437323+0000","last_peered":"2024-09-19T04:18:02.437323+0000","last_clean":"2024-09-19T04:18:02.437323+0000","last_became_active":"2024-09-19T04:18:02.429546+0000","last_became_peered":"2024-09-19T04:18:02.429546+0000","last_unstale":"2024-09-19T04:18:02.437323+0000","last_undegraded":"2024-09-19T04:18:02.437323+0000","last_fullsized":"2024-09-19T04:18:02.437323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:41.471819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423046+0000","last_change":"2024-09-19T04:18:02.413963+0000","last_active":"2024-09-19T04:18:02.423046+0000","last_peered":"2024-09-19T04:18:02.423046+0000","last_clean":"2024-09-19T04:18:02.423046+0000","last_became_active":"2024-09-19T04:18:02.413423+0000","last_became_peered":"2024-09-19T04:18:02.413423+0000","last_unstale":"2024-09-19T04:18:02.423046+0000","last_undegraded":"2024-09-19T04:18:02.423046+0000","last_fullsized":"2024-09-19T04:18:02.423046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:00.783384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.464522+0000","last_change":"2024-09-19T04:18:02.447090+0000","last_active":"2024-09-19T04:18:02.464522+0000","last_peered":"2024-09-19T04:18:02.464522+0000","last_clean":"2024-09-19T04:18:02.464522+0000","last_became_active":"2024-09-19T04:18:02.444837+0000","last_became_peered":"2024-09-19T04:18:02.444837+0000","last_unstale":"2024-09-19T04:18:02.464522+0000","last_undegraded":"2024-09-19T04:18:02.464522+0000","last_fullsized":"2024-09-19T04:18:02.464522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:05.698611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.443317+0000","last_change":"2024-09-19T04:18:02.440751+0000","last_active":"2024-09-19T04:18:02.443317+0000","last_peered":"2024-09-19T04:18:02.443317+0000","last_clean":"2024-09-19T04:18:02.443317+0000","last_became_active":"2024-09-19T04:18:02.440037+0000","last_became_peered":"2024-09-19T04:18:02.440037+0000","last_unstale":"2024-09-19T04:18:02.443317+0000","last_undegraded":"2024-09-19T04:18:02.443317+0000","last_fullsized":"2024-09-19T04:18:02.443317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:54.166915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459122+0000","last_change":"2024-09-19T04:18:02.451259+0000","last_active":"2024-09-19T04:18:02.459122+0000","last_peered":"2024-09-19T04:18:02.459122+0000","last_clean":"2024-09-19T04:18:02.459122+0000","last_became_active":"2024-09-19T04:18:02.450761+0000","last_became_peered":"2024-09-19T04:18:02.450761+0000","last_unstale":"2024-09-19T04:18:02.459122+0000","last_undegraded":"2024-09-19T04:18:02.459122+0000","last_fullsized":"2024-09-19T04:18:02.459122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:41.436795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.455185+0000","last_change":"2024-09-19T04:18:02.447129+0000","last_active":"2024-09-19T04:18:02.455185+0000","last_peered":"2024-09-19T04:18:02.455185+0000","last_clean":"2024-09-19T04:18:02.455185+0000","last_became_active":"2024-09-19T04:18:02.446335+0000","last_became_peered":"2024-09-19T04:18:02.446335+0000","last_unstale":"2024-09-19T04:18:02.455185+0000","last_undegraded":"2024-09-19T04:18:02.455185+0000","last_fullsized":"2024-09-19T04:18:02.455185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:42:20.333888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.442352+0000","last_change":"2024-09-19T04:18:02.435230+0000","last_active":"2024-09-19T04:18:02.442352+0000","last_peered":"2024-09-19T04:18:02.442352+0000","last_clean":"2024-09-19T04:18:02.442352+0000","last_became_active":"2024-09-19T04:18:02.434857+0000","last_became_peered":"2024-09-19T04:18:02.434857+0000","last_unstale":"2024-09-19T04:18:02.442352+0000","last_undegraded":"2024-09-19T04:18:02.442352+0000","last_fullsized":"2024-09-19T04:18:02.442352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:21.965193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457914+0000","last_change":"2024-09-19T04:18:02.452107+0000","last_active":"2024-09-19T04:18:02.457914+0000","last_peered":"2024-09-19T04:18:02.457914+0000","last_clean":"2024-09-19T04:18:02.457914+0000","last_became_active":"2024-09-19T04:18:02.451439+0000","last_became_peered":"2024-09-19T04:18:02.451439+0000","last_unstale":"2024-09-19T04:18:02.457914+0000","last_undegraded":"2024-09-19T04:18:02.457914+0000","last_fullsized":"2024-09-19T04:18:02.457914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:40:33.491809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.453390+0000","last_change":"2024-09-19T04:18:02.441369+0000","last_active":"2024-09-19T04:18:02.453390+0000","last_peered":"2024-09-19T04:18:02.453390+0000","last_clean":"2024-09-19T04:18:02.453390+0000","last_became_active":"2024-09-19T04:18:02.440610+0000","last_became_peered":"2024-09-19T04:18:02.440610+0000","last_unstale":"2024-09-19T04:18:02.453390+0000","last_undegraded":"2024-09-19T04:18:02.453390+0000","last_fullsized":"2024-09-19T04:18:02.453390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:33.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435167+0000","last_change":"2024-09-19T04:18:02.429760+0000","last_active":"2024-09-19T04:18:02.435167+0000","last_peered":"2024-09-19T04:18:02.435167+0000","last_clean":"2024-09-19T04:18:02.435167+0000","last_became_active":"2024-09-19T04:18:02.428809+0000","last_became_peered":"2024-09-19T04:18:02.428809+0000","last_unstale":"2024-09-19T04:18:02.435167+0000","last_undegraded":"2024-09-19T04:18:02.435167+0000","last_fullsized":"2024-09-19T04:18:02.435167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.500411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.426941+0000","last_change":"2024-09-19T04:18:02.424318+0000","last_active":"2024-09-19T04:18:02.426941+0000","last_peered":"2024-09-19T04:18:02.426941+0000","last_clean":"2024-09-19T04:18:02.426941+0000","last_became_active":"2024-09-19T04:18:02.421773+0000","last_became_peered":"2024-09-19T04:18:02.421773+0000","last_unstale":"2024-09-19T04:18:02.426941+0000","last_undegraded":"2024-09-19T04:18:02.426941+0000","last_fullsized":"2024-09-19T04:18:02.426941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:48.455240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445616+0000","last_change":"2024-09-19T04:18:02.439881+0000","last_active":"2024-09-19T04:18:02.445616+0000","last_peered":"2024-09-19T04:18:02.445616+0000","last_clean":"2024-09-19T04:18:02.445616+0000","last_became_active":"2024-09-19T04:18:02.439558+0000","last_became_peered":"2024-09-19T04:18:02.439558+0000","last_unstale":"2024-09-19T04:18:02.445616+0000","last_undegraded":"2024-09-19T04:18:02.445616+0000","last_fullsized":"2024-09-19T04:18:02.445616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:32.786261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445992+0000","last_change":"2024-09-19T04:18:02.443178+0000","last_active":"2024-09-19T04:18:02.445992+0000","last_peered":"2024-09-19T04:18:02.445992+0000","last_clean":"2024-09-19T04:18:02.445992+0000","last_became_active":"2024-09-19T04:18:02.442893+0000","last_became_peered":"2024-09-19T04:18:02.442893+0000","last_unstale":"2024-09-19T04:18:02.445992+0000","last_undegraded":"2024-09-19T04:18:02.445992+0000","last_fullsized":"2024-09-19T04:18:02.445992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:36.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435208+0000","last_change":"2024-09-19T04:18:02.427154+0000","last_active":"2024-09-19T04:18:02.435208+0000","last_peered":"2024-09-19T04:18:02.435208+0000","last_clean":"2024-09-19T04:18:02.435208+0000","last_became_active":"2024-09-19T04:18:02.424520+0000","last_became_peered":"2024-09-19T04:18:02.424520+0000","last_unstale":"2024-09-19T04:18:02.435208+0000","last_undegraded":"2024-09-19T04:18:02.435208+0000","last_fullsized":"2024-09-19T04:18:02.435208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:49.099149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.449270+0000","last_change":"2024-09-19T04:18:02.437662+0000","last_active":"2024-09-19T04:18:02.449270+0000","last_peered":"2024-09-19T04:18:02.449270+0000","last_clean":"2024-09-19T04:18:02.449270+0000","last_became_active":"2024-09-19T04:18:02.436581+0000","last_became_peered":"2024-09-19T04:18:02.436581+0000","last_unstale":"2024-09-19T04:18:02.449270+0000","last_undegraded":"2024-09-19T04:18:02.449270+0000","last_fullsized":"2024-09-19T04:18:02.449270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:37.644518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448360+0000","last_change":"2024-09-19T04:18:02.442819+0000","last_active":"2024-09-19T04:18:02.448360+0000","last_peered":"2024-09-19T04:18:02.448360+0000","last_clean":"2024-09-19T04:18:02.448360+0000","last_became_active":"2024-09-19T04:18:02.442506+0000","last_became_peered":"2024-09-19T04:18:02.442506+0000","last_unstale":"2024-09-19T04:18:02.448360+0000","last_undegraded":"2024-09-19T04:18:02.448360+0000","last_fullsized":"2024-09-19T04:18:02.448360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:36.064475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445879+0000","last_change":"2024-09-19T04:18:02.426728+0000","last_active":"2024-09-19T04:18:02.445879+0000","last_peered":"2024-09-19T04:18:02.445879+0000","last_clean":"2024-09-19T04:18:02.445879+0000","last_became_active":"2024-09-19T04:18:02.426015+0000","last_became_peered":"2024-09-19T04:18:02.426015+0000","last_unstale":"2024-09-19T04:18:02.445879+0000","last_undegraded":"2024-09-19T04:18:02.445879+0000","last_fullsized":"2024-09-19T04:18:02.445879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:01.707512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463945+0000","last_change":"2024-09-19T04:18:02.462616+0000","last_active":"2024-09-19T04:18:02.463945+0000","last_peered":"2024-09-19T04:18:02.463945+0000","last_clean":"2024-09-19T04:18:02.463945+0000","last_became_active":"2024-09-19T04:18:02.462265+0000","last_became_peered":"2024-09-19T04:18:02.462265+0000","last_unstale":"2024-09-19T04:18:02.463945+0000","last_undegraded":"2024-09-19T04:18:02.463945+0000","last_fullsized":"2024-09-19T04:18:02.463945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:46.255336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463886+0000","last_change":"2024-09-19T04:18:02.462486+0000","last_active":"2024-09-19T04:18:02.463886+0000","last_peered":"2024-09-19T04:18:02.463886+0000","last_clean":"2024-09-19T04:18:02.463886+0000","last_became_active":"2024-09-19T04:18:02.462170+0000","last_became_peered":"2024-09-19T04:18:02.462170+0000","last_unstale":"2024-09-19T04:18:02.463886+0000","last_undegraded":"2024-09-19T04:18:02.463886+0000","last_fullsized":"2024-09-19T04:18:02.463886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:24.469786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.454928+0000","last_change":"2024-09-19T04:18:02.447293+0000","last_active":"2024-09-19T04:18:02.454928+0000","last_peered":"2024-09-19T04:18:02.454928+0000","last_clean":"2024-09-19T04:18:02.454928+0000","last_became_active":"2024-09-19T04:18:02.446533+0000","last_became_peered":"2024-09-19T04:18:02.446533+0000","last_unstale":"2024-09-19T04:18:02.454928+0000","last_undegraded":"2024-09-19T04:18:02.454928+0000","last_fullsized":"2024-09-19T04:18:02.454928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:35:13.954242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435407+0000","last_change":"2024-09-19T04:18:02.426760+0000","last_active":"2024-09-19T04:18:02.435407+0000","last_peered":"2024-09-19T04:18:02.435407+0000","last_clean":"2024-09-19T04:18:02.435407+0000","last_became_active":"2024-09-19T04:18:02.425891+0000","last_became_peered":"2024-09-19T04:18:02.425891+0000","last_unstale":"2024-09-19T04:18:02.435407+0000","last_undegraded":"2024-09-19T04:18:02.435407+0000","last_fullsized":"2024-09-19T04:18:02.435407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:24.217697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.442342+0000","last_change":"2024-09-19T04:18:02.431035+0000","last_active":"2024-09-19T04:18:02.442342+0000","last_peered":"2024-09-19T04:18:02.442342+0000","last_clean":"2024-09-19T04:18:02.442342+0000","last_became_active":"2024-09-19T04:18:02.430249+0000","last_became_peered":"2024-09-19T04:18:02.430249+0000","last_unstale":"2024-09-19T04:18:02.442342+0000","last_undegraded":"2024-09-19T04:18:02.442342+0000","last_fullsized":"2024-09-19T04:18:02.442342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:16.185194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.465751+0000","last_change":"2024-09-19T04:18:02.464622+0000","last_active":"2024-09-19T04:18:02.465751+0000","last_peered":"2024-09-19T04:18:02.465751+0000","last_clean":"2024-09-19T04:18:02.465751+0000","last_became_active":"2024-09-19T04:18:02.464360+0000","last_became_peered":"2024-09-19T04:18:02.464360+0000","last_unstale":"2024-09-19T04:18:02.465751+0000","last_undegraded":"2024-09-19T04:18:02.465751+0000","last_fullsized":"2024-09-19T04:18:02.465751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:36.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.460456+0000","last_change":"2024-09-19T04:18:02.458572+0000","last_active":"2024-09-19T04:18:02.460456+0000","last_peered":"2024-09-19T04:18:02.460456+0000","last_clean":"2024-09-19T04:18:02.460456+0000","last_became_active":"2024-09-19T04:18:02.458004+0000","last_became_peered":"2024-09-19T04:18:02.458004+0000","last_unstale":"2024-09-19T04:18:02.460456+0000","last_undegraded":"2024-09-19T04:18:02.460456+0000","last_fullsized":"2024-09-19T04:18:02.460456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:42.734883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.441587+0000","last_change":"2024-09-19T04:18:02.431051+0000","last_active":"2024-09-19T04:18:02.441587+0000","last_peered":"2024-09-19T04:18:02.441587+0000","last_clean":"2024-09-19T04:18:02.441587+0000","last_became_active":"2024-09-19T04:18:02.429026+0000","last_became_peered":"2024-09-19T04:18:02.429026+0000","last_unstale":"2024-09-19T04:18:02.441587+0000","last_undegraded":"2024-09-19T04:18:02.441587+0000","last_fullsized":"2024-09-19T04:18:02.441587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.338194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435106+0000","last_change":"2024-09-19T04:18:02.427556+0000","last_active":"2024-09-19T04:18:02.435106+0000","last_peered":"2024-09-19T04:18:02.435106+0000","last_clean":"2024-09-19T04:18:02.435106+0000","last_became_active":"2024-09-19T04:18:02.426909+0000","last_became_peered":"2024-09-19T04:18:02.426909+0000","last_unstale":"2024-09-19T04:18:02.435106+0000","last_undegraded":"2024-09-19T04:18:02.435106+0000","last_fullsized":"2024-09-19T04:18:02.435106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:40.267633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445774+0000","last_change":"2024-09-19T04:18:02.439897+0000","last_active":"2024-09-19T04:18:02.445774+0000","last_peered":"2024-09-19T04:18:02.445774+0000","last_clean":"2024-09-19T04:18:02.445774+0000","last_became_active":"2024-09-19T04:18:02.439592+0000","last_became_peered":"2024-09-19T04:18:02.439592+0000","last_unstale":"2024-09-19T04:18:02.445774+0000","last_undegraded":"2024-09-19T04:18:02.445774+0000","last_fullsized":"2024-09-19T04:18:02.445774+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:41.471819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457741+0000","last_change":"2024-09-19T04:18:01.419252+0000","last_active":"2024-09-19T04:18:02.457741+0000","last_peered":"2024-09-19T04:18:02.457741+0000","last_clean":"2024-09-19T04:18:02.457741+0000","last_became_active":"2024-09-19T04:18:01.418935+0000","last_became_peered":"2024-09-19T04:18:01.418935+0000","last_unstale":"2024-09-19T04:18:02.457741+0000","last_undegraded":"2024-09-19T04:18:02.457741+0000","last_fullsized":"2024-09-19T04:18:02.457741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:24.046686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.453416+0000","last_change":"2024-09-19T04:18:02.442719+0000","last_active":"2024-09-19T04:18:02.453416+0000","last_peered":"2024-09-19T04:18:02.453416+0000","last_clean":"2024-09-19T04:18:02.453416+0000","last_became_active":"2024-09-19T04:18:02.442060+0000","last_became_peered":"2024-09-19T04:18:02.442060+0000","last_unstale":"2024-09-19T04:18:02.453416+0000","last_undegraded":"2024-09-19T04:18:02.453416+0000","last_fullsized":"2024-09-19T04:18:02.453416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:06:01.348680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463582+0000","last_change":"2024-09-19T04:18:01.444434+0000","last_active":"2024-09-19T04:18:02.463582+0000","last_peered":"2024-09-19T04:18:02.463582+0000","last_clean":"2024-09-19T04:18:02.463582+0000","last_became_active":"2024-09-19T04:18:01.443795+0000","last_became_peered":"2024-09-19T04:18:01.443795+0000","last_unstale":"2024-09-19T04:18:02.463582+0000","last_undegraded":"2024-09-19T04:18:02.463582+0000","last_fullsized":"2024-09-19T04:18:02.463582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:06.327779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.467314+0000","last_change":"2024-09-19T04:18:02.452288+0000","last_active":"2024-09-19T04:18:02.467314+0000","last_peered":"2024-09-19T04:18:02.467314+0000","last_clean":"2024-09-19T04:18:02.467314+0000","last_became_active":"2024-09-19T04:18:02.448700+0000","last_became_peered":"2024-09-19T04:18:02.448700+0000","last_unstale":"2024-09-19T04:18:02.467314+0000","last_undegraded":"2024-09-19T04:18:02.467314+0000","last_fullsized":"2024-09-19T04:18:02.467314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:41.710546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463468+0000","last_change":"2024-09-19T04:18:01.444895+0000","last_active":"2024-09-19T04:18:02.463468+0000","last_peered":"2024-09-19T04:18:02.463468+0000","last_clean":"2024-09-19T04:18:02.463468+0000","last_became_active":"2024-09-19T04:18:01.443565+0000","last_became_peered":"2024-09-19T04:18:01.443565+0000","last_unstale":"2024-09-19T04:18:02.463468+0000","last_undegraded":"2024-09-19T04:18:02.463468+0000","last_fullsized":"2024-09-19T04:18:02.463468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:35.483830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.446098+0000","last_change":"2024-09-19T04:18:02.437114+0000","last_active":"2024-09-19T04:18:02.446098+0000","last_peered":"2024-09-19T04:18:02.446098+0000","last_clean":"2024-09-19T04:18:02.446098+0000","last_became_active":"2024-09-19T04:18:02.436436+0000","last_became_peered":"2024-09-19T04:18:02.436436+0000","last_unstale":"2024-09-19T04:18:02.446098+0000","last_undegraded":"2024-09-19T04:18:02.446098+0000","last_fullsized":"2024-09-19T04:18:02.446098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:06.063752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463495+0000","last_change":"2024-09-19T04:18:01.433700+0000","last_active":"2024-09-19T04:18:02.463495+0000","last_peered":"2024-09-19T04:18:02.463495+0000","last_clean":"2024-09-19T04:18:02.463495+0000","last_became_active":"2024-09-19T04:18:01.432148+0000","last_became_peered":"2024-09-19T04:18:01.432148+0000","last_unstale":"2024-09-19T04:18:02.463495+0000","last_undegraded":"2024-09-19T04:18:02.463495+0000","last_fullsized":"2024-09-19T04:18:02.463495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.700851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459088+0000","last_change":"2024-09-19T04:18:02.444971+0000","last_active":"2024-09-19T04:18:02.459088+0000","last_peered":"2024-09-19T04:18:02.459088+0000","last_clean":"2024-09-19T04:18:02.459088+0000","last_became_active":"2024-09-19T04:18:02.444421+0000","last_became_peered":"2024-09-19T04:18:02.444421+0000","last_unstale":"2024-09-19T04:18:02.459088+0000","last_undegraded":"2024-09-19T04:18:02.459088+0000","last_fullsized":"2024-09-19T04:18:02.459088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:52.876014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468636+0000","last_change":"2024-09-19T04:18:01.432369+0000","last_active":"2024-09-19T04:18:02.468636+0000","last_peered":"2024-09-19T04:18:02.468636+0000","last_clean":"2024-09-19T04:18:02.468636+0000","last_became_active":"2024-09-19T04:18:01.431423+0000","last_became_peered":"2024-09-19T04:18:01.431423+0000","last_unstale":"2024-09-19T04:18:02.468636+0000","last_undegraded":"2024-09-19T04:18:02.468636+0000","last_fullsized":"2024-09-19T04:18:02.468636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:29.307422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459161+0000","last_change":"2024-09-19T04:18:02.452072+0000","last_active":"2024-09-19T04:18:02.459161+0000","last_peered":"2024-09-19T04:18:02.459161+0000","last_clean":"2024-09-19T04:18:02.459161+0000","last_became_active":"2024-09-19T04:18:02.451432+0000","last_became_peered":"2024-09-19T04:18:02.451432+0000","last_unstale":"2024-09-19T04:18:02.459161+0000","last_undegraded":"2024-09-19T04:18:02.459161+0000","last_fullsized":"2024-09-19T04:18:02.459161+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:15.323523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.462563+0000","last_change":"2024-09-19T04:18:01.419082+0000","last_active":"2024-09-19T04:18:02.462563+0000","last_peered":"2024-09-19T04:18:02.462563+0000","last_clean":"2024-09-19T04:18:02.462563+0000","last_became_active":"2024-09-19T04:18:01.418294+0000","last_became_peered":"2024-09-19T04:18:01.418294+0000","last_unstale":"2024-09-19T04:18:02.462563+0000","last_undegraded":"2024-09-19T04:18:02.462563+0000","last_fullsized":"2024-09-19T04:18:02.462563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:13.846251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.464465+0000","last_change":"2024-09-19T04:18:02.445661+0000","last_active":"2024-09-19T04:18:02.464465+0000","last_peered":"2024-09-19T04:18:02.464465+0000","last_clean":"2024-09-19T04:18:02.464465+0000","last_became_active":"2024-09-19T04:18:02.444841+0000","last_became_peered":"2024-09-19T04:18:02.444841+0000","last_unstale":"2024-09-19T04:18:02.464465+0000","last_undegraded":"2024-09-19T04:18:02.464465+0000","last_fullsized":"2024-09-19T04:18:02.464465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:57:31.059273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.426791+0000","last_change":"2024-09-19T04:18:01.444532+0000","last_active":"2024-09-19T04:18:02.426791+0000","last_peered":"2024-09-19T04:18:02.426791+0000","last_clean":"2024-09-19T04:18:02.426791+0000","last_became_active":"2024-09-19T04:18:01.444261+0000","last_became_peered":"2024-09-19T04:18:01.444261+0000","last_unstale":"2024-09-19T04:18:02.426791+0000","last_undegraded":"2024-09-19T04:18:02.426791+0000","last_fullsized":"2024-09-19T04:18:02.426791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:02.152154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437147+0000","last_change":"2024-09-19T04:18:02.426626+0000","last_active":"2024-09-19T04:18:02.437147+0000","last_peered":"2024-09-19T04:18:02.437147+0000","last_clean":"2024-09-19T04:18:02.437147+0000","last_became_active":"2024-09-19T04:18:02.425410+0000","last_became_peered":"2024-09-19T04:18:02.425410+0000","last_unstale":"2024-09-19T04:18:02.437147+0000","last_undegraded":"2024-09-19T04:18:02.437147+0000","last_fullsized":"2024-09-19T04:18:02.437147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:32.786261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406054+0000","last_change":"2024-09-19T04:18:01.429905+0000","last_active":"2024-09-19T04:18:02.406054+0000","last_peered":"2024-09-19T04:18:02.406054+0000","last_clean":"2024-09-19T04:18:02.406054+0000","last_became_active":"2024-09-19T04:18:01.429563+0000","last_became_peered":"2024-09-19T04:18:01.429563+0000","last_unstale":"2024-09-19T04:18:02.406054+0000","last_undegraded":"2024-09-19T04:18:02.406054+0000","last_fullsized":"2024-09-19T04:18:02.406054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:12:37.679637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463687+0000","last_change":"2024-09-19T04:18:02.460855+0000","last_active":"2024-09-19T04:18:02.463687+0000","last_peered":"2024-09-19T04:18:02.463687+0000","last_clean":"2024-09-19T04:18:02.463687+0000","last_became_active":"2024-09-19T04:18:02.460559+0000","last_became_peered":"2024-09-19T04:18:02.460559+0000","last_unstale":"2024-09-19T04:18:02.463687+0000","last_undegraded":"2024-09-19T04:18:02.463687+0000","last_fullsized":"2024-09-19T04:18:02.463687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:42.134180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.407407+0000","last_change":"2024-09-19T04:18:01.444708+0000","last_active":"2024-09-19T04:18:02.407407+0000","last_peered":"2024-09-19T04:18:02.407407+0000","last_clean":"2024-09-19T04:18:02.407407+0000","last_became_active":"2024-09-19T04:18:01.444016+0000","last_became_peered":"2024-09-19T04:18:01.444016+0000","last_unstale":"2024-09-19T04:18:02.407407+0000","last_undegraded":"2024-09-19T04:18:02.407407+0000","last_fullsized":"2024-09-19T04:18:02.407407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:05.837521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.455560+0000","last_change":"2024-09-19T04:18:02.448557+0000","last_active":"2024-09-19T04:18:02.455560+0000","last_peered":"2024-09-19T04:18:02.455560+0000","last_clean":"2024-09-19T04:18:02.455560+0000","last_became_active":"2024-09-19T04:18:02.447853+0000","last_became_peered":"2024-09-19T04:18:02.447853+0000","last_unstale":"2024-09-19T04:18:02.455560+0000","last_undegraded":"2024-09-19T04:18:02.455560+0000","last_fullsized":"2024-09-19T04:18:02.455560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:58.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423242+0000","last_change":"2024-09-19T04:18:01.428836+0000","last_active":"2024-09-19T04:18:02.423242+0000","last_peered":"2024-09-19T04:18:02.423242+0000","last_clean":"2024-09-19T04:18:02.423242+0000","last_became_active":"2024-09-19T04:18:01.428122+0000","last_became_peered":"2024-09-19T04:18:01.428122+0000","last_unstale":"2024-09-19T04:18:02.423242+0000","last_undegraded":"2024-09-19T04:18:02.423242+0000","last_fullsized":"2024-09-19T04:18:02.423242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:00.969190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457829+0000","last_change":"2024-09-19T04:18:02.446619+0000","last_active":"2024-09-19T04:18:02.457829+0000","last_peered":"2024-09-19T04:18:02.457829+0000","last_clean":"2024-09-19T04:18:02.457829+0000","last_became_active":"2024-09-19T04:18:02.446365+0000","last_became_peered":"2024-09-19T04:18:02.446365+0000","last_unstale":"2024-09-19T04:18:02.457829+0000","last_undegraded":"2024-09-19T04:18:02.457829+0000","last_fullsized":"2024-09-19T04:18:02.457829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:38.619995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.458331+0000","last_change":"2024-09-19T04:18:01.415693+0000","last_active":"2024-09-19T04:18:02.458331+0000","last_peered":"2024-09-19T04:18:02.458331+0000","last_clean":"2024-09-19T04:18:02.458331+0000","last_became_active":"2024-09-19T04:18:01.415409+0000","last_became_peered":"2024-09-19T04:18:01.415409+0000","last_unstale":"2024-09-19T04:18:02.458331+0000","last_undegraded":"2024-09-19T04:18:02.458331+0000","last_fullsized":"2024-09-19T04:18:02.458331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:29.237363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448918+0000","last_change":"2024-09-19T04:18:02.444586+0000","last_active":"2024-09-19T04:18:02.448918+0000","last_peered":"2024-09-19T04:18:02.448918+0000","last_clean":"2024-09-19T04:18:02.448918+0000","last_became_active":"2024-09-19T04:18:02.444274+0000","last_became_peered":"2024-09-19T04:18:02.444274+0000","last_unstale":"2024-09-19T04:18:02.448918+0000","last_undegraded":"2024-09-19T04:18:02.448918+0000","last_fullsized":"2024-09-19T04:18:02.448918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:02.081069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.472266+0000","last_change":"2024-09-19T04:18:01.440750+0000","last_active":"2024-09-19T04:18:02.472266+0000","last_peered":"2024-09-19T04:18:02.472266+0000","last_clean":"2024-09-19T04:18:02.472266+0000","last_became_active":"2024-09-19T04:18:01.439842+0000","last_became_peered":"2024-09-19T04:18:01.439842+0000","last_unstale":"2024-09-19T04:18:02.472266+0000","last_undegraded":"2024-09-19T04:18:02.472266+0000","last_fullsized":"2024-09-19T04:18:02.472266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:50.685522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.443208+0000","last_change":"2024-09-19T04:18:02.439877+0000","last_active":"2024-09-19T04:18:02.443208+0000","last_peered":"2024-09-19T04:18:02.443208+0000","last_clean":"2024-09-19T04:18:02.443208+0000","last_became_active":"2024-09-19T04:18:02.439168+0000","last_became_peered":"2024-09-19T04:18:02.439168+0000","last_unstale":"2024-09-19T04:18:02.443208+0000","last_undegraded":"2024-09-19T04:18:02.443208+0000","last_fullsized":"2024-09-19T04:18:02.443208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:17.908329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468115+0000","last_change":"2024-09-19T04:18:01.433865+0000","last_active":"2024-09-19T04:18:02.468115+0000","last_peered":"2024-09-19T04:18:02.468115+0000","last_clean":"2024-09-19T04:18:02.468115+0000","last_became_active":"2024-09-19T04:18:01.433084+0000","last_became_peered":"2024-09-19T04:18:01.433084+0000","last_unstale":"2024-09-19T04:18:02.468115+0000","last_undegraded":"2024-09-19T04:18:02.468115+0000","last_fullsized":"2024-09-19T04:18:02.468115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:47.965198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.438107+0000","last_change":"2024-09-19T04:18:02.431066+0000","last_active":"2024-09-19T04:18:02.438107+0000","last_peered":"2024-09-19T04:18:02.438107+0000","last_clean":"2024-09-19T04:18:02.438107+0000","last_became_active":"2024-09-19T04:18:02.430018+0000","last_became_peered":"2024-09-19T04:18:02.430018+0000","last_unstale":"2024-09-19T04:18:02.438107+0000","last_undegraded":"2024-09-19T04:18:02.438107+0000","last_fullsized":"2024-09-19T04:18:02.438107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:23.831213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406091+0000","last_change":"2024-09-19T04:18:01.437255+0000","last_active":"2024-09-19T04:18:02.406091+0000","last_peered":"2024-09-19T04:18:02.406091+0000","last_clean":"2024-09-19T04:18:02.406091+0000","last_became_active":"2024-09-19T04:18:01.436414+0000","last_became_peered":"2024-09-19T04:18:01.436414+0000","last_unstale":"2024-09-19T04:18:02.406091+0000","last_undegraded":"2024-09-19T04:18:02.406091+0000","last_fullsized":"2024-09-19T04:18:02.406091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:34.210598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.444278+0000","last_change":"2024-09-19T04:18:02.440292+0000","last_active":"2024-09-19T04:18:02.444278+0000","last_peered":"2024-09-19T04:18:02.444278+0000","last_clean":"2024-09-19T04:18:02.444278+0000","last_became_active":"2024-09-19T04:18:02.439573+0000","last_became_peered":"2024-09-19T04:18:02.439573+0000","last_unstale":"2024-09-19T04:18:02.444278+0000","last_undegraded":"2024-09-19T04:18:02.444278+0000","last_fullsized":"2024-09-19T04:18:02.444278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:56.528286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406207+0000","last_change":"2024-09-19T04:18:01.419197+0000","last_active":"2024-09-19T04:18:02.406207+0000","last_peered":"2024-09-19T04:18:02.406207+0000","last_clean":"2024-09-19T04:18:02.406207+0000","last_became_active":"2024-09-19T04:18:01.418894+0000","last_became_peered":"2024-09-19T04:18:01.418894+0000","last_unstale":"2024-09-19T04:18:02.406207+0000","last_undegraded":"2024-09-19T04:18:02.406207+0000","last_fullsized":"2024-09-19T04:18:02.406207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:55.740984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437245+0000","last_change":"2024-09-19T04:18:02.432056+0000","last_active":"2024-09-19T04:18:02.437245+0000","last_peered":"2024-09-19T04:18:02.437245+0000","last_clean":"2024-09-19T04:18:02.437245+0000","last_became_active":"2024-09-19T04:18:02.431365+0000","last_became_peered":"2024-09-19T04:18:02.431365+0000","last_unstale":"2024-09-19T04:18:02.437245+0000","last_undegraded":"2024-09-19T04:18:02.437245+0000","last_fullsized":"2024-09-19T04:18:02.437245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:02.081069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.467046+0000","last_change":"2024-09-19T04:18:01.433055+0000","last_active":"2024-09-19T04:18:02.467046+0000","last_peered":"2024-09-19T04:18:02.467046+0000","last_clean":"2024-09-19T04:18:02.467046+0000","last_became_active":"2024-09-19T04:18:01.432780+0000","last_became_peered":"2024-09-19T04:18:01.432780+0000","last_unstale":"2024-09-19T04:18:02.467046+0000","last_undegraded":"2024-09-19T04:18:02.467046+0000","last_fullsized":"2024-09-19T04:18:02.467046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:42.620115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.442724+0000","last_change":"2024-09-19T04:18:02.440343+0000","last_active":"2024-09-19T04:18:02.442724+0000","last_peered":"2024-09-19T04:18:02.442724+0000","last_clean":"2024-09-19T04:18:02.442724+0000","last_became_active":"2024-09-19T04:18:02.440069+0000","last_became_peered":"2024-09-19T04:18:02.440069+0000","last_unstale":"2024-09-19T04:18:02.442724+0000","last_undegraded":"2024-09-19T04:18:02.442724+0000","last_fullsized":"2024-09-19T04:18:02.442724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:46.255336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.462327+0000","last_change":"2024-09-19T04:18:01.439501+0000","last_active":"2024-09-19T04:18:02.462327+0000","last_peered":"2024-09-19T04:18:02.462327+0000","last_clean":"2024-09-19T04:18:02.462327+0000","last_became_active":"2024-09-19T04:18:01.438869+0000","last_became_peered":"2024-09-19T04:18:01.438869+0000","last_unstale":"2024-09-19T04:18:02.462327+0000","last_undegraded":"2024-09-19T04:18:02.462327+0000","last_fullsized":"2024-09-19T04:18:02.462327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:28.945268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.439744+0000","last_change":"2024-09-19T04:18:02.433490+0000","last_active":"2024-09-19T04:18:02.439744+0000","last_peered":"2024-09-19T04:18:02.439744+0000","last_clean":"2024-09-19T04:18:02.439744+0000","last_became_active":"2024-09-19T04:18:02.430975+0000","last_became_peered":"2024-09-19T04:18:02.430975+0000","last_unstale":"2024-09-19T04:18:02.439744+0000","last_undegraded":"2024-09-19T04:18:02.439744+0000","last_fullsized":"2024-09-19T04:18:02.439744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:03.274491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468992+0000","last_change":"2024-09-19T04:18:01.437357+0000","last_active":"2024-09-19T04:18:02.468992+0000","last_peered":"2024-09-19T04:18:02.468992+0000","last_clean":"2024-09-19T04:18:02.468992+0000","last_became_active":"2024-09-19T04:18:01.436248+0000","last_became_peered":"2024-09-19T04:18:01.436248+0000","last_unstale":"2024-09-19T04:18:02.468992+0000","last_undegraded":"2024-09-19T04:18:02.468992+0000","last_fullsized":"2024-09-19T04:18:02.468992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:52.562016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.436754+0000","last_change":"2024-09-19T04:18:02.430509+0000","last_active":"2024-09-19T04:18:02.436754+0000","last_peered":"2024-09-19T04:18:02.436754+0000","last_clean":"2024-09-19T04:18:02.436754+0000","last_became_active":"2024-09-19T04:18:02.429845+0000","last_became_peered":"2024-09-19T04:18:02.429845+0000","last_unstale":"2024-09-19T04:18:02.436754+0000","last_undegraded":"2024-09-19T04:18:02.436754+0000","last_fullsized":"2024-09-19T04:18:02.436754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:52.670666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468113+0000","last_change":"2024-09-19T04:18:01.429737+0000","last_active":"2024-09-19T04:18:02.468113+0000","last_peered":"2024-09-19T04:18:02.468113+0000","last_clean":"2024-09-19T04:18:02.468113+0000","last_became_active":"2024-09-19T04:18:01.429512+0000","last_became_peered":"2024-09-19T04:18:01.429512+0000","last_unstale":"2024-09-19T04:18:02.468113+0000","last_undegraded":"2024-09-19T04:18:02.468113+0000","last_fullsized":"2024-09-19T04:18:02.468113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:17.225238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.440828+0000","last_change":"2024-09-19T04:18:02.431032+0000","last_active":"2024-09-19T04:18:02.440828+0000","last_peered":"2024-09-19T04:18:02.440828+0000","last_clean":"2024-09-19T04:18:02.440828+0000","last_became_active":"2024-09-19T04:18:02.430493+0000","last_became_peered":"2024-09-19T04:18:02.430493+0000","last_unstale":"2024-09-19T04:18:02.440828+0000","last_undegraded":"2024-09-19T04:18:02.440828+0000","last_fullsized":"2024-09-19T04:18:02.440828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:59.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.414752+0000","last_change":"2024-09-19T04:18:01.412564+0000","last_active":"2024-09-19T04:18:02.414752+0000","last_peered":"2024-09-19T04:18:02.414752+0000","last_clean":"2024-09-19T04:18:02.414752+0000","last_became_active":"2024-09-19T04:18:01.411102+0000","last_became_peered":"2024-09-19T04:18:01.411102+0000","last_unstale":"2024-09-19T04:18:02.414752+0000","last_undegraded":"2024-09-19T04:18:02.414752+0000","last_fullsized":"2024-09-19T04:18:02.414752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.700851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459188+0000","last_change":"2024-09-19T04:18:02.445199+0000","last_active":"2024-09-19T04:18:02.459188+0000","last_peered":"2024-09-19T04:18:02.459188+0000","last_clean":"2024-09-19T04:18:02.459188+0000","last_became_active":"2024-09-19T04:18:02.444474+0000","last_became_peered":"2024-09-19T04:18:02.444474+0000","last_unstale":"2024-09-19T04:18:02.459188+0000","last_undegraded":"2024-09-19T04:18:02.459188+0000","last_fullsized":"2024-09-19T04:18:02.459188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:05.343882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457368+0000","last_change":"2024-09-19T04:18:01.416161+0000","last_active":"2024-09-19T04:18:02.457368+0000","last_peered":"2024-09-19T04:18:02.457368+0000","last_clean":"2024-09-19T04:18:02.457368+0000","last_became_active":"2024-09-19T04:18:01.415873+0000","last_became_peered":"2024-09-19T04:18:01.415873+0000","last_unstale":"2024-09-19T04:18:02.457368+0000","last_undegraded":"2024-09-19T04:18:02.457368+0000","last_fullsized":"2024-09-19T04:18:02.457368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:14.313827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.470986+0000","last_change":"2024-09-19T04:18:02.469895+0000","last_active":"2024-09-19T04:18:02.470986+0000","last_peered":"2024-09-19T04:18:02.470986+0000","last_clean":"2024-09-19T04:18:02.470986+0000","last_became_active":"2024-09-19T04:18:02.469624+0000","last_became_peered":"2024-09-19T04:18:02.469624+0000","last_unstale":"2024-09-19T04:18:02.470986+0000","last_undegraded":"2024-09-19T04:18:02.470986+0000","last_fullsized":"2024-09-19T04:18:02.470986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:38.619995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423965+0000","last_change":"2024-09-19T04:18:01.440551+0000","last_active":"2024-09-19T04:18:02.423965+0000","last_peered":"2024-09-19T04:18:02.423965+0000","last_clean":"2024-09-19T04:18:02.423965+0000","last_became_active":"2024-09-19T04:18:01.440276+0000","last_became_peered":"2024-09-19T04:18:01.440276+0000","last_unstale":"2024-09-19T04:18:02.423965+0000","last_undegraded":"2024-09-19T04:18:02.423965+0000","last_fullsized":"2024-09-19T04:18:02.423965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:41.124484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437199+0000","last_change":"2024-09-19T04:18:02.431584+0000","last_active":"2024-09-19T04:18:02.437199+0000","last_peered":"2024-09-19T04:18:02.437199+0000","last_clean":"2024-09-19T04:18:02.437199+0000","last_became_active":"2024-09-19T04:18:02.430854+0000","last_became_peered":"2024-09-19T04:18:02.430854+0000","last_unstale":"2024-09-19T04:18:02.437199+0000","last_undegraded":"2024-09-19T04:18:02.437199+0000","last_fullsized":"2024-09-19T04:18:02.437199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:05:25.365046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423123+0000","last_change":"2024-09-19T04:18:01.432749+0000","last_active":"2024-09-19T04:18:02.423123+0000","last_peered":"2024-09-19T04:18:02.423123+0000","last_clean":"2024-09-19T04:18:02.423123+0000","last_became_active":"2024-09-19T04:18:01.432481+0000","last_became_peered":"2024-09-19T04:18:01.432481+0000","last_unstale":"2024-09-19T04:18:02.423123+0000","last_undegraded":"2024-09-19T04:18:02.423123+0000","last_fullsized":"2024-09-19T04:18:02.423123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:28.315508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445743+0000","last_change":"2024-09-19T04:18:02.436751+0000","last_active":"2024-09-19T04:18:02.445743+0000","last_peered":"2024-09-19T04:18:02.445743+0000","last_clean":"2024-09-19T04:18:02.445743+0000","last_became_active":"2024-09-19T04:18:02.429731+0000","last_became_peered":"2024-09-19T04:18:02.429731+0000","last_unstale":"2024-09-19T04:18:02.445743+0000","last_undegraded":"2024-09-19T04:18:02.445743+0000","last_fullsized":"2024-09-19T04:18:02.445743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:57.429637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.460326+0000","last_change":"2024-09-19T04:18:01.420205+0000","last_active":"2024-09-19T04:18:02.460326+0000","last_peered":"2024-09-19T04:18:02.460326+0000","last_clean":"2024-09-19T04:18:02.460326+0000","last_became_active":"2024-09-19T04:18:01.419399+0000","last_became_peered":"2024-09-19T04:18:01.419399+0000","last_unstale":"2024-09-19T04:18:02.460326+0000","last_undegraded":"2024-09-19T04:18:02.460326+0000","last_fullsized":"2024-09-19T04:18:02.460326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:19.000405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.461338+0000","last_change":"2024-09-19T04:18:02.442173+0000","last_active":"2024-09-19T04:18:02.461338+0000","last_peered":"2024-09-19T04:18:02.461338+0000","last_clean":"2024-09-19T04:18:02.461338+0000","last_became_active":"2024-09-19T04:18:02.441273+0000","last_became_peered":"2024-09-19T04:18:02.441273+0000","last_unstale":"2024-09-19T04:18:02.461338+0000","last_undegraded":"2024-09-19T04:18:02.461338+0000","last_fullsized":"2024-09-19T04:18:02.461338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.504879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.436770+0000","last_change":"2024-09-19T04:18:01.410252+0000","last_active":"2024-09-19T04:18:02.436770+0000","last_peered":"2024-09-19T04:18:02.436770+0000","last_clean":"2024-09-19T04:18:02.436770+0000","last_became_active":"2024-09-19T04:18:01.409901+0000","last_became_peered":"2024-09-19T04:18:01.409901+0000","last_unstale":"2024-09-19T04:18:02.436770+0000","last_undegraded":"2024-09-19T04:18:02.436770+0000","last_fullsized":"2024-09-19T04:18:02.436770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:35.911828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.467601+0000","last_change":"2024-09-19T04:18:02.460997+0000","last_active":"2024-09-19T04:18:02.467601+0000","last_peered":"2024-09-19T04:18:02.467601+0000","last_clean":"2024-09-19T04:18:02.467601+0000","last_became_active":"2024-09-19T04:18:02.460141+0000","last_became_peered":"2024-09-19T04:18:02.460141+0000","last_unstale":"2024-09-19T04:18:02.467601+0000","last_undegraded":"2024-09-19T04:18:02.467601+0000","last_fullsized":"2024-09-19T04:18:02.467601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:51.695218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.407725+0000","last_change":"2024-09-19T04:18:01.444473+0000","last_active":"2024-09-19T04:18:02.407725+0000","last_peered":"2024-09-19T04:18:02.407725+0000","last_clean":"2024-09-19T04:18:02.407725+0000","last_became_active":"2024-09-19T04:18:01.443651+0000","last_became_peered":"2024-09-19T04:18:01.443651+0000","last_unstale":"2024-09-19T04:18:02.407725+0000","last_undegraded":"2024-09-19T04:18:02.407725+0000","last_fullsized":"2024-09-19T04:18:02.407725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:45.921951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463545+0000","last_change":"2024-09-19T04:18:02.461892+0000","last_active":"2024-09-19T04:18:02.463545+0000","last_peered":"2024-09-19T04:18:02.463545+0000","last_clean":"2024-09-19T04:18:02.463545+0000","last_became_active":"2024-09-19T04:18:02.460635+0000","last_became_peered":"2024-09-19T04:18:02.460635+0000","last_unstale":"2024-09-19T04:18:02.463545+0000","last_undegraded":"2024-09-19T04:18:02.463545+0000","last_fullsized":"2024-09-19T04:18:02.463545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:59.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406694+0000","last_change":"2024-09-19T04:18:01.437617+0000","last_active":"2024-09-19T04:18:02.406694+0000","last_peered":"2024-09-19T04:18:02.406694+0000","last_clean":"2024-09-19T04:18:02.406694+0000","last_became_active":"2024-09-19T04:18:01.436825+0000","last_became_peered":"2024-09-19T04:18:01.436825+0000","last_unstale":"2024-09-19T04:18:02.406694+0000","last_undegraded":"2024-09-19T04:18:02.406694+0000","last_fullsized":"2024-09-19T04:18:02.406694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:53.838198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.441808+0000","last_change":"2024-09-19T04:18:02.434122+0000","last_active":"2024-09-19T04:18:02.441808+0000","last_peered":"2024-09-19T04:18:02.441808+0000","last_clean":"2024-09-19T04:18:02.441808+0000","last_became_active":"2024-09-19T04:18:02.433034+0000","last_became_peered":"2024-09-19T04:18:02.433034+0000","last_unstale":"2024-09-19T04:18:02.441808+0000","last_undegraded":"2024-09-19T04:18:02.441808+0000","last_fullsized":"2024-09-19T04:18:02.441808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:45.835099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.409645+0000","last_change":"2024-09-19T04:18:01.434879+0000","last_active":"2024-09-19T04:18:02.409645+0000","last_peered":"2024-09-19T04:18:02.409645+0000","last_clean":"2024-09-19T04:18:02.409645+0000","last_became_active":"2024-09-19T04:18:01.432576+0000","last_became_peered":"2024-09-19T04:18:01.432576+0000","last_unstale":"2024-09-19T04:18:02.409645+0000","last_undegraded":"2024-09-19T04:18:02.409645+0000","last_fullsized":"2024-09-19T04:18:02.409645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:04.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.438052+0000","last_change":"2024-09-19T04:18:02.426424+0000","last_active":"2024-09-19T04:18:02.438052+0000","last_peered":"2024-09-19T04:18:02.438052+0000","last_clean":"2024-09-19T04:18:02.438052+0000","last_became_active":"2024-09-19T04:18:02.425433+0000","last_became_peered":"2024-09-19T04:18:02.425433+0000","last_unstale":"2024-09-19T04:18:02.438052+0000","last_undegraded":"2024-09-19T04:18:02.438052+0000","last_fullsized":"2024-09-19T04:18:02.438052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.338194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406755+0000","last_change":"2024-09-19T04:18:01.419456+0000","last_active":"2024-09-19T04:18:02.406755+0000","last_peered":"2024-09-19T04:18:02.406755+0000","last_clean":"2024-09-19T04:18:02.406755+0000","last_became_active":"2024-09-19T04:18:01.419159+0000","last_became_peered":"2024-09-19T04:18:01.419159+0000","last_unstale":"2024-09-19T04:18:02.406755+0000","last_undegraded":"2024-09-19T04:18:02.406755+0000","last_fullsized":"2024-09-19T04:18:02.406755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:45.576305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448879+0000","last_change":"2024-09-19T04:18:02.444568+0000","last_active":"2024-09-19T04:18:02.448879+0000","last_peered":"2024-09-19T04:18:02.448879+0000","last_clean":"2024-09-19T04:18:02.448879+0000","last_became_active":"2024-09-19T04:18:02.444205+0000","last_became_peered":"2024-09-19T04:18:02.444205+0000","last_unstale":"2024-09-19T04:18:02.448879+0000","last_undegraded":"2024-09-19T04:18:02.448879+0000","last_fullsized":"2024-09-19T04:18:02.448879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:05:25.365046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.413897+0000","last_change":"2024-09-19T04:18:01.428145+0000","last_active":"2024-09-19T04:18:02.413897+0000","last_peered":"2024-09-19T04:18:02.413897+0000","last_clean":"2024-09-19T04:18:02.413897+0000","last_became_active":"2024-09-19T04:18:01.425709+0000","last_became_peered":"2024-09-19T04:18:01.425709+0000","last_unstale":"2024-09-19T04:18:02.413897+0000","last_undegraded":"2024-09-19T04:18:02.413897+0000","last_fullsized":"2024-09-19T04:18:02.413897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:58.560859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.440445+0000","last_change":"2024-09-19T04:18:02.428097+0000","last_active":"2024-09-19T04:18:02.440445+0000","last_peered":"2024-09-19T04:18:02.440445+0000","last_clean":"2024-09-19T04:18:02.440445+0000","last_became_active":"2024-09-19T04:18:02.427332+0000","last_became_peered":"2024-09-19T04:18:02.427332+0000","last_unstale":"2024-09-19T04:18:02.440445+0000","last_undegraded":"2024-09-19T04:18:02.440445+0000","last_fullsized":"2024-09-19T04:18:02.440445+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:59.570554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.422366+0000","last_change":"2024-09-19T04:18:01.416528+0000","last_active":"2024-09-19T04:18:02.422366+0000","last_peered":"2024-09-19T04:18:02.422366+0000","last_clean":"2024-09-19T04:18:02.422366+0000","last_became_active":"2024-09-19T04:18:01.416163+0000","last_became_peered":"2024-09-19T04:18:01.416163+0000","last_unstale":"2024-09-19T04:18:02.422366+0000","last_undegraded":"2024-09-19T04:18:02.422366+0000","last_fullsized":"2024-09-19T04:18:02.422366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:50.685522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435463+0000","last_change":"2024-09-19T04:18:02.431813+0000","last_active":"2024-09-19T04:18:02.435463+0000","last_peered":"2024-09-19T04:18:02.435463+0000","last_clean":"2024-09-19T04:18:02.435463+0000","last_became_active":"2024-09-19T04:18:02.431068+0000","last_became_peered":"2024-09-19T04:18:02.431068+0000","last_unstale":"2024-09-19T04:18:02.435463+0000","last_undegraded":"2024-09-19T04:18:02.435463+0000","last_fullsized":"2024-09-19T04:18:02.435463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.903407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437474+0000","last_change":"2024-09-19T04:18:01.435376+0000","last_active":"2024-09-19T04:18:02.437474+0000","last_peered":"2024-09-19T04:18:02.437474+0000","last_clean":"2024-09-19T04:18:02.437474+0000","last_became_active":"2024-09-19T04:18:01.435106+0000","last_became_peered":"2024-09-19T04:18:01.435106+0000","last_unstale":"2024-09-19T04:18:02.437474+0000","last_undegraded":"2024-09-19T04:18:02.437474+0000","last_fullsized":"2024-09-19T04:18:02.437474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:23.460090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":278,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.405144+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:18:02.405144+0000","last_peered":"2024-09-19T04:18:02.405144+0000","last_clean":"2024-09-19T04:18:02.405144+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:18:02.405144+0000","last_undegraded":"2024-09-19T04:18:02.405144+0000","last_fullsized":"2024-09-19T04:18:02.405144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":1027039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":950272,"data_stored":436671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":443872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":950272,"data_stored":436671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":444420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":974848,"data_stored":444580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542206,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":436671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":1034206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:18:06.461 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:18:06.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3f59c6640 1 Processor -- start 2024-09-19T04:18:06.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3f59c6640 1 -- start start 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3f59c6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa3f006fef0 0x7fa3f006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3f59c6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f00b3d50 0x7fa3f00b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3f59c6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3f006ee50 con 0x7fa3f0073e60 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3f59c6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa3f00b46c0 con 0x7fa3f006fef0 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3f59c6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f006f130 con 0x7fa3f00b3d50 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3eeffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa3f0073e60 0x7fa3f015b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45010/0 (socket says 172.21.15.76:45010) 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3eeffd640 1 -- 172.21.15.76:0/443011934 learned_addr learned my addr 172.21.15.76:0/443011934 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3ee7fc640 1 --2- 172.21.15.76:0/443011934 >> v2:172.21.15.76:3300/0 conn(0x7fa3f006fef0 0x7fa3f006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3ef7fe640 1 --2- 172.21.15.76:0/443011934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f00b3d50 0x7fa3f00b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3ee7fc640 1 -- 172.21.15.76:0/443011934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f00b3d50 msgr2=0x7fa3f00b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:06.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3ee7fc640 1 --2- 172.21.15.76:0/443011934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f00b3d50 0x7fa3f00b4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3ee7fc640 1 -- 172.21.15.76:0/443011934 >> v1:172.21.15.76:6789/0 conn(0x7fa3f0073e60 legacy=0x7fa3f015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3ee7fc640 1 -- 172.21.15.76:0/443011934 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f006f410 con 0x7fa3f006fef0 2024-09-19T04:18:06.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.626+0000 7fa3edffb640 1 -- 172.21.15.76:0/443011934 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1581788192 0 0) 0x7fa3f006ee50 con 0x7fa3f0073e60 2024-09-19T04:18:06.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3ee7fc640 1 --2- 172.21.15.76:0/443011934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f006fef0 0x7fa3f006e800 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00097c0 tx=0x7fa3e001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cbc5c2bf1bb264 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3edffb640 1 -- 172.21.15.76:0/443011934 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00b0070 con 0x7fa3f006fef0 2024-09-19T04:18:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3edffb640 1 -- 172.21.15.76:0/443011934 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e00ab600 con 0x7fa3f006fef0 2024-09-19T04:18:06.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3edffb640 1 -- 172.21.15.76:0/443011934 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00ab9d0 con 0x7fa3f006fef0 2024-09-19T04:18:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 -- 172.21.15.76:0/443011934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f006fef0 msgr2=0x7fa3f006e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/443011934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f006fef0 0x7fa3f006e800 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fa3e00097c0 tx=0x7fa3e001c410 comp rx=0 tx=0).stop 2024-09-19T04:18:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 -- 172.21.15.76:0/443011934 shutdown_connections 2024-09-19T04:18:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/443011934 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f00b3d50 0x7fa3f00b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/443011934 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f006fef0 0x7fa3f006e800 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 -- 172.21.15.76:0/443011934 >> 172.21.15.76:0/443011934 conn(0x7fa3f00b1f90 msgr2=0x7fa3f00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:06.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 -- 172.21.15.76:0/443011934 shutdown_connections 2024-09-19T04:18:06.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.630+0000 7fa3f59c6640 1 -- 172.21.15.76:0/443011934 wait complete. 2024-09-19T04:18:06.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 Processor -- start 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 -- start start 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f006fef0 0x7fa3f014c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f0073e60 0x7fa3f0150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3f00b3d50 0x7fa3f01512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f00729a0 con 0x7fa3f0073e60 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0072820 con 0x7fa3f00b3d50 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f0072b20 con 0x7fa3f006fef0 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f0073e60 0x7fa3f0150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f0073e60 0x7fa3f0150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41502/0 (socket says 172.21.15.76:41502) 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 -- 172.21.15.76:0/3420260404 learned_addr learned my addr 172.21.15.76:0/3420260404 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3eeffd640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f006fef0 0x7fa3f014c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ef7fe640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3f00b3d50 0x7fa3f01512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 -- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f006fef0 msgr2=0x7fa3f014c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f006fef0 0x7fa3f014c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 -- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3f00b3d50 msgr2=0x7fa3f01512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3f00b3d50 0x7fa3f01512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 -- 172.21.15.76:0/3420260404 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f00af550 con 0x7fa3f0073e60 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ef7fe640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3f00b3d50 0x7fa3f01512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3eeffd640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f006fef0 0x7fa3f014c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3ee7fc640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f0073e60 0x7fa3f0150d70 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa3e001a040 tx=0x7fa3e00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3d7fff640 1 -- 172.21.15.76:0/3420260404 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00b8020 con 0x7fa3f0073e60 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3d7fff640 1 -- 172.21.15.76:0/3420260404 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3e00b4410 con 0x7fa3f0073e60 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f00ae4b0 con 0x7fa3f0073e60 2024-09-19T04:18:06.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3d7fff640 1 -- 172.21.15.76:0/3420260404 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e00b0040 con 0x7fa3f0073e60 2024-09-19T04:18:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f00aea70 con 0x7fa3f0073e60 2024-09-19T04:18:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f006e800 con 0x7fa3f0073e60 2024-09-19T04:18:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.634+0000 7fa3d7fff640 1 -- 172.21.15.76:0/3420260404 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa3e00bb020 con 0x7fa3f0073e60 2024-09-19T04:18:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.638+0000 7fa3d7fff640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa3dc0678c0 0x7fa3dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:06.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.638+0000 7fa3d7fff640 1 -- 172.21.15.76:0/3420260404 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa3e011f9f0 con 0x7fa3f0073e60 2024-09-19T04:18:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.638+0000 7fa3eeffd640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa3dc0678c0 0x7fa3dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:06.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.638+0000 7fa3eeffd640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa3dc0678c0 0x7fa3dc069d80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4000f70 tx=0x7fa3e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:06.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.642+0000 7fa3d7fff640 1 -- 172.21.15.76:0/3420260404 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa3e00c2030 con 0x7fa3f0073e60 2024-09-19T04:18:06.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.786+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3f0152f90 con 0x7fa3dc0678c0 2024-09-19T04:18:06.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.806+0000 7fa3d7fff640 1 -- 172.21.15.76:0/3420260404 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252005 (secure 0 0 0) 0x7fa3f0152f90 con 0x7fa3dc0678c0 2024-09-19T04:18:06.810 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:06.812 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:18:06.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa3dc0678c0 msgr2=0x7fa3dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa3dc0678c0 0x7fa3dc069d80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4000f70 tx=0x7fa3e4009040 comp rx=0 tx=0).stop 2024-09-19T04:18:06.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f0073e60 msgr2=0x7fa3f0150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:06.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f0073e60 0x7fa3f0150d70 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fa3e001a040 tx=0x7fa3e00191b0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 shutdown_connections 2024-09-19T04:18:06.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa3dc0678c0 0x7fa3dc069d80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa3f00b3d50 0x7fa3f01512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa3f0073e60 0x7fa3f0150d70 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 --2- 172.21.15.76:0/3420260404 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa3f006fef0 0x7fa3f014c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:06.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 >> 172.21.15.76:0/3420260404 conn(0x7fa3f00b1f90 msgr2=0x7fa3f0158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:06.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 shutdown_connections 2024-09-19T04:18:06.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.810+0000 7fa3f59c6640 1 -- 172.21.15.76:0/3420260404 wait complete. 2024-09-19T04:18:06.827 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-19T04:18:05.828929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":39,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":245704,"kb_used_data":8652,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754729016,"statfs":{"total":773094113280,"available":772842512384,"internally_reserved":0,"allocated":8859648,"data_stored":4704130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":44,"apply_latency_ms":44,"commit_latency_ns":44000000,"apply_latency_ns":44000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.442864"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445222+0000","last_change":"2024-09-19T04:18:02.441306+0000","last_active":"2024-09-19T04:18:02.445222+0000","last_peered":"2024-09-19T04:18:02.445222+0000","last_clean":"2024-09-19T04:18:02.445222+0000","last_became_active":"2024-09-19T04:18:02.440575+0000","last_became_peered":"2024-09-19T04:18:02.440575+0000","last_unstale":"2024-09-19T04:18:02.445222+0000","last_undegraded":"2024-09-19T04:18:02.445222+0000","last_fullsized":"2024-09-19T04:18:02.445222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:57:34.152269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459175+0000","last_change":"2024-09-19T04:18:02.452772+0000","last_active":"2024-09-19T04:18:02.459175+0000","last_peered":"2024-09-19T04:18:02.459175+0000","last_clean":"2024-09-19T04:18:02.459175+0000","last_became_active":"2024-09-19T04:18:02.445274+0000","last_became_peered":"2024-09-19T04:18:02.445274+0000","last_unstale":"2024-09-19T04:18:02.459175+0000","last_undegraded":"2024-09-19T04:18:02.459175+0000","last_fullsized":"2024-09-19T04:18:02.459175+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:29.325204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448950+0000","last_change":"2024-09-19T04:18:02.444475+0000","last_active":"2024-09-19T04:18:02.448950+0000","last_peered":"2024-09-19T04:18:02.448950+0000","last_clean":"2024-09-19T04:18:02.448950+0000","last_became_active":"2024-09-19T04:18:02.444096+0000","last_became_peered":"2024-09-19T04:18:02.444096+0000","last_unstale":"2024-09-19T04:18:02.448950+0000","last_undegraded":"2024-09-19T04:18:02.448950+0000","last_fullsized":"2024-09-19T04:18:02.448950+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:23.831213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.455437+0000","last_change":"2024-09-19T04:18:02.450165+0000","last_active":"2024-09-19T04:18:02.455437+0000","last_peered":"2024-09-19T04:18:02.455437+0000","last_clean":"2024-09-19T04:18:02.455437+0000","last_became_active":"2024-09-19T04:18:02.448643+0000","last_became_peered":"2024-09-19T04:18:02.448643+0000","last_unstale":"2024-09-19T04:18:02.455437+0000","last_undegraded":"2024-09-19T04:18:02.455437+0000","last_fullsized":"2024-09-19T04:18:02.455437+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:26:56.594682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.422394+0000","last_change":"2024-09-19T04:18:02.415450+0000","last_active":"2024-09-19T04:18:02.422394+0000","last_peered":"2024-09-19T04:18:02.422394+0000","last_clean":"2024-09-19T04:18:02.422394+0000","last_became_active":"2024-09-19T04:18:02.413682+0000","last_became_peered":"2024-09-19T04:18:02.413682+0000","last_unstale":"2024-09-19T04:18:02.422394+0000","last_undegraded":"2024-09-19T04:18:02.422394+0000","last_fullsized":"2024-09-19T04:18:02.422394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:49:44.955712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.461127+0000","last_change":"2024-09-19T04:18:02.453536+0000","last_active":"2024-09-19T04:18:02.461127+0000","last_peered":"2024-09-19T04:18:02.461127+0000","last_clean":"2024-09-19T04:18:02.461127+0000","last_became_active":"2024-09-19T04:18:02.453270+0000","last_became_peered":"2024-09-19T04:18:02.453270+0000","last_unstale":"2024-09-19T04:18:02.461127+0000","last_undegraded":"2024-09-19T04:18:02.461127+0000","last_fullsized":"2024-09-19T04:18:02.461127+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:29.935876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437323+0000","last_change":"2024-09-19T04:18:02.430489+0000","last_active":"2024-09-19T04:18:02.437323+0000","last_peered":"2024-09-19T04:18:02.437323+0000","last_clean":"2024-09-19T04:18:02.437323+0000","last_became_active":"2024-09-19T04:18:02.429546+0000","last_became_peered":"2024-09-19T04:18:02.429546+0000","last_unstale":"2024-09-19T04:18:02.437323+0000","last_undegraded":"2024-09-19T04:18:02.437323+0000","last_fullsized":"2024-09-19T04:18:02.437323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:41.471819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423046+0000","last_change":"2024-09-19T04:18:02.413963+0000","last_active":"2024-09-19T04:18:02.423046+0000","last_peered":"2024-09-19T04:18:02.423046+0000","last_clean":"2024-09-19T04:18:02.423046+0000","last_became_active":"2024-09-19T04:18:02.413423+0000","last_became_peered":"2024-09-19T04:18:02.413423+0000","last_unstale":"2024-09-19T04:18:02.423046+0000","last_undegraded":"2024-09-19T04:18:02.423046+0000","last_fullsized":"2024-09-19T04:18:02.423046+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:00.783384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.464522+0000","last_change":"2024-09-19T04:18:02.447090+0000","last_active":"2024-09-19T04:18:02.464522+0000","last_peered":"2024-09-19T04:18:02.464522+0000","last_clean":"2024-09-19T04:18:02.464522+0000","last_became_active":"2024-09-19T04:18:02.444837+0000","last_became_peered":"2024-09-19T04:18:02.444837+0000","last_unstale":"2024-09-19T04:18:02.464522+0000","last_undegraded":"2024-09-19T04:18:02.464522+0000","last_fullsized":"2024-09-19T04:18:02.464522+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:05.698611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.443317+0000","last_change":"2024-09-19T04:18:02.440751+0000","last_active":"2024-09-19T04:18:02.443317+0000","last_peered":"2024-09-19T04:18:02.443317+0000","last_clean":"2024-09-19T04:18:02.443317+0000","last_became_active":"2024-09-19T04:18:02.440037+0000","last_became_peered":"2024-09-19T04:18:02.440037+0000","last_unstale":"2024-09-19T04:18:02.443317+0000","last_undegraded":"2024-09-19T04:18:02.443317+0000","last_fullsized":"2024-09-19T04:18:02.443317+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:05:54.166915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459122+0000","last_change":"2024-09-19T04:18:02.451259+0000","last_active":"2024-09-19T04:18:02.459122+0000","last_peered":"2024-09-19T04:18:02.459122+0000","last_clean":"2024-09-19T04:18:02.459122+0000","last_became_active":"2024-09-19T04:18:02.450761+0000","last_became_peered":"2024-09-19T04:18:02.450761+0000","last_unstale":"2024-09-19T04:18:02.459122+0000","last_undegraded":"2024-09-19T04:18:02.459122+0000","last_fullsized":"2024-09-19T04:18:02.459122+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:55:41.436795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.455185+0000","last_change":"2024-09-19T04:18:02.447129+0000","last_active":"2024-09-19T04:18:02.455185+0000","last_peered":"2024-09-19T04:18:02.455185+0000","last_clean":"2024-09-19T04:18:02.455185+0000","last_became_active":"2024-09-19T04:18:02.446335+0000","last_became_peered":"2024-09-19T04:18:02.446335+0000","last_unstale":"2024-09-19T04:18:02.455185+0000","last_undegraded":"2024-09-19T04:18:02.455185+0000","last_fullsized":"2024-09-19T04:18:02.455185+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:42:20.333888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.442352+0000","last_change":"2024-09-19T04:18:02.435230+0000","last_active":"2024-09-19T04:18:02.442352+0000","last_peered":"2024-09-19T04:18:02.442352+0000","last_clean":"2024-09-19T04:18:02.442352+0000","last_became_active":"2024-09-19T04:18:02.434857+0000","last_became_peered":"2024-09-19T04:18:02.434857+0000","last_unstale":"2024-09-19T04:18:02.442352+0000","last_undegraded":"2024-09-19T04:18:02.442352+0000","last_fullsized":"2024-09-19T04:18:02.442352+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:21.965193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457914+0000","last_change":"2024-09-19T04:18:02.452107+0000","last_active":"2024-09-19T04:18:02.457914+0000","last_peered":"2024-09-19T04:18:02.457914+0000","last_clean":"2024-09-19T04:18:02.457914+0000","last_became_active":"2024-09-19T04:18:02.451439+0000","last_became_peered":"2024-09-19T04:18:02.451439+0000","last_unstale":"2024-09-19T04:18:02.457914+0000","last_undegraded":"2024-09-19T04:18:02.457914+0000","last_fullsized":"2024-09-19T04:18:02.457914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:40:33.491809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.453390+0000","last_change":"2024-09-19T04:18:02.441369+0000","last_active":"2024-09-19T04:18:02.453390+0000","last_peered":"2024-09-19T04:18:02.453390+0000","last_clean":"2024-09-19T04:18:02.453390+0000","last_became_active":"2024-09-19T04:18:02.440610+0000","last_became_peered":"2024-09-19T04:18:02.440610+0000","last_unstale":"2024-09-19T04:18:02.453390+0000","last_undegraded":"2024-09-19T04:18:02.453390+0000","last_fullsized":"2024-09-19T04:18:02.453390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:33.941345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435167+0000","last_change":"2024-09-19T04:18:02.429760+0000","last_active":"2024-09-19T04:18:02.435167+0000","last_peered":"2024-09-19T04:18:02.435167+0000","last_clean":"2024-09-19T04:18:02.435167+0000","last_became_active":"2024-09-19T04:18:02.428809+0000","last_became_peered":"2024-09-19T04:18:02.428809+0000","last_unstale":"2024-09-19T04:18:02.435167+0000","last_undegraded":"2024-09-19T04:18:02.435167+0000","last_fullsized":"2024-09-19T04:18:02.435167+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:31.500411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.426941+0000","last_change":"2024-09-19T04:18:02.424318+0000","last_active":"2024-09-19T04:18:02.426941+0000","last_peered":"2024-09-19T04:18:02.426941+0000","last_clean":"2024-09-19T04:18:02.426941+0000","last_became_active":"2024-09-19T04:18:02.421773+0000","last_became_peered":"2024-09-19T04:18:02.421773+0000","last_unstale":"2024-09-19T04:18:02.426941+0000","last_undegraded":"2024-09-19T04:18:02.426941+0000","last_fullsized":"2024-09-19T04:18:02.426941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:36:48.455240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445616+0000","last_change":"2024-09-19T04:18:02.439881+0000","last_active":"2024-09-19T04:18:02.445616+0000","last_peered":"2024-09-19T04:18:02.445616+0000","last_clean":"2024-09-19T04:18:02.445616+0000","last_became_active":"2024-09-19T04:18:02.439558+0000","last_became_peered":"2024-09-19T04:18:02.439558+0000","last_unstale":"2024-09-19T04:18:02.445616+0000","last_undegraded":"2024-09-19T04:18:02.445616+0000","last_fullsized":"2024-09-19T04:18:02.445616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:32.786261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445992+0000","last_change":"2024-09-19T04:18:02.443178+0000","last_active":"2024-09-19T04:18:02.445992+0000","last_peered":"2024-09-19T04:18:02.445992+0000","last_clean":"2024-09-19T04:18:02.445992+0000","last_became_active":"2024-09-19T04:18:02.442893+0000","last_became_peered":"2024-09-19T04:18:02.442893+0000","last_unstale":"2024-09-19T04:18:02.445992+0000","last_undegraded":"2024-09-19T04:18:02.445992+0000","last_fullsized":"2024-09-19T04:18:02.445992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:36.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435208+0000","last_change":"2024-09-19T04:18:02.427154+0000","last_active":"2024-09-19T04:18:02.435208+0000","last_peered":"2024-09-19T04:18:02.435208+0000","last_clean":"2024-09-19T04:18:02.435208+0000","last_became_active":"2024-09-19T04:18:02.424520+0000","last_became_peered":"2024-09-19T04:18:02.424520+0000","last_unstale":"2024-09-19T04:18:02.435208+0000","last_undegraded":"2024-09-19T04:18:02.435208+0000","last_fullsized":"2024-09-19T04:18:02.435208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:13:49.099149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.449270+0000","last_change":"2024-09-19T04:18:02.437662+0000","last_active":"2024-09-19T04:18:02.449270+0000","last_peered":"2024-09-19T04:18:02.449270+0000","last_clean":"2024-09-19T04:18:02.449270+0000","last_became_active":"2024-09-19T04:18:02.436581+0000","last_became_peered":"2024-09-19T04:18:02.436581+0000","last_unstale":"2024-09-19T04:18:02.449270+0000","last_undegraded":"2024-09-19T04:18:02.449270+0000","last_fullsized":"2024-09-19T04:18:02.449270+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:37.644518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448360+0000","last_change":"2024-09-19T04:18:02.442819+0000","last_active":"2024-09-19T04:18:02.448360+0000","last_peered":"2024-09-19T04:18:02.448360+0000","last_clean":"2024-09-19T04:18:02.448360+0000","last_became_active":"2024-09-19T04:18:02.442506+0000","last_became_peered":"2024-09-19T04:18:02.442506+0000","last_unstale":"2024-09-19T04:18:02.448360+0000","last_undegraded":"2024-09-19T04:18:02.448360+0000","last_fullsized":"2024-09-19T04:18:02.448360+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:36.064475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445879+0000","last_change":"2024-09-19T04:18:02.426728+0000","last_active":"2024-09-19T04:18:02.445879+0000","last_peered":"2024-09-19T04:18:02.445879+0000","last_clean":"2024-09-19T04:18:02.445879+0000","last_became_active":"2024-09-19T04:18:02.426015+0000","last_became_peered":"2024-09-19T04:18:02.426015+0000","last_unstale":"2024-09-19T04:18:02.445879+0000","last_undegraded":"2024-09-19T04:18:02.445879+0000","last_fullsized":"2024-09-19T04:18:02.445879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:45:01.707512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463945+0000","last_change":"2024-09-19T04:18:02.462616+0000","last_active":"2024-09-19T04:18:02.463945+0000","last_peered":"2024-09-19T04:18:02.463945+0000","last_clean":"2024-09-19T04:18:02.463945+0000","last_became_active":"2024-09-19T04:18:02.462265+0000","last_became_peered":"2024-09-19T04:18:02.462265+0000","last_unstale":"2024-09-19T04:18:02.463945+0000","last_undegraded":"2024-09-19T04:18:02.463945+0000","last_fullsized":"2024-09-19T04:18:02.463945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:46.255336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463886+0000","last_change":"2024-09-19T04:18:02.462486+0000","last_active":"2024-09-19T04:18:02.463886+0000","last_peered":"2024-09-19T04:18:02.463886+0000","last_clean":"2024-09-19T04:18:02.463886+0000","last_became_active":"2024-09-19T04:18:02.462170+0000","last_became_peered":"2024-09-19T04:18:02.462170+0000","last_unstale":"2024-09-19T04:18:02.463886+0000","last_undegraded":"2024-09-19T04:18:02.463886+0000","last_fullsized":"2024-09-19T04:18:02.463886+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:24.469786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.454928+0000","last_change":"2024-09-19T04:18:02.447293+0000","last_active":"2024-09-19T04:18:02.454928+0000","last_peered":"2024-09-19T04:18:02.454928+0000","last_clean":"2024-09-19T04:18:02.454928+0000","last_became_active":"2024-09-19T04:18:02.446533+0000","last_became_peered":"2024-09-19T04:18:02.446533+0000","last_unstale":"2024-09-19T04:18:02.454928+0000","last_undegraded":"2024-09-19T04:18:02.454928+0000","last_fullsized":"2024-09-19T04:18:02.454928+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:35:13.954242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435407+0000","last_change":"2024-09-19T04:18:02.426760+0000","last_active":"2024-09-19T04:18:02.435407+0000","last_peered":"2024-09-19T04:18:02.435407+0000","last_clean":"2024-09-19T04:18:02.435407+0000","last_became_active":"2024-09-19T04:18:02.425891+0000","last_became_peered":"2024-09-19T04:18:02.425891+0000","last_unstale":"2024-09-19T04:18:02.435407+0000","last_undegraded":"2024-09-19T04:18:02.435407+0000","last_fullsized":"2024-09-19T04:18:02.435407+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:24.217697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.442342+0000","last_change":"2024-09-19T04:18:02.431035+0000","last_active":"2024-09-19T04:18:02.442342+0000","last_peered":"2024-09-19T04:18:02.442342+0000","last_clean":"2024-09-19T04:18:02.442342+0000","last_became_active":"2024-09-19T04:18:02.430249+0000","last_became_peered":"2024-09-19T04:18:02.430249+0000","last_unstale":"2024-09-19T04:18:02.442342+0000","last_undegraded":"2024-09-19T04:18:02.442342+0000","last_fullsized":"2024-09-19T04:18:02.442342+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:16.185194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.465751+0000","last_change":"2024-09-19T04:18:02.464622+0000","last_active":"2024-09-19T04:18:02.465751+0000","last_peered":"2024-09-19T04:18:02.465751+0000","last_clean":"2024-09-19T04:18:02.465751+0000","last_became_active":"2024-09-19T04:18:02.464360+0000","last_became_peered":"2024-09-19T04:18:02.464360+0000","last_unstale":"2024-09-19T04:18:02.465751+0000","last_undegraded":"2024-09-19T04:18:02.465751+0000","last_fullsized":"2024-09-19T04:18:02.465751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:21:36.563468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.460456+0000","last_change":"2024-09-19T04:18:02.458572+0000","last_active":"2024-09-19T04:18:02.460456+0000","last_peered":"2024-09-19T04:18:02.460456+0000","last_clean":"2024-09-19T04:18:02.460456+0000","last_became_active":"2024-09-19T04:18:02.458004+0000","last_became_peered":"2024-09-19T04:18:02.458004+0000","last_unstale":"2024-09-19T04:18:02.460456+0000","last_undegraded":"2024-09-19T04:18:02.460456+0000","last_fullsized":"2024-09-19T04:18:02.460456+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:58:42.734883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.441587+0000","last_change":"2024-09-19T04:18:02.431051+0000","last_active":"2024-09-19T04:18:02.441587+0000","last_peered":"2024-09-19T04:18:02.441587+0000","last_clean":"2024-09-19T04:18:02.441587+0000","last_became_active":"2024-09-19T04:18:02.429026+0000","last_became_peered":"2024-09-19T04:18:02.429026+0000","last_unstale":"2024-09-19T04:18:02.441587+0000","last_undegraded":"2024-09-19T04:18:02.441587+0000","last_fullsized":"2024-09-19T04:18:02.441587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.338194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435106+0000","last_change":"2024-09-19T04:18:02.427556+0000","last_active":"2024-09-19T04:18:02.435106+0000","last_peered":"2024-09-19T04:18:02.435106+0000","last_clean":"2024-09-19T04:18:02.435106+0000","last_became_active":"2024-09-19T04:18:02.426909+0000","last_became_peered":"2024-09-19T04:18:02.426909+0000","last_unstale":"2024-09-19T04:18:02.435106+0000","last_undegraded":"2024-09-19T04:18:02.435106+0000","last_fullsized":"2024-09-19T04:18:02.435106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:53:40.267633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445774+0000","last_change":"2024-09-19T04:18:02.439897+0000","last_active":"2024-09-19T04:18:02.445774+0000","last_peered":"2024-09-19T04:18:02.445774+0000","last_clean":"2024-09-19T04:18:02.445774+0000","last_became_active":"2024-09-19T04:18:02.439592+0000","last_became_peered":"2024-09-19T04:18:02.439592+0000","last_unstale":"2024-09-19T04:18:02.445774+0000","last_undegraded":"2024-09-19T04:18:02.445774+0000","last_fullsized":"2024-09-19T04:18:02.445774+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:07:41.471819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"79'4","reported_seq":16,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457741+0000","last_change":"2024-09-19T04:18:01.419252+0000","last_active":"2024-09-19T04:18:02.457741+0000","last_peered":"2024-09-19T04:18:02.457741+0000","last_clean":"2024-09-19T04:18:02.457741+0000","last_became_active":"2024-09-19T04:18:01.418935+0000","last_became_peered":"2024-09-19T04:18:01.418935+0000","last_unstale":"2024-09-19T04:18:02.457741+0000","last_undegraded":"2024-09-19T04:18:02.457741+0000","last_fullsized":"2024-09-19T04:18:02.457741+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:24.046686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.453416+0000","last_change":"2024-09-19T04:18:02.442719+0000","last_active":"2024-09-19T04:18:02.453416+0000","last_peered":"2024-09-19T04:18:02.453416+0000","last_clean":"2024-09-19T04:18:02.453416+0000","last_became_active":"2024-09-19T04:18:02.442060+0000","last_became_peered":"2024-09-19T04:18:02.442060+0000","last_unstale":"2024-09-19T04:18:02.453416+0000","last_undegraded":"2024-09-19T04:18:02.453416+0000","last_fullsized":"2024-09-19T04:18:02.453416+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:06:01.348680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463582+0000","last_change":"2024-09-19T04:18:01.444434+0000","last_active":"2024-09-19T04:18:02.463582+0000","last_peered":"2024-09-19T04:18:02.463582+0000","last_clean":"2024-09-19T04:18:02.463582+0000","last_became_active":"2024-09-19T04:18:01.443795+0000","last_became_peered":"2024-09-19T04:18:01.443795+0000","last_unstale":"2024-09-19T04:18:02.463582+0000","last_undegraded":"2024-09-19T04:18:02.463582+0000","last_fullsized":"2024-09-19T04:18:02.463582+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:37:06.327779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.467314+0000","last_change":"2024-09-19T04:18:02.452288+0000","last_active":"2024-09-19T04:18:02.467314+0000","last_peered":"2024-09-19T04:18:02.467314+0000","last_clean":"2024-09-19T04:18:02.467314+0000","last_became_active":"2024-09-19T04:18:02.448700+0000","last_became_peered":"2024-09-19T04:18:02.448700+0000","last_unstale":"2024-09-19T04:18:02.467314+0000","last_undegraded":"2024-09-19T04:18:02.467314+0000","last_fullsized":"2024-09-19T04:18:02.467314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:41.710546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463468+0000","last_change":"2024-09-19T04:18:01.444895+0000","last_active":"2024-09-19T04:18:02.463468+0000","last_peered":"2024-09-19T04:18:02.463468+0000","last_clean":"2024-09-19T04:18:02.463468+0000","last_became_active":"2024-09-19T04:18:01.443565+0000","last_became_peered":"2024-09-19T04:18:01.443565+0000","last_unstale":"2024-09-19T04:18:02.463468+0000","last_undegraded":"2024-09-19T04:18:02.463468+0000","last_fullsized":"2024-09-19T04:18:02.463468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:35.483830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.446098+0000","last_change":"2024-09-19T04:18:02.437114+0000","last_active":"2024-09-19T04:18:02.446098+0000","last_peered":"2024-09-19T04:18:02.446098+0000","last_clean":"2024-09-19T04:18:02.446098+0000","last_became_active":"2024-09-19T04:18:02.436436+0000","last_became_peered":"2024-09-19T04:18:02.436436+0000","last_unstale":"2024-09-19T04:18:02.446098+0000","last_undegraded":"2024-09-19T04:18:02.446098+0000","last_fullsized":"2024-09-19T04:18:02.446098+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:06.063752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463495+0000","last_change":"2024-09-19T04:18:01.433700+0000","last_active":"2024-09-19T04:18:02.463495+0000","last_peered":"2024-09-19T04:18:02.463495+0000","last_clean":"2024-09-19T04:18:02.463495+0000","last_became_active":"2024-09-19T04:18:01.432148+0000","last_became_peered":"2024-09-19T04:18:01.432148+0000","last_unstale":"2024-09-19T04:18:02.463495+0000","last_undegraded":"2024-09-19T04:18:02.463495+0000","last_fullsized":"2024-09-19T04:18:02.463495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.700851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459088+0000","last_change":"2024-09-19T04:18:02.444971+0000","last_active":"2024-09-19T04:18:02.459088+0000","last_peered":"2024-09-19T04:18:02.459088+0000","last_clean":"2024-09-19T04:18:02.459088+0000","last_became_active":"2024-09-19T04:18:02.444421+0000","last_became_peered":"2024-09-19T04:18:02.444421+0000","last_unstale":"2024-09-19T04:18:02.459088+0000","last_undegraded":"2024-09-19T04:18:02.459088+0000","last_fullsized":"2024-09-19T04:18:02.459088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:07:52.876014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468636+0000","last_change":"2024-09-19T04:18:01.432369+0000","last_active":"2024-09-19T04:18:02.468636+0000","last_peered":"2024-09-19T04:18:02.468636+0000","last_clean":"2024-09-19T04:18:02.468636+0000","last_became_active":"2024-09-19T04:18:01.431423+0000","last_became_peered":"2024-09-19T04:18:01.431423+0000","last_unstale":"2024-09-19T04:18:02.468636+0000","last_undegraded":"2024-09-19T04:18:02.468636+0000","last_fullsized":"2024-09-19T04:18:02.468636+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:10:29.307422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459161+0000","last_change":"2024-09-19T04:18:02.452072+0000","last_active":"2024-09-19T04:18:02.459161+0000","last_peered":"2024-09-19T04:18:02.459161+0000","last_clean":"2024-09-19T04:18:02.459161+0000","last_became_active":"2024-09-19T04:18:02.451432+0000","last_became_peered":"2024-09-19T04:18:02.451432+0000","last_unstale":"2024-09-19T04:18:02.459161+0000","last_undegraded":"2024-09-19T04:18:02.459161+0000","last_fullsized":"2024-09-19T04:18:02.459161+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:15.323523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.462563+0000","last_change":"2024-09-19T04:18:01.419082+0000","last_active":"2024-09-19T04:18:02.462563+0000","last_peered":"2024-09-19T04:18:02.462563+0000","last_clean":"2024-09-19T04:18:02.462563+0000","last_became_active":"2024-09-19T04:18:01.418294+0000","last_became_peered":"2024-09-19T04:18:01.418294+0000","last_unstale":"2024-09-19T04:18:02.462563+0000","last_undegraded":"2024-09-19T04:18:02.462563+0000","last_fullsized":"2024-09-19T04:18:02.462563+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:13.846251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.464465+0000","last_change":"2024-09-19T04:18:02.445661+0000","last_active":"2024-09-19T04:18:02.464465+0000","last_peered":"2024-09-19T04:18:02.464465+0000","last_clean":"2024-09-19T04:18:02.464465+0000","last_became_active":"2024-09-19T04:18:02.444841+0000","last_became_peered":"2024-09-19T04:18:02.444841+0000","last_unstale":"2024-09-19T04:18:02.464465+0000","last_undegraded":"2024-09-19T04:18:02.464465+0000","last_fullsized":"2024-09-19T04:18:02.464465+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:57:31.059273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.426791+0000","last_change":"2024-09-19T04:18:01.444532+0000","last_active":"2024-09-19T04:18:02.426791+0000","last_peered":"2024-09-19T04:18:02.426791+0000","last_clean":"2024-09-19T04:18:02.426791+0000","last_became_active":"2024-09-19T04:18:01.444261+0000","last_became_peered":"2024-09-19T04:18:01.444261+0000","last_unstale":"2024-09-19T04:18:02.426791+0000","last_undegraded":"2024-09-19T04:18:02.426791+0000","last_fullsized":"2024-09-19T04:18:02.426791+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:02.152154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437147+0000","last_change":"2024-09-19T04:18:02.426626+0000","last_active":"2024-09-19T04:18:02.437147+0000","last_peered":"2024-09-19T04:18:02.437147+0000","last_clean":"2024-09-19T04:18:02.437147+0000","last_became_active":"2024-09-19T04:18:02.425410+0000","last_became_peered":"2024-09-19T04:18:02.425410+0000","last_unstale":"2024-09-19T04:18:02.437147+0000","last_undegraded":"2024-09-19T04:18:02.437147+0000","last_fullsized":"2024-09-19T04:18:02.437147+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:09:32.786261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406054+0000","last_change":"2024-09-19T04:18:01.429905+0000","last_active":"2024-09-19T04:18:02.406054+0000","last_peered":"2024-09-19T04:18:02.406054+0000","last_clean":"2024-09-19T04:18:02.406054+0000","last_became_active":"2024-09-19T04:18:01.429563+0000","last_became_peered":"2024-09-19T04:18:01.429563+0000","last_unstale":"2024-09-19T04:18:02.406054+0000","last_undegraded":"2024-09-19T04:18:02.406054+0000","last_fullsized":"2024-09-19T04:18:02.406054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:12:37.679637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463687+0000","last_change":"2024-09-19T04:18:02.460855+0000","last_active":"2024-09-19T04:18:02.463687+0000","last_peered":"2024-09-19T04:18:02.463687+0000","last_clean":"2024-09-19T04:18:02.463687+0000","last_became_active":"2024-09-19T04:18:02.460559+0000","last_became_peered":"2024-09-19T04:18:02.460559+0000","last_unstale":"2024-09-19T04:18:02.463687+0000","last_undegraded":"2024-09-19T04:18:02.463687+0000","last_fullsized":"2024-09-19T04:18:02.463687+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:42.134180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.407407+0000","last_change":"2024-09-19T04:18:01.444708+0000","last_active":"2024-09-19T04:18:02.407407+0000","last_peered":"2024-09-19T04:18:02.407407+0000","last_clean":"2024-09-19T04:18:02.407407+0000","last_became_active":"2024-09-19T04:18:01.444016+0000","last_became_peered":"2024-09-19T04:18:01.444016+0000","last_unstale":"2024-09-19T04:18:02.407407+0000","last_undegraded":"2024-09-19T04:18:02.407407+0000","last_fullsized":"2024-09-19T04:18:02.407407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:58:05.837521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.455560+0000","last_change":"2024-09-19T04:18:02.448557+0000","last_active":"2024-09-19T04:18:02.455560+0000","last_peered":"2024-09-19T04:18:02.455560+0000","last_clean":"2024-09-19T04:18:02.455560+0000","last_became_active":"2024-09-19T04:18:02.447853+0000","last_became_peered":"2024-09-19T04:18:02.447853+0000","last_unstale":"2024-09-19T04:18:02.455560+0000","last_undegraded":"2024-09-19T04:18:02.455560+0000","last_fullsized":"2024-09-19T04:18:02.455560+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:20:58.846942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423242+0000","last_change":"2024-09-19T04:18:01.428836+0000","last_active":"2024-09-19T04:18:02.423242+0000","last_peered":"2024-09-19T04:18:02.423242+0000","last_clean":"2024-09-19T04:18:02.423242+0000","last_became_active":"2024-09-19T04:18:01.428122+0000","last_became_peered":"2024-09-19T04:18:01.428122+0000","last_unstale":"2024-09-19T04:18:02.423242+0000","last_undegraded":"2024-09-19T04:18:02.423242+0000","last_fullsized":"2024-09-19T04:18:02.423242+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:27:00.969190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457829+0000","last_change":"2024-09-19T04:18:02.446619+0000","last_active":"2024-09-19T04:18:02.457829+0000","last_peered":"2024-09-19T04:18:02.457829+0000","last_clean":"2024-09-19T04:18:02.457829+0000","last_became_active":"2024-09-19T04:18:02.446365+0000","last_became_peered":"2024-09-19T04:18:02.446365+0000","last_unstale":"2024-09-19T04:18:02.457829+0000","last_undegraded":"2024-09-19T04:18:02.457829+0000","last_fullsized":"2024-09-19T04:18:02.457829+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:38.619995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"79'3","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.458331+0000","last_change":"2024-09-19T04:18:01.415693+0000","last_active":"2024-09-19T04:18:02.458331+0000","last_peered":"2024-09-19T04:18:02.458331+0000","last_clean":"2024-09-19T04:18:02.458331+0000","last_became_active":"2024-09-19T04:18:01.415409+0000","last_became_peered":"2024-09-19T04:18:01.415409+0000","last_unstale":"2024-09-19T04:18:02.458331+0000","last_undegraded":"2024-09-19T04:18:02.458331+0000","last_fullsized":"2024-09-19T04:18:02.458331+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:52:29.237363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448918+0000","last_change":"2024-09-19T04:18:02.444586+0000","last_active":"2024-09-19T04:18:02.448918+0000","last_peered":"2024-09-19T04:18:02.448918+0000","last_clean":"2024-09-19T04:18:02.448918+0000","last_became_active":"2024-09-19T04:18:02.444274+0000","last_became_peered":"2024-09-19T04:18:02.444274+0000","last_unstale":"2024-09-19T04:18:02.448918+0000","last_undegraded":"2024-09-19T04:18:02.448918+0000","last_fullsized":"2024-09-19T04:18:02.448918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:02.081069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"79'6","reported_seq":18,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.472266+0000","last_change":"2024-09-19T04:18:01.440750+0000","last_active":"2024-09-19T04:18:02.472266+0000","last_peered":"2024-09-19T04:18:02.472266+0000","last_clean":"2024-09-19T04:18:02.472266+0000","last_became_active":"2024-09-19T04:18:01.439842+0000","last_became_peered":"2024-09-19T04:18:01.439842+0000","last_unstale":"2024-09-19T04:18:02.472266+0000","last_undegraded":"2024-09-19T04:18:02.472266+0000","last_fullsized":"2024-09-19T04:18:02.472266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:50.685522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.443208+0000","last_change":"2024-09-19T04:18:02.439877+0000","last_active":"2024-09-19T04:18:02.443208+0000","last_peered":"2024-09-19T04:18:02.443208+0000","last_clean":"2024-09-19T04:18:02.443208+0000","last_became_active":"2024-09-19T04:18:02.439168+0000","last_became_peered":"2024-09-19T04:18:02.439168+0000","last_unstale":"2024-09-19T04:18:02.443208+0000","last_undegraded":"2024-09-19T04:18:02.443208+0000","last_fullsized":"2024-09-19T04:18:02.443208+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:17.908329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468115+0000","last_change":"2024-09-19T04:18:01.433865+0000","last_active":"2024-09-19T04:18:02.468115+0000","last_peered":"2024-09-19T04:18:02.468115+0000","last_clean":"2024-09-19T04:18:02.468115+0000","last_became_active":"2024-09-19T04:18:01.433084+0000","last_became_peered":"2024-09-19T04:18:01.433084+0000","last_unstale":"2024-09-19T04:18:02.468115+0000","last_undegraded":"2024-09-19T04:18:02.468115+0000","last_fullsized":"2024-09-19T04:18:02.468115+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:47.965198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.438107+0000","last_change":"2024-09-19T04:18:02.431066+0000","last_active":"2024-09-19T04:18:02.438107+0000","last_peered":"2024-09-19T04:18:02.438107+0000","last_clean":"2024-09-19T04:18:02.438107+0000","last_became_active":"2024-09-19T04:18:02.430018+0000","last_became_peered":"2024-09-19T04:18:02.430018+0000","last_unstale":"2024-09-19T04:18:02.438107+0000","last_undegraded":"2024-09-19T04:18:02.438107+0000","last_fullsized":"2024-09-19T04:18:02.438107+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:47:23.831213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406091+0000","last_change":"2024-09-19T04:18:01.437255+0000","last_active":"2024-09-19T04:18:02.406091+0000","last_peered":"2024-09-19T04:18:02.406091+0000","last_clean":"2024-09-19T04:18:02.406091+0000","last_became_active":"2024-09-19T04:18:01.436414+0000","last_became_peered":"2024-09-19T04:18:01.436414+0000","last_unstale":"2024-09-19T04:18:02.406091+0000","last_undegraded":"2024-09-19T04:18:02.406091+0000","last_fullsized":"2024-09-19T04:18:02.406091+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:53:34.210598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.444278+0000","last_change":"2024-09-19T04:18:02.440292+0000","last_active":"2024-09-19T04:18:02.444278+0000","last_peered":"2024-09-19T04:18:02.444278+0000","last_clean":"2024-09-19T04:18:02.444278+0000","last_became_active":"2024-09-19T04:18:02.439573+0000","last_became_peered":"2024-09-19T04:18:02.439573+0000","last_unstale":"2024-09-19T04:18:02.444278+0000","last_undegraded":"2024-09-19T04:18:02.444278+0000","last_fullsized":"2024-09-19T04:18:02.444278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:31:56.528286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406207+0000","last_change":"2024-09-19T04:18:01.419197+0000","last_active":"2024-09-19T04:18:02.406207+0000","last_peered":"2024-09-19T04:18:02.406207+0000","last_clean":"2024-09-19T04:18:02.406207+0000","last_became_active":"2024-09-19T04:18:01.418894+0000","last_became_peered":"2024-09-19T04:18:01.418894+0000","last_unstale":"2024-09-19T04:18:02.406207+0000","last_undegraded":"2024-09-19T04:18:02.406207+0000","last_fullsized":"2024-09-19T04:18:02.406207+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:28:55.740984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437245+0000","last_change":"2024-09-19T04:18:02.432056+0000","last_active":"2024-09-19T04:18:02.437245+0000","last_peered":"2024-09-19T04:18:02.437245+0000","last_clean":"2024-09-19T04:18:02.437245+0000","last_became_active":"2024-09-19T04:18:02.431365+0000","last_became_peered":"2024-09-19T04:18:02.431365+0000","last_unstale":"2024-09-19T04:18:02.437245+0000","last_undegraded":"2024-09-19T04:18:02.437245+0000","last_fullsized":"2024-09-19T04:18:02.437245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:02.081069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"79'8","reported_seq":20,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.467046+0000","last_change":"2024-09-19T04:18:01.433055+0000","last_active":"2024-09-19T04:18:02.467046+0000","last_peered":"2024-09-19T04:18:02.467046+0000","last_clean":"2024-09-19T04:18:02.467046+0000","last_became_active":"2024-09-19T04:18:01.432780+0000","last_became_peered":"2024-09-19T04:18:01.432780+0000","last_unstale":"2024-09-19T04:18:02.467046+0000","last_undegraded":"2024-09-19T04:18:02.467046+0000","last_fullsized":"2024-09-19T04:18:02.467046+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:14:42.620115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.442724+0000","last_change":"2024-09-19T04:18:02.440343+0000","last_active":"2024-09-19T04:18:02.442724+0000","last_peered":"2024-09-19T04:18:02.442724+0000","last_clean":"2024-09-19T04:18:02.442724+0000","last_became_active":"2024-09-19T04:18:02.440069+0000","last_became_peered":"2024-09-19T04:18:02.440069+0000","last_unstale":"2024-09-19T04:18:02.442724+0000","last_undegraded":"2024-09-19T04:18:02.442724+0000","last_fullsized":"2024-09-19T04:18:02.442724+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:46.255336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.462327+0000","last_change":"2024-09-19T04:18:01.439501+0000","last_active":"2024-09-19T04:18:02.462327+0000","last_peered":"2024-09-19T04:18:02.462327+0000","last_clean":"2024-09-19T04:18:02.462327+0000","last_became_active":"2024-09-19T04:18:01.438869+0000","last_became_peered":"2024-09-19T04:18:01.438869+0000","last_unstale":"2024-09-19T04:18:02.462327+0000","last_undegraded":"2024-09-19T04:18:02.462327+0000","last_fullsized":"2024-09-19T04:18:02.462327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:21:28.945268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.439744+0000","last_change":"2024-09-19T04:18:02.433490+0000","last_active":"2024-09-19T04:18:02.439744+0000","last_peered":"2024-09-19T04:18:02.439744+0000","last_clean":"2024-09-19T04:18:02.439744+0000","last_became_active":"2024-09-19T04:18:02.430975+0000","last_became_peered":"2024-09-19T04:18:02.430975+0000","last_unstale":"2024-09-19T04:18:02.439744+0000","last_undegraded":"2024-09-19T04:18:02.439744+0000","last_fullsized":"2024-09-19T04:18:02.439744+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:17:03.274491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468992+0000","last_change":"2024-09-19T04:18:01.437357+0000","last_active":"2024-09-19T04:18:02.468992+0000","last_peered":"2024-09-19T04:18:02.468992+0000","last_clean":"2024-09-19T04:18:02.468992+0000","last_became_active":"2024-09-19T04:18:01.436248+0000","last_became_peered":"2024-09-19T04:18:01.436248+0000","last_unstale":"2024-09-19T04:18:02.468992+0000","last_undegraded":"2024-09-19T04:18:02.468992+0000","last_fullsized":"2024-09-19T04:18:02.468992+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:52.562016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.436754+0000","last_change":"2024-09-19T04:18:02.430509+0000","last_active":"2024-09-19T04:18:02.436754+0000","last_peered":"2024-09-19T04:18:02.436754+0000","last_clean":"2024-09-19T04:18:02.436754+0000","last_became_active":"2024-09-19T04:18:02.429845+0000","last_became_peered":"2024-09-19T04:18:02.429845+0000","last_unstale":"2024-09-19T04:18:02.436754+0000","last_undegraded":"2024-09-19T04:18:02.436754+0000","last_fullsized":"2024-09-19T04:18:02.436754+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:49:52.670666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"79'1","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.468113+0000","last_change":"2024-09-19T04:18:01.429737+0000","last_active":"2024-09-19T04:18:02.468113+0000","last_peered":"2024-09-19T04:18:02.468113+0000","last_clean":"2024-09-19T04:18:02.468113+0000","last_became_active":"2024-09-19T04:18:01.429512+0000","last_became_peered":"2024-09-19T04:18:01.429512+0000","last_unstale":"2024-09-19T04:18:02.468113+0000","last_undegraded":"2024-09-19T04:18:02.468113+0000","last_fullsized":"2024-09-19T04:18:02.468113+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:17.225238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.440828+0000","last_change":"2024-09-19T04:18:02.431032+0000","last_active":"2024-09-19T04:18:02.440828+0000","last_peered":"2024-09-19T04:18:02.440828+0000","last_clean":"2024-09-19T04:18:02.440828+0000","last_became_active":"2024-09-19T04:18:02.430493+0000","last_became_peered":"2024-09-19T04:18:02.430493+0000","last_unstale":"2024-09-19T04:18:02.440828+0000","last_undegraded":"2024-09-19T04:18:02.440828+0000","last_fullsized":"2024-09-19T04:18:02.440828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:59.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.414752+0000","last_change":"2024-09-19T04:18:01.412564+0000","last_active":"2024-09-19T04:18:02.414752+0000","last_peered":"2024-09-19T04:18:02.414752+0000","last_clean":"2024-09-19T04:18:02.414752+0000","last_became_active":"2024-09-19T04:18:01.411102+0000","last_became_peered":"2024-09-19T04:18:01.411102+0000","last_unstale":"2024-09-19T04:18:02.414752+0000","last_undegraded":"2024-09-19T04:18:02.414752+0000","last_fullsized":"2024-09-19T04:18:02.414752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:05:40.700851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.459188+0000","last_change":"2024-09-19T04:18:02.445199+0000","last_active":"2024-09-19T04:18:02.459188+0000","last_peered":"2024-09-19T04:18:02.459188+0000","last_clean":"2024-09-19T04:18:02.459188+0000","last_became_active":"2024-09-19T04:18:02.444474+0000","last_became_peered":"2024-09-19T04:18:02.444474+0000","last_unstale":"2024-09-19T04:18:02.459188+0000","last_undegraded":"2024-09-19T04:18:02.459188+0000","last_fullsized":"2024-09-19T04:18:02.459188+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:05.343882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.457368+0000","last_change":"2024-09-19T04:18:01.416161+0000","last_active":"2024-09-19T04:18:02.457368+0000","last_peered":"2024-09-19T04:18:02.457368+0000","last_clean":"2024-09-19T04:18:02.457368+0000","last_became_active":"2024-09-19T04:18:01.415873+0000","last_became_peered":"2024-09-19T04:18:01.415873+0000","last_unstale":"2024-09-19T04:18:02.457368+0000","last_undegraded":"2024-09-19T04:18:02.457368+0000","last_fullsized":"2024-09-19T04:18:02.457368+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:14.313827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.470986+0000","last_change":"2024-09-19T04:18:02.469895+0000","last_active":"2024-09-19T04:18:02.470986+0000","last_peered":"2024-09-19T04:18:02.470986+0000","last_clean":"2024-09-19T04:18:02.470986+0000","last_became_active":"2024-09-19T04:18:02.469624+0000","last_became_peered":"2024-09-19T04:18:02.469624+0000","last_unstale":"2024-09-19T04:18:02.470986+0000","last_undegraded":"2024-09-19T04:18:02.470986+0000","last_fullsized":"2024-09-19T04:18:02.470986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:38.619995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423965+0000","last_change":"2024-09-19T04:18:01.440551+0000","last_active":"2024-09-19T04:18:02.423965+0000","last_peered":"2024-09-19T04:18:02.423965+0000","last_clean":"2024-09-19T04:18:02.423965+0000","last_became_active":"2024-09-19T04:18:01.440276+0000","last_became_peered":"2024-09-19T04:18:01.440276+0000","last_unstale":"2024-09-19T04:18:02.423965+0000","last_undegraded":"2024-09-19T04:18:02.423965+0000","last_fullsized":"2024-09-19T04:18:02.423965+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:44:41.124484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437199+0000","last_change":"2024-09-19T04:18:02.431584+0000","last_active":"2024-09-19T04:18:02.437199+0000","last_peered":"2024-09-19T04:18:02.437199+0000","last_clean":"2024-09-19T04:18:02.437199+0000","last_became_active":"2024-09-19T04:18:02.430854+0000","last_became_peered":"2024-09-19T04:18:02.430854+0000","last_unstale":"2024-09-19T04:18:02.437199+0000","last_undegraded":"2024-09-19T04:18:02.437199+0000","last_fullsized":"2024-09-19T04:18:02.437199+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:05:25.365046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.423123+0000","last_change":"2024-09-19T04:18:01.432749+0000","last_active":"2024-09-19T04:18:02.423123+0000","last_peered":"2024-09-19T04:18:02.423123+0000","last_clean":"2024-09-19T04:18:02.423123+0000","last_became_active":"2024-09-19T04:18:01.432481+0000","last_became_peered":"2024-09-19T04:18:01.432481+0000","last_unstale":"2024-09-19T04:18:02.423123+0000","last_undegraded":"2024-09-19T04:18:02.423123+0000","last_fullsized":"2024-09-19T04:18:02.423123+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:20:28.315508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.445743+0000","last_change":"2024-09-19T04:18:02.436751+0000","last_active":"2024-09-19T04:18:02.445743+0000","last_peered":"2024-09-19T04:18:02.445743+0000","last_clean":"2024-09-19T04:18:02.445743+0000","last_became_active":"2024-09-19T04:18:02.429731+0000","last_became_peered":"2024-09-19T04:18:02.429731+0000","last_unstale":"2024-09-19T04:18:02.445743+0000","last_undegraded":"2024-09-19T04:18:02.445743+0000","last_fullsized":"2024-09-19T04:18:02.445743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:48:57.429637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"79'2","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.460326+0000","last_change":"2024-09-19T04:18:01.420205+0000","last_active":"2024-09-19T04:18:02.460326+0000","last_peered":"2024-09-19T04:18:02.460326+0000","last_clean":"2024-09-19T04:18:02.460326+0000","last_became_active":"2024-09-19T04:18:01.419399+0000","last_became_peered":"2024-09-19T04:18:01.419399+0000","last_unstale":"2024-09-19T04:18:02.460326+0000","last_undegraded":"2024-09-19T04:18:02.460326+0000","last_fullsized":"2024-09-19T04:18:02.460326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:15:19.000405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.461338+0000","last_change":"2024-09-19T04:18:02.442173+0000","last_active":"2024-09-19T04:18:02.461338+0000","last_peered":"2024-09-19T04:18:02.461338+0000","last_clean":"2024-09-19T04:18:02.461338+0000","last_became_active":"2024-09-19T04:18:02.441273+0000","last_became_peered":"2024-09-19T04:18:02.441273+0000","last_unstale":"2024-09-19T04:18:02.461338+0000","last_undegraded":"2024-09-19T04:18:02.461338+0000","last_fullsized":"2024-09-19T04:18:02.461338+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:05.504879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.436770+0000","last_change":"2024-09-19T04:18:01.410252+0000","last_active":"2024-09-19T04:18:02.436770+0000","last_peered":"2024-09-19T04:18:02.436770+0000","last_clean":"2024-09-19T04:18:02.436770+0000","last_became_active":"2024-09-19T04:18:01.409901+0000","last_became_peered":"2024-09-19T04:18:01.409901+0000","last_unstale":"2024-09-19T04:18:02.436770+0000","last_undegraded":"2024-09-19T04:18:02.436770+0000","last_fullsized":"2024-09-19T04:18:02.436770+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:14:35.911828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.467601+0000","last_change":"2024-09-19T04:18:02.460997+0000","last_active":"2024-09-19T04:18:02.467601+0000","last_peered":"2024-09-19T04:18:02.467601+0000","last_clean":"2024-09-19T04:18:02.467601+0000","last_became_active":"2024-09-19T04:18:02.460141+0000","last_became_peered":"2024-09-19T04:18:02.460141+0000","last_unstale":"2024-09-19T04:18:02.467601+0000","last_undegraded":"2024-09-19T04:18:02.467601+0000","last_fullsized":"2024-09-19T04:18:02.467601+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:51.695218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.407725+0000","last_change":"2024-09-19T04:18:01.444473+0000","last_active":"2024-09-19T04:18:02.407725+0000","last_peered":"2024-09-19T04:18:02.407725+0000","last_clean":"2024-09-19T04:18:02.407725+0000","last_became_active":"2024-09-19T04:18:01.443651+0000","last_became_peered":"2024-09-19T04:18:01.443651+0000","last_unstale":"2024-09-19T04:18:02.407725+0000","last_undegraded":"2024-09-19T04:18:02.407725+0000","last_fullsized":"2024-09-19T04:18:02.407725+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:06:45.921951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.463545+0000","last_change":"2024-09-19T04:18:02.461892+0000","last_active":"2024-09-19T04:18:02.463545+0000","last_peered":"2024-09-19T04:18:02.463545+0000","last_clean":"2024-09-19T04:18:02.463545+0000","last_became_active":"2024-09-19T04:18:02.460635+0000","last_became_peered":"2024-09-19T04:18:02.460635+0000","last_unstale":"2024-09-19T04:18:02.463545+0000","last_undegraded":"2024-09-19T04:18:02.463545+0000","last_fullsized":"2024-09-19T04:18:02.463545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:31:59.774897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406694+0000","last_change":"2024-09-19T04:18:01.437617+0000","last_active":"2024-09-19T04:18:02.406694+0000","last_peered":"2024-09-19T04:18:02.406694+0000","last_clean":"2024-09-19T04:18:02.406694+0000","last_became_active":"2024-09-19T04:18:01.436825+0000","last_became_peered":"2024-09-19T04:18:01.436825+0000","last_unstale":"2024-09-19T04:18:02.406694+0000","last_undegraded":"2024-09-19T04:18:02.406694+0000","last_fullsized":"2024-09-19T04:18:02.406694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:39:53.838198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.441808+0000","last_change":"2024-09-19T04:18:02.434122+0000","last_active":"2024-09-19T04:18:02.441808+0000","last_peered":"2024-09-19T04:18:02.441808+0000","last_clean":"2024-09-19T04:18:02.441808+0000","last_became_active":"2024-09-19T04:18:02.433034+0000","last_became_peered":"2024-09-19T04:18:02.433034+0000","last_unstale":"2024-09-19T04:18:02.441808+0000","last_undegraded":"2024-09-19T04:18:02.441808+0000","last_fullsized":"2024-09-19T04:18:02.441808+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:12:45.835099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.409645+0000","last_change":"2024-09-19T04:18:01.434879+0000","last_active":"2024-09-19T04:18:02.409645+0000","last_peered":"2024-09-19T04:18:02.409645+0000","last_clean":"2024-09-19T04:18:02.409645+0000","last_became_active":"2024-09-19T04:18:01.432576+0000","last_became_peered":"2024-09-19T04:18:01.432576+0000","last_unstale":"2024-09-19T04:18:02.409645+0000","last_undegraded":"2024-09-19T04:18:02.409645+0000","last_fullsized":"2024-09-19T04:18:02.409645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:39:04.688916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.438052+0000","last_change":"2024-09-19T04:18:02.426424+0000","last_active":"2024-09-19T04:18:02.438052+0000","last_peered":"2024-09-19T04:18:02.438052+0000","last_clean":"2024-09-19T04:18:02.438052+0000","last_became_active":"2024-09-19T04:18:02.425433+0000","last_became_peered":"2024-09-19T04:18:02.425433+0000","last_unstale":"2024-09-19T04:18:02.438052+0000","last_undegraded":"2024-09-19T04:18:02.438052+0000","last_fullsized":"2024-09-19T04:18:02.438052+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:11:17.338194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.406755+0000","last_change":"2024-09-19T04:18:01.419456+0000","last_active":"2024-09-19T04:18:02.406755+0000","last_peered":"2024-09-19T04:18:02.406755+0000","last_clean":"2024-09-19T04:18:02.406755+0000","last_became_active":"2024-09-19T04:18:01.419159+0000","last_became_peered":"2024-09-19T04:18:01.419159+0000","last_unstale":"2024-09-19T04:18:02.406755+0000","last_undegraded":"2024-09-19T04:18:02.406755+0000","last_fullsized":"2024-09-19T04:18:02.406755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:45.576305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.448879+0000","last_change":"2024-09-19T04:18:02.444568+0000","last_active":"2024-09-19T04:18:02.448879+0000","last_peered":"2024-09-19T04:18:02.448879+0000","last_clean":"2024-09-19T04:18:02.448879+0000","last_became_active":"2024-09-19T04:18:02.444205+0000","last_became_peered":"2024-09-19T04:18:02.444205+0000","last_unstale":"2024-09-19T04:18:02.448879+0000","last_undegraded":"2024-09-19T04:18:02.448879+0000","last_fullsized":"2024-09-19T04:18:02.448879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:05:25.365046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.413897+0000","last_change":"2024-09-19T04:18:01.428145+0000","last_active":"2024-09-19T04:18:02.413897+0000","last_peered":"2024-09-19T04:18:02.413897+0000","last_clean":"2024-09-19T04:18:02.413897+0000","last_became_active":"2024-09-19T04:18:01.425709+0000","last_became_peered":"2024-09-19T04:18:01.425709+0000","last_unstale":"2024-09-19T04:18:02.413897+0000","last_undegraded":"2024-09-19T04:18:02.413897+0000","last_fullsized":"2024-09-19T04:18:02.413897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:58.560859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.440445+0000","last_change":"2024-09-19T04:18:02.428097+0000","last_active":"2024-09-19T04:18:02.440445+0000","last_peered":"2024-09-19T04:18:02.440445+0000","last_clean":"2024-09-19T04:18:02.440445+0000","last_became_active":"2024-09-19T04:18:02.427332+0000","last_became_peered":"2024-09-19T04:18:02.427332+0000","last_unstale":"2024-09-19T04:18:02.440445+0000","last_undegraded":"2024-09-19T04:18:02.440445+0000","last_fullsized":"2024-09-19T04:18:02.440445+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:41:59.570554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.422366+0000","last_change":"2024-09-19T04:18:01.416528+0000","last_active":"2024-09-19T04:18:02.422366+0000","last_peered":"2024-09-19T04:18:02.422366+0000","last_clean":"2024-09-19T04:18:02.422366+0000","last_became_active":"2024-09-19T04:18:01.416163+0000","last_became_peered":"2024-09-19T04:18:01.416163+0000","last_unstale":"2024-09-19T04:18:02.422366+0000","last_undegraded":"2024-09-19T04:18:02.422366+0000","last_fullsized":"2024-09-19T04:18:02.422366+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:50.685522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":9,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.435463+0000","last_change":"2024-09-19T04:18:02.431813+0000","last_active":"2024-09-19T04:18:02.435463+0000","last_peered":"2024-09-19T04:18:02.435463+0000","last_clean":"2024-09-19T04:18:02.435463+0000","last_became_active":"2024-09-19T04:18:02.431068+0000","last_became_peered":"2024-09-19T04:18:02.431068+0000","last_unstale":"2024-09-19T04:18:02.435463+0000","last_undegraded":"2024-09-19T04:18:02.435463+0000","last_fullsized":"2024-09-19T04:18:02.435463+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:01.383704+0000","last_clean_scrub_stamp":"2024-09-19T04:18:01.383704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:00.903407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.437474+0000","last_change":"2024-09-19T04:18:01.435376+0000","last_active":"2024-09-19T04:18:02.437474+0000","last_peered":"2024-09-19T04:18:02.437474+0000","last_clean":"2024-09-19T04:18:02.437474+0000","last_became_active":"2024-09-19T04:18:01.435106+0000","last_became_peered":"2024-09-19T04:18:01.435106+0000","last_unstale":"2024-09-19T04:18:02.437474+0000","last_undegraded":"2024-09-19T04:18:02.437474+0000","last_fullsized":"2024-09-19T04:18:02.437474+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:00.374008+0000","last_clean_scrub_stamp":"2024-09-19T04:18:00.374008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:23.460090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":278,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-19T04:18:02.405144+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:18:02.405144+0000","last_peered":"2024-09-19T04:18:02.405144+0000","last_clean":"2024-09-19T04:18:02.405144+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:18:02.405144+0000","last_undegraded":"2024-09-19T04:18:02.405144+0000","last_fullsized":"2024-09-19T04:18:02.405144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1544192,"data_stored":1027039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":950272,"data_stored":436671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":966656,"data_stored":443872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":950272,"data_stored":436671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":444420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542206,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":974848,"data_stored":444580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542206,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":950272,"data_stored":436671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1556480,"data_stored":1034206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:18:06.832 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:18:06.832 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T04:18:07.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.998+0000 7f2a1bf70640 1 Processor -- start 2024-09-19T04:18:07.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:06.998+0000 7f2a1bf70640 1 -- start start 2024-09-19T04:18:07.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1bf70640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1bf70640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2a1415d850 0x7f2a1415dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1bf70640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a140b58a0 con 0x7f2a14159d30 2024-09-19T04:18:07.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1bf70640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2a1415a110 con 0x7f2a1415d850 2024-09-19T04:18:07.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1bf70640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1409b790 con 0x7f2a140b5c10 2024-09-19T04:18:07.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2a1415d850 0x7f2a1415dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1950b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2a1415d850 0x7f2a1415dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:39450/0 (socket says 172.21.15.76:39450) 2024-09-19T04:18:07.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 -- 172.21.15.76:0/1525326355 learned_addr learned my addr 172.21.15.76:0/1525326355 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:07.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 -- 172.21.15.76:0/1525326355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 msgr2=0x7f2a140b6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 --2- 172.21.15.76:0/1525326355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 -- 172.21.15.76:0/1525326355 >> v1:172.21.15.116:6789/0 conn(0x7f2a14159d30 legacy=0x7f2a140b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 -- 172.21.15.76:0/1525326355 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a140b6570 con 0x7f2a1415d850 2024-09-19T04:18:07.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a1a50d640 1 --2- 172.21.15.76:0/1525326355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1415dcc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009e50 tx=0x7f2a0c01ca00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25df6343c25ee628 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a18d0a640 1 -- 172.21.15.76:0/1525326355 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0a58d0 con 0x7f2a1415d850 2024-09-19T04:18:07.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a18d0a640 1 -- 172.21.15.76:0/1525326355 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0c0a5a70 con 0x7f2a1415d850 2024-09-19T04:18:07.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.002+0000 7f2a18d0a640 1 -- 172.21.15.76:0/1525326355 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0a5e40 con 0x7f2a1415d850 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- 172.21.15.76:0/1525326355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 msgr2=0x7f2a1415dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/1525326355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1415dcc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009e50 tx=0x7f2a0c01ca00 comp rx=0 tx=0).stop 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- 172.21.15.76:0/1525326355 shutdown_connections 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/1525326355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1415dcc0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/1525326355 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- 172.21.15.76:0/1525326355 >> 172.21.15.76:0/1525326355 conn(0x7f2a140b3e90 msgr2=0x7f2a140b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- 172.21.15.76:0/1525326355 shutdown_connections 2024-09-19T04:18:07.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- 172.21.15.76:0/1525326355 wait complete. 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 Processor -- start 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- start start 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14159d30 0x7f2a14150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1409e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1415d580 con 0x7f2a14159d30 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2a1409b420 con 0x7f2a1415d850 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2a1409bc00 con 0x7f2a140b5c10 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14159d30 0x7f2a14150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14159d30 0x7f2a14150300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41514/0 (socket says 172.21.15.76:41514) 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 -- 172.21.15.76:0/2505566479 learned_addr learned my addr 172.21.15.76:0/2505566479 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a19d0c640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1a50d640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1409e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 -- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 msgr2=0x7f2a140b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 -- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 msgr2=0x7f2a1409e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1409e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 -- 172.21.15.76:0/2505566479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a1409f1c0 con 0x7f2a14159d30 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a19d0c640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:07.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1a50d640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1409e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:07.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1950b640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14159d30 0x7f2a14150300 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e930 tx=0x7f2a0400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a0a7fc640 1 -- 172.21.15.76:0/2505566479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a04017070 con 0x7f2a14159d30 2024-09-19T04:18:07.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a0a7fc640 1 -- 172.21.15.76:0/2505566479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a0400ca00 con 0x7f2a14159d30 2024-09-19T04:18:07.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a1409f4a0 con 0x7f2a14159d30 2024-09-19T04:18:07.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a0a7fc640 1 -- 172.21.15.76:0/2505566479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0400cdb0 con 0x7f2a14159d30 2024-09-19T04:18:07.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.006+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a1409c560 con 0x7f2a14159d30 2024-09-19T04:18:07.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.010+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a1415e610 con 0x7f2a14159d30 2024-09-19T04:18:07.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.010+0000 7f2a0a7fc640 1 -- 172.21.15.76:0/2505566479 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2a0401a070 con 0x7f2a14159d30 2024-09-19T04:18:07.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.010+0000 7f2a0a7fc640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29f80678f0 0x7f29f8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.010+0000 7f2a0a7fc640 1 -- 172.21.15.76:0/2505566479 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f2a04086220 con 0x7f2a14159d30 2024-09-19T04:18:07.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.010+0000 7f2a19d0c640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29f80678f0 0x7f29f8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.010+0000 7f2a19d0c640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29f80678f0 0x7f29f8069db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2a100016f0 tx=0x7f2a10009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.014+0000 7f2a0a7fc640 1 -- 172.21.15.76:0/2505566479 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2a04056c10 con 0x7f2a14159d30 2024-09-19T04:18:07.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.218+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f2a140b4b60 con 0x7f2a14159d30 2024-09-19T04:18:07.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a0a7fc640 1 -- 172.21.15.76:0/2505566479 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f2a04056510 con 0x7f2a14159d30 2024-09-19T04:18:07.225 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-19T04:18:07.225 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:18:07.225 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2024-09-19T04:18:07.226 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2024-09-19T04:18:07.226 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T04:18:07.226 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2024-09-19T04:18:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29f80678f0 msgr2=0x7f29f8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29f80678f0 0x7f29f8069db0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2a100016f0 tx=0x7f2a10009040 comp rx=0 tx=0).stop 2024-09-19T04:18:07.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14159d30 msgr2=0x7f2a14150300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14159d30 0x7f2a14150300 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e930 tx=0x7f2a0400edf0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 shutdown_connections 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29f80678f0 0x7f29f8069db0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a1415d850 0x7f2a1409e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2a14159d30 0x7f2a14150300 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 --2- 172.21.15.76:0/2505566479 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2a140b5c10 0x7f2a140b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 >> 172.21.15.76:0/2505566479 conn(0x7f2a140b3e90 msgr2=0x7f2a1415bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 shutdown_connections 2024-09-19T04:18:07.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.226+0000 7f2a1bf70640 1 -- 172.21.15.76:0/2505566479 wait complete. 2024-09-19T04:18:07.243 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:07.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c47d58640 1 Processor -- start 2024-09-19T04:18:07.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c47d58640 1 -- start start 2024-09-19T04:18:07.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c47d58640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6c400afb70 0x7f6c4015f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c47d58640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c4015fa60 con 0x7f6c4006d3e0 2024-09-19T04:18:07.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c47d58640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c400cfa60 con 0x7f6c400cf5c0 2024-09-19T04:18:07.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c47d58640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6c400cfd40 con 0x7f6c400afb70 2024-09-19T04:18:07.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c45af4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6c400afb70 0x7f6c4015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c45af4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6c400afb70 0x7f6c4015f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41530/0 (socket says 172.21.15.76:41530) 2024-09-19T04:18:07.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c45af4640 1 -- 172.21.15.76:0/2125288963 learned_addr learned my addr 172.21.15.76:0/2125288963 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:07.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c45af4640 1 -- 172.21.15.76:0/2125288963 >> v1:172.21.15.116:6789/0 conn(0x7f6c400cf5c0 legacy=0x7f6c4006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:07.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c45af4640 1 -- 172.21.15.76:0/2125288963 >> v1:172.21.15.76:6789/0 conn(0x7f6c4006d3e0 legacy=0x7f6c4006f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c45af4640 1 -- 172.21.15.76:0/2125288963 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c400d0020 con 0x7f6c400afb70 2024-09-19T04:18:07.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.362+0000 7f6c44af2640 1 -- 172.21.15.76:0/2125288963 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 445208437 0 0) 0x7f6c4015fa60 con 0x7f6c4006d3e0 2024-09-19T04:18:07.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 --2- 172.21.15.76:0/2125288963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c400afb70 0x7f6c4015f520 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70c85d6f21f11d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c44af2640 1 -- 172.21.15.76:0/2125288963 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0b1070 con 0x7f6c400afb70 2024-09-19T04:18:07.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c44af2640 1 -- 172.21.15.76:0/2125288963 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3c0ac410 con 0x7f6c400afb70 2024-09-19T04:18:07.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c44af2640 1 -- 172.21.15.76:0/2125288963 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0ac7e0 con 0x7f6c400afb70 2024-09-19T04:18:07.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- 172.21.15.76:0/2125288963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c400afb70 msgr2=0x7f6c4015f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 --2- 172.21.15.76:0/2125288963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c400afb70 0x7f6c4015f520 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0098a0 tx=0x7f6c3c01c3b0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- 172.21.15.76:0/2125288963 shutdown_connections 2024-09-19T04:18:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 --2- 172.21.15.76:0/2125288963 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c400afb70 0x7f6c4015f520 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- 172.21.15.76:0/2125288963 >> 172.21.15.76:0/2125288963 conn(0x7f6c40072670 msgr2=0x7f6c40072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- 172.21.15.76:0/2125288963 shutdown_connections 2024-09-19T04:18:07.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- 172.21.15.76:0/2125288963 wait complete. 2024-09-19T04:18:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 Processor -- start 2024-09-19T04:18:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- start start 2024-09-19T04:18:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c4006d3e0 0x7f6c4015a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c400afb70 0x7f6c401548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c400cf5c0 0x7f6c40139d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400700d0 con 0x7f6c4006d3e0 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6c400aff50 con 0x7f6c400cf5c0 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6c40070250 con 0x7f6c400afb70 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c4006d3e0 0x7f6c4015a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c4006d3e0 0x7f6c4015a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41538/0 (socket says 172.21.15.76:41538) 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 -- 172.21.15.76:0/812494191 learned_addr learned my addr 172.21.15.76:0/812494191 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 -- 172.21.15.76:0/812494191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c400afb70 msgr2=0x7f6c401548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c400afb70 0x7f6c401548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 -- 172.21.15.76:0/812494191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c400cf5c0 msgr2=0x7f6c40139d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c400cf5c0 0x7f6c40139d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 -- 172.21.15.76:0/812494191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4015aee0 con 0x7f6c4006d3e0 2024-09-19T04:18:07.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c45af4640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c4006d3e0 0x7f6c4015a9a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0af040 tx=0x7f6c3c01abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c367fc640 1 -- 172.21.15.76:0/812494191 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0bb070 con 0x7f6c4006d3e0 2024-09-19T04:18:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c367fc640 1 -- 172.21.15.76:0/812494191 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c3c0acd80 con 0x7f6c4006d3e0 2024-09-19T04:18:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4013aaa0 con 0x7f6c4006d3e0 2024-09-19T04:18:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c367fc640 1 -- 172.21.15.76:0/812494191 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0b1040 con 0x7f6c4006d3e0 2024-09-19T04:18:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.366+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c40138100 con 0x7f6c4006d3e0 2024-09-19T04:18:07.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.370+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4006f1b0 con 0x7f6c4006d3e0 2024-09-19T04:18:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.370+0000 7f6c367fc640 1 -- 172.21.15.76:0/812494191 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6c3c004220 con 0x7f6c4006d3e0 2024-09-19T04:18:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.370+0000 7f6c367fc640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6c240678c0 0x7f6c24069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.370+0000 7f6c367fc640 1 -- 172.21.15.76:0/812494191 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6c3c11fef0 con 0x7f6c4006d3e0 2024-09-19T04:18:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.370+0000 7f6c452f3640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6c240678c0 0x7f6c24069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:07.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.370+0000 7f6c452f3640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6c240678c0 0x7f6c24069d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6c401f83b0 tx=0x7f6c30006bd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:07.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.374+0000 7f6c367fc640 1 -- 172.21.15.76:0/812494191 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6c3c0160f0 con 0x7f6c4006d3e0 2024-09-19T04:18:07.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.570+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c4006d1c0 con 0x7f6c4006d3e0 2024-09-19T04:18:07.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.570+0000 7f6c367fc640 1 -- 172.21.15.76:0/812494191 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5225 (secure 0 0 0) 0x7f6c3c0b6380 con 0x7f6c4006d3e0 2024-09-19T04:18:07.572 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:07.573 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":67,"btime":"2024-09-19T04:18:05:447857+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5336,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":6780,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:04.811666+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6783},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[6783]},"id":5}]} 2024-09-19T04:18:07.573 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 67 2024-09-19T04:18:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6c240678c0 msgr2=0x7f6c24069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6c240678c0 0x7f6c24069d80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6c401f83b0 tx=0x7f6c30006bd0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c4006d3e0 msgr2=0x7f6c4015a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c4006d3e0 0x7f6c4015a9a0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0af040 tx=0x7f6c3c01abf0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 shutdown_connections 2024-09-19T04:18:07.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6c240678c0 0x7f6c24069d80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c400cf5c0 0x7f6c40139d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6c400afb70 0x7f6c401548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 --2- 172.21.15.76:0/812494191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6c4006d3e0 0x7f6c4015a9a0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 >> 172.21.15.76:0/812494191 conn(0x7f6c40072670 msgr2=0x7f6c4015c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 shutdown_connections 2024-09-19T04:18:07.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:07.574+0000 7f6c47d58640 1 -- 172.21.15.76:0/812494191 wait complete. 2024-09-19T04:18:07.994 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:18:01.762504+0000', 'modified': '2024-09-19T04:18:04.811666+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6783}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6783': {'gid': 6783, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3569622848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3569622848}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3569622848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6783, 'qdb_cluster': [6783]} 2024-09-19T04:18:07.994 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:18:07.994 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:18:08.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d8996a640 1 Processor -- start 2024-09-19T04:18:08.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d8996a640 1 -- start start 2024-09-19T04:18:08.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d8996a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4d84160900 0x7f4d8415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d8996a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84071e60 0x7f4d8415cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d8996a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d840d0d70 con 0x7f4d84160520 2024-09-19T04:18:08.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d8996a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4d840d1640 con 0x7f4d84160900 2024-09-19T04:18:08.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d8996a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8415d390 con 0x7f4d84071e60 2024-09-19T04:18:08.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d827fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4d84160900 0x7f4d8415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84071e60 0x7f4d8415cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d827fc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4d84160900 0x7f4d8415c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:39476/0 (socket says 172.21.15.76:39476) 2024-09-19T04:18:08.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d827fc640 1 -- 172.21.15.76:0/3067248196 learned_addr learned my addr 172.21.15.76:0/3067248196 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:08.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d82ffd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4d84160520 0x7f4d84071750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:46830/0 (socket says 172.21.15.76:46830) 2024-09-19T04:18:08.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d837fe640 1 -- 172.21.15.76:0/3067248196 >> v2:172.21.15.116:3300/0 conn(0x7f4d84160900 msgr2=0x7f4d8415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d837fe640 1 --2- 172.21.15.76:0/3067248196 >> v2:172.21.15.116:3300/0 conn(0x7f4d84160900 0x7f4d8415c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d837fe640 1 -- 172.21.15.76:0/3067248196 >> v1:172.21.15.116:6789/0 conn(0x7f4d84160520 legacy=0x7f4d84071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.126+0000 7f4d837fe640 1 -- 172.21.15.76:0/3067248196 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d840d0ff0 con 0x7f4d84071e60 2024-09-19T04:18:08.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d837fe640 1 --2- 172.21.15.76:0/3067248196 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84071e60 0x7f4d8415cb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d740097c0 tx=0x7f4d7401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f72b4e6c9baa0088 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d81ffb640 1 -- 172.21.15.76:0/3067248196 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740b0070 con 0x7f4d84071e60 2024-09-19T04:18:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d81ffb640 1 -- 172.21.15.76:0/3067248196 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d740ab600 con 0x7f4d84071e60 2024-09-19T04:18:08.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d81ffb640 1 -- 172.21.15.76:0/3067248196 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740ab9d0 con 0x7f4d84071e60 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 -- 172.21.15.76:0/3067248196 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84071e60 msgr2=0x7f4d8415cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3067248196 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84071e60 0x7f4d8415cb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4d740097c0 tx=0x7f4d7401c410 comp rx=0 tx=0).stop 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 -- 172.21.15.76:0/3067248196 shutdown_connections 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3067248196 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84071e60 0x7f4d8415cb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3067248196 >> v2:172.21.15.116:3300/0 conn(0x7f4d84160900 0x7f4d8415c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 -- 172.21.15.76:0/3067248196 >> 172.21.15.76:0/3067248196 conn(0x7f4d8406c0c0 msgr2=0x7f4d8406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 -- 172.21.15.76:0/3067248196 shutdown_connections 2024-09-19T04:18:08.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.130+0000 7f4d8996a640 1 -- 172.21.15.76:0/3067248196 wait complete. 2024-09-19T04:18:08.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 Processor -- start 2024-09-19T04:18:08.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 -- start start 2024-09-19T04:18:08.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d84071e60 0x7f4d8413e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84160520 0x7f4d840d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84160900 0x7f4d840d2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84141190 con 0x7f4d84160900 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8415d070 con 0x7f4d84071e60 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4d84141310 con 0x7f4d84160520 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84160900 0x7f4d840d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84160900 0x7f4d840d2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41550/0 (socket says 172.21.15.76:41550) 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 -- 172.21.15.76:0/3988054292 learned_addr learned my addr 172.21.15.76:0/3988054292 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d84071e60 0x7f4d8413e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d827fc640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84160520 0x7f4d840d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 -- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84160520 msgr2=0x7f4d840d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84160520 0x7f4d840d1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 -- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d84071e60 msgr2=0x7f4d8413e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d84071e60 0x7f4d8413e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 -- 172.21.15.76:0/3988054292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d84167270 con 0x7f4d84160900 2024-09-19T04:18:08.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d837fe640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84160900 0x7f4d840d2010 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4d740ae040 tx=0x7f4d740191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d6bfff640 1 -- 172.21.15.76:0/3988054292 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740b8070 con 0x7f4d84160900 2024-09-19T04:18:08.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d6bfff640 1 -- 172.21.15.76:0/3988054292 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d740b4410 con 0x7f4d84160900 2024-09-19T04:18:08.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d841438c0 con 0x7f4d84160900 2024-09-19T04:18:08.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d6bfff640 1 -- 172.21.15.76:0/3988054292 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d740b0040 con 0x7f4d84160900 2024-09-19T04:18:08.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.134+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d84143e80 con 0x7f4d84160900 2024-09-19T04:18:08.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d6bfff640 1 -- 172.21.15.76:0/3988054292 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4d740bb070 con 0x7f4d84160900 2024-09-19T04:18:08.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d6bfff640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d700678c0 0x7f4d70069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d6bfff640 1 -- 172.21.15.76:0/3988054292 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f4d7411fa30 con 0x7f4d84160900 2024-09-19T04:18:08.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d700678c0 0x7f4d70069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 Processor -- start 2024-09-19T04:18:08.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- start start 2024-09-19T04:18:08.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d700678c0 0x7f4d70069d80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4d84145be0 tx=0x7f4d7c007cf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7dd70 0x55a50de7e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7e690 0x55a50de34160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de34a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a50dd17b60 con 0x55a50de7dd70 2024-09-19T04:18:08.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a50dc7ec70 con 0x55a50de346a0 2024-09-19T04:18:08.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a50dc24ed0 con 0x55a50de7e690 2024-09-19T04:18:08.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7dd70 0x55a50de7e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7dd70 0x55a50de7e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41566/0 (socket says 172.21.15.76:41566) 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 -- 172.21.15.76:0/2234399470 learned_addr learned my addr 172.21.15.76:0/2234399470 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d837fe640 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de34a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 -- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7e690 msgr2=0x55a50de34160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d827fc640 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7e690 0x55a50de34160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7e690 0x55a50de34160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 -- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 msgr2=0x55a50de34a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de34a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 -- 172.21.15.76:0/2234399470 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50dc36e50 con 0x55a50de7dd70 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d837fe640 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de34a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7dd70 0x55a50de7e150 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0076c0 tx=0x7f4d7c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310d76bc91a77d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d697fa640 1 -- 172.21.15.76:0/2234399470 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c001c10 con 0x55a50de7dd70 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d697fa640 1 -- 172.21.15.76:0/2234399470 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c005050 con 0x55a50de7dd70 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d697fa640 1 -- 172.21.15.76:0/2234399470 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c005400 con 0x55a50de7dd70 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2234399470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7dd70 msgr2=0x55a50de7e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7dd70 0x55a50de7e150 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0076c0 tx=0x7f4d7c011000 comp rx=0 tx=0).stop 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2234399470 shutdown_connections 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de34a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7e690 0x55a50de34160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2234399470 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7dd70 0x55a50de7e150 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2234399470 >> 172.21.15.76:0/2234399470 conn(0x55a50de36600 msgr2=0x55a50de2e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2234399470 shutdown_connections 2024-09-19T04:18:08.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.138+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2234399470 wait complete. 2024-09-19T04:18:08.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- start start 2024-09-19T04:18:08.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 Processor -- start 2024-09-19T04:18:08.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de7dd70 0x55a50de83eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7e690 0x55a50de843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de6a490 0x55a50de6a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a50dce0b00 con 0x55a50de7e690 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a50dc59360 con 0x55a50de7dd70 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a50dc562d0 con 0x55a50de6a490 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 10 client.0 _ll_register_callbacks cb 0x55a50de1b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7e690 0x55a50de843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7e690 0x55a50de843f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41568/0 (socket says 172.21.15.76:41568) 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 -- 172.21.15.76:0/1902044006 learned_addr learned my addr 172.21.15.76:0/1902044006 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d837fe640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de6a490 0x55a50de6a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de7dd70 0x55a50de83eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 -- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de6a490 msgr2=0x55a50de6a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de6a490 0x55a50de6a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 -- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de7dd70 msgr2=0x55a50de83eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de7dd70 0x55a50de83eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 -- 172.21.15.76:0/1902044006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50dc25120 con 0x55a50de7e690 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d837fe640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de6a490 0x55a50de6a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d827fc640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7e690 0x55a50de843f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007970 tx=0x7f4d78007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b45119779f962fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 1 -- 172.21.15.76:0/1902044006 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7801a020 con 0x55a50de7e690 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 1 -- 172.21.15.76:0/1902044006 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f4d78011940 con 0x55a50de7e690 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c005050 con 0x55a50de7e690 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 1 -- 172.21.15.76:0/1902044006 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f4d78024050 con 0x55a50de7e690 2024-09-19T04:18:08.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 10 client.6927 fetch_fsmap learned FSMap version 67 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55a50dc1e450 con 0x55a50de7e690 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 1 -- 172.21.15.76:0/1902044006 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c005050 con 0x55a50de7e690 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 1 -- 172.21.15.76:0/1902044006 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7f4d78027020 con 0x55a50de7e690 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 10 client.6927 fetch_fsmap finished waiting for FSMap version 67 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 10 client.6927 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 20 client.6927 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de6b280 0x55a50de6b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 4 client.6927 mds_command: new command op to 6783 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 --> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a50dc18c00 con 0x55a50de6b280 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de6b280 0x55a50de6b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 10 client.6927 ms_handle_connect on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de6b280 0x55a50de6b660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6b7f52773a2dc88 server_cookie=6113c5892dcde06c in_seq=0 out_seq=0 2024-09-19T04:18:08.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 1 -- 172.21.15.76:0/1902044006 <== mds.0 v2:172.21.15.76:6836/3569622848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a50dc18c00 con 0x55a50de6b280 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 10 client.6927 handle_command_reply: tid=0 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 client.6927 shutdown 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7e690 msgr2=0x55a50de843f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7e690 0x55a50de843f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007970 tx=0x7f4d78007e30 comp rx=0 tx=0).stop 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 shutdown_connections 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de6a490 0x55a50de6a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de6b280 0x55a50de6b660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de7e690 0x55a50de843f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/1902044006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de7dd70 0x55a50de83eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d627fc640 0 client.6927 ms_handle_reset on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.142+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 >> 172.21.15.76:0/1902044006 conn(0x55a50de346a0 msgr2=0x55a50de359b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.146+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 shutdown_connections 2024-09-19T04:18:08.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.146+0000 7f4d8ca26000 1 -- 172.21.15.76:0/1902044006 wait complete. 2024-09-19T04:18:08.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.146+0000 7f4d8ca26000 20 client.6927 trim_cache size 0 max 16384 2024-09-19T04:18:08.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 Processor -- start 2024-09-19T04:18:08.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 -- start start 2024-09-19T04:18:08.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de346a0 0x55a50de7c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de6a490 0x55a50de7cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de89ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8406ea30 con 0x55a50de346a0 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4d841f5b90 con 0x55a50de6a490 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8406e3e0 con 0x55a50de7dd70 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de346a0 0x55a50de7c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de346a0 0x55a50de7c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41580/0 (socket says 172.21.15.76:41580) 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 -- 172.21.15.76:0/3636816084 learned_addr learned my addr 172.21.15.76:0/3636816084 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 -- 172.21.15.76:0/3636816084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 msgr2=0x55a50de89ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/3636816084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de89ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 -- 172.21.15.76:0/3636816084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de6a490 msgr2=0x55a50de7cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/3636816084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de6a490 0x55a50de7cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 -- 172.21.15.76:0/3636816084 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a50dc05fd0 con 0x55a50de346a0 2024-09-19T04:18:08.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/3636816084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de346a0 0x55a50de7c6f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c007b70 tx=0x7f4d7c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f859b875b37aa6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.162+0000 7f4d68ff9640 1 -- 172.21.15.76:0/3636816084 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c004f80 con 0x55a50de346a0 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d68ff9640 1 -- 172.21.15.76:0/3636816084 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c005120 con 0x55a50de346a0 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d68ff9640 1 -- 172.21.15.76:0/3636816084 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c0054d0 con 0x55a50de346a0 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/3636816084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de346a0 msgr2=0x55a50de7c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/3636816084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de346a0 0x55a50de7c6f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c007b70 tx=0x7f4d7c012000 comp rx=0 tx=0).stop 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/3636816084 shutdown_connections 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/3636816084 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de89ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/3636816084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de6a490 0x55a50de7cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/3636816084 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de346a0 0x55a50de7c6f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/3636816084 >> 172.21.15.76:0/3636816084 conn(0x55a50de69ce0 msgr2=0x55a50de2eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/3636816084 shutdown_connections 2024-09-19T04:18:08.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/3636816084 wait complete. 2024-09-19T04:18:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- start start 2024-09-19T04:18:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 Processor -- start 2024-09-19T04:18:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de83ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de6a490 0x55a50de86ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de871e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a50dc7df80 con 0x55a50de6a490 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a50dc36ca0 con 0x55a50de346a0 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a50dc832a0 con 0x55a50de7dd70 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 10 client.0 _ll_register_callbacks cb 0x7f4d780091d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de6a490 0x55a50de86ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de6a490 0x55a50de86ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41586/0 (socket says 172.21.15.76:41586) 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 -- 172.21.15.76:0/2966248507 learned_addr learned my addr 172.21.15.76:0/2966248507 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de83ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d837fe640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de871e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 -- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 msgr2=0x55a50de871e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de871e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 -- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 msgr2=0x55a50de83ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de83ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 -- 172.21.15.76:0/2966248507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a50dd01750 con 0x55a50de6a490 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de83ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d837fe640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de871e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d827fc640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de6a490 0x55a50de86ca0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007e70 tx=0x7f4d78004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8c9a17297f51ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 1 -- 172.21.15.76:0/2966248507 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7801a070 con 0x55a50de6a490 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 1 -- 172.21.15.76:0/2966248507 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f4d78009790 con 0x55a50de6a490 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d7c005120 con 0x55a50de6a490 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 1 -- 172.21.15.76:0/2966248507 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f4d78032050 con 0x55a50de6a490 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 10 client.6933 fetch_fsmap learned FSMap version 67 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55a50dc0ca30 con 0x55a50de6a490 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 1 -- 172.21.15.76:0/2966248507 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c005120 con 0x55a50de6a490 2024-09-19T04:18:08.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 1 -- 172.21.15.76:0/2966248507 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 67) ==== 3502+0+0 (secure 0 0 0) 0x7f4d78035020 con 0x55a50de6a490 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 10 client.6933 fetch_fsmap finished waiting for FSMap version 67 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 10 client.6933 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 20 client.6933 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de7e690 0x55a50de87720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 4 client.6933 mds_command: new command op to 6783 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 --> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] -- command(tid 0: {"prefix": "status"}) -- 0x55a50dc8c3a0 con 0x55a50de7e690 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de7e690 0x55a50de87720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d82ffd640 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de7e690 0x55a50de87720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9916447ea5b9688 server_cookie=228ecec31116341d in_seq=0 out_seq=0 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 10 client.6933 ms_handle_connect on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.166+0000 7f4d627fc640 1 -- 172.21.15.76:0/2966248507 <== mds.0 v2:172.21.15.76:6836/3569622848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55a50dc8c3a0 con 0x55a50de7e690 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d627fc640 10 client.6933 handle_command_reply: tid=0 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 client.6933 shutdown 2024-09-19T04:18:08.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de6a490 msgr2=0x55a50de86ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de6a490 0x55a50de86ca0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007e70 tx=0x7f4d78004ac0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 shutdown_connections 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a50de7dd70 0x55a50de871e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55a50de7e690 0x55a50de87720 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a50de6a490 0x55a50de86ca0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 --2- 172.21.15.76:0/2966248507 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a50de346a0 0x55a50de83ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 >> 172.21.15.76:0/2966248507 conn(0x55a50de69ce0 msgr2=0x55a50de93250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d627fc640 0 client.6933 ms_handle_reset on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 shutdown_connections 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 1 -- 172.21.15.76:0/2966248507 wait complete. 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8ca26000 20 client.6933 trim_cache size 0 max 16384 2024-09-19T04:18:08.175 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6783, 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 5.7323010270000001, 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 67, 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 80, 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 10.768855537 2024-09-19T04:18:08.176 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d700678c0 msgr2=0x7f4d70069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d700678c0 0x7f4d70069d80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4d84145be0 tx=0x7f4d7c007cf0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84160900 msgr2=0x7f4d840d2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84160900 0x7f4d840d2010 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4d740ae040 tx=0x7f4d740191b0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 shutdown_connections 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4d700678c0 0x7f4d70069d80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4d84160900 0x7f4d840d2010 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4d84160520 0x7f4d840d1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 --2- 172.21.15.76:0/3988054292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d84071e60 0x7f4d8413e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 >> 172.21.15.76:0/3988054292 conn(0x7f4d8406c0c0 msgr2=0x7f4d840b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 shutdown_connections 2024-09-19T04:18:08.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.170+0000 7f4d8996a640 1 -- 172.21.15.76:0/3988054292 wait complete. 2024-09-19T04:18:08.185 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:18:08.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb774422640 1 Processor -- start 2024-09-19T04:18:08.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb774422640 1 -- start start 2024-09-19T04:18:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb774422640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb76c1587a0 0x7fb76c0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb774422640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb76c06e680 0x7fb76c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb774422640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb76c06ec60 con 0x7fb76c0b37d0 2024-09-19T04:18:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb774422640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb76c0b3c40 con 0x7fb76c1587a0 2024-09-19T04:18:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb774422640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb76c06f870 con 0x7fb76c06e680 2024-09-19T04:18:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb7721be640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb76c1587a0 0x7fb76c0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb7719bd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb76c0b37d0 0x7fb76c06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45034/0 (socket says 172.21.15.76:45034) 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.294+0000 7fb7719bd640 1 -- 172.21.15.76:0/4238351942 learned_addr learned my addr 172.21.15.76:0/4238351942 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7729bf640 1 --2- 172.21.15.76:0/4238351942 >> v2:172.21.15.116:3300/0 conn(0x7fb76c06e680 0x7fb76c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7721be640 1 -- 172.21.15.76:0/4238351942 >> v2:172.21.15.116:3300/0 conn(0x7fb76c06e680 msgr2=0x7fb76c072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7721be640 1 --2- 172.21.15.76:0/4238351942 >> v2:172.21.15.116:3300/0 conn(0x7fb76c06e680 0x7fb76c072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7721be640 1 -- 172.21.15.76:0/4238351942 >> v1:172.21.15.76:6789/0 conn(0x7fb76c0b37d0 legacy=0x7fb76c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7711bc640 1 -- 172.21.15.76:0/4238351942 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1032830989 0 0) 0x7fb76c06ec60 con 0x7fb76c0b37d0 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7721be640 1 -- 172.21.15.76:0/4238351942 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c06ef40 con 0x7fb76c1587a0 2024-09-19T04:18:08.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7729bf640 1 --2- 172.21.15.76:0/4238351942 >> v2:172.21.15.116:3300/0 conn(0x7fb76c06e680 0x7fb76c072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7721be640 1 --2- 172.21.15.76:0/4238351942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c1587a0 0x7fb76c0b3290 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0af930 tx=0x7fb76801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da73d5786bbbcee6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7711bc640 1 -- 172.21.15.76:0/4238351942 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680b2070 con 0x7fb76c1587a0 2024-09-19T04:18:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7711bc640 1 -- 172.21.15.76:0/4238351942 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7680a58d0 con 0x7fb76c1587a0 2024-09-19T04:18:08.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb7711bc640 1 -- 172.21.15.76:0/4238351942 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7680a5ca0 con 0x7fb76c1587a0 2024-09-19T04:18:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 -- 172.21.15.76:0/4238351942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c1587a0 msgr2=0x7fb76c0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 --2- 172.21.15.76:0/4238351942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c1587a0 0x7fb76c0b3290 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fb76c0af930 tx=0x7fb76801c900 comp rx=0 tx=0).stop 2024-09-19T04:18:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 -- 172.21.15.76:0/4238351942 shutdown_connections 2024-09-19T04:18:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 --2- 172.21.15.76:0/4238351942 >> v2:172.21.15.116:3300/0 conn(0x7fb76c06e680 0x7fb76c072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 --2- 172.21.15.76:0/4238351942 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c1587a0 0x7fb76c0b3290 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 -- 172.21.15.76:0/4238351942 >> 172.21.15.76:0/4238351942 conn(0x7fb76c0b1f90 msgr2=0x7fb76c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 -- 172.21.15.76:0/4238351942 shutdown_connections 2024-09-19T04:18:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.298+0000 7fb774422640 1 -- 172.21.15.76:0/4238351942 wait complete. 2024-09-19T04:18:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 Processor -- start 2024-09-19T04:18:08.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 -- start start 2024-09-19T04:18:08.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76c06e680 0x7fb76c1511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76c0b37d0 0x7fb76c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c0b0b40 0x7fb76c096580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c073030 con 0x7fb76c0b0b40 2024-09-19T04:18:08.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c072eb0 con 0x7fb76c0b37d0 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb76c0731b0 con 0x7fb76c06e680 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c0b0b40 0x7fb76c096580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c0b0b40 0x7fb76c096580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41600/0 (socket says 172.21.15.76:41600) 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 -- 172.21.15.76:0/3737905961 learned_addr learned my addr 172.21.15.76:0/3737905961 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7721be640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76c06e680 0x7fb76c1511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7719bd640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76c0b37d0 0x7fb76c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 -- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76c06e680 msgr2=0x7fb76c1511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76c06e680 0x7fb76c1511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 -- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76c0b37d0 msgr2=0x7fb76c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76c0b37d0 0x7fb76c0b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 -- 172.21.15.76:0/3737905961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c1463e0 con 0x7fb76c0b0b40 2024-09-19T04:18:08.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7721be640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76c06e680 0x7fb76c1511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7729bf640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c0b0b40 0x7fb76c096580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb76400c7f0 tx=0x7fb76400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb762ffd640 1 -- 172.21.15.76:0/3737905961 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764015070 con 0x7fb76c0b0b40 2024-09-19T04:18:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb762ffd640 1 -- 172.21.15.76:0/3737905961 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb764007800 con 0x7fb76c0b0b40 2024-09-19T04:18:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb762ffd640 1 -- 172.21.15.76:0/3737905961 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb764007bf0 con 0x7fb76c0b0b40 2024-09-19T04:18:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c1466c0 con 0x7fb76c0b0b40 2024-09-19T04:18:08.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c146c80 con 0x7fb76c0b0b40 2024-09-19T04:18:08.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c0b3290 con 0x7fb76c0b0b40 2024-09-19T04:18:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb762ffd640 1 -- 172.21.15.76:0/3737905961 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb764018070 con 0x7fb76c0b0b40 2024-09-19T04:18:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb762ffd640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7500678f0 0x7fb750069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb762ffd640 1 -- 172.21.15.76:0/3737905961 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb764085f30 con 0x7fb76c0b0b40 2024-09-19T04:18:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.302+0000 7fb7721be640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7500678f0 0x7fb750069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:08.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.306+0000 7fb7721be640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7500678f0 0x7fb750069db0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb7680af000 tx=0x7fb7680a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:08.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.310+0000 7fb762ffd640 1 -- 172.21.15.76:0/3737905961 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb7640568e0 con 0x7fb76c0b0b40 2024-09-19T04:18:08.382 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:18:08.382+0000 7ff5fba83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:18:08.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.502+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb76c099e50 con 0x7fb76c0b0b40 2024-09-19T04:18:08.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb762ffd640 1 -- 172.21.15.76:0/3737905961 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7fb7640561e0 con 0x7fb76c0b0b40 2024-09-19T04:18:08.505 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7500678f0 msgr2=0x7fb750069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7500678f0 0x7fb750069db0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb7680af000 tx=0x7fb7680a7070 comp rx=0 tx=0).stop 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c0b0b40 msgr2=0x7fb76c096580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c0b0b40 0x7fb76c096580 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb76400c7f0 tx=0x7fb76400ccb0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 shutdown_connections 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7500678f0 0x7fb750069db0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76c0b0b40 0x7fb76c096580 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76c0b37d0 0x7fb76c0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 --2- 172.21.15.76:0/3737905961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76c06e680 0x7fb76c1511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:08.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 >> 172.21.15.76:0/3737905961 conn(0x7fb76c0b1f90 msgr2=0x7fb76c0b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:08.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 shutdown_connections 2024-09-19T04:18:08.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08.506+0000 7fb774422640 1 -- 172.21.15.76:0/3737905961 wait complete. 2024-09-19T04:18:08.524 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:18:08.524 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:18:08.524 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:18:08.524 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:18:08.524 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:08.524 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:18:08.524 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:18:08.525 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:18:08.525 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:18:08.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:08.526 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:18:08.532 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:18:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:18:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:18:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:18:08.532 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42232sec preferred_lft 42232sec 2024-09-19T04:18:08.533 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:18:08.534 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:08.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:08.535 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:18:08.582 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:18:08.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:08.583 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:18:08.633 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:18:08.633 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:18:08.634 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:18:08.635 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:18:08.635 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:18:08.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:18:08.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:18:08.635 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:18:08.635 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:18:08.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:18:08.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:18:08.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:08.753 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:18:08.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T04:18:08.802 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:18:08.802 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:18:08.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:08.802 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:18:08.858 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T04:18:08.858 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:18:08.858 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:08.906 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:18:08.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:18:08.906 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:08.953 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:18:09.011 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:18:09.058 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:18:09.059 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:18:09.060 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:18:09.061 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:18:09.061 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:18:09.061 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:18:09.061 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:09.061 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:09.061 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:09.061 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:09.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:09.062 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:18:09.110 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:18:09.110 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T04:18:09.154 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:18:09.168 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:18:09.218 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:18:09.218 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:18:09.218 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:18:09.218 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:18:09.218 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:18:09.218 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:18:09.219 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:09.220 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:09.221 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:09.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:09.222 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:18:09.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-09-19T04:18:09.234+0000 7f88ac17a580 -1 init, newargv = 0x562410f7ae90 newargc=15 2024-09-19T04:18:09.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[44668]: starting ceph client 2024-09-19T04:18:09.243 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[44668]: starting fuse 2024-09-19T04:18:10.341 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:18:10.358 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:18:10.408 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:18:10.409 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:10.410 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:10.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:10.411 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:18:10.458 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:18:10.458 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:18:10.458 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:18:10.458 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:18:10.458 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:18:10.458 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:18:10.458 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:18:10.458 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:18:10.459 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-09-19T04:18:10.460 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:18:10.582 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.44668.asok 2024-09-19T04:18:10.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:18:10.594 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.44668.asok 2024-09-19T04:18:10.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:10.594 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.44668.asok status 2024-09-19T04:18:10.754 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:10.754 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:18:10.754 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:18:10.754 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:18:10.754 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-09-19T04:18:10.754 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:18:10.754 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "44668", 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6948, 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 6948 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:18:10.755 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1849549066 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 1849549066 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.6948 192.168.0.1:0/1849549066", 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/1849549066", 2024-09-19T04:18:10.756 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:18:10.757 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 67, 2024-09-19T04:18:10.757 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 80, 2024-09-19T04:18:10.757 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:18:10.757 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:18:10.757 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:18:10.757 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:10.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:10.768 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:18:10.768 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:10.775 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:18:10.775 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:10.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:10.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:10.828 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-19T04:18:10.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.993+0000 7ff92ae8e640 1 Processor -- start 2024-09-19T04:18:10.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.993+0000 7ff92ae8e640 1 -- start start 2024-09-19T04:18:10.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff92ae8e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff92415bec0 0x7ff9240ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:10.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff92ae8e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff92415c2a0 0x7ff9240aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:10.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff92ae8e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9240b3290 con 0x7ff9240af2a0 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff92ae8e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff9240b3570 con 0x7ff92415bec0 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff92ae8e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff9240b3850 con 0x7ff92415c2a0 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff928c2a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff92415bec0 0x7ff9240ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff92942b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff9240af2a0 0x7ff92415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45042/0 (socket says 172.21.15.76:45042) 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff92942b640 1 -- 172.21.15.76:0/393265237 learned_addr learned my addr 172.21.15.76:0/393265237 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff923fff640 1 --2- 172.21.15.76:0/393265237 >> v2:172.21.15.116:3300/0 conn(0x7ff92415c2a0 0x7ff9240aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff928c2a640 1 -- 172.21.15.76:0/393265237 >> v2:172.21.15.116:3300/0 conn(0x7ff92415c2a0 msgr2=0x7ff9240aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff928c2a640 1 --2- 172.21.15.76:0/393265237 >> v2:172.21.15.116:3300/0 conn(0x7ff92415c2a0 0x7ff9240aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff928c2a640 1 -- 172.21.15.76:0/393265237 >> v1:172.21.15.76:6789/0 conn(0x7ff9240af2a0 legacy=0x7ff92415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:10.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff928c2a640 1 -- 172.21.15.76:0/393265237 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9240b3b30 con 0x7ff92415bec0 2024-09-19T04:18:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:10.997+0000 7ff928c2a640 1 --2- 172.21.15.76:0/393265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92415bec0 0x7ff9240ad200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff9240ae510 tx=0x7ff91401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49d90971689321a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff9237fe640 1 -- 172.21.15.76:0/393265237 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140ab650 con 0x7ff92415bec0 2024-09-19T04:18:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff9237fe640 1 -- 172.21.15.76:0/393265237 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9140b0070 con 0x7ff92415bec0 2024-09-19T04:18:11.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff9237fe640 1 -- 172.21.15.76:0/393265237 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140aba10 con 0x7ff92415bec0 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- 172.21.15.76:0/393265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92415bec0 msgr2=0x7ff9240ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/393265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92415bec0 0x7ff9240ad200 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff9240ae510 tx=0x7ff91401c450 comp rx=0 tx=0).stop 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- 172.21.15.76:0/393265237 shutdown_connections 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/393265237 >> v2:172.21.15.116:3300/0 conn(0x7ff92415c2a0 0x7ff9240aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/393265237 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92415bec0 0x7ff9240ad200 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- 172.21.15.76:0/393265237 >> 172.21.15.76:0/393265237 conn(0x7ff9240b1f90 msgr2=0x7ff9240b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- 172.21.15.76:0/393265237 shutdown_connections 2024-09-19T04:18:11.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- 172.21.15.76:0/393265237 wait complete. 2024-09-19T04:18:11.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 Processor -- start 2024-09-19T04:18:11.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- start start 2024-09-19T04:18:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9240af2a0 0x7ff924151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92415bec0 0x7ff9241664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff92415c2a0 0x7ff9240cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240af6b0 con 0x7ff9240af2a0 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff9240ad740 con 0x7ff92415bec0 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff924169000 con 0x7ff92415c2a0 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9240af2a0 0x7ff924151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9240af2a0 0x7ff924151eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41612/0 (socket says 172.21.15.76:41612) 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 -- 172.21.15.76:0/2863593203 learned_addr learned my addr 172.21.15.76:0/2863593203 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 -- 172.21.15.76:0/2863593203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff92415c2a0 msgr2=0x7ff9240cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff92415c2a0 0x7ff9240cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 -- 172.21.15.76:0/2863593203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92415bec0 msgr2=0x7ff9241664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92415bec0 0x7ff9241664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 -- 172.21.15.76:0/2863593203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92406fa80 con 0x7ff9240af2a0 2024-09-19T04:18:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff928c2a640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9240af2a0 0x7ff924151eb0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ad040 tx=0x7ff91401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff9217fa640 1 -- 172.21.15.76:0/2863593203 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140bf610 con 0x7ff9240af2a0 2024-09-19T04:18:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff9217fa640 1 -- 172.21.15.76:0/2863593203 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9140bf7b0 con 0x7ff9240af2a0 2024-09-19T04:18:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff92415eac0 con 0x7ff9240af2a0 2024-09-19T04:18:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff9217fa640 1 -- 172.21.15.76:0/2863593203 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140bfb80 con 0x7ff9240af2a0 2024-09-19T04:18:11.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.001+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff92415f080 con 0x7ff9240af2a0 2024-09-19T04:18:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.005+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9240ad200 con 0x7ff9240af2a0 2024-09-19T04:18:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.005+0000 7ff9217fa640 1 -- 172.21.15.76:0/2863593203 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff9140bd020 con 0x7ff9240af2a0 2024-09-19T04:18:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.005+0000 7ff9217fa640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9100678c0 0x7ff910069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.005+0000 7ff9217fa640 1 -- 172.21.15.76:0/2863593203 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7ff914120bd0 con 0x7ff9240af2a0 2024-09-19T04:18:11.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.005+0000 7ff923fff640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9100678c0 0x7ff910069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.005+0000 7ff923fff640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9100678c0 0x7ff910069d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff924159c40 tx=0x7ff918008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.009+0000 7ff9217fa640 1 -- 172.21.15.76:0/2863593203 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff914016070 con 0x7ff9240af2a0 2024-09-19T04:18:11.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.201+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ff92406fcd0 con 0x7ff9240af2a0 2024-09-19T04:18:11.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.485+0000 7ff9217fa640 1 -- 172.21.15.76:0/2863593203 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v68) ==== 105+0+0 (secure 0 0 0) 0x7ff9140f16f0 con 0x7ff9240af2a0 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9100678c0 msgr2=0x7ff910069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9100678c0 0x7ff910069d80 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff924159c40 tx=0x7ff918008040 comp rx=0 tx=0).stop 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9240af2a0 msgr2=0x7ff924151eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9240af2a0 0x7ff924151eb0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ad040 tx=0x7ff91401ac90 comp rx=0 tx=0).stop 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 shutdown_connections 2024-09-19T04:18:11.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9100678c0 0x7ff910069d80 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff92415c2a0 0x7ff9240cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff92415bec0 0x7ff9241664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 --2- 172.21.15.76:0/2863593203 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9240af2a0 0x7ff924151eb0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 >> 172.21.15.76:0/2863593203 conn(0x7ff9240b1f90 msgr2=0x7ff9240b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 shutdown_connections 2024-09-19T04:18:11.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.489+0000 7ff92ae8e640 1 -- 172.21.15.76:0/2863593203 wait complete. 2024-09-19T04:18:11.516 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:11.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.629+0000 7f15e955c640 1 Processor -- start 2024-09-19T04:18:11.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.629+0000 7f15e955c640 1 -- start start 2024-09-19T04:18:11.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e955c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15e415bec0 0x7f15e40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e955c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e40b3290 con 0x7f15e40af470 2024-09-19T04:18:11.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e955c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15e40b3570 con 0x7f15e415c2a0 2024-09-19T04:18:11.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e955c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f15e40b3850 con 0x7f15e415bec0 2024-09-19T04:18:11.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f15e40af470 0x7f15e406e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45054/0 (socket says 172.21.15.76:45054) 2024-09-19T04:18:11.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 -- 172.21.15.76:0/4082741574 learned_addr learned my addr 172.21.15.76:0/4082741574 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:11.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e2ffd640 1 --2- 172.21.15.76:0/4082741574 >> v2:172.21.15.116:3300/0 conn(0x7f15e415bec0 0x7f15e40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2559439596 0 0) 0x7f15e40b3290 con 0x7f15e40af470 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f15cc003530 con 0x7f15e40af470 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2993111848 0 0) 0x7f15cc003530 con 0x7f15e40af470 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 >> v2:172.21.15.116:3300/0 conn(0x7f15e415bec0 msgr2=0x7f15e40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 --2- 172.21.15.76:0/4082741574 >> v2:172.21.15.116:3300/0 conn(0x7f15e415bec0 0x7f15e40ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 >> v1:172.21.15.116:6789/0 conn(0x7f15e415c2a0 legacy=0x7f15e40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e40b3b30 con 0x7f15e40af470 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f15dc001a00 con 0x7f15e40af470 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e2ffd640 1 --2- 172.21.15.76:0/4082741574 >> v2:172.21.15.116:3300/0 conn(0x7f15e415bec0 0x7f15e40ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 >> v1:172.21.15.76:6789/0 conn(0x7f15e40af470 legacy=0x7f15e406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0039c0 0x7f15cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc004810 0x7f15cc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15cc0051d0 0x7f15cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40b3850 con 0x7f15cc0051d0 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15cc00c120 con 0x7f15cc004810 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f15cc00c320 con 0x7f15cc0039c0 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15cc0051d0 0x7f15cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e2ffd640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0039c0 0x7f15cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e27fc640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc004810 0x7f15cc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 -- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0039c0 msgr2=0x7f15cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0039c0 0x7f15cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 -- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc004810 msgr2=0x7f15cc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc004810 0x7f15cc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 -- 172.21.15.76:0/4082741574 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15cc003530 con 0x7f15cc0051d0 2024-09-19T04:18:11.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e2ffd640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0039c0 0x7f15cc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 -- 172.21.15.76:0/4082741574 reap_dead start 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.633+0000 7f15e37fe640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15cc0051d0 0x7f15cc00b9d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00a840 tx=0x7f15dc01d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc66796a65c46d52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc00a460 con 0x7f15cc0051d0 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15dc0ac640 con 0x7f15cc0051d0 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e1ffb640 1 -- 172.21.15.76:0/4082741574 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc0aca30 con 0x7f15cc0051d0 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- 172.21.15.76:0/4082741574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15cc0051d0 msgr2=0x7f15cc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15cc0051d0 0x7f15cc00b9d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00a840 tx=0x7f15dc01d360 comp rx=0 tx=0).stop 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- 172.21.15.76:0/4082741574 shutdown_connections 2024-09-19T04:18:11.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 --2- 172.21.15.76:0/4082741574 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15cc0051d0 0x7f15cc00b9d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- 172.21.15.76:0/4082741574 >> 172.21.15.76:0/4082741574 conn(0x7f15e40b1f90 msgr2=0x7f15e40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- 172.21.15.76:0/4082741574 shutdown_connections 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- 172.21.15.76:0/4082741574 wait complete. 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 Processor -- start 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- start start 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0051d0 0x7f15e409d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e409a4b0 0x7f15e41f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e41f5a20 0x7f15e414e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40afc90 con 0x7f15e41f5a20 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15e40b40f0 con 0x7f15e409a4b0 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f15e40aed60 con 0x7f15cc0051d0 2024-09-19T04:18:11.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e41f5a20 0x7f15e414e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e41f5a20 0x7f15e414e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:41632/0 (socket says 172.21.15.76:41632) 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 -- 172.21.15.76:0/1261328484 learned_addr learned my addr 172.21.15.76:0/1261328484 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e27fc640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e409a4b0 0x7f15e41f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e2ffd640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0051d0 0x7f15e409d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 -- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0051d0 msgr2=0x7f15e409d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0051d0 0x7f15e409d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 -- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e409a4b0 msgr2=0x7f15e41f54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e409a4b0 0x7f15e41f54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 -- 172.21.15.76:0/1261328484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e41f5f30 con 0x7f15e41f5a20 2024-09-19T04:18:11.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e37fe640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e41f5a20 0x7f15e414e280 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00a030 tx=0x7f15cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15cb7fe640 1 -- 172.21.15.76:0/1261328484 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc001fc0 con 0x7f15e41f5a20 2024-09-19T04:18:11.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15cb7fe640 1 -- 172.21.15.76:0/1261328484 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15dc001c00 con 0x7f15e41f5a20 2024-09-19T04:18:11.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e41f6210 con 0x7f15e41f5a20 2024-09-19T04:18:11.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15cb7fe640 1 -- 172.21.15.76:0/1261328484 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc001940 con 0x7f15e41f5a20 2024-09-19T04:18:11.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.637+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e40c56f0 con 0x7f15e41f5a20 2024-09-19T04:18:11.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.641+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e40c5200 con 0x7f15e41f5a20 2024-09-19T04:18:11.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.641+0000 7f15cb7fe640 1 -- 172.21.15.76:0/1261328484 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f15dc0b5030 con 0x7f15e41f5a20 2024-09-19T04:18:11.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.641+0000 7f15cb7fe640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15d00678c0 0x7f15d0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:11.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.641+0000 7f15cb7fe640 1 -- 172.21.15.76:0/1261328484 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f15dc11f620 con 0x7f15e41f5a20 2024-09-19T04:18:11.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.641+0000 7f15e2ffd640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15d00678c0 0x7f15d0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:11.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.641+0000 7f15e2ffd640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15d00678c0 0x7f15d0069d80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f15d400e990 tx=0x7f15d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:11.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.645+0000 7f15cb7fe640 1 -- 172.21.15.76:0/1261328484 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f15dc017070 con 0x7f15e41f5a20 2024-09-19T04:18:11.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.841+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15e40af0f0 con 0x7f15e41f5a20 2024-09-19T04:18:11.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15cb7fe640 1 -- 172.21.15.76:0/1261328484 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5241 (secure 0 0 0) 0x7f15dc0f4e20 con 0x7f15e41f5a20 2024-09-19T04:18:11.846 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:11.846 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":69,"btime":"2024-09-19T04:18:11:491672+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5336,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:11.491650+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":6783,"mds_1":6780},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[6783]},"id":5}]} 2024-09-19T04:18:11.846 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 69 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15d00678c0 msgr2=0x7f15d0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15d00678c0 0x7f15d0069d80 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f15d400e990 tx=0x7f15d4008040 comp rx=0 tx=0).stop 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e41f5a20 msgr2=0x7f15e414e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e41f5a20 0x7f15e414e280 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00a030 tx=0x7f15cc005840 comp rx=0 tx=0).stop 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 shutdown_connections 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15d00678c0 0x7f15d0069d80 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15e41f5a20 0x7f15e414e280 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e409a4b0 0x7f15e41f54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 --2- 172.21.15.76:0/1261328484 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15cc0051d0 0x7f15e409d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:11.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 >> 172.21.15.76:0/1261328484 conn(0x7f15e40b1f90 msgr2=0x7f15e4159db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:11.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 shutdown_connections 2024-09-19T04:18:11.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:11.845+0000 7f15e955c640 1 -- 172.21.15.76:0/1261328484 wait complete. 2024-09-19T04:18:12.266 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:18:01.762504+0000', 'modified': '2024-09-19T04:18:11.491650+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 6783, 'mds_1': 6780}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6780': {'gid': 6780, 'name': 'a', 'rank': 1, 'incarnation': 69, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.76:6835/603375312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 603375312}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 603375312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6783': {'gid': 6783, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3569622848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3569622848}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3569622848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6783, 'qdb_cluster': [6783]} 2024-09-19T04:18:12.266 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6780:up:creating 2024-09-19T04:18:13.268 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:13.382 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:18:13.381+0000 7ff5fba83640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T04:18:13.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.397+0000 7fc971075640 1 Processor -- start 2024-09-19T04:18:13.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc971075640 1 -- start start 2024-09-19T04:18:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc971075640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc96c0b38e0 0x7fc96c0d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc971075640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc96c0d4750 0x7fc96c06d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc971075640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c0d2b20 con 0x7fc96c15e470 2024-09-19T04:18:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc971075640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc96c0d3bc0 con 0x7fc96c0d4750 2024-09-19T04:18:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc971075640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc96c0d4ba0 con 0x7fc96c0b38e0 2024-09-19T04:18:13.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96b59e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc96c0d4750 0x7fc96c06d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc96c15e470 0x7fc96c0d15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55140/0 (socket says 172.21.15.76:55140) 2024-09-19T04:18:13.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96ad9d640 1 -- 172.21.15.76:0/3178264593 learned_addr learned my addr 172.21.15.76:0/3178264593 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:13.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96b59e640 1 -- 172.21.15.76:0/3178264593 >> v2:172.21.15.116:3300/0 conn(0x7fc96c0b38e0 msgr2=0x7fc96c0d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:13.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96b59e640 1 --2- 172.21.15.76:0/3178264593 >> v2:172.21.15.116:3300/0 conn(0x7fc96c0b38e0 0x7fc96c0d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96b59e640 1 -- 172.21.15.76:0/3178264593 >> v1:172.21.15.76:6789/0 conn(0x7fc96c15e470 legacy=0x7fc96c0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc969d9b640 1 -- 172.21.15.76:0/3178264593 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48864374 0 0) 0x7fc96c0d2b20 con 0x7fc96c15e470 2024-09-19T04:18:13.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96b59e640 1 -- 172.21.15.76:0/3178264593 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0d2e00 con 0x7fc96c0d4750 2024-09-19T04:18:13.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc96b59e640 1 --2- 172.21.15.76:0/3178264593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0d4750 0x7fc96c06d1b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc96c09dda0 tx=0x7fc96401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a1bf19858ba1b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc969d9b640 1 -- 172.21.15.76:0/3178264593 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9640b2070 con 0x7fc96c0d4750 2024-09-19T04:18:13.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc969d9b640 1 -- 172.21.15.76:0/3178264593 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9640a5760 con 0x7fc96c0d4750 2024-09-19T04:18:13.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.401+0000 7fc969d9b640 1 -- 172.21.15.76:0/3178264593 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9640a5b30 con 0x7fc96c0d4750 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- 172.21.15.76:0/3178264593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0d4750 msgr2=0x7fc96c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 --2- 172.21.15.76:0/3178264593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0d4750 0x7fc96c06d1b0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc96c09dda0 tx=0x7fc96401c8a0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- 172.21.15.76:0/3178264593 shutdown_connections 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 --2- 172.21.15.76:0/3178264593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c0d4750 0x7fc96c06d1b0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 --2- 172.21.15.76:0/3178264593 >> v2:172.21.15.116:3300/0 conn(0x7fc96c0b38e0 0x7fc96c0d4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- 172.21.15.76:0/3178264593 >> 172.21.15.76:0/3178264593 conn(0x7fc96c09d730 msgr2=0x7fc96c08bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- 172.21.15.76:0/3178264593 shutdown_connections 2024-09-19T04:18:13.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- 172.21.15.76:0/3178264593 wait complete. 2024-09-19T04:18:13.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 Processor -- start 2024-09-19T04:18:13.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- start start 2024-09-19T04:18:13.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc96c0b38e0 0x7fc96c0b97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96c0d4750 0x7fc96c156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c15e470 0x7fc96c160380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c06e2d0 con 0x7fc96c15e470 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c06e150 con 0x7fc96c0d4750 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c06e450 con 0x7fc96c0b38e0 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c15e470 0x7fc96c160380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c15e470 0x7fc96c160380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47010/0 (socket says 172.21.15.76:47010) 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 -- 172.21.15.76:0/4004286275 learned_addr learned my addr 172.21.15.76:0/4004286275 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 -- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc96c0b38e0 msgr2=0x7fc96c0b97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96ad9d640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc96c0b38e0 0x7fc96c0b97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96a59c640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96c0d4750 0x7fc96c156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc96c0b38e0 0x7fc96c0b97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 -- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96c0d4750 msgr2=0x7fc96c156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96c0d4750 0x7fc96c156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 -- 172.21.15.76:0/4004286275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0b7e20 con 0x7fc96c15e470 2024-09-19T04:18:13.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc96b59e640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c15e470 0x7fc96c160380 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc96401a4d0 tx=0x7fc964019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc9537fe640 1 -- 172.21.15.76:0/4004286275 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9640bd020 con 0x7fc96c15e470 2024-09-19T04:18:13.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc9537fe640 1 -- 172.21.15.76:0/4004286275 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9640b8670 con 0x7fc96c15e470 2024-09-19T04:18:13.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c15f8a0 con 0x7fc96c15e470 2024-09-19T04:18:13.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc9537fe640 1 -- 172.21.15.76:0/4004286275 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9640b2040 con 0x7fc96c15e470 2024-09-19T04:18:13.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.405+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c15fe60 con 0x7fc96c15e470 2024-09-19T04:18:13.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.409+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96c0d4210 con 0x7fc96c15e470 2024-09-19T04:18:13.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.409+0000 7fc9537fe640 1 -- 172.21.15.76:0/4004286275 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc9640c0020 con 0x7fc96c15e470 2024-09-19T04:18:13.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.409+0000 7fc9537fe640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9580678c0 0x7fc958069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:13.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.409+0000 7fc9537fe640 1 -- 172.21.15.76:0/4004286275 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fc964120de0 con 0x7fc96c15e470 2024-09-19T04:18:13.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.409+0000 7fc96ad9d640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9580678c0 0x7fc958069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:13.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.409+0000 7fc96ad9d640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9580678c0 0x7fc958069d80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000f70 tx=0x7fc95c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:13.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.413+0000 7fc9537fe640 1 -- 172.21.15.76:0/4004286275 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc9640c7030 con 0x7fc96c15e470 2024-09-19T04:18:13.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.613+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc96c15f680 con 0x7fc96c15e470 2024-09-19T04:18:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.613+0000 7fc9537fe640 1 -- 172.21.15.76:0/4004286275 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5263 (secure 0 0 0) 0x7fc964016520 con 0x7fc96c15e470 2024-09-19T04:18:13.615 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:13.616 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":72,"btime":"2024-09-19T04:18:13:500921+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:13.500909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6783,"mds_1":6780,"mds_2":5336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5336":{"gid":5336,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[5336,6780,6783]},"id":5}]} 2024-09-19T04:18:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 72 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9580678c0 msgr2=0x7fc958069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9580678c0 0x7fc958069d80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc95c000f70 tx=0x7fc95c009040 comp rx=0 tx=0).stop 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c15e470 msgr2=0x7fc96c160380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c15e470 0x7fc96c160380 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc96401a4d0 tx=0x7fc964019640 comp rx=0 tx=0).stop 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 shutdown_connections 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc9580678c0 0x7fc958069d80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc96c15e470 0x7fc96c160380 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96c0d4750 0x7fc96c156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 --2- 172.21.15.76:0/4004286275 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc96c0b38e0 0x7fc96c0b97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 >> 172.21.15.76:0/4004286275 conn(0x7fc96c09d730 msgr2=0x7fc96c15d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 shutdown_connections 2024-09-19T04:18:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:13.617+0000 7fc971075640 1 -- 172.21.15.76:0/4004286275 wait complete. 2024-09-19T04:18:14.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 72, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:18:01.762504+0000', 'modified': '2024-09-19T04:18:13.500909+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 6783, 'mds_1': 6780, 'mds_2': 5336}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5336': {'gid': 5336, 'name': 'd', 'rank': 2, 'incarnation': 71, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6835/3224374251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3224374251}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3224374251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6780': {'gid': 6780, 'name': 'a', 'rank': 1, 'incarnation': 69, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6835/603375312', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 603375312}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 603375312}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_6783': {'gid': 6783, 'name': 'c', 'rank': 0, 'incarnation': 63, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3569622848', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3569622848}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3569622848}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6783, 'qdb_cluster': [5336, 6780, 6783]} 2024-09-19T04:18:14.041 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:18:14.041 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T04:18:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cdc08b640 1 Processor -- start 2024-09-19T04:18:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cdc08b640 1 -- start start 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cdc08b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1cd415bec0 0x7f1cd40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cdc08b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 0x7f1cd40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cdc08b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cd40b3290 con 0x7f1cd40af2a0 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cdc08b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1cd40b3570 con 0x7f1cd415bec0 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cdc08b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd40b3850 con 0x7f1cd415c2a0 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1cd40af2a0 0x7f1cd415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55150/0 (socket says 172.21.15.76:55150) 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 -- 172.21.15.76:0/1689016675 learned_addr learned my addr 172.21.15.76:0/1689016675 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/1689016675 >> v2:172.21.15.116:3300/0 conn(0x7f1cd415bec0 0x7f1cd40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 0x7f1cd40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2804576634 0 0) 0x7f1cd40b3290 con 0x7f1cd40af2a0 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cbc003530 con 0x7f1cd40af2a0 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2701170854 0 0) 0x7f1cbc003530 con 0x7f1cd40af2a0 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 msgr2=0x7f1cd40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 0x7f1cd40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 >> v2:172.21.15.116:3300/0 conn(0x7f1cd415bec0 msgr2=0x7f1cd40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 --2- 172.21.15.76:0/1689016675 >> v2:172.21.15.116:3300/0 conn(0x7f1cd415bec0 0x7f1cd40ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd40b3b30 con 0x7f1cd40af2a0 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f1cc4001680 con 0x7f1cd40af2a0 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 >> v1:172.21.15.76:6789/0 conn(0x7f1cd40af2a0 legacy=0x7f1cd415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 0x7f1cbc0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cbc003f40 0x7f1cbc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cbc004d50 0x7f1cbc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd40b3570 con 0x7f1cbc004d50 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd40b3850 con 0x7f1cbc003f40 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc00b9d0 con 0x7f1cd415c2a0 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cbc004d50 0x7f1cbc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cbc003f40 0x7f1cbc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 -- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 msgr2=0x7f1cbc0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 0x7f1cbc0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd415c2a0 0x7f1cbc0039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 -- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cbc003f40 msgr2=0x7f1cbc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cbc003f40 0x7f1cbc004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.173+0000 7f1cda628640 1 -- 172.21.15.76:0/1689016675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc003530 con 0x7f1cbc004d50 2024-09-19T04:18:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cda628640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cbc004d50 0x7f1cbc0051d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400c240 tx=0x7f1cc401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f349ca431750bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4007d40 con 0x7f1cbc004d50 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc40a9020 con 0x7f1cbc004d50 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cd8e25640 1 -- 172.21.15.76:0/1689016675 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc40a5bc0 con 0x7f1cbc004d50 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cdc08b640 1 -- 172.21.15.76:0/1689016675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cbc004d50 msgr2=0x7f1cbc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cdc08b640 1 --2- 172.21.15.76:0/1689016675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cbc004d50 0x7f1cbc0051d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400c240 tx=0x7f1cc401cc40 comp rx=0 tx=0).stop 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cda628640 1 -- 172.21.15.76:0/1689016675 reap_dead start 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cdc08b640 1 -- 172.21.15.76:0/1689016675 shutdown_connections 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cdc08b640 1 -- 172.21.15.76:0/1689016675 >> 172.21.15.76:0/1689016675 conn(0x7f1cd40b1f90 msgr2=0x7f1cd40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cdc08b640 1 -- 172.21.15.76:0/1689016675 shutdown_connections 2024-09-19T04:18:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.177+0000 7f1cdc08b640 1 -- 172.21.15.76:0/1689016675 wait complete. 2024-09-19T04:18:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 Processor -- start 2024-09-19T04:18:14.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 -- start start 2024-09-19T04:18:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd40cec80 0x7f1cd415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cd415cc20 0x7f1cd415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd415e580 0x7f1cd415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd40af730 con 0x7f1cd415cc20 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd40b40f0 con 0x7f1cd415e580 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd40afc90 con 0x7f1cd40cec80 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cd415cc20 0x7f1cd415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cd415cc20 0x7f1cd415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47014/0 (socket says 172.21.15.76:47014) 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 -- 172.21.15.76:0/703696492 learned_addr learned my addr 172.21.15.76:0/703696492 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cda628640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd415e580 0x7f1cd415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd40cec80 0x7f1cd415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 -- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd40cec80 msgr2=0x7f1cd415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd40cec80 0x7f1cd415c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 -- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd415e580 msgr2=0x7f1cd415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd415e580 0x7f1cd415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 -- 172.21.15.76:0/703696492 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd4072b70 con 0x7f1cd415cc20 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cda628640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd415e580 0x7f1cd415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd9626640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cd415cc20 0x7f1cd415d0a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8010890 tx=0x7f1cc8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd27fc640 1 -- 172.21.15.76:0/703696492 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc800f040 con 0x7f1cd415cc20 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd27fc640 1 -- 172.21.15.76:0/703696492 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc800eb60 con 0x7f1cd415cc20 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd4072e50 con 0x7f1cd415cc20 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cd27fc640 1 -- 172.21.15.76:0/703696492 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc801c400 con 0x7f1cd415cc20 2024-09-19T04:18:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.181+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd4073410 con 0x7f1cd415cc20 2024-09-19T04:18:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd27fc640 1 -- 172.21.15.76:0/703696492 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1cc801c5a0 con 0x7f1cd415cc20 2024-09-19T04:18:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd27fc640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1cc00678c0 0x7f1cc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1cc00678c0 0x7f1cc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd27fc640 1 -- 172.21.15.76:0/703696492 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f1cc8086400 con 0x7f1cd415cc20 2024-09-19T04:18:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1cc00678c0 0x7f1cc0069d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0074a0 tx=0x7f1ccc00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 Processor -- start 2024-09-19T04:18:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 -- start start 2024-09-19T04:18:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a00682f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00638d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00639290 0x561a00639670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561a00520b60 con 0x561a00683440 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561a00487c70 con 0x561a00682b20 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561a0042ded0 con 0x561a00639290 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00638d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00638d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47028/0 (socket says 172.21.15.76:47028) 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 -- 172.21.15.76:0/4223667822 learned_addr learned my addr 172.21.15.76:0/4223667822 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a00682f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cda628640 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00639290 0x561a00639670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 -- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00639290 msgr2=0x561a00639670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00639290 0x561a00639670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 -- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 msgr2=0x561a00682f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a00682f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 -- 172.21.15.76:0/4223667822 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0043fe50 con 0x561a00683440 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a00682f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cd9626640 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00638d50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009310 tx=0x7f1cc8056da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4149ad1f96956e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cbb7fe640 1 -- 172.21.15.76:0/4223667822 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc805f280 con 0x561a00683440 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cbb7fe640 1 -- 172.21.15.76:0/4223667822 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc80549a0 con 0x561a00683440 2024-09-19T04:18:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.185+0000 7f1cbb7fe640 1 -- 172.21.15.76:0/4223667822 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8052d10 con 0x561a00683440 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/4223667822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 msgr2=0x561a00638d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00638d50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8009310 tx=0x7f1cc8056da0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/4223667822 shutdown_connections 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00639290 0x561a00639670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00638d50 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- 172.21.15.76:0/4223667822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a00682f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/4223667822 >> 172.21.15.76:0/4223667822 conn(0x561a0063f600 msgr2=0x561a00637630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/4223667822 shutdown_connections 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/4223667822 wait complete. 2024-09-19T04:18:14.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- start start 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 Processor -- start 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a0063ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00643cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 0x561a00644230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561a004e9b00 con 0x561a00683440 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561a00462360 con 0x561a00682b20 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561a0045f2d0 con 0x561a0063e330 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 10 client.0 _ll_register_callbacks cb 0x561a00624820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00643cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00643cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47034/0 (socket says 172.21.15.76:47034) 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 -- 172.21.15.76:0/997899065 learned_addr learned my addr 172.21.15.76:0/997899065 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cda628640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 0x561a00644230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a0063ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 -- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 msgr2=0x561a00644230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 0x561a00644230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 -- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 msgr2=0x561a0063ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a0063ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 -- 172.21.15.76:0/997899065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0042e120 con 0x561a00683440 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a0063ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9626640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00643cf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8056630 tx=0x7f1cc80092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4598d2b95202e276 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/997899065 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8072070 con 0x561a00683440 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/997899065 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f1cc805bbe0 con 0x561a00683440 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc80549a0 con 0x561a00683440 2024-09-19T04:18:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/997899065 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f1cc807c050 con 0x561a00683440 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 10 client.6984 fetch_fsmap learned FSMap version 72 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x561a00427450 con 0x561a00683440 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/997899065 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc80549a0 con 0x561a00683440 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/997899065 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 72) ==== 3542+0+0 (secure 0 0 0) 0x7f1cc8024030 con 0x561a00683440 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 10 client.6984 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 10 client.6984 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 20 client.6984 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a00691280 0x561a00693670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 4 client.6984 mds_command: new command op to 5336 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 --> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a00421c00 con 0x561a00691280 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.189+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a00691280 0x561a00693670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a00691280 0x561a00693670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=787759fbe58dbaa8 server_cookie=5445a708809cfeff in_seq=0 out_seq=0 2024-09-19T04:18:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cb8ff9640 10 client.6984 ms_handle_connect on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/997899065 <== mds.2 v2:172.21.15.116:6834/3224374251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a00421c00 con 0x561a00691280 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cb8ff9640 10 client.6984 handle_command_reply: tid=0 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 client.6984 shutdown 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 msgr2=0x561a00643cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00643cf0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8056630 tx=0x7f1cc80092a0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 shutdown_connections 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a00691280 0x561a00693670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 0x561a00644230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cb8ff9640 0 client.6984 ms_handle_reset on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00683440 0x561a00643cf0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 --2- 172.21.15.76:0/997899065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a0063ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 >> 172.21.15.76:0/997899065 conn(0x561a00639290 msgr2=0x561a00637d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 shutdown_connections 2024-09-19T04:18:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 1 -- 172.21.15.76:0/997899065 wait complete. 2024-09-19T04:18:14.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.193+0000 7f1cdf147000 20 client.6984 trim_cache size 0 max 16384 2024-09-19T04:18:14.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 Processor -- start 2024-09-19T04:18:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- start start 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 0x561a00681c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00682b20 0x561a00682170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00683440 0x561a00693670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4200e50 con 0x561a00682b20 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4201520 con 0x561a00683440 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd41fff20 con 0x561a0063e330 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00682b20 0x561a00682170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00682b20 0x561a00682170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47038/0 (socket says 172.21.15.76:47038) 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 -- 172.21.15.76:0/1609368907 learned_addr learned my addr 172.21.15.76:0/1609368907 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 -- 172.21.15.76:0/1609368907 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 msgr2=0x561a00681c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1609368907 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 0x561a00681c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 -- 172.21.15.76:0/1609368907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00683440 msgr2=0x561a00693670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1609368907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00683440 0x561a00693670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 -- 172.21.15.76:0/1609368907 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0040efd0 con 0x561a00682b20 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1609368907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00682b20 0x561a00682170 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e540 tx=0x7f1cc8081000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8fdef65eaf4ad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cbaffd640 1 -- 172.21.15.76:0/1609368907 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8072040 con 0x561a00682b20 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cbaffd640 1 -- 172.21.15.76:0/1609368907 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc805bbe0 con 0x561a00682b20 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cbaffd640 1 -- 172.21.15.76:0/1609368907 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc805edf0 con 0x561a00682b20 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- 172.21.15.76:0/1609368907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00682b20 msgr2=0x561a00682170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1609368907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00682b20 0x561a00682170 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f1cc800e540 tx=0x7f1cc8081000 comp rx=0 tx=0).stop 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- 172.21.15.76:0/1609368907 shutdown_connections 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1609368907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00683440 0x561a00693670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1609368907 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a00682b20 0x561a00682170 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1609368907 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a0063e330 0x561a00681c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- 172.21.15.76:0/1609368907 >> 172.21.15.76:0/1609368907 conn(0x561a00639290 msgr2=0x561a0063d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- 172.21.15.76:0/1609368907 shutdown_connections 2024-09-19T04:18:14.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- 172.21.15.76:0/1609368907 wait complete. 2024-09-19T04:18:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 -- start start 2024-09-19T04:18:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.213+0000 7f1cdf147000 1 Processor -- start 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a0063e330 0x561a006963b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a006968f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00683440 0x561a006b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561a00486f80 con 0x561a0063e330 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561a0043fca0 con 0x561a00682b20 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561a0048c2a0 con 0x561a00683440 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 10 client.0 _ll_register_callbacks cb 0x7f1cc8056520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a0063e330 0x561a006963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a0063e330 0x561a006963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47040/0 (socket says 172.21.15.76:47040) 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 -- 172.21.15.76:0/1782543771 learned_addr learned my addr 172.21.15.76:0/1782543771 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a006968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cda628640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00683440 0x561a006b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 -- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00683440 msgr2=0x561a006b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00683440 0x561a006b1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 -- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 msgr2=0x561a006968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a006968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 -- 172.21.15.76:0/1782543771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a0050a750 con 0x561a0063e330 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a006968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9e27640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a0063e330 0x561a006963b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x561a00489e40 tx=0x7f1ccc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c389700fda27e12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/1782543771 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc017be0 con 0x561a0063e330 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/1782543771 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f1ccc017d80 con 0x561a0063e330 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cc805bbe0 con 0x561a0063e330 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/1782543771 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f1ccc035050 con 0x561a0063e330 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 10 client.6990 fetch_fsmap learned FSMap version 72 2024-09-19T04:18:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x561a00415a30 con 0x561a0063e330 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/1782543771 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cc805bbe0 con 0x561a0063e330 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/1782543771 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 72) ==== 3542+0+0 (secure 0 0 0) 0x7f1ccc038020 con 0x561a0063e330 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 10 client.6990 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 10 client.6990 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 20 client.6990 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a006f5c00 0x561a006f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 4 client.6990 mds_command: new command op to 5336 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 --> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] -- command(tid 0: {"prefix": "status"}) -- 0x561a004953a0 con 0x561a006f5c00 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a006f5c00 0x561a006f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 10 client.6990 ms_handle_connect on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cd9626640 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a006f5c00 0x561a006f5fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=119ef0ef6d3c8775 server_cookie=39830746ac9a2de in_seq=0 out_seq=0 2024-09-19T04:18:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 1 -- 172.21.15.76:0/1782543771 <== mds.2 v2:172.21.15.116:6834/3224374251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x561a004953a0 con 0x561a006f5c00 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 10 client.6990 handle_command_reply: tid=0 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 client.6990 shutdown 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a0063e330 msgr2=0x561a006963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a0063e330 0x561a006963b0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x561a00489e40 tx=0x7f1ccc009750 comp rx=0 tx=0).stop 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 shutdown_connections 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x561a006f5c00 0x561a006f5fe0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561a00683440 0x561a006b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561a00682b20 0x561a006968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 --2- 172.21.15.76:0/1782543771 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561a0063e330 0x561a006963b0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cb8ff9640 0 client.6990 ms_handle_reset on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 >> 172.21.15.76:0/1782543771 conn(0x561a00639290 msgr2=0x561a00680c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 shutdown_connections 2024-09-19T04:18:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 1 -- 172.21.15.76:0/1782543771 wait complete. 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.217+0000 7f1cdf147000 20 client.6990 trim_cache size 0 max 16384 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5336, 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:14.226 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 1.7187224059999999, 2024-09-19T04:18:14.227 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 72, 2024-09-19T04:18:14.227 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 80, 2024-09-19T04:18:14.227 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T04:18:14.227 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 16.806784453999999 2024-09-19T04:18:14.227 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1cc00678c0 msgr2=0x7f1cc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1cc00678c0 0x7f1cc0069d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0074a0 tx=0x7f1ccc00b040 comp rx=0 tx=0).stop 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cd415cc20 msgr2=0x7f1cd415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cd415cc20 0x7f1cd415d0a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1cc8010890 tx=0x7f1cc8010d50 comp rx=0 tx=0).stop 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 shutdown_connections 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1cc00678c0 0x7f1cc0069d80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1cd415e580 0x7f1cd415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1cd415cc20 0x7f1cd415d0a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 --2- 172.21.15.76:0/703696492 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1cd40cec80 0x7f1cd415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 >> 172.21.15.76:0/703696492 conn(0x7f1cd40b1f90 msgr2=0x7f1cd40b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 shutdown_connections 2024-09-19T04:18:14.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.221+0000 7f1cdc08b640 1 -- 172.21.15.76:0/703696492 wait complete. 2024-09-19T04:18:14.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:18:14.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc2f5c640 1 Processor -- start 2024-09-19T04:18:14.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc2f5c640 1 -- start start 2024-09-19T04:18:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc2f5c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fbc161a60 0x7f6fbc0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc2f5c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6fbc0cf620 0x7f6fbc06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc2f5c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fbc0d32b0 con 0x7f6fbc0d2e90 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc2f5c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6fbc0d0300 con 0x7f6fbc161a60 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc2f5c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6fbc0cfa90 con 0x7f6fbc0cf620 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc0cf8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fbc161a60 0x7f6fbc0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc0cf8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6fbc161a60 0x7f6fbc0d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47050/0 (socket says 172.21.15.76:47050) 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/1719905930 learned_addr learned my addr 172.21.15.76:0/1719905930 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/1719905930 >> v2:172.21.15.116:3300/0 conn(0x7f6fbc0cf620 0x7f6fbc06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/1719905930 >> v2:172.21.15.116:3300/0 conn(0x7f6fbc0cf620 msgr2=0x7f6fbc06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/1719905930 >> v2:172.21.15.116:3300/0 conn(0x7f6fbc0cf620 0x7f6fbc06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/1719905930 >> v1:172.21.15.116:6789/0 conn(0x7f6fbc0d2e90 legacy=0x7f6fbc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/1719905930 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc0b0f60 con 0x7f6fbc161a60 2024-09-19T04:18:14.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.349+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/1719905930 >> v2:172.21.15.116:3300/0 conn(0x7f6fbc0cf620 0x7f6fbc06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/1719905930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc161a60 0x7f6fbc0d2950 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0ce950 tx=0x7f6fac01c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bd205bc83d429 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fbb7fe640 1 -- 172.21.15.76:0/1719905930 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0a57b0 con 0x7f6fbc161a60 2024-09-19T04:18:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fbb7fe640 1 -- 172.21.15.76:0/1719905930 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac0a5950 con 0x7f6fbc161a60 2024-09-19T04:18:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fbb7fe640 1 -- 172.21.15.76:0/1719905930 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0a5d20 con 0x7f6fbc161a60 2024-09-19T04:18:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/1719905930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc161a60 msgr2=0x7f6fbc0d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/1719905930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc161a60 0x7f6fbc0d2950 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0ce950 tx=0x7f6fac01c8f0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/1719905930 shutdown_connections 2024-09-19T04:18:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/1719905930 >> v2:172.21.15.116:3300/0 conn(0x7f6fbc0cf620 0x7f6fbc06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/1719905930 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc161a60 0x7f6fbc0d2950 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/1719905930 >> 172.21.15.76:0/1719905930 conn(0x7f6fbc08bf90 msgr2=0x7f6fbc09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/1719905930 shutdown_connections 2024-09-19T04:18:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/1719905930 wait complete. 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 Processor -- start 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- start start 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc0cf620 0x7f6fbc0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fbc0d2e90 0x7f6fbc152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fbc161a60 0x7f6fbc152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc06e400 con 0x7f6fbc0cf620 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc06d190 con 0x7f6fbc0d2e90 2024-09-19T04:18:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc2f5c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc06e580 con 0x7f6fbc161a60 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc0cf620 0x7f6fbc0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc0cf620 0x7f6fbc0b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47062/0 (socket says 172.21.15.76:47062) 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/159918997 learned_addr learned my addr 172.21.15.76:0/159918997 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fbc0d2e90 0x7f6fbc152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fbc161a60 0x7f6fbc152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fbc161a60 msgr2=0x7f6fbc152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fbc161a60 0x7f6fbc152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fbc0d2e90 msgr2=0x7f6fbc152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fbc0d2e90 0x7f6fbc152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/159918997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc0c7c90 con 0x7f6fbc0cf620 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fbc0d2e90 0x7f6fbc152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc0cf620 0x7f6fbc0b4420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0af040 tx=0x7f6fac019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fb97fa640 1 -- 172.21.15.76:0/159918997 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0bc020 con 0x7f6fbc0cf620 2024-09-19T04:18:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.353+0000 7f6fb97fa640 1 -- 172.21.15.76:0/159918997 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fac0b88a0 con 0x7f6fbc0cf620 2024-09-19T04:18:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc15be90 con 0x7f6fbc0cf620 2024-09-19T04:18:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fb97fa640 1 -- 172.21.15.76:0/159918997 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0b8c90 con 0x7f6fbc0cf620 2024-09-19T04:18:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc15c450 con 0x7f6fbc0cf620 2024-09-19T04:18:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fb97fa640 1 -- 172.21.15.76:0/159918997 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6fac0bf020 con 0x7f6fbc0cf620 2024-09-19T04:18:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fb97fa640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6fa80678c0 0x7f6fa8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fb97fa640 1 -- 172.21.15.76:0/159918997 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6fac1210a0 con 0x7f6fbc0cf620 2024-09-19T04:18:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6fa80678c0 0x7f6fa8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6fa80678c0 0x7f6fa8069d80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0b4ec0 tx=0x7f6fb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 Processor -- start 2024-09-19T04:18:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 -- start start 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c435f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3ebd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3ec290 0x56239c3ec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56239c2d3b60 con 0x56239c3ec290 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56239c23ac70 con 0x56239c436440 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56239c1e0ed0 con 0x56239c435b20 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.357+0000 7f6fc14f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3ec290 0x56239c3ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3ec290 0x56239c3ec670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47078/0 (socket says 172.21.15.76:47078) 2024-09-19T04:18:14.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/2104562828 learned_addr learned my addr 172.21.15.76:0/2104562828 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c435f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3ebd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 msgr2=0x56239c435f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c435f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 msgr2=0x56239c3ebd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3ebd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/2104562828 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239c1f2e50 con 0x56239c3ec290 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3ec290 0x56239c3ec670 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400b8d0 tx=0x7f6fb400bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53da1d03d9e6cbc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fa27fc640 1 -- 172.21.15.76:0/2104562828 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4016070 con 0x56239c3ec290 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fa27fc640 1 -- 172.21.15.76:0/2104562828 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb400dbb0 con 0x56239c3ec290 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fa27fc640 1 -- 172.21.15.76:0/2104562828 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4012460 con 0x56239c3ec290 2024-09-19T04:18:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- 172.21.15.76:0/2104562828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3ec290 msgr2=0x56239c3ec670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3ec290 0x56239c3ec670 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400b8d0 tx=0x7f6fb400bd90 comp rx=0 tx=0).stop 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- 172.21.15.76:0/2104562828 shutdown_connections 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3ec290 0x56239c3ec670 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3ebd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 --2- 172.21.15.76:0/2104562828 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c435f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- 172.21.15.76:0/2104562828 >> 172.21.15.76:0/2104562828 conn(0x56239c3f2600 msgr2=0x56239c3ea630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- 172.21.15.76:0/2104562828 shutdown_connections 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- 172.21.15.76:0/2104562828 wait complete. 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- start start 2024-09-19T04:18:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 Processor -- start 2024-09-19T04:18:14.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c3f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3f6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c3f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56239c29cb00 con 0x56239c3f1330 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56239c215360 con 0x56239c436440 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56239c2122d0 con 0x56239c435b20 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 10 client.0 _ll_register_callbacks cb 0x56239c3d7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c3f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c3f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47090/0 (socket says 172.21.15.76:47090) 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/3047218640 learned_addr learned my addr 172.21.15.76:0/3047218640 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c3f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3f6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 msgr2=0x56239c3f0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c3f0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 msgr2=0x56239c3f6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3f6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 -- 172.21.15.76:0/3047218640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239c1e1120 con 0x56239c3f1330 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c3f7230 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400bdd0 tx=0x7f6fb4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679dfc924beb6abb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6f9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3047218640 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb4016040 con 0x56239c3f1330 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3047218640 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6fb400dd60 con 0x56239c3f1330 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb400dbb0 con 0x56239c3f1330 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3047218640 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f6fb4030050 con 0x56239c3f1330 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 10 client.7002 fetch_fsmap learned FSMap version 72 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x56239c1da450 con 0x56239c3f1330 2024-09-19T04:18:14.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.361+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3047218640 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb400dbb0 con 0x56239c3f1330 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3047218640 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 72) ==== 3542+0+0 (secure 0 0 0) 0x7f6fb4033020 con 0x56239c3f1330 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 10 client.7002 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 10 client.7002 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 20 client.7002 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c444280 0x56239c446670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 4 client.7002 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56239c1d4c00 con 0x56239c444280 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c444280 0x56239c446670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6f9b7fe640 10 client.7002 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c444280 0x56239c446670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd5fec9bd3d10889 server_cookie=7b95d9c2e7c18261 in_seq=0 out_seq=0 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3047218640 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56239c1d4c00 con 0x56239c444280 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6f9b7fe640 10 client.7002 handle_command_reply: tid=0 2024-09-19T04:18:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 client.7002 shutdown 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 msgr2=0x56239c3f7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c3f7230 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f6fb400bdd0 tx=0x7f6fb4004740 comp rx=0 tx=0).stop 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 shutdown_connections 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c444280 0x56239c446670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c3f7230 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c436440 0x56239c3f6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6f9b7fe640 0 client.7002 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3047218640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c435b20 0x56239c3f0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 >> 172.21.15.76:0/3047218640 conn(0x56239c3ec290 msgr2=0x56239c3ead40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 shutdown_connections 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 1 -- 172.21.15.76:0/3047218640 wait complete. 2024-09-19T04:18:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.365+0000 7f6fc6018000 20 client.7002 trim_cache size 0 max 16384 2024-09-19T04:18:14.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.381+0000 7f6fc6018000 1 Processor -- start 2024-09-19T04:18:14.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.381+0000 7f6fc6018000 1 -- start start 2024-09-19T04:18:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c3f1330 0x56239c434cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c435b20 0x56239c435230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c446670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc202fa0 con 0x56239c435b20 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc203670 con 0x56239c3f1330 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc0b9190 con 0x56239c436440 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c435b20 0x56239c435230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c435b20 0x56239c435230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47100/0 (socket says 172.21.15.76:47100) 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 -- 172.21.15.76:0/1920487661 learned_addr learned my addr 172.21.15.76:0/1920487661 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c446670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c3f1330 0x56239c434cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 -- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 msgr2=0x56239c446670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c446670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 -- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c3f1330 msgr2=0x56239c434cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c3f1330 0x56239c434cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.387 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 -- 172.21.15.76:0/1920487661 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56239c436270 con 0x56239c435b20 2024-09-19T04:18:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc14f9640 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c446670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c3f1330 0x56239c434cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c435b20 0x56239c435230 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000caf0 tx=0x7f6fb0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37e2924d18846f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fa1ffb640 1 -- 172.21.15.76:0/1920487661 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb001f480 con 0x56239c435b20 2024-09-19T04:18:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fa1ffb640 1 -- 172.21.15.76:0/1920487661 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb0024070 con 0x56239c435b20 2024-09-19T04:18:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fa1ffb640 1 -- 172.21.15.76:0/1920487661 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb001f840 con 0x56239c435b20 2024-09-19T04:18:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- 172.21.15.76:0/1920487661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c435b20 msgr2=0x56239c435230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c435b20 0x56239c435230 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000caf0 tx=0x7f6fb0012000 comp rx=0 tx=0).stop 2024-09-19T04:18:14.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- 172.21.15.76:0/1920487661 shutdown_connections 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c446670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c435b20 0x56239c435230 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- 172.21.15.76:0/1920487661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c3f1330 0x56239c434cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- 172.21.15.76:0/1920487661 >> 172.21.15.76:0/1920487661 conn(0x56239c3ec290 msgr2=0x56239c3f05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- 172.21.15.76:0/1920487661 shutdown_connections 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- 172.21.15.76:0/1920487661 wait complete. 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- start start 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 Processor -- start 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c449b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c435b20 0x56239c464960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c464ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56239c239f80 con 0x56239c3f1330 2024-09-19T04:18:14.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56239c1f2ca0 con 0x56239c435b20 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56239c23f2a0 con 0x56239c436440 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc6018000 10 client.0 _ll_register_callbacks cb 0x7f6fb400b720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc0cf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c449b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc0cf8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c449b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47112/0 (socket says 172.21.15.76:47112) 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.385+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/3280089973 learned_addr learned my addr 172.21.15.76:0/3280089973 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c435b20 0x56239c464960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/3280089973 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 msgr2=0x56239c464ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c464ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/3280089973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c435b20 msgr2=0x56239c464960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c435b20 0x56239c464960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc0cf8640 1 -- 172.21.15.76:0/3280089973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56239c1c1fd0 con 0x56239c3f1330 2024-09-19T04:18:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc0cf8640 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c449b40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0f1170 tx=0x7f6fac11f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a83ceaedd14aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3280089973 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0fa050 con 0x56239c3f1330 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3280089973 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6fac0ef770 con 0x56239c3f1330 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fb0024070 con 0x56239c3f1330 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3280089973 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f6fac11d050 con 0x56239c3f1330 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 10 client.7008 fetch_fsmap learned FSMap version 72 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x56239c2bd750 con 0x56239c3f1330 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3280089973 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fb0024070 con 0x56239c3f1330 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3280089973 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 72) ==== 3542+0+0 (secure 0 0 0) 0x7f6fac0edae0 con 0x56239c3f1330 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 10 client.7008 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 10 client.7008 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 20 client.7008 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c4653e0 0x56239c4a8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 4 client.7008 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "status"}) -- 0x56239c2483a0 con 0x56239c4653e0 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c4653e0 0x56239c4a8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 10 client.7008 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fbbfff640 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c4653e0 0x56239c4a8f30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3114d357d82bd515 server_cookie=be9f3b9915899798 in_seq=0 out_seq=0 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 1 -- 172.21.15.76:0/3280089973 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56239c2483a0 con 0x56239c4653e0 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 10 client.7008 handle_command_reply: tid=0 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 client.7008 shutdown 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 msgr2=0x56239c449b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c449b40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0f1170 tx=0x7f6fac11f310 comp rx=0 tx=0).stop 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 shutdown_connections 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56239c436440 0x56239c464ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56239c435b20 0x56239c464960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x56239c4653e0 0x56239c4a8f30 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 --2- 172.21.15.76:0/3280089973 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56239c3f1330 0x56239c449b40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 >> 172.21.15.76:0/3280089973 conn(0x56239c3ec290 msgr2=0x56239c433ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6f9b7fe640 0 client.7008 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 shutdown_connections 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 1 -- 172.21.15.76:0/3280089973 wait complete. 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc6018000 20 client.7008 trim_cache size 0 max 16384 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:14.397 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6780, 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 2.8976777070000002, 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 72, 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 80, 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 16.994212255000001 2024-09-19T04:18:14.398 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6fa80678c0 msgr2=0x7f6fa8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6fa80678c0 0x7f6fa8069d80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0b4ec0 tx=0x7f6fb0008040 comp rx=0 tx=0).stop 2024-09-19T04:18:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc0cf620 msgr2=0x7f6fbc0b4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc0cf620 0x7f6fbc0b4420 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f6fac0af040 tx=0x7f6fac019690 comp rx=0 tx=0).stop 2024-09-19T04:18:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 shutdown_connections 2024-09-19T04:18:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6fa80678c0 0x7f6fa8069d80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6fbc161a60 0x7f6fbc152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fbc0d2e90 0x7f6fbc152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 --2- 172.21.15.76:0/159918997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6fbc0cf620 0x7f6fbc0b4420 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 >> 172.21.15.76:0/159918997 conn(0x7f6fbc08bf90 msgr2=0x7f6fbc0b1990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 shutdown_connections 2024-09-19T04:18:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.389+0000 7f6fc2f5c640 1 -- 172.21.15.76:0/159918997 wait complete. 2024-09-19T04:18:14.407 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:18:14.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef6ac6640 1 Processor -- start 2024-09-19T04:18:14.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef6ac6640 1 -- start start 2024-09-19T04:18:14.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef6ac6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015c2a0 0x7f8ef00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef6ac6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ef00af2a0 0x7f8ef006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef6ac6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ef00b3290 con 0x7f8ef015bec0 2024-09-19T04:18:14.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef6ac6640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8ef00b3570 con 0x7f8ef00af2a0 2024-09-19T04:18:14.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef6ac6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef00b3850 con 0x7f8ef015c2a0 2024-09-19T04:18:14.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015c2a0 0x7f8ef00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef5063640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ef00af2a0 0x7f8ef006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef4862640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8ef015bec0 0x7f8ef00ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:35646/0 (socket says 172.21.15.76:35646) 2024-09-19T04:18:14.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8ef4862640 1 -- 172.21.15.76:0/2082506226 learned_addr learned my addr 172.21.15.76:0/2082506226 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:14.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8eef7fe640 1 -- 172.21.15.76:0/2082506226 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2531266228 0 0) 0x7f8ef00b3290 con 0x7f8ef015bec0 2024-09-19T04:18:14.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8eef7fe640 1 -- 172.21.15.76:0/2082506226 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8ed8003530 con 0x7f8ef015bec0 2024-09-19T04:18:14.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8eeffff640 1 -- 172.21.15.76:0/2082506226 >> v2:172.21.15.116:3300/0 conn(0x7f8ef00af2a0 msgr2=0x7f8ef006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2082506226 >> v2:172.21.15.116:3300/0 conn(0x7f8ef00af2a0 0x7f8ef006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8eeffff640 1 -- 172.21.15.76:0/2082506226 >> v1:172.21.15.116:6789/0 conn(0x7f8ef015bec0 legacy=0x7f8ef00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.517+0000 7f8eeffff640 1 -- 172.21.15.76:0/2082506226 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef00b3b30 con 0x7f8ef015c2a0 2024-09-19T04:18:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2082506226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015c2a0 0x7f8ef00aed60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009d90 tx=0x7f8ee401c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f019cd889ba60b60 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8eef7fe640 1 -- 172.21.15.76:0/2082506226 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40b2070 con 0x7f8ef015c2a0 2024-09-19T04:18:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8eef7fe640 1 -- 172.21.15.76:0/2082506226 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee40a5a00 con 0x7f8ef015c2a0 2024-09-19T04:18:14.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8eef7fe640 1 -- 172.21.15.76:0/2082506226 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40a5dd0 con 0x7f8ef015c2a0 2024-09-19T04:18:14.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2082506226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015c2a0 msgr2=0x7f8ef00aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2082506226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015c2a0 0x7f8ef00aed60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009d90 tx=0x7f8ee401c940 comp rx=0 tx=0).stop 2024-09-19T04:18:14.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2082506226 shutdown_connections 2024-09-19T04:18:14.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2082506226 >> v2:172.21.15.116:3300/0 conn(0x7f8ef00af2a0 0x7f8ef006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2082506226 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015c2a0 0x7f8ef00aed60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2082506226 >> 172.21.15.76:0/2082506226 conn(0x7f8ef00b1f90 msgr2=0x7f8ef00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2082506226 shutdown_connections 2024-09-19T04:18:14.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.521+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2082506226 wait complete. 2024-09-19T04:18:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 Processor -- start 2024-09-19T04:18:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 -- start start 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ef00af2a0 0x7f8ef0165350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015bec0 0x7f8ef009d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef015c2a0 0x7f8ef009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef006ed50 con 0x7f8ef015c2a0 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef00af6b0 con 0x7f8ef00af2a0 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef006eed0 con 0x7f8ef015bec0 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef015c2a0 0x7f8ef009d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef015c2a0 0x7f8ef009d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47120/0 (socket says 172.21.15.76:47120) 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 -- 172.21.15.76:0/2773299411 learned_addr learned my addr 172.21.15.76:0/2773299411 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ef00af2a0 0x7f8ef0165350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015bec0 0x7f8ef009d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 -- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015bec0 msgr2=0x7f8ef009d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015bec0 0x7f8ef009d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 -- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ef00af2a0 msgr2=0x7f8ef0165350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ef00af2a0 0x7f8ef0165350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 -- 172.21.15.76:0/2773299411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef015ed60 con 0x7f8ef015c2a0 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef5063640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef015c2a0 0x7f8ef009d7d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800e930 tx=0x7f8ee800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8eed7fa640 1 -- 172.21.15.76:0/2773299411 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8017070 con 0x7f8ef015c2a0 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8eed7fa640 1 -- 172.21.15.76:0/2773299411 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee800c9a0 con 0x7f8ef015c2a0 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef015f040 con 0x7f8ef015c2a0 2024-09-19T04:18:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8eed7fa640 1 -- 172.21.15.76:0/2773299411 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee800cd70 con 0x7f8ef015c2a0 2024-09-19T04:18:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef009ae00 con 0x7f8ef015c2a0 2024-09-19T04:18:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.525+0000 7f8eed7fa640 1 -- 172.21.15.76:0/2773299411 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ee801a070 con 0x7f8ef015c2a0 2024-09-19T04:18:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eed7fa640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8edc0678c0 0x7f8edc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eed7fa640 1 -- 172.21.15.76:0/2773299411 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f8ee8086280 con 0x7f8ef015c2a0 2024-09-19T04:18:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8edc0678c0 0x7f8edc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8edc0678c0 0x7f8edc069d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005890 tx=0x7f8ee0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 Processor -- start 2024-09-19T04:18:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- start start 2024-09-19T04:18:14.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 0x55d8bf2baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf270d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf271290 0x55d8bf271670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d8bf158b60 con 0x55d8bf2bb440 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d8bf0bfc70 con 0x55d8bf271290 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d8bf065ed0 con 0x55d8bf2bab20 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf270d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf270d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47130/0 (socket says 172.21.15.76:47130) 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 -- 172.21.15.76:0/4233724318 learned_addr learned my addr 172.21.15.76:0/4233724318 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef5063640 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf271290 0x55d8bf271670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef4862640 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 0x55d8bf2baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 -- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 msgr2=0x55d8bf2baf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 0x55d8bf2baf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 -- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf271290 msgr2=0x55d8bf271670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf271290 0x55d8bf271670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 -- 172.21.15.76:0/4233724318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bf077e50 con 0x55d8bf2bb440 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef5063640 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf271290 0x55d8bf271670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8eeffff640 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf270d50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8ee401a570 tx=0x7f8ee4004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18d8e7e7ad5f1c34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ed67fc640 1 -- 172.21.15.76:0/4233724318 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40b2040 con 0x55d8bf2bb440 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ed67fc640 1 -- 172.21.15.76:0/4233724318 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee40a5a00 con 0x55d8bf2bb440 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ed67fc640 1 -- 172.21.15.76:0/4233724318 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40a5dd0 con 0x55d8bf2bb440 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- 172.21.15.76:0/4233724318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 msgr2=0x55d8bf270d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf270d50 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f8ee401a570 tx=0x7f8ee4004ac0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- 172.21.15.76:0/4233724318 shutdown_connections 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf271290 0x55d8bf271670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf270d50 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/4233724318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 0x55d8bf2baf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- 172.21.15.76:0/4233724318 >> 172.21.15.76:0/4233724318 conn(0x55d8bf277600 msgr2=0x55d8bf26f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- 172.21.15.76:0/4233724318 shutdown_connections 2024-09-19T04:18:14.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- 172.21.15.76:0/4233724318 wait complete. 2024-09-19T04:18:14.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 -- start start 2024-09-19T04:18:14.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.529+0000 7f8ef9b82000 1 Processor -- start 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf275df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf27bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 0x55d8bf27c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d8bf121b00 con 0x55d8bf2bab20 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d8bf09a360 con 0x55d8bf2bb440 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d8bf0972d0 con 0x55d8bf276330 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 10 client.0 _ll_register_callbacks cb 0x55d8bf25c820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf275df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf275df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47146/0 (socket says 172.21.15.76:47146) 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 -- 172.21.15.76:0/2310002567 learned_addr learned my addr 172.21.15.76:0/2310002567 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf27bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef5063640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 0x55d8bf27c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 -- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 msgr2=0x55d8bf27c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 0x55d8bf27c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 -- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 msgr2=0x55d8bf27bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf27bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 -- 172.21.15.76:0/2310002567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bf066120 con 0x55d8bf2bab20 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf27bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf275df0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005f70 tx=0x7f8ee0000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe11c5fa097640a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2310002567 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0027060 con 0x55d8bf2bab20 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2310002567 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f8ee0010040 con 0x55d8bf2bab20 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee40a5a00 con 0x55d8bf2bab20 2024-09-19T04:18:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2310002567 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f8ee0007560 con 0x55d8bf2bab20 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 10 client.7017 fetch_fsmap learned FSMap version 72 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55d8bf05f450 con 0x55d8bf2bab20 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2310002567 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee40a5a00 con 0x55d8bf2bab20 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2310002567 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 72) ==== 3542+0+0 (secure 0 0 0) 0x7f8ee0022460 con 0x55d8bf2bab20 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 10 client.7017 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 10 client.7017 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 20 client.7017 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2c9280 0x55d8bf2cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 4 client.7017 mds_command: new command op to 6783 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 --> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8bf059c00 con 0x55d8bf2c9280 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2c9280 0x55d8bf2cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 10 client.7017 ms_handle_connect on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2c9280 0x55d8bf2cb670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5a69fa585df41fb server_cookie=33bfd3931eb819df in_seq=0 out_seq=0 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2310002567 <== mds.0 v2:172.21.15.76:6836/3569622848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d8bf059c00 con 0x55d8bf2c9280 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 10 client.7017 handle_command_reply: tid=0 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 client.7017 shutdown 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 msgr2=0x55d8bf275df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf275df0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005f70 tx=0x7f8ee0000ee0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 shutdown_connections 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 0x55d8bf27c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf27bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2c9280 0x55d8bf2cb670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2310002567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf275df0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 >> 172.21.15.76:0/2310002567 conn(0x55d8bf271290 msgr2=0x55d8bf26fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ecf7fe640 0 client.7017 ms_handle_reset on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 shutdown_connections 2024-09-19T04:18:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2310002567 wait complete. 2024-09-19T04:18:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.533+0000 7f8ef9b82000 20 client.7017 trim_cache size 0 max 16384 2024-09-19T04:18:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 Processor -- start 2024-09-19T04:18:14.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- start start 2024-09-19T04:18:14.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf276330 0x55d8bf2b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 0x55d8bf2ba1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf2cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0200c20 con 0x55d8bf2bb440 2024-09-19T04:18:14.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef0201270 con 0x55d8bf276330 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef0155410 con 0x55d8bf2bab20 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf2cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf2cb670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47152/0 (socket says 172.21.15.76:47152) 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 -- 172.21.15.76:0/3002564798 learned_addr learned my addr 172.21.15.76:0/3002564798 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 -- 172.21.15.76:0/3002564798 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 msgr2=0x55d8bf2ba1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 --2- 172.21.15.76:0/3002564798 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 0x55d8bf2ba1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 -- 172.21.15.76:0/3002564798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf276330 msgr2=0x55d8bf2b9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 --2- 172.21.15.76:0/3002564798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf276330 0x55d8bf2b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 -- 172.21.15.76:0/3002564798 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8bf2bb270 con 0x55d8bf2bb440 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef5063640 1 --2- 172.21.15.76:0/3002564798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf2cb670 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8ee805ec20 tx=0x7f8ee80861c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755dd2ee7010c513 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ed5ffb640 1 -- 172.21.15.76:0/3002564798 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8021030 con 0x55d8bf2bb440 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ed5ffb640 1 -- 172.21.15.76:0/3002564798 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee805f150 con 0x55d8bf2bb440 2024-09-19T04:18:14.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ed5ffb640 1 -- 172.21.15.76:0/3002564798 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8054870 con 0x55d8bf2bb440 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- 172.21.15.76:0/3002564798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 msgr2=0x55d8bf2cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/3002564798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf2cb670 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8ee805ec20 tx=0x7f8ee80861c0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- 172.21.15.76:0/3002564798 shutdown_connections 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/3002564798 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bb440 0x55d8bf2cb670 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/3002564798 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf2bab20 0x55d8bf2ba1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/3002564798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf276330 0x55d8bf2b9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- 172.21.15.76:0/3002564798 >> 172.21.15.76:0/3002564798 conn(0x55d8bf271290 msgr2=0x55d8bf2755f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- 172.21.15.76:0/3002564798 shutdown_connections 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.553+0000 7f8ef9b82000 1 -- 172.21.15.76:0/3002564798 wait complete. 2024-09-19T04:18:14.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- start start 2024-09-19T04:18:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 Processor -- start 2024-09-19T04:18:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 0x55d8bf2ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf2e9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf2e9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d8bf0bef80 con 0x55d8bf2bab20 2024-09-19T04:18:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d8bf077ca0 con 0x55d8bf2bb440 2024-09-19T04:18:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d8bf0c42a0 con 0x55d8bf276330 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 10 client.0 _ll_register_callbacks cb 0x7f8ee0001440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf2e9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf2e9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47154/0 (socket says 172.21.15.76:47154) 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 -- 172.21.15.76:0/2250995887 learned_addr learned my addr 172.21.15.76:0/2250995887 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 -- 172.21.15.76:0/2250995887 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 msgr2=0x55d8bf2ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 0x55d8bf2ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 -- 172.21.15.76:0/2250995887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 msgr2=0x55d8bf2e9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf2e9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 -- 172.21.15.76:0/2250995887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8bf046fd0 con 0x55d8bf2bab20 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8eeffff640 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf2e9960 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8ee401a570 tx=0x7f8ee40196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ead219ad667a1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2250995887 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee40b2050 con 0x55d8bf2bab20 2024-09-19T04:18:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2250995887 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f8ee40a5a00 con 0x55d8bf2bab20 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ee805f150 con 0x55d8bf2bab20 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2250995887 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7f8ee40039a0 con 0x55d8bf2bab20 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 10 client.7023 fetch_fsmap learned FSMap version 72 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=2+}) -- 0x55d8bf142750 con 0x55d8bf2bab20 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2250995887 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ee805f150 con 0x55d8bf2bab20 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2250995887 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 72) ==== 3542+0+0 (secure 0 0 0) 0x7f8ee4003b40 con 0x55d8bf2bab20 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 10 client.7023 fetch_fsmap finished waiting for FSMap version 72 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 10 client.7023 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 20 client.7023 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2ea3e0 0x55d8bf32e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 4 client.7023 mds_command: new command op to 6783 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 --> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] -- command(tid 0: {"prefix": "status"}) -- 0x55d8bf0cd3a0 con 0x55d8bf2ea3e0 2024-09-19T04:18:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2ea3e0 0x55d8bf32e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 10 client.7023 ms_handle_connect on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef4862640 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2ea3e0 0x55d8bf32e2b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9812befc8d28d205 server_cookie=65a7e9424ee7e398 in_seq=0 out_seq=0 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 1 -- 172.21.15.76:0/2250995887 <== mds.0 v2:172.21.15.76:6836/3569622848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d8bf0cd3a0 con 0x55d8bf2ea3e0 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 10 client.7023 handle_command_reply: tid=0 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 client.7023 shutdown 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 msgr2=0x55d8bf2e9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf2e9960 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8ee401a570 tx=0x7f8ee40196e0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 shutdown_connections 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d8bf2bb440 0x55d8bf2e9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x55d8bf2ea3e0 0x55d8bf32e2b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d8bf2bab20 0x55d8bf2e9960 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ecf7fe640 0 client.7023 ms_handle_reset on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 --2- 172.21.15.76:0/2250995887 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d8bf276330 0x55d8bf2ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 >> 172.21.15.76:0/2250995887 conn(0x55d8bf271290 msgr2=0x55d8bf2b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 shutdown_connections 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 1 -- 172.21.15.76:0/2250995887 wait complete. 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef9b82000 20 client.7023 trim_cache size 0 max 16384 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:18:14.568 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6783, 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 12.123332316999999, 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 72, 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 80, 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 80, 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 17.159887442999999 2024-09-19T04:18:14.569 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8edc0678c0 msgr2=0x7f8edc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8edc0678c0 0x7f8edc069d80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8ee0005890 tx=0x7f8ee0009040 comp rx=0 tx=0).stop 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef015c2a0 msgr2=0x7f8ef009d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef015c2a0 0x7f8ef009d7d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8ee800e930 tx=0x7f8ee800edf0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 shutdown_connections 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8edc0678c0 0x7f8edc069d80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ef015c2a0 0x7f8ef009d7d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ef015bec0 0x7f8ef009d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 --2- 172.21.15.76:0/2773299411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ef00af2a0 0x7f8ef0165350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.557+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 >> 172.21.15.76:0/2773299411 conn(0x7f8ef00b1f90 msgr2=0x7f8ef01597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.561+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 shutdown_connections 2024-09-19T04:18:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.561+0000 7f8ef6ac6640 1 -- 172.21.15.76:0/2773299411 wait complete. 2024-09-19T04:18:14.576 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.685+0000 7eff77758640 1 Processor -- start 2024-09-19T04:18:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.685+0000 7eff77758640 1 -- start start 2024-09-19T04:18:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff77758640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff7015bec0 0x7eff700ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff77758640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff7015c2a0 0x7eff700aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff77758640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff700b3290 con 0x7eff700af2a0 2024-09-19T04:18:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff77758640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7eff700b3570 con 0x7eff7015c2a0 2024-09-19T04:18:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff77758640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7eff700b3850 con 0x7eff7015bec0 2024-09-19T04:18:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff74cf3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7eff7015c2a0 0x7eff700aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff75cf5640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7eff700af2a0 0x7eff7015fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55164/0 (socket says 172.21.15.76:55164) 2024-09-19T04:18:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff75cf5640 1 -- 172.21.15.76:0/1174522816 learned_addr learned my addr 172.21.15.76:0/1174522816 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff754f4640 1 --2- 172.21.15.76:0/1174522816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff7015bec0 0x7eff700ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff74cf3640 1 -- 172.21.15.76:0/1174522816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff7015bec0 msgr2=0x7eff700ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff67fff640 1 -- 172.21.15.76:0/1174522816 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2935535586 0 0) 0x7eff700b3290 con 0x7eff700af2a0 2024-09-19T04:18:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff74cf3640 1 --2- 172.21.15.76:0/1174522816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff7015bec0 0x7eff700ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff74cf3640 1 -- 172.21.15.76:0/1174522816 >> v1:172.21.15.76:6789/0 conn(0x7eff700af2a0 legacy=0x7eff7015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff74cf3640 1 -- 172.21.15.76:0/1174522816 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff700b3b30 con 0x7eff7015c2a0 2024-09-19T04:18:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff74cf3640 1 --2- 172.21.15.76:0/1174522816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff700aed60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7eff600097c0 tx=0x7eff6001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40748abe44e1da3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff67fff640 1 -- 172.21.15.76:0/1174522816 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600a7070 con 0x7eff7015c2a0 2024-09-19T04:18:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff67fff640 1 -- 172.21.15.76:0/1174522816 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff600b0070 con 0x7eff7015c2a0 2024-09-19T04:18:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff67fff640 1 -- 172.21.15.76:0/1174522816 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff600ab890 con 0x7eff7015c2a0 2024-09-19T04:18:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff77758640 1 -- 172.21.15.76:0/1174522816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 msgr2=0x7eff700aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.689+0000 7eff77758640 1 --2- 172.21.15.76:0/1174522816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff700aed60 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7eff600097c0 tx=0x7eff6001c470 comp rx=0 tx=0).stop 2024-09-19T04:18:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- 172.21.15.76:0/1174522816 shutdown_connections 2024-09-19T04:18:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 --2- 172.21.15.76:0/1174522816 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff700aed60 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 --2- 172.21.15.76:0/1174522816 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff7015bec0 0x7eff700ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- 172.21.15.76:0/1174522816 >> 172.21.15.76:0/1174522816 conn(0x7eff700b1f90 msgr2=0x7eff700b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- 172.21.15.76:0/1174522816 shutdown_connections 2024-09-19T04:18:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- 172.21.15.76:0/1174522816 wait complete. 2024-09-19T04:18:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 Processor -- start 2024-09-19T04:18:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- start start 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff700af2a0 0x7eff7015cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff7015bec0 0x7eff7015f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff7014edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7eff700af6f0 con 0x7eff7015c2a0 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7eff700ad740 con 0x7eff7015bec0 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7eff70169000 con 0x7eff700af2a0 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff7014edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff7014edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:47176/0 (socket says 172.21.15.76:47176) 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 -- 172.21.15.76:0/639707640 learned_addr learned my addr 172.21.15.76:0/639707640 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff74cf3640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff7015bec0 0x7eff7015f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 -- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff700af2a0 msgr2=0x7eff7015cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff754f4640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff700af2a0 0x7eff7015cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff700af2a0 0x7eff7015cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 -- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff7015bec0 msgr2=0x7eff7015f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff7015bec0 0x7eff7015f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 -- 172.21.15.76:0/639707640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff7014e060 con 0x7eff7015c2a0 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff75cf5640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff7014edb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c810 tx=0x7eff6800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff65ffb640 1 -- 172.21.15.76:0/639707640 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff68018070 con 0x7eff7015c2a0 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff65ffb640 1 -- 172.21.15.76:0/639707640 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff6800a640 con 0x7eff7015c2a0 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff7014f2f0 con 0x7eff7015c2a0 2024-09-19T04:18:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff65ffb640 1 -- 172.21.15.76:0/639707640 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff6800aa30 con 0x7eff7015c2a0 2024-09-19T04:18:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff7014e5b0 con 0x7eff7015c2a0 2024-09-19T04:18:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.693+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff700ad200 con 0x7eff7015c2a0 2024-09-19T04:18:14.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.697+0000 7eff65ffb640 1 -- 172.21.15.76:0/639707640 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7eff6801b070 con 0x7eff7015c2a0 2024-09-19T04:18:14.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.697+0000 7eff65ffb640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7eff540678c0 0x7eff54069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:14.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.697+0000 7eff65ffb640 1 -- 172.21.15.76:0/639707640 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7eff680853b0 con 0x7eff7015c2a0 2024-09-19T04:18:14.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.697+0000 7eff754f4640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7eff540678c0 0x7eff54069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:14.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.697+0000 7eff754f4640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7eff540678c0 0x7eff54069d80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff701f83d0 tx=0x7eff6c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:14.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.701+0000 7eff65ffb640 1 -- 172.21.15.76:0/639707640 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7eff68055df0 con 0x7eff7015c2a0 2024-09-19T04:18:14.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.885+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff7006fcd0 con 0x7eff7015c2a0 2024-09-19T04:18:14.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.885+0000 7eff65ffb640 1 -- 172.21.15.76:0/639707640 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5263 (secure 0 0 0) 0x7eff680556f0 con 0x7eff7015c2a0 2024-09-19T04:18:14.888 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:14.888 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":72,"btime":"2024-09-19T04:18:13:500921+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":72,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:13.500909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6783,"mds_1":6780,"mds_2":5336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5336":{"gid":5336,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[5336,6780,6783]},"id":5}]} 2024-09-19T04:18:14.888 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 72 2024-09-19T04:18:14.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7eff540678c0 msgr2=0x7eff54069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7eff540678c0 0x7eff54069d80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff701f83d0 tx=0x7eff6c008040 comp rx=0 tx=0).stop 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 msgr2=0x7eff7014edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff7014edb0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7eff6800c810 tx=0x7eff6800ccd0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 shutdown_connections 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7eff540678c0 0x7eff54069d80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7eff7015c2a0 0x7eff7014edb0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff7015bec0 0x7eff7015f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 --2- 172.21.15.76:0/639707640 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7eff700af2a0 0x7eff7015cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 >> 172.21.15.76:0/639707640 conn(0x7eff700b1f90 msgr2=0x7eff70159ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 shutdown_connections 2024-09-19T04:18:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:14.889+0000 7eff77758640 1 -- 172.21.15.76:0/639707640 wait complete. 2024-09-19T04:18:14.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:14.907 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:18:14.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:14.932 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:18:14.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:14.989 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-19T04:18:15.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:15.040 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-19T04:18:15.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:15.088 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-19T04:18:15.137 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:18:15.318 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:15.319 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:15.319 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:18:17.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:18:18.103 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:18.103 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:18.103 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:18:18.383 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:18:18.381+0000 7ff5fba83640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:18:20.104 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:18:20.249 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:20.250 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:20.250 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:18:22.252 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:18:22.398 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:22.399 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:22.399 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:18:23.857 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:18:23.860 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:18:24.401 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:18:24.549 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:24.550 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:24.550 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:18:24.550 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:18:24.688 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:24.688 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:24.689 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:18:26.690 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:18:26.836 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:26.837 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:26.837 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:18:28.839 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:18:28.987 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:28.987 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:28.987 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:18:28.988 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-19T04:18:29.134 INFO:teuthology.orchestra.run.smithi116.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:18:29.134 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:29.135 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:18:29.135 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 flush journal 2024-09-19T04:18:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.261+0000 7ffb30af7640 1 Processor -- start 2024-09-19T04:18:29.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.261+0000 7ffb30af7640 1 -- start start 2024-09-19T04:18:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb2c0d3060 0x7ffb2c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c0cf450 0x7ffb2c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb2c0d0300 con 0x7ffb2c161a60 2024-09-19T04:18:29.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ffb2c0b0f60 con 0x7ffb2c0d3060 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2c0cfd40 con 0x7ffb2c0cf450 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb29d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb2c0d3060 0x7ffb2c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2a59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffb2c161a60 0x7ffb2c0d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37376/0 (socket says 172.21.15.76:37376) 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2a59c640 1 -- 172.21.15.76:0/3121231558 learned_addr learned my addr 172.21.15.76:0/3121231558 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb29d9b640 1 -- 172.21.15.76:0/3121231558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c0cf450 msgr2=0x7ffb2c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/3121231558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c0cf450 0x7ffb2c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/3121231558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c0cf450 0x7ffb2c06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb29d9b640 1 -- 172.21.15.76:0/3121231558 >> v1:172.21.15.76:6789/0 conn(0x7ffb2c161a60 legacy=0x7ffb2c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2959a640 1 -- 172.21.15.76:0/3121231558 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3726674068 0 0) 0x7ffb1c001120 con 0x7ffb2c161a60 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb29d9b640 1 -- 172.21.15.76:0/3121231558 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c0d0020 con 0x7ffb2c0d3060 2024-09-19T04:18:29.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/3121231558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c0cf450 0x7ffb2c06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:29.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/3121231558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0d3060 0x7ffb2c0cef10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ffb200097c0 tx=0x7ffb2001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7143631a3ac8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2959a640 1 -- 172.21.15.76:0/3121231558 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200b1070 con 0x7ffb2c0d3060 2024-09-19T04:18:29.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2959a640 1 -- 172.21.15.76:0/3121231558 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb200a8050 con 0x7ffb2c0d3060 2024-09-19T04:18:29.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb2959a640 1 -- 172.21.15.76:0/3121231558 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200ac860 con 0x7ffb2c0d3060 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- 172.21.15.76:0/3121231558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0d3060 msgr2=0x7ffb2c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3121231558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0d3060 0x7ffb2c0cef10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7ffb200097c0 tx=0x7ffb2001c3e0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- 172.21.15.76:0/3121231558 shutdown_connections 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3121231558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c0cf450 0x7ffb2c06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3121231558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0d3060 0x7ffb2c0cef10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- 172.21.15.76:0/3121231558 >> 172.21.15.76:0/3121231558 conn(0x7ffb2c08bf90 msgr2=0x7ffb2c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- 172.21.15.76:0/3121231558 shutdown_connections 2024-09-19T04:18:29.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.265+0000 7ffb30af7640 1 -- 172.21.15.76:0/3121231558 wait complete. 2024-09-19T04:18:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 Processor -- start 2024-09-19T04:18:29.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 -- start start 2024-09-19T04:18:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0cf450 0x7ffb2c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb2c0d3060 0x7ffb2c1586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c161a60 0x7ffb2c158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c16af20 con 0x7ffb2c0cf450 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c06e690 con 0x7ffb2c0d3060 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2c16b0a0 con 0x7ffb2c161a60 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0cf450 0x7ffb2c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0cf450 0x7ffb2c0cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42456/0 (socket says 172.21.15.76:42456) 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 -- 172.21.15.76:0/3637441535 learned_addr learned my addr 172.21.15.76:0/3637441535 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 -- 172.21.15.76:0/3637441535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c161a60 msgr2=0x7ffb2c158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c161a60 0x7ffb2c158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 -- 172.21.15.76:0/3637441535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb2c0d3060 msgr2=0x7ffb2c1586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb2c0d3060 0x7ffb2c1586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 -- 172.21.15.76:0/3637441535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb2c0cc410 con 0x7ffb2c0cf450 2024-09-19T04:18:29.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0cf450 0x7ffb2c0cbe30 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c780 tx=0x7ffb1c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb1affd640 1 -- 172.21.15.76:0/3637441535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c018070 con 0x7ffb2c0cf450 2024-09-19T04:18:29.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb1affd640 1 -- 172.21.15.76:0/3637441535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb1c0072f0 con 0x7ffb2c0cf450 2024-09-19T04:18:29.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb2c0b7240 con 0x7ffb2c0cf450 2024-09-19T04:18:29.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb1affd640 1 -- 172.21.15.76:0/3637441535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c0047f0 con 0x7ffb2c0cf450 2024-09-19T04:18:29.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.269+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb2c0b7800 con 0x7ffb2c0cf450 2024-09-19T04:18:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb1affd640 1 -- 172.21.15.76:0/3637441535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb1c01b070 con 0x7ffb2c0cf450 2024-09-19T04:18:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb1affd640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb0c0678c0 0x7ffb0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb1affd640 1 -- 172.21.15.76:0/3637441535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7ffb1c085200 con 0x7ffb2c0cf450 2024-09-19T04:18:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb0c0678c0 0x7ffb0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb0c0678c0 0x7ffb0c069d80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ae000 tx=0x7ffb20008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 Processor -- start 2024-09-19T04:18:29.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 -- start start 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf80c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf7bf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf7c0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5564bf6a9960 con 0x5564bf7bfdb0 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5564bf5ed900 con 0x5564bf80bd70 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5564bf610d40 con 0x5564bf80c690 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf7c0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf7c0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42464/0 (socket says 172.21.15.76:42464) 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 -- 172.21.15.76:0/795710633 learned_addr learned my addr 172.21.15.76:0/795710633 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf80c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf7bf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 -- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 msgr2=0x5564bf7bf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf7bf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 -- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 msgr2=0x5564bf80c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf80c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 -- 172.21.15.76:0/795710633 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564bf5b7580 con 0x5564bf7bfdb0 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf80c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf7c0190 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffb24008e60 tx=0x7ffb2400d6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5464ec6c0be72d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb0bfff640 1 -- 172.21.15.76:0/795710633 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400f800 con 0x5564bf7bfdb0 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb0bfff640 1 -- 172.21.15.76:0/795710633 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2400f9a0 con 0x5564bf7bfdb0 2024-09-19T04:18:29.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb0bfff640 1 -- 172.21.15.76:0/795710633 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400fd70 con 0x5564bf7bfdb0 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 -- 172.21.15.76:0/795710633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 msgr2=0x5564bf7c0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf7c0190 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7ffb24008e60 tx=0x7ffb2400d6e0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 -- 172.21.15.76:0/795710633 shutdown_connections 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf7c0190 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf7bf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/795710633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf80c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.273+0000 7ffb33bb3000 1 -- 172.21.15.76:0/795710633 >> 172.21.15.76:0/795710633 conn(0x5564bf7ed040 msgr2=0x5564bf7ef430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- 172.21.15.76:0/795710633 shutdown_connections 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- 172.21.15.76:0/795710633 wait complete. 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- start start 2024-09-19T04:18:29.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 Processor -- start 2024-09-19T04:18:29.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf812ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf813220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf813760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5564bf60cd00 con 0x5564bf7bfdb0 2024-09-19T04:18:29.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5564bf5cbaa0 con 0x5564bf80bd70 2024-09-19T04:18:29.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5564bf5aa340 con 0x5564bf80c690 2024-09-19T04:18:29.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 10 client.0 _ll_register_callbacks cb 0x5564bf7ad760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf812ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf812ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42480/0 (socket says 172.21.15.76:42480) 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 -- 172.21.15.76:0/857081458 learned_addr learned my addr 172.21.15.76:0/857081458 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf813760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 -- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 msgr2=0x5564bf813760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf813220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf813760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 -- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 msgr2=0x5564bf813220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf813220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 -- 172.21.15.76:0/857081458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564bf5b3dc0 con 0x5564bf7bfdb0 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf812ce0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c000fc0 tx=0x7ffb1c055c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f7d7e27d1d75eb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:29.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 1 -- 172.21.15.76:0/857081458 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c054b90 con 0x5564bf7bfdb0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 1 -- 172.21.15.76:0/857081458 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7ffb1c051290 con 0x5564bf7bfdb0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb2400f9a0 con 0x5564bf7bfdb0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 1 -- 172.21.15.76:0/857081458 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7ffb1c05e120 con 0x5564bf7bfdb0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 10 client.7047 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5564bf5b07f0 con 0x5564bf7bfdb0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 1 -- 172.21.15.76:0/857081458 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb2400f9a0 con 0x5564bf7bfdb0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 1 -- 172.21.15.76:0/857081458 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7ffb1c083080 con 0x5564bf7bfdb0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 10 client.7047 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 10 client.7047 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 20 client.7047 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf81b0b0 0x5564bf81d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 4 client.7047 mds_command: new command op to 6783 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 --> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564bf5aae10 con 0x5564bf81b0b0 2024-09-19T04:18:29.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf81b0b0 0x5564bf81d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 10 client.7047 ms_handle_connect on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf81b0b0 0x5564bf81d4a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52f8da5d53ae94f server_cookie=78d60f5677bb76b2 in_seq=0 out_seq=0 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 1 -- 172.21.15.76:0/857081458 <== mds.0 v2:172.21.15.76:6836/3569622848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5564bf5aae10 con 0x5564bf81b0b0 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb097fa640 10 client.7047 handle_command_reply: tid=0 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.277+0000 7ffb33bb3000 1 client.7047 shutdown 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 msgr2=0x5564bf812ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf812ce0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c000fc0 tx=0x7ffb1c055c40 comp rx=0 tx=0).stop 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 shutdown_connections 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80c690 0x5564bf813760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf80bd70 0x5564bf813220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf81b0b0 0x5564bf81d4a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/857081458 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf812ce0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 >> 172.21.15.76:0/857081458 conn(0x5564bf7ed040 msgr2=0x5564bf7bef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb097fa640 0 client.7047 ms_handle_reset on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:29.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 shutdown_connections 2024-09-19T04:18:29.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 1 -- 172.21.15.76:0/857081458 wait complete. 2024-09-19T04:18:29.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.281+0000 7ffb33bb3000 20 client.7047 trim_cache size 0 max 16384 2024-09-19T04:18:29.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.293+0000 7ffb33bb3000 1 Processor -- start 2024-09-19T04:18:29.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.293+0000 7ffb33bb3000 1 -- start start 2024-09-19T04:18:29.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7ed040 0x5564bf7c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf8191e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf819720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c204330 con 0x5564bf7bfdb0 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c203500 con 0x5564bf7ed040 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb2c203c00 con 0x5564bf80bd70 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf8191e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf8191e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42492/0 (socket says 172.21.15.76:42492) 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/3769724003 learned_addr learned my addr 172.21.15.76:0/3769724003 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 msgr2=0x5564bf819720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf819720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7ed040 0x5564bf7c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf819720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7ed040 msgr2=0x5564bf7c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7ed040 0x5564bf7c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/3769724003 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564bf5ce470 con 0x5564bf7bfdb0 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf8191e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffb200af040 tx=0x7ffb20004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ccb3c1b72949a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb0b7fe640 1 -- 172.21.15.76:0/3769724003 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200d2070 con 0x5564bf7bfdb0 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb0b7fe640 1 -- 172.21.15.76:0/3769724003 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb200a8040 con 0x5564bf7bfdb0 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb0b7fe640 1 -- 172.21.15.76:0/3769724003 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200b1040 con 0x5564bf7bfdb0 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- 172.21.15.76:0/3769724003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 msgr2=0x5564bf8191e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf8191e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ffb200af040 tx=0x7ffb20004740 comp rx=0 tx=0).stop 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- 172.21.15.76:0/3769724003 shutdown_connections 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf819720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7bfdb0 0x5564bf8191e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/3769724003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7ed040 0x5564bf7c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- 172.21.15.76:0/3769724003 >> 172.21.15.76:0/3769724003 conn(0x5564bf7c4890 msgr2=0x5564bf818ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- 172.21.15.76:0/3769724003 shutdown_connections 2024-09-19T04:18:29.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- 172.21.15.76:0/3769724003 wait complete. 2024-09-19T04:18:29.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- start start 2024-09-19T04:18:29.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 Processor -- start 2024-09-19T04:18:29.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7bfdb0 0x5564bf81f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7ed040 0x5564bf87ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf880470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5564bf61efd0 con 0x5564bf7ed040 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb2c153520 con 0x5564bf7bfdb0 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5564bf615370 con 0x5564bf80bd70 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 10 client.0 _ll_register_callbacks cb 0x7ffb1c001d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7ed040 0x5564bf87ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7ed040 0x5564bf87ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42496/0 (socket says 172.21.15.76:42496) 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/1826959081 learned_addr learned my addr 172.21.15.76:0/1826959081 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb2ad9d640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf880470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7bfdb0 0x5564bf81f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 msgr2=0x5564bf880470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf880470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7bfdb0 msgr2=0x5564bf81f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7bfdb0 0x5564bf81f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 -- 172.21.15.76:0/1826959081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564bf5a4640 con 0x5564bf7ed040 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7bfdb0 0x5564bf81f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb29d9b640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7ed040 0x5564bf87ff30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ffb2001aa90 tx=0x7ffb2001ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9167daa22ce9c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb097fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb097fa640 1 -- 172.21.15.76:0/1826959081 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200d2070 con 0x5564bf7ed040 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb097fa640 1 -- 172.21.15.76:0/1826959081 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7ffb200b7c60 con 0x5564bf7ed040 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.297+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb200a8040 con 0x5564bf7ed040 2024-09-19T04:18:29.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb097fa640 1 -- 172.21.15.76:0/1826959081 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7ffb200e0050 con 0x5564bf7ed040 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 10 client.7053 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5564bf5a83b0 con 0x5564bf7ed040 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb097fa640 1 -- 172.21.15.76:0/1826959081 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb200a8040 con 0x5564bf7ed040 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb097fa640 1 -- 172.21.15.76:0/1826959081 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7ffb200e3020 con 0x5564bf7ed040 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 10 client.7053 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 10 client.7053 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 20 client.7053 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf80c690 0x5564bf8809b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 4 client.7053 mds_command: new command op to 6783 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 --> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5564bf61e470 con 0x5564bf80c690 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf80c690 0x5564bf8809b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb097fa640 10 client.7053 ms_handle_connect on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:29.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.301+0000 7ffb2a59c640 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf80c690 0x5564bf8809b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc0a846dbcec0b5 server_cookie=76161184ed59e0d1 in_seq=0 out_seq=0 2024-09-19T04:18:29.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb097fa640 1 -- 172.21.15.76:0/1826959081 <== mds.0 v2:172.21.15.76:6836/3569622848 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5564bf61e470 con 0x5564bf80c690 2024-09-19T04:18:29.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb097fa640 10 client.7053 handle_command_reply: tid=0 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 client.7053 shutdown 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7ed040 msgr2=0x5564bf87ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7ed040 0x5564bf87ff30 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ffb2001aa90 tx=0x7ffb2001ac20 comp rx=0 tx=0).stop 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 shutdown_connections 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5564bf80bd70 0x5564bf880470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:6836/3569622848,v1:172.21.15.76:6837/3569622848] conn(0x5564bf80c690 0x5564bf8809b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5564bf7ed040 0x5564bf87ff30 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 --2- 172.21.15.76:0/1826959081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5564bf7bfdb0 0x5564bf81f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb097fa640 0 client.7053 ms_handle_reset on v2:172.21.15.76:6836/3569622848 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 >> 172.21.15.76:0/1826959081 conn(0x5564bf7c4890 msgr2=0x5564bf818ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 shutdown_connections 2024-09-19T04:18:29.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 1 -- 172.21.15.76:0/1826959081 wait complete. 2024-09-19T04:18:29.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb33bb3000 20 client.7053 trim_cache size 0 max 16384 2024-09-19T04:18:29.328 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:29.328 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:18:29.328 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:18:29.328 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:29.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb0c0678c0 msgr2=0x7ffb0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb0c0678c0 0x7ffb0c069d80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ae000 tx=0x7ffb20008040 comp rx=0 tx=0).stop 2024-09-19T04:18:29.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0cf450 msgr2=0x7ffb2c0cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0cf450 0x7ffb2c0cbe30 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c780 tx=0x7ffb1c00cc40 comp rx=0 tx=0).stop 2024-09-19T04:18:29.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 shutdown_connections 2024-09-19T04:18:29.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb0c0678c0 0x7ffb0c069d80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb2c161a60 0x7ffb2c158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb2c0d3060 0x7ffb2c1586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 --2- 172.21.15.76:0/3637441535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb2c0cf450 0x7ffb2c0cbe30 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 >> 172.21.15.76:0/3637441535 conn(0x7ffb2c08bf90 msgr2=0x7ffb2c0b4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 shutdown_connections 2024-09-19T04:18:29.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.325+0000 7ffb30af7640 1 -- 172.21.15.76:0/3637441535 wait complete. 2024-09-19T04:18:29.343 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:1 flush journal 2024-09-19T04:18:29.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783cd99640 1 Processor -- start 2024-09-19T04:18:29.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783cd99640 1 -- start start 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783cd99640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f78380d3060 0x7f78380cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783cd99640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78380cf450 0x7f783806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783cd99640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78380d0300 con 0x7f7838161a60 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783cd99640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f78380b0f60 con 0x7f78380cf450 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783cd99640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f78380cfd40 con 0x7f78380d3060 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7838161a60 0x7f78380d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37380/0 (socket says 172.21.15.76:37380) 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f7836d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f78380cf450 0x7f783806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f783659c640 1 -- 172.21.15.76:0/1967756380 learned_addr learned my addr 172.21.15.76:0/1967756380 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:29.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f782ffff640 1 --2- 172.21.15.76:0/1967756380 >> v2:172.21.15.116:3300/0 conn(0x7f78380d3060 0x7f78380cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f7835d9b640 1 -- 172.21.15.76:0/1967756380 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4026778016 0 0) 0x7f7828001120 con 0x7f7838161a60 2024-09-19T04:18:29.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f7835d9b640 1 -- 172.21.15.76:0/1967756380 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7818003530 con 0x7f7838161a60 2024-09-19T04:18:29.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f7836d9d640 1 -- 172.21.15.76:0/1967756380 >> v2:172.21.15.116:3300/0 conn(0x7f78380d3060 msgr2=0x7f78380cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f7836d9d640 1 --2- 172.21.15.76:0/1967756380 >> v2:172.21.15.116:3300/0 conn(0x7f78380d3060 0x7f78380cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f7836d9d640 1 -- 172.21.15.76:0/1967756380 >> v1:172.21.15.76:6789/0 conn(0x7f7838161a60 legacy=0x7f78380d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.457+0000 7f7836d9d640 1 -- 172.21.15.76:0/1967756380 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78380d0020 con 0x7f78380cf450 2024-09-19T04:18:29.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f7836d9d640 1 --2- 172.21.15.76:0/1967756380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380cf450 0x7f783806d6b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f783808d850 tx=0x7f783001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12095ea58224607e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f7835d9b640 1 -- 172.21.15.76:0/1967756380 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78300b2070 con 0x7f78380cf450 2024-09-19T04:18:29.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f7835d9b640 1 -- 172.21.15.76:0/1967756380 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78300a5930 con 0x7f78380cf450 2024-09-19T04:18:29.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f7835d9b640 1 -- 172.21.15.76:0/1967756380 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78300a5d00 con 0x7f78380cf450 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 -- 172.21.15.76:0/1967756380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380cf450 msgr2=0x7f783806d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 --2- 172.21.15.76:0/1967756380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380cf450 0x7f783806d6b0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f783808d850 tx=0x7f783001c870 comp rx=0 tx=0).stop 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 -- 172.21.15.76:0/1967756380 shutdown_connections 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 --2- 172.21.15.76:0/1967756380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380cf450 0x7f783806d6b0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 --2- 172.21.15.76:0/1967756380 >> v2:172.21.15.116:3300/0 conn(0x7f78380d3060 0x7f78380cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 -- 172.21.15.76:0/1967756380 >> 172.21.15.76:0/1967756380 conn(0x7f783808bf90 msgr2=0x7f783809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 -- 172.21.15.76:0/1967756380 shutdown_connections 2024-09-19T04:18:29.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 -- 172.21.15.76:0/1967756380 wait complete. 2024-09-19T04:18:29.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 Processor -- start 2024-09-19T04:18:29.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.461+0000 7f783cd99640 1 -- start start 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78380cf450 0x7f78380b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380d3060 0x7f7838155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7838161a60 0x7f78380b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f783806e400 con 0x7f78380d3060 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f78380cfad0 con 0x7f78380cf450 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f783806e580 con 0x7f7838161a60 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380d3060 0x7f7838155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380d3060 0x7f7838155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42524/0 (socket says 172.21.15.76:42524) 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 -- 172.21.15.76:0/797562997 learned_addr learned my addr 172.21.15.76:0/797562997 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 -- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7838161a60 msgr2=0x7f78380b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f7836d9d640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7838161a60 0x7f78380b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7838161a60 0x7f78380b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 -- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78380cf450 msgr2=0x7f78380b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783659c640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78380cf450 0x7f78380b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78380cf450 0x7f78380b2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 -- 172.21.15.76:0/797562997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78380b3840 con 0x7f78380d3060 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782ffff640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380d3060 0x7f7838155d40 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f782000d5f0 tx=0x7f782000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782effd640 1 -- 172.21.15.76:0/797562997 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78200125f0 con 0x7f78380d3060 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782effd640 1 -- 172.21.15.76:0/797562997 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7820019070 con 0x7f78380d3060 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78380b3560 con 0x7f78380d3060 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782effd640 1 -- 172.21.15.76:0/797562997 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78200129b0 con 0x7f78380d3060 2024-09-19T04:18:29.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f783815b110 con 0x7f78380d3060 2024-09-19T04:18:29.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782effd640 1 -- 172.21.15.76:0/797562997 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7820012b50 con 0x7f78380d3060 2024-09-19T04:18:29.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782effd640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f781c0678f0 0x7f781c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783659c640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f781c0678f0 0x7f781c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f782effd640 1 -- 172.21.15.76:0/797562997 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7820086190 con 0x7f78380d3060 2024-09-19T04:18:29.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783659c640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f781c0678f0 0x7f781c069db0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f78280012c0 tx=0x7f7828009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.465+0000 7f783fe55000 1 Processor -- start 2024-09-19T04:18:29.469 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- start start 2024-09-19T04:18:29.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadde9d70 0x55feaddea150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddea690 0x55feadd9d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feadd9e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55feadc87960 con 0x55feadde9d70 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55feadbcb900 con 0x55feaddea690 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55feadbeed40 con 0x55feadd9ddb0 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadde9d70 0x55feaddea150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadde9d70 0x55feaddea150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42532/0 (socket says 172.21.15.76:42532) 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 -- 172.21.15.76:0/4227168411 learned_addr learned my addr 172.21.15.76:0/4227168411 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f7836d9d640 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feadd9e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f782ffff640 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddea690 0x55feadd9d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 -- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 msgr2=0x55feadd9e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feadd9e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 -- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddea690 msgr2=0x55feadd9d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddea690 0x55feadd9d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 -- 172.21.15.76:0/4227168411 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feadb95580 con 0x55feadde9d70 2024-09-19T04:18:29.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783659c640 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadde9d70 0x55feaddea150 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7828005d50 tx=0x7f7828010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a231e9516cd497f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f7827fff640 1 -- 172.21.15.76:0/4227168411 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828012820 con 0x55feadde9d70 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f7827fff640 1 -- 172.21.15.76:0/4227168411 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78280129c0 con 0x55feadde9d70 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f7827fff640 1 -- 172.21.15.76:0/4227168411 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7828012d90 con 0x55feadde9d70 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- 172.21.15.76:0/4227168411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadde9d70 msgr2=0x55feaddea150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadde9d70 0x55feaddea150 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7828005d50 tx=0x7f7828010000 comp rx=0 tx=0).stop 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- 172.21.15.76:0/4227168411 shutdown_connections 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feadd9e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddea690 0x55feadd9d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- 172.21.15.76:0/4227168411 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadde9d70 0x55feaddea150 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- 172.21.15.76:0/4227168411 >> 172.21.15.76:0/4227168411 conn(0x55feaddcb040 msgr2=0x55feaddcd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- 172.21.15.76:0/4227168411 shutdown_connections 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- 172.21.15.76:0/4227168411 wait complete. 2024-09-19T04:18:29.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- start start 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 Processor -- start 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadd9ddb0 0x55feaddf0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 0x55feaddf1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddea690 0x55feaddf1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55feadbead00 con 0x55feaddea690 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55feadba9aa0 con 0x55feadd9ddb0 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55feadb88340 con 0x55feadde9d70 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f783fe55000 10 client.0 _ll_register_callbacks cb 0x55feadd8b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f7836d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddea690 0x55feaddf1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f7836d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddea690 0x55feaddf1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42534/0 (socket says 172.21.15.76:42534) 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.469+0000 7f7836d9d640 1 -- 172.21.15.76:0/260874303 learned_addr learned my addr 172.21.15.76:0/260874303 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f782ffff640 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 0x55feaddf1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783659c640 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadd9ddb0 0x55feaddf0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f7836d9d640 1 -- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 msgr2=0x55feaddf1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f7836d9d640 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 0x55feaddf1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f7836d9d640 1 -- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadd9ddb0 msgr2=0x55feaddf0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f7836d9d640 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadd9ddb0 0x55feaddf0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f7836d9d640 1 -- 172.21.15.76:0/260874303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feadb91dc0 con 0x55feaddea690 2024-09-19T04:18:29.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f7836d9d640 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddea690 0x55feaddf1760 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f78300b0040 tx=0x7f783001b0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c251a379f50b64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:29.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 1 -- 172.21.15.76:0/260874303 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78300bd070 con 0x55feaddea690 2024-09-19T04:18:29.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 1 -- 172.21.15.76:0/260874303 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f78300b8440 con 0x55feaddea690 2024-09-19T04:18:29.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78280129c0 con 0x55feaddea690 2024-09-19T04:18:29.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 1 -- 172.21.15.76:0/260874303 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f78300c8050 con 0x55feaddea690 2024-09-19T04:18:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 10 client.7068 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55feadb8e7f0 con 0x55feaddea690 2024-09-19T04:18:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 1 -- 172.21.15.76:0/260874303 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78280129c0 con 0x55feaddea690 2024-09-19T04:18:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 1 -- 172.21.15.76:0/260874303 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f78300cb020 con 0x55feaddea690 2024-09-19T04:18:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 10 client.7068 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:29.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 10 client.7068 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.a 2024-09-19T04:18:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 20 client.7068 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddf90b0 0x55feaddfb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 4 client.7068 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55feadb88e10 con 0x55feaddf90b0 2024-09-19T04:18:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783659c640 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddf90b0 0x55feaddfb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 10 client.7068 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:29.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783659c640 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddf90b0 0x55feaddfb4a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eb65126843a64da7 server_cookie=e98bc5de08addcc in_seq=0 out_seq=0 2024-09-19T04:18:29.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 1 -- 172.21.15.76:0/260874303 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55feadb88e10 con 0x55feaddf90b0 2024-09-19T04:18:29.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 10 client.7068 handle_command_reply: tid=0 2024-09-19T04:18:29.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 client.7068 shutdown 2024-09-19T04:18:29.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddea690 msgr2=0x55feaddf1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddea690 0x55feaddf1760 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f78300b0040 tx=0x7f783001b0b0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 shutdown_connections 2024-09-19T04:18:29.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddf90b0 0x55feaddfb4a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddea690 0x55feaddf1760 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f78257fa640 0 client.7068 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:29.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 0x55feaddf1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 --2- 172.21.15.76:0/260874303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadd9ddb0 0x55feaddf0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 >> 172.21.15.76:0/260874303 conn(0x55feaddcb040 msgr2=0x55feadd9cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 shutdown_connections 2024-09-19T04:18:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 1 -- 172.21.15.76:0/260874303 wait complete. 2024-09-19T04:18:29.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.473+0000 7f783fe55000 20 client.7068 trim_cache size 0 max 16384 2024-09-19T04:18:29.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 Processor -- start 2024-09-19T04:18:29.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- start start 2024-09-19T04:18:29.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddcb040 0x55feaddfed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadd9ddb0 0x55feaddf73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 0x55feaddf7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7838203f00 con 0x55feadd9ddb0 2024-09-19T04:18:29.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f78382030f0 con 0x55feaddcb040 2024-09-19T04:18:29.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f78382037d0 con 0x55feadde9d70 2024-09-19T04:18:29.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadd9ddb0 0x55feaddf73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadd9ddb0 0x55feaddf73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42550/0 (socket says 172.21.15.76:42550) 2024-09-19T04:18:29.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 -- 172.21.15.76:0/4279275883 learned_addr learned my addr 172.21.15.76:0/4279275883 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 -- 172.21.15.76:0/4279275883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 msgr2=0x55feaddf7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 --2- 172.21.15.76:0/4279275883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 0x55feaddf7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 -- 172.21.15.76:0/4279275883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddcb040 msgr2=0x55feaddfed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 --2- 172.21.15.76:0/4279275883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddcb040 0x55feaddfed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 -- 172.21.15.76:0/4279275883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55feadbac470 con 0x55feadd9ddb0 2024-09-19T04:18:29.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f782ffff640 1 --2- 172.21.15.76:0/4279275883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadd9ddb0 0x55feaddf73f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f782005eb80 tx=0x7f7820011340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48fb675dbec3893 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f78277fe640 1 -- 172.21.15.76:0/4279275883 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820082070 con 0x55feadd9ddb0 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f78277fe640 1 -- 172.21.15.76:0/4279275883 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78200564d0 con 0x55feadd9ddb0 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f78277fe640 1 -- 172.21.15.76:0/4279275883 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f782006e070 con 0x55feadd9ddb0 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- 172.21.15.76:0/4279275883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadd9ddb0 msgr2=0x55feaddf73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 --2- 172.21.15.76:0/4279275883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadd9ddb0 0x55feaddf73f0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f782005eb80 tx=0x7f7820011340 comp rx=0 tx=0).stop 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- 172.21.15.76:0/4279275883 shutdown_connections 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 --2- 172.21.15.76:0/4279275883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadde9d70 0x55feaddf7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 --2- 172.21.15.76:0/4279275883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feadd9ddb0 0x55feaddf73f0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 --2- 172.21.15.76:0/4279275883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feaddcb040 0x55feaddfed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- 172.21.15.76:0/4279275883 >> 172.21.15.76:0/4279275883 conn(0x55feadda2890 msgr2=0x55feaddf6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- 172.21.15.76:0/4279275883 shutdown_connections 2024-09-19T04:18:29.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.489+0000 7f783fe55000 1 -- 172.21.15.76:0/4279275883 wait complete. 2024-09-19T04:18:29.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 -- start start 2024-09-19T04:18:29.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 Processor -- start 2024-09-19T04:18:29.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feaddfe330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddcb040 0x55feade21fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadde9d70 0x55feade224e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55feadbee050 con 0x55feaddcb040 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f78380b98a0 con 0x55feadde9d70 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55feadcab0c0 con 0x55feadd9ddb0 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 10 client.0 _ll_register_callbacks cb 0x7f78300b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddcb040 0x55feade21fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddcb040 0x55feade21fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42562/0 (socket says 172.21.15.76:42562) 2024-09-19T04:18:29.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 -- 172.21.15.76:0/966349322 learned_addr learned my addr 172.21.15.76:0/966349322 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783659c640 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feaddfe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f7836d9d640 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadde9d70 0x55feade224e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 -- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 msgr2=0x55feaddfe330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feaddfe330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 -- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadde9d70 msgr2=0x55feade224e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadde9d70 0x55feade224e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 -- 172.21.15.76:0/966349322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55feadb82640 con 0x55feaddcb040 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f782ffff640 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddcb040 0x55feade21fa0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7820011380 tx=0x7f7820000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c993d74cebec0284 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f78257fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f78257fa640 1 -- 172.21.15.76:0/966349322 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7820082070 con 0x55feaddcb040 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f78257fa640 1 -- 172.21.15.76:0/966349322 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f78200547d0 con 0x55feaddcb040 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78200564d0 con 0x55feaddcb040 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f78257fa640 1 -- 172.21.15.76:0/966349322 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f782006e040 con 0x55feaddcb040 2024-09-19T04:18:29.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 10 client.7074 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55feadb863b0 con 0x55feaddcb040 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f78257fa640 1 -- 172.21.15.76:0/966349322 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78200564d0 con 0x55feaddcb040 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f78257fa640 1 -- 172.21.15.76:0/966349322 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f7820013070 con 0x55feaddcb040 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 10 client.7074 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 10 client.7074 resolve_mds: resolved 5:1 to role '5:1' aka daemon mds.a 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 20 client.7074 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddea690 0x55feade22a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 4 client.7074 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55feadbfc470 con 0x55feaddea690 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783659c640 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddea690 0x55feade22a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f78257fa640 10 client.7074 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:29.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.493+0000 7f783659c640 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddea690 0x55feade22a20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f3246ff3edfb332 server_cookie=c81484ee55f79484 in_seq=0 out_seq=0 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f78257fa640 1 -- 172.21.15.76:0/966349322 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55feadbfc470 con 0x55feaddea690 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f78257fa640 10 client.7074 handle_command_reply: tid=0 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 client.7074 shutdown 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddcb040 msgr2=0x55feade21fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddcb040 0x55feade21fa0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7820011380 tx=0x7f7820000bd0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 shutdown_connections 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55feadde9d70 0x55feade224e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55feaddea690 0x55feade22a20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55feaddcb040 0x55feade21fa0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 --2- 172.21.15.76:0/966349322 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55feadd9ddb0 0x55feaddfe330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f78257fa640 0 client.7074 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 >> 172.21.15.76:0/966349322 conn(0x55feadda2890 msgr2=0x55feaddf72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 shutdown_connections 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 1 -- 172.21.15.76:0/966349322 wait complete. 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.509+0000 7f783fe55000 20 client.7074 trim_cache size 0 max 16384 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:18:29.514 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:29.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f781c0678f0 msgr2=0x7f781c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f781c0678f0 0x7f781c069db0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f78280012c0 tx=0x7f7828009040 comp rx=0 tx=0).stop 2024-09-19T04:18:29.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380d3060 msgr2=0x7f7838155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380d3060 0x7f7838155d40 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f782000d5f0 tx=0x7f782000dab0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 shutdown_connections 2024-09-19T04:18:29.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f781c0678f0 0x7f781c069db0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7838161a60 0x7f78380b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f78380d3060 0x7f7838155d40 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 --2- 172.21.15.76:0/797562997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78380cf450 0x7f78380b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 >> 172.21.15.76:0/797562997 conn(0x7f783808bf90 msgr2=0x7f783815f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 shutdown_connections 2024-09-19T04:18:29.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.513+0000 7f783cd99640 1 -- 172.21.15.76:0/797562997 wait complete. 2024-09-19T04:18:29.530 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:2 flush journal 2024-09-19T04:18:29.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5789335640 1 Processor -- start 2024-09-19T04:18:29.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5789335640 1 -- start start 2024-09-19T04:18:29.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5789335640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 0x7f5784165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5789335640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57840aede0 con 0x7f5784070020 2024-09-19T04:18:29.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5789335640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f578415fa10 con 0x7f57841520a0 2024-09-19T04:18:29.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5789335640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5784152480 con 0x7f578415f3a0 2024-09-19T04:18:29.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57827fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5784070020 0x7f578415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37390/0 (socket says 172.21.15.76:37390) 2024-09-19T04:18:29.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57827fc640 1 -- 172.21.15.76:0/2188942493 learned_addr learned my addr 172.21.15.76:0/2188942493 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:29.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57837fe640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 0x7f5784165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 676927065 0 0) 0x7f57840aede0 con 0x7f5784070020 2024-09-19T04:18:29.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f576c003530 con 0x7f5784070020 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 375717854 0 0) 0x7f576c003530 con 0x7f5784070020 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 msgr2=0x7f5784165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 0x7f5784165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 >> v1:172.21.15.116:6789/0 conn(0x7f57841520a0 legacy=0x7f578406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5784074250 con 0x7f5784070020 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5778001680 con 0x7f5784070020 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 >> v1:172.21.15.76:6789/0 conn(0x7f5784070020 legacy=0x7f578415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 0x7f576c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f576c003f70 0x7f576c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576c004d50 0x7f576c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f576c00ba80 con 0x7f576c004d50 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f576c00bc30 con 0x7f576c003f70 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f576c00be30 con 0x7f578415f3a0 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57837fe640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576c004d50 0x7f576c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57827fc640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f576c003f70 0x7f576c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f5782ffd640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 0x7f576c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57837fe640 1 -- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 msgr2=0x7f576c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57837fe640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578415f3a0 0x7f576c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57837fe640 1 -- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f576c003f70 msgr2=0x7f576c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57837fe640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f576c003f70 0x7f576c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.645+0000 7f57837fe640 1 -- 172.21.15.76:0/2188942493 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c003530 con 0x7f576c004d50 2024-09-19T04:18:29.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f57837fe640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576c004d50 0x7f576c0051d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f577c009fa0 tx=0x7f577c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf679a0157d5cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0b2070 con 0x7f576c004d50 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c0a5d10 con 0x7f576c004d50 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5781ffb640 1 -- 172.21.15.76:0/2188942493 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c009bf0 con 0x7f576c004d50 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- 172.21.15.76:0/2188942493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576c004d50 msgr2=0x7f576c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 --2- 172.21.15.76:0/2188942493 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f576c004d50 0x7f576c0051d0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f577c009fa0 tx=0x7f577c01cc40 comp rx=0 tx=0).stop 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f57837fe640 1 -- 172.21.15.76:0/2188942493 reap_dead start 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- 172.21.15.76:0/2188942493 shutdown_connections 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- 172.21.15.76:0/2188942493 >> 172.21.15.76:0/2188942493 conn(0x7f57840b1f90 msgr2=0x7f57840b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- 172.21.15.76:0/2188942493 shutdown_connections 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- 172.21.15.76:0/2188942493 wait complete. 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 Processor -- start 2024-09-19T04:18:29.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- start start 2024-09-19T04:18:29.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578409c070 0x7f578409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5784147ca0 0x7f5784154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f578409c450 0x7f5784154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5784073a50 con 0x7f578409c450 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5784073770 con 0x7f5784147ca0 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.649+0000 7f5789335640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5784073d30 con 0x7f578409c070 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f578409c450 0x7f5784154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f578409c450 0x7f5784154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42588/0 (socket says 172.21.15.76:42588) 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 -- 172.21.15.76:0/2434411182 learned_addr learned my addr 172.21.15.76:0/2434411182 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57827fc640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5784147ca0 0x7f5784154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f5782ffd640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578409c070 0x7f578409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 -- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578409c070 msgr2=0x7f578409db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578409c070 0x7f578409db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 -- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5784147ca0 msgr2=0x7f5784154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5784147ca0 0x7f5784154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 -- 172.21.15.76:0/2434411182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5784153030 con 0x7f578409c450 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57827fc640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5784147ca0 0x7f5784154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f57837fe640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f578409c450 0x7f5784154e80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b0040 tx=0x7f577c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f576b7fe640 1 -- 172.21.15.76:0/2434411182 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c0b2040 con 0x7f578409c450 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f576b7fe640 1 -- 172.21.15.76:0/2434411182 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577c0a5b10 con 0x7f578409c450 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57841532d0 con 0x7f578409c450 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f576b7fe640 1 -- 172.21.15.76:0/2434411182 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c009bf0 con 0x7f578409c450 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5784153890 con 0x7f578409c450 2024-09-19T04:18:29.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f576b7fe640 1 -- 172.21.15.76:0/2434411182 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f577c0b8420 con 0x7f578409c450 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f576b7fe640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57700678c0 0x7f5770069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f5782ffd640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57700678c0 0x7f5770069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f576b7fe640 1 -- 172.21.15.76:0/2434411182 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f577c120160 con 0x7f578409c450 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f5782ffd640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57700678c0 0x7f5770069d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f57740015e0 tx=0x7f5774009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f578c3f1000 1 Processor -- start 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.653+0000 7f578c3f1000 1 -- start start 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add164870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add165190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555add04e960 con 0x555add1b0d70 2024-09-19T04:18:29.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555adcf92900 con 0x555add1b1690 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555adcfb5d40 con 0x555add164db0 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42602/0 (socket says 172.21.15.76:42602) 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 -- 172.21.15.76:0/1126741049 learned_addr learned my addr 172.21.15.76:0/1126741049 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 -- 172.21.15.76:0/1126741049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 msgr2=0x555add165190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 --2- 172.21.15.76:0/1126741049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add165190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 -- 172.21.15.76:0/1126741049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 msgr2=0x555add164870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 --2- 172.21.15.76:0/1126741049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 -- 172.21.15.76:0/1126741049 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adcf5c580 con 0x555add1b0d70 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 --2- 172.21.15.76:0/1126741049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b1150 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f577400f870 tx=0x7f5774012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fdc584cd6e245c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5768ff9640 1 -- 172.21.15.76:0/1126741049 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774025070 con 0x555add1b0d70 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5768ff9640 1 -- 172.21.15.76:0/1126741049 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577401b050 con 0x555add1b0d70 2024-09-19T04:18:29.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5768ff9640 1 -- 172.21.15.76:0/1126741049 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774014c10 con 0x555add1b0d70 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- 172.21.15.76:0/1126741049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 msgr2=0x555add1b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1126741049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b1150 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f577400f870 tx=0x7f5774012000 comp rx=0 tx=0).stop 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- 172.21.15.76:0/1126741049 shutdown_connections 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1126741049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add165190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1126741049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add164870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1126741049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b1150 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- 172.21.15.76:0/1126741049 >> 172.21.15.76:0/1126741049 conn(0x555add192040 msgr2=0x555add194430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- 172.21.15.76:0/1126741049 shutdown_connections 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- 172.21.15.76:0/1126741049 wait complete. 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- start start 2024-09-19T04:18:29.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 Processor -- start 2024-09-19T04:18:29.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add1b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add1b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555adcfb1d00 con 0x555add1b0d70 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555adcf70aa0 con 0x555add1b1690 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555adcf4f340 con 0x555add164db0 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f578c3f1000 10 client.0 _ll_register_callbacks cb 0x555add152760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42612/0 (socket says 172.21.15.76:42612) 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 -- 172.21.15.76:0/1064890476 learned_addr learned my addr 172.21.15.76:0/1064890476 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5782ffd640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add1b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57837fe640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add1b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 -- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 msgr2=0x555add1b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add1b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 -- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 msgr2=0x555add1b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add1b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 -- 172.21.15.76:0/1064890476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555adcf58dc0 con 0x555add1b0d70 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57837fe640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add1b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f57827fc640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b8220 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f577800f3d0 tx=0x7f577800f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8cfb69cf689caa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.657+0000 7f5761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 1 -- 172.21.15.76:0/1064890476 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5778019060 con 0x555add1b0d70 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 1 -- 172.21.15.76:0/1064890476 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f57780145f0 con 0x555add1b0d70 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577401b050 con 0x555add1b0d70 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 1 -- 172.21.15.76:0/1064890476 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f577801f050 con 0x555add1b0d70 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 10 client.7086 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:29.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x555adcf557f0 con 0x555add1b0d70 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 1 -- 172.21.15.76:0/1064890476 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577401b050 con 0x555add1b0d70 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 1 -- 172.21.15.76:0/1064890476 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f5778022020 con 0x555add1b0d70 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 10 client.7086 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 10 client.7086 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.d 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 20 client.7086 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add1c00b0 0x555add1c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 4 client.7086 mds_command: new command op to 5336 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 --> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555adcf4fe10 con 0x555add1c00b0 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5782ffd640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add1c00b0 0x555add1c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5782ffd640 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add1c00b0 0x555add1c24a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f9ecca08e9d8d419 server_cookie=6814cb1439efa2c9 in_seq=0 out_seq=0 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 10 client.7086 ms_handle_connect on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:29.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 1 -- 172.21.15.76:0/1064890476 <== mds.2 v2:172.21.15.116:6834/3224374251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555adcf4fe10 con 0x555add1c00b0 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 10 client.7086 handle_command_reply: tid=0 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 client.7086 shutdown 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 msgr2=0x555add1b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b8220 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f577800f3d0 tx=0x7f577800f890 comp rx=0 tx=0).stop 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 shutdown_connections 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add1c00b0 0x555add1c24a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add1b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f5761ffb640 0 client.7086 ms_handle_reset on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add1b0d70 0x555add1b8220 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 --2- 172.21.15.76:0/1064890476 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add1b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 >> 172.21.15.76:0/1064890476 conn(0x555add192040 msgr2=0x555add163f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 shutdown_connections 2024-09-19T04:18:29.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 1 -- 172.21.15.76:0/1064890476 wait complete. 2024-09-19T04:18:29.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.661+0000 7f578c3f1000 20 client.7086 trim_cache size 0 max 16384 2024-09-19T04:18:29.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 Processor -- start 2024-09-19T04:18:29.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 -- start start 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add192040 0x555add16cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add16d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b0d70 0x555add1bad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f57841ff9a0 con 0x555add192040 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f57841feb70 con 0x555add1b0d70 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f57841ff270 con 0x555add164db0 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add192040 0x555add16cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add192040 0x555add16cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42624/0 (socket says 172.21.15.76:42624) 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 -- 172.21.15.76:0/3670461138 learned_addr learned my addr 172.21.15.76:0/3670461138 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 -- 172.21.15.76:0/3670461138 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 msgr2=0x555add16d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 --2- 172.21.15.76:0/3670461138 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add16d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 -- 172.21.15.76:0/3670461138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b0d70 msgr2=0x555add1bad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 --2- 172.21.15.76:0/3670461138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b0d70 0x555add1bad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 -- 172.21.15.76:0/3670461138 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555adcf73470 con 0x555add192040 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5782ffd640 1 --2- 172.21.15.76:0/3670461138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add192040 0x555add16cba0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f577400f870 tx=0x7f5774014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53adc9c79e7ec699 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5763fff640 1 -- 172.21.15.76:0/3670461138 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774039070 con 0x555add192040 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5763fff640 1 -- 172.21.15.76:0/3670461138 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577401b040 con 0x555add192040 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.677+0000 7f5763fff640 1 -- 172.21.15.76:0/3670461138 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774025040 con 0x555add192040 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3670461138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add192040 msgr2=0x555add16cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3670461138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add192040 0x555add16cba0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f577400f870 tx=0x7f5774014000 comp rx=0 tx=0).stop 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3670461138 shutdown_connections 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3670461138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b0d70 0x555add1bad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3670461138 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add164db0 0x555add16d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3670461138 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add192040 0x555add16cba0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3670461138 >> 172.21.15.76:0/3670461138 conn(0x555add169890 msgr2=0x555add1b7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3670461138 shutdown_connections 2024-09-19T04:18:29.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3670461138 wait complete. 2024-09-19T04:18:29.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- start start 2024-09-19T04:18:29.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 Processor -- start 2024-09-19T04:18:29.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add164db0 0x555add1bd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add1b0d70 0x555add224d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add225260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555add16b9d0 con 0x555add164db0 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555adcf4f480 con 0x555add1b1690 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555adcf6bac0 con 0x555add1b0d70 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 10 client.0 _ll_register_callbacks cb 0x7f5778001560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add164db0 0x555add1bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add164db0 0x555add1bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42632/0 (socket says 172.21.15.76:42632) 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 -- 172.21.15.76:0/3684168093 learned_addr learned my addr 172.21.15.76:0/3684168093 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f57827fc640 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add1b0d70 0x555add224d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f57837fe640 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add225260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 -- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add1b0d70 msgr2=0x555add224d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add1b0d70 0x555add224d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 -- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 msgr2=0x555add225260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add225260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 -- 172.21.15.76:0/3684168093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555adcf49640 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5782ffd640 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add164db0 0x555add1bd700 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5774011010 tx=0x7f57740011a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd944a6d4ef6ef84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5761ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5761ffb640 1 -- 172.21.15.76:0/3684168093 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774039070 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5761ffb640 1 -- 172.21.15.76:0/3684168093 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f577403ec40 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f577401b040 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5761ffb640 1 -- 172.21.15.76:0/3684168093 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f5774025040 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 10 client.7092 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x555adcf4d3b0 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5761ffb640 1 -- 172.21.15.76:0/3684168093 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f577401b040 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5761ffb640 1 -- 172.21.15.76:0/3684168093 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f577402b040 con 0x555add164db0 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 10 client.7092 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:29.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 10 client.7092 resolve_mds: resolved 5:2 to role '5:2' aka daemon mds.d 2024-09-19T04:18:29.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 20 client.7092 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:29.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add2257a0 0x555add221710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 4 client.7092 mds_command: new command op to 5336 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:18:29.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 --> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555adcfc3470 con 0x555add2257a0 2024-09-19T04:18:29.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f57827fc640 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add2257a0 0x555add221710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f5761ffb640 10 client.7092 ms_handle_connect on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:29.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.681+0000 7f57827fc640 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add2257a0 0x555add221710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cd10c026592ea177 server_cookie=ddcdb6a3e1f02b69 in_seq=0 out_seq=0 2024-09-19T04:18:29.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5761ffb640 1 -- 172.21.15.76:0/3684168093 <== mds.2 v2:172.21.15.116:6834/3224374251 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555adcfc3470 con 0x555add2257a0 2024-09-19T04:18:29.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5761ffb640 10 client.7092 handle_command_reply: tid=0 2024-09-19T04:18:29.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 client.7092 shutdown 2024-09-19T04:18:29.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add164db0 msgr2=0x555add1bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add164db0 0x555add1bd700 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5774011010 tx=0x7f57740011a0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 shutdown_connections 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:6834/3224374251,v1:172.21.15.116:6835/3224374251] conn(0x555add2257a0 0x555add221710 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555add1b1690 0x555add225260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5761ffb640 0 client.7092 ms_handle_reset on v2:172.21.15.116:6834/3224374251 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555add1b0d70 0x555add224d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 --2- 172.21.15.76:0/3684168093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555add164db0 0x555add1bd700 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 >> 172.21.15.76:0/3684168093 conn(0x555add169890 msgr2=0x555add16bb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 shutdown_connections 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 1 -- 172.21.15.76:0/3684168093 wait complete. 2024-09-19T04:18:29.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f578c3f1000 20 client.7092 trim_cache size 0 max 16384 2024-09-19T04:18:29.699 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:29.699 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:18:29.699 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:18:29.699 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57700678c0 msgr2=0x7f5770069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57700678c0 0x7f5770069d80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f57740015e0 tx=0x7f5774009040 comp rx=0 tx=0).stop 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f578409c450 msgr2=0x7f5784154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f578409c450 0x7f5784154e80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f577c0b0040 tx=0x7f577c009750 comp rx=0 tx=0).stop 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 shutdown_connections 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57700678c0 0x7f5770069d80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f578409c450 0x7f5784154e80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5784147ca0 0x7f5784154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 --2- 172.21.15.76:0/2434411182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f578409c070 0x7f578409db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 >> 172.21.15.76:0/2434411182 conn(0x7f57840b1f90 msgr2=0x7f57840b4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 shutdown_connections 2024-09-19T04:18:29.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.697+0000 7f5789335640 1 -- 172.21.15.76:0/2434411182 wait complete. 2024-09-19T04:18:29.713 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155c9c8640 1 Processor -- start 2024-09-19T04:18:29.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155c9c8640 1 -- start start 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155c9c8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15580cf620 0x7f1558168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155c9c8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15580d0300 con 0x7f1558161a60 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155c9c8640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f155806e6c0 con 0x7f15580d3060 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155c9c8640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f15580cfa90 con 0x7f15580cf620 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1558161a60 0x7f15580d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37392/0 (socket says 172.21.15.76:37392) 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155659c640 1 -- 172.21.15.76:0/2775652379 learned_addr learned my addr 172.21.15.76:0/2775652379 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1556d9d640 1 --2- 172.21.15.76:0/2775652379 >> v2:172.21.15.116:3300/0 conn(0x7f15580cf620 0x7f1558168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 82849642 0 0) 0x7f1548001120 con 0x7f1558161a60 2024-09-19T04:18:29.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1544003530 con 0x7f1558161a60 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3534222022 0 0) 0x7f1544003530 con 0x7f1558161a60 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 >> v2:172.21.15.116:3300/0 conn(0x7f15580cf620 msgr2=0x7f1558168f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 --2- 172.21.15.76:0/2775652379 >> v2:172.21.15.116:3300/0 conn(0x7f15580cf620 0x7f1558168f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 >> v1:172.21.15.116:6789/0 conn(0x7f15580d3060 legacy=0x7f15580cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15580b0f60 con 0x7f1558161a60 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f1548001850 con 0x7f1558161a60 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 >> v1:172.21.15.76:6789/0 conn(0x7f1558161a60 legacy=0x7f15580d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15440039c0 0x7f1544003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f1544004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15440051d0 0x7f154400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15580cfa90 con 0x7f1544004810 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f154400c170 con 0x7f15440039c0 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f154400c370 con 0x7f15440051d0 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f154ffff640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f1544004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f154ffff640 1 -- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15440051d0 msgr2=0x7f154400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f154ffff640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15440051d0 0x7f154400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155659c640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15440039c0 0x7f1544003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f154ffff640 1 -- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15440039c0 msgr2=0x7f1544003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f154ffff640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15440039c0 0x7f1544003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f154ffff640 1 -- 172.21.15.76:0/2775652379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1544003530 con 0x7f1544004810 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f1556d9d640 1 -- 172.21.15.76:0/2775652379 reap_dead start 2024-09-19T04:18:29.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.821+0000 7f155659c640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15440039c0 0x7f1544003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f154ffff640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f1544004c90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1540001780 tx=0x7f154001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330f0241707ee31b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15400013a0 con 0x7f1544004810 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15400052f0 con 0x7f1544004810 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f1555d9b640 1 -- 172.21.15.76:0/2775652379 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540003840 con 0x7f1544004810 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- 172.21.15.76:0/2775652379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 msgr2=0x7f1544004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f1544004c90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1540001780 tx=0x7f154001c3d0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- 172.21.15.76:0/2775652379 shutdown_connections 2024-09-19T04:18:29.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 --2- 172.21.15.76:0/2775652379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f1544004c90 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- 172.21.15.76:0/2775652379 >> 172.21.15.76:0/2775652379 conn(0x7f155808bf90 msgr2=0x7f155809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:29.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- 172.21.15.76:0/2775652379 shutdown_connections 2024-09-19T04:18:29.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- 172.21.15.76:0/2775652379 wait complete. 2024-09-19T04:18:29.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 Processor -- start 2024-09-19T04:18:29.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- start start 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f155815d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15580cd390 0x7f155815d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f155815de90 0x7f15580b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15580d2d60 con 0x7f1544004810 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f155806d190 con 0x7f15580cd390 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155c9c8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f15580d2ee0 con 0x7f155815de90 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f155815d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f155815d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42648/0 (socket says 172.21.15.76:42648) 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 -- 172.21.15.76:0/1244793758 learned_addr learned my addr 172.21.15.76:0/1244793758 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f1556d9d640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f155815de90 0x7f15580b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f154ffff640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15580cd390 0x7f155815d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 -- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f155815de90 msgr2=0x7f15580b4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f155815de90 0x7f15580b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 -- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15580cd390 msgr2=0x7f155815d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15580cd390 0x7f155815d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 -- 172.21.15.76:0/1244793758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15580cc2f0 con 0x7f1544004810 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f155659c640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f155815d3b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f154800d820 tx=0x7f154800dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.825+0000 7f154effd640 1 -- 172.21.15.76:0/1244793758 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548019070 con 0x7f1544004810 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f154effd640 1 -- 172.21.15.76:0/1244793758 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15480074e0 con 0x7f1544004810 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15580cc5d0 con 0x7f1544004810 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f154effd640 1 -- 172.21.15.76:0/1244793758 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1548004a10 con 0x7f1544004810 2024-09-19T04:18:29.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15580ccb90 con 0x7f1544004810 2024-09-19T04:18:29.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15580cf370 con 0x7f1544004810 2024-09-19T04:18:29.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f154effd640 1 -- 172.21.15.76:0/1244793758 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f154801c070 con 0x7f1544004810 2024-09-19T04:18:29.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f154effd640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15380678f0 0x7f1538069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:29.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f154effd640 1 -- 172.21.15.76:0/1244793758 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f1548085740 con 0x7f1544004810 2024-09-19T04:18:29.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f154ffff640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15380678f0 0x7f1538069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:29.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.829+0000 7f154ffff640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15380678f0 0x7f1538069db0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1540001750 tx=0x7f15400011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:29.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:29.833+0000 7f154effd640 1 -- 172.21.15.76:0/1244793758 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f15580d0300 con 0x7f1544004810 2024-09-19T04:18:30.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.025+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15580b2910 con 0x7f1544004810 2024-09-19T04:18:30.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.025+0000 7f154effd640 1 -- 172.21.15.76:0/1244793758 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5267 (secure 0 0 0) 0x7f15480560f0 con 0x7f1544004810 2024-09-19T04:18:30.027 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:30.027 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"btime":"2024-09-19T04:18:27:920425+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:27.400688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6783,"mds_1":6780,"mds_2":5336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5336":{"gid":5336,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[5336,6780,6783]},"id":5}]} 2024-09-19T04:18:30.028 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 74 2024-09-19T04:18:30.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15380678f0 msgr2=0x7f1538069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15380678f0 0x7f1538069db0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1540001750 tx=0x7f15400011a0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 msgr2=0x7f155815d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f155815d3b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f154800d820 tx=0x7f154800dce0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 shutdown_connections 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15380678f0 0x7f1538069db0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f155815de90 0x7f15580b4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15580cd390 0x7f155815d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 --2- 172.21.15.76:0/1244793758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1544004810 0x7f155815d3b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 >> 172.21.15.76:0/1244793758 conn(0x7f155808bf90 msgr2=0x7f15580b33f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 shutdown_connections 2024-09-19T04:18:30.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.029+0000 7f155c9c8640 1 -- 172.21.15.76:0/1244793758 wait complete. 2024-09-19T04:18:30.045 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start /d1/d2/d3 2024-09-19T04:18:30.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 Processor -- start 2024-09-19T04:18:30.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 -- start start 2024-09-19T04:18:30.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f730815bec0 0x7f73080ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815c2a0 0x7f73080aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73080af2a0 0x7f730815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f73080b3290 con 0x7f73080af2a0 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f73080b3570 con 0x7f730815bec0 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730e697640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f73080b3850 con 0x7f730815c2a0 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73080af2a0 0x7f730815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f73080af2a0 0x7f730815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42652/0 (socket says 172.21.15.76:42652) 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 -- 172.21.15.76:0/1957542334 learned_addr learned my addr 172.21.15.76:0/1957542334 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/1957542334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815c2a0 0x7f73080aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f7307fff640 1 --2- 172.21.15.76:0/1957542334 >> v2:172.21.15.116:3300/0 conn(0x7f730815bec0 0x7f73080ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 -- 172.21.15.76:0/1957542334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815c2a0 msgr2=0x7f73080aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 --2- 172.21.15.76:0/1957542334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815c2a0 0x7f73080aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 -- 172.21.15.76:0/1957542334 >> v2:172.21.15.116:3300/0 conn(0x7f730815bec0 msgr2=0x7f73080ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 --2- 172.21.15.76:0/1957542334 >> v2:172.21.15.116:3300/0 conn(0x7f730815bec0 0x7f73080ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.169+0000 7f730cc34640 1 -- 172.21.15.76:0/1957542334 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73080b3b30 con 0x7f73080af2a0 2024-09-19T04:18:30.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 --2- 172.21.15.76:0/1957542334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73080af2a0 0x7f730815fc60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7308071710 tx=0x7f730001c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768f460f1645af31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f73077fe640 1 -- 172.21.15.76:0/1957542334 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000a5580 con 0x7f73080af2a0 2024-09-19T04:18:30.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f73077fe640 1 -- 172.21.15.76:0/1957542334 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73000a5720 con 0x7f73080af2a0 2024-09-19T04:18:30.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f73077fe640 1 -- 172.21.15.76:0/1957542334 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000a5af0 con 0x7f73080af2a0 2024-09-19T04:18:30.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- 172.21.15.76:0/1957542334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73080af2a0 msgr2=0x7f730815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 --2- 172.21.15.76:0/1957542334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73080af2a0 0x7f730815fc60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f7308071710 tx=0x7f730001c700 comp rx=0 tx=0).stop 2024-09-19T04:18:30.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- 172.21.15.76:0/1957542334 shutdown_connections 2024-09-19T04:18:30.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 --2- 172.21.15.76:0/1957542334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f73080af2a0 0x7f730815fc60 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 --2- 172.21.15.76:0/1957542334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815c2a0 0x7f73080aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 --2- 172.21.15.76:0/1957542334 >> v2:172.21.15.116:3300/0 conn(0x7f730815bec0 0x7f73080ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- 172.21.15.76:0/1957542334 >> 172.21.15.76:0/1957542334 conn(0x7f73080b1f90 msgr2=0x7f73080b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- 172.21.15.76:0/1957542334 shutdown_connections 2024-09-19T04:18:30.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- 172.21.15.76:0/1957542334 wait complete. 2024-09-19T04:18:30.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 Processor -- start 2024-09-19T04:18:30.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- start start 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73080af2a0 0x7f730814c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815bec0 0x7f730815a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f730815c2a0 0x7f730815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f73080ad740 con 0x7f730815c2a0 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f73080afc90 con 0x7f73080af2a0 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f730806e000 con 0x7f730815bec0 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f730815c2a0 0x7f730815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f730815c2a0 0x7f730815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42658/0 (socket says 172.21.15.76:42658) 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 -- 172.21.15.76:0/2871218671 learned_addr learned my addr 172.21.15.76:0/2871218671 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815bec0 0x7f730815a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f7307fff640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73080af2a0 0x7f730814c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 -- 172.21.15.76:0/2871218671 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815bec0 msgr2=0x7f730815a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f730815bec0 0x7f730815a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 -- 172.21.15.76:0/2871218671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73080af2a0 msgr2=0x7f730814c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73080af2a0 0x7f730814c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 -- 172.21.15.76:0/2871218671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7308151230 con 0x7f730815c2a0 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f7307fff640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f73080af2a0 0x7f730814c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730cc34640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f730815c2a0 0x7f730815ac30 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f730001adb0 tx=0x7f73000194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f73057fa640 1 -- 172.21.15.76:0/2871218671 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7300009510 con 0x7f730815c2a0 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f73057fa640 1 -- 172.21.15.76:0/2871218671 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73000be070 con 0x7f730815c2a0 2024-09-19T04:18:30.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- 172.21.15.76:0/2871218671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7308151510 con 0x7f730815c2a0 2024-09-19T04:18:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f73057fa640 1 -- 172.21.15.76:0/2871218671 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000b88d0 con 0x7f730815c2a0 2024-09-19T04:18:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.173+0000 7f730e697640 1 -- 172.21.15.76:0/2871218671 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7308151ad0 con 0x7f730815c2a0 2024-09-19T04:18:30.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f73057fa640 1 -- 172.21.15.76:0/2871218671 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f73000b8a70 con 0x7f730815c2a0 2024-09-19T04:18:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f73057fa640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72e80678f0 0x7f72e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7307fff640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72e80678f0 0x7f72e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f73057fa640 1 -- 172.21.15.76:0/2871218671 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7300120930 con 0x7f730815c2a0 2024-09-19T04:18:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7307fff640 1 --2- 172.21.15.76:0/2871218671 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72e80678f0 0x7f72e8069db0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f72f8009840 tx=0x7f72f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 Processor -- start 2024-09-19T04:18:30.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 -- start start 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239fca800 0x559239fcabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f80ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239f81400 0x559239f817e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559239dceaa0 con 0x559239f81400 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559239d74e40 con 0x559239fcb120 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559239dcaa60 con 0x559239fca800 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f730cc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239f81400 0x559239f817e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f730cc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239f81400 0x559239f817e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42662/0 (socket says 172.21.15.76:42662) 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.177+0000 7f730cc34640 1 -- 172.21.15.76:0/2315563225 learned_addr learned my addr 172.21.15.76:0/2315563225 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f80ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f730cc34640 1 -- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239fca800 msgr2=0x559239fcabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7307fff640 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239fca800 0x559239fcabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f730cc34640 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239fca800 0x559239fcabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f730cc34640 1 -- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 msgr2=0x559239f80ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f730cc34640 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f80ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f730cc34640 1 -- 172.21.15.76:0/2315563225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559239d91c10 con 0x559239f81400 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f80ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f730cc34640 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239f81400 0x559239f817e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f73000f0ad0 tx=0x7f73000c2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9828fee0f64b034 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72fdffb640 1 -- 172.21.15.76:0/2315563225 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000f1240 con 0x559239f81400 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72fdffb640 1 -- 172.21.15.76:0/2315563225 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73000f0b40 con 0x559239f81400 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72fdffb640 1 -- 172.21.15.76:0/2315563225 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000f4ef0 con 0x559239f81400 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/2315563225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239f81400 msgr2=0x559239f817e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239f81400 0x559239f817e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f73000f0ad0 tx=0x7f73000c2000 comp rx=0 tx=0).stop 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/2315563225 shutdown_connections 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239f81400 0x559239f817e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f80ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- 172.21.15.76:0/2315563225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239fca800 0x559239fcabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/2315563225 >> 172.21.15.76:0/2315563225 conn(0x559239f7d3b0 msgr2=0x559239f7f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/2315563225 shutdown_connections 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/2315563225 wait complete. 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- start start 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 Processor -- start 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239f8b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239f8baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f8c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559239dceaa0 con 0x559239fca800 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559239d8c400 con 0x559239fcb120 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559239d74e40 con 0x559239f81400 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 10 client.0 _ll_register_callbacks cb 0x559239f6c300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239f8baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239f8baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42666/0 (socket says 172.21.15.76:42666) 2024-09-19T04:18:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 -- 172.21.15.76:0/762389749 learned_addr learned my addr 172.21.15.76:0/762389749 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7307fff640 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239f8b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 -- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 msgr2=0x559239f8b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f730cc34640 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f8c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239f8b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 -- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 msgr2=0x559239f8c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f8c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 -- 172.21.15.76:0/762389749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559239d75090 con 0x559239fca800 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239f8baf0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f72f000efd0 tx=0x7f72f0010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d528df56881d3da4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72e6ffd640 1 -- 172.21.15.76:0/762389749 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f000f040 con 0x559239fca800 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72e6ffd640 1 -- 172.21.15.76:0/762389749 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f72f001dc40 con 0x559239fca800 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73000f0b40 con 0x559239fca800 2024-09-19T04:18:30.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72e6ffd640 1 -- 172.21.15.76:0/762389749 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f72f001de40 con 0x559239fca800 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 10 client.7110 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x559239d6e440 con 0x559239fca800 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72e6ffd640 1 -- 172.21.15.76:0/762389749 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73000f0b40 con 0x559239fca800 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f72e6ffd640 1 -- 172.21.15.76:0/762389749 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f72f0014800 con 0x559239fca800 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 10 client.7110 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 10 client.7110 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 20 client.7110 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x559239fd9890 0x559239fdbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 4 client.7110 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559239d68bf0 con 0x559239fd9890 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.181+0000 7f7307fff640 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x559239fd9890 0x559239fdbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7307fff640 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x559239fd9890 0x559239fdbc80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8acfeb8485a13bbb server_cookie=e39b753057eb2b7a in_seq=0 out_seq=0 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f72e6ffd640 10 client.7110 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f72e6ffd640 1 -- 172.21.15.76:0/762389749 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559239d68bf0 con 0x559239fd9890 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f72e6ffd640 10 client.7110 handle_command_reply: tid=0 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 client.7110 shutdown 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 msgr2=0x559239f8baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239f8baf0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f72f000efd0 tx=0x7f72f0010680 comp rx=0 tx=0).stop 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 shutdown_connections 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x559239f8c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x559239fd9890 0x559239fdbc80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239f8baf0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 --2- 172.21.15.76:0/762389749 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239f8b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f72e6ffd640 0 client.7110 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 >> 172.21.15.76:0/762389749 conn(0x559239f7d3b0 msgr2=0x559239f7ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 shutdown_connections 2024-09-19T04:18:30.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 1 -- 172.21.15.76:0/762389749 wait complete. 2024-09-19T04:18:30.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.185+0000 7f7311753000 20 client.7110 trim_cache size 0 max 16384 2024-09-19T04:18:30.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.201+0000 7f7311753000 1 Processor -- start 2024-09-19T04:18:30.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.201+0000 7f7311753000 1 -- start start 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239fdf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fca800 0x559239f807b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fcb120 0x559239f80cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7308201c40 con 0x559239fcb120 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7308200e10 con 0x559239fca800 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7308201510 con 0x559239f81400 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fcb120 0x559239f80cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fcb120 0x559239f80cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42672/0 (socket says 172.21.15.76:42672) 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 -- 172.21.15.76:0/757763230 learned_addr learned my addr 172.21.15.76:0/757763230 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 -- 172.21.15.76:0/757763230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 msgr2=0x559239fdf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 --2- 172.21.15.76:0/757763230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239fdf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 -- 172.21.15.76:0/757763230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fca800 msgr2=0x559239f807b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 --2- 172.21.15.76:0/757763230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fca800 0x559239f807b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 -- 172.21.15.76:0/757763230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559239e29100 con 0x559239fcb120 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f730cc34640 1 --2- 172.21.15.76:0/757763230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fcb120 0x559239f80cf0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f73000f0840 tx=0x7f73000acf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543d9d2d613f14a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72fd7fa640 1 -- 172.21.15.76:0/757763230 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000f0b40 con 0x559239fcb120 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72fd7fa640 1 -- 172.21.15.76:0/757763230 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73000f1240 con 0x559239fcb120 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72fd7fa640 1 -- 172.21.15.76:0/757763230 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73000f9720 con 0x559239fcb120 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- 172.21.15.76:0/757763230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fcb120 msgr2=0x559239f80cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- 172.21.15.76:0/757763230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fcb120 0x559239f80cf0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f73000f0840 tx=0x7f73000acf90 comp rx=0 tx=0).stop 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- 172.21.15.76:0/757763230 shutdown_connections 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- 172.21.15.76:0/757763230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fcb120 0x559239f80cf0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- 172.21.15.76:0/757763230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fca800 0x559239f807b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- 172.21.15.76:0/757763230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239fdf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- 172.21.15.76:0/757763230 >> 172.21.15.76:0/757763230 conn(0x559239f7d3b0 msgr2=0x559239fc97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- 172.21.15.76:0/757763230 shutdown_connections 2024-09-19T04:18:30.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- 172.21.15.76:0/757763230 wait complete. 2024-09-19T04:18:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- start start 2024-09-19T04:18:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 Processor -- start 2024-09-19T04:18:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239fde4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239fdea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x55923a002780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559239d53a70 con 0x559239fca800 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7308154bc0 con 0x559239fcb120 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559239dceaa0 con 0x559239f81400 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 10 client.0 _ll_register_callbacks cb 0x7f72f0006ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239fdea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239fdea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42674/0 (socket says 172.21.15.76:42674) 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 -- 172.21.15.76:0/2234431882 learned_addr learned my addr 172.21.15.76:0/2234431882 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 -- 172.21.15.76:0/2234431882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 msgr2=0x559239fde4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239fde4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 -- 172.21.15.76:0/2234431882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 msgr2=0x55923a002780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x55923a002780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 -- 172.21.15.76:0/2234431882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559239deafa0 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72ff7fe640 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239fdea10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f72f00046a0 tx=0x7f72f0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f6361c94dd8b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72e6ffd640 1 -- 172.21.15.76:0/2234431882 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72f0026830 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72e6ffd640 1 -- 172.21.15.76:0/2234431882 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f72f0010b80 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73000f1240 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72e6ffd640 1 -- 172.21.15.76:0/2234431882 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f72f0036050 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 10 client.7116 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x559239d5ca20 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72e6ffd640 1 -- 172.21.15.76:0/2234431882 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73000f1240 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.205+0000 7f72e6ffd640 1 -- 172.21.15.76:0/2234431882 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f72f0039020 con 0x559239fca800 2024-09-19T04:18:30.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 10 client.7116 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 10 client.7116 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 20 client.7116 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55923a045fd0 0x55923a0463b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 4 client.7116 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3"}] 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3"}) -- 0x559239ddc1d0 con 0x55923a045fd0 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7307fff640 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55923a045fd0 0x55923a0463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7307fff640 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55923a045fd0 0x55923a0463b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9df032c6116cae6e server_cookie=e6bfffd7f36f3b44 in_seq=0 out_seq=0 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f72e6ffd640 10 client.7116 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f72e6ffd640 1 -- 172.21.15.76:0/2234431882 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x559239ddc1d0 con 0x55923a045fd0 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f72e6ffd640 10 client.7116 handle_command_reply: tid=0 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 client.7116 shutdown 2024-09-19T04:18:30.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 msgr2=0x559239fdea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239fdea10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f72f00046a0 tx=0x7f72f0004740 comp rx=0 tx=0).stop 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 shutdown_connections 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559239fcb120 0x55923a002780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x55923a045fd0 0x55923a0463b0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559239fca800 0x559239fdea10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 --2- 172.21.15.76:0/2234431882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559239f81400 0x559239fde4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f72e6ffd640 0 client.7116 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 >> 172.21.15.76:0/2234431882 conn(0x559239f7d3b0 msgr2=0x559239fc9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 shutdown_connections 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 1 -- 172.21.15.76:0/2234431882 wait complete. 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.209+0000 7f7311753000 20 client.7116 trim_cache size 0 max 16384 2024-09-19T04:18:30.215 INFO:teuthology.orchestra.run.smithi076.stderr:Error EXDEV: Not rank 0 2024-09-19T04:18:30.216 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-19T04:18:30.217 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub abort 2024-09-19T04:18:30.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.325+0000 7f675438c640 1 Processor -- start 2024-09-19T04:18:30.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.325+0000 7f675438c640 1 -- start start 2024-09-19T04:18:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 0x7f674c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f674c15f3a0 0x7f674c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f674c070250 con 0x7f674c06fe50 2024-09-19T04:18:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f674c152480 con 0x7f674c15f3a0 2024-09-19T04:18:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f674c0aede0 con 0x7f674c1520a0 2024-09-19T04:18:30.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f674c15f3a0 0x7f674c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f674c15f3a0 0x7f674c0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42686/0 (socket says 172.21.15.76:42686) 2024-09-19T04:18:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 -- 172.21.15.76:0/118415510 learned_addr learned my addr 172.21.15.76:0/118415510 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 -- 172.21.15.76:0/118415510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 msgr2=0x7f674c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 --2- 172.21.15.76:0/118415510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 0x7f674c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 -- 172.21.15.76:0/118415510 >> v1:172.21.15.116:6789/0 conn(0x7f674c06fe50 legacy=0x7f674c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 -- 172.21.15.76:0/118415510 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c15fa10 con 0x7f674c15f3a0 2024-09-19T04:18:30.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6752929640 1 --2- 172.21.15.76:0/118415510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 0x7f674c0adc80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6744009e50 tx=0x7f674401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24de33f05d04502 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6751126640 1 -- 172.21.15.76:0/118415510 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440a5870 con 0x7f674c15f3a0 2024-09-19T04:18:30.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6751126640 1 -- 172.21.15.76:0/118415510 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67440a5a10 con 0x7f674c15f3a0 2024-09-19T04:18:30.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f6751126640 1 -- 172.21.15.76:0/118415510 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440a5de0 con 0x7f674c15f3a0 2024-09-19T04:18:30.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- 172.21.15.76:0/118415510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 msgr2=0x7f674c0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 --2- 172.21.15.76:0/118415510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 0x7f674c0adc80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6744009e50 tx=0x7f674401c9f0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- 172.21.15.76:0/118415510 shutdown_connections 2024-09-19T04:18:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 --2- 172.21.15.76:0/118415510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 0x7f674c0adc80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 --2- 172.21.15.76:0/118415510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 0x7f674c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- 172.21.15.76:0/118415510 >> 172.21.15.76:0/118415510 conn(0x7f674c0b1f90 msgr2=0x7f674c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- 172.21.15.76:0/118415510 shutdown_connections 2024-09-19T04:18:30.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.329+0000 7f675438c640 1 -- 172.21.15.76:0/118415510 wait complete. 2024-09-19T04:18:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 Processor -- start 2024-09-19T04:18:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 -- start start 2024-09-19T04:18:30.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f674c06fe50 0x7f674c14c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 0x7f674c1453e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 0x7f674c15d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f674c073d30 con 0x7f674c15f3a0 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f674c073a50 con 0x7f674c06fe50 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f674c167f10 con 0x7f674c1520a0 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 0x7f674c15d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 0x7f674c15d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42696/0 (socket says 172.21.15.76:42696) 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 -- 172.21.15.76:0/2260886354 learned_addr learned my addr 172.21.15.76:0/2260886354 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752128640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f674c06fe50 0x7f674c14c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6751927640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 0x7f674c1453e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 -- 172.21.15.76:0/2260886354 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 msgr2=0x7f674c1453e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f674c1520a0 0x7f674c1453e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 -- 172.21.15.76:0/2260886354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f674c06fe50 msgr2=0x7f674c14c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f674c06fe50 0x7f674c14c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 -- 172.21.15.76:0/2260886354 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f674c14c7a0 con 0x7f674c15f3a0 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752128640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f674c06fe50 0x7f674c14c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6752929640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f674c15f3a0 0x7f674c15d880 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f674401b0a0 tx=0x7f6744019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6742ffd640 1 -- 172.21.15.76:0/2260886354 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440b8650 con 0x7f674c15f3a0 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6742ffd640 1 -- 172.21.15.76:0/2260886354 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67440b8810 con 0x7f674c15f3a0 2024-09-19T04:18:30.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 -- 172.21.15.76:0/2260886354 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f674c14ebe0 con 0x7f674c15f3a0 2024-09-19T04:18:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6742ffd640 1 -- 172.21.15.76:0/2260886354 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440b8c00 con 0x7f674c15f3a0 2024-09-19T04:18:30.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f675438c640 1 -- 172.21.15.76:0/2260886354 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f674c14f1a0 con 0x7f674c15f3a0 2024-09-19T04:18:30.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.333+0000 7f6742ffd640 1 -- 172.21.15.76:0/2260886354 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f67440c1020 con 0x7f674c15f3a0 2024-09-19T04:18:30.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6742ffd640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f67300678c0 0x7f6730069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6742ffd640 1 -- 172.21.15.76:0/2260886354 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f67441204e0 con 0x7f674c15f3a0 2024-09-19T04:18:30.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f67300678c0 0x7f6730069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 --2- 172.21.15.76:0/2260886354 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f67300678c0 0x7f6730069d80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f674c074c20 tx=0x7f6748008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 Processor -- start 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- start start 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af256e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 0x561af2521ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af2522800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561af23275b0 con 0x561af256e380 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561af2371bf0 con 0x561af2522420 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561af232c900 con 0x561af256eca0 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af256e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af256e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42702/0 (socket says 172.21.15.76:42702) 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 -- 172.21.15.76:0/1775679583 learned_addr learned my addr 172.21.15.76:0/1775679583 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 -- 172.21.15.76:0/1775679583 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 msgr2=0x561af2521ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 --2- 172.21.15.76:0/1775679583 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 0x561af2521ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 -- 172.21.15.76:0/1775679583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 msgr2=0x561af2522800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 --2- 172.21.15.76:0/1775679583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af2522800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 -- 172.21.15.76:0/1775679583 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af232b9d0 con 0x561af256e380 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6752128640 1 --2- 172.21.15.76:0/1775679583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af256e760 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f674800cab0 tx=0x7f674800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba8c6cc5c8438fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f672ffff640 1 -- 172.21.15.76:0/1775679583 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6748023070 con 0x561af256e380 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f672ffff640 1 -- 172.21.15.76:0/1775679583 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6748013db0 con 0x561af256e380 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f672ffff640 1 -- 172.21.15.76:0/1775679583 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674801e600 con 0x561af256e380 2024-09-19T04:18:30.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- 172.21.15.76:0/1775679583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 msgr2=0x561af256e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 --2- 172.21.15.76:0/1775679583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af256e760 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f674800cab0 tx=0x7f674800f000 comp rx=0 tx=0).stop 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- 172.21.15.76:0/1775679583 shutdown_connections 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 --2- 172.21.15.76:0/1775679583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af2522800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 --2- 172.21.15.76:0/1775679583 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 0x561af2521ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 --2- 172.21.15.76:0/1775679583 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af256e760 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- 172.21.15.76:0/1775679583 >> 172.21.15.76:0/1775679583 conn(0x561af251bf30 msgr2=0x561af2551410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- 172.21.15.76:0/1775679583 shutdown_connections 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- 172.21.15.76:0/1775679583 wait complete. 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 -- start start 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.337+0000 7f6757448000 1 Processor -- start 2024-09-19T04:18:30.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af25750e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af2575620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 0x561af2575b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561af230b190 con 0x561af256e380 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561af236dbb0 con 0x561af2522420 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561af232f120 con 0x561af256eca0 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 10 client.0 _ll_register_callbacks cb 0x561af250f910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af2575620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af2575620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42708/0 (socket says 172.21.15.76:42708) 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 -- 172.21.15.76:0/1968483593 learned_addr learned my addr 172.21.15.76:0/1968483593 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6752128640 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af25750e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6752929640 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 0x561af2575b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 -- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 msgr2=0x561af2575b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 0x561af2575b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 -- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 msgr2=0x561af25750e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af25750e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 -- 172.21.15.76:0/1968483593 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af232e570 con 0x561af256e380 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6751927640 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af2575620 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f674c1fadb0 tx=0x7f673c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d96f3b30c5749eb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 1 -- 172.21.15.76:0/1968483593 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c015070 con 0x561af256e380 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 1 -- 172.21.15.76:0/1968483593 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f673c0115f0 con 0x561af256e380 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6748013db0 con 0x561af256e380 2024-09-19T04:18:30.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 1 -- 172.21.15.76:0/1968483593 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f673c020050 con 0x561af256e380 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 10 client.7128 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x561af23323c0 con 0x561af256e380 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 1 -- 172.21.15.76:0/1968483593 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6748013db0 con 0x561af256e380 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 1 -- 172.21.15.76:0/1968483593 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f673c023020 con 0x561af256e380 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 10 client.7128 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 10 client.7128 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 20 client.7128 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af257d4b0 0x561af257f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 4 client.7128 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561af240a040 con 0x561af257d4b0 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6752128640 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af257d4b0 0x561af257f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 10 client.7128 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6752128640 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af257d4b0 0x561af257f8a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b7af17cbb1dd782 server_cookie=7e5a52be3e75306b in_seq=0 out_seq=0 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 1 -- 172.21.15.76:0/1968483593 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561af240a040 con 0x561af257d4b0 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 10 client.7128 handle_command_reply: tid=0 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 client.7128 shutdown 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 msgr2=0x561af2575620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af2575620 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f674c1fadb0 tx=0x7f673c00c520 comp rx=0 tx=0).stop 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 shutdown_connections 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256eca0 0x561af2575b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af257d4b0 0x561af257f8a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af2575620 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f672d7fa640 0 client.7128 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 --2- 172.21.15.76:0/1968483593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af25750e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 >> 172.21.15.76:0/1968483593 conn(0x561af251bf30 msgr2=0x561af25215e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 shutdown_connections 2024-09-19T04:18:30.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 1 -- 172.21.15.76:0/1968483593 wait complete. 2024-09-19T04:18:30.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.341+0000 7f6757448000 20 client.7128 trim_cache size 0 max 16384 2024-09-19T04:18:30.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.357+0000 7f6757448000 1 Processor -- start 2024-09-19T04:18:30.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.357+0000 7f6757448000 1 -- start start 2024-09-19T04:18:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af251bf30 0x561af2553610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af2522420 0x561af2553b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256e380 0x561af257f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f674c2042a0 con 0x561af2522420 2024-09-19T04:18:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f674c2034d0 con 0x561af251bf30 2024-09-19T04:18:30.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f674c203b70 con 0x561af256e380 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af2522420 0x561af2553b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af2522420 0x561af2553b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42712/0 (socket says 172.21.15.76:42712) 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 -- 172.21.15.76:0/2156976024 learned_addr learned my addr 172.21.15.76:0/2156976024 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 -- 172.21.15.76:0/2156976024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256e380 msgr2=0x561af257f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 --2- 172.21.15.76:0/2156976024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256e380 0x561af257f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 -- 172.21.15.76:0/2156976024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af251bf30 msgr2=0x561af2553610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 --2- 172.21.15.76:0/2156976024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af251bf30 0x561af2553610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 -- 172.21.15.76:0/2156976024 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561af23115c0 con 0x561af2522420 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6751927640 1 --2- 172.21.15.76:0/2156976024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af2522420 0x561af2553b50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f673c00a6c0 tx=0x7f673c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523237fd752c42b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672f7fe640 1 -- 172.21.15.76:0/2156976024 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c015040 con 0x561af2522420 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672f7fe640 1 -- 172.21.15.76:0/2156976024 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c0115f0 con 0x561af2522420 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672f7fe640 1 -- 172.21.15.76:0/2156976024 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c004220 con 0x561af2522420 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2156976024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af2522420 msgr2=0x561af2553b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- 172.21.15.76:0/2156976024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af2522420 0x561af2553b50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f673c00a6c0 tx=0x7f673c011000 comp rx=0 tx=0).stop 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2156976024 shutdown_connections 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- 172.21.15.76:0/2156976024 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af256e380 0x561af257f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- 172.21.15.76:0/2156976024 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af2522420 0x561af2553b50 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- 172.21.15.76:0/2156976024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af251bf30 0x561af2553610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2156976024 >> 172.21.15.76:0/2156976024 conn(0x561af252a210 msgr2=0x561af2552f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2156976024 shutdown_connections 2024-09-19T04:18:30.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2156976024 wait complete. 2024-09-19T04:18:30.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- start start 2024-09-19T04:18:30.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 Processor -- start 2024-09-19T04:18:30.369 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-19T04:18:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af251bf30 0x561af2582cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af259db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af259e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561af252d5a0 con 0x561af256e380 2024-09-19T04:18:30.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f674c0b09f0 con 0x561af2522420 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561af2370ae0 con 0x561af251bf30 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 10 client.0 _ll_register_callbacks cb 0x7f673c006da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af259e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af259e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42726/0 (socket says 172.21.15.76:42726) 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 -- 172.21.15.76:0/2726924525 learned_addr learned my addr 172.21.15.76:0/2726924525 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 -- 172.21.15.76:0/2726924525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af251bf30 msgr2=0x561af2582cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af251bf30 0x561af2582cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 -- 172.21.15.76:0/2726924525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 msgr2=0x561af259db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af259db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 -- 172.21.15.76:0/2726924525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561af23f3280 con 0x561af256e380 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6752929640 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af259e0d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f67440ecbf0 tx=0x7f67440f05e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca57312b1b9d1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672d7fa640 1 -- 172.21.15.76:0/2726924525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67440f90e0 con 0x561af256e380 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672d7fa640 1 -- 172.21.15.76:0/2726924525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f6744120070 con 0x561af256e380 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f673c0115f0 con 0x561af256e380 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672d7fa640 1 -- 172.21.15.76:0/2726924525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f67440a5a10 con 0x561af256e380 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 10 client.7134 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x561af23ca7d0 con 0x561af256e380 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672d7fa640 1 -- 172.21.15.76:0/2726924525 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f673c0115f0 con 0x561af256e380 2024-09-19T04:18:30.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f672d7fa640 1 -- 172.21.15.76:0/2726924525 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f67440efbe0 con 0x561af256e380 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 10 client.7134 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 10 client.7134 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 20 client.7134 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af256eca0 0x561af259e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 4 client.7134 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.361+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x561af230b930 con 0x561af256eca0 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6752128640 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af256eca0 0x561af259e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f672d7fa640 10 client.7134 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6752128640 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af256eca0 0x561af259e610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f27d3148286be1ae server_cookie=d9480cab2d52db71 in_seq=0 out_seq=0 2024-09-19T04:18:30.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f672d7fa640 1 -- 172.21.15.76:0/2726924525 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x561af230b930 con 0x561af256eca0 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f672d7fa640 10 client.7134 handle_command_reply: tid=0 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 client.7134 shutdown 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 msgr2=0x561af259e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af259e0d0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f67440ecbf0 tx=0x7f67440f05e0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 shutdown_connections 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561af256eca0 0x561af259e610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561af256e380 0x561af259e0d0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f672d7fa640 0 client.7134 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561af2522420 0x561af259db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 --2- 172.21.15.76:0/2726924525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561af251bf30 0x561af2582cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 >> 172.21.15.76:0/2726924525 conn(0x561af252a210 msgr2=0x561af2553720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 shutdown_connections 2024-09-19T04:18:30.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 1 -- 172.21.15.76:0/2726924525 wait complete. 2024-09-19T04:18:30.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.365+0000 7f6757448000 20 client.7134 trim_cache size 0 max 16384 2024-09-19T04:18:30.374 INFO:teuthology.orchestra.run.smithi076.stderr:Error EXDEV: Not rank 0 2024-09-19T04:18:30.374 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2024-09-19T04:18:30.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b84405640 1 Processor -- start 2024-09-19T04:18:30.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b84405640 1 -- start start 2024-09-19T04:18:30.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b84405640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7b7c1520a0 0x7f7b7c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b84405640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b7c070250 con 0x7f7b7c06fe50 2024-09-19T04:18:30.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b84405640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b7c0aede0 con 0x7f7b7c15f3a0 2024-09-19T04:18:30.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b84405640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7b7c152480 con 0x7f7b7c1520a0 2024-09-19T04:18:30.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b819a0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7b7c06fe50 0x7f7b7c15ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37398/0 (socket says 172.21.15.76:37398) 2024-09-19T04:18:30.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.481+0000 7f7b819a0640 1 -- 172.21.15.76:0/10007309 learned_addr learned my addr 172.21.15.76:0/10007309 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 --2- 172.21.15.76:0/10007309 >> v2:172.21.15.116:3300/0 conn(0x7f7b7c1520a0 0x7f7b7c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3616897852 0 0) 0x7f7b7c070250 con 0x7f7b7c06fe50 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b68003530 con 0x7f7b7c06fe50 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3061204804 0 0) 0x7f7b68003530 con 0x7f7b7c06fe50 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 >> v2:172.21.15.116:3300/0 conn(0x7f7b7c1520a0 msgr2=0x7f7b7c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 --2- 172.21.15.76:0/10007309 >> v2:172.21.15.116:3300/0 conn(0x7f7b7c1520a0 0x7f7b7c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 >> v1:172.21.15.116:6789/0 conn(0x7f7b7c15f3a0 legacy=0x7f7b7c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c15fa10 con 0x7f7b7c06fe50 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f7b6c001680 con 0x7f7b7c06fe50 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 >> v1:172.21.15.76:6789/0 conn(0x7f7b7c06fe50 legacy=0x7f7b7c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b680039c0 0x7f7b68003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b68004810 0x7f7b68004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b680051d0 0x7f7b6800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6800c040 con 0x7f7b680039c0 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6800c1f0 con 0x7f7b68004810 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6800c3f0 con 0x7f7b680051d0 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b680039c0 0x7f7b68003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b819a0640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b68004810 0x7f7b68004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 -- 172.21.15.76:0/10007309 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b680051d0 msgr2=0x7f7b6800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b680051d0 0x7f7b6800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 -- 172.21.15.76:0/10007309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b68004810 msgr2=0x7f7b68004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b68004810 0x7f7b68004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b829a2640 1 -- 172.21.15.76:0/10007309 reap_dead start 2024-09-19T04:18:30.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 -- 172.21.15.76:0/10007309 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b68003530 con 0x7f7b680039c0 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b821a1640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b680039c0 0x7f7b68003de0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009ae0 tx=0x7f7b7801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be9b81f0ac97105 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780b2070 con 0x7f7b680039c0 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b780a5600 con 0x7f7b680039c0 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.485+0000 7f7b8119f640 1 -- 172.21.15.76:0/10007309 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780096b0 con 0x7f7b680039c0 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- 172.21.15.76:0/10007309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b680039c0 msgr2=0x7f7b68003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b680039c0 0x7f7b68003de0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7b78009ae0 tx=0x7f7b7801ca90 comp rx=0 tx=0).stop 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- 172.21.15.76:0/10007309 shutdown_connections 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 --2- 172.21.15.76:0/10007309 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b680039c0 0x7f7b68003de0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- 172.21.15.76:0/10007309 >> 172.21.15.76:0/10007309 conn(0x7f7b7c0b1f90 msgr2=0x7f7b7c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- 172.21.15.76:0/10007309 shutdown_connections 2024-09-19T04:18:30.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- 172.21.15.76:0/10007309 wait complete. 2024-09-19T04:18:30.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 Processor -- start 2024-09-19T04:18:30.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- start start 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b680039c0 0x7f7b7c148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b7c14bc50 0x7f7b7c09dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b7c15d830 0x7f7b7c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c073a50 con 0x7f7b7c14bc50 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c073770 con 0x7f7b680039c0 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7c073d30 con 0x7f7b7c15d830 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b7c14bc50 0x7f7b7c09dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b7c14bc50 0x7f7b7c09dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42736/0 (socket says 172.21.15.76:42736) 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 -- 172.21.15.76:0/2449854416 learned_addr learned my addr 172.21.15.76:0/2449854416 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b829a2640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b7c15d830 0x7f7b7c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b821a1640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b680039c0 0x7f7b7c148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 -- 172.21.15.76:0/2449854416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b7c15d830 msgr2=0x7f7b7c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7b7c15d830 0x7f7b7c15dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 -- 172.21.15.76:0/2449854416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b680039c0 msgr2=0x7f7b7c148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b680039c0 0x7f7b7c148840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 -- 172.21.15.76:0/2449854416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b7c148d80 con 0x7f7b7c14bc50 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b819a0640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7b7c14bc50 0x7f7b7c09dc40 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00d700 tx=0x7f7b6c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b72ffd640 1 -- 172.21.15.76:0/2449854416 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c019070 con 0x7f7b7c14bc50 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b72ffd640 1 -- 172.21.15.76:0/2449854416 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b6c0074e0 con 0x7f7b7c14bc50 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- 172.21.15.76:0/2449854416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b7c14c1a0 con 0x7f7b7c14bc50 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b72ffd640 1 -- 172.21.15.76:0/2449854416 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0049f0 con 0x7f7b7c14bc50 2024-09-19T04:18:30.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.489+0000 7f7b84405640 1 -- 172.21.15.76:0/2449854416 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b7c14c760 con 0x7f7b7c14bc50 2024-09-19T04:18:30.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b72ffd640 1 -- 172.21.15.76:0/2449854416 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7b6c01c070 con 0x7f7b7c14bc50 2024-09-19T04:18:30.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b72ffd640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b600678f0 0x7f7b60069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b821a1640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b600678f0 0x7f7b60069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b72ffd640 1 -- 172.21.15.76:0/2449854416 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7b6c085890 con 0x7f7b7c14bc50 2024-09-19T04:18:30.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b821a1640 1 --2- 172.21.15.76:0/2449854416 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7b600678f0 0x7f7b60069db0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b780af000 tx=0x7f7b78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 Processor -- start 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- start start 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc10d9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc108d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561fc0e925b0 con 0x561fc108d420 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561fc0edcbf0 con 0x561fc10d9ca0 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561fc0e97900 con 0x561fc10d9380 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc108d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc108d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42742/0 (socket says 172.21.15.76:42742) 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 -- 172.21.15.76:0/1920808893 learned_addr learned my addr 172.21.15.76:0/1920808893 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 -- 172.21.15.76:0/1920808893 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 msgr2=0x561fc10d9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b819a0640 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc10d9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 -- 172.21.15.76:0/1920808893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 msgr2=0x561fc108cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 -- 172.21.15.76:0/1920808893 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc0e969d0 con 0x561fc108d420 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b829a2640 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc108d800 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0744a0 tx=0x7f7b74001ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8765940b9e210440 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b5ffff640 1 -- 172.21.15.76:0/1920808893 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b74003980 con 0x561fc108d420 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b5ffff640 1 -- 172.21.15.76:0/1920808893 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b74003b40 con 0x561fc108d420 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b5ffff640 1 -- 172.21.15.76:0/1920808893 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b7400d6d0 con 0x561fc108d420 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- 172.21.15.76:0/1920808893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 msgr2=0x561fc108d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc108d800 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f7b7c0744a0 tx=0x7f7b74001ff0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- 172.21.15.76:0/1920808893 shutdown_connections 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc108d800 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1920808893 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc10d9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- 172.21.15.76:0/1920808893 >> 172.21.15.76:0/1920808893 conn(0x561fc1086f30 msgr2=0x561fc10bc410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- 172.21.15.76:0/1920808893 shutdown_connections 2024-09-19T04:18:30.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.493+0000 7f7b874c1000 1 -- 172.21.15.76:0/1920808893 wait complete. 2024-09-19T04:18:30.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- start start 2024-09-19T04:18:30.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 Processor -- start 2024-09-19T04:18:30.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10b9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc1094d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc1095280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561fc0e76190 con 0x561fc108d420 2024-09-19T04:18:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561fc0ed8bb0 con 0x561fc10d9ca0 2024-09-19T04:18:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561fc0ea6d90 con 0x561fc10d9380 2024-09-19T04:18:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 10 client.0 _ll_register_callbacks cb 0x561fc107a910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10b9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10b9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42752/0 (socket says 172.21.15.76:42752) 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 -- 172.21.15.76:0/4207219961 learned_addr learned my addr 172.21.15.76:0/4207219961 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 -- 172.21.15.76:0/4207219961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 msgr2=0x561fc1094d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc1094d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 -- 172.21.15.76:0/4207219961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 msgr2=0x561fc1095280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc1095280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 -- 172.21.15.76:0/4207219961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc0e99570 con 0x561fc108d420 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b821a1640 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10b9a20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7b78000c10 tx=0x7f7b68005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ad6da5293d4a6b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/4207219961 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780b2040 con 0x561fc108d420 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/4207219961 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7b780ad640 con 0x561fc108d420 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b74003b40 con 0x561fc108d420 2024-09-19T04:18:30.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/4207219961 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f7b780cf050 con 0x561fc108d420 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 10 client.7146 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x561fc0e9d3c0 con 0x561fc108d420 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/4207219961 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b74003b40 con 0x561fc108d420 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/4207219961 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f7b780d2020 con 0x561fc108d420 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 10 client.7146 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 10 client.7146 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 20 client.7146 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10bb920 0x561fc10bbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 4 client.7146 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fc0f75040 con 0x561fc10bb920 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b819a0640 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10bb920 0x561fc10bbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 10 client.7146 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b819a0640 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10bb920 0x561fc10bbd00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbdf8f60b501e989 server_cookie=bcf2ab87cbeefc51 in_seq=0 out_seq=0 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/4207219961 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561fc0f75040 con 0x561fc10bb920 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 10 client.7146 handle_command_reply: tid=0 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 client.7146 shutdown 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 msgr2=0x561fc10b9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10b9a20 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7b78000c10 tx=0x7f7b68005840 comp rx=0 tx=0).stop 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 shutdown_connections 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc1095280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc1094d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10bb920 0x561fc10bbd00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 --2- 172.21.15.76:0/4207219961 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10b9a20 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b5d7fa640 0 client.7146 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 >> 172.21.15.76:0/4207219961 conn(0x561fc1086f30 msgr2=0x561fc108c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 shutdown_connections 2024-09-19T04:18:30.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 1 -- 172.21.15.76:0/4207219961 wait complete. 2024-09-19T04:18:30.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.497+0000 7f7b874c1000 20 client.7146 trim_cache size 0 max 16384 2024-09-19T04:18:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 Processor -- start 2024-09-19T04:18:30.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- start start 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc108b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c201470 con 0x561fc108d420 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7b7c201b10 con 0x561fc10d9ca0 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7c14ef00 con 0x561fc10d9380 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10f08a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42762/0 (socket says 172.21.15.76:42762) 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/1857834460 learned_addr learned my addr 172.21.15.76:0/1857834460 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b819a0640 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc108b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 msgr2=0x561fc108b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc108b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 msgr2=0x561fc108bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b829a2640 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/1857834460 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fc0e7c5c0 con 0x561fc108d420 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10f08a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7b7801cad0 tx=0x7f7b780bf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899ea22c74d9ff12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b5f7fe640 1 -- 172.21.15.76:0/1857834460 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780b2040 con 0x561fc108d420 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b5f7fe640 1 -- 172.21.15.76:0/1857834460 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b780c7070 con 0x561fc108d420 2024-09-19T04:18:30.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b5f7fe640 1 -- 172.21.15.76:0/1857834460 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780ada00 con 0x561fc108d420 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- 172.21.15.76:0/1857834460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 msgr2=0x561fc10f08a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10f08a0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f7b7801cad0 tx=0x7f7b780bf000 comp rx=0 tx=0).stop 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- 172.21.15.76:0/1857834460 shutdown_connections 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9ca0 0x561fc108bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9380 0x561fc108b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- 172.21.15.76:0/1857834460 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc10f08a0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- 172.21.15.76:0/1857834460 >> 172.21.15.76:0/1857834460 conn(0x561fc1086f30 msgr2=0x561fc10e6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- 172.21.15.76:0/1857834460 shutdown_connections 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- 172.21.15.76:0/1857834460 wait complete. 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- start start 2024-09-19T04:18:30.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 Processor -- start 2024-09-19T04:18:30.525 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc114cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9380 0x561fc114d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9ca0 0x561fc10edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561fc0edec40 con 0x561fc108d420 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561fc0edbae0 con 0x561fc10d9380 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561fc0ee39f0 con 0x561fc10d9ca0 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b874c1000 10 client.0 _ll_register_callbacks cb 0x7f7b780b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc114cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc114cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42778/0 (socket says 172.21.15.76:42778) 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/2489992293 learned_addr learned my addr 172.21.15.76:0/2489992293 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/2489992293 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9ca0 msgr2=0x561fc10edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9ca0 0x561fc10edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/2489992293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9380 msgr2=0x561fc114d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9380 0x561fc114d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 -- 172.21.15.76:0/2489992293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fc0f5e280 con 0x561fc108d420 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b821a1640 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc114cd90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7b7801d000 tx=0x7f7b780046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854da79f453b253d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.517+0000 7f7b5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/2489992293 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b780e2060 con 0x561fc108d420 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/2489992293 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f7b780c48b0 con 0x561fc108d420 2024-09-19T04:18:30.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b780c7070 con 0x561fc108d420 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/2489992293 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f7b780ade10 con 0x561fc108d420 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 10 client.7152 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x561fc0f357d0 con 0x561fc108d420 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/2489992293 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b780c7070 con 0x561fc108d420 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/2489992293 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7f7b780d2040 con 0x561fc108d420 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 10 client.7152 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 10 client.7152 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 20 client.7152 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10ee110 0x561fc10ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 4 client.7152 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-19T04:18:30.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x561fc0e76930 con 0x561fc10ee110 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b819a0640 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10ee110 0x561fc10ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 10 client.7152 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b819a0640 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10ee110 0x561fc10ee4f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e57005282508b30b server_cookie=a4ce9c68ddc565f2 in_seq=0 out_seq=0 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 1 -- 172.21.15.76:0/2489992293 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x561fc0e76930 con 0x561fc10ee110 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 10 client.7152 handle_command_reply: tid=0 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 client.7152 shutdown 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 msgr2=0x561fc114cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc114cd90 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f7b7801d000 tx=0x7f7b780046d0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 shutdown_connections 2024-09-19T04:18:30.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561fc10d9ca0 0x561fc10edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561fc10d9380 0x561fc114d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x561fc10ee110 0x561fc10ee4f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 --2- 172.21.15.76:0/2489992293 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561fc108d420 0x561fc114cd90 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b5d7fa640 0 client.7152 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 >> 172.21.15.76:0/2489992293 conn(0x561fc1086f30 msgr2=0x561fc10d7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 shutdown_connections 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 1 -- 172.21.15.76:0/2489992293 wait complete. 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.521+0000 7f7b874c1000 20 client.7152 trim_cache size 0 max 16384 2024-09-19T04:18:30.530 INFO:teuthology.orchestra.run.smithi076.stderr:Error EXDEV: Not rank 0 2024-09-19T04:18:30.531 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub resume 2024-09-19T04:18:30.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.633+0000 7fa8c456d640 1 Processor -- start 2024-09-19T04:18:30.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.633+0000 7fa8c456d640 1 -- start start 2024-09-19T04:18:30.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c456d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc072220 0x7fa8bc06da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c456d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc06e060 con 0x7fa8bc071730 2024-09-19T04:18:30.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c456d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc06e340 con 0x7fa8bc160480 2024-09-19T04:18:30.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c456d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc15efe0 con 0x7fa8bc072220 2024-09-19T04:18:30.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2309640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa8bc071730 0x7fa8bc071b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37402/0 (socket says 172.21.15.76:37402) 2024-09-19T04:18:30.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2309640 1 -- 172.21.15.76:0/3905539193 learned_addr learned my addr 172.21.15.76:0/3905539193 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc072220 0x7fa8bc06da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209391852 0 0) 0x7fa8bc06e060 con 0x7fa8bc071730 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa8a8003530 con 0x7fa8bc071730 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1245910285 0 0) 0x7fa8a8003530 con 0x7fa8bc071730 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc072220 msgr2=0x7fa8bc06da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc072220 0x7fa8bc06da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 >> v1:172.21.15.116:6789/0 conn(0x7fa8bc160480 legacy=0x7fa8bc06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc15e990 con 0x7fa8bc071730 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fa8b8001680 con 0x7fa8bc071730 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 >> v1:172.21.15.76:6789/0 conn(0x7fa8bc071730 legacy=0x7fa8bc071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80039f0 0x7fa8a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a8004810 0x7fa8a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a80051d0 0x7fa8a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc15efe0 con 0x7fa8a80051d0 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a800c0a0 con 0x7fa8a80039f0 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a800c2a0 con 0x7fa8a8004810 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc072220 0x7fa8bc06da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a80051d0 0x7fa8a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a8004810 0x7fa8a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2309640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80039f0 0x7fa8a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a8004810 msgr2=0x7fa8a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8a8004810 0x7fa8a8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80039f0 msgr2=0x7fa8a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80039f0 0x7fa8a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/3905539193 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a8003530 con 0x7fa8a80051d0 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a80051d0 0x7fa8a800b9d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc072600 tx=0x7fa8b401cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db5e5d62b3193a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40ab070 con 0x7fa8a80051d0 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b40a9070 con 0x7fa8a80051d0 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.637+0000 7fa8c1307640 1 -- 172.21.15.76:0/3905539193 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4009b50 con 0x7fa8a80051d0 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- 172.21.15.76:0/3905539193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a80051d0 msgr2=0x7fa8a800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 --2- 172.21.15.76:0/3905539193 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8a80051d0 0x7fa8a800b9d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc072600 tx=0x7fa8b401cd10 comp rx=0 tx=0).stop 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/3905539193 reap_dead start 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- 172.21.15.76:0/3905539193 shutdown_connections 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- 172.21.15.76:0/3905539193 >> 172.21.15.76:0/3905539193 conn(0x7fa8bc0b3e70 msgr2=0x7fa8bc0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- 172.21.15.76:0/3905539193 shutdown_connections 2024-09-19T04:18:30.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- 172.21.15.76:0/3905539193 wait complete. 2024-09-19T04:18:30.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 Processor -- start 2024-09-19T04:18:30.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- start start 2024-09-19T04:18:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80051d0 0x7fa8bc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc15dcf0 0x7fa8bc140740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8bc15d7a0 0x7fa8bc140c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc148080 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc147f00 con 0x7fa8a80051d0 2024-09-19T04:18:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc06da00 con 0x7fa8bc15dcf0 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8bc15d7a0 0x7fa8bc140c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8bc15d7a0 0x7fa8bc140c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42794/0 (socket says 172.21.15.76:42794) 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/249122165 learned_addr learned my addr 172.21.15.76:0/249122165 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc15dcf0 0x7fa8bc140740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/249122165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc15dcf0 msgr2=0x7fa8bc140740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2309640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80051d0 0x7fa8bc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa8bc15dcf0 0x7fa8bc140740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/249122165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80051d0 msgr2=0x7fa8bc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8a80051d0 0x7fa8bc15d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 -- 172.21.15.76:0/249122165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc15d580 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c2b0a640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa8bc15d7a0 0x7fa8bc140c80 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa8b401d5b0 tx=0x7fa8b4019ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8b2ffd640 1 -- 172.21.15.76:0/249122165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b40ab070 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8b2ffd640 1 -- 172.21.15.76:0/249122165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b4004250 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8b2ffd640 1 -- 172.21.15.76:0/249122165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4009b50 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- 172.21.15.76:0/249122165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc144400 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.641+0000 7fa8c456d640 1 -- 172.21.15.76:0/249122165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc1449c0 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8b2ffd640 1 -- 172.21.15.76:0/249122165 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa8b4004640 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8b2ffd640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8a00678c0 0x7fa8a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8b2ffd640 1 -- 172.21.15.76:0/249122165 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa8b411fbb0 con 0x7fa8bc15d7a0 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8a00678c0 0x7fa8a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 --2- 172.21.15.76:0/249122165 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa8a00678c0 0x7fa8a0069d80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8001060 tx=0x7fa8b8001160 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 Processor -- start 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- start start 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a9361ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 0x5654a9315240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 0x5654a9315b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654a9166cc0 con 0x5654a9361c10 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5654a9121a20 con 0x5654a9362530 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5654a9162c80 con 0x5654a9315780 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a9361ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a9361ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42796/0 (socket says 172.21.15.76:42796) 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 -- 172.21.15.76:0/1918863510 learned_addr learned my addr 172.21.15.76:0/1918863510 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 -- 172.21.15.76:0/1918863510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 msgr2=0x5654a9315b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 0x5654a9315b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 -- 172.21.15.76:0/1918863510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 msgr2=0x5654a9315240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 0x5654a9315240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 0x5654a9315240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 -- 172.21.15.76:0/1918863510 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a9109d40 con 0x5654a9361c10 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a9361ff0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800f8f0 tx=0x7fa8b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a728bdd42b0ad4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa89ffff640 1 -- 172.21.15.76:0/1918863510 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8010040 con 0x5654a9361c10 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa89ffff640 1 -- 172.21.15.76:0/1918863510 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b800fe70 con 0x5654a9361c10 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa89ffff640 1 -- 172.21.15.76:0/1918863510 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8016bd0 con 0x5654a9361c10 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- 172.21.15.76:0/1918863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 msgr2=0x5654a9361ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a9361ff0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800f8f0 tx=0x7fa8b8012000 comp rx=0 tx=0).stop 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- 172.21.15.76:0/1918863510 shutdown_connections 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 0x5654a9315b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 0x5654a9315240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1918863510 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a9361ff0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- 172.21.15.76:0/1918863510 >> 172.21.15.76:0/1918863510 conn(0x5654a93429b0 msgr2=0x5654a9344da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- 172.21.15.76:0/1918863510 shutdown_connections 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.645+0000 7fa8c7629000 1 -- 172.21.15.76:0/1918863510 wait complete. 2024-09-19T04:18:30.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- start start 2024-09-19T04:18:30.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 Processor -- start 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a931c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 0x5654a931c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 0x5654a931cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654a91002c0 con 0x5654a9315780 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5654a9125370 con 0x5654a9361c10 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5654a9174f50 con 0x5654a9362530 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 10 client.0 _ll_register_callbacks cb 0x5654a91c7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a931c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a931c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42806/0 (socket says 172.21.15.76:42806) 2024-09-19T04:18:30.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 -- 172.21.15.76:0/356029471 learned_addr learned my addr 172.21.15.76:0/356029471 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 -- 172.21.15.76:0/356029471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 msgr2=0x5654a931cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 0x5654a931cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 -- 172.21.15.76:0/356029471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 msgr2=0x5654a931c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 0x5654a931c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 -- 172.21.15.76:0/356029471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a9106770 con 0x5654a9315780 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c2309640 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a931c040 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x5654a9126470 tx=0x7fa8bc06e230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dab5a83431345e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 1 -- 172.21.15.76:0/356029471 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8022660 con 0x5654a9315780 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 1 -- 172.21.15.76:0/356029471 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa8b802dc60 con 0x5654a9315780 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8b800fe70 con 0x5654a9315780 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 1 -- 172.21.15.76:0/356029471 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa8b8034050 con 0x5654a9315780 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 10 client.7164 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5654a91243f0 con 0x5654a9315780 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 1 -- 172.21.15.76:0/356029471 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b800fe70 con 0x5654a9315780 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 1 -- 172.21.15.76:0/356029471 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7fa8b8037020 con 0x5654a9315780 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 10 client.7164 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 10 client.7164 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 20 client.7164 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a9371cb0 0x5654a93740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 4 client.7164 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654a9100d90 con 0x5654a9371cb0 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a9371cb0 0x5654a93740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a9371cb0 0x5654a93740a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6389c46c45587e9 server_cookie=425271395bc6301a in_seq=0 out_seq=0 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 10 client.7164 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 1 -- 172.21.15.76:0/356029471 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5654a9100d90 con 0x5654a9371cb0 2024-09-19T04:18:30.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 10 client.7164 handle_command_reply: tid=0 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 client.7164 shutdown 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 msgr2=0x5654a931c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a931c040 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x5654a9126470 tx=0x7fa8bc06e230 comp rx=0 tx=0).stop 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 shutdown_connections 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 0x5654a931cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 0x5654a931c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a9371cb0 0x5654a93740a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 --2- 172.21.15.76:0/356029471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a931c040 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa89d7fa640 0 client.7164 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 >> 172.21.15.76:0/356029471 conn(0x5654a93429b0 msgr2=0x5654a9314940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 shutdown_connections 2024-09-19T04:18:30.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 1 -- 172.21.15.76:0/356029471 wait complete. 2024-09-19T04:18:30.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.649+0000 7fa8c7629000 20 client.7164 trim_cache size 0 max 16384 2024-09-19T04:18:30.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.665+0000 7fa8c7629000 1 Processor -- start 2024-09-19T04:18:30.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.665+0000 7fa8c7629000 1 -- start start 2024-09-19T04:18:30.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a93d0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 0x5654a93d0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 0x5654a93d1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc200870 con 0x5654a9315780 2024-09-19T04:18:30.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc200f70 con 0x5654a9361c10 2024-09-19T04:18:30.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc1ff9e0 con 0x5654a9362530 2024-09-19T04:18:30.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a93d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a93d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42818/0 (socket says 172.21.15.76:42818) 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 -- 172.21.15.76:0/1362946876 learned_addr learned my addr 172.21.15.76:0/1362946876 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 -- 172.21.15.76:0/1362946876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 msgr2=0x5654a93d1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1362946876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 0x5654a93d1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 -- 172.21.15.76:0/1362946876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 msgr2=0x5654a93d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1362946876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 0x5654a93d0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 -- 172.21.15.76:0/1362946876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654a91e8320 con 0x5654a9315780 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1362946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a93d0990 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa8b802efc0 tx=0x7fa8b8027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f189e8f1643b2b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa89f7fe640 1 -- 172.21.15.76:0/1362946876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8016cb0 con 0x5654a9315780 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa89f7fe640 1 -- 172.21.15.76:0/1362946876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8b8022660 con 0x5654a9315780 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa89f7fe640 1 -- 172.21.15.76:0/1362946876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8022af0 con 0x5654a9315780 2024-09-19T04:18:30.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- 172.21.15.76:0/1362946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 msgr2=0x5654a93d0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1362946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a93d0990 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fa8b802efc0 tx=0x7fa8b8027000 comp rx=0 tx=0).stop 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- 172.21.15.76:0/1362946876 shutdown_connections 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1362946876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9362530 0x5654a93d1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1362946876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9361c10 0x5654a93d0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1362946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9315780 0x5654a93d0990 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- 172.21.15.76:0/1362946876 >> 172.21.15.76:0/1362946876 conn(0x5654a93429b0 msgr2=0x5654a9313a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- 172.21.15.76:0/1362946876 shutdown_connections 2024-09-19T04:18:30.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- 172.21.15.76:0/1362946876 wait complete. 2024-09-19T04:18:30.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- start start 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 Processor -- start 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 0x5654a93d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a93d7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 0x5654a93da3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5654a910e120 con 0x5654a9361c10 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5654a91e7e90 con 0x5654a9362530 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5654a9110b90 con 0x5654a9315780 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 10 client.0 _ll_register_callbacks cb 0x7fa8b800f7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a93d7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a93d7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42826/0 (socket says 172.21.15.76:42826) 2024-09-19T04:18:30.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 -- 172.21.15.76:0/1806167049 learned_addr learned my addr 172.21.15.76:0/1806167049 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 -- 172.21.15.76:0/1806167049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 msgr2=0x5654a93d2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 0x5654a93d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 -- 172.21.15.76:0/1806167049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 msgr2=0x5654a93da3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 0x5654a93da3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 -- 172.21.15.76:0/1806167049 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654a90fa5c0 con 0x5654a9361c10 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c1b08640 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a93d7e70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac001540 tx=0x7fa8ac001d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05544eabaed410f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa89d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa89d7fa640 1 -- 172.21.15.76:0/1806167049 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac023060 con 0x5654a9361c10 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5654a91743f0 con 0x5654a9361c10 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa89d7fa640 1 -- 172.21.15.76:0/1806167049 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa8ac0159e0 con 0x5654a9361c10 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa89d7fa640 1 -- 172.21.15.76:0/1806167049 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fa8ac029050 con 0x5654a9361c10 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.669+0000 7fa8c7629000 10 client.7170 fetch_fsmap learned FSMap version 74 2024-09-19T04:18:30.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x5654a90fe330 con 0x5654a9361c10 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa89d7fa640 1 -- 172.21.15.76:0/1806167049 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5654a91743f0 con 0x5654a9361c10 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa89d7fa640 1 -- 172.21.15.76:0/1806167049 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 74) ==== 3554+0+0 (secure 0 0 0) 0x7fa8ac02c020 con 0x5654a9361c10 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 10 client.7170 fetch_fsmap finished waiting for FSMap version 74 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 10 client.7170 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 20 client.7170 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a93dab10 0x5654a93dcf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 4 client.7170 mds_command: new command op to 6780 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 --> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x5654a916e740 con 0x5654a93dab10 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a93dab10 0x5654a93dcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa89d7fa640 10 client.7170 ms_handle_connect on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c2309640 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a93dab10 0x5654a93dcf00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d9b4f18d05ec698f server_cookie=186f3bc17bbba526 in_seq=0 out_seq=0 2024-09-19T04:18:30.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa89d7fa640 1 -- 172.21.15.76:0/1806167049 <== mds.1 v2:172.21.15.76:6834/603375312 1 ==== command_reply(tid 0: -18 Not rank 0) ==== 18+0+0 (crc 0 0 0) 0x5654a916e740 con 0x5654a93dab10 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa89d7fa640 10 client.7170 handle_command_reply: tid=0 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 client.7170 shutdown 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 msgr2=0x5654a93d7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a93d7e70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac001540 tx=0x7fa8ac001d90 comp rx=0 tx=0).stop 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 shutdown_connections 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5654a9362530 0x5654a93da3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:6834/603375312,v1:172.21.15.76:6835/603375312] conn(0x5654a93dab10 0x5654a93dcf00 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5654a9361c10 0x5654a93d7e70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 --2- 172.21.15.76:0/1806167049 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5654a9315780 0x5654a93d2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa89d7fa640 0 client.7170 ms_handle_reset on v2:172.21.15.76:6834/603375312 2024-09-19T04:18:30.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 >> 172.21.15.76:0/1806167049 conn(0x5654a93429b0 msgr2=0x5654a9314230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:30.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 shutdown_connections 2024-09-19T04:18:30.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 1 -- 172.21.15.76:0/1806167049 wait complete. 2024-09-19T04:18:30.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.673+0000 7fa8c7629000 20 client.7170 trim_cache size 0 max 16384 2024-09-19T04:18:30.680 INFO:teuthology.orchestra.run.smithi076.stderr:Error EXDEV: Not rank 0 2024-09-19T04:18:30.680 DEBUG:teuthology.orchestra.run:got remote process result: 18 2024-09-19T04:18:30.681 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:18:30.681 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:18:30.742 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:18:30.829 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:18:30.829 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:18:30.830 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:18:30.830 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:30.835 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:18:30.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90e0005640 1 Processor -- start 2024-09-19T04:18:30.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90e0005640 1 -- start start 2024-09-19T04:18:31.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90e0005640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90d815f3a0 0x7f90d8165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90e0005640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d80aede0 con 0x7f90d81520a0 2024-09-19T04:18:31.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90e0005640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90d815fa10 con 0x7f90d8070020 2024-09-19T04:18:31.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90e0005640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f90d8152480 con 0x7f90d815f3a0 2024-09-19T04:18:31.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90ddda1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f90d81520a0 0x7f90d806f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37418/0 (socket says 172.21.15.76:37418) 2024-09-19T04:18:31.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90ddda1640 1 -- 172.21.15.76:0/3920962646 learned_addr learned my addr 172.21.15.76:0/3920962646 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90de5a2640 1 --2- 172.21.15.76:0/3920962646 >> v2:172.21.15.116:3300/0 conn(0x7f90d815f3a0 0x7f90d8165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 595321015 0 0) 0x7f90d80aede0 con 0x7f90d81520a0 2024-09-19T04:18:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:30.997+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f90c4003530 con 0x7f90d81520a0 2024-09-19T04:18:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1709529315 0 0) 0x7f90c4003530 con 0x7f90d81520a0 2024-09-19T04:18:31.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 >> v2:172.21.15.116:3300/0 conn(0x7f90d815f3a0 msgr2=0x7f90d8165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 --2- 172.21.15.76:0/3920962646 >> v2:172.21.15.116:3300/0 conn(0x7f90d815f3a0 0x7f90d8165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 >> v1:172.21.15.116:6789/0 conn(0x7f90d8070020 legacy=0x7f90d815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d8074250 con 0x7f90d81520a0 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f90d4001a00 con 0x7f90d81520a0 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 >> v1:172.21.15.76:6789/0 conn(0x7f90d81520a0 legacy=0x7f90d806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c4003a10 0x7f90c4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4004810 0x7f90c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90c40051d0 0x7f90c400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f90c400c0a0 con 0x7f90c4003a10 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90c400c250 con 0x7f90c4004810 2024-09-19T04:18:31.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f90c400c450 con 0x7f90c40051d0 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90ddda1640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c4003a10 0x7f90c4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dd5a0640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4004810 0x7f90c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90ddda1640 1 -- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90c40051d0 msgr2=0x7f90c400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90ddda1640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90c40051d0 0x7f90c400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90ddda1640 1 -- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4004810 msgr2=0x7f90c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90ddda1640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4004810 0x7f90c4004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90de5a2640 1 -- 172.21.15.76:0/3920962646 reap_dead start 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90ddda1640 1 -- 172.21.15.76:0/3920962646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90c4003530 con 0x7f90c4003a10 2024-09-19T04:18:31.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dd5a0640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4004810 0x7f90c4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90ddda1640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c4003a10 0x7f90c4003e30 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f90d400a790 tx=0x7f90d401d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c320985a17d9b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d400a3b0 con 0x7f90c4003a10 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90d40ad650 con 0x7f90c4003a10 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.001+0000 7f90dcd9f640 1 -- 172.21.15.76:0/3920962646 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d40ada80 con 0x7f90c4003a10 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/3920962646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c4003a10 msgr2=0x7f90c4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c4003a10 0x7f90c4003e30 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f90d400a790 tx=0x7f90d401d280 comp rx=0 tx=0).stop 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/3920962646 shutdown_connections 2024-09-19T04:18:31.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 --2- 172.21.15.76:0/3920962646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90c4003a10 0x7f90c4003e30 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/3920962646 >> 172.21.15.76:0/3920962646 conn(0x7f90d80b1f90 msgr2=0x7f90d80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/3920962646 shutdown_connections 2024-09-19T04:18:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/3920962646 wait complete. 2024-09-19T04:18:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 Processor -- start 2024-09-19T04:18:31.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- start start 2024-09-19T04:18:31.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4003a10 0x7f90d81f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d814b020 0x7f90d815cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90d814b4a0 0x7f90d815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8073d30 con 0x7f90d814b020 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90d8073a50 con 0x7f90c4003a10 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f90d815f0a0 con 0x7f90d814b4a0 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d814b020 0x7f90d815cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d814b020 0x7f90d815cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42842/0 (socket says 172.21.15.76:42842) 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 -- 172.21.15.76:0/2483098310 learned_addr learned my addr 172.21.15.76:0/2483098310 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90de5a2640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90d814b4a0 0x7f90d815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90ddda1640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4003a10 0x7f90d81f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 -- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90d814b4a0 msgr2=0x7f90d815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90d814b4a0 0x7f90d815d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 -- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4003a10 msgr2=0x7f90d81f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4003a10 0x7f90d81f3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 -- 172.21.15.76:0/2483098310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90d8153220 con 0x7f90d814b020 2024-09-19T04:18:31.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90dd5a0640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d814b020 0x7f90d815cc70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f90c800d4a0 tx=0x7f90c800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90ce7fc640 1 -- 172.21.15.76:0/2483098310 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8019070 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90ce7fc640 1 -- 172.21.15.76:0/2483098310 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90c8005480 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90d8153ee0 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90ce7fc640 1 -- 172.21.15.76:0/2483098310 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90c8007250 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90d81f3f40 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.005+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90d81ed4d0 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.009+0000 7f90ce7fc640 1 -- 172.21.15.76:0/2483098310 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f90c801c070 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.009+0000 7f90ce7fc640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f90bc0678f0 0x7f90bc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.009+0000 7f90ce7fc640 1 -- 172.21.15.76:0/2483098310 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f90c8085470 con 0x7f90d814b020 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.009+0000 7f90ddda1640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f90bc0678f0 0x7f90bc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.009+0000 7f90ddda1640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f90bc0678f0 0x7f90bc069db0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f90d40ad000 tx=0x7f90d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.013+0000 7f90ce7fc640 1 -- 172.21.15.76:0/2483098310 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f90d81ed4d0 con 0x7f90d814b020 2024-09-19T04:18:31.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.173+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f90d8068e50 con 0x7f90d814b020 2024-09-19T04:18:31.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90ce7fc640 1 -- 172.21.15.76:0/2483098310 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v80) ==== 80+0+0 (secure 0 0 0) 0x7f90c8055eb0 con 0x7f90d814b020 2024-09-19T04:18:31.177 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:18:31.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f90bc0678f0 msgr2=0x7f90bc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f90bc0678f0 0x7f90bc069db0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f90d40ad000 tx=0x7f90d4009040 comp rx=0 tx=0).stop 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d814b020 msgr2=0x7f90d815cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d814b020 0x7f90d815cc70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f90c800d4a0 tx=0x7f90c800d960 comp rx=0 tx=0).stop 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 shutdown_connections 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f90bc0678f0 0x7f90bc069db0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f90d814b4a0 0x7f90d815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90d814b020 0x7f90d815cc70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 --2- 172.21.15.76:0/2483098310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90c4003a10 0x7f90d81f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 >> 172.21.15.76:0/2483098310 conn(0x7f90d80b1f90 msgr2=0x7f90d806e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 shutdown_connections 2024-09-19T04:18:31.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.177+0000 7f90e0005640 1 -- 172.21.15.76:0/2483098310 wait complete. 2024-09-19T04:18:31.195 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:18:31.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:31.195 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:31.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[44668]: fuse finished with error 0 and tester_r 0 2024-09-19T04:18:31.207 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:18:31.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:31.207 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:18:31.296 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:18:31.296 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:18:31.297 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:31.303 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:18:31.303 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:18:31.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:31.303 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T04:18:31.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:18:31.350 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T04:18:31.351 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:18:31.351 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:18:31.351 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:31.356 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:18:31.356 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:18:31.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:31.356 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T04:18:31.405 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T04:18:31.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:18:31.406 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:18:31.406 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:18:31.406 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:31.412 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:18:31.412 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:18:31.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:31.412 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T04:18:31.458 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T04:18:31.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:18:31.459 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:31.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.581+0000 7f1b3f58d640 1 Processor -- start 2024-09-19T04:18:31.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.581+0000 7f1b3f58d640 1 -- start start 2024-09-19T04:18:31.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3f58d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1b38070020 0x7f1b3815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3f58d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3f58d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b380aede0 con 0x7f1b381520a0 2024-09-19T04:18:31.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3f58d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1b38152480 con 0x7f1b38070020 2024-09-19T04:18:31.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3f58d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1b3815fa10 con 0x7f1b3815f1d0 2024-09-19T04:18:31.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3cb28640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1b38070020 0x7f1b3815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3db2a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3db2a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:55600/0 (socket says 172.21.15.76:55600) 2024-09-19T04:18:31.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3d329640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1b381520a0 0x7f1b3806f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:56108/0 (socket says 172.21.15.76:56108) 2024-09-19T04:18:31.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3db2a640 1 -- 172.21.15.76:0/2125409657 learned_addr learned my addr 172.21.15.76:0/2125409657 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:31.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3db2a640 1 -- 172.21.15.76:0/2125409657 >> v2:172.21.15.116:3300/0 conn(0x7f1b38070020 msgr2=0x7f1b3815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:31.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3db2a640 1 --2- 172.21.15.76:0/2125409657 >> v2:172.21.15.116:3300/0 conn(0x7f1b38070020 0x7f1b3815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3db2a640 1 -- 172.21.15.76:0/2125409657 >> v1:172.21.15.116:6789/0 conn(0x7f1b381520a0 legacy=0x7f1b3806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3db2a640 1 -- 172.21.15.76:0/2125409657 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b38074250 con 0x7f1b3815f1d0 2024-09-19T04:18:31.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.585+0000 7f1b3cb28640 1 --2- 172.21.15.76:0/2125409657 >> v2:172.21.15.116:3300/0 conn(0x7f1b38070020 0x7f1b3815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:31.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3db2a640 1 --2- 172.21.15.76:0/2125409657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380adc80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009e50 tx=0x7f1b3001ca00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=977d5099ea7a8173 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b2ffff640 1 -- 172.21.15.76:0/2125409657 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300a58d0 con 0x7f1b3815f1d0 2024-09-19T04:18:31.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b2ffff640 1 -- 172.21.15.76:0/2125409657 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b300a5a70 con 0x7f1b3815f1d0 2024-09-19T04:18:31.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b2ffff640 1 -- 172.21.15.76:0/2125409657 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b300a5e40 con 0x7f1b3815f1d0 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- 172.21.15.76:0/2125409657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 msgr2=0x7f1b380adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/2125409657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380adc80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009e50 tx=0x7f1b3001ca00 comp rx=0 tx=0).stop 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- 172.21.15.76:0/2125409657 shutdown_connections 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/2125409657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380adc80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/2125409657 >> v2:172.21.15.116:3300/0 conn(0x7f1b38070020 0x7f1b3815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- 172.21.15.76:0/2125409657 >> 172.21.15.76:0/2125409657 conn(0x7f1b380b1f90 msgr2=0x7f1b380b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- 172.21.15.76:0/2125409657 shutdown_connections 2024-09-19T04:18:31.590 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- 172.21.15.76:0/2125409657 wait complete. 2024-09-19T04:18:31.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 Processor -- start 2024-09-19T04:18:31.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- start start 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b38070020 0x7f1b3814b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b381520a0 0x7f1b38146940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3815f680 con 0x7f1b381520a0 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1b38073d30 con 0x7f1b38070020 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3f58d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1b38167f10 con 0x7f1b3815f1d0 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b381520a0 0x7f1b38146940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b381520a0 0x7f1b38146940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42850/0 (socket says 172.21.15.76:42850) 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 -- 172.21.15.76:0/729801341 learned_addr learned my addr 172.21.15.76:0/729801341 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3db2a640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3d329640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b38070020 0x7f1b3814b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 -- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 msgr2=0x7f1b380d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380d02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 -- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b38070020 msgr2=0x7f1b3814b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b38070020 0x7f1b3814b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.589+0000 7f1b3cb28640 1 -- 172.21.15.76:0/729801341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3815a9d0 con 0x7f1b381520a0 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b3cb28640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b381520a0 0x7f1b38146940 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e930 tx=0x7f1b2800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b2dffb640 1 -- 172.21.15.76:0/729801341 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b28017070 con 0x7f1b381520a0 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b2dffb640 1 -- 172.21.15.76:0/729801341 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b2800c9a0 con 0x7f1b381520a0 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3815acb0 con 0x7f1b381520a0 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b2dffb640 1 -- 172.21.15.76:0/729801341 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b2800cd70 con 0x7f1b381520a0 2024-09-19T04:18:31.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3814ade0 con 0x7f1b381520a0 2024-09-19T04:18:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3806fe00 con 0x7f1b381520a0 2024-09-19T04:18:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b2dffb640 1 -- 172.21.15.76:0/729801341 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1b2801a070 con 0x7f1b381520a0 2024-09-19T04:18:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b2dffb640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b1c0678c0 0x7f1b1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b2dffb640 1 -- 172.21.15.76:0/729801341 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f1b28085f60 con 0x7f1b381520a0 2024-09-19T04:18:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b3d329640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b1c0678c0 0x7f1b1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.593+0000 7f1b3d329640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b1c0678c0 0x7f1b1c069d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b34000f70 tx=0x7f1b34007c90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.597+0000 7f1b2dffb640 1 -- 172.21.15.76:0/729801341 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1b280569a0 con 0x7f1b381520a0 2024-09-19T04:18:31.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.789+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b3806e540 con 0x7f1b381520a0 2024-09-19T04:18:31.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b2dffb640 1 -- 172.21.15.76:0/729801341 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5267 (secure 0 0 0) 0x7f1b280562a0 con 0x7f1b381520a0 2024-09-19T04:18:31.793 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:31.794 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"btime":"2024-09-19T04:18:27:920425+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:27.400688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6783,"mds_1":6780,"mds_2":5336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5336":{"gid":5336,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[5336,6780,6783]},"id":5}]} 2024-09-19T04:18:31.794 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 74 2024-09-19T04:18:31.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b1c0678c0 msgr2=0x7f1b1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b1c0678c0 0x7f1b1c069d80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1b34000f70 tx=0x7f1b34007c90 comp rx=0 tx=0).stop 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b381520a0 msgr2=0x7f1b38146940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b381520a0 0x7f1b38146940 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800e930 tx=0x7f1b2800edf0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 shutdown_connections 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b1c0678c0 0x7f1b1c069d80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1b3815f1d0 0x7f1b380d02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1b381520a0 0x7f1b38146940 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 --2- 172.21.15.76:0/729801341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b38070020 0x7f1b3814b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 >> 172.21.15.76:0/729801341 conn(0x7f1b380b1f90 msgr2=0x7f1b380b41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 shutdown_connections 2024-09-19T04:18:31.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.793+0000 7f1b3f58d640 1 -- 172.21.15.76:0/729801341 wait complete. 2024-09-19T04:18:31.812 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:31.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f32180de640 1 Processor -- start 2024-09-19T04:18:31.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f32180de640 1 -- start start 2024-09-19T04:18:31.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f32180de640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f321015c2a0 0x7f32100aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f32180de640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32100b3290 con 0x7f321015bec0 2024-09-19T04:18:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f32180de640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32100b3570 con 0x7f32100af2a0 2024-09-19T04:18:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f32180de640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f32100b3850 con 0x7f321015c2a0 2024-09-19T04:18:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f3215e7a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f321015bec0 0x7f32100ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:37422/0 (socket says 172.21.15.76:37422) 2024-09-19T04:18:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f3215e7a640 1 -- 172.21.15.76:0/4233281317 learned_addr learned my addr 172.21.15.76:0/4233281317 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f3215679640 1 --2- 172.21.15.76:0/4233281317 >> v2:172.21.15.116:3300/0 conn(0x7f321015c2a0 0x7f32100aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f3215679640 1 -- 172.21.15.76:0/4233281317 >> v1:172.21.15.116:6789/0 conn(0x7f32100af2a0 legacy=0x7f321006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f3215679640 1 -- 172.21.15.76:0/4233281317 >> v1:172.21.15.76:6789/0 conn(0x7f321015bec0 legacy=0x7f32100ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.933+0000 7f3215679640 1 -- 172.21.15.76:0/4233281317 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32100b3b30 con 0x7f321015c2a0 2024-09-19T04:18:31.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f3215679640 1 --2- 172.21.15.76:0/4233281317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015c2a0 0x7f32100aed60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097c0 tx=0x7f320c01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6004e25c41402d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f3214e78640 1 -- 172.21.15.76:0/4233281317 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0a7070 con 0x7f321015c2a0 2024-09-19T04:18:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f3214e78640 1 -- 172.21.15.76:0/4233281317 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f320c0b0070 con 0x7f321015c2a0 2024-09-19T04:18:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f3214e78640 1 -- 172.21.15.76:0/4233281317 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320c0ab890 con 0x7f321015c2a0 2024-09-19T04:18:31.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f32180de640 1 -- 172.21.15.76:0/4233281317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015c2a0 msgr2=0x7f32100aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f32180de640 1 --2- 172.21.15.76:0/4233281317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015c2a0 0x7f32100aed60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f320c0097c0 tx=0x7f320c01c470 comp rx=0 tx=0).stop 2024-09-19T04:18:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f32180de640 1 -- 172.21.15.76:0/4233281317 shutdown_connections 2024-09-19T04:18:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f32180de640 1 --2- 172.21.15.76:0/4233281317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015c2a0 0x7f32100aed60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f32180de640 1 -- 172.21.15.76:0/4233281317 >> 172.21.15.76:0/4233281317 conn(0x7f32100b1f90 msgr2=0x7f32100b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f32180de640 1 -- 172.21.15.76:0/4233281317 shutdown_connections 2024-09-19T04:18:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.937+0000 7f32180de640 1 -- 172.21.15.76:0/4233281317 wait complete. 2024-09-19T04:18:31.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 Processor -- start 2024-09-19T04:18:31.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 -- start start 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32100af2a0 0x7f321015ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015bec0 0x7f32101547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321015c2a0 0x7f3210154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f321006ef20 con 0x7f321015c2a0 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f32100af6b0 con 0x7f321015bec0 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f321006f0a0 con 0x7f32100af2a0 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321015c2a0 0x7f3210154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321015c2a0 0x7f3210154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42866/0 (socket says 172.21.15.76:42866) 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 -- 172.21.15.76:0/4128244379 learned_addr learned my addr 172.21.15.76:0/4128244379 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f3215e7a640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32100af2a0 0x7f321015ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f3215679640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015bec0 0x7f32101547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 -- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32100af2a0 msgr2=0x7f321015ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32100af2a0 0x7f321015ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 -- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015bec0 msgr2=0x7f32101547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:31.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015bec0 0x7f32101547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:31.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 -- 172.21.15.76:0/4128244379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f321015f180 con 0x7f321015c2a0 2024-09-19T04:18:31.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f321667b640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321015c2a0 0x7f3210154d10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f320800c710 tx=0x7f320800cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32067fc640 1 -- 172.21.15.76:0/4128244379 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208016020 con 0x7f321015c2a0 2024-09-19T04:18:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32067fc640 1 -- 172.21.15.76:0/4128244379 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3208006d60 con 0x7f321015c2a0 2024-09-19T04:18:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3210152d40 con 0x7f321015c2a0 2024-09-19T04:18:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32067fc640 1 -- 172.21.15.76:0/4128244379 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3208003bc0 con 0x7f321015c2a0 2024-09-19T04:18:31.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.941+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32101532c0 con 0x7f321015c2a0 2024-09-19T04:18:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.945+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32100aed60 con 0x7f321015c2a0 2024-09-19T04:18:31.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.945+0000 7f32067fc640 1 -- 172.21.15.76:0/4128244379 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f320800a850 con 0x7f321015c2a0 2024-09-19T04:18:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.945+0000 7f32067fc640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31f40678c0 0x7f31f4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.945+0000 7f3215e7a640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31f40678c0 0x7f31f4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.945+0000 7f32067fc640 1 -- 172.21.15.76:0/4128244379 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f3208085910 con 0x7f321015c2a0 2024-09-19T04:18:31.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.945+0000 7f3215e7a640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31f40678c0 0x7f31f4069d80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3210167db0 tx=0x7f3200007e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:31.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:31.949+0000 7f32067fc640 1 -- 172.21.15.76:0/4128244379 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3208020020 con 0x7f321015c2a0 2024-09-19T04:18:32.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.141+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32100753b0 con 0x7f321015c2a0 2024-09-19T04:18:32.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.141+0000 7f32067fc640 1 -- 172.21.15.76:0/4128244379 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5267 (secure 0 0 0) 0x7f320800ce20 con 0x7f321015c2a0 2024-09-19T04:18:32.144 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:32.144 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"btime":"2024-09-19T04:18:27:920425+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:27.400688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6783,"mds_1":6780,"mds_2":5336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5336":{"gid":5336,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[5336,6780,6783]},"id":5}]} 2024-09-19T04:18:32.144 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 74 2024-09-19T04:18:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31f40678c0 msgr2=0x7f31f4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31f40678c0 0x7f31f4069d80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3210167db0 tx=0x7f3200007e90 comp rx=0 tx=0).stop 2024-09-19T04:18:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321015c2a0 msgr2=0x7f3210154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321015c2a0 0x7f3210154d10 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f320800c710 tx=0x7f320800cbd0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 shutdown_connections 2024-09-19T04:18:32.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f31f40678c0 0x7f31f4069d80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f321015c2a0 0x7f3210154d10 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f321015bec0 0x7f32101547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 --2- 172.21.15.76:0/4128244379 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f32100af2a0 0x7f321015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 >> 172.21.15.76:0/4128244379 conn(0x7f32100b1f90 msgr2=0x7f32100cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:32.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 shutdown_connections 2024-09-19T04:18:32.147 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.145+0000 7f32180de640 1 -- 172.21.15.76:0/4128244379 wait complete. 2024-09-19T04:18:32.162 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:32.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a825f7640 1 Processor -- start 2024-09-19T04:18:32.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a825f7640 1 -- start start 2024-09-19T04:18:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a825f7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a7c070020 0x7f4a7c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a825f7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a7c15f1d0 0x7f4a7c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a825f7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a7c0aede0 con 0x7f4a7c1520a0 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a825f7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7c152480 con 0x7f4a7c15f1d0 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a825f7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4a7c15fa10 con 0x7f4a7c070020 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a7c15f1d0 0x7f4a7c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4a7c15f1d0 0x7f4a7c0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42880/0 (socket says 172.21.15.76:42880) 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 -- 172.21.15.76:0/2722754863 learned_addr learned my addr 172.21.15.76:0/2722754863 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a7b7fe640 1 --2- 172.21.15.76:0/2722754863 >> v2:172.21.15.116:3300/0 conn(0x7f4a7c070020 0x7f4a7c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 -- 172.21.15.76:0/2722754863 >> v2:172.21.15.116:3300/0 conn(0x7f4a7c070020 msgr2=0x7f4a7c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 --2- 172.21.15.76:0/2722754863 >> v2:172.21.15.116:3300/0 conn(0x7f4a7c070020 0x7f4a7c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 -- 172.21.15.76:0/2722754863 >> v1:172.21.15.116:6789/0 conn(0x7f4a7c1520a0 legacy=0x7f4a7c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:32.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 -- 172.21.15.76:0/2722754863 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c074250 con 0x7f4a7c15f1d0 2024-09-19T04:18:32.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.277+0000 7f4a80b94640 1 --2- 172.21.15.76:0/2722754863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c15f1d0 0x7f4a7c0adc80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c06e510 tx=0x7f4a7401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed0691c4e08bb78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7affd640 1 -- 172.21.15.76:0/2722754863 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a740aa610 con 0x7f4a7c15f1d0 2024-09-19T04:18:32.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7affd640 1 -- 172.21.15.76:0/2722754863 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a740af070 con 0x7f4a7c15f1d0 2024-09-19T04:18:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7affd640 1 -- 172.21.15.76:0/2722754863 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a740aa9d0 con 0x7f4a7c15f1d0 2024-09-19T04:18:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- 172.21.15.76:0/2722754863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c15f1d0 msgr2=0x7f4a7c0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2722754863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c15f1d0 0x7f4a7c0adc80 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c06e510 tx=0x7f4a7401c4b0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- 172.21.15.76:0/2722754863 shutdown_connections 2024-09-19T04:18:32.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2722754863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c15f1d0 0x7f4a7c0adc80 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2722754863 >> v2:172.21.15.116:3300/0 conn(0x7f4a7c070020 0x7f4a7c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- 172.21.15.76:0/2722754863 >> 172.21.15.76:0/2722754863 conn(0x7f4a7c0b1f90 msgr2=0x7f4a7c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- 172.21.15.76:0/2722754863 shutdown_connections 2024-09-19T04:18:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- 172.21.15.76:0/2722754863 wait complete. 2024-09-19T04:18:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 Processor -- start 2024-09-19T04:18:32.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- start start 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c070020 0x7f4a7c06f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a7c1520a0 0x7f4a7c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a7c15f1d0 0x7f4a7c15dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c15f680 con 0x7f4a7c070020 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7c073d30 con 0x7f4a7c15f1d0 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7c167f10 con 0x7f4a7c1520a0 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c070020 0x7f4a7c06f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c070020 0x7f4a7c06f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42892/0 (socket says 172.21.15.76:42892) 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 -- 172.21.15.76:0/2632753318 learned_addr learned my addr 172.21.15.76:0/2632753318 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7b7fe640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a7c1520a0 0x7f4a7c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a80b94640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a7c15f1d0 0x7f4a7c15dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 -- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a7c1520a0 msgr2=0x7f4a7c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a7c1520a0 0x7f4a7c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 -- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a7c15f1d0 msgr2=0x7f4a7c15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a7c15f1d0 0x7f4a7c15dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 -- 172.21.15.76:0/2632753318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7c09a810 con 0x7f4a7c070020 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a7bfff640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c070020 0x7f4a7c06f210 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00c7f0 tx=0x7f4a6c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a78ff9640 1 -- 172.21.15.76:0/2632753318 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c016020 con 0x7f4a7c070020 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a78ff9640 1 -- 172.21.15.76:0/2632753318 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a6c00aa50 con 0x7f4a7c070020 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a7c14e280 con 0x7f4a7c070020 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a78ff9640 1 -- 172.21.15.76:0/2632753318 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6c00ae40 con 0x7f4a7c070020 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.281+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7c06fca0 con 0x7f4a7c070020 2024-09-19T04:18:32.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.285+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7c15ec90 con 0x7f4a7c070020 2024-09-19T04:18:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.285+0000 7f4a78ff9640 1 -- 172.21.15.76:0/2632753318 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4a6c019020 con 0x7f4a7c070020 2024-09-19T04:18:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.285+0000 7f4a78ff9640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4a680678c0 0x7f4a68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.285+0000 7f4a78ff9640 1 -- 172.21.15.76:0/2632753318 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7f4a6c085e70 con 0x7f4a7c070020 2024-09-19T04:18:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.285+0000 7f4a7b7fe640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4a680678c0 0x7f4a68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.285+0000 7f4a7b7fe640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4a680678c0 0x7f4a68069d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0b0e30 tx=0x7f4a70008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.289+0000 7f4a78ff9640 1 -- 172.21.15.76:0/2632753318 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4a6c020030 con 0x7f4a7c070020 2024-09-19T04:18:32.439 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.437+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a7c150480 con 0x7f4a7c070020 2024-09-19T04:18:32.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a78ff9640 1 -- 172.21.15.76:0/2632753318 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17029 (secure 0 0 0) 0x7f4a6c056820 con 0x7f4a7c070020 2024-09-19T04:18:32.442 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:32.442 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":80,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:03.399539+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:17:59.452038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:00.756649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4a680678c0 msgr2=0x7f4a68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4a680678c0 0x7f4a68069d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f4a7c0b0e30 tx=0x7f4a70008040 comp rx=0 tx=0).stop 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c070020 msgr2=0x7f4a7c06f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c070020 0x7f4a7c06f210 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c00c7f0 tx=0x7f4a6c00ccb0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 shutdown_connections 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4a680678c0 0x7f4a68069d80 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a7c15f1d0 0x7f4a7c15dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4a7c1520a0 0x7f4a7c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 --2- 172.21.15.76:0/2632753318 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4a7c070020 0x7f4a7c06f210 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 >> 172.21.15.76:0/2632753318 conn(0x7f4a7c0b1f90 msgr2=0x7f4a7c151540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:32.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 shutdown_connections 2024-09-19T04:18:32.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.441+0000 7f4a825f7640 1 -- 172.21.15.76:0/2632753318 wait complete. 2024-09-19T04:18:32.460 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:18:32.460 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8aab55640 1 Processor -- start 2024-09-19T04:18:32.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8aab55640 1 -- start start 2024-09-19T04:18:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8aab55640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb8a415c2a0 0x7fb8a40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8aab55640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a40af2a0 0x7fb8a406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8aab55640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8a40b3290 con 0x7fb8a415bec0 2024-09-19T04:18:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8aab55640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb8a40b3570 con 0x7fb8a415c2a0 2024-09-19T04:18:32.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8aab55640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a40b3850 con 0x7fb8a40af2a0 2024-09-19T04:18:32.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb8a415c2a0 0x7fb8a40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a90f2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a40af2a0 0x7fb8a406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb8a415c2a0 0x7fb8a40aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:48752/0 (socket says 172.21.15.76:48752) 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a90f2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a40af2a0 0x7fb8a406e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:55636/0 (socket says 172.21.15.76:55636) 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a3fff640 1 -- 172.21.15.76:0/2830692285 learned_addr learned my addr 172.21.15.76:0/2830692285 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a88f1640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb8a415bec0 0x7fb8a40ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:56152/0 (socket says 172.21.15.76:56152) 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a3fff640 1 -- 172.21.15.76:0/2830692285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a40af2a0 msgr2=0x7fb8a406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a3fff640 1 --2- 172.21.15.76:0/2830692285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a40af2a0 0x7fb8a406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a3fff640 1 -- 172.21.15.76:0/2830692285 >> v1:172.21.15.116:6789/0 conn(0x7fb8a415bec0 legacy=0x7fb8a40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.577+0000 7fb8a3fff640 1 -- 172.21.15.76:0/2830692285 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a40b3b30 con 0x7fb8a415c2a0 2024-09-19T04:18:32.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8a3fff640 1 --2- 172.21.15.76:0/2830692285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a415c2a0 0x7fb8a40aed60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb894009870 tx=0x7fb89401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ace494def24c9c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8a37fe640 1 -- 172.21.15.76:0/2830692285 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940a95f0 con 0x7fb8a415c2a0 2024-09-19T04:18:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8a37fe640 1 -- 172.21.15.76:0/2830692285 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8940a9790 con 0x7fb8a415c2a0 2024-09-19T04:18:32.582 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8a37fe640 1 -- 172.21.15.76:0/2830692285 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940a9b60 con 0x7fb8a415c2a0 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 -- 172.21.15.76:0/2830692285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a415c2a0 msgr2=0x7fb8a40aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 --2- 172.21.15.76:0/2830692285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a415c2a0 0x7fb8a40aed60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb894009870 tx=0x7fb89401c530 comp rx=0 tx=0).stop 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 -- 172.21.15.76:0/2830692285 shutdown_connections 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 --2- 172.21.15.76:0/2830692285 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a40af2a0 0x7fb8a406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 --2- 172.21.15.76:0/2830692285 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a415c2a0 0x7fb8a40aed60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 -- 172.21.15.76:0/2830692285 >> 172.21.15.76:0/2830692285 conn(0x7fb8a40b1f90 msgr2=0x7fb8a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 -- 172.21.15.76:0/2830692285 shutdown_connections 2024-09-19T04:18:32.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 -- 172.21.15.76:0/2830692285 wait complete. 2024-09-19T04:18:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 Processor -- start 2024-09-19T04:18:32.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.581+0000 7fb8aab55640 1 -- start start 2024-09-19T04:18:32.585 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a40af2a0 0x7fb8a415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8a415bec0 0x7fb8a4156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a415c2a0 0x7fb8a4154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a406ed50 con 0x7fb8a415bec0 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a40af6b0 con 0x7fb8a40af2a0 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb8a406eed0 con 0x7fb8a415c2a0 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8a415bec0 0x7fb8a4156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8a415bec0 0x7fb8a4156b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:42898/0 (socket says 172.21.15.76:42898) 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 -- 172.21.15.76:0/3716781547 learned_addr learned my addr 172.21.15.76:0/3716781547 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a88f1640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a40af2a0 0x7fb8a415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a90f2640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a415c2a0 0x7fb8a4154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 -- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a415c2a0 msgr2=0x7fb8a4154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a415c2a0 0x7fb8a4154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 -- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a40af2a0 msgr2=0x7fb8a415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a40af2a0 0x7fb8a415ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 -- 172.21.15.76:0/3716781547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a415f4c0 con 0x7fb8a415bec0 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a88f1640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a40af2a0 0x7fb8a415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a3fff640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8a415bec0 0x7fb8a4156b40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb89401a160 tx=0x7fb8940192d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a17fa640 1 -- 172.21.15.76:0/3716781547 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940bf070 con 0x7fb8a415bec0 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a17fa640 1 -- 172.21.15.76:0/3716781547 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8940ba410 con 0x7fb8a415bec0 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a415df40 con 0x7fb8a415bec0 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a17fa640 1 -- 172.21.15.76:0/3716781547 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8940ba820 con 0x7fb8a415bec0 2024-09-19T04:18:32.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a415e780 con 0x7fb8a415bec0 2024-09-19T04:18:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8a406e000 con 0x7fb8a415bec0 2024-09-19T04:18:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a17fa640 1 -- 172.21.15.76:0/3716781547 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb8940c2070 con 0x7fb8a415bec0 2024-09-19T04:18:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a17fa640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8900678c0 0x7fb890069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a17fa640 1 -- 172.21.15.76:0/3716781547 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fb89411fd30 con 0x7fb8a415bec0 2024-09-19T04:18:32.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a88f1640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8900678c0 0x7fb890069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.585+0000 7fb8a88f1640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8900678c0 0x7fb890069d80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb898000f70 tx=0x7fb898007d70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.589+0000 7fb8a17fa640 1 -- 172.21.15.76:0/3716781547 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb8a415df40 con 0x7fb8a415bec0 2024-09-19T04:18:32.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.785+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8a406fcd0 con 0x7fb8a415bec0 2024-09-19T04:18:32.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8a17fa640 1 -- 172.21.15.76:0/3716781547 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5267 (secure 0 0 0) 0x7fb8940ad020 con 0x7fb8a415bec0 2024-09-19T04:18:32.789 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:32.789 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":74,"btime":"2024-09-19T04:18:27:920425+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:27.400688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6783,"mds_1":6780,"mds_2":5336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5336":{"gid":5336,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[5336,6780,6783]},"id":5}]} 2024-09-19T04:18:32.790 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 74 2024-09-19T04:18:32.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8900678c0 msgr2=0x7fb890069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8900678c0 0x7fb890069d80 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fb898000f70 tx=0x7fb898007d70 comp rx=0 tx=0).stop 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8a415bec0 msgr2=0x7fb8a4156b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8a415bec0 0x7fb8a4156b40 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb89401a160 tx=0x7fb8940192d0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 shutdown_connections 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8900678c0 0x7fb890069d80 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8a415c2a0 0x7fb8a4154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8a415bec0 0x7fb8a4156b40 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 --2- 172.21.15.76:0/3716781547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8a40af2a0 0x7fb8a415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 >> 172.21.15.76:0/3716781547 conn(0x7fb8a40b1f90 msgr2=0x7fb8a40cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 shutdown_connections 2024-09-19T04:18:32.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.789+0000 7fb8aab55640 1 -- 172.21.15.76:0/3716781547 wait complete. 2024-09-19T04:18:32.808 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:32.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.925+0000 7fa07bc62640 1 Processor -- start 2024-09-19T04:18:32.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.925+0000 7fa07bc62640 1 -- start start 2024-09-19T04:18:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07bc62640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 0x7fa0740cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07bc62640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0740cf450 0x7fa07406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07bc62640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0740d0300 con 0x7fa074161a60 2024-09-19T04:18:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07bc62640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa0740b0f60 con 0x7fa0740cf450 2024-09-19T04:18:32.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07bc62640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa0740cfd40 con 0x7fa0740d3060 2024-09-19T04:18:32.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa074161a60 0x7fa0740d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59410/0 (socket says 172.21.15.76:59410) 2024-09-19T04:18:32.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 -- 172.21.15.76:0/2751504932 learned_addr learned my addr 172.21.15.76:0/2751504932 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:32.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07a1ff640 1 --2- 172.21.15.76:0/2751504932 >> v2:172.21.15.116:3300/0 conn(0x7fa0740cf450 0x7fa07406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0791fd640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 0x7fa0740cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4247694095 0 0) 0x7fa070001120 con 0x7fa074161a60 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa060003530 con 0x7fa074161a60 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4271206565 0 0) 0x7fa060003530 con 0x7fa074161a60 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 msgr2=0x7fa0740cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 0x7fa0740cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 >> v2:172.21.15.116:3300/0 conn(0x7fa0740cf450 msgr2=0x7fa07406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 --2- 172.21.15.76:0/2751504932 >> v2:172.21.15.116:3300/0 conn(0x7fa0740cf450 0x7fa07406d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740d0020 con 0x7fa074161a60 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fa070001850 con 0x7fa074161a60 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 >> v1:172.21.15.76:6789/0 conn(0x7fa074161a60 legacy=0x7fa0740d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0600039c0 0x7fa060003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa060004810 0x7fa060004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 0x7fa0600051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740b0f60 con 0x7fa0600039c0 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa0740cfd40 con 0x7fa060004810 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa06000b9d0 con 0x7fa0740d3060 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0600039c0 0x7fa060003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0791fd640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa060004810 0x7fa060004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07a1ff640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 0x7fa0600051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 -- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 msgr2=0x7fa0600051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 0x7fa0600051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 -- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa060004810 msgr2=0x7fa060004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa060004810 0x7fa060004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa0799fe640 1 -- 172.21.15.76:0/2751504932 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa060003530 con 0x7fa0600039c0 2024-09-19T04:18:32.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.929+0000 7fa07a1ff640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0740d3060 0x7fa0600051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0799fe640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0600039c0 0x7fa060003de0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4f0 tx=0x7fa07001cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1db9fec0c8c46c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000c0e0 con 0x7fa0600039c0 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0700a9020 con 0x7fa0600039c0 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0789fc640 1 -- 172.21.15.76:0/2751504932 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070005b00 con 0x7fa0600039c0 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- 172.21.15.76:0/2751504932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0600039c0 msgr2=0x7fa060003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 --2- 172.21.15.76:0/2751504932 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0600039c0 0x7fa060003de0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4f0 tx=0x7fa07001cc90 comp rx=0 tx=0).stop 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07a1ff640 1 -- 172.21.15.76:0/2751504932 reap_dead start 2024-09-19T04:18:32.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- 172.21.15.76:0/2751504932 shutdown_connections 2024-09-19T04:18:32.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- 172.21.15.76:0/2751504932 >> 172.21.15.76:0/2751504932 conn(0x7fa07408bf90 msgr2=0x7fa07409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:32.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- 172.21.15.76:0/2751504932 shutdown_connections 2024-09-19T04:18:32.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- 172.21.15.76:0/2751504932 wait complete. 2024-09-19T04:18:32.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 Processor -- start 2024-09-19T04:18:32.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- start start 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0600039c0 0x7fa0741648c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa07414b450 0x7fa074160070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0741605b0 0x7fa0740b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa074168f10 con 0x7fa07414b450 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa07406e610 con 0x7fa0741605b0 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07bc62640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa074169090 con 0x7fa0600039c0 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa07414b450 0x7fa074160070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa07414b450 0x7fa074160070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51218/0 (socket says 172.21.15.76:51218) 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 -- 172.21.15.76:0/807122584 learned_addr learned my addr 172.21.15.76:0/807122584 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa07a1ff640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0741605b0 0x7fa0740b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0799fe640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0600039c0 0x7fa0741648c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 -- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0600039c0 msgr2=0x7fa0741648c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0600039c0 0x7fa0741648c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 -- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0741605b0 msgr2=0x7fa0740b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0741605b0 0x7fa0740b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0791fd640 1 -- 172.21.15.76:0/807122584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0740b91e0 con 0x7fa07414b450 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.933+0000 7fa0799fe640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0600039c0 0x7fa0741648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa0791fd640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa07414b450 0x7fa074160070 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa0640115c0 tx=0x7fa064011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa06a7fc640 1 -- 172.21.15.76:0/807122584 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa064015930 con 0x7fa07414b450 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa06a7fc640 1 -- 172.21.15.76:0/807122584 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa064015ad0 con 0x7fa07414b450 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa06a7fc640 1 -- 172.21.15.76:0/807122584 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa06401d400 con 0x7fa07414b450 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0740b94c0 con 0x7fa07414b450 2024-09-19T04:18:32.939 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0740b9a80 con 0x7fa07414b450 2024-09-19T04:18:32.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa07406d6b0 con 0x7fa07414b450 2024-09-19T04:18:32.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa06a7fc640 1 -- 172.21.15.76:0/807122584 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa06401d5a0 con 0x7fa07414b450 2024-09-19T04:18:32.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa06a7fc640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0580678f0 0x7fa058069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:32.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa06a7fc640 1 -- 172.21.15.76:0/807122584 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fa064086570 con 0x7fa07414b450 2024-09-19T04:18:32.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa0799fe640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0580678f0 0x7fa058069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:32.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.937+0000 7fa0799fe640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0580678f0 0x7fa058069db0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4c0 tx=0x7fa070001c20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:32.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:32.941+0000 7fa06a7fc640 1 -- 172.21.15.76:0/807122584 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa064019030 con 0x7fa07414b450 2024-09-19T04:18:33.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.125+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa074160990 con 0x7fa07414b450 2024-09-19T04:18:33.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.125+0000 7fa06a7fc640 1 -- 172.21.15.76:0/807122584 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5269 (secure 0 0 0) 0x7fa064056f20 con 0x7fa07414b450 2024-09-19T04:18:33.128 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:33.128 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":75,"btime":"2024-09-19T04:18:32:944861+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:01.762504+0000","modified":"2024-09-19T04:18:32.400502+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":6783,"mds_1":6780,"mds_2":5336},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5336":{"gid":5336,"name":"d","rank":2,"incarnation":71,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/3224374251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3224374251},{"type":"v1","addr":"172.21.15.116:6835","nonce":3224374251}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6780":{"gid":6780,"name":"a","rank":1,"incarnation":69,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/603375312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":603375312},{"type":"v1","addr":"172.21.15.76:6835","nonce":603375312}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_6783":{"gid":6783,"name":"c","rank":0,"incarnation":63,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3569622848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3569622848},{"type":"v1","addr":"172.21.15.76:6837","nonce":3569622848}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6783,"qdb_cluster":[6783,6780,5336]},"id":5}]} 2024-09-19T04:18:33.128 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 75 2024-09-19T04:18:33.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0580678f0 msgr2=0x7fa058069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0580678f0 0x7fa058069db0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa07000c4c0 tx=0x7fa070001c20 comp rx=0 tx=0).stop 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa07414b450 msgr2=0x7fa074160070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa07414b450 0x7fa074160070 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa0640115c0 tx=0x7fa064011a80 comp rx=0 tx=0).stop 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 shutdown_connections 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0580678f0 0x7fa058069db0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0741605b0 0x7fa0740b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa07414b450 0x7fa074160070 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 --2- 172.21.15.76:0/807122584 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0600039c0 0x7fa0741648c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 >> 172.21.15.76:0/807122584 conn(0x7fa07408bf90 msgr2=0x7fa0740d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 shutdown_connections 2024-09-19T04:18:33.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.129+0000 7fa07bc62640 1 -- 172.21.15.76:0/807122584 wait complete. 2024-09-19T04:18:33.145 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:33.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9c0e1f640 1 Processor -- start 2024-09-19T04:18:33.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9c0e1f640 1 -- start start 2024-09-19T04:18:33.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9c0e1f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd9bc161a60 0x7fd9bc0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9c0e1f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9bc0d32b0 con 0x7fd9bc0d2e90 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9c0e1f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9bc0d0300 con 0x7fd9bc0cf620 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9c0e1f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd9bc0cfa90 con 0x7fd9bc161a60 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd9bc0d2e90 0x7fd9bc0cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59422/0 (socket says 172.21.15.76:59422) 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 -- 172.21.15.76:0/1916145033 learned_addr learned my addr 172.21.15.76:0/1916145033 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376398150 0 0) 0x7fd9bc0d32b0 con 0x7fd9bc0d2e90 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9ba59c640 1 --2- 172.21.15.76:0/1916145033 >> v2:172.21.15.116:3300/0 conn(0x7fd9bc161a60 0x7fd9bc0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9a4003530 con 0x7fd9bc0d2e90 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 426934842 0 0) 0x7fd9a4003530 con 0x7fd9bc0d2e90 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 >> v2:172.21.15.116:3300/0 conn(0x7fd9bc161a60 msgr2=0x7fd9bc0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 --2- 172.21.15.76:0/1916145033 >> v2:172.21.15.116:3300/0 conn(0x7fd9bc161a60 0x7fd9bc0d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 >> v1:172.21.15.116:6789/0 conn(0x7fd9bc0cf620 legacy=0x7fd9bc06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc0b0f60 con 0x7fd9bc0d2e90 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fd9b0001680 con 0x7fd9bc0d2e90 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 >> v1:172.21.15.76:6789/0 conn(0x7fd9bc0d2e90 legacy=0x7fd9bc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a40039c0 0x7fd9a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9a4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a40051d0 0x7fd9a400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0cfa90 con 0x7fd9a4004810 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a400c170 con 0x7fd9a40039c0 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a400c370 con 0x7fd9a40051d0 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9a4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9ba59c640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a40039c0 0x7fd9a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.257 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 -- 172.21.15.76:0/1916145033 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a40051d0 msgr2=0x7fd9a400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:33.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9a40051d0 0x7fd9a400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 -- 172.21.15.76:0/1916145033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a40039c0 msgr2=0x7fd9a4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:33.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9a40039c0 0x7fd9a4003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9b9d9b640 1 -- 172.21.15.76:0/1916145033 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a4003530 con 0x7fd9a4004810 2024-09-19T04:18:33.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.253+0000 7fd9bad9d640 1 -- 172.21.15.76:0/1916145033 reap_dead start 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9b9d9b640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9a4004c90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c210 tx=0x7fd9b001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7e822bcf30a02c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0007d40 con 0x7fd9a4004810 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b00a9070 con 0x7fd9a4004810 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9b959a640 1 -- 172.21.15.76:0/1916145033 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0005d80 con 0x7fd9a4004810 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/1916145033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 msgr2=0x7fd9a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9a4004c90 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c210 tx=0x7fd9b001cc70 comp rx=0 tx=0).stop 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/1916145033 shutdown_connections 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/1916145033 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9a4004c90 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/1916145033 >> 172.21.15.76:0/1916145033 conn(0x7fd9bc08bf90 msgr2=0x7fd9bc09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/1916145033 shutdown_connections 2024-09-19T04:18:33.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/1916145033 wait complete. 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 Processor -- start 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.257+0000 7fd9c0e1f640 1 -- start start 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9bc0b9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9bc1555e0 0x7fd9bc1e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9bc1ea1b0 0x7fd9bc1ea660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc0cf320 con 0x7fd9a4004810 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9bc06d190 con 0x7fd9bc1ea1b0 2024-09-19T04:18:33.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd9bc0cf4a0 con 0x7fd9bc1555e0 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9bc0b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9bc0b9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51230/0 (socket says 172.21.15.76:51230) 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 -- 172.21.15.76:0/3695215996 learned_addr learned my addr 172.21.15.76:0/3695215996 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9bad9d640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9bc1ea1b0 0x7fd9bc1ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9b9d9b640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9bc1555e0 0x7fd9bc1e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 -- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9bc1555e0 msgr2=0x7fd9bc1e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9bc1555e0 0x7fd9bc1e9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 -- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9bc1ea1b0 msgr2=0x7fd9bc1ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9bc1ea1b0 0x7fd9bc1ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 -- 172.21.15.76:0/3695215996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9bc1ec710 con 0x7fd9a4004810 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9bad9d640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9bc1ea1b0 0x7fd9bc1ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9ba59c640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9bc0b9990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac010800 tx=0x7fd9ac010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9aaffd640 1 -- 172.21.15.76:0/3695215996 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac00f040 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9aaffd640 1 -- 172.21.15.76:0/3695215996 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ac00eb00 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9bc0ba100 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9aaffd640 1 -- 172.21.15.76:0/3695215996 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac01c400 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9bc0b80e0 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9bc0d12b0 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9aaffd640 1 -- 172.21.15.76:0/3695215996 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9ac01c5a0 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9aaffd640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd99c0678f0 0x7fd99c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9b9d9b640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd99c0678f0 0x7fd99c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9aaffd640 1 -- 172.21.15.76:0/3695215996 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fd9ac086440 con 0x7fd9a4004810 2024-09-19T04:18:33.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.261+0000 7fd9b9d9b640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd99c0678f0 0x7fd99c069db0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00ab000 tx=0x7fd9b00a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.265+0000 7fd9aaffd640 1 -- 172.21.15.76:0/3695215996 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd9ac018030 con 0x7fd9a4004810 2024-09-19T04:18:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.429+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9bc06e4a0 con 0x7fd9a4004810 2024-09-19T04:18:33.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9aaffd640 1 -- 172.21.15.76:0/3695215996 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+17029 (secure 0 0 0) 0x7fd9ac056df0 con 0x7fd9a4004810 2024-09-19T04:18:33.434 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:33.434 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":80,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:03.399539+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:17:59.452038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:00.756649+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:33.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd99c0678f0 msgr2=0x7fd99c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd99c0678f0 0x7fd99c069db0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00ab000 tx=0x7fd9b00a7070 comp rx=0 tx=0).stop 2024-09-19T04:18:33.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 msgr2=0x7fd9bc0b9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9bc0b9990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac010800 tx=0x7fd9ac010cc0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.436 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 shutdown_connections 2024-09-19T04:18:33.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd99c0678f0 0x7fd99c069db0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9bc1ea1b0 0x7fd9bc1ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9bc1555e0 0x7fd9bc1e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 --2- 172.21.15.76:0/3695215996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9a4004810 0x7fd9bc0b9990 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.433+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 >> 172.21.15.76:0/3695215996 conn(0x7fd9bc08bf90 msgr2=0x7fd9bc0d4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.437+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 shutdown_connections 2024-09-19T04:18:33.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.437+0000 7fd9c0e1f640 1 -- 172.21.15.76:0/3695215996 wait complete. 2024-09-19T04:18:33.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T04:18:33.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd84ab45640 1 Processor -- start 2024-09-19T04:18:33.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd84ab45640 1 -- start start 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd84ab45640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8441520a0 0x7fd84406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd84ab45640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd84415f3a0 0x7fd8440adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd84ab45640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd844070250 con 0x7fd84406fe50 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd84ab45640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd844152480 con 0x7fd8441520a0 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd84ab45640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd8440aede0 con 0x7fd84415f3a0 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd843fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd84406fe50 0x7fd84415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59430/0 (socket says 172.21.15.76:59430) 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd843fff640 1 -- 172.21.15.76:0/3651339525 learned_addr learned my addr 172.21.15.76:0/3651339525 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8488e1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8441520a0 0x7fd84406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8490e2640 1 --2- 172.21.15.76:0/3651339525 >> v2:172.21.15.116:3300/0 conn(0x7fd84415f3a0 0x7fd8440adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8488e1640 1 -- 172.21.15.76:0/3651339525 >> v2:172.21.15.116:3300/0 conn(0x7fd84415f3a0 msgr2=0x7fd8440adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:33.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8488e1640 1 --2- 172.21.15.76:0/3651339525 >> v2:172.21.15.116:3300/0 conn(0x7fd84415f3a0 0x7fd8440adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8488e1640 1 -- 172.21.15.76:0/3651339525 >> v1:172.21.15.76:6789/0 conn(0x7fd84406fe50 legacy=0x7fd84415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8488e1640 1 -- 172.21.15.76:0/3651339525 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84415fa10 con 0x7fd8441520a0 2024-09-19T04:18:33.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8437fe640 1 -- 172.21.15.76:0/3651339525 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3265718838 0 0) 0x7fd844070250 con 0x7fd84406fe50 2024-09-19T04:18:33.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.569+0000 7fd8490e2640 1 --2- 172.21.15.76:0/3651339525 >> v2:172.21.15.116:3300/0 conn(0x7fd84415f3a0 0x7fd8440adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:33.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd8488e1640 1 --2- 172.21.15.76:0/3651339525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84406f910 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd844155b40 tx=0x7fd83401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbe10b3a77e02ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd8437fe640 1 -- 172.21.15.76:0/3651339525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340b0070 con 0x7fd8441520a0 2024-09-19T04:18:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd8437fe640 1 -- 172.21.15.76:0/3651339525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8340ab600 con 0x7fd8441520a0 2024-09-19T04:18:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd8437fe640 1 -- 172.21.15.76:0/3651339525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340ab9d0 con 0x7fd8441520a0 2024-09-19T04:18:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 -- 172.21.15.76:0/3651339525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 msgr2=0x7fd84406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 --2- 172.21.15.76:0/3651339525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84406f910 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fd844155b40 tx=0x7fd83401c480 comp rx=0 tx=0).stop 2024-09-19T04:18:33.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 -- 172.21.15.76:0/3651339525 shutdown_connections 2024-09-19T04:18:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 --2- 172.21.15.76:0/3651339525 >> v2:172.21.15.116:3300/0 conn(0x7fd84415f3a0 0x7fd8440adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 --2- 172.21.15.76:0/3651339525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84406f910 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 -- 172.21.15.76:0/3651339525 >> 172.21.15.76:0/3651339525 conn(0x7fd8440b1f90 msgr2=0x7fd8440b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 -- 172.21.15.76:0/3651339525 shutdown_connections 2024-09-19T04:18:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.573+0000 7fd84ab45640 1 -- 172.21.15.76:0/3651339525 wait complete. 2024-09-19T04:18:33.576 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 Processor -- start 2024-09-19T04:18:33.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 -- start start 2024-09-19T04:18:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd84406fe50 0x7fd844146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84415f3a0 0x7fd844147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd844073d30 con 0x7fd8441520a0 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd844073a50 con 0x7fd84415f3a0 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd844167f10 con 0x7fd84406fe50 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84414f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51246/0 (socket says 172.21.15.76:51246) 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 -- 172.21.15.76:0/1807158958 learned_addr learned my addr 172.21.15.76:0/1807158958 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd8490e2640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84415f3a0 0x7fd844147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 -- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd84406fe50 msgr2=0x7fd844146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd8488e1640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd84406fe50 0x7fd844146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd84406fe50 0x7fd844146700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 -- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84415f3a0 msgr2=0x7fd844147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84415f3a0 0x7fd844147750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 -- 172.21.15.76:0/1807158958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84414e860 con 0x7fd8441520a0 2024-09-19T04:18:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd843fff640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84414f900 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c4b0 tx=0x7fd83800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd8417fa640 1 -- 172.21.15.76:0/1807158958 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838018070 con 0x7fd8441520a0 2024-09-19T04:18:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd8417fa640 1 -- 172.21.15.76:0/1807158958 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd838007500 con 0x7fd8441520a0 2024-09-19T04:18:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd8417fa640 1 -- 172.21.15.76:0/1807158958 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd838004a30 con 0x7fd8441520a0 2024-09-19T04:18:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84414eb40 con 0x7fd8441520a0 2024-09-19T04:18:33.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84414f100 con 0x7fd8441520a0 2024-09-19T04:18:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84406f910 con 0x7fd8441520a0 2024-09-19T04:18:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd8417fa640 1 -- 172.21.15.76:0/1807158958 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd838004bd0 con 0x7fd8441520a0 2024-09-19T04:18:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.577+0000 7fd8417fa640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8300678c0 0x7fd830069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.581+0000 7fd8417fa640 1 -- 172.21.15.76:0/1807158958 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6101+0+0 (secure 0 0 0) 0x7fd838085060 con 0x7fd8441520a0 2024-09-19T04:18:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.581+0000 7fd8488e1640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8300678c0 0x7fd830069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.581 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.581+0000 7fd8488e1640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8300678c0 0x7fd830069d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd8340ad000 tx=0x7fd834008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.584 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.585+0000 7fd8417fa640 1 -- 172.21.15.76:0/1807158958 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd838055a10 con 0x7fd8441520a0 2024-09-19T04:18:33.769 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.769+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd844146c40 con 0x7fd8441520a0 2024-09-19T04:18:33.770 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:18:33.769+0000 7fa674384640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:18:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.773+0000 7fd8417fa640 1 -- 172.21.15.76:0/1807158958 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v76) ==== 161+0+0 (secure 0 0 0) 0x7fd838055310 con 0x7fd8441520a0 2024-09-19T04:18:33.773 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:18:33.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.773+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8300678c0 msgr2=0x7fd830069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.773+0000 7fd84ab45640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8300678c0 0x7fd830069d80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd8340ad000 tx=0x7fd834008040 comp rx=0 tx=0).stop 2024-09-19T04:18:33.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.773+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 msgr2=0x7fd84414f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.773+0000 7fd84ab45640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84414f900 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c4b0 tx=0x7fd83800c970 comp rx=0 tx=0).stop 2024-09-19T04:18:33.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 shutdown_connections 2024-09-19T04:18:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8300678c0 0x7fd830069d80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd84415f3a0 0x7fd844147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8441520a0 0x7fd84414f900 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 --2- 172.21.15.76:0/1807158958 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd84406fe50 0x7fd844146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 >> 172.21.15.76:0/1807158958 conn(0x7fd8440b1f90 msgr2=0x7fd8440b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 shutdown_connections 2024-09-19T04:18:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.781+0000 7fd84ab45640 1 -- 172.21.15.76:0/1807158958 wait complete. 2024-09-19T04:18:33.782 INFO:tasks.ceph.mds.c.smithi076.stderr: -8277> 2024-09-19T04:18:08.382+0000 7ff5fba83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:18:33.783 INFO:tasks.ceph.mds.c.smithi076.stderr: -7727> 2024-09-19T04:18:13.381+0000 7ff5fba83640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T04:18:33.788 INFO:tasks.ceph.mds.c.smithi076.stderr: -4103> 2024-09-19T04:18:18.381+0000 7ff5fba83640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:18:33.798 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:18:33.809 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:18:33.817 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:18:33.845 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:18:33.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83c976640 1 Processor -- start 2024-09-19T04:18:33.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83c976640 1 -- start start 2024-09-19T04:18:33.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83c976640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc83815c2a0 0x7fc8380aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83c976640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8380af2a0 0x7fc83806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83c976640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8380b3290 con 0x7fc83815bec0 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83c976640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc8380b3570 con 0x7fc83815c2a0 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83c976640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc8380b3850 con 0x7fc8380af2a0 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc835d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc83815c2a0 0x7fc8380aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc83815bec0 0x7fc8380ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59448/0 (socket says 172.21.15.76:59448) 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83659c640 1 -- 172.21.15.76:0/724620159 learned_addr learned my addr 172.21.15.76:0/724620159 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc835d9b640 1 --2- 172.21.15.76:0/724620159 >> v2:172.21.15.76:3300/0 conn(0x7fc83815c2a0 0x7fc8380aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51278/0 (socket says 172.21.15.76:51278) 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc835d9b640 1 -- 172.21.15.76:0/724620159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8380af2a0 msgr2=0x7fc83806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc836d9d640 1 --2- 172.21.15.76:0/724620159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8380af2a0 0x7fc83806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc83559a640 1 -- 172.21.15.76:0/724620159 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2234585376 0 0) 0x7fc8380b3290 con 0x7fc83815bec0 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc835d9b640 1 --2- 172.21.15.76:0/724620159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8380af2a0 0x7fc83806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc835d9b640 1 -- 172.21.15.76:0/724620159 >> v1:172.21.15.76:6789/0 conn(0x7fc83815bec0 legacy=0x7fc8380ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc835d9b640 1 -- 172.21.15.76:0/724620159 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8380b3b30 con 0x7fc83815c2a0 2024-09-19T04:18:33.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.913+0000 7fc836d9d640 1 --2- 172.21.15.76:0/724620159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8380af2a0 0x7fc83806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc835d9b640 1 --2- 172.21.15.76:0/724620159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc83815c2a0 0x7fc8380aed60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097c0 tx=0x7fc82c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7851f12eb61c8840 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83559a640 1 -- 172.21.15.76:0/724620159 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0a7070 con 0x7fc83815c2a0 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83559a640 1 -- 172.21.15.76:0/724620159 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc82c0b0070 con 0x7fc83815c2a0 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83559a640 1 -- 172.21.15.76:0/724620159 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0ab890 con 0x7fc83815c2a0 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- 172.21.15.76:0/724620159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc83815c2a0 msgr2=0x7fc8380aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 --2- 172.21.15.76:0/724620159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc83815c2a0 0x7fc8380aed60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0097c0 tx=0x7fc82c01c470 comp rx=0 tx=0).stop 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- 172.21.15.76:0/724620159 shutdown_connections 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 --2- 172.21.15.76:0/724620159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc8380af2a0 0x7fc83806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 --2- 172.21.15.76:0/724620159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc83815c2a0 0x7fc8380aed60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- 172.21.15.76:0/724620159 >> 172.21.15.76:0/724620159 conn(0x7fc8380b1f90 msgr2=0x7fc8380b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:33.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- 172.21.15.76:0/724620159 shutdown_connections 2024-09-19T04:18:33.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- 172.21.15.76:0/724620159 wait complete. 2024-09-19T04:18:33.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 Processor -- start 2024-09-19T04:18:33.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- start start 2024-09-19T04:18:33.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8380af2a0 0x7fc83815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc83815bec0 0x7fc8381656d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc83815c2a0 0x7fc838166520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc83806ed50 con 0x7fc8380af2a0 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc8380af6b0 con 0x7fc83815bec0 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83c976640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc83806eed0 con 0x7fc83815c2a0 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8380af2a0 0x7fc83815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8380af2a0 0x7fc83815d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51280/0 (socket says 172.21.15.76:51280) 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 -- 172.21.15.76:0/855885327 learned_addr learned my addr 172.21.15.76:0/855885327 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc836d9d640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc83815c2a0 0x7fc838166520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 -- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc83815c2a0 msgr2=0x7fc838166520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc83815c2a0 0x7fc838166520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc835d9b640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc83815bec0 0x7fc8381656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 -- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc83815bec0 msgr2=0x7fc8381656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:33.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc83815bec0 0x7fc8381656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 -- 172.21.15.76:0/855885327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83815c750 con 0x7fc8380af2a0 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.917+0000 7fc83659c640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8380af2a0 0x7fc83815d7f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc8280011f0 tx=0x7fc82800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc826ffd640 1 -- 172.21.15.76:0/855885327 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828018070 con 0x7fc8380af2a0 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc826ffd640 1 -- 172.21.15.76:0/855885327 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8280074e0 con 0x7fc8380af2a0 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83815ca30 con 0x7fc8380af2a0 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc826ffd640 1 -- 172.21.15.76:0/855885327 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828004a10 con 0x7fc8380af2a0 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83815cff0 con 0x7fc8380af2a0 2024-09-19T04:18:33.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc83806e000 con 0x7fc8380af2a0 2024-09-19T04:18:33.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc826ffd640 1 -- 172.21.15.76:0/855885327 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc82801b070 con 0x7fc8380af2a0 2024-09-19T04:18:33.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc826ffd640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc8180678c0 0x7fc818069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:33.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc826ffd640 1 -- 172.21.15.76:0/855885327 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6359+0+0 (secure 0 0 0) 0x7fc828085230 con 0x7fc8380af2a0 2024-09-19T04:18:33.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc835d9b640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc8180678c0 0x7fc818069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:33.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.921+0000 7fc835d9b640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc8180678c0 0x7fc818069d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0af000 tx=0x7fc82c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:33.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:33.925+0000 7fc826ffd640 1 -- 172.21.15.76:0/855885327 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc828055b70 con 0x7fc8380af2a0 2024-09-19T04:18:34.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.109+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc83806fcd0 con 0x7fc8380af2a0 2024-09-19T04:18:34.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.781+0000 7fc826ffd640 1 -- 172.21.15.76:0/855885327 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v77) ==== 104+0+0 (secure 0 0 0) 0x7fc828055470 con 0x7fc8380af2a0 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc8180678c0 msgr2=0x7fc818069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc8180678c0 0x7fc818069d80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc82c0af000 tx=0x7fc82c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8380af2a0 msgr2=0x7fc83815d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8380af2a0 0x7fc83815d7f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fc8280011f0 tx=0x7fc82800c9b0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 shutdown_connections 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc8180678c0 0x7fc818069d80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc83815c2a0 0x7fc838166520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc83815bec0 0x7fc8381656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 --2- 172.21.15.76:0/855885327 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc8380af2a0 0x7fc83815d7f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 >> 172.21.15.76:0/855885327 conn(0x7fc8380b1f90 msgr2=0x7fc83815a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:34.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 shutdown_connections 2024-09-19T04:18:34.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.785+0000 7fc83c976640 1 -- 172.21.15.76:0/855885327 wait complete. 2024-09-19T04:18:34.812 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:18:34.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.981+0000 7f9a26a7f640 1 Processor -- start 2024-09-19T04:18:34.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.981+0000 7f9a26a7f640 1 -- start start 2024-09-19T04:18:34.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a26a7f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9a2009ac60 0x7f9a2009ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:34.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a26a7f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9a2009b040 0x7f9a200af730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:34.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a26a7f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a200afd80 con 0x7f9a200b0e20 2024-09-19T04:18:34.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a26a7f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9a200b0060 con 0x7f9a2009b040 2024-09-19T04:18:34.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a26a7f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9a200b0340 con 0x7f9a2009ac60 2024-09-19T04:18:34.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9a2009b040 0x7f9a200af730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:34.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a2501c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9a200b0e20 0x7f9a200b1bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59454/0 (socket says 172.21.15.76:59454) 2024-09-19T04:18:34.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a2501c640 1 -- 172.21.15.76:0/1078252442 learned_addr learned my addr 172.21.15.76:0/1078252442 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:34.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1ffff640 1 -- 172.21.15.76:0/1078252442 >> v2:172.21.15.116:3300/0 conn(0x7f9a2009ac60 msgr2=0x7f9a2009ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:34.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1ffff640 1 --2- 172.21.15.76:0/1078252442 >> v2:172.21.15.116:3300/0 conn(0x7f9a2009ac60 0x7f9a2009ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1ffff640 1 -- 172.21.15.76:0/1078252442 >> v1:172.21.15.76:6789/0 conn(0x7f9a200b0e20 legacy=0x7f9a200b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:34.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1ffff640 1 -- 172.21.15.76:0/1078252442 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a200b0620 con 0x7f9a2009b040 2024-09-19T04:18:34.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1f7fe640 1 -- 172.21.15.76:0/1078252442 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1389415516 0 0) 0x7f9a200afd80 con 0x7f9a200b0e20 2024-09-19T04:18:34.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1ffff640 1 --2- 172.21.15.76:0/1078252442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009b040 0x7f9a200af730 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f9a200b7d70 tx=0x7f9a1401ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6305080a33e24a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:34.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1f7fe640 1 -- 172.21.15.76:0/1078252442 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140b1070 con 0x7f9a2009b040 2024-09-19T04:18:34.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1f7fe640 1 -- 172.21.15.76:0/1078252442 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a140a8050 con 0x7f9a2009b040 2024-09-19T04:18:34.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a1f7fe640 1 -- 172.21.15.76:0/1078252442 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a140ac860 con 0x7f9a2009b040 2024-09-19T04:18:34.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a26a7f640 1 -- 172.21.15.76:0/1078252442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009b040 msgr2=0x7f9a200af730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:34.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.985+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/1078252442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009b040 0x7f9a200af730 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f9a200b7d70 tx=0x7f9a1401ce20 comp rx=0 tx=0).stop 2024-09-19T04:18:34.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- 172.21.15.76:0/1078252442 shutdown_connections 2024-09-19T04:18:34.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/1078252442 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009b040 0x7f9a200af730 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/1078252442 >> v2:172.21.15.116:3300/0 conn(0x7f9a2009ac60 0x7f9a2009ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- 172.21.15.76:0/1078252442 >> 172.21.15.76:0/1078252442 conn(0x7f9a200b78d0 msgr2=0x7f9a200b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:34.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- 172.21.15.76:0/1078252442 shutdown_connections 2024-09-19T04:18:34.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- 172.21.15.76:0/1078252442 wait complete. 2024-09-19T04:18:34.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 Processor -- start 2024-09-19T04:18:34.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- start start 2024-09-19T04:18:34.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009ac60 0x7f9a2014d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:34.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9a2009b040 0x7f9a2006cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a200b0e20 0x7f9a2006d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2015d3e0 con 0x7f9a2009ac60 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9a2015d260 con 0x7f9a200b0e20 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9a2015d560 con 0x7f9a2009b040 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009ac60 0x7f9a2014d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009ac60 0x7f9a2014d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51310/0 (socket says 172.21.15.76:51310) 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 -- 172.21.15.76:0/716788680 learned_addr learned my addr 172.21.15.76:0/716788680 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a1ffff640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9a2009b040 0x7f9a2006cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 -- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9a2009b040 msgr2=0x7f9a2006cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9a2009b040 0x7f9a2006cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2501c640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a200b0e20 0x7f9a2006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 -- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a200b0e20 msgr2=0x7f9a2006d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a200b0e20 0x7f9a2006d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 -- 172.21.15.76:0/716788680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a20156ee0 con 0x7f9a2009ac60 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a1ffff640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9a2009b040 0x7f9a2006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2501c640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a200b0e20 0x7f9a2006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a2481b640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009ac60 0x7f9a2014d7c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b550 tx=0x7f9a1000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a1d7fa640 1 -- 172.21.15.76:0/716788680 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10014070 con 0x7f9a2009ac60 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a1d7fa640 1 -- 172.21.15.76:0/716788680 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a10010890 con 0x7f9a2009ac60 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a201571c0 con 0x7f9a2009ac60 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a1d7fa640 1 -- 172.21.15.76:0/716788680 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a10010c80 con 0x7f9a2009ac60 2024-09-19T04:18:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.989+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a20157a60 con 0x7f9a2009ac60 2024-09-19T04:18:34.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.993+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a20157c60 con 0x7f9a2009ac60 2024-09-19T04:18:34.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.993+0000 7f9a1d7fa640 1 -- 172.21.15.76:0/716788680 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9a10017070 con 0x7f9a2009ac60 2024-09-19T04:18:34.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.993+0000 7f9a1d7fa640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9a0c0678c0 0x7f9a0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:34.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.993+0000 7f9a1d7fa640 1 -- 172.21.15.76:0/716788680 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 6359+0+0 (secure 0 0 0) 0x7f9a10085e00 con 0x7f9a2009ac60 2024-09-19T04:18:34.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.993+0000 7f9a1ffff640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9a0c0678c0 0x7f9a0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:34.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.993+0000 7f9a1ffff640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9a0c0678c0 0x7f9a0c069d80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9a140ae000 tx=0x7f9a14004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:34.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:34.997+0000 7f9a1d7fa640 1 -- 172.21.15.76:0/716788680 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9a20157c60 con 0x7f9a2009ac60 2024-09-19T04:18:35.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.157+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9a201552e0 con 0x7f9a2009ac60 2024-09-19T04:18:35.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.965+0000 7f9a1d7fa640 1 -- 172.21.15.76:0/716788680 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v82) ==== 181+0+0 (secure 0 0 0) 0x7f9a10010e20 con 0x7f9a2009ac60 2024-09-19T04:18:35.966 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:18:35.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9a0c0678c0 msgr2=0x7f9a0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:35.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9a0c0678c0 0x7f9a0c069d80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9a140ae000 tx=0x7f9a14004740 comp rx=0 tx=0).stop 2024-09-19T04:18:35.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009ac60 msgr2=0x7f9a2014d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:35.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009ac60 0x7f9a2014d7c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b550 tx=0x7f9a1000ba10 comp rx=0 tx=0).stop 2024-09-19T04:18:35.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 shutdown_connections 2024-09-19T04:18:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9a0c0678c0 0x7f9a0c069d80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a200b0e20 0x7f9a2006d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9a2009b040 0x7f9a2006cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 --2- 172.21.15.76:0/716788680 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9a2009ac60 0x7f9a2014d7c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 >> 172.21.15.76:0/716788680 conn(0x7f9a200b78d0 msgr2=0x7f9a2014e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:35.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 shutdown_connections 2024-09-19T04:18:35.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:35.969+0000 7f9a26a7f640 1 -- 172.21.15.76:0/716788680 wait complete. 2024-09-19T04:18:35.991 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:18:36.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062f8a2640 1 Processor -- start 2024-09-19T04:18:36.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062f8a2640 1 -- start start 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062f8a2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0628165ea0 0x7f06280ae070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062f8a2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f06280b2200 0x7f06280b4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062f8a2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0628166280 con 0x7f06280b45d0 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062f8a2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f062809aa00 con 0x7f0628165ea0 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062f8a2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f06280b5d90 con 0x7f06280b2200 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062d63e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0628165ea0 0x7f06280ae070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062de3f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f06280b45d0 0x7f06280b0730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59460/0 (socket says 172.21.15.76:59460) 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062d63e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0628165ea0 0x7f06280ae070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51324/0 (socket says 172.21.15.76:51324) 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062de3f640 1 -- 172.21.15.76:0/980789938 learned_addr learned my addr 172.21.15.76:0/980789938 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062ce3d640 1 --2- 172.21.15.76:0/980789938 >> v2:172.21.15.116:3300/0 conn(0x7f06280b2200 0x7f06280b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:36.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062d63e640 1 -- 172.21.15.76:0/980789938 >> v2:172.21.15.116:3300/0 conn(0x7f06280b2200 msgr2=0x7f06280b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:36.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062d63e640 1 --2- 172.21.15.76:0/980789938 >> v2:172.21.15.116:3300/0 conn(0x7f06280b2200 0x7f06280b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062d63e640 1 -- 172.21.15.76:0/980789938 >> v1:172.21.15.76:6789/0 conn(0x7f06280b45d0 legacy=0x7f06280b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:36.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.117+0000 7f062d63e640 1 -- 172.21.15.76:0/980789938 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06280b6550 con 0x7f0628165ea0 2024-09-19T04:18:36.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062d63e640 1 --2- 172.21.15.76:0/980789938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0628165ea0 0x7f06280ae070 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0628072110 tx=0x7f061c01c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=929eff8e3ebe2ca2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:36.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f061bfff640 1 -- 172.21.15.76:0/980789938 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0b0070 con 0x7f0628165ea0 2024-09-19T04:18:36.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f061bfff640 1 -- 172.21.15.76:0/980789938 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c0ab600 con 0x7f0628165ea0 2024-09-19T04:18:36.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f061bfff640 1 -- 172.21.15.76:0/980789938 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0ab9d0 con 0x7f0628165ea0 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 -- 172.21.15.76:0/980789938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0628165ea0 msgr2=0x7f06280ae070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 --2- 172.21.15.76:0/980789938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0628165ea0 0x7f06280ae070 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f0628072110 tx=0x7f061c01c310 comp rx=0 tx=0).stop 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 -- 172.21.15.76:0/980789938 shutdown_connections 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 --2- 172.21.15.76:0/980789938 >> v2:172.21.15.116:3300/0 conn(0x7f06280b2200 0x7f06280b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 --2- 172.21.15.76:0/980789938 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0628165ea0 0x7f06280ae070 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 -- 172.21.15.76:0/980789938 >> 172.21.15.76:0/980789938 conn(0x7f0628073380 msgr2=0x7f06280b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 -- 172.21.15.76:0/980789938 shutdown_connections 2024-09-19T04:18:36.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 -- 172.21.15.76:0/980789938 wait complete. 2024-09-19T04:18:36.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 Processor -- start 2024-09-19T04:18:36.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.121+0000 7f062f8a2640 1 -- start start 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06280b2200 0x7f06280cfde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06280b45d0 0x7f062814fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0628165ea0 0x7f06280b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f06280b6030 con 0x7f06280b2200 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f06280b2950 con 0x7f0628165ea0 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f06280ae5b0 con 0x7f06280b45d0 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06280b2200 0x7f06280cfde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06280b2200 0x7f06280cfde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51328/0 (socket says 172.21.15.76:51328) 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 -- 172.21.15.76:0/270905944 learned_addr learned my addr 172.21.15.76:0/270905944 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:36.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062ce3d640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06280b45d0 0x7f062814fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 -- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06280b45d0 msgr2=0x7f062814fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062de3f640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0628165ea0 0x7f06280b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06280b45d0 0x7f062814fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 -- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0628165ea0 msgr2=0x7f06280b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0628165ea0 0x7f06280b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 -- 172.21.15.76:0/270905944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0628150a90 con 0x7f06280b2200 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062d63e640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06280b2200 0x7f06280cfde0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f061c0ae040 tx=0x7f061c0190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f0619ffb640 1 -- 172.21.15.76:0/270905944 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0b8020 con 0x7f06280b2200 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f0619ffb640 1 -- 172.21.15.76:0/270905944 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f061c0aa730 con 0x7f06280b2200 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0628150d70 con 0x7f06280b2200 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f0619ffb640 1 -- 172.21.15.76:0/270905944 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c0b0040 con 0x7f06280b2200 2024-09-19T04:18:36.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06280d0320 con 0x7f06280b2200 2024-09-19T04:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0628164880 con 0x7f06280b2200 2024-09-19T04:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f0619ffb640 1 -- 172.21.15.76:0/270905944 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f061c0bb020 con 0x7f06280b2200 2024-09-19T04:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f0619ffb640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f060c0678c0 0x7f060c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:36.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f0619ffb640 1 -- 172.21.15.76:0/270905944 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5921+0+0 (secure 0 0 0) 0x7f061c11fd60 con 0x7f06280b2200 2024-09-19T04:18:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062ce3d640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f060c0678c0 0x7f060c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:36.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.125+0000 7f062ce3d640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f060c0678c0 0x7f060c069d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f06280b1420 tx=0x7f062000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:36.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.129+0000 7f0619ffb640 1 -- 172.21.15.76:0/270905944 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f061c0c2030 con 0x7f06280b2200 2024-09-19T04:18:36.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.281+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0628150f40 con 0x7f06280b2200 2024-09-19T04:18:36.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.973+0000 7f0619ffb640 1 -- 172.21.15.76:0/270905944 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v83) ==== 169+0+0 (secure 0 0 0) 0x7f061c015f90 con 0x7f06280b2200 2024-09-19T04:18:36.975 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-09-19T04:18:36.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.977+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f060c0678c0 msgr2=0x7f060c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:36.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.977+0000 7f062f8a2640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f060c0678c0 0x7f060c069d80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f06280b1420 tx=0x7f062000b040 comp rx=0 tx=0).stop 2024-09-19T04:18:36.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.977+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06280b2200 msgr2=0x7f06280cfde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:36.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.977+0000 7f062f8a2640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06280b2200 0x7f06280cfde0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f061c0ae040 tx=0x7f061c0190b0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 shutdown_connections 2024-09-19T04:18:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f060c0678c0 0x7f060c069d80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0628165ea0 0x7f06280b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f06280b45d0 0x7f062814fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 --2- 172.21.15.76:0/270905944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f06280b2200 0x7f06280cfde0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 >> 172.21.15.76:0/270905944 conn(0x7f0628073380 msgr2=0x7f06280b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:36.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 shutdown_connections 2024-09-19T04:18:36.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:36.981+0000 7f062f8a2640 1 -- 172.21.15.76:0/270905944 wait complete. 2024-09-19T04:18:37.004 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T04:18:37.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.125+0000 7fb625401640 1 Processor -- start 2024-09-19T04:18:37.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.125+0000 7fb625401640 1 -- start start 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.125+0000 7fb625401640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb62015bec0 0x7fb6200ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.125+0000 7fb625401640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6200b3290 con 0x7fb6200af470 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.125+0000 7fb625401640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6200b3570 con 0x7fb62015c2a0 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.125+0000 7fb625401640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb6200b3850 con 0x7fb62015bec0 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb6200af470 0x7fb62006e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59466/0 (socket says 172.21.15.76:59466) 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61f7fe640 1 -- 172.21.15.76:0/4029011281 learned_addr learned my addr 172.21.15.76:0/4029011281 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 --2- 172.21.15.76:0/4029011281 >> v2:172.21.15.116:3300/0 conn(0x7fb62015bec0 0x7fb6200ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 485115358 0 0) 0x7fb6200b3290 con 0x7fb6200af470 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb604003530 con 0x7fb6200af470 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 312470124 0 0) 0x7fb604003530 con 0x7fb6200af470 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 >> v2:172.21.15.116:3300/0 conn(0x7fb62015bec0 msgr2=0x7fb6200ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:37.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 --2- 172.21.15.76:0/4029011281 >> v2:172.21.15.116:3300/0 conn(0x7fb62015bec0 0x7fb6200ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 >> v1:172.21.15.116:6789/0 conn(0x7fb62015c2a0 legacy=0x7fb6200aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6200b3b30 con 0x7fb6200af470 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb618001a00 con 0x7fb6200af470 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 >> v1:172.21.15.76:6789/0 conn(0x7fb6200af470 legacy=0x7fb62006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6040039c0 0x7fb604003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb604004810 0x7fb604004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040051d0 0x7fb60400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6200b3850 con 0x7fb6040039c0 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb60400c070 con 0x7fb604004810 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb60400c270 con 0x7fb6040051d0 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6040039c0 0x7fb604003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61e7fc640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb604004810 0x7fb604004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 -- 172.21.15.76:0/4029011281 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040051d0 msgr2=0x7fb60400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:37.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040051d0 0x7fb60400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 -- 172.21.15.76:0/4029011281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb604004810 msgr2=0x7fb604004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:37.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb604004810 0x7fb604004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 -- 172.21.15.76:0/4029011281 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb604003530 con 0x7fb6040039c0 2024-09-19T04:18:37.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61f7fe640 1 -- 172.21.15.76:0/4029011281 reap_dead start 2024-09-19T04:18:37.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61effd640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6040039c0 0x7fb604003de0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb610009ae0 tx=0x7fb61001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c984a4072053999 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6100b2070 con 0x7fb6040039c0 2024-09-19T04:18:37.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6100a5520 con 0x7fb6040039c0 2024-09-19T04:18:37.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb61dffb640 1 -- 172.21.15.76:0/4029011281 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6100096b0 con 0x7fb6040039c0 2024-09-19T04:18:37.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb625401640 1 -- 172.21.15.76:0/4029011281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6040039c0 msgr2=0x7fb604003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.129+0000 7fb625401640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6040039c0 0x7fb604003de0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb610009ae0 tx=0x7fb61001ca40 comp rx=0 tx=0).stop 2024-09-19T04:18:37.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- 172.21.15.76:0/4029011281 shutdown_connections 2024-09-19T04:18:37.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 --2- 172.21.15.76:0/4029011281 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb6040039c0 0x7fb604003de0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- 172.21.15.76:0/4029011281 >> 172.21.15.76:0/4029011281 conn(0x7fb6200b1f90 msgr2=0x7fb6200b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:37.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- 172.21.15.76:0/4029011281 shutdown_connections 2024-09-19T04:18:37.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- 172.21.15.76:0/4029011281 wait complete. 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 Processor -- start 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- start start 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040039c0 0x7fb6201e9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb620146190 0x7fb620146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6201f5b70 0x7fb6201f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb6200af2f0 con 0x7fb620146190 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb6200af170 con 0x7fb6201f5b70 2024-09-19T04:18:37.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb6200ad610 con 0x7fb6040039c0 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb620146190 0x7fb620146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb620146190 0x7fb620146610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51336/0 (socket says 172.21.15.76:51336) 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 -- 172.21.15.76:0/2281886011 learned_addr learned my addr 172.21.15.76:0/2281886011 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61effd640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040039c0 0x7fb6201e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61f7fe640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6201f5b70 0x7fb6201f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 -- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040039c0 msgr2=0x7fb6201e9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040039c0 0x7fb6201e9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 -- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6201f5b70 msgr2=0x7fb6201f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6201f5b70 0x7fb6201f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 -- 172.21.15.76:0/2281886011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6201f8eb0 con 0x7fb620146190 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61f7fe640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6201f5b70 0x7fb6201f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb61e7fc640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb620146190 0x7fb620146610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00d570 tx=0x7fb60c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb6177fe640 1 -- 172.21.15.76:0/2281886011 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c005480 con 0x7fb620146190 2024-09-19T04:18:37.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb6177fe640 1 -- 172.21.15.76:0/2281886011 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb60c017070 con 0x7fb620146190 2024-09-19T04:18:37.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6201f9150 con 0x7fb620146190 2024-09-19T04:18:37.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb6177fe640 1 -- 172.21.15.76:0/2281886011 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60c003820 con 0x7fb620146190 2024-09-19T04:18:37.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201f9710 con 0x7fb620146190 2024-09-19T04:18:37.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.133+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb620160470 con 0x7fb620146190 2024-09-19T04:18:37.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.137+0000 7fb6177fe640 1 -- 172.21.15.76:0/2281886011 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb60c0039c0 con 0x7fb620146190 2024-09-19T04:18:37.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.137+0000 7fb6177fe640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6080678f0 0x7fb608069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.137+0000 7fb6177fe640 1 -- 172.21.15.76:0/2281886011 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7fb60c0864a0 con 0x7fb620146190 2024-09-19T04:18:37.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.137+0000 7fb61effd640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6080678f0 0x7fb608069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.137+0000 7fb61effd640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6080678f0 0x7fb608069db0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb6100af000 tx=0x7fb610008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.141+0000 7fb6177fe640 1 -- 172.21.15.76:0/2281886011 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb60c057080 con 0x7fb620146190 2024-09-19T04:18:37.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.325+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb62006fcd0 con 0x7fb620146190 2024-09-19T04:18:37.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.329+0000 7fb6177fe640 1 -- 172.21.15.76:0/2281886011 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7fb60c056980 con 0x7fb620146190 2024-09-19T04:18:37.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6080678f0 msgr2=0x7fb608069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6080678f0 0x7fb608069db0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb6100af000 tx=0x7fb610008040 comp rx=0 tx=0).stop 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb620146190 msgr2=0x7fb620146610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb620146190 0x7fb620146610 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb60c00d570 tx=0x7fb60c00da30 comp rx=0 tx=0).stop 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 shutdown_connections 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6080678f0 0x7fb608069db0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6201f5b70 0x7fb6201f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb620146190 0x7fb620146610 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 --2- 172.21.15.76:0/2281886011 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb6040039c0 0x7fb6201e9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 >> 172.21.15.76:0/2281886011 conn(0x7fb6200b1f90 msgr2=0x7fb6200cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 shutdown_connections 2024-09-19T04:18:37.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.333+0000 7fb625401640 1 -- 172.21.15.76:0/2281886011 wait complete. 2024-09-19T04:18:37.351 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0' 2024-09-19T04:18:37.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.469+0000 7fd64426c640 1 Processor -- start 2024-09-19T04:18:37.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.469+0000 7fd64426c640 1 -- start start 2024-09-19T04:18:37.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd64426c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd63c070020 0x7fd63c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd64426c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c15f1d0 0x7fd63c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd64426c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd63c0aede0 con 0x7fd63c1520a0 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd64426c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd63c152480 con 0x7fd63c070020 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd64426c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c15fa10 con 0x7fd63c15f1d0 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd63c070020 0x7fd63c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd63c070020 0x7fd63c15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51348/0 (socket says 172.21.15.76:51348) 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 -- 172.21.15.76:0/857192821 learned_addr learned my addr 172.21.15.76:0/857192821 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd642809640 1 --2- 172.21.15.76:0/857192821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c15f1d0 0x7fd63c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 -- 172.21.15.76:0/857192821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c15f1d0 msgr2=0x7fd63c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 --2- 172.21.15.76:0/857192821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c15f1d0 0x7fd63c0adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 -- 172.21.15.76:0/857192821 >> v1:172.21.15.116:6789/0 conn(0x7fd63c1520a0 legacy=0x7fd63c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:37.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 -- 172.21.15.76:0/857192821 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c074250 con 0x7fd63c070020 2024-09-19T04:18:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641807640 1 --2- 172.21.15.76:0/857192821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c070020 0x7fd63c15ec90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd638009870 tx=0x7fd63801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff8a6c93b3a17aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641006640 1 -- 172.21.15.76:0/857192821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380aa610 con 0x7fd63c070020 2024-09-19T04:18:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641006640 1 -- 172.21.15.76:0/857192821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6380af070 con 0x7fd63c070020 2024-09-19T04:18:37.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.473+0000 7fd641006640 1 -- 172.21.15.76:0/857192821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380aa9d0 con 0x7fd63c070020 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- 172.21.15.76:0/857192821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c070020 msgr2=0x7fd63c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 --2- 172.21.15.76:0/857192821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c070020 0x7fd63c15ec90 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd638009870 tx=0x7fd63801c4f0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- 172.21.15.76:0/857192821 shutdown_connections 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 --2- 172.21.15.76:0/857192821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c15f1d0 0x7fd63c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 --2- 172.21.15.76:0/857192821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c070020 0x7fd63c15ec90 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- 172.21.15.76:0/857192821 >> 172.21.15.76:0/857192821 conn(0x7fd63c0b1f90 msgr2=0x7fd63c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- 172.21.15.76:0/857192821 shutdown_connections 2024-09-19T04:18:37.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- 172.21.15.76:0/857192821 wait complete. 2024-09-19T04:18:37.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 Processor -- start 2024-09-19T04:18:37.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- start start 2024-09-19T04:18:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c070020 0x7fd63c15ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c1520a0 0x7fd63c165120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd63c15f1d0 0x7fd63c165660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c15f5e0 con 0x7fd63c1520a0 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c073d30 con 0x7fd63c15f1d0 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd63c15f760 con 0x7fd63c070020 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c1520a0 0x7fd63c165120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c1520a0 0x7fd63c165120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51364/0 (socket says 172.21.15.76:51364) 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 -- 172.21.15.76:0/250181713 learned_addr learned my addr 172.21.15.76:0/250181713 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd642008640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c070020 0x7fd63c15ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd642809640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd63c15f1d0 0x7fd63c165660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 -- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c070020 msgr2=0x7fd63c15ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c070020 0x7fd63c15ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 -- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd63c15f1d0 msgr2=0x7fd63c165660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd63c15f1d0 0x7fd63c165660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 -- 172.21.15.76:0/250181713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c15b060 con 0x7fd63c1520a0 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd642008640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c070020 0x7fd63c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd641807640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c1520a0 0x7fd63c165120 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd6380ac040 tx=0x7fd63801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd632ffd640 1 -- 172.21.15.76:0/250181713 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380bd5f0 con 0x7fd63c1520a0 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd632ffd640 1 -- 172.21.15.76:0/250181713 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6380bd790 con 0x7fd63c1520a0 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c167d00 con 0x7fd63c1520a0 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd632ffd640 1 -- 172.21.15.76:0/250181713 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6380bdb80 con 0x7fd63c1520a0 2024-09-19T04:18:37.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.477+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c1682c0 con 0x7fd63c1520a0 2024-09-19T04:18:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.481+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c15ec90 con 0x7fd63c1520a0 2024-09-19T04:18:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.481+0000 7fd632ffd640 1 -- 172.21.15.76:0/250181713 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd6380b33f0 con 0x7fd63c1520a0 2024-09-19T04:18:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.481+0000 7fd632ffd640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd6200678c0 0x7fd620069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.481+0000 7fd632ffd640 1 -- 172.21.15.76:0/250181713 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7fd6381208c0 con 0x7fd63c1520a0 2024-09-19T04:18:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.481+0000 7fd642008640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd6200678c0 0x7fd620069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:37.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.481+0000 7fd642008640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd6200678c0 0x7fd620069d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0098b0 tx=0x7fd62c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:37.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.485+0000 7fd632ffd640 1 -- 172.21.15.76:0/250181713 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd6380c3060 con 0x7fd63c1520a0 2024-09-19T04:18:37.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.705+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]} v 0) -- 0x7fd63c15ec90 con 0x7fd63c1520a0 2024-09-19T04:18:37.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.977+0000 7fd632ffd640 1 -- 172.21.15.76:0/250181713 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_non_mds0"]}]=0 v266) ==== 141+0+0 (secure 0 0 0) 0x7fd6380a7070 con 0x7fd63c1520a0 2024-09-19T04:18:37.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.977+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd6200678c0 msgr2=0x7fd620069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.977+0000 7fd64426c640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd6200678c0 0x7fd620069d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd62c0098b0 tx=0x7fd62c008040 comp rx=0 tx=0).stop 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.977+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c1520a0 msgr2=0x7fd63c165120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.977+0000 7fd64426c640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c1520a0 0x7fd63c165120 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fd6380ac040 tx=0x7fd63801ad30 comp rx=0 tx=0).stop 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 shutdown_connections 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd6200678c0 0x7fd620069d80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd63c15f1d0 0x7fd63c165660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd63c1520a0 0x7fd63c165120 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 --2- 172.21.15.76:0/250181713 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd63c070020 0x7fd63c15ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 >> 172.21.15.76:0/250181713 conn(0x7fd63c0b1f90 msgr2=0x7fd63c06f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 shutdown_connections 2024-09-19T04:18:37.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:37.981+0000 7fd64426c640 1 -- 172.21.15.76:0/250181713 wait complete. 2024-09-19T04:18:38.000 INFO:tasks.cephfs_test_runner:test_scrub_non_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-19T04:18:38.001 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-19T04:18:38.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:38.123 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.121+0000 7f9b51068640 1 Processor -- start 2024-09-19T04:18:38.124 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.121+0000 7f9b51068640 1 -- start start 2024-09-19T04:18:38.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b51068640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9b4c06c3d0 0x7f9b4c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.126 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b51068640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b4c073740 con 0x7f9b4c1627d0 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b51068640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b4c0b4630 con 0x7f9b4c0b40c0 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b51068640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9b4c0b6050 con 0x7f9b4c06c3d0 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9b4c1627d0 0x7f9b4c163f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59474/0 (socket says 172.21.15.76:59474) 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4ad9d640 1 -- 172.21.15.76:0/374029508 learned_addr learned my addr 172.21.15.76:0/374029508 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1906630863 0 0) 0x7f9b4c073740 con 0x7f9b4c1627d0 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b34003530 con 0x7f9b4c1627d0 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4b59e640 1 --2- 172.21.15.76:0/374029508 >> v2:172.21.15.116:3300/0 conn(0x7f9b4c06c3d0 0x7f9b4c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165408727 0 0) 0x7f9b34003530 con 0x7f9b4c1627d0 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 >> v2:172.21.15.116:3300/0 conn(0x7f9b4c06c3d0 msgr2=0x7f9b4c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 --2- 172.21.15.76:0/374029508 >> v2:172.21.15.116:3300/0 conn(0x7f9b4c06c3d0 0x7f9b4c09aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 >> v1:172.21.15.116:6789/0 conn(0x7f9b4c0b40c0 legacy=0x7f9b4c06bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.127 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c0b6330 con 0x7f9b4c1627d0 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f9b44001a00 con 0x7f9b4c1627d0 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 >> v1:172.21.15.76:6789/0 conn(0x7f9b4c1627d0 legacy=0x7f9b4c163f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b340039c0 0x7f9b34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b34004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b340051d0 0x7f9b3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c0b6050 con 0x7f9b34004810 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3400c100 con 0x7f9b340039c0 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3400c300 con 0x7f9b340051d0 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4a59c640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b34004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4ad9d640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b340039c0 0x7f9b34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4b59e640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b340051d0 0x7f9b3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.128 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4a59c640 1 -- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b340051d0 msgr2=0x7f9b3400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4a59c640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b340051d0 0x7f9b3400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4a59c640 1 -- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b340039c0 msgr2=0x7f9b34003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4a59c640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b340039c0 0x7f9b34003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4a59c640 1 -- 172.21.15.76:0/374029508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b34003530 con 0x7f9b34004810 2024-09-19T04:18:38.129 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4b59e640 1 -- 172.21.15.76:0/374029508 reap_dead start 2024-09-19T04:18:38.130 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.125+0000 7f9b4a59c640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b34004c90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9b40007fd0 tx=0x7f9b4001c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293411c88e276b05 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400b0070 con 0x7f9b34004810 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b400ab620 con 0x7f9b34004810 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b49d9b640 1 -- 172.21.15.76:0/374029508 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b400039b0 con 0x7f9b34004810 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- 172.21.15.76:0/374029508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 msgr2=0x7f9b34004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b34004c90 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9b40007fd0 tx=0x7f9b4001c2c0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- 172.21.15.76:0/374029508 shutdown_connections 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 --2- 172.21.15.76:0/374029508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b34004c90 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- 172.21.15.76:0/374029508 >> 172.21.15.76:0/374029508 conn(0x7f9b4c0b6800 msgr2=0x7f9b4c0b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- 172.21.15.76:0/374029508 shutdown_connections 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- 172.21.15.76:0/374029508 wait complete. 2024-09-19T04:18:38.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 Processor -- start 2024-09-19T04:18:38.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- start start 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b4c157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b4c154e20 0x7f9b4c1545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b4c0ae0a0 0x7f9b4c0700d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c09ab80 con 0x7f9b34004810 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9b4c09aa00 con 0x7f9b4c0ae0a0 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9b4c09ad00 con 0x7f9b4c154e20 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b4c157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b4c157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51376/0 (socket says 172.21.15.76:51376) 2024-09-19T04:18:38.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 -- 172.21.15.76:0/696461692 learned_addr learned my addr 172.21.15.76:0/696461692 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:38.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4a59c640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b4c154e20 0x7f9b4c1545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4b59e640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b4c0ae0a0 0x7f9b4c0700d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 -- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b4c154e20 msgr2=0x7f9b4c1545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b4c154e20 0x7f9b4c1545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 -- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b4c0ae0a0 msgr2=0x7f9b4c0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b4c0ae0a0 0x7f9b4c0700d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 -- 172.21.15.76:0/696461692 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b4c0b0b50 con 0x7f9b34004810 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4b59e640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b4c0ae0a0 0x7f9b4c0700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b4ad9d640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b4c157da0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9b44007a00 tx=0x7f9b440115c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b337fe640 1 -- 172.21.15.76:0/696461692 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4401c070 con 0x7f9b34004810 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b337fe640 1 -- 172.21.15.76:0/696461692 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b44004d70 con 0x7f9b34004810 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b4c0b0df0 con 0x7f9b34004810 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b337fe640 1 -- 172.21.15.76:0/696461692 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44010040 con 0x7f9b34004810 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.129+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b4c0ae520 con 0x7f9b34004810 2024-09-19T04:18:38.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.133+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b4c06bcc0 con 0x7f9b34004810 2024-09-19T04:18:38.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.133+0000 7f9b337fe640 1 -- 172.21.15.76:0/696461692 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9b4401f070 con 0x7f9b34004810 2024-09-19T04:18:38.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.133+0000 7f9b337fe640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9b380678c0 0x7f9b38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.133+0000 7f9b337fe640 1 -- 172.21.15.76:0/696461692 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f9b44088d20 con 0x7f9b34004810 2024-09-19T04:18:38.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.133+0000 7f9b4a59c640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9b380678c0 0x7f9b38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.133+0000 7f9b4a59c640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9b380678c0 0x7f9b38069d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9b400ad000 tx=0x7f9b40003770 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.137+0000 7f9b337fe640 1 -- 172.21.15.76:0/696461692 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9b4408d070 con 0x7f9b34004810 2024-09-19T04:18:38.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.297+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b4c06e7c0 con 0x7f9b34004810 2024-09-19T04:18:38.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b337fe640 1 -- 172.21.15.76:0/696461692 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+13590 (secure 0 0 0) 0x7f9b44059a50 con 0x7f9b34004810 2024-09-19T04:18:38.302 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:38.303 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":83,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:36.967824+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6835/3224374251":"2024-09-20T04:18:33.771333+0000","172.21.15.76:6835/603375312":"2024-09-20T04:18:33.771372+0000","172.21.15.76:6834/603375312":"2024-09-20T04:18:33.771372+0000","172.21.15.76:6837/3569622848":"2024-09-20T04:18:33.771395+0000","172.21.15.116:6834/3224374251":"2024-09-20T04:18:33.771333+0000","172.21.15.76:6836/3569622848":"2024-09-20T04:18:33.771395+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:38.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9b380678c0 msgr2=0x7f9b38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9b380678c0 0x7f9b38069d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9b400ad000 tx=0x7f9b40003770 comp rx=0 tx=0).stop 2024-09-19T04:18:38.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 msgr2=0x7f9b4c157da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b4c157da0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9b44007a00 tx=0x7f9b440115c0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 shutdown_connections 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9b380678c0 0x7f9b38069d80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b4c0ae0a0 0x7f9b4c0700d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9b4c154e20 0x7f9b4c1545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 --2- 172.21.15.76:0/696461692 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9b34004810 0x7f9b4c157da0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 >> 172.21.15.76:0/696461692 conn(0x7f9b4c0b6800 msgr2=0x7f9b4c160ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 shutdown_connections 2024-09-19T04:18:38.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.301+0000 7f9b51068640 1 -- 172.21.15.76:0/696461692 wait complete. 2024-09-19T04:18:38.320 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:13.812655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:18:38.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:18:38.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.441+0000 7fd7429e7640 1 Processor -- start 2024-09-19T04:18:38.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.441+0000 7fd7429e7640 1 -- start start 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd7429e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd73c15bec0 0x7fd73c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd7429e7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd73c0af470 0x7fd73c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd7429e7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73c0b3290 con 0x7fd73c15c2a0 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd7429e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0b3570 con 0x7fd73c15bec0 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd7429e7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd73c0b3850 con 0x7fd73c0af470 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd73bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd73c15bec0 0x7fd73c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd73bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd73c15bec0 0x7fd73c0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51384/0 (socket says 172.21.15.76:51384) 2024-09-19T04:18:38.446 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd73bfff640 1 -- 172.21.15.76:0/3782589335 learned_addr learned my addr 172.21.15.76:0/3782589335 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:38.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd740f84640 1 --2- 172.21.15.76:0/3782589335 >> v2:172.21.15.116:3300/0 conn(0x7fd73c0af470 0x7fd73c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd73bfff640 1 -- 172.21.15.76:0/3782589335 >> v2:172.21.15.116:3300/0 conn(0x7fd73c0af470 msgr2=0x7fd73c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd73bfff640 1 --2- 172.21.15.76:0/3782589335 >> v2:172.21.15.116:3300/0 conn(0x7fd73c0af470 0x7fd73c06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd73bfff640 1 -- 172.21.15.76:0/3782589335 >> v1:172.21.15.116:6789/0 conn(0x7fd73c15c2a0 legacy=0x7fd73c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd73bfff640 1 -- 172.21.15.76:0/3782589335 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c0b3b30 con 0x7fd73c15bec0 2024-09-19T04:18:38.447 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.445+0000 7fd740f84640 1 --2- 172.21.15.76:0/3782589335 >> v2:172.21.15.116:3300/0 conn(0x7fd73c0af470 0x7fd73c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:38.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd73bfff640 1 --2- 172.21.15.76:0/3782589335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c15bec0 0x7fd73c0ad200 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0b2fb0 tx=0x7fd72401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdf3df5ab3feb13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.449 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd73affd640 1 -- 172.21.15.76:0/3782589335 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240ab600 con 0x7fd73c15bec0 2024-09-19T04:18:38.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd73affd640 1 -- 172.21.15.76:0/3782589335 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7240b0070 con 0x7fd73c15bec0 2024-09-19T04:18:38.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd73affd640 1 -- 172.21.15.76:0/3782589335 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240ab9c0 con 0x7fd73c15bec0 2024-09-19T04:18:38.450 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 -- 172.21.15.76:0/3782589335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c15bec0 msgr2=0x7fd73c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 --2- 172.21.15.76:0/3782589335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c15bec0 0x7fd73c0ad200 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0b2fb0 tx=0x7fd72401c4f0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 -- 172.21.15.76:0/3782589335 shutdown_connections 2024-09-19T04:18:38.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 --2- 172.21.15.76:0/3782589335 >> v2:172.21.15.116:3300/0 conn(0x7fd73c0af470 0x7fd73c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 --2- 172.21.15.76:0/3782589335 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c15bec0 0x7fd73c0ad200 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 -- 172.21.15.76:0/3782589335 >> 172.21.15.76:0/3782589335 conn(0x7fd73c0b1f90 msgr2=0x7fd73c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:38.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 -- 172.21.15.76:0/3782589335 shutdown_connections 2024-09-19T04:18:38.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 -- 172.21.15.76:0/3782589335 wait complete. 2024-09-19T04:18:38.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 Processor -- start 2024-09-19T04:18:38.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.449+0000 7fd7429e7640 1 -- start start 2024-09-19T04:18:38.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c0af470 0x7fd73c1575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd73c15bec0 0x7fd73c1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd73c15c2a0 0x7fd73c150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c06ed50 con 0x7fd73c0af470 2024-09-19T04:18:38.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd73c0ad740 con 0x7fd73c15c2a0 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd73c06eed0 con 0x7fd73c15bec0 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c0af470 0x7fd73c1575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c0af470 0x7fd73c1575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51394/0 (socket says 172.21.15.76:51394) 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 -- 172.21.15.76:0/2537789207 learned_addr learned my addr 172.21.15.76:0/2537789207 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd740f84640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd73c15c2a0 0x7fd73c150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73b7fe640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd73c15bec0 0x7fd73c1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 -- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd73c15bec0 msgr2=0x7fd73c1f5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd73c15bec0 0x7fd73c1f5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 -- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd73c15c2a0 msgr2=0x7fd73c150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd73c15c2a0 0x7fd73c150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 -- 172.21.15.76:0/2537789207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd73c1f5140 con 0x7fd73c0af470 2024-09-19T04:18:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73bfff640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c0af470 0x7fd73c1575f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd7240ad040 tx=0x7fd72401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd738ff9640 1 -- 172.21.15.76:0/2537789207 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240be5f0 con 0x7fd73c0af470 2024-09-19T04:18:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd738ff9640 1 -- 172.21.15.76:0/2537789207 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7240be790 con 0x7fd73c0af470 2024-09-19T04:18:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73c157d40 con 0x7fd73c0af470 2024-09-19T04:18:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd738ff9640 1 -- 172.21.15.76:0/2537789207 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7240beb60 con 0x7fd73c0af470 2024-09-19T04:18:38.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd73c1f5400 con 0x7fd73c0af470 2024-09-19T04:18:38.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd73c06f230 con 0x7fd73c0af470 2024-09-19T04:18:38.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd738ff9640 1 -- 172.21.15.76:0/2537789207 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd7240aa070 con 0x7fd73c0af470 2024-09-19T04:18:38.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd738ff9640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd71c0678c0 0x7fd71c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd738ff9640 1 -- 172.21.15.76:0/2537789207 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7fd7241205b0 con 0x7fd73c0af470 2024-09-19T04:18:38.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73b7fe640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd71c0678c0 0x7fd71c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.453+0000 7fd73b7fe640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd71c0678c0 0x7fd71c069d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd73c15b280 tx=0x7fd72c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.457+0000 7fd738ff9640 1 -- 172.21.15.76:0/2537789207 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd7240c4060 con 0x7fd73c0af470 2024-09-19T04:18:38.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.621+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd73c06fcd0 con 0x7fd73c0af470 2024-09-19T04:18:38.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.621+0000 7fd738ff9640 1 -- 172.21.15.76:0/2537789207 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) ==== 93+0+10 (secure 0 0 0) 0x7fd724016110 con 0x7fd73c0af470 2024-09-19T04:18:38.624 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-09-19T04:18:38.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd71c0678c0 msgr2=0x7fd71c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd71c0678c0 0x7fd71c069d80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd73c15b280 tx=0x7fd72c009040 comp rx=0 tx=0).stop 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c0af470 msgr2=0x7fd73c1575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c0af470 0x7fd73c1575f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7fd7240ad040 tx=0x7fd72401ad30 comp rx=0 tx=0).stop 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 shutdown_connections 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd71c0678c0 0x7fd71c069d80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd73c15c2a0 0x7fd73c150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd73c15bec0 0x7fd73c1f5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 --2- 172.21.15.76:0/2537789207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd73c0af470 0x7fd73c1575f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 >> 172.21.15.76:0/2537789207 conn(0x7fd73c0b1f90 msgr2=0x7fd73c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 shutdown_connections 2024-09-19T04:18:38.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.625+0000 7fd7429e7640 1 -- 172.21.15.76:0/2537789207 wait complete. 2024-09-19T04:18:38.643 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-09-19T04:18:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c80c1640 1 Processor -- start 2024-09-19T04:18:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c80c1640 1 -- start start 2024-09-19T04:18:38.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c80c1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 0x7fb7c00d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c80c1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c00d32b0 con 0x7fb7c00d2e90 2024-09-19T04:18:38.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c80c1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7c00cfad0 con 0x7fb7c00cf620 2024-09-19T04:18:38.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c80c1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c00d0300 con 0x7fb7c0161a60 2024-09-19T04:18:38.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c565c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb7c00d2e90 0x7fb7c00cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59488/0 (socket says 172.21.15.76:59488) 2024-09-19T04:18:38.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c565c640 1 -- 172.21.15.76:0/2500741599 learned_addr learned my addr 172.21.15.76:0/2500741599 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c5e5d640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 0x7fb7c00d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357250603 0 0) 0x7fb7c00d32b0 con 0x7fb7c00d2e90 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7ac003530 con 0x7fb7c00d2e90 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1013891599 0 0) 0x7fb7ac003530 con 0x7fb7c00d2e90 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 msgr2=0x7fb7c00d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 0x7fb7c00d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 >> v1:172.21.15.116:6789/0 conn(0x7fb7c00cf620 legacy=0x7fb7c006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c00b0f60 con 0x7fb7c00d2e90 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb7b0001680 con 0x7fb7c00d2e90 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 >> v1:172.21.15.76:6789/0 conn(0x7fb7c00d2e90 legacy=0x7fb7c00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 0x7fb7ac0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7ac003f70 0x7fb7ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7ac004d50 0x7fb7ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c00d0300 con 0x7fb7ac004d50 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7ac00bb60 con 0x7fb7ac003f70 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb7ac00bd60 con 0x7fb7c0161a60 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c665e640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7ac004d50 0x7fb7ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c565c640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7ac003f70 0x7fb7ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c5e5d640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 0x7fb7ac0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c665e640 1 -- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 msgr2=0x7fb7ac0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c665e640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c0161a60 0x7fb7ac0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c665e640 1 -- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7ac003f70 msgr2=0x7fb7ac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c665e640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7ac003f70 0x7fb7ac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.761+0000 7fb7c665e640 1 -- 172.21.15.76:0/2500741599 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7ac003530 con 0x7fb7ac004d50 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7ac004d50 0x7fb7ac0051d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001e70 tx=0x7fb7b801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91e8d0985046281 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8001a90 con 0x7fb7ac004d50 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b8003b80 con 0x7fb7ac004d50 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c4e5b640 1 -- 172.21.15.76:0/2500741599 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8005080 con 0x7fb7ac004d50 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2500741599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7ac004d50 msgr2=0x7fb7ac0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 --2- 172.21.15.76:0/2500741599 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7ac004d50 0x7fb7ac0051d0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8001e70 tx=0x7fb7b801c780 comp rx=0 tx=0).stop 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 -- 172.21.15.76:0/2500741599 reap_dead start 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2500741599 shutdown_connections 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2500741599 >> 172.21.15.76:0/2500741599 conn(0x7fb7c008bf90 msgr2=0x7fb7c009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2500741599 shutdown_connections 2024-09-19T04:18:38.767 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2500741599 wait complete. 2024-09-19T04:18:38.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 Processor -- start 2024-09-19T04:18:38.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- start start 2024-09-19T04:18:38.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7c015f820 0x7fb7c00cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c00a1c10 0x7fb7c00c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7c00ca360 0x7fb7c01f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c006ceb0 con 0x7fb7c00ca360 2024-09-19T04:18:38.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7c00d0020 con 0x7fb7c015f820 2024-09-19T04:18:38.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c80c1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb7c006d190 con 0x7fb7c00a1c10 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7c00ca360 0x7fb7c01f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7c00ca360 0x7fb7c01f7cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51416/0 (socket says 172.21.15.76:51416) 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 -- 172.21.15.76:0/2421732211 learned_addr learned my addr 172.21.15.76:0/2421732211 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c565c640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c00a1c10 0x7fb7c00c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c5e5d640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7c015f820 0x7fb7c00cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 -- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c00a1c10 msgr2=0x7fb7c00c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c00a1c10 0x7fb7c00c9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 -- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7c015f820 msgr2=0x7fb7c00cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7c015f820 0x7fb7c00cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 -- 172.21.15.76:0/2421732211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7c00cbdb0 con 0x7fb7c00ca360 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.765+0000 7fb7c665e640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7c00ca360 0x7fb7c01f7cf0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8098080 tx=0x7fb7ac005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7b67fc640 1 -- 172.21.15.76:0/2421732211 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8007dc0 con 0x7fb7c00ca360 2024-09-19T04:18:38.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7b67fc640 1 -- 172.21.15.76:0/2421732211 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7b80b8070 con 0x7fb7c00ca360 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7c00cc090 con 0x7fb7c00ca360 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7b67fc640 1 -- 172.21.15.76:0/2421732211 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8001a90 con 0x7fb7c00ca360 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7c00cc650 con 0x7fb7c00ca360 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7c00cf3d0 con 0x7fb7c00ca360 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7b67fc640 1 -- 172.21.15.76:0/2421732211 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb7b8006b60 con 0x7fb7c00ca360 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7b67fc640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7a40678c0 0x7fb7a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7c5e5d640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7a40678c0 0x7fb7a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7b67fc640 1 -- 172.21.15.76:0/2421732211 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7fb7b811fae0 con 0x7fb7c00ca360 2024-09-19T04:18:38.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.769+0000 7fb7c5e5d640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7a40678c0 0x7fb7a4069d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009970 tx=0x7fb7bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:38.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.773+0000 7fb7b67fc640 1 -- 172.21.15.76:0/2421732211 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb7b80ad020 con 0x7fb7c00ca360 2024-09-19T04:18:38.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:38.981+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7fb7c00b4130 con 0x7fb7c00ca360 2024-09-19T04:18:39.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.985+0000 7fb7b67fc640 1 -- 172.21.15.76:0/2421732211 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v268) ==== 157+0+0 (secure 0 0 0) 0x7fb7b8016490 con 0x7fb7c00ca360 2024-09-19T04:18:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.989+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7a40678c0 msgr2=0x7fb7a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:39.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.989+0000 7fb7c80c1640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7a40678c0 0x7fb7a4069d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc009970 tx=0x7fb7bc008040 comp rx=0 tx=0).stop 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.989+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7c00ca360 msgr2=0x7fb7c01f7cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.989+0000 7fb7c80c1640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7c00ca360 0x7fb7c01f7cf0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8098080 tx=0x7fb7ac005840 comp rx=0 tx=0).stop 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.989+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 shutdown_connections 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.989+0000 7fb7c80c1640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7a40678c0 0x7fb7a4069d80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.993+0000 7fb7c80c1640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7c00ca360 0x7fb7c01f7cf0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.993+0000 7fb7c80c1640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7c00a1c10 0x7fb7c00c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.993+0000 7fb7c80c1640 1 --2- 172.21.15.76:0/2421732211 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7c015f820 0x7fb7c00cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.993+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 >> 172.21.15.76:0/2421732211 conn(0x7fb7c008bf90 msgr2=0x7fb7c00b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.993+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 shutdown_connections 2024-09-19T04:18:39.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:39.993+0000 7fb7c80c1640 1 -- 172.21.15.76:0/2421732211 wait complete. 2024-09-19T04:18:40.012 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T04:18:40.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.137+0000 7ffb366a0640 1 Processor -- start 2024-09-19T04:18:40.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb366a0640 1 -- start start 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb366a0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb30161a60 0x7ffb300d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb366a0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffb300d2e90 0x7ffb300cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb366a0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb300d32b0 con 0x7ffb300cf450 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb366a0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ffb300d0300 con 0x7ffb30161a60 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb366a0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ffb3006e6c0 con 0x7ffb300d2e90 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb30161a60 0x7ffb300d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ffb30161a60 0x7ffb300d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51428/0 (socket says 172.21.15.76:51428) 2024-09-19T04:18:40.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 -- 172.21.15.76:0/1416944983 learned_addr learned my addr 172.21.15.76:0/1416944983 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:40.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2f7fe640 1 --2- 172.21.15.76:0/1416944983 >> v2:172.21.15.116:3300/0 conn(0x7ffb300d2e90 0x7ffb300cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 -- 172.21.15.76:0/1416944983 >> v2:172.21.15.116:3300/0 conn(0x7ffb300d2e90 msgr2=0x7ffb300cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:40.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 --2- 172.21.15.76:0/1416944983 >> v2:172.21.15.116:3300/0 conn(0x7ffb300d2e90 0x7ffb300cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 -- 172.21.15.76:0/1416944983 >> v1:172.21.15.116:6789/0 conn(0x7ffb300cf450 legacy=0x7ffb300d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:40.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 -- 172.21.15.76:0/1416944983 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb300b0f60 con 0x7ffb30161a60 2024-09-19T04:18:40.143 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2f7fe640 1 --2- 172.21.15.76:0/1416944983 >> v2:172.21.15.116:3300/0 conn(0x7ffb300d2e90 0x7ffb300cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:40.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.141+0000 7ffb2ffff640 1 --2- 172.21.15.76:0/1416944983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30161a60 0x7ffb300d2950 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ffb3008d850 tx=0x7ffb2001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82f18eed76352cc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:40.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2effd640 1 -- 172.21.15.76:0/1416944983 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb300d1930 con 0x7ffb30161a60 2024-09-19T04:18:40.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2effd640 1 -- 172.21.15.76:0/1416944983 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb200b0070 con 0x7ffb30161a60 2024-09-19T04:18:40.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2effd640 1 -- 172.21.15.76:0/1416944983 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200ab840 con 0x7ffb30161a60 2024-09-19T04:18:40.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- 172.21.15.76:0/1416944983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30161a60 msgr2=0x7ffb300d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 --2- 172.21.15.76:0/1416944983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30161a60 0x7ffb300d2950 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ffb3008d850 tx=0x7ffb2001c420 comp rx=0 tx=0).stop 2024-09-19T04:18:40.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- 172.21.15.76:0/1416944983 shutdown_connections 2024-09-19T04:18:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 --2- 172.21.15.76:0/1416944983 >> v2:172.21.15.116:3300/0 conn(0x7ffb300d2e90 0x7ffb300cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 --2- 172.21.15.76:0/1416944983 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb30161a60 0x7ffb300d2950 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- 172.21.15.76:0/1416944983 >> 172.21.15.76:0/1416944983 conn(0x7ffb3008bf90 msgr2=0x7ffb3009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- 172.21.15.76:0/1416944983 shutdown_connections 2024-09-19T04:18:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- 172.21.15.76:0/1416944983 wait complete. 2024-09-19T04:18:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 Processor -- start 2024-09-19T04:18:40.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- start start 2024-09-19T04:18:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300cf450 0x7ffb300cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb300d2e90 0x7ffb30154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb30161a60 0x7ffb301540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb300cf8c0 con 0x7ffb300cf450 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3006d190 con 0x7ffb30161a60 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb300cfa40 con 0x7ffb300d2e90 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300cf450 0x7ffb300cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300cf450 0x7ffb300cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51440/0 (socket says 172.21.15.76:51440) 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 -- 172.21.15.76:0/2411868142 learned_addr learned my addr 172.21.15.76:0/2411868142 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 -- 172.21.15.76:0/2411868142 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb300d2e90 msgr2=0x7ffb30154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb300d2e90 0x7ffb30154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 -- 172.21.15.76:0/2411868142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb30161a60 msgr2=0x7ffb301540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb30161a60 0x7ffb301540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 -- 172.21.15.76:0/2411868142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb300cbb00 con 0x7ffb300cf450 2024-09-19T04:18:40.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2ffff640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300cf450 0x7ffb300cb440 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ad040 tx=0x7ffb2001ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2cff9640 1 -- 172.21.15.76:0/2411868142 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200abd10 con 0x7ffb300cf450 2024-09-19T04:18:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2cff9640 1 -- 172.21.15.76:0/2411868142 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb200be3f0 con 0x7ffb300cf450 2024-09-19T04:18:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb300cbde0 con 0x7ffb300cf450 2024-09-19T04:18:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb2cff9640 1 -- 172.21.15.76:0/2411868142 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb200be7e0 con 0x7ffb300cf450 2024-09-19T04:18:40.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.145+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb300c7e80 con 0x7ffb300cf450 2024-09-19T04:18:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.149+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb300cef10 con 0x7ffb300cf450 2024-09-19T04:18:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.149+0000 7ffb2cff9640 1 -- 172.21.15.76:0/2411868142 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb200c7440 con 0x7ffb300cf450 2024-09-19T04:18:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.149+0000 7ffb2cff9640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb1c0678c0 0x7ffb1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.149+0000 7ffb2cff9640 1 -- 172.21.15.76:0/2411868142 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7ffb20120060 con 0x7ffb300cf450 2024-09-19T04:18:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.149+0000 7ffb2f7fe640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb1c0678c0 0x7ffb1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.149+0000 7ffb2f7fe640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb1c0678c0 0x7ffb1c069d80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb3006dab0 tx=0x7ffb24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:40.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.153+0000 7ffb2cff9640 1 -- 172.21.15.76:0/2411868142 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffb200160d0 con 0x7ffb300cf450 2024-09-19T04:18:40.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.301+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ffb3006d800 con 0x7ffb300cf450 2024-09-19T04:18:40.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb2cff9640 1 -- 172.21.15.76:0/2411868142 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7ffb200f0cf0 con 0x7ffb300cf450 2024-09-19T04:18:40.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb1c0678c0 msgr2=0x7ffb1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb1c0678c0 0x7ffb1c069d80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ffb3006dab0 tx=0x7ffb24008040 comp rx=0 tx=0).stop 2024-09-19T04:18:40.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300cf450 msgr2=0x7ffb300cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300cf450 0x7ffb300cb440 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7ffb200ad040 tx=0x7ffb2001ac60 comp rx=0 tx=0).stop 2024-09-19T04:18:40.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 shutdown_connections 2024-09-19T04:18:40.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb1c0678c0 0x7ffb1c069d80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb30161a60 0x7ffb301540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb300d2e90 0x7ffb30154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 --2- 172.21.15.76:0/2411868142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb300cf450 0x7ffb300cb440 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 >> 172.21.15.76:0/2411868142 conn(0x7ffb3008bf90 msgr2=0x7ffb3006dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:40.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 shutdown_connections 2024-09-19T04:18:40.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.309+0000 7ffb366a0640 1 -- 172.21.15.76:0/2411868142 wait complete. 2024-09-19T04:18:40.327 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:18:40.327 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:18:40.386 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:18:40.455 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:18:40.455 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:18:40.455 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:40.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:18:40.460 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:18:40.460 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:40.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:18:40.467 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:18:40.467 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:40.508 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:18:40.509 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:18:40.509 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:18:40.514 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:18:40.514 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T04:18:40.514 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:40.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 Processor -- start 2024-09-19T04:18:40.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 -- start start 2024-09-19T04:18:40.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30581520a0 0x7f305806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 0x7f305815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f305815f1d0 0x7f3058071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3058152480 con 0x7f305815f1d0 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3058070250 con 0x7f30581520a0 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f305d3ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f30580aede0 con 0x7f305806fe50 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f305815f1d0 0x7f3058071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f305815f1d0 0x7f3058071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51450/0 (socket says 172.21.15.76:51450) 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 -- 172.21.15.76:0/3151946876 learned_addr learned my addr 172.21.15.76:0/3151946876 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30567fc640 1 --2- 172.21.15.76:0/3151946876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 0x7f305815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f3056ffd640 1 --2- 172.21.15.76:0/3151946876 >> v2:172.21.15.116:3300/0 conn(0x7f30581520a0 0x7f305806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 -- 172.21.15.76:0/3151946876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 msgr2=0x7f305815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 --2- 172.21.15.76:0/3151946876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 0x7f305815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 -- 172.21.15.76:0/3151946876 >> v2:172.21.15.116:3300/0 conn(0x7f30581520a0 msgr2=0x7f305806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 --2- 172.21.15.76:0/3151946876 >> v2:172.21.15.116:3300/0 conn(0x7f30581520a0 0x7f305806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f30577fe640 1 -- 172.21.15.76:0/3151946876 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305815fa10 con 0x7f305815f1d0 2024-09-19T04:18:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.677+0000 7f3056ffd640 1 --2- 172.21.15.76:0/3151946876 >> v2:172.21.15.116:3300/0 conn(0x7f30581520a0 0x7f305806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30577fe640 1 --2- 172.21.15.76:0/3151946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305815f1d0 0x7f3058071f10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3058071710 tx=0x7f305001e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f550764515e5b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f3055ffb640 1 -- 172.21.15.76:0/3151946876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305000b2d0 con 0x7f305815f1d0 2024-09-19T04:18:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f3055ffb640 1 -- 172.21.15.76:0/3151946876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f305000b470 con 0x7f305815f1d0 2024-09-19T04:18:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f3055ffb640 1 -- 172.21.15.76:0/3151946876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30500a79c0 con 0x7f305815f1d0 2024-09-19T04:18:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- 172.21.15.76:0/3151946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305815f1d0 msgr2=0x7f3058071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 --2- 172.21.15.76:0/3151946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305815f1d0 0x7f3058071f10 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f3058071710 tx=0x7f305001e910 comp rx=0 tx=0).stop 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- 172.21.15.76:0/3151946876 shutdown_connections 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 --2- 172.21.15.76:0/3151946876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f305815f1d0 0x7f3058071f10 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 --2- 172.21.15.76:0/3151946876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 0x7f305815ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 --2- 172.21.15.76:0/3151946876 >> v2:172.21.15.116:3300/0 conn(0x7f30581520a0 0x7f305806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- 172.21.15.76:0/3151946876 >> 172.21.15.76:0/3151946876 conn(0x7f30580b1f90 msgr2=0x7f30580b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- 172.21.15.76:0/3151946876 shutdown_connections 2024-09-19T04:18:40.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- 172.21.15.76:0/3151946876 wait complete. 2024-09-19T04:18:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 Processor -- start 2024-09-19T04:18:40.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- start start 2024-09-19T04:18:40.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 0x7f305814b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581520a0 0x7f3058149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f305815f1d0 0x7f305814c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3058073d30 con 0x7f30581520a0 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3058073a50 con 0x7f305815f1d0 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f305815f680 con 0x7f305806fe50 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581520a0 0x7f3058149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581520a0 0x7f3058149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51452/0 (socket says 172.21.15.76:51452) 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 -- 172.21.15.76:0/1534043311 learned_addr learned my addr 172.21.15.76:0/1534043311 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30577fe640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f305815f1d0 0x7f305814c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 -- 172.21.15.76:0/1534043311 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 msgr2=0x7f305814b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 0x7f305814b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 -- 172.21.15.76:0/1534043311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f305815f1d0 msgr2=0x7f305814c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f305815f1d0 0x7f305814c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 -- 172.21.15.76:0/1534043311 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f305814d9f0 con 0x7f30581520a0 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f30567fc640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581520a0 0x7f3058149620 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f304800d5f0 tx=0x7f304800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f303f7fe640 1 -- 172.21.15.76:0/1534043311 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048004240 con 0x7f30581520a0 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f303f7fe640 1 -- 172.21.15.76:0/1534043311 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30480043e0 con 0x7f30581520a0 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f305814dc90 con 0x7f30581520a0 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f303f7fe640 1 -- 172.21.15.76:0/1534043311 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3048003620 con 0x7f30581520a0 2024-09-19T04:18:40.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.681+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f305814e250 con 0x7f30581520a0 2024-09-19T04:18:40.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.685+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305815ec90 con 0x7f30581520a0 2024-09-19T04:18:40.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.685+0000 7f303f7fe640 1 -- 172.21.15.76:0/1534043311 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f304801b020 con 0x7f30581520a0 2024-09-19T04:18:40.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.685+0000 7f303f7fe640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30440678c0 0x7f3044069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:40.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.685+0000 7f303f7fe640 1 -- 172.21.15.76:0/1534043311 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f3048085a70 con 0x7f30581520a0 2024-09-19T04:18:40.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.685+0000 7f3056ffd640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30440678c0 0x7f3044069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:40.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.685+0000 7f3056ffd640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30440678c0 0x7f3044069d80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f304c00a8d0 tx=0x7f304c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:40.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.689+0000 7f303f7fe640 1 -- 172.21.15.76:0/1534043311 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3048056540 con 0x7f30581520a0 2024-09-19T04:18:40.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f305806e540 con 0x7f30581520a0 2024-09-19T04:18:40.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f303f7fe640 1 -- 172.21.15.76:0/1534043311 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) ==== 94+0+3914 (secure 0 0 0) 0x7f3048055e40 con 0x7f30581520a0 2024-09-19T04:18:40.873 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:40.873 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":77,"btime":"2024-09-19T04:18:34:775030+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/526594831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":526594831},{"type":"v1","addr":"172.21.15.116:6833","nonce":526594831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62},{"gid":5746,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2693960919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2693960919},{"type":"v1","addr":"172.21.15.116:6835","nonce":2693960919}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7218,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2027233163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2027233163},{"type":"v1","addr":"172.21.15.76:6835","nonce":2027233163}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7224,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2633035131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2633035131},{"type":"v1","addr":"172.21.15.76:6837","nonce":2633035131}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[]} 2024-09-19T04:18:40.874 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 77 2024-09-19T04:18:40.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30440678c0 msgr2=0x7f3044069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30440678c0 0x7f3044069d80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f304c00a8d0 tx=0x7f304c008040 comp rx=0 tx=0).stop 2024-09-19T04:18:40.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581520a0 msgr2=0x7f3058149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581520a0 0x7f3058149620 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f304800d5f0 tx=0x7f304800dab0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 shutdown_connections 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f30440678c0 0x7f3044069d80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f305815f1d0 0x7f305814c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f30581520a0 0x7f3058149620 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 --2- 172.21.15.76:0/1534043311 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f305806fe50 0x7f305814b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 >> 172.21.15.76:0/1534043311 conn(0x7f30580b1f90 msgr2=0x7f30580adc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 shutdown_connections 2024-09-19T04:18:40.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:40.873+0000 7f305d3ee640 1 -- 172.21.15.76:0/1534043311 wait complete. 2024-09-19T04:18:40.890 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T04:18:40.890 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T04:18:40.890 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T04:18:40.891 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:18:40.891 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T04:18:40.891 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T04:18:40.891 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T04:18:40.891 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:18:40.891 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T04:18:40.891 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T04:18:40.892 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T04:18:40.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:18:40.892 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T04:18:40.892 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T04:18:40.892 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T04:18:40.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:18:40.900 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T04:18:40.901 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T04:18:40.902 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T04:18:40.902 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T04:18:40.905 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T04:18:40.905 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T04:18:40.906 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T04:18:40.906 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T04:18:40.945 INFO:tasks.ceph.mds.c:Started 2024-09-19T04:18:40.946 DEBUG:teuthology.parallel:result is None 2024-09-19T04:18:40.947 INFO:tasks.ceph.mds.a:Started 2024-09-19T04:18:40.947 DEBUG:teuthology.parallel:result is None 2024-09-19T04:18:40.951 INFO:tasks.ceph.mds.b:Started 2024-09-19T04:18:40.951 DEBUG:teuthology.parallel:result is None 2024-09-19T04:18:40.951 INFO:tasks.ceph.mds.d:Started 2024-09-19T04:18:40.951 DEBUG:teuthology.parallel:result is None 2024-09-19T04:18:40.952 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T04:18:41.004 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:18:41.011 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:18:41.023 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:18:41.026 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:18:41.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20ae4c2640 1 Processor -- start 2024-09-19T04:18:41.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20ae4c2640 1 -- start start 2024-09-19T04:18:41.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20ae4c2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f20a809b110 0x7f20a81690e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20ae4c2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a806da60 con 0x7f20a815f660 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20ae4c2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a806dd30 con 0x7f20a80b3790 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20ae4c2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f20a806e5d0 con 0x7f20a809b110 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20a77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f20a815f660 0x7f20a809aa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59500/0 (socket says 172.21.15.76:59500) 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20a77fe640 1 -- 172.21.15.76:0/1914324089 learned_addr learned my addr 172.21.15.76:0/1914324089 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20aca5f640 1 --2- 172.21.15.76:0/1914324089 >> v2:172.21.15.116:3300/0 conn(0x7f20a809b110 0x7f20a81690e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20a6ffd640 1 -- 172.21.15.76:0/1914324089 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 351481571 0 0) 0x7f20a806da60 con 0x7f20a815f660 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20a6ffd640 1 -- 172.21.15.76:0/1914324089 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2090003530 con 0x7f20a815f660 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20a6ffd640 1 -- 172.21.15.76:0/1914324089 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 6413204 0 0) 0x7f2090003530 con 0x7f20a815f660 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20aca5f640 1 -- 172.21.15.76:0/1914324089 >> v1:172.21.15.116:6789/0 conn(0x7f20a80b3790 legacy=0x7f20a806d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20aca5f640 1 -- 172.21.15.76:0/1914324089 >> v1:172.21.15.76:6789/0 conn(0x7f20a815f660 legacy=0x7f20a809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.065+0000 7f20aca5f640 1 -- 172.21.15.76:0/1914324089 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a806e010 con 0x7f20a809b110 2024-09-19T04:18:41.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20aca5f640 1 --2- 172.21.15.76:0/1914324089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a809b110 0x7f20a81690e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009d90 tx=0x7f20a001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbc4637545acae42 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:41.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20a6ffd640 1 -- 172.21.15.76:0/1914324089 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00b2070 con 0x7f20a809b110 2024-09-19T04:18:41.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20a6ffd640 1 -- 172.21.15.76:0/1914324089 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a00a55c0 con 0x7f20a809b110 2024-09-19T04:18:41.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20a6ffd640 1 -- 172.21.15.76:0/1914324089 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c001680 con 0x7f20a809b110 2024-09-19T04:18:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 -- 172.21.15.76:0/1914324089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a809b110 msgr2=0x7f20a81690e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/1914324089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a809b110 0x7f20a81690e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009d90 tx=0x7f20a001c890 comp rx=0 tx=0).stop 2024-09-19T04:18:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 -- 172.21.15.76:0/1914324089 shutdown_connections 2024-09-19T04:18:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/1914324089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a809b110 0x7f20a81690e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 -- 172.21.15.76:0/1914324089 >> 172.21.15.76:0/1914324089 conn(0x7f20a80b78d0 msgr2=0x7f20a80b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 -- 172.21.15.76:0/1914324089 shutdown_connections 2024-09-19T04:18:41.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 -- 172.21.15.76:0/1914324089 wait complete. 2024-09-19T04:18:41.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 Processor -- start 2024-09-19T04:18:41.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.069+0000 7f20ae4c2640 1 -- start start 2024-09-19T04:18:41.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a809b110 0x7f20a80cec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a80b3790 0x7f20a81f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20a815f660 0x7f20a8150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f20a80b75b0 con 0x7f20a809b110 2024-09-19T04:18:41.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f20a80b7430 con 0x7f20a80b3790 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f20a816be40 con 0x7f20a815f660 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a809b110 0x7f20a80cec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a809b110 0x7f20a80cec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51504/0 (socket says 172.21.15.76:51504) 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 -- 172.21.15.76:0/3742849081 learned_addr learned my addr 172.21.15.76:0/3742849081 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20aca5f640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20a815f660 0x7f20a8150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a77fe640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a80b3790 0x7f20a81f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 -- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20a815f660 msgr2=0x7f20a8150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20a815f660 0x7f20a8150680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 -- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a80b3790 msgr2=0x7f20a81f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a80b3790 0x7f20a81f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 -- 172.21.15.76:0/3742849081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a813c840 con 0x7f20a809b110 2024-09-19T04:18:41.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a7fff640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a809b110 0x7f20a80cec10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f209800c3c0 tx=0x7f209800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:41.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a4ff9640 1 -- 172.21.15.76:0/3742849081 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098017070 con 0x7f20a809b110 2024-09-19T04:18:41.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a4ff9640 1 -- 172.21.15.76:0/3742849081 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20980052d0 con 0x7f20a809b110 2024-09-19T04:18:41.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a80cf1f0 con 0x7f20a809b110 2024-09-19T04:18:41.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a4ff9640 1 -- 172.21.15.76:0/3742849081 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098003840 con 0x7f20a809b110 2024-09-19T04:18:41.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a813cde0 con 0x7f20a809b110 2024-09-19T04:18:41.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a809aa00 con 0x7f20a809b110 2024-09-19T04:18:41.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a4ff9640 1 -- 172.21.15.76:0/3742849081 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f209801a070 con 0x7f20a809b110 2024-09-19T04:18:41.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a4ff9640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20940678c0 0x7f2094069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:41.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a4ff9640 1 -- 172.21.15.76:0/3742849081 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5507+0+0 (secure 0 0 0) 0x7f2098084df0 con 0x7f20a809b110 2024-09-19T04:18:41.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a77fe640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20940678c0 0x7f2094069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:41.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.073+0000 7f20a77fe640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20940678c0 0x7f2094069d80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f209c0013d0 tx=0x7f209c001360 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:41.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.077+0000 7f20a4ff9640 1 -- 172.21.15.76:0/3742849081 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2098005480 con 0x7f20a809b110 2024-09-19T04:18:41.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:41.253+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f20a8151ba0 con 0x7f20a809b110 2024-09-19T04:18:42.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.009+0000 7f20a4ff9640 1 -- 172.21.15.76:0/3742849081 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) ==== 97+0+0 (secure 0 0 0) 0x7f2098055430 con 0x7f20a809b110 2024-09-19T04:18:42.011 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2024-09-19T04:18:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20940678c0 msgr2=0x7f2094069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20940678c0 0x7f2094069d80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f209c0013d0 tx=0x7f209c001360 comp rx=0 tx=0).stop 2024-09-19T04:18:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a809b110 msgr2=0x7f20a80cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a809b110 0x7f20a80cec10 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f209800c3c0 tx=0x7f209800c880 comp rx=0 tx=0).stop 2024-09-19T04:18:42.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 shutdown_connections 2024-09-19T04:18:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20940678c0 0x7f2094069d80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f20a815f660 0x7f20a8150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20a80b3790 0x7f20a81f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 --2- 172.21.15.76:0/3742849081 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f20a809b110 0x7f20a80cec10 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 >> 172.21.15.76:0/3742849081 conn(0x7f20a80b78d0 msgr2=0x7f20a815fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 shutdown_connections 2024-09-19T04:18:42.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.013+0000 7f20ae4c2640 1 -- 172.21.15.76:0/3742849081 wait complete. 2024-09-19T04:18:42.035 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T04:18:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6d0a4640 1 Processor -- start 2024-09-19T04:18:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6d0a4640 1 -- start start 2024-09-19T04:18:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6d0a4640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcf6806f010 0x7fcf68166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6d0a4640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680b3290 con 0x7fcf6815c2a0 2024-09-19T04:18:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6d0a4640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf680b3570 con 0x7fcf6815bec0 2024-09-19T04:18:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6d0a4640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcf680b3850 con 0x7fcf6806f010 2024-09-19T04:18:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcf6815c2a0 0x7fcf680aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59516/0 (socket says 172.21.15.76:59516) 2024-09-19T04:18:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6659c640 1 -- 172.21.15.76:0/3094796976 learned_addr learned my addr 172.21.15.76:0/3094796976 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf6759e640 1 --2- 172.21.15.76:0/3094796976 >> v2:172.21.15.116:3300/0 conn(0x7fcf6806f010 0x7fcf68166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 677894594 0 0) 0x7fcf680b3290 con 0x7fcf6815c2a0 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.153+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf54003530 con 0x7fcf6815c2a0 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3855985631 0 0) 0x7fcf54003530 con 0x7fcf6815c2a0 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 >> v2:172.21.15.116:3300/0 conn(0x7fcf6806f010 msgr2=0x7fcf68166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 --2- 172.21.15.76:0/3094796976 >> v2:172.21.15.116:3300/0 conn(0x7fcf6806f010 0x7fcf68166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 >> v1:172.21.15.116:6789/0 conn(0x7fcf6815bec0 legacy=0x7fcf680ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf680b3b30 con 0x7fcf6815c2a0 2024-09-19T04:18:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fcf50001680 con 0x7fcf6815c2a0 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 >> v1:172.21.15.76:6789/0 conn(0x7fcf6815c2a0 legacy=0x7fcf680aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540039c0 0x7fcf54003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf54004810 0x7fcf54004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf540051d0 0x7fcf5400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680b3850 con 0x7fcf540051d0 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf5400c070 con 0x7fcf540039c0 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcf5400c270 con 0x7fcf54004810 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf540051d0 0x7fcf5400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf66d9d640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540039c0 0x7fcf54003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6659c640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf54004810 0x7fcf54004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 -- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf54004810 msgr2=0x7fcf54004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf54004810 0x7fcf54004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 -- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540039c0 msgr2=0x7fcf54003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540039c0 0x7fcf54003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 -- 172.21.15.76:0/3094796976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf54003530 con 0x7fcf540051d0 2024-09-19T04:18:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf66d9d640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540039c0 0x7fcf54003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 -- 172.21.15.76:0/3094796976 reap_dead start 2024-09-19T04:18:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6759e640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf540051d0 0x7fcf5400b9d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009a90 tx=0x7fcf5801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bae5045f3788bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009690 con 0x7fcf540051d0 2024-09-19T04:18:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf580a5660 con 0x7fcf540051d0 2024-09-19T04:18:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf65d9b640 1 -- 172.21.15.76:0/3094796976 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580a5a30 con 0x7fcf540051d0 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/3094796976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf540051d0 msgr2=0x7fcf5400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf540051d0 0x7fcf5400b9d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009a90 tx=0x7fcf5801ca70 comp rx=0 tx=0).stop 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/3094796976 shutdown_connections 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/3094796976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf540051d0 0x7fcf5400b9d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/3094796976 >> 172.21.15.76:0/3094796976 conn(0x7fcf680b1f90 msgr2=0x7fcf680b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/3094796976 shutdown_connections 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.157+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/3094796976 wait complete. 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 Processor -- start 2024-09-19T04:18:42.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 -- start start 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540051d0 0x7fcf681574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf6809a280 0x7fcf681e5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf6809df60 0x7fcf681e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680afc90 con 0x7fcf6809df60 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf680b40f0 con 0x7fcf540051d0 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcf680ad200 con 0x7fcf6809a280 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf6809df60 0x7fcf681e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf6809df60 0x7fcf681e6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51534/0 (socket says 172.21.15.76:51534) 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 -- 172.21.15.76:0/2483077853 learned_addr learned my addr 172.21.15.76:0/2483077853 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6659c640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf6809a280 0x7fcf681e5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 -- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf6809a280 msgr2=0x7fcf681e5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf6809a280 0x7fcf681e5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 -- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540051d0 msgr2=0x7fcf681574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf66d9d640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540051d0 0x7fcf681574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540051d0 0x7fcf681574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 -- 172.21.15.76:0/2483077853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf6809bb70 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6659c640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf6809a280 0x7fcf681e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6759e640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf6809df60 0x7fcf681e6000 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcf5801cf50 tx=0x7fcf58019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf5f7fe640 1 -- 172.21.15.76:0/2483077853 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580bc070 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf5f7fe640 1 -- 172.21.15.76:0/2483077853 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf580ad440 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf6809b2d0 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf5f7fe640 1 -- 172.21.15.76:0/2483077853 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf58009690 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf6809b890 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.161+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf681f1fb0 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.165+0000 7fcf5f7fe640 1 -- 172.21.15.76:0/2483077853 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcf580bf070 con 0x7fcf6809df60 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.165+0000 7fcf5f7fe640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf480678c0 0x7fcf48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.165+0000 7fcf5f7fe640 1 -- 172.21.15.76:0/2483077853 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7fcf5811f280 con 0x7fcf6809df60 2024-09-19T04:18:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.165+0000 7fcf66d9d640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf480678c0 0x7fcf48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.165+0000 7fcf66d9d640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf480678c0 0x7fcf48069d80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcf60001c40 tx=0x7fcf60009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.169+0000 7fcf5f7fe640 1 -- 172.21.15.76:0/2483077853 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcf58016780 con 0x7fcf6809df60 2024-09-19T04:18:42.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.389+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fcf6806fcd0 con 0x7fcf6809df60 2024-09-19T04:18:42.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf5f7fe640 1 -- 172.21.15.76:0/2483077853 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fcf580ef930 con 0x7fcf6809df60 2024-09-19T04:18:42.393 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:42.393 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:42.393 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2024-09-19T04:18:42.393 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w==", 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA==", 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.394 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw==", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.395 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA==", 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2024-09-19T04:18:42.396 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ==", 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w==", 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.397 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ==", 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw==", 2024-09-19T04:18:42.398 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA==", 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.399 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw==", 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg==", 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.400 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA==", 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.401 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg==", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg==", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:18:42.402 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ==", 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.403 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA==", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCopOtmGhUPLBAA9AkjLm1xF81Mjf1IYlN6Fg==", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:18:42.404 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A==", 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:42.405 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg==", 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-19T04:18:42.406 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf480678c0 msgr2=0x7fcf48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf480678c0 0x7fcf48069d80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fcf60001c40 tx=0x7fcf60009040 comp rx=0 tx=0).stop 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf6809df60 msgr2=0x7fcf681e6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf6809df60 0x7fcf681e6000 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fcf5801cf50 tx=0x7fcf58019810 comp rx=0 tx=0).stop 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 shutdown_connections 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf480678c0 0x7fcf48069d80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf6809df60 0x7fcf681e6000 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf6809a280 0x7fcf681e5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 --2- 172.21.15.76:0/2483077853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf540051d0 0x7fcf681574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 >> 172.21.15.76:0/2483077853 conn(0x7fcf680b1f90 msgr2=0x7fcf6815a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:42.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 shutdown_connections 2024-09-19T04:18:42.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.393+0000 7fcf6d0a4640 1 -- 172.21.15.76:0/2483077853 wait complete. 2024-09-19T04:18:42.412 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T04:18:42.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.529+0000 7f04c4655640 1 Processor -- start 2024-09-19T04:18:42.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.529+0000 7f04c4655640 1 -- start start 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c4655640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04bc15bec0 0x7f04bc0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c4655640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f04bc15c2a0 0x7f04bc0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c4655640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04bc0b3290 con 0x7f04bc0af2a0 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c4655640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f04bc0b3570 con 0x7f04bc15bec0 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c4655640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f04bc0b3850 con 0x7f04bc15c2a0 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04bc15bec0 0x7f04bc0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f04bc15bec0 0x7f04bc0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51542/0 (socket says 172.21.15.76:51542) 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 -- 172.21.15.76:0/357984602 learned_addr learned my addr 172.21.15.76:0/357984602 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:42.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c2bf2640 1 --1- 172.21.15.76:0/357984602 >> v1:172.21.15.76:6789/0 conn(0x7f04bc0af2a0 0x7f04bc15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:59524/0 (socket says 172.21.15.76:59524) 2024-09-19T04:18:42.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c1bf0640 1 --2- 172.21.15.76:0/357984602 >> v2:172.21.15.116:3300/0 conn(0x7f04bc15c2a0 0x7f04bc0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 -- 172.21.15.76:0/357984602 >> v2:172.21.15.116:3300/0 conn(0x7f04bc15c2a0 msgr2=0x7f04bc0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:42.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 --2- 172.21.15.76:0/357984602 >> v2:172.21.15.116:3300/0 conn(0x7f04bc15c2a0 0x7f04bc0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 -- 172.21.15.76:0/357984602 >> v1:172.21.15.76:6789/0 conn(0x7f04bc0af2a0 legacy=0x7f04bc15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 -- 172.21.15.76:0/357984602 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc0b3b30 con 0x7f04bc15bec0 2024-09-19T04:18:42.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c13ef640 1 -- 172.21.15.76:0/357984602 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2200113936 0 0) 0x7f04bc0b3290 con 0x7f04bc0af2a0 2024-09-19T04:18:42.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c1bf0640 1 --2- 172.21.15.76:0/357984602 >> v2:172.21.15.116:3300/0 conn(0x7f04bc15c2a0 0x7f04bc0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:42.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c23f1640 1 --2- 172.21.15.76:0/357984602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc0ad200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0ae510 tx=0x7f04b801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e947d06f2e76c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c13ef640 1 -- 172.21.15.76:0/357984602 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b80aa650 con 0x7f04bc15bec0 2024-09-19T04:18:42.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c13ef640 1 -- 172.21.15.76:0/357984602 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b80af070 con 0x7f04bc15bec0 2024-09-19T04:18:42.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.533+0000 7f04c13ef640 1 -- 172.21.15.76:0/357984602 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b80aaa10 con 0x7f04bc15bec0 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/357984602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 msgr2=0x7f04bc0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 --2- 172.21.15.76:0/357984602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc0ad200 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f04bc0ae510 tx=0x7f04b801c4f0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/357984602 shutdown_connections 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 --2- 172.21.15.76:0/357984602 >> v2:172.21.15.116:3300/0 conn(0x7f04bc15c2a0 0x7f04bc0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 --2- 172.21.15.76:0/357984602 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc0ad200 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/357984602 >> 172.21.15.76:0/357984602 conn(0x7f04bc0b1f90 msgr2=0x7f04bc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/357984602 shutdown_connections 2024-09-19T04:18:42.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/357984602 wait complete. 2024-09-19T04:18:42.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 Processor -- start 2024-09-19T04:18:42.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- start start 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04bc0af2a0 0x7f04bc1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc09cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04bc15c2a0 0x7f04bc1546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0af6d0 con 0x7f04bc15bec0 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0ad740 con 0x7f04bc15c2a0 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc169000 con 0x7f04bc0af2a0 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc09cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc09cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51548/0 (socket says 172.21.15.76:51548) 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 -- 172.21.15.76:0/3773223533 learned_addr learned my addr 172.21.15.76:0/3773223533 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:42.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c23f1640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04bc0af2a0 0x7f04bc1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c2bf2640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04bc15c2a0 0x7f04bc1546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 -- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04bc0af2a0 msgr2=0x7f04bc1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04bc0af2a0 0x7f04bc1521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 -- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04bc15c2a0 msgr2=0x7f04bc1546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04bc15c2a0 0x7f04bc1546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 -- 172.21.15.76:0/3773223533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc09d680 con 0x7f04bc15bec0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c2bf2640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04bc15c2a0 0x7f04bc1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c1bf0640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc09cc80 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c820 tx=0x7f04ac00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04b2ffd640 1 -- 172.21.15.76:0/3773223533 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac015070 con 0x7f04bc15bec0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04b2ffd640 1 -- 172.21.15.76:0/3773223533 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ac004220 con 0x7f04bc15bec0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04b2ffd640 1 -- 172.21.15.76:0/3773223533 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ac00ce10 con 0x7f04bc15bec0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc09d960 con 0x7f04bc15bec0 2024-09-19T04:18:42.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc09df20 con 0x7f04bc15bec0 2024-09-19T04:18:42.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.537+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc0ad200 con 0x7f04bc15bec0 2024-09-19T04:18:42.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.541+0000 7f04b2ffd640 1 -- 172.21.15.76:0/3773223533 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f04ac007830 con 0x7f04bc15bec0 2024-09-19T04:18:42.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.541+0000 7f04b2ffd640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04a00678c0 0x7f04a0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.541+0000 7f04b2ffd640 1 -- 172.21.15.76:0/3773223533 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f04ac085c10 con 0x7f04bc15bec0 2024-09-19T04:18:42.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.541+0000 7f04c23f1640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04a00678c0 0x7f04a0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.541+0000 7f04c23f1640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04a00678c0 0x7f04a0069d80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f04b80ae000 tx=0x7f04b80a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.545+0000 7f04b2ffd640 1 -- 172.21.15.76:0/3773223533 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f04ac0569a0 con 0x7f04bc15bec0 2024-09-19T04:18:42.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.697+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f04bc06fcd0 con 0x7f04bc15bec0 2024-09-19T04:18:42.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.697+0000 7f04b2ffd640 1 -- 172.21.15.76:0/3773223533 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v87) ==== 84+0+65 (secure 0 0 0) 0x7f04ac0562a0 con 0x7f04bc15bec0 2024-09-19T04:18:42.699 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:42.699 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:18:42.699 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:18:42.699 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-09-19T04:18:42.699 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-09-19T04:18:42.699 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:42.700 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:18:42.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04a00678c0 msgr2=0x7f04a0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04a00678c0 0x7f04a0069d80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f04b80ae000 tx=0x7f04b80a7000 comp rx=0 tx=0).stop 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 msgr2=0x7f04bc09cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc09cc80 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f04ac00c820 tx=0x7f04ac00cce0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 shutdown_connections 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f04a00678c0 0x7f04a0069d80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04bc15c2a0 0x7f04bc1546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04bc15bec0 0x7f04bc09cc80 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 --2- 172.21.15.76:0/3773223533 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f04bc0af2a0 0x7f04bc1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 >> 172.21.15.76:0/3773223533 conn(0x7f04bc0b1f90 msgr2=0x7f04bc0b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 shutdown_connections 2024-09-19T04:18:42.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.701+0000 7f04c4655640 1 -- 172.21.15.76:0/3773223533 wait complete. 2024-09-19T04:18:42.717 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:18:42.717 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:18:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.837+0000 7f0aff7ab640 1 Processor -- start 2024-09-19T04:18:42.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.837+0000 7f0aff7ab640 1 -- start start 2024-09-19T04:18:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0af8161a60 0x7f0af80d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0af80d2e90 0x7f0af80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0af80d32b0 con 0x7f0af80cf450 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0af80d0300 con 0x7f0af80d2e90 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0af806e6c0 con 0x7f0af8161a60 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0af80d2e90 0x7f0af80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0af80d2e90 0x7f0af80cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46406/0 (socket says 172.21.15.76:46406) 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 -- 172.21.15.76:0/4242086448 learned_addr learned my addr 172.21.15.76:0/4242086448 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afd547640 1 --2- 172.21.15.76:0/4242086448 >> v2:172.21.15.116:3300/0 conn(0x7f0af8161a60 0x7f0af80d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 -- 172.21.15.76:0/4242086448 >> v2:172.21.15.116:3300/0 conn(0x7f0af8161a60 msgr2=0x7f0af80d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 --2- 172.21.15.76:0/4242086448 >> v2:172.21.15.116:3300/0 conn(0x7f0af8161a60 0x7f0af80d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 -- 172.21.15.76:0/4242086448 >> v1:172.21.15.116:6789/0 conn(0x7f0af80cf450 legacy=0x7f0af80d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:42.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 -- 172.21.15.76:0/4242086448 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af80b0f60 con 0x7f0af80d2e90 2024-09-19T04:18:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0afcd46640 1 --2- 172.21.15.76:0/4242086448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80d2e90 0x7f0af80cef10 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0af808e010 tx=0x7f0ae801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c08dc41db4591f07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aeffff640 1 -- 172.21.15.76:0/4242086448 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80d1930 con 0x7f0af80d2e90 2024-09-19T04:18:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aeffff640 1 -- 172.21.15.76:0/4242086448 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae80b0070 con 0x7f0af80d2e90 2024-09-19T04:18:42.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aeffff640 1 -- 172.21.15.76:0/4242086448 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80ab840 con 0x7f0af80d2e90 2024-09-19T04:18:42.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- 172.21.15.76:0/4242086448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80d2e90 msgr2=0x7f0af80cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/4242086448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80d2e90 0x7f0af80cef10 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f0af808e010 tx=0x7f0ae801c420 comp rx=0 tx=0).stop 2024-09-19T04:18:42.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- 172.21.15.76:0/4242086448 shutdown_connections 2024-09-19T04:18:42.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/4242086448 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80d2e90 0x7f0af80cef10 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/4242086448 >> v2:172.21.15.116:3300/0 conn(0x7f0af8161a60 0x7f0af80d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- 172.21.15.76:0/4242086448 >> 172.21.15.76:0/4242086448 conn(0x7f0af808bf90 msgr2=0x7f0af809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:42.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- 172.21.15.76:0/4242086448 shutdown_connections 2024-09-19T04:18:42.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.841+0000 7f0aff7ab640 1 -- 172.21.15.76:0/4242086448 wait complete. 2024-09-19T04:18:42.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 Processor -- start 2024-09-19T04:18:42.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 -- start start 2024-09-19T04:18:42.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80cf450 0x7f0af80b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af80d2e90 0x7f0af8150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0af8161a60 0x7f0af815f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0af80cf8c0 con 0x7f0af80cf450 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0af806d190 con 0x7f0af80d2e90 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0af80cfa40 con 0x7f0af8161a60 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80cf450 0x7f0af80b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80cf450 0x7f0af80b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46410/0 (socket says 172.21.15.76:46410) 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 -- 172.21.15.76:0/2787010256 learned_addr learned my addr 172.21.15.76:0/2787010256 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afdd48640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0af8161a60 0x7f0af815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 -- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0af8161a60 msgr2=0x7f0af815f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0af8161a60 0x7f0af815f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afcd46640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af80d2e90 0x7f0af8150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 -- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af80d2e90 msgr2=0x7f0af8150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af80d2e90 0x7f0af8150140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 -- 172.21.15.76:0/2787010256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0af80cb8d0 con 0x7f0af80cf450 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afdd48640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0af8161a60 0x7f0af815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0afd547640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80cf450 0x7f0af80b71a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d5f0 tx=0x7f0af400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aedffb640 1 -- 172.21.15.76:0/2787010256 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af40125f0 con 0x7f0af80cf450 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aedffb640 1 -- 172.21.15.76:0/2787010256 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af4019070 con 0x7f0af80cf450 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0af80cbbb0 con 0x7f0af80cf450 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aedffb640 1 -- 172.21.15.76:0/2787010256 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af40129b0 con 0x7f0af80cf450 2024-09-19T04:18:42.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0af80cc170 con 0x7f0af80cf450 2024-09-19T04:18:42.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.845+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0af80cef10 con 0x7f0af80cf450 2024-09-19T04:18:42.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.849+0000 7f0aedffb640 1 -- 172.21.15.76:0/2787010256 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0af4012b50 con 0x7f0af80cf450 2024-09-19T04:18:42.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.849+0000 7f0aedffb640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0adc0678c0 0x7f0adc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:42.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.849+0000 7f0aedffb640 1 -- 172.21.15.76:0/2787010256 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5249+0+0 (secure 0 0 0) 0x7f0af4085f80 con 0x7f0af80cf450 2024-09-19T04:18:42.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.849+0000 7f0afcd46640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0adc0678c0 0x7f0adc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:42.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.849+0000 7f0afcd46640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0adc0678c0 0x7f0adc069d80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80af000 tx=0x7f0ae8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:42.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:42.853+0000 7f0aedffb640 1 -- 172.21.15.76:0/2787010256 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0af4011340 con 0x7f0af80cf450 2024-09-19T04:18:43.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:43.017+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0af80b5960 con 0x7f0af80cf450 2024-09-19T04:18:44.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.017+0000 7f0aedffb640 1 -- 172.21.15.76:0/2787010256 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v88) ==== 138+0+0 (secure 0 0 0) 0x7f0af4056be0 con 0x7f0af80cf450 2024-09-19T04:18:44.017 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0adc0678c0 msgr2=0x7f0adc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0adc0678c0 0x7f0adc069d80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80af000 tx=0x7f0ae8008040 comp rx=0 tx=0).stop 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80cf450 msgr2=0x7f0af80b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80cf450 0x7f0af80b71a0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f0af400d5f0 tx=0x7f0af400dab0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 shutdown_connections 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0adc0678c0 0x7f0adc069d80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0af8161a60 0x7f0af815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0af80d2e90 0x7f0af8150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 --2- 172.21.15.76:0/2787010256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0af80cf450 0x7f0af80b71a0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 >> 172.21.15.76:0/2787010256 conn(0x7f0af808bf90 msgr2=0x7f0af815cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:44.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 shutdown_connections 2024-09-19T04:18:44.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.021+0000 7f0aff7ab640 1 -- 172.21.15.76:0/2787010256 wait complete. 2024-09-19T04:18:44.045 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:18:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.173+0000 7ff4b2669640 1 Processor -- start 2024-09-19T04:18:44.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.173+0000 7ff4b2669640 1 -- start start 2024-09-19T04:18:44.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.173+0000 7ff4b2669640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff4ac15e4f0 0x7ff4ac0d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.173+0000 7ff4b2669640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4ac0d4e20 0x7ff4ac0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.173+0000 7ff4b2669640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4ac0d0350 con 0x7ff4ac162070 2024-09-19T04:18:44.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.173+0000 7ff4b2669640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff4ac0cf070 con 0x7ff4ac15e4f0 2024-09-19T04:18:44.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.173+0000 7ff4b2669640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d1c20 con 0x7ff4ac0d4e20 2024-09-19T04:18:44.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4b0c06640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff4ac162070 0x7ff4ac0b7260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46946/0 (socket says 172.21.15.76:46946) 2024-09-19T04:18:44.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4b0c06640 1 -- 172.21.15.76:0/654070762 learned_addr learned my addr 172.21.15.76:0/654070762 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:44.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4ac0d4e20 0x7ff4ac0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4abfff640 1 --2- 172.21.15.76:0/654070762 >> v2:172.21.15.116:3300/0 conn(0x7ff4ac15e4f0 0x7ff4ac0d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 305680425 0 0) 0x7ff4ac0d0350 con 0x7ff4ac162070 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff494003530 con 0x7ff4ac162070 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3798821353 0 0) 0x7ff494003530 con 0x7ff4ac162070 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4ac0d4e20 msgr2=0x7ff4ac0cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4ac0d4e20 0x7ff4ac0cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 >> v2:172.21.15.116:3300/0 conn(0x7ff4ac15e4f0 msgr2=0x7ff4ac0d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 --2- 172.21.15.76:0/654070762 >> v2:172.21.15.116:3300/0 conn(0x7ff4ac15e4f0 0x7ff4ac0d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0cf330 con 0x7ff4ac162070 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7ff4a4001680 con 0x7ff4ac162070 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 >> v1:172.21.15.76:6789/0 conn(0x7ff4ac162070 legacy=0x7ff4ac0b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4940039c0 0x7ff494003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff494004810 0x7ff494004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4ac0d4e20 0x7ff4940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0cf070 con 0x7ff494004810 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d1c20 con 0x7ff4940039c0 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff49400ba00 con 0x7ff4ac0d4e20 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff494004810 0x7ff494004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4abfff640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4940039c0 0x7ff494003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 -- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4ac0d4e20 msgr2=0x7ff4940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4ac0d4e20 0x7ff4940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 -- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4940039c0 msgr2=0x7ff494003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4940039c0 0x7ff494003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 -- 172.21.15.76:0/654070762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff494003530 con 0x7ff494004810 2024-09-19T04:18:44.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff494004810 0x7ff494004c90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009ac0 tx=0x7ff4a001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8f8756bdc173c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00b2070 con 0x7ff494004810 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a00a55a0 con 0x7ff494004810 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.177+0000 7ff4aaffd640 1 -- 172.21.15.76:0/654070762 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0009690 con 0x7ff494004810 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- 172.21.15.76:0/654070762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff494004810 msgr2=0x7ff494004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 --2- 172.21.15.76:0/654070762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff494004810 0x7ff494004c90 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0009ac0 tx=0x7ff4a001ca90 comp rx=0 tx=0).stop 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b0c06640 1 -- 172.21.15.76:0/654070762 reap_dead start 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- 172.21.15.76:0/654070762 shutdown_connections 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- 172.21.15.76:0/654070762 >> 172.21.15.76:0/654070762 conn(0x7ff4ac08bf90 msgr2=0x7ff4ac0d4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- 172.21.15.76:0/654070762 shutdown_connections 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- 172.21.15.76:0/654070762 wait complete. 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 Processor -- start 2024-09-19T04:18:44.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- start start 2024-09-19T04:18:44.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff494004810 0x7ff4ac14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0c8d00 0x7ff4ac161270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ac1617b0 0x7ff4ac0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac15e8d0 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4ac0cf8f0 con 0x7ff4ac1617b0 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff4ac0d5270 con 0x7ff494004810 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0c8d00 0x7ff4ac161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0c8d00 0x7ff4ac161270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46428/0 (socket says 172.21.15.76:46428) 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 -- 172.21.15.76:0/1176181854 learned_addr learned my addr 172.21.15.76:0/1176181854 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4abfff640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff494004810 0x7ff4ac14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b0c06640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ac1617b0 0x7ff4ac0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 -- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff494004810 msgr2=0x7ff4ac14f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff494004810 0x7ff4ac14f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 -- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ac1617b0 msgr2=0x7ff4ac0c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ac1617b0 0x7ff4ac0c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 -- 172.21.15.76:0/1176181854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4ac0ca4c0 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b0c06640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ac1617b0 0x7ff4ac0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4abfff640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff494004810 0x7ff4ac14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4ab7fe640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0c8d00 0x7ff4ac161270 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff4a001c8c0 tx=0x7ff494005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4a8ff9640 1 -- 172.21.15.76:0/1176181854 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00c0070 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4a8ff9640 1 -- 172.21.15.76:0/1176181854 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4a00a5e50 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4a8ff9640 1 -- 172.21.15.76:0/1176181854 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00b2040 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4ac0c7f40 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.181+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4ac0c8500 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.185+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4ac161bc0 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.185+0000 7ff4a8ff9640 1 -- 172.21.15.76:0/1176181854 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff4a00c3070 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.185+0000 7ff4a8ff9640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4980678c0 0x7ff498069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.185+0000 7ff4abfff640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4980678c0 0x7ff498069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.185+0000 7ff4a8ff9640 1 -- 172.21.15.76:0/1176181854 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5619+0+0 (secure 0 0 0) 0x7ff4a011f830 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.185+0000 7ff4abfff640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4980678c0 0x7ff498069d80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0074d0 tx=0x7ff49c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:44.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.189+0000 7ff4a8ff9640 1 -- 172.21.15.76:0/1176181854 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff4ac161bc0 con 0x7ff4ac0c8d00 2024-09-19T04:18:44.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:44.349+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff4ac0b9670 con 0x7ff4ac0c8d00 2024-09-19T04:18:45.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.021+0000 7ff4a8ff9640 1 -- 172.21.15.76:0/1176181854 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v89) ==== 170+0+0 (secure 0 0 0) 0x7ff4a00167a0 con 0x7ff4ac0c8d00 2024-09-19T04:18:45.022 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-09-19T04:18:45.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4980678c0 msgr2=0x7ff498069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4980678c0 0x7ff498069d80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff49c0074d0 tx=0x7ff49c00b040 comp rx=0 tx=0).stop 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0c8d00 msgr2=0x7ff4ac161270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0c8d00 0x7ff4ac161270 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7ff4a001c8c0 tx=0x7ff494005840 comp rx=0 tx=0).stop 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 shutdown_connections 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4980678c0 0x7ff498069d80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ac1617b0 0x7ff4ac0c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4ac0c8d00 0x7ff4ac161270 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 --2- 172.21.15.76:0/1176181854 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff494004810 0x7ff4ac14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 >> 172.21.15.76:0/1176181854 conn(0x7ff4ac08bf90 msgr2=0x7ff4ac0cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 shutdown_connections 2024-09-19T04:18:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.025+0000 7ff4b2669640 1 -- 172.21.15.76:0/1176181854 wait complete. 2024-09-19T04:18:45.060 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:18:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f8e34640 1 Processor -- start 2024-09-19T04:18:45.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f8e34640 1 -- start start 2024-09-19T04:18:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f8e34640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43f40d3060 0x7f43f40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f8e34640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40cf450 0x7f43f406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f8e34640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43f40d0300 con 0x7f43f4161a60 2024-09-19T04:18:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f8e34640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f43f40b0f60 con 0x7f43f40d3060 2024-09-19T04:18:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f8e34640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f43f40cfd40 con 0x7f43f40cf450 2024-09-19T04:18:45.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f1d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43f40d3060 0x7f43f40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f1d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f43f40d3060 0x7f43f40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46432/0 (socket says 172.21.15.76:46432) 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f1d9b640 1 -- 172.21.15.76:0/3229728003 learned_addr learned my addr 172.21.15.76:0/3229728003 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f2d9d640 1 --2- 172.21.15.76:0/3229728003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40cf450 0x7f43f406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f1d9b640 1 -- 172.21.15.76:0/3229728003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40cf450 msgr2=0x7f43f406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f1d9b640 1 --2- 172.21.15.76:0/3229728003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40cf450 0x7f43f406d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f1d9b640 1 -- 172.21.15.76:0/3229728003 >> v1:172.21.15.116:6789/0 conn(0x7f43f4161a60 legacy=0x7f43f40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.189+0000 7f43f1d9b640 1 -- 172.21.15.76:0/3229728003 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f40d0020 con 0x7f43f40d3060 2024-09-19T04:18:45.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f1d9b640 1 --2- 172.21.15.76:0/3229728003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f40d3060 0x7f43f40cef10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009870 tx=0x7f43e801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fbed54ef476f93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:45.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f159a640 1 -- 172.21.15.76:0/3229728003 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e80aa600 con 0x7f43f40d3060 2024-09-19T04:18:45.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f159a640 1 -- 172.21.15.76:0/3229728003 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43e80af070 con 0x7f43f40d3060 2024-09-19T04:18:45.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f159a640 1 -- 172.21.15.76:0/3229728003 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e80aa9c0 con 0x7f43f40d3060 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- 172.21.15.76:0/3229728003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f40d3060 msgr2=0x7f43f40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 --2- 172.21.15.76:0/3229728003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f40d3060 0x7f43f40cef10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f43e8009870 tx=0x7f43e801c4a0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- 172.21.15.76:0/3229728003 shutdown_connections 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 --2- 172.21.15.76:0/3229728003 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40cf450 0x7f43f406d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 --2- 172.21.15.76:0/3229728003 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f40d3060 0x7f43f40cef10 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- 172.21.15.76:0/3229728003 >> 172.21.15.76:0/3229728003 conn(0x7f43f408bf90 msgr2=0x7f43f409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- 172.21.15.76:0/3229728003 shutdown_connections 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- 172.21.15.76:0/3229728003 wait complete. 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 Processor -- start 2024-09-19T04:18:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- start start 2024-09-19T04:18:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43f40cf450 0x7f43f414fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40d3060 0x7f43f41584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f4161a60 0x7f43f40b4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f43f416af20 con 0x7f43f4161a60 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f43f406e690 con 0x7f43f40cf450 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.193+0000 7f43f8e34640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f43f416b0a0 con 0x7f43f40d3060 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f4161a60 0x7f43f40b4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f4161a60 0x7f43f40b4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46446/0 (socket says 172.21.15.76:46446) 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 -- 172.21.15.76:0/519416984 learned_addr learned my addr 172.21.15.76:0/519416984 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f1d9b640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40d3060 0x7f43f41584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f259c640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43f40cf450 0x7f43f414fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 -- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40d3060 msgr2=0x7f43f41584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40d3060 0x7f43f41584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 -- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43f40cf450 msgr2=0x7f43f414fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43f40cf450 0x7f43f414fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 -- 172.21.15.76:0/519416984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43f41545c0 con 0x7f43f4161a60 2024-09-19T04:18:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f2d9d640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f4161a60 0x7f43f40b4a60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00d590 tx=0x7f43ec00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43e2ffd640 1 -- 172.21.15.76:0/519416984 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0125f0 con 0x7f43f4161a60 2024-09-19T04:18:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43e2ffd640 1 -- 172.21.15.76:0/519416984 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43ec019070 con 0x7f43f4161a60 2024-09-19T04:18:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43f4154840 con 0x7f43f4161a60 2024-09-19T04:18:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43e2ffd640 1 -- 172.21.15.76:0/519416984 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43ec0129f0 con 0x7f43f4161a60 2024-09-19T04:18:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43f4154e00 con 0x7f43f4161a60 2024-09-19T04:18:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43f406d6b0 con 0x7f43f4161a60 2024-09-19T04:18:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43e2ffd640 1 -- 172.21.15.76:0/519416984 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f43ec012b90 con 0x7f43f4161a60 2024-09-19T04:18:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43e2ffd640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43d40678c0 0x7f43d4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43e2ffd640 1 -- 172.21.15.76:0/519416984 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6001+0+0 (secure 0 0 0) 0x7f43ec086130 con 0x7f43f4161a60 2024-09-19T04:18:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f259c640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43d40678c0 0x7f43d4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.197+0000 7f43f259c640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43d40678c0 0x7f43d4069d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43f40b4020 tx=0x7f43e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:45.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.201+0000 7f43e2ffd640 1 -- 172.21.15.76:0/519416984 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f43f41545c0 con 0x7f43f4161a60 2024-09-19T04:18:45.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:45.401+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f43f40c7c40 con 0x7f43f4161a60 2024-09-19T04:18:46.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.085+0000 7f43e2ffd640 1 -- 172.21.15.76:0/519416984 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v80) ==== 174+0+0 (secure 0 0 0) 0x7f43ec056bd0 con 0x7f43f4161a60 2024-09-19T04:18:46.092 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 13 and data pool 14 2024-09-19T04:18:46.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43d40678c0 msgr2=0x7f43d4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43d40678c0 0x7f43d4069d80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f43f40b4020 tx=0x7f43e4009040 comp rx=0 tx=0).stop 2024-09-19T04:18:46.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f4161a60 msgr2=0x7f43f40b4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f4161a60 0x7f43f40b4a60 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00d590 tx=0x7f43ec00da50 comp rx=0 tx=0).stop 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 shutdown_connections 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43d40678c0 0x7f43d4069d80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43f4161a60 0x7f43f40b4a60 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43f40d3060 0x7f43f41584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 --2- 172.21.15.76:0/519416984 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43f40cf450 0x7f43f414fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 >> 172.21.15.76:0/519416984 conn(0x7f43f408bf90 msgr2=0x7f43f415d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 shutdown_connections 2024-09-19T04:18:46.106 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.105+0000 7f43f8e34640 1 -- 172.21.15.76:0/519416984 wait complete. 2024-09-19T04:18:46.133 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:46.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fac2dd640 1 Processor -- start 2024-09-19T04:18:46.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fac2dd640 1 -- start start 2024-09-19T04:18:46.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fac2dd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7fa40d3060 0x7f7fa40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fac2dd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa40cf450 0x7f7fa406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fac2dd640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fa40d0300 con 0x7f7fa4161a60 2024-09-19T04:18:46.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fac2dd640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f7fa40b0f60 con 0x7f7fa40d3060 2024-09-19T04:18:46.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fac2dd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa40cfd40 con 0x7f7fa40cf450 2024-09-19T04:18:46.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa87a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa40cf450 0x7f7fa406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fa9878640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7fa40d3060 0x7f7fa40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa079640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7fa4161a60 0x7f7fa40d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:43994/0 (socket says 172.21.15.76:43994) 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa87a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa40cf450 0x7f7fa406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:37850/0 (socket says 172.21.15.76:37850) 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa079640 1 -- 172.21.15.76:0/1455517754 learned_addr learned my addr 172.21.15.76:0/1455517754 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa87a640 1 -- 172.21.15.76:0/1455517754 >> v2:172.21.15.116:3300/0 conn(0x7f7fa40d3060 msgr2=0x7f7fa40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa87a640 1 --2- 172.21.15.76:0/1455517754 >> v2:172.21.15.116:3300/0 conn(0x7f7fa40d3060 0x7f7fa40cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa87a640 1 -- 172.21.15.76:0/1455517754 >> v1:172.21.15.116:6789/0 conn(0x7f7fa4161a60 legacy=0x7f7fa40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7faa87a640 1 -- 172.21.15.76:0/1455517754 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa40d0020 con 0x7f7fa40cf450 2024-09-19T04:18:46.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.249+0000 7f7fa9878640 1 --2- 172.21.15.76:0/1455517754 >> v2:172.21.15.116:3300/0 conn(0x7f7fa40d3060 0x7f7fa40cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:18:46.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7faa87a640 1 --2- 172.21.15.76:0/1455517754 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa40cf450 0x7f7fa406d6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009d90 tx=0x7f7f9c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe979a92ac16361 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fa9077640 1 -- 172.21.15.76:0/1455517754 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0b2070 con 0x7f7fa40cf450 2024-09-19T04:18:46.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fa9077640 1 -- 172.21.15.76:0/1455517754 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f9c0a5970 con 0x7f7fa40cf450 2024-09-19T04:18:46.254 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fa9077640 1 -- 172.21.15.76:0/1455517754 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0a5d40 con 0x7f7fa40cf450 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 -- 172.21.15.76:0/1455517754 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa40cf450 msgr2=0x7f7fa406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/1455517754 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa40cf450 0x7f7fa406d6b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009d90 tx=0x7f7f9c01c8b0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 -- 172.21.15.76:0/1455517754 shutdown_connections 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/1455517754 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa40cf450 0x7f7fa406d6b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/1455517754 >> v2:172.21.15.116:3300/0 conn(0x7f7fa40d3060 0x7f7fa40cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 -- 172.21.15.76:0/1455517754 >> 172.21.15.76:0/1455517754 conn(0x7f7fa408bf90 msgr2=0x7f7fa409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 -- 172.21.15.76:0/1455517754 shutdown_connections 2024-09-19T04:18:46.255 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 -- 172.21.15.76:0/1455517754 wait complete. 2024-09-19T04:18:46.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 Processor -- start 2024-09-19T04:18:46.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.253+0000 7f7fac2dd640 1 -- start start 2024-09-19T04:18:46.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fa40cf450 0x7f7fa415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa40d3060 0x7f7fa4151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa4161a60 0x7f7fa4152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa406e580 con 0x7f7fa40cf450 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa406e400 con 0x7f7fa40d3060 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7fa406e700 con 0x7f7fa4161a60 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fa40cf450 0x7f7fa415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fa40cf450 0x7f7fa415d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46448/0 (socket says 172.21.15.76:46448) 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 -- 172.21.15.76:0/3685416500 learned_addr learned my addr 172.21.15.76:0/3685416500 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa87a640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa4161a60 0x7f7fa4152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 -- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa4161a60 msgr2=0x7f7fa4152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa4161a60 0x7f7fa4152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 -- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa40d3060 msgr2=0x7f7fa4151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fa9878640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa40d3060 0x7f7fa4151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa40d3060 0x7f7fa4151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 -- 172.21.15.76:0/3685416500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa40cc450 con 0x7f7fa40cf450 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa87a640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa4161a60 0x7f7fa4152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7faa079640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fa40cf450 0x7f7fa415d3b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c420 tx=0x7f7fa000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7f9affd640 1 -- 172.21.15.76:0/3685416500 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa0017070 con 0x7f7fa40cf450 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7f9affd640 1 -- 172.21.15.76:0/3685416500 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fa0005cf0 con 0x7f7fa40cf450 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa40cc730 con 0x7f7fa40cf450 2024-09-19T04:18:46.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7f9affd640 1 -- 172.21.15.76:0/3685416500 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa00072f0 con 0x7f7fa40cf450 2024-09-19T04:18:46.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa40ccfd0 con 0x7f7fa40cf450 2024-09-19T04:18:46.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa40cc900 con 0x7f7fa40cf450 2024-09-19T04:18:46.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7f9affd640 1 -- 172.21.15.76:0/3685416500 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7fa001a070 con 0x7f7fa40cf450 2024-09-19T04:18:46.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7f9affd640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f880678f0 0x7f7f88069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7f9affd640 1 -- 172.21.15.76:0/3685416500 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f7fa0085230 con 0x7f7fa40cf450 2024-09-19T04:18:46.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fa9878640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f880678f0 0x7f7f88069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.257+0000 7f7fa9878640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f880678f0 0x7f7f88069db0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40b2320 tx=0x7f7f94008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.261+0000 7f7f9affd640 1 -- 172.21.15.76:0/3685416500 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7fa40d0300 con 0x7f7fa40cf450 2024-09-19T04:18:46.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.425+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fa406d6b0 con 0x7f7fa40cf450 2024-09-19T04:18:46.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7f9affd640 1 -- 172.21.15.76:0/3685416500 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16982 (secure 0 0 0) 0x7f7fa0055b60 con 0x7f7fa40cf450 2024-09-19T04:18:46.429 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:46.429 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":90,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:45.404177+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:43.054193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:44.388097+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:46.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f880678f0 msgr2=0x7f7f88069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.431 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f880678f0 0x7f7f88069db0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40b2320 tx=0x7f7f94008040 comp rx=0 tx=0).stop 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fa40cf450 msgr2=0x7f7fa415d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fa40cf450 0x7f7fa415d3b0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f7fa000c420 tx=0x7f7fa000c8e0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 shutdown_connections 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7f880678f0 0x7f7f88069db0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fa4161a60 0x7f7fa4152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fa40d3060 0x7f7fa4151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 --2- 172.21.15.76:0/3685416500 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fa40cf450 0x7f7fa415d3b0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 >> 172.21.15.76:0/3685416500 conn(0x7f7fa408bf90 msgr2=0x7f7fa40d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 shutdown_connections 2024-09-19T04:18:46.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.429+0000 7f7fac2dd640 1 -- 172.21.15.76:0/3685416500 wait complete. 2024-09-19T04:18:46.448 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:46.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.557+0000 7f8854bbb640 1 Processor -- start 2024-09-19T04:18:46.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.557+0000 7f8854bbb640 1 -- start start 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f8854bbb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f88500d3060 0x7f88500cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f8854bbb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500cf450 0x7f885006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f8854bbb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88500d0300 con 0x7f8850161a60 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f8854bbb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f88500b0f60 con 0x7f88500d3060 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f8854bbb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f88500cfd40 con 0x7f88500cf450 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884e59c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8850161a60 0x7f88500d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46956/0 (socket says 172.21.15.76:46956) 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884e59c640 1 -- 172.21.15.76:0/2599074772 learned_addr learned my addr 172.21.15.76:0/2599074772 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884dd9b640 1 --2- 172.21.15.76:0/2599074772 >> v2:172.21.15.76:3300/0 conn(0x7f88500d3060 0x7f88500cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884dd9b640 1 -- 172.21.15.76:0/2599074772 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500cf450 msgr2=0x7f885006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884dd9b640 1 --2- 172.21.15.76:0/2599074772 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500cf450 0x7f885006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884dd9b640 1 -- 172.21.15.76:0/2599074772 >> v1:172.21.15.76:6789/0 conn(0x7f8850161a60 legacy=0x7f88500d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884d59a640 1 -- 172.21.15.76:0/2599074772 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3732803005 0 0) 0x7f8840001120 con 0x7f8850161a60 2024-09-19T04:18:46.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884dd9b640 1 -- 172.21.15.76:0/2599074772 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88500d0020 con 0x7f88500d3060 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884dd9b640 1 --2- 172.21.15.76:0/2599074772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500d3060 0x7f88500cef10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f88440097c0 tx=0x7f884401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29a39f98c423303 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884d59a640 1 -- 172.21.15.76:0/2599074772 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88440b1070 con 0x7f88500d3060 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884d59a640 1 -- 172.21.15.76:0/2599074772 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88440a8050 con 0x7f88500d3060 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.561+0000 7f884d59a640 1 -- 172.21.15.76:0/2599074772 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88440ac860 con 0x7f88500d3060 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/2599074772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500d3060 msgr2=0x7f88500cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 --2- 172.21.15.76:0/2599074772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500d3060 0x7f88500cef10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f88440097c0 tx=0x7f884401c3e0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/2599074772 shutdown_connections 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 --2- 172.21.15.76:0/2599074772 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500cf450 0x7f885006d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 --2- 172.21.15.76:0/2599074772 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500d3060 0x7f88500cef10 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/2599074772 >> 172.21.15.76:0/2599074772 conn(0x7f885008bf90 msgr2=0x7f885009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:46.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/2599074772 shutdown_connections 2024-09-19T04:18:46.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/2599074772 wait complete. 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 Processor -- start 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- start start 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500cf450 0x7f88501538d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500d3060 0x7f8850159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8850161a60 0x7f88500cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f885006e400 con 0x7f88500cf450 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f885006e580 con 0x7f8850161a60 2024-09-19T04:18:46.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f885006e700 con 0x7f88500d3060 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500cf450 0x7f88501538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500cf450 0x7f88501538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46474/0 (socket says 172.21.15.76:46474) 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 -- 172.21.15.76:0/4283868852 learned_addr learned my addr 172.21.15.76:0/4283868852 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 -- 172.21.15.76:0/4283868852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500d3060 msgr2=0x7f8850159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500d3060 0x7f8850159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 -- 172.21.15.76:0/4283868852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8850161a60 msgr2=0x7f88500cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8850161a60 0x7f88500cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 -- 172.21.15.76:0/4283868852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88500cb980 con 0x7f88500cf450 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f884e59c640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500cf450 0x7f88501538d0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f884000c780 tx=0x7f884000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f883effd640 1 -- 172.21.15.76:0/4283868852 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8840018070 con 0x7f88500cf450 2024-09-19T04:18:46.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f883effd640 1 -- 172.21.15.76:0/4283868852 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88400072f0 con 0x7f88500cf450 2024-09-19T04:18:46.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f885015f850 con 0x7f88500cf450 2024-09-19T04:18:46.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f883effd640 1 -- 172.21.15.76:0/4283868852 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88400047f0 con 0x7f88500cf450 2024-09-19T04:18:46.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88501583d0 con 0x7f88500cf450 2024-09-19T04:18:46.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.565+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f885006d6b0 con 0x7f88500cf450 2024-09-19T04:18:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.569+0000 7f883effd640 1 -- 172.21.15.76:0/4283868852 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f884001b070 con 0x7f88500cf450 2024-09-19T04:18:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.569+0000 7f883effd640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f88300678c0 0x7f8830069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.569+0000 7f883effd640 1 -- 172.21.15.76:0/4283868852 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7f88400851a0 con 0x7f88500cf450 2024-09-19T04:18:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.569+0000 7f884dd9b640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f88300678c0 0x7f8830069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.569+0000 7f884dd9b640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f88300678c0 0x7f8830069d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f88440ae000 tx=0x7f8844008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.573+0000 7f883effd640 1 -- 172.21.15.76:0/4283868852 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f88500d0300 con 0x7f88500cf450 2024-09-19T04:18:46.749 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.749+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f885015a880 con 0x7f88500cf450 2024-09-19T04:18:46.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.749+0000 7f883effd640 1 -- 172.21.15.76:0/4283868852 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16982 (secure 0 0 0) 0x7f8840055be0 con 0x7f88500cf450 2024-09-19T04:18:46.751 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:46.752 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":90,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:45.404177+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:43.054193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:44.388097+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f88300678c0 msgr2=0x7f8830069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f88300678c0 0x7f8830069d80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f88440ae000 tx=0x7f8844008040 comp rx=0 tx=0).stop 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500cf450 msgr2=0x7f88501538d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500cf450 0x7f88501538d0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f884000c780 tx=0x7f884000cc40 comp rx=0 tx=0).stop 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 shutdown_connections 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f88300678c0 0x7f8830069d80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8850161a60 0x7f88500cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f88500d3060 0x7f8850159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 --2- 172.21.15.76:0/4283868852 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f88500cf450 0x7f88501538d0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 >> 172.21.15.76:0/4283868852 conn(0x7f885008bf90 msgr2=0x7f88500b4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:46.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 shutdown_connections 2024-09-19T04:18:46.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.753+0000 7f8854bbb640 1 -- 172.21.15.76:0/4283868852 wait complete. 2024-09-19T04:18:46.771 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:18:46.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf1a867640 1 Processor -- start 2024-09-19T04:18:46.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf1a867640 1 -- start start 2024-09-19T04:18:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf1a867640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf140b39a0 0x7fbf1406df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf1a867640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf1a867640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1406ec60 con 0x7fbf141587a0 2024-09-19T04:18:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf1a867640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1406f870 con 0x7fbf140b39a0 2024-09-19T04:18:46.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf1a867640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbf1406ef40 con 0x7fbf1406e4b0 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf137fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf140b39a0 0x7fbf1406df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf137fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf140b39a0 0x7fbf1406df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46486/0 (socket says 172.21.15.76:46486) 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf137fe640 1 -- 172.21.15.76:0/1891302208 learned_addr learned my addr 172.21.15.76:0/1891302208 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf18e04640 1 --2- 172.21.15.76:0/1891302208 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf137fe640 1 -- 172.21.15.76:0/1891302208 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 msgr2=0x7fbf14072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf137fe640 1 --2- 172.21.15.76:0/1891302208 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf137fe640 1 -- 172.21.15.76:0/1891302208 >> v1:172.21.15.116:6789/0 conn(0x7fbf141587a0 legacy=0x7fbf140b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf137fe640 1 -- 172.21.15.76:0/1891302208 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1406f220 con 0x7fbf140b39a0 2024-09-19T04:18:46.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.885+0000 7fbf18e04640 1 --2- 172.21.15.76:0/1891302208 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:46.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf137fe640 1 --2- 172.21.15.76:0/1891302208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf140b39a0 0x7fbf1406df70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009870 tx=0x7fbf0401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56facecf7c7fa65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf12ffd640 1 -- 172.21.15.76:0/1891302208 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040b1070 con 0x7fbf140b39a0 2024-09-19T04:18:46.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf12ffd640 1 -- 172.21.15.76:0/1891302208 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf040ac630 con 0x7fbf140b39a0 2024-09-19T04:18:46.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf12ffd640 1 -- 172.21.15.76:0/1891302208 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a8050 con 0x7fbf140b39a0 2024-09-19T04:18:46.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- 172.21.15.76:0/1891302208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf140b39a0 msgr2=0x7fbf1406df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 --2- 172.21.15.76:0/1891302208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf140b39a0 0x7fbf1406df70 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009870 tx=0x7fbf0401c4f0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- 172.21.15.76:0/1891302208 shutdown_connections 2024-09-19T04:18:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 --2- 172.21.15.76:0/1891302208 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 --2- 172.21.15.76:0/1891302208 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf140b39a0 0x7fbf1406df70 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- 172.21.15.76:0/1891302208 >> 172.21.15.76:0/1891302208 conn(0x7fbf140b1f90 msgr2=0x7fbf140b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- 172.21.15.76:0/1891302208 shutdown_connections 2024-09-19T04:18:46.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- 172.21.15.76:0/1891302208 wait complete. 2024-09-19T04:18:46.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 Processor -- start 2024-09-19T04:18:46.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- start start 2024-09-19T04:18:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf140b39a0 0x7fbf141530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf14150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14073030 con 0x7fbf141587a0 2024-09-19T04:18:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14072eb0 con 0x7fbf140b39a0 2024-09-19T04:18:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf1a867640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbf140731b0 con 0x7fbf1406e4b0 2024-09-19T04:18:46.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf18e04640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf14150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf18e04640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf14150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46502/0 (socket says 172.21.15.76:46502) 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.889+0000 7fbf18e04640 1 -- 172.21.15.76:0/3673025581 learned_addr learned my addr 172.21.15.76:0/3673025581 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf13fff640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf137fe640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf140b39a0 0x7fbf141530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf18e04640 1 -- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 msgr2=0x7fbf14155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf18e04640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf18e04640 1 -- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf140b39a0 msgr2=0x7fbf141530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf18e04640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf140b39a0 0x7fbf141530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf18e04640 1 -- 172.21.15.76:0/3673025581 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf14154e30 con 0x7fbf141587a0 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf137fe640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf140b39a0 0x7fbf141530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf18e04640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf14150910 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c007f20 tx=0x7fbf0c00d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf10ff9640 1 -- 172.21.15.76:0/3673025581 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c01a070 con 0x7fbf141587a0 2024-09-19T04:18:46.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf10ff9640 1 -- 172.21.15.76:0/3673025581 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c003ba0 con 0x7fbf141587a0 2024-09-19T04:18:46.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf10ff9640 1 -- 172.21.15.76:0/3673025581 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c004030 con 0x7fbf141587a0 2024-09-19T04:18:46.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf141fb730 con 0x7fbf141587a0 2024-09-19T04:18:46.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf141553b0 con 0x7fbf141587a0 2024-09-19T04:18:46.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf14072160 con 0x7fbf141587a0 2024-09-19T04:18:46.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf10ff9640 1 -- 172.21.15.76:0/3673025581 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf0c01d070 con 0x7fbf141587a0 2024-09-19T04:18:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf10ff9640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf000678c0 0x7fbf00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf10ff9640 1 -- 172.21.15.76:0/3673025581 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6101+0+0 (secure 0 0 0) 0x7fbf0c086160 con 0x7fbf141587a0 2024-09-19T04:18:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf13fff640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf000678c0 0x7fbf00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:46.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.893+0000 7fbf13fff640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf000678c0 0x7fbf00069d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800b810 tx=0x7fbf08008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:46.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:46.897+0000 7fbf10ff9640 1 -- 172.21.15.76:0/3673025581 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbf0c010070 con 0x7fbf141587a0 2024-09-19T04:18:47.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:47.093+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbf140d1090 con 0x7fbf141587a0 2024-09-19T04:18:48.014 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:18:48.013+0000 7f4997ef7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:18:48.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.105+0000 7fbf10ff9640 1 -- 172.21.15.76:0/3673025581 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v83) ==== 118+0+0 (secure 0 0 0) 0x7fbf0c056b10 con 0x7fbf141587a0 2024-09-19T04:18:48.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf000678c0 msgr2=0x7fbf00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf000678c0 0x7fbf00069d80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fbf0800b810 tx=0x7fbf08008040 comp rx=0 tx=0).stop 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 msgr2=0x7fbf14150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf14150910 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c007f20 tx=0x7fbf0c00d560 comp rx=0 tx=0).stop 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 shutdown_connections 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbf000678c0 0x7fbf00069d80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf14150910 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf140b39a0 0x7fbf141530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 --2- 172.21.15.76:0/3673025581 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf1406e4b0 0x7fbf14155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 >> 172.21.15.76:0/3673025581 conn(0x7fbf140b1f90 msgr2=0x7fbf140ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 shutdown_connections 2024-09-19T04:18:48.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.109+0000 7fbf1a867640 1 -- 172.21.15.76:0/3673025581 wait complete. 2024-09-19T04:18:48.133 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T04:18:48.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T04:18:48.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3fbb8640 1 Processor -- start 2024-09-19T04:18:48.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3fbb8640 1 -- start start 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3fbb8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc3815f3a0 0x7fdc38165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3fbb8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc380aede0 con 0x7fdc381520a0 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3fbb8640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc3815fa10 con 0x7fdc38070020 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3fbb8640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdc38152480 con 0x7fdc3815f3a0 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3e155640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc3815f3a0 0x7fdc38165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3e155640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdc3815f3a0 0x7fdc38165f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46504/0 (socket says 172.21.15.76:46504) 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3e155640 1 -- 172.21.15.76:0/701265754 learned_addr learned my addr 172.21.15.76:0/701265754 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3e155640 1 -- 172.21.15.76:0/701265754 >> v1:172.21.15.116:6789/0 conn(0x7fdc38070020 legacy=0x7fdc3815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3e155640 1 -- 172.21.15.76:0/701265754 >> v1:172.21.15.76:6789/0 conn(0x7fdc381520a0 legacy=0x7fdc3806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:48.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3e155640 1 -- 172.21.15.76:0/701265754 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc38074250 con 0x7fdc3815f3a0 2024-09-19T04:18:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.261+0000 7fdc3e155640 1 --2- 172.21.15.76:0/701265754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc3815f3a0 0x7fdc38165f00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009e50 tx=0x7fdc3001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0b1768f43144c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3c952640 1 -- 172.21.15.76:0/701265754 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300a58b0 con 0x7fdc3815f3a0 2024-09-19T04:18:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3c952640 1 -- 172.21.15.76:0/701265754 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc300a5a50 con 0x7fdc3815f3a0 2024-09-19T04:18:48.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3c952640 1 -- 172.21.15.76:0/701265754 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300a5e20 con 0x7fdc3815f3a0 2024-09-19T04:18:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/701265754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc3815f3a0 msgr2=0x7fdc38165f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/701265754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc3815f3a0 0x7fdc38165f00 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdc30009e50 tx=0x7fdc3001c9f0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/701265754 shutdown_connections 2024-09-19T04:18:48.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/701265754 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc3815f3a0 0x7fdc38165f00 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/701265754 >> 172.21.15.76:0/701265754 conn(0x7fdc380b1f90 msgr2=0x7fdc380b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/701265754 shutdown_connections 2024-09-19T04:18:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/701265754 wait complete. 2024-09-19T04:18:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 Processor -- start 2024-09-19T04:18:48.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- start start 2024-09-19T04:18:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc38070020 0x7fdc38099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc381520a0 0x7fdc38150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:48.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc3815f3a0 0x7fdc380adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdc38168c60 con 0x7fdc38070020 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdc38073d30 con 0x7fdc3815f3a0 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3fbb8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdc38168de0 con 0x7fdc381520a0 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc38070020 0x7fdc38099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc38070020 0x7fdc38099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46514/0 (socket says 172.21.15.76:46514) 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 -- 172.21.15.76:0/359065727 learned_addr learned my addr 172.21.15.76:0/359065727 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d153640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc381520a0 0x7fdc38150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 -- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc381520a0 msgr2=0x7fdc38150700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3e155640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc3815f3a0 0x7fdc380adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc381520a0 0x7fdc38150700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 -- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc3815f3a0 msgr2=0x7fdc380adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:48.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc3815f3a0 0x7fdc380adc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d954640 1 -- 172.21.15.76:0/359065727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc3806e930 con 0x7fdc38070020 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.265+0000 7fdc3d153640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc381520a0 0x7fdc38150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc3d954640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc38070020 0x7fdc38099a40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c420 tx=0x7fdc3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc2e7fc640 1 -- 172.21.15.76:0/359065727 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc340052d0 con 0x7fdc38070020 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc2e7fc640 1 -- 172.21.15.76:0/359065727 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc34005470 con 0x7fdc38070020 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc381511c0 con 0x7fdc38070020 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc2e7fc640 1 -- 172.21.15.76:0/359065727 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34007260 con 0x7fdc38070020 2024-09-19T04:18:48.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc3814d180 con 0x7fdc38070020 2024-09-19T04:18:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc3815ec90 con 0x7fdc38070020 2024-09-19T04:18:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc2e7fc640 1 -- 172.21.15.76:0/359065727 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdc34007400 con 0x7fdc38070020 2024-09-19T04:18:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc2e7fc640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc1c0678c0 0x7fdc1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc2e7fc640 1 -- 172.21.15.76:0/359065727 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fdc34085310 con 0x7fdc38070020 2024-09-19T04:18:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc3d153640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc1c0678c0 0x7fdc1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:48.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.269+0000 7fdc3d153640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc1c0678c0 0x7fdc1c069d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdc38148650 tx=0x7fdc28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:48.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.273+0000 7fdc2e7fc640 1 -- 172.21.15.76:0/359065727 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fdc34055d50 con 0x7fdc38070020 2024-09-19T04:18:48.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:48.469+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdc3806e540 con 0x7fdc38070020 2024-09-19T04:18:49.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.113+0000 7fdc2e7fc640 1 -- 172.21.15.76:0/359065727 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v84) ==== 122+0+0 (secure 0 0 0) 0x7fdc34055650 con 0x7fdc38070020 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc1c0678c0 msgr2=0x7fdc1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc1c0678c0 0x7fdc1c069d80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdc38148650 tx=0x7fdc28008040 comp rx=0 tx=0).stop 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc38070020 msgr2=0x7fdc38099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc38070020 0x7fdc38099a40 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c420 tx=0x7fdc3400c8e0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 shutdown_connections 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdc1c0678c0 0x7fdc1c069d80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdc3815f3a0 0x7fdc380adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdc381520a0 0x7fdc38150700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 --2- 172.21.15.76:0/359065727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdc38070020 0x7fdc38099a40 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 >> 172.21.15.76:0/359065727 conn(0x7fdc380b1f90 msgr2=0x7fdc3815a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 shutdown_connections 2024-09-19T04:18:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.117+0000 7fdc3fbb8640 1 -- 172.21.15.76:0/359065727 wait complete. 2024-09-19T04:18:49.140 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T04:18:49.141 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:49.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86f814640 1 Processor -- start 2024-09-19T04:18:49.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86f814640 1 -- start start 2024-09-19T04:18:49.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86f814640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb868161a60 0x7fb8680d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86f814640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 0x7fb8680cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86f814640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8680d32b0 con 0x7fb8680cf450 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86f814640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb8680d0300 con 0x7fb868161a60 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86f814640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb86806e6c0 con 0x7fb8680d2e90 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86d5b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb868161a60 0x7fb8680d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86d5b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb868161a60 0x7fb8680d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46530/0 (socket says 172.21.15.76:46530) 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86d5b0640 1 -- 172.21.15.76:0/3700986178 learned_addr learned my addr 172.21.15.76:0/3700986178 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86cdaf640 1 --2- 172.21.15.76:0/3700986178 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 0x7fb8680cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86d5b0640 1 -- 172.21.15.76:0/3700986178 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 msgr2=0x7fb8680cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86d5b0640 1 --2- 172.21.15.76:0/3700986178 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 0x7fb8680cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86d5b0640 1 -- 172.21.15.76:0/3700986178 >> v1:172.21.15.116:6789/0 conn(0x7fb8680cf450 legacy=0x7fb8680d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:49.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.265+0000 7fb86d5b0640 1 -- 172.21.15.76:0/3700986178 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8680b0f60 con 0x7fb868161a60 2024-09-19T04:18:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86d5b0640 1 --2- 172.21.15.76:0/3700986178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb868161a60 0x7fb8680d2950 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb860009870 tx=0x7fb86001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fae032013e8a62c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb867fff640 1 -- 172.21.15.76:0/3700986178 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8680d1930 con 0x7fb868161a60 2024-09-19T04:18:49.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb867fff640 1 -- 172.21.15.76:0/3700986178 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8600b0070 con 0x7fb868161a60 2024-09-19T04:18:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb867fff640 1 -- 172.21.15.76:0/3700986178 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8600ab840 con 0x7fb868161a60 2024-09-19T04:18:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- 172.21.15.76:0/3700986178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb868161a60 msgr2=0x7fb8680d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 --2- 172.21.15.76:0/3700986178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb868161a60 0x7fb8680d2950 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb860009870 tx=0x7fb86001c460 comp rx=0 tx=0).stop 2024-09-19T04:18:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- 172.21.15.76:0/3700986178 shutdown_connections 2024-09-19T04:18:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 --2- 172.21.15.76:0/3700986178 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 0x7fb8680cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 --2- 172.21.15.76:0/3700986178 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb868161a60 0x7fb8680d2950 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- 172.21.15.76:0/3700986178 >> 172.21.15.76:0/3700986178 conn(0x7fb86808bf90 msgr2=0x7fb86809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- 172.21.15.76:0/3700986178 shutdown_connections 2024-09-19T04:18:49.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- 172.21.15.76:0/3700986178 wait complete. 2024-09-19T04:18:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 Processor -- start 2024-09-19T04:18:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- start start 2024-09-19T04:18:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8680cf450 0x7fb8680b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 0x7fb8680b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb868150cc0 0x7fb868151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb8680cf8e0 con 0x7fb8680cf450 2024-09-19T04:18:49.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb86806d190 con 0x7fb868150cc0 2024-09-19T04:18:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86f814640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb8680cfa60 con 0x7fb8680d2e90 2024-09-19T04:18:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.269+0000 7fb86d5b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8680cf450 0x7fb8680b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8680cf450 0x7fb8680b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46538/0 (socket says 172.21.15.76:46538) 2024-09-19T04:18:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 -- 172.21.15.76:0/2957878362 learned_addr learned my addr 172.21.15.76:0/2957878362 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 -- 172.21.15.76:0/2957878362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 msgr2=0x7fb8680b5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 0x7fb8680b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 -- 172.21.15.76:0/2957878362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb868150cc0 msgr2=0x7fb868151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb868150cc0 0x7fb868151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 -- 172.21.15.76:0/2957878362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb868160c40 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86d5b0640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8680cf450 0x7fb8680b4fd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb8600ad040 tx=0x7fb86001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb865ffb640 1 -- 172.21.15.76:0/2957878362 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8600abd10 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb865ffb640 1 -- 172.21.15.76:0/2957878362 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8600be3f0 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb865ffb640 1 -- 172.21.15.76:0/2957878362 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8600be7e0 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8680b2210 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8680b27d0 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8680b3290 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb865ffb640 1 -- 172.21.15.76:0/2957878362 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb8600b43f0 con 0x7fb8680cf450 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb865ffb640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8540678f0 0x7fb854069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb865ffb640 1 -- 172.21.15.76:0/2957878362 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb860120130 con 0x7fb8680cf450 2024-09-19T04:18:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86cdaf640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8540678f0 0x7fb854069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.273+0000 7fb86cdaf640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8540678f0 0x7fb854069db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0070b0 tx=0x7fb85c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.277+0000 7fb865ffb640 1 -- 172.21.15.76:0/2957878362 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb8680b3290 con 0x7fb8680cf450 2024-09-19T04:18:49.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.473+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8680cdc40 con 0x7fb8680cf450 2024-09-19T04:18:49.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.473+0000 7fb865ffb640 1 -- 172.21.15.76:0/2957878362 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5228 (secure 0 0 0) 0x7fb8600c4060 con 0x7fb8680cf450 2024-09-19T04:18:49.475 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:49.475 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"btime":"2024-09-19T04:18:49:115048+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":5794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:18:48.474617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-19T04:18:49.475 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 84 2024-09-19T04:18:49.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8540678f0 msgr2=0x7fb854069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8540678f0 0x7fb854069db0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0070b0 tx=0x7fb85c007040 comp rx=0 tx=0).stop 2024-09-19T04:18:49.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8680cf450 msgr2=0x7fb8680b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8680cf450 0x7fb8680b4fd0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb8600ad040 tx=0x7fb86001aca0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 shutdown_connections 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb8540678f0 0x7fb854069db0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb868150cc0 0x7fb868151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb8680d2e90 0x7fb8680b5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 --2- 172.21.15.76:0/2957878362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb8680cf450 0x7fb8680b4fd0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 >> 172.21.15.76:0/2957878362 conn(0x7fb86808bf90 msgr2=0x7fb8680cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 shutdown_connections 2024-09-19T04:18:49.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.477+0000 7fb86f814640 1 -- 172.21.15.76:0/2957878362 wait complete. 2024-09-19T04:18:49.495 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:18:49.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.613+0000 7fb70624a640 1 Processor -- start 2024-09-19T04:18:49.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.613+0000 7fb70624a640 1 -- start start 2024-09-19T04:18:49.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb70624a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7000d3060 0x7fb7000cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb70624a640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7000cf450 0x7fb70006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb70624a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7000d0300 con 0x7fb700161a60 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb70624a640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb7000b0f60 con 0x7fb7000cf450 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb70624a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb7000cfd40 con 0x7fb7000d3060 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7000cf450 0x7fb70006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb7000cf450 0x7fb70006d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46548/0 (socket says 172.21.15.76:46548) 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 -- 172.21.15.76:0/3162712027 learned_addr learned my addr 172.21.15.76:0/3162712027 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6feffd640 1 --2- 172.21.15.76:0/3162712027 >> v2:172.21.15.116:3300/0 conn(0x7fb7000d3060 0x7fb7000cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 -- 172.21.15.76:0/3162712027 >> v2:172.21.15.116:3300/0 conn(0x7fb7000d3060 msgr2=0x7fb7000cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 --2- 172.21.15.76:0/3162712027 >> v2:172.21.15.116:3300/0 conn(0x7fb7000d3060 0x7fb7000cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 -- 172.21.15.76:0/3162712027 >> v1:172.21.15.116:6789/0 conn(0x7fb700161a60 legacy=0x7fb7000d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:49.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 -- 172.21.15.76:0/3162712027 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7000d0020 con 0x7fb7000cf450 2024-09-19T04:18:49.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fffff640 1 --2- 172.21.15.76:0/3162712027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb70006d6b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb70008d850 tx=0x7fb6f801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5adabb5031dd7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fe7fc640 1 -- 172.21.15.76:0/3162712027 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f80a55d0 con 0x7fb7000cf450 2024-09-19T04:18:49.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fe7fc640 1 -- 172.21.15.76:0/3162712027 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f80a5770 con 0x7fb7000cf450 2024-09-19T04:18:49.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.617+0000 7fb6fe7fc640 1 -- 172.21.15.76:0/3162712027 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f80a5b40 con 0x7fb7000cf450 2024-09-19T04:18:49.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- 172.21.15.76:0/3162712027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 msgr2=0x7fb70006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 --2- 172.21.15.76:0/3162712027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb70006d6b0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb70008d850 tx=0x7fb6f801c710 comp rx=0 tx=0).stop 2024-09-19T04:18:49.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- 172.21.15.76:0/3162712027 shutdown_connections 2024-09-19T04:18:49.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 --2- 172.21.15.76:0/3162712027 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb70006d6b0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 --2- 172.21.15.76:0/3162712027 >> v2:172.21.15.116:3300/0 conn(0x7fb7000d3060 0x7fb7000cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- 172.21.15.76:0/3162712027 >> 172.21.15.76:0/3162712027 conn(0x7fb70008bf90 msgr2=0x7fb70009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- 172.21.15.76:0/3162712027 shutdown_connections 2024-09-19T04:18:49.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- 172.21.15.76:0/3162712027 wait complete. 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 Processor -- start 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- start start 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb7000b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7000d3060 0x7fb7000ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb700161a60 0x7fb700152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb70006e400 con 0x7fb7000cf450 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb7000cfad0 con 0x7fb700161a60 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb70006e580 con 0x7fb7000d3060 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb7000b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb7000b9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46562/0 (socket says 172.21.15.76:46562) 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 -- 172.21.15.76:0/3209612612 learned_addr learned my addr 172.21.15.76:0/3209612612 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6feffd640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7000d3060 0x7fb7000ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6fffff640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb700161a60 0x7fb700152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 -- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7000d3060 msgr2=0x7fb7000ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7000d3060 0x7fb7000ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 -- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb700161a60 msgr2=0x7fb700152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb700161a60 0x7fb700152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 -- 172.21.15.76:0/3209612612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7000c8260 con 0x7fb7000cf450 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6ff7fe640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb7000b9e40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b410 tx=0x7fb6f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6e7fff640 1 -- 172.21.15.76:0/3209612612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0015020 con 0x7fb7000cf450 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6e7fff640 1 -- 172.21.15.76:0/3209612612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6f0011830 con 0x7fb7000cf450 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb6e7fff640 1 -- 172.21.15.76:0/3209612612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0011c20 con 0x7fb7000cf450 2024-09-19T04:18:49.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7000c8540 con 0x7fb7000cf450 2024-09-19T04:18:49.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.621+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7000c8b00 con 0x7fb7000cf450 2024-09-19T04:18:49.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.625+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7000c8f00 con 0x7fb7000cf450 2024-09-19T04:18:49.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.625+0000 7fb6e7fff640 1 -- 172.21.15.76:0/3209612612 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb6f0018020 con 0x7fb7000cf450 2024-09-19T04:18:49.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.625+0000 7fb6e7fff640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6ec0678c0 0x7fb6ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.625+0000 7fb6e7fff640 1 -- 172.21.15.76:0/3209612612 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb6f0085df0 con 0x7fb7000cf450 2024-09-19T04:18:49.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.625+0000 7fb6feffd640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6ec0678c0 0x7fb6ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.625+0000 7fb6feffd640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6ec0678c0 0x7fb6ec069d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb7001fa5d0 tx=0x7fb6f4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.629+0000 7fb6e7fff640 1 -- 172.21.15.76:0/3209612612 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb6f001f020 con 0x7fb7000cf450 2024-09-19T04:18:49.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.793+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb70006d6b0 con 0x7fb7000cf450 2024-09-19T04:18:49.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.793+0000 7fb6e7fff640 1 -- 172.21.15.76:0/3209612612 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16969 (secure 0 0 0) 0x7fb6f00172c0 con 0x7fb7000cf450 2024-09-19T04:18:49.795 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:49.795 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:18:47.086918+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:43.054193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:44.388097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:18:49.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6ec0678c0 msgr2=0x7fb6ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6ec0678c0 0x7fb6ec069d80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb7001fa5d0 tx=0x7fb6f4008040 comp rx=0 tx=0).stop 2024-09-19T04:18:49.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 msgr2=0x7fb7000b9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb7000b9e40 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b410 tx=0x7fb6f000b8d0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 shutdown_connections 2024-09-19T04:18:49.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb6ec0678c0 0x7fb6ec069d80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb700161a60 0x7fb700152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7000d3060 0x7fb7000ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 --2- 172.21.15.76:0/3209612612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb7000cf450 0x7fb7000b9e40 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 >> 172.21.15.76:0/3209612612 conn(0x7fb70008bf90 msgr2=0x7fb70015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 shutdown_connections 2024-09-19T04:18:49.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.797+0000 7fb70624a640 1 -- 172.21.15.76:0/3209612612 wait complete. 2024-09-19T04:18:49.814 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:18:49.814 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:18:49.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.929+0000 7f64202a9640 1 Processor -- start 2024-09-19T04:18:49.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.929+0000 7f64202a9640 1 -- start start 2024-09-19T04:18:49.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f64202a9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f641815bec0 0x7f64180ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f64202a9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64180b3290 con 0x7f64180af470 2024-09-19T04:18:49.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f64202a9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64180b3570 con 0x7f641815c2a0 2024-09-19T04:18:49.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f64202a9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f64180b3850 con 0x7f641815bec0 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e045640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f641815bec0 0x7f64180ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e045640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f641815bec0 0x7f64180ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46572/0 (socket says 172.21.15.76:46572) 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e045640 1 -- 172.21.15.76:0/1091776523 learned_addr learned my addr 172.21.15.76:0/1091776523 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e846640 1 --1- 172.21.15.76:0/1091776523 >> v1:172.21.15.76:6789/0 conn(0x7f64180af470 0x7f641806e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46984/0 (socket says 172.21.15.76:46984) 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e045640 1 -- 172.21.15.76:0/1091776523 >> v1:172.21.15.116:6789/0 conn(0x7f641815c2a0 legacy=0x7f64180aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e045640 1 -- 172.21.15.76:0/1091776523 >> v1:172.21.15.76:6789/0 conn(0x7f64180af470 legacy=0x7f641806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e045640 1 -- 172.21.15.76:0/1091776523 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64180b3b30 con 0x7f641815bec0 2024-09-19T04:18:49.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641d043640 1 -- 172.21.15.76:0/1091776523 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1451795921 0 0) 0x7f64180b3290 con 0x7f64180af470 2024-09-19T04:18:49.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641e045640 1 --2- 172.21.15.76:0/1091776523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815bec0 0x7f64180ad200 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6414009870 tx=0x7f641401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3087d5a40997627c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.935 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641d043640 1 -- 172.21.15.76:0/1091776523 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140aa650 con 0x7f641815bec0 2024-09-19T04:18:49.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641d043640 1 -- 172.21.15.76:0/1091776523 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64140af070 con 0x7f641815bec0 2024-09-19T04:18:49.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.933+0000 7f641d043640 1 -- 172.21.15.76:0/1091776523 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64140aaa10 con 0x7f641815bec0 2024-09-19T04:18:49.936 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1091776523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815bec0 msgr2=0x7f64180ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 --2- 172.21.15.76:0/1091776523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815bec0 0x7f64180ad200 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f6414009870 tx=0x7f641401c4f0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1091776523 shutdown_connections 2024-09-19T04:18:49.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 --2- 172.21.15.76:0/1091776523 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815bec0 0x7f64180ad200 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1091776523 >> 172.21.15.76:0/1091776523 conn(0x7f64180b1f90 msgr2=0x7f64180b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:49.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1091776523 shutdown_connections 2024-09-19T04:18:49.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1091776523 wait complete. 2024-09-19T04:18:49.937 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 Processor -- start 2024-09-19T04:18:49.938 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- start start 2024-09-19T04:18:49.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64180af470 0x7f641815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f641815bec0 0x7f6418157920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815c2a0 0x7f6418147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f641806ef20 con 0x7f641815c2a0 2024-09-19T04:18:49.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64180ad740 con 0x7f64180af470 2024-09-19T04:18:49.940 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f641806f0a0 con 0x7f641815bec0 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815c2a0 0x7f6418147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815c2a0 0x7f6418147bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46578/0 (socket says 172.21.15.76:46578) 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 -- 172.21.15.76:0/1344196357 learned_addr learned my addr 172.21.15.76:0/1344196357 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641d844640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f641815bec0 0x7f6418157920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e045640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64180af470 0x7f641815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 -- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f641815bec0 msgr2=0x7f6418157920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f641815bec0 0x7f6418157920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 -- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64180af470 msgr2=0x7f641815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64180af470 0x7f641815c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 -- 172.21.15.76:0/1344196357 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64180add70 con 0x7f641815c2a0 2024-09-19T04:18:49.941 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641d844640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f641815bec0 0x7f6418157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:49.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f641e846640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815c2a0 0x7f6418147bf0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6410001570 tx=0x7f641000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f640effd640 1 -- 172.21.15.76:0/1344196357 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6410018070 con 0x7f641815c2a0 2024-09-19T04:18:49.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f640effd640 1 -- 172.21.15.76:0/1344196357 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f641000a640 con 0x7f641815c2a0 2024-09-19T04:18:49.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6418154eb0 con 0x7f641815c2a0 2024-09-19T04:18:49.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f640effd640 1 -- 172.21.15.76:0/1344196357 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f641000aa30 con 0x7f641815c2a0 2024-09-19T04:18:49.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6418148130 con 0x7f641815c2a0 2024-09-19T04:18:49.942 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.937+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64180ad200 con 0x7f641815c2a0 2024-09-19T04:18:49.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.941+0000 7f640effd640 1 -- 172.21.15.76:0/1344196357 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f641001b070 con 0x7f641815c2a0 2024-09-19T04:18:49.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.941+0000 7f640effd640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63fc0678c0 0x7f63fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:49.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.941+0000 7f640effd640 1 -- 172.21.15.76:0/1344196357 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f64100854a0 con 0x7f641815c2a0 2024-09-19T04:18:49.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.941+0000 7f641e045640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63fc0678c0 0x7f63fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:49.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.941+0000 7f641e045640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63fc0678c0 0x7f63fc069d80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f64140ae000 tx=0x7f64140a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:49.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:49.945+0000 7f640effd640 1 -- 172.21.15.76:0/1344196357 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6410055e40 con 0x7f641815c2a0 2024-09-19T04:18:50.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.105+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f641806fcd0 con 0x7f63fc0678c0 2024-09-19T04:18:50.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.113+0000 7f640effd640 1 -- 172.21.15.76:0/1344196357 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7f641806fcd0 con 0x7f63fc0678c0 2024-09-19T04:18:50.117 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:50.119 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:18:50.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63fc0678c0 msgr2=0x7f63fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63fc0678c0 0x7f63fc069d80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f64140ae000 tx=0x7f64140a7000 comp rx=0 tx=0).stop 2024-09-19T04:18:50.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815c2a0 msgr2=0x7f6418147bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815c2a0 0x7f6418147bf0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f6410001570 tx=0x7f641000cca0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 shutdown_connections 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f63fc0678c0 0x7f63fc069d80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f641815c2a0 0x7f6418147bf0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f641815bec0 0x7f6418157920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 --2- 172.21.15.76:0/1344196357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64180af470 0x7f641815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 >> 172.21.15.76:0/1344196357 conn(0x7f64180b1f90 msgr2=0x7f64180b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 shutdown_connections 2024-09-19T04:18:50.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.117+0000 7f64202a9640 1 -- 172.21.15.76:0/1344196357 wait complete. 2024-09-19T04:18:50.136 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-19T04:18:49.842473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255284,"kb_used_data":9912,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719436,"statfs":{"total":773094113280,"available":772832702464,"internally_reserved":0,"allocated":10149888,"data_stored":5311124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.822903"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098166+0000","last_change":"2024-09-19T04:18:46.097510+0000","last_active":"2024-09-19T04:18:47.098166+0000","last_peered":"2024-09-19T04:18:47.098166+0000","last_clean":"2024-09-19T04:18:47.098166+0000","last_became_active":"2024-09-19T04:18:46.097122+0000","last_became_peered":"2024-09-19T04:18:46.097122+0000","last_unstale":"2024-09-19T04:18:47.098166+0000","last_undegraded":"2024-09-19T04:18:47.098166+0000","last_fullsized":"2024-09-19T04:18:47.098166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:01.024313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100226+0000","last_change":"2024-09-19T04:18:46.105337+0000","last_active":"2024-09-19T04:18:47.100226+0000","last_peered":"2024-09-19T04:18:47.100226+0000","last_clean":"2024-09-19T04:18:47.100226+0000","last_became_active":"2024-09-19T04:18:46.104585+0000","last_became_peered":"2024-09-19T04:18:46.104585+0000","last_unstale":"2024-09-19T04:18:47.100226+0000","last_undegraded":"2024-09-19T04:18:47.100226+0000","last_fullsized":"2024-09-19T04:18:47.100226+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:41:32.235161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098039+0000","last_change":"2024-09-19T04:18:46.111561+0000","last_active":"2024-09-19T04:18:47.098039+0000","last_peered":"2024-09-19T04:18:47.098039+0000","last_clean":"2024-09-19T04:18:47.098039+0000","last_became_active":"2024-09-19T04:18:46.111293+0000","last_became_peered":"2024-09-19T04:18:46.111293+0000","last_unstale":"2024-09-19T04:18:47.098039+0000","last_undegraded":"2024-09-19T04:18:47.098039+0000","last_fullsized":"2024-09-19T04:18:47.098039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:30.291475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727271+0000","last_change":"2024-09-19T04:18:46.112337+0000","last_active":"2024-09-19T04:18:47.727271+0000","last_peered":"2024-09-19T04:18:47.727271+0000","last_clean":"2024-09-19T04:18:47.727271+0000","last_became_active":"2024-09-19T04:18:46.111978+0000","last_became_peered":"2024-09-19T04:18:46.111978+0000","last_unstale":"2024-09-19T04:18:47.727271+0000","last_undegraded":"2024-09-19T04:18:47.727271+0000","last_fullsized":"2024-09-19T04:18:47.727271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:44.014241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101949+0000","last_change":"2024-09-19T04:18:46.089524+0000","last_active":"2024-09-19T04:18:47.101949+0000","last_peered":"2024-09-19T04:18:47.101949+0000","last_clean":"2024-09-19T04:18:47.101949+0000","last_became_active":"2024-09-19T04:18:46.088754+0000","last_became_peered":"2024-09-19T04:18:46.088754+0000","last_unstale":"2024-09-19T04:18:47.101949+0000","last_undegraded":"2024-09-19T04:18:47.101949+0000","last_fullsized":"2024-09-19T04:18:47.101949+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:45.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728622+0000","last_change":"2024-09-19T04:18:46.092499+0000","last_active":"2024-09-19T04:18:47.728622+0000","last_peered":"2024-09-19T04:18:47.728622+0000","last_clean":"2024-09-19T04:18:47.728622+0000","last_became_active":"2024-09-19T04:18:46.092122+0000","last_became_peered":"2024-09-19T04:18:46.092122+0000","last_unstale":"2024-09-19T04:18:47.728622+0000","last_undegraded":"2024-09-19T04:18:47.728622+0000","last_fullsized":"2024-09-19T04:18:47.728622+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:19.113397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725042+0000","last_change":"2024-09-19T04:18:46.102200+0000","last_active":"2024-09-19T04:18:47.725042+0000","last_peered":"2024-09-19T04:18:47.725042+0000","last_clean":"2024-09-19T04:18:47.725042+0000","last_became_active":"2024-09-19T04:18:46.101361+0000","last_became_peered":"2024-09-19T04:18:46.101361+0000","last_unstale":"2024-09-19T04:18:47.725042+0000","last_undegraded":"2024-09-19T04:18:47.725042+0000","last_fullsized":"2024-09-19T04:18:47.725042+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:20.844483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.730320+0000","last_change":"2024-09-19T04:18:46.092397+0000","last_active":"2024-09-19T04:18:47.730320+0000","last_peered":"2024-09-19T04:18:47.730320+0000","last_clean":"2024-09-19T04:18:47.730320+0000","last_became_active":"2024-09-19T04:18:46.092124+0000","last_became_peered":"2024-09-19T04:18:46.092124+0000","last_unstale":"2024-09-19T04:18:47.730320+0000","last_undegraded":"2024-09-19T04:18:47.730320+0000","last_fullsized":"2024-09-19T04:18:47.730320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:16.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102590+0000","last_change":"2024-09-19T04:18:46.081787+0000","last_active":"2024-09-19T04:18:47.102590+0000","last_peered":"2024-09-19T04:18:47.102590+0000","last_clean":"2024-09-19T04:18:47.102590+0000","last_became_active":"2024-09-19T04:18:46.081025+0000","last_became_peered":"2024-09-19T04:18:46.081025+0000","last_unstale":"2024-09-19T04:18:47.102590+0000","last_undegraded":"2024-09-19T04:18:47.102590+0000","last_fullsized":"2024-09-19T04:18:47.102590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:08.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727985+0000","last_change":"2024-09-19T04:18:46.104166+0000","last_active":"2024-09-19T04:18:47.727985+0000","last_peered":"2024-09-19T04:18:47.727985+0000","last_clean":"2024-09-19T04:18:47.727985+0000","last_became_active":"2024-09-19T04:18:46.103506+0000","last_became_peered":"2024-09-19T04:18:46.103506+0000","last_unstale":"2024-09-19T04:18:47.727985+0000","last_undegraded":"2024-09-19T04:18:47.727985+0000","last_fullsized":"2024-09-19T04:18:47.727985+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:51.573366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101959+0000","last_change":"2024-09-19T04:18:46.097343+0000","last_active":"2024-09-19T04:18:47.101959+0000","last_peered":"2024-09-19T04:18:47.101959+0000","last_clean":"2024-09-19T04:18:47.101959+0000","last_became_active":"2024-09-19T04:18:46.092130+0000","last_became_peered":"2024-09-19T04:18:46.092130+0000","last_unstale":"2024-09-19T04:18:47.101959+0000","last_undegraded":"2024-09-19T04:18:47.101959+0000","last_fullsized":"2024-09-19T04:18:47.101959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:06.256564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727380+0000","last_change":"2024-09-19T04:18:46.101082+0000","last_active":"2024-09-19T04:18:47.727380+0000","last_peered":"2024-09-19T04:18:47.727380+0000","last_clean":"2024-09-19T04:18:47.727380+0000","last_became_active":"2024-09-19T04:18:46.100683+0000","last_became_peered":"2024-09-19T04:18:46.100683+0000","last_unstale":"2024-09-19T04:18:47.727380+0000","last_undegraded":"2024-09-19T04:18:47.727380+0000","last_fullsized":"2024-09-19T04:18:47.727380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.000061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728479+0000","last_change":"2024-09-19T04:18:46.091532+0000","last_active":"2024-09-19T04:18:47.728479+0000","last_peered":"2024-09-19T04:18:47.728479+0000","last_clean":"2024-09-19T04:18:47.728479+0000","last_became_active":"2024-09-19T04:18:46.090826+0000","last_became_peered":"2024-09-19T04:18:46.090826+0000","last_unstale":"2024-09-19T04:18:47.728479+0000","last_undegraded":"2024-09-19T04:18:47.728479+0000","last_fullsized":"2024-09-19T04:18:47.728479+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:32.959237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099291+0000","last_change":"2024-09-19T04:18:46.105958+0000","last_active":"2024-09-19T04:18:47.099291+0000","last_peered":"2024-09-19T04:18:47.099291+0000","last_clean":"2024-09-19T04:18:47.099291+0000","last_became_active":"2024-09-19T04:18:46.100355+0000","last_became_peered":"2024-09-19T04:18:46.100355+0000","last_unstale":"2024-09-19T04:18:47.099291+0000","last_undegraded":"2024-09-19T04:18:47.099291+0000","last_fullsized":"2024-09-19T04:18:47.099291+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:44.014241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729078+0000","last_change":"2024-09-19T04:18:46.099766+0000","last_active":"2024-09-19T04:18:47.729078+0000","last_peered":"2024-09-19T04:18:47.729078+0000","last_clean":"2024-09-19T04:18:47.729078+0000","last_became_active":"2024-09-19T04:18:46.099457+0000","last_became_peered":"2024-09-19T04:18:46.099457+0000","last_unstale":"2024-09-19T04:18:47.729078+0000","last_undegraded":"2024-09-19T04:18:47.729078+0000","last_fullsized":"2024-09-19T04:18:47.729078+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:45.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727272+0000","last_change":"2024-09-19T04:18:46.103105+0000","last_active":"2024-09-19T04:18:47.727272+0000","last_peered":"2024-09-19T04:18:47.727272+0000","last_clean":"2024-09-19T04:18:47.727272+0000","last_became_active":"2024-09-19T04:18:46.102099+0000","last_became_peered":"2024-09-19T04:18:46.102099+0000","last_unstale":"2024-09-19T04:18:47.727272+0000","last_undegraded":"2024-09-19T04:18:47.727272+0000","last_fullsized":"2024-09-19T04:18:47.727272+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:47.549289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729035+0000","last_change":"2024-09-19T04:18:46.098621+0000","last_active":"2024-09-19T04:18:47.729035+0000","last_peered":"2024-09-19T04:18:47.729035+0000","last_clean":"2024-09-19T04:18:47.729035+0000","last_became_active":"2024-09-19T04:18:46.098105+0000","last_became_peered":"2024-09-19T04:18:46.098105+0000","last_unstale":"2024-09-19T04:18:47.729035+0000","last_undegraded":"2024-09-19T04:18:47.729035+0000","last_fullsized":"2024-09-19T04:18:47.729035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:47.769886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.097930+0000","last_change":"2024-09-19T04:18:46.098650+0000","last_active":"2024-09-19T04:18:47.097930+0000","last_peered":"2024-09-19T04:18:47.097930+0000","last_clean":"2024-09-19T04:18:47.097930+0000","last_became_active":"2024-09-19T04:18:46.098316+0000","last_became_peered":"2024-09-19T04:18:46.098316+0000","last_unstale":"2024-09-19T04:18:47.097930+0000","last_undegraded":"2024-09-19T04:18:47.097930+0000","last_fullsized":"2024-09-19T04:18:47.097930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.081156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102399+0000","last_change":"2024-09-19T04:18:46.111648+0000","last_active":"2024-09-19T04:18:47.102399+0000","last_peered":"2024-09-19T04:18:47.102399+0000","last_clean":"2024-09-19T04:18:47.102399+0000","last_became_active":"2024-09-19T04:18:46.108501+0000","last_became_peered":"2024-09-19T04:18:46.108501+0000","last_unstale":"2024-09-19T04:18:47.102399+0000","last_undegraded":"2024-09-19T04:18:47.102399+0000","last_fullsized":"2024-09-19T04:18:47.102399+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:27.775882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727681+0000","last_change":"2024-09-19T04:18:46.101315+0000","last_active":"2024-09-19T04:18:47.727681+0000","last_peered":"2024-09-19T04:18:47.727681+0000","last_clean":"2024-09-19T04:18:47.727681+0000","last_became_active":"2024-09-19T04:18:46.100922+0000","last_became_peered":"2024-09-19T04:18:46.100922+0000","last_unstale":"2024-09-19T04:18:47.727681+0000","last_undegraded":"2024-09-19T04:18:47.727681+0000","last_fullsized":"2024-09-19T04:18:47.727681+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:16.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101232+0000","last_change":"2024-09-19T04:18:46.090088+0000","last_active":"2024-09-19T04:18:47.101232+0000","last_peered":"2024-09-19T04:18:47.101232+0000","last_clean":"2024-09-19T04:18:47.101232+0000","last_became_active":"2024-09-19T04:18:46.089305+0000","last_became_peered":"2024-09-19T04:18:46.089305+0000","last_unstale":"2024-09-19T04:18:47.101232+0000","last_undegraded":"2024-09-19T04:18:47.101232+0000","last_fullsized":"2024-09-19T04:18:47.101232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:32.959237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099662+0000","last_change":"2024-09-19T04:18:46.096177+0000","last_active":"2024-09-19T04:18:47.099662+0000","last_peered":"2024-09-19T04:18:47.099662+0000","last_clean":"2024-09-19T04:18:47.099662+0000","last_became_active":"2024-09-19T04:18:46.092091+0000","last_became_peered":"2024-09-19T04:18:46.092091+0000","last_unstale":"2024-09-19T04:18:47.099662+0000","last_undegraded":"2024-09-19T04:18:47.099662+0000","last_fullsized":"2024-09-19T04:18:47.099662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:16.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725426+0000","last_change":"2024-09-19T04:18:46.099423+0000","last_active":"2024-09-19T04:18:47.725426+0000","last_peered":"2024-09-19T04:18:47.725426+0000","last_clean":"2024-09-19T04:18:47.725426+0000","last_became_active":"2024-09-19T04:18:46.098728+0000","last_became_peered":"2024-09-19T04:18:46.098728+0000","last_unstale":"2024-09-19T04:18:47.725426+0000","last_undegraded":"2024-09-19T04:18:47.725426+0000","last_fullsized":"2024-09-19T04:18:47.725426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:08.793346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.097986+0000","last_change":"2024-09-19T04:18:46.098484+0000","last_active":"2024-09-19T04:18:47.097986+0000","last_peered":"2024-09-19T04:18:47.097986+0000","last_clean":"2024-09-19T04:18:47.097986+0000","last_became_active":"2024-09-19T04:18:46.098244+0000","last_became_peered":"2024-09-19T04:18:46.098244+0000","last_unstale":"2024-09-19T04:18:47.097986+0000","last_undegraded":"2024-09-19T04:18:47.097986+0000","last_fullsized":"2024-09-19T04:18:47.097986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:29.608036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100462+0000","last_change":"2024-09-19T04:18:46.108052+0000","last_active":"2024-09-19T04:18:47.100462+0000","last_peered":"2024-09-19T04:18:47.100462+0000","last_clean":"2024-09-19T04:18:47.100462+0000","last_became_active":"2024-09-19T04:18:46.107296+0000","last_became_peered":"2024-09-19T04:18:46.107296+0000","last_unstale":"2024-09-19T04:18:47.100462+0000","last_undegraded":"2024-09-19T04:18:47.100462+0000","last_fullsized":"2024-09-19T04:18:47.100462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:28.803233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101636+0000","last_change":"2024-09-19T04:18:46.109379+0000","last_active":"2024-09-19T04:18:47.101636+0000","last_peered":"2024-09-19T04:18:47.101636+0000","last_clean":"2024-09-19T04:18:47.101636+0000","last_became_active":"2024-09-19T04:18:46.106745+0000","last_became_peered":"2024-09-19T04:18:46.106745+0000","last_unstale":"2024-09-19T04:18:47.101636+0000","last_undegraded":"2024-09-19T04:18:47.101636+0000","last_fullsized":"2024-09-19T04:18:47.101636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:08.793346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726882+0000","last_change":"2024-09-19T04:18:46.111471+0000","last_active":"2024-09-19T04:18:47.726882+0000","last_peered":"2024-09-19T04:18:47.726882+0000","last_clean":"2024-09-19T04:18:47.726882+0000","last_became_active":"2024-09-19T04:18:46.111090+0000","last_became_peered":"2024-09-19T04:18:46.111090+0000","last_unstale":"2024-09-19T04:18:47.726882+0000","last_undegraded":"2024-09-19T04:18:47.726882+0000","last_fullsized":"2024-09-19T04:18:47.726882+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:17.187828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728069+0000","last_change":"2024-09-19T04:18:46.099777+0000","last_active":"2024-09-19T04:18:47.728069+0000","last_peered":"2024-09-19T04:18:47.728069+0000","last_clean":"2024-09-19T04:18:47.728069+0000","last_became_active":"2024-09-19T04:18:46.099077+0000","last_became_peered":"2024-09-19T04:18:46.099077+0000","last_unstale":"2024-09-19T04:18:47.728069+0000","last_undegraded":"2024-09-19T04:18:47.728069+0000","last_fullsized":"2024-09-19T04:18:47.728069+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:02.640637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726265+0000","last_change":"2024-09-19T04:18:46.099467+0000","last_active":"2024-09-19T04:18:47.726265+0000","last_peered":"2024-09-19T04:18:47.726265+0000","last_clean":"2024-09-19T04:18:47.726265+0000","last_became_active":"2024-09-19T04:18:46.098753+0000","last_became_peered":"2024-09-19T04:18:46.098753+0000","last_unstale":"2024-09-19T04:18:47.726265+0000","last_undegraded":"2024-09-19T04:18:47.726265+0000","last_fullsized":"2024-09-19T04:18:47.726265+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:37.984225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100273+0000","last_change":"2024-09-19T04:18:46.082066+0000","last_active":"2024-09-19T04:18:47.100273+0000","last_peered":"2024-09-19T04:18:47.100273+0000","last_clean":"2024-09-19T04:18:47.100273+0000","last_became_active":"2024-09-19T04:18:46.078672+0000","last_became_peered":"2024-09-19T04:18:46.078672+0000","last_unstale":"2024-09-19T04:18:47.100273+0000","last_undegraded":"2024-09-19T04:18:47.100273+0000","last_fullsized":"2024-09-19T04:18:47.100273+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:02.640637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099317+0000","last_change":"2024-09-19T04:18:46.104497+0000","last_active":"2024-09-19T04:18:47.099317+0000","last_peered":"2024-09-19T04:18:47.099317+0000","last_clean":"2024-09-19T04:18:47.099317+0000","last_became_active":"2024-09-19T04:18:46.102334+0000","last_became_peered":"2024-09-19T04:18:46.102334+0000","last_unstale":"2024-09-19T04:18:47.099317+0000","last_undegraded":"2024-09-19T04:18:47.099317+0000","last_fullsized":"2024-09-19T04:18:47.099317+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:04.293681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729696+0000","last_change":"2024-09-19T04:18:46.102254+0000","last_active":"2024-09-19T04:18:47.729696+0000","last_peered":"2024-09-19T04:18:47.729696+0000","last_clean":"2024-09-19T04:18:47.729696+0000","last_became_active":"2024-09-19T04:18:46.101967+0000","last_became_peered":"2024-09-19T04:18:46.101967+0000","last_unstale":"2024-09-19T04:18:47.729696+0000","last_undegraded":"2024-09-19T04:18:47.729696+0000","last_fullsized":"2024-09-19T04:18:47.729696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:48.534500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727993+0000","last_change":"2024-09-19T04:18:46.092823+0000","last_active":"2024-09-19T04:18:47.727993+0000","last_peered":"2024-09-19T04:18:47.727993+0000","last_clean":"2024-09-19T04:18:47.727993+0000","last_became_active":"2024-09-19T04:18:46.092231+0000","last_became_peered":"2024-09-19T04:18:46.092231+0000","last_unstale":"2024-09-19T04:18:47.727993+0000","last_undegraded":"2024-09-19T04:18:47.727993+0000","last_fullsized":"2024-09-19T04:18:47.727993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:35.328519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726057+0000","last_change":"2024-09-19T04:18:45.061693+0000","last_active":"2024-09-19T04:18:47.726057+0000","last_peered":"2024-09-19T04:18:47.726057+0000","last_clean":"2024-09-19T04:18:47.726057+0000","last_became_active":"2024-09-19T04:18:45.060982+0000","last_became_peered":"2024-09-19T04:18:45.060982+0000","last_unstale":"2024-09-19T04:18:47.726057+0000","last_undegraded":"2024-09-19T04:18:47.726057+0000","last_fullsized":"2024-09-19T04:18:47.726057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:16.115150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728297+0000","last_change":"2024-09-19T04:18:46.096754+0000","last_active":"2024-09-19T04:18:47.728297+0000","last_peered":"2024-09-19T04:18:47.728297+0000","last_clean":"2024-09-19T04:18:47.728297+0000","last_became_active":"2024-09-19T04:18:46.096455+0000","last_became_peered":"2024-09-19T04:18:46.096455+0000","last_unstale":"2024-09-19T04:18:47.728297+0000","last_undegraded":"2024-09-19T04:18:47.728297+0000","last_fullsized":"2024-09-19T04:18:47.728297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:01.943423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101962+0000","last_change":"2024-09-19T04:18:45.065663+0000","last_active":"2024-09-19T04:18:47.101962+0000","last_peered":"2024-09-19T04:18:47.101962+0000","last_clean":"2024-09-19T04:18:47.101962+0000","last_became_active":"2024-09-19T04:18:45.063514+0000","last_became_peered":"2024-09-19T04:18:45.063514+0000","last_unstale":"2024-09-19T04:18:47.101962+0000","last_undegraded":"2024-09-19T04:18:47.101962+0000","last_fullsized":"2024-09-19T04:18:47.101962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:47.967223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726613+0000","last_change":"2024-09-19T04:18:46.099715+0000","last_active":"2024-09-19T04:18:47.726613+0000","last_peered":"2024-09-19T04:18:47.726613+0000","last_clean":"2024-09-19T04:18:47.726613+0000","last_became_active":"2024-09-19T04:18:46.098947+0000","last_became_peered":"2024-09-19T04:18:46.098947+0000","last_unstale":"2024-09-19T04:18:47.726613+0000","last_undegraded":"2024-09-19T04:18:47.726613+0000","last_fullsized":"2024-09-19T04:18:47.726613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:25.791085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726892+0000","last_change":"2024-09-19T04:18:45.064570+0000","last_active":"2024-09-19T04:18:47.726892+0000","last_peered":"2024-09-19T04:18:47.726892+0000","last_clean":"2024-09-19T04:18:47.726892+0000","last_became_active":"2024-09-19T04:18:45.063857+0000","last_became_peered":"2024-09-19T04:18:45.063857+0000","last_unstale":"2024-09-19T04:18:47.726892+0000","last_undegraded":"2024-09-19T04:18:47.726892+0000","last_fullsized":"2024-09-19T04:18:47.726892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:56.475914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729351+0000","last_change":"2024-09-19T04:18:46.107583+0000","last_active":"2024-09-19T04:18:47.729351+0000","last_peered":"2024-09-19T04:18:47.729351+0000","last_clean":"2024-09-19T04:18:47.729351+0000","last_became_active":"2024-09-19T04:18:46.107272+0000","last_became_peered":"2024-09-19T04:18:46.107272+0000","last_unstale":"2024-09-19T04:18:47.729351+0000","last_undegraded":"2024-09-19T04:18:47.729351+0000","last_fullsized":"2024-09-19T04:18:47.729351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:59.412298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725815+0000","last_change":"2024-09-19T04:18:45.067978+0000","last_active":"2024-09-19T04:18:47.725815+0000","last_peered":"2024-09-19T04:18:47.725815+0000","last_clean":"2024-09-19T04:18:47.725815+0000","last_became_active":"2024-09-19T04:18:45.060466+0000","last_became_peered":"2024-09-19T04:18:45.060466+0000","last_unstale":"2024-09-19T04:18:47.725815+0000","last_undegraded":"2024-09-19T04:18:47.725815+0000","last_fullsized":"2024-09-19T04:18:47.725815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.243336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727454+0000","last_change":"2024-09-19T04:18:46.099897+0000","last_active":"2024-09-19T04:18:47.727454+0000","last_peered":"2024-09-19T04:18:47.727454+0000","last_clean":"2024-09-19T04:18:47.727454+0000","last_became_active":"2024-09-19T04:18:46.099619+0000","last_became_peered":"2024-09-19T04:18:46.099619+0000","last_unstale":"2024-09-19T04:18:47.727454+0000","last_undegraded":"2024-09-19T04:18:47.727454+0000","last_fullsized":"2024-09-19T04:18:47.727454+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:16.434612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101756+0000","last_change":"2024-09-19T04:18:45.056632+0000","last_active":"2024-09-19T04:18:47.101756+0000","last_peered":"2024-09-19T04:18:47.101756+0000","last_clean":"2024-09-19T04:18:47.101756+0000","last_became_active":"2024-09-19T04:18:45.055905+0000","last_became_peered":"2024-09-19T04:18:45.055905+0000","last_unstale":"2024-09-19T04:18:47.101756+0000","last_undegraded":"2024-09-19T04:18:47.101756+0000","last_fullsized":"2024-09-19T04:18:47.101756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:57.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725908+0000","last_change":"2024-09-19T04:18:46.095507+0000","last_active":"2024-09-19T04:18:47.725908+0000","last_peered":"2024-09-19T04:18:47.725908+0000","last_clean":"2024-09-19T04:18:47.725908+0000","last_became_active":"2024-09-19T04:18:46.094690+0000","last_became_peered":"2024-09-19T04:18:46.094690+0000","last_unstale":"2024-09-19T04:18:47.725908+0000","last_undegraded":"2024-09-19T04:18:47.725908+0000","last_fullsized":"2024-09-19T04:18:47.725908+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:45.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.730444+0000","last_change":"2024-09-19T04:18:45.055425+0000","last_active":"2024-09-19T04:18:47.730444+0000","last_peered":"2024-09-19T04:18:47.730444+0000","last_clean":"2024-09-19T04:18:47.730444+0000","last_became_active":"2024-09-19T04:18:45.055023+0000","last_became_peered":"2024-09-19T04:18:45.055023+0000","last_unstale":"2024-09-19T04:18:47.730444+0000","last_undegraded":"2024-09-19T04:18:47.730444+0000","last_fullsized":"2024-09-19T04:18:47.730444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:11.948545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728984+0000","last_change":"2024-09-19T04:18:46.092574+0000","last_active":"2024-09-19T04:18:47.728984+0000","last_peered":"2024-09-19T04:18:47.728984+0000","last_clean":"2024-09-19T04:18:47.728984+0000","last_became_active":"2024-09-19T04:18:46.091536+0000","last_became_peered":"2024-09-19T04:18:46.091536+0000","last_unstale":"2024-09-19T04:18:47.728984+0000","last_undegraded":"2024-09-19T04:18:47.728984+0000","last_fullsized":"2024-09-19T04:18:47.728984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:04.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.097868+0000","last_change":"2024-09-19T04:18:45.057250+0000","last_active":"2024-09-19T04:18:47.097868+0000","last_peered":"2024-09-19T04:18:47.097868+0000","last_clean":"2024-09-19T04:18:47.097868+0000","last_became_active":"2024-09-19T04:18:45.056733+0000","last_became_peered":"2024-09-19T04:18:45.056733+0000","last_unstale":"2024-09-19T04:18:47.097868+0000","last_undegraded":"2024-09-19T04:18:47.097868+0000","last_fullsized":"2024-09-19T04:18:47.097868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:27.791923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728849+0000","last_change":"2024-09-19T04:18:46.107825+0000","last_active":"2024-09-19T04:18:47.728849+0000","last_peered":"2024-09-19T04:18:47.728849+0000","last_clean":"2024-09-19T04:18:47.728849+0000","last_became_active":"2024-09-19T04:18:46.107559+0000","last_became_peered":"2024-09-19T04:18:46.107559+0000","last_unstale":"2024-09-19T04:18:47.728849+0000","last_undegraded":"2024-09-19T04:18:47.728849+0000","last_fullsized":"2024-09-19T04:18:47.728849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:08.793346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.724940+0000","last_change":"2024-09-19T04:18:45.068393+0000","last_active":"2024-09-19T04:18:47.724940+0000","last_peered":"2024-09-19T04:18:47.724940+0000","last_clean":"2024-09-19T04:18:47.724940+0000","last_became_active":"2024-09-19T04:18:45.066713+0000","last_became_peered":"2024-09-19T04:18:45.066713+0000","last_unstale":"2024-09-19T04:18:47.724940+0000","last_undegraded":"2024-09-19T04:18:47.724940+0000","last_fullsized":"2024-09-19T04:18:47.724940+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:57.653022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101971+0000","last_change":"2024-09-19T04:18:46.109048+0000","last_active":"2024-09-19T04:18:47.101971+0000","last_peered":"2024-09-19T04:18:47.101971+0000","last_clean":"2024-09-19T04:18:47.101971+0000","last_became_active":"2024-09-19T04:18:46.107899+0000","last_became_peered":"2024-09-19T04:18:46.107899+0000","last_unstale":"2024-09-19T04:18:47.101971+0000","last_undegraded":"2024-09-19T04:18:47.101971+0000","last_fullsized":"2024-09-19T04:18:47.101971+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:16.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100779+0000","last_change":"2024-09-19T04:18:45.044817+0000","last_active":"2024-09-19T04:18:47.100779+0000","last_peered":"2024-09-19T04:18:47.100779+0000","last_clean":"2024-09-19T04:18:47.100779+0000","last_became_active":"2024-09-19T04:18:45.043707+0000","last_became_peered":"2024-09-19T04:18:45.043707+0000","last_unstale":"2024-09-19T04:18:47.100779+0000","last_undegraded":"2024-09-19T04:18:47.100779+0000","last_fullsized":"2024-09-19T04:18:47.100779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:44.518485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726745+0000","last_change":"2024-09-19T04:18:46.112976+0000","last_active":"2024-09-19T04:18:47.726745+0000","last_peered":"2024-09-19T04:18:47.726745+0000","last_clean":"2024-09-19T04:18:47.726745+0000","last_became_active":"2024-09-19T04:18:46.112238+0000","last_became_peered":"2024-09-19T04:18:46.112238+0000","last_unstale":"2024-09-19T04:18:47.726745+0000","last_undegraded":"2024-09-19T04:18:47.726745+0000","last_fullsized":"2024-09-19T04:18:47.726745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:19.159205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098068+0000","last_change":"2024-09-19T04:18:45.053010+0000","last_active":"2024-09-19T04:18:47.098068+0000","last_peered":"2024-09-19T04:18:47.098068+0000","last_clean":"2024-09-19T04:18:47.098068+0000","last_became_active":"2024-09-19T04:18:45.052558+0000","last_became_peered":"2024-09-19T04:18:45.052558+0000","last_unstale":"2024-09-19T04:18:47.098068+0000","last_undegraded":"2024-09-19T04:18:47.098068+0000","last_fullsized":"2024-09-19T04:18:47.098068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:55.151611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098222+0000","last_change":"2024-09-19T04:18:46.099306+0000","last_active":"2024-09-19T04:18:47.098222+0000","last_peered":"2024-09-19T04:18:47.098222+0000","last_clean":"2024-09-19T04:18:47.098222+0000","last_became_active":"2024-09-19T04:18:46.098946+0000","last_became_peered":"2024-09-19T04:18:46.098946+0000","last_unstale":"2024-09-19T04:18:47.098222+0000","last_undegraded":"2024-09-19T04:18:47.098222+0000","last_fullsized":"2024-09-19T04:18:47.098222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:01.283235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102401+0000","last_change":"2024-09-19T04:18:45.071213+0000","last_active":"2024-09-19T04:18:47.102401+0000","last_peered":"2024-09-19T04:18:47.102401+0000","last_clean":"2024-09-19T04:18:47.102401+0000","last_became_active":"2024-09-19T04:18:45.070552+0000","last_became_peered":"2024-09-19T04:18:45.070552+0000","last_unstale":"2024-09-19T04:18:47.102401+0000","last_undegraded":"2024-09-19T04:18:47.102401+0000","last_fullsized":"2024-09-19T04:18:47.102401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:16.564686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728976+0000","last_change":"2024-09-19T04:18:46.092781+0000","last_active":"2024-09-19T04:18:47.728976+0000","last_peered":"2024-09-19T04:18:47.728976+0000","last_clean":"2024-09-19T04:18:47.728976+0000","last_became_active":"2024-09-19T04:18:46.090172+0000","last_became_peered":"2024-09-19T04:18:46.090172+0000","last_unstale":"2024-09-19T04:18:47.728976+0000","last_undegraded":"2024-09-19T04:18:47.728976+0000","last_fullsized":"2024-09-19T04:18:47.728976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:21.590821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728277+0000","last_change":"2024-09-19T04:18:45.054695+0000","last_active":"2024-09-19T04:18:47.728277+0000","last_peered":"2024-09-19T04:18:47.728277+0000","last_clean":"2024-09-19T04:18:47.728277+0000","last_became_active":"2024-09-19T04:18:45.054264+0000","last_became_peered":"2024-09-19T04:18:45.054264+0000","last_unstale":"2024-09-19T04:18:47.728277+0000","last_undegraded":"2024-09-19T04:18:47.728277+0000","last_fullsized":"2024-09-19T04:18:47.728277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:17.069845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729837+0000","last_change":"2024-09-19T04:18:46.092871+0000","last_active":"2024-09-19T04:18:47.729837+0000","last_peered":"2024-09-19T04:18:47.729837+0000","last_clean":"2024-09-19T04:18:47.729837+0000","last_became_active":"2024-09-19T04:18:46.092404+0000","last_became_peered":"2024-09-19T04:18:46.092404+0000","last_unstale":"2024-09-19T04:18:47.729837+0000","last_undegraded":"2024-09-19T04:18:47.729837+0000","last_fullsized":"2024-09-19T04:18:47.729837+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:06.256564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726249+0000","last_change":"2024-09-19T04:18:45.062957+0000","last_active":"2024-09-19T04:18:47.726249+0000","last_peered":"2024-09-19T04:18:47.726249+0000","last_clean":"2024-09-19T04:18:47.726249+0000","last_became_active":"2024-09-19T04:18:45.062159+0000","last_became_peered":"2024-09-19T04:18:45.062159+0000","last_unstale":"2024-09-19T04:18:47.726249+0000","last_undegraded":"2024-09-19T04:18:47.726249+0000","last_fullsized":"2024-09-19T04:18:47.726249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:22.991055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099293+0000","last_change":"2024-09-19T04:18:46.090069+0000","last_active":"2024-09-19T04:18:47.099293+0000","last_peered":"2024-09-19T04:18:47.099293+0000","last_clean":"2024-09-19T04:18:47.099293+0000","last_became_active":"2024-09-19T04:18:46.089282+0000","last_became_peered":"2024-09-19T04:18:46.089282+0000","last_unstale":"2024-09-19T04:18:47.099293+0000","last_undegraded":"2024-09-19T04:18:47.099293+0000","last_fullsized":"2024-09-19T04:18:47.099293+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:16.434612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729222+0000","last_change":"2024-09-19T04:18:45.068440+0000","last_active":"2024-09-19T04:18:47.729222+0000","last_peered":"2024-09-19T04:18:47.729222+0000","last_clean":"2024-09-19T04:18:47.729222+0000","last_became_active":"2024-09-19T04:18:45.068087+0000","last_became_peered":"2024-09-19T04:18:45.068087+0000","last_unstale":"2024-09-19T04:18:47.729222+0000","last_undegraded":"2024-09-19T04:18:47.729222+0000","last_fullsized":"2024-09-19T04:18:47.729222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:58.883567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726955+0000","last_change":"2024-09-19T04:18:46.103807+0000","last_active":"2024-09-19T04:18:47.726955+0000","last_peered":"2024-09-19T04:18:47.726955+0000","last_clean":"2024-09-19T04:18:47.726955+0000","last_became_active":"2024-09-19T04:18:46.103512+0000","last_became_peered":"2024-09-19T04:18:46.103512+0000","last_unstale":"2024-09-19T04:18:47.726955+0000","last_undegraded":"2024-09-19T04:18:47.726955+0000","last_fullsized":"2024-09-19T04:18:47.726955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:00.526824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729526+0000","last_change":"2024-09-19T04:18:45.066053+0000","last_active":"2024-09-19T04:18:47.729526+0000","last_peered":"2024-09-19T04:18:47.729526+0000","last_clean":"2024-09-19T04:18:47.729526+0000","last_became_active":"2024-09-19T04:18:45.065679+0000","last_became_peered":"2024-09-19T04:18:45.065679+0000","last_unstale":"2024-09-19T04:18:47.729526+0000","last_undegraded":"2024-09-19T04:18:47.729526+0000","last_fullsized":"2024-09-19T04:18:47.729526+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:24.757521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727169+0000","last_change":"2024-09-19T04:18:46.102100+0000","last_active":"2024-09-19T04:18:47.727169+0000","last_peered":"2024-09-19T04:18:47.727169+0000","last_clean":"2024-09-19T04:18:47.727169+0000","last_became_active":"2024-09-19T04:18:46.101819+0000","last_became_peered":"2024-09-19T04:18:46.101819+0000","last_unstale":"2024-09-19T04:18:47.727169+0000","last_undegraded":"2024-09-19T04:18:47.727169+0000","last_fullsized":"2024-09-19T04:18:47.727169+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:07.050105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.730122+0000","last_change":"2024-09-19T04:18:45.055284+0000","last_active":"2024-09-19T04:18:47.730122+0000","last_peered":"2024-09-19T04:18:47.730122+0000","last_clean":"2024-09-19T04:18:47.730122+0000","last_became_active":"2024-09-19T04:18:45.054898+0000","last_became_peered":"2024-09-19T04:18:45.054898+0000","last_unstale":"2024-09-19T04:18:47.730122+0000","last_undegraded":"2024-09-19T04:18:47.730122+0000","last_fullsized":"2024-09-19T04:18:47.730122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:24.060136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728340+0000","last_change":"2024-09-19T04:18:46.092514+0000","last_active":"2024-09-19T04:18:47.728340+0000","last_peered":"2024-09-19T04:18:47.728340+0000","last_clean":"2024-09-19T04:18:47.728340+0000","last_became_active":"2024-09-19T04:18:46.091581+0000","last_became_peered":"2024-09-19T04:18:46.091581+0000","last_unstale":"2024-09-19T04:18:47.728340+0000","last_undegraded":"2024-09-19T04:18:47.728340+0000","last_fullsized":"2024-09-19T04:18:47.728340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:54.041420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726332+0000","last_change":"2024-09-19T04:18:45.068182+0000","last_active":"2024-09-19T04:18:47.726332+0000","last_peered":"2024-09-19T04:18:47.726332+0000","last_clean":"2024-09-19T04:18:47.726332+0000","last_became_active":"2024-09-19T04:18:45.067149+0000","last_became_peered":"2024-09-19T04:18:45.067149+0000","last_unstale":"2024-09-19T04:18:47.726332+0000","last_undegraded":"2024-09-19T04:18:47.726332+0000","last_fullsized":"2024-09-19T04:18:47.726332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:12.559217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725929+0000","last_change":"2024-09-19T04:18:46.084203+0000","last_active":"2024-09-19T04:18:47.725929+0000","last_peered":"2024-09-19T04:18:47.725929+0000","last_clean":"2024-09-19T04:18:47.725929+0000","last_became_active":"2024-09-19T04:18:46.083457+0000","last_became_peered":"2024-09-19T04:18:46.083457+0000","last_unstale":"2024-09-19T04:18:47.725929+0000","last_undegraded":"2024-09-19T04:18:47.725929+0000","last_fullsized":"2024-09-19T04:18:47.725929+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:06.256564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726631+0000","last_change":"2024-09-19T04:18:45.070055+0000","last_active":"2024-09-19T04:18:47.726631+0000","last_peered":"2024-09-19T04:18:47.726631+0000","last_clean":"2024-09-19T04:18:47.726631+0000","last_became_active":"2024-09-19T04:18:45.069313+0000","last_became_peered":"2024-09-19T04:18:45.069313+0000","last_unstale":"2024-09-19T04:18:47.726631+0000","last_undegraded":"2024-09-19T04:18:47.726631+0000","last_fullsized":"2024-09-19T04:18:47.726631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:17:04.342215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101891+0000","last_change":"2024-09-19T04:18:46.108730+0000","last_active":"2024-09-19T04:18:47.101891+0000","last_peered":"2024-09-19T04:18:47.101891+0000","last_clean":"2024-09-19T04:18:47.101891+0000","last_became_active":"2024-09-19T04:18:46.100297+0000","last_became_peered":"2024-09-19T04:18:46.100297+0000","last_unstale":"2024-09-19T04:18:47.101891+0000","last_undegraded":"2024-09-19T04:18:47.101891+0000","last_fullsized":"2024-09-19T04:18:47.101891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:21.590821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102612+0000","last_change":"2024-09-19T04:18:45.055509+0000","last_active":"2024-09-19T04:18:47.102612+0000","last_peered":"2024-09-19T04:18:47.102612+0000","last_clean":"2024-09-19T04:18:47.102612+0000","last_became_active":"2024-09-19T04:18:45.054662+0000","last_became_peered":"2024-09-19T04:18:45.054662+0000","last_unstale":"2024-09-19T04:18:47.102612+0000","last_undegraded":"2024-09-19T04:18:47.102612+0000","last_fullsized":"2024-09-19T04:18:47.102612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:06:43.972021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727701+0000","last_change":"2024-09-19T04:18:46.093493+0000","last_active":"2024-09-19T04:18:47.727701+0000","last_peered":"2024-09-19T04:18:47.727701+0000","last_clean":"2024-09-19T04:18:47.727701+0000","last_became_active":"2024-09-19T04:18:46.093197+0000","last_became_peered":"2024-09-19T04:18:46.093197+0000","last_unstale":"2024-09-19T04:18:47.727701+0000","last_undegraded":"2024-09-19T04:18:47.727701+0000","last_fullsized":"2024-09-19T04:18:47.727701+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:49.190050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099375+0000","last_change":"2024-09-19T04:18:45.055060+0000","last_active":"2024-09-19T04:18:47.099375+0000","last_peered":"2024-09-19T04:18:47.099375+0000","last_clean":"2024-09-19T04:18:47.099375+0000","last_became_active":"2024-09-19T04:18:45.052574+0000","last_became_peered":"2024-09-19T04:18:45.052574+0000","last_unstale":"2024-09-19T04:18:47.099375+0000","last_undegraded":"2024-09-19T04:18:47.099375+0000","last_fullsized":"2024-09-19T04:18:47.099375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:20.579511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099390+0000","last_change":"2024-09-19T04:18:46.103856+0000","last_active":"2024-09-19T04:18:47.099390+0000","last_peered":"2024-09-19T04:18:47.099390+0000","last_clean":"2024-09-19T04:18:47.099390+0000","last_became_active":"2024-09-19T04:18:46.101379+0000","last_became_peered":"2024-09-19T04:18:46.101379+0000","last_unstale":"2024-09-19T04:18:47.099390+0000","last_undegraded":"2024-09-19T04:18:47.099390+0000","last_fullsized":"2024-09-19T04:18:47.099390+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:48.052777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727202+0000","last_change":"2024-09-19T04:18:45.049798+0000","last_active":"2024-09-19T04:18:47.727202+0000","last_peered":"2024-09-19T04:18:47.727202+0000","last_clean":"2024-09-19T04:18:47.727202+0000","last_became_active":"2024-09-19T04:18:45.049428+0000","last_became_peered":"2024-09-19T04:18:45.049428+0000","last_unstale":"2024-09-19T04:18:47.727202+0000","last_undegraded":"2024-09-19T04:18:47.727202+0000","last_fullsized":"2024-09-19T04:18:47.727202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:55.241517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099922+0000","last_change":"2024-09-19T04:18:46.103821+0000","last_active":"2024-09-19T04:18:47.099922+0000","last_peered":"2024-09-19T04:18:47.099922+0000","last_clean":"2024-09-19T04:18:47.099922+0000","last_became_active":"2024-09-19T04:18:46.101231+0000","last_became_peered":"2024-09-19T04:18:46.101231+0000","last_unstale":"2024-09-19T04:18:47.099922+0000","last_undegraded":"2024-09-19T04:18:47.099922+0000","last_fullsized":"2024-09-19T04:18:47.099922+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:57.771748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099624+0000","last_change":"2024-09-19T04:18:45.052821+0000","last_active":"2024-09-19T04:18:47.099624+0000","last_peered":"2024-09-19T04:18:47.099624+0000","last_clean":"2024-09-19T04:18:47.099624+0000","last_became_active":"2024-09-19T04:18:45.052111+0000","last_became_peered":"2024-09-19T04:18:45.052111+0000","last_unstale":"2024-09-19T04:18:47.099624+0000","last_undegraded":"2024-09-19T04:18:47.099624+0000","last_fullsized":"2024-09-19T04:18:47.099624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:05.245253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727538+0000","last_change":"2024-09-19T04:18:46.097301+0000","last_active":"2024-09-19T04:18:47.727538+0000","last_peered":"2024-09-19T04:18:47.727538+0000","last_clean":"2024-09-19T04:18:47.727538+0000","last_became_active":"2024-09-19T04:18:46.096923+0000","last_became_peered":"2024-09-19T04:18:46.096923+0000","last_unstale":"2024-09-19T04:18:47.727538+0000","last_undegraded":"2024-09-19T04:18:47.727538+0000","last_fullsized":"2024-09-19T04:18:47.727538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:32.294118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726581+0000","last_change":"2024-09-19T04:18:45.048359+0000","last_active":"2024-09-19T04:18:47.726581+0000","last_peered":"2024-09-19T04:18:47.726581+0000","last_clean":"2024-09-19T04:18:47.726581+0000","last_became_active":"2024-09-19T04:18:45.047972+0000","last_became_peered":"2024-09-19T04:18:45.047972+0000","last_unstale":"2024-09-19T04:18:47.726581+0000","last_undegraded":"2024-09-19T04:18:47.726581+0000","last_fullsized":"2024-09-19T04:18:47.726581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:37.891701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725528+0000","last_change":"2024-09-19T04:18:46.095443+0000","last_active":"2024-09-19T04:18:47.725528+0000","last_peered":"2024-09-19T04:18:47.725528+0000","last_clean":"2024-09-19T04:18:47.725528+0000","last_became_active":"2024-09-19T04:18:46.086131+0000","last_became_peered":"2024-09-19T04:18:46.086131+0000","last_unstale":"2024-09-19T04:18:47.725528+0000","last_undegraded":"2024-09-19T04:18:47.725528+0000","last_fullsized":"2024-09-19T04:18:47.725528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:21.590821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726815+0000","last_change":"2024-09-19T04:18:45.066540+0000","last_active":"2024-09-19T04:18:47.726815+0000","last_peered":"2024-09-19T04:18:47.726815+0000","last_clean":"2024-09-19T04:18:47.726815+0000","last_became_active":"2024-09-19T04:18:45.065648+0000","last_became_peered":"2024-09-19T04:18:45.065648+0000","last_unstale":"2024-09-19T04:18:47.726815+0000","last_undegraded":"2024-09-19T04:18:47.726815+0000","last_fullsized":"2024-09-19T04:18:47.726815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:57.172726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729417+0000","last_change":"2024-09-19T04:18:46.092324+0000","last_active":"2024-09-19T04:18:47.729417+0000","last_peered":"2024-09-19T04:18:47.729417+0000","last_clean":"2024-09-19T04:18:47.729417+0000","last_became_active":"2024-09-19T04:18:46.091943+0000","last_became_peered":"2024-09-19T04:18:46.091943+0000","last_unstale":"2024-09-19T04:18:47.729417+0000","last_undegraded":"2024-09-19T04:18:47.729417+0000","last_fullsized":"2024-09-19T04:18:47.729417+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:51.883485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102037+0000","last_change":"2024-09-19T04:18:45.047755+0000","last_active":"2024-09-19T04:18:47.102037+0000","last_peered":"2024-09-19T04:18:47.102037+0000","last_clean":"2024-09-19T04:18:47.102037+0000","last_became_active":"2024-09-19T04:18:45.046948+0000","last_became_peered":"2024-09-19T04:18:45.046948+0000","last_unstale":"2024-09-19T04:18:47.102037+0000","last_undegraded":"2024-09-19T04:18:47.102037+0000","last_fullsized":"2024-09-19T04:18:47.102037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:24.060136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728731+0000","last_change":"2024-09-19T04:18:46.097027+0000","last_active":"2024-09-19T04:18:47.728731+0000","last_peered":"2024-09-19T04:18:47.728731+0000","last_clean":"2024-09-19T04:18:47.728731+0000","last_became_active":"2024-09-19T04:18:46.096656+0000","last_became_peered":"2024-09-19T04:18:46.096656+0000","last_unstale":"2024-09-19T04:18:47.728731+0000","last_undegraded":"2024-09-19T04:18:47.728731+0000","last_fullsized":"2024-09-19T04:18:47.728731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:13.087989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102660+0000","last_change":"2024-09-19T04:18:45.066608+0000","last_active":"2024-09-19T04:18:47.102660+0000","last_peered":"2024-09-19T04:18:47.102660+0000","last_clean":"2024-09-19T04:18:47.102660+0000","last_became_active":"2024-09-19T04:18:45.064427+0000","last_became_peered":"2024-09-19T04:18:45.064427+0000","last_unstale":"2024-09-19T04:18:47.102660+0000","last_undegraded":"2024-09-19T04:18:47.102660+0000","last_fullsized":"2024-09-19T04:18:47.102660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:35.499355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100654+0000","last_change":"2024-09-19T04:18:46.091808+0000","last_active":"2024-09-19T04:18:47.100654+0000","last_peered":"2024-09-19T04:18:47.100654+0000","last_clean":"2024-09-19T04:18:47.100654+0000","last_became_active":"2024-09-19T04:18:46.091051+0000","last_became_peered":"2024-09-19T04:18:46.091051+0000","last_unstale":"2024-09-19T04:18:47.100654+0000","last_undegraded":"2024-09-19T04:18:47.100654+0000","last_fullsized":"2024-09-19T04:18:47.100654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:56.162921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726082+0000","last_change":"2024-09-19T04:18:45.062064+0000","last_active":"2024-09-19T04:18:47.726082+0000","last_peered":"2024-09-19T04:18:47.726082+0000","last_clean":"2024-09-19T04:18:47.726082+0000","last_became_active":"2024-09-19T04:18:45.059487+0000","last_became_peered":"2024-09-19T04:18:45.059487+0000","last_unstale":"2024-09-19T04:18:47.726082+0000","last_undegraded":"2024-09-19T04:18:47.726082+0000","last_fullsized":"2024-09-19T04:18:47.726082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:27.832132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726319+0000","last_change":"2024-09-19T04:18:46.096522+0000","last_active":"2024-09-19T04:18:47.726319+0000","last_peered":"2024-09-19T04:18:47.726319+0000","last_clean":"2024-09-19T04:18:47.726319+0000","last_became_active":"2024-09-19T04:18:46.096266+0000","last_became_peered":"2024-09-19T04:18:46.096266+0000","last_unstale":"2024-09-19T04:18:47.726319+0000","last_undegraded":"2024-09-19T04:18:47.726319+0000","last_fullsized":"2024-09-19T04:18:47.726319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:34:07.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728218+0000","last_change":"2024-09-19T04:18:45.067236+0000","last_active":"2024-09-19T04:18:47.728218+0000","last_peered":"2024-09-19T04:18:47.728218+0000","last_clean":"2024-09-19T04:18:47.728218+0000","last_became_active":"2024-09-19T04:18:45.066928+0000","last_became_peered":"2024-09-19T04:18:45.066928+0000","last_unstale":"2024-09-19T04:18:47.728218+0000","last_undegraded":"2024-09-19T04:18:47.728218+0000","last_fullsized":"2024-09-19T04:18:47.728218+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:28.596726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098136+0000","last_change":"2024-09-19T04:18:46.103544+0000","last_active":"2024-09-19T04:18:47.098136+0000","last_peered":"2024-09-19T04:18:47.098136+0000","last_clean":"2024-09-19T04:18:47.098136+0000","last_became_active":"2024-09-19T04:18:46.103277+0000","last_became_peered":"2024-09-19T04:18:46.103277+0000","last_unstale":"2024-09-19T04:18:47.098136+0000","last_undegraded":"2024-09-19T04:18:47.098136+0000","last_fullsized":"2024-09-19T04:18:47.098136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.077041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099847+0000","last_change":"2024-09-19T04:18:45.053835+0000","last_active":"2024-09-19T04:18:47.099847+0000","last_peered":"2024-09-19T04:18:47.099847+0000","last_clean":"2024-09-19T04:18:47.099847+0000","last_became_active":"2024-09-19T04:18:45.052396+0000","last_became_peered":"2024-09-19T04:18:45.052396+0000","last_unstale":"2024-09-19T04:18:47.099847+0000","last_undegraded":"2024-09-19T04:18:47.099847+0000","last_fullsized":"2024-09-19T04:18:47.099847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:15.790489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725436+0000","last_change":"2024-09-19T04:18:46.105631+0000","last_active":"2024-09-19T04:18:47.725436+0000","last_peered":"2024-09-19T04:18:47.725436+0000","last_clean":"2024-09-19T04:18:47.725436+0000","last_became_active":"2024-09-19T04:18:46.099065+0000","last_became_peered":"2024-09-19T04:18:46.099065+0000","last_unstale":"2024-09-19T04:18:47.725436+0000","last_undegraded":"2024-09-19T04:18:47.725436+0000","last_fullsized":"2024-09-19T04:18:47.725436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:32.959237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725774+0000","last_change":"2024-09-19T04:18:45.060688+0000","last_active":"2024-09-19T04:18:47.725774+0000","last_peered":"2024-09-19T04:18:47.725774+0000","last_clean":"2024-09-19T04:18:47.725774+0000","last_became_active":"2024-09-19T04:18:45.059926+0000","last_became_peered":"2024-09-19T04:18:45.059926+0000","last_unstale":"2024-09-19T04:18:47.725774+0000","last_undegraded":"2024-09-19T04:18:47.725774+0000","last_fullsized":"2024-09-19T04:18:47.725774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:19.186809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727949+0000","last_change":"2024-09-19T04:18:46.099468+0000","last_active":"2024-09-19T04:18:47.727949+0000","last_peered":"2024-09-19T04:18:47.727949+0000","last_clean":"2024-09-19T04:18:47.727949+0000","last_became_active":"2024-09-19T04:18:46.097621+0000","last_became_peered":"2024-09-19T04:18:46.097621+0000","last_unstale":"2024-09-19T04:18:47.727949+0000","last_undegraded":"2024-09-19T04:18:47.727949+0000","last_fullsized":"2024-09-19T04:18:47.727949+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:06.824615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.724888+0000","last_change":"2024-09-19T04:18:45.067388+0000","last_active":"2024-09-19T04:18:47.724888+0000","last_peered":"2024-09-19T04:18:47.724888+0000","last_clean":"2024-09-19T04:18:47.724888+0000","last_became_active":"2024-09-19T04:18:45.065842+0000","last_became_peered":"2024-09-19T04:18:45.065842+0000","last_unstale":"2024-09-19T04:18:47.724888+0000","last_undegraded":"2024-09-19T04:18:47.724888+0000","last_fullsized":"2024-09-19T04:18:47.724888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:23.707349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726740+0000","last_change":"2024-09-19T04:18:46.098202+0000","last_active":"2024-09-19T04:18:47.726740+0000","last_peered":"2024-09-19T04:18:47.726740+0000","last_clean":"2024-09-19T04:18:47.726740+0000","last_became_active":"2024-09-19T04:18:46.097446+0000","last_became_peered":"2024-09-19T04:18:46.097446+0000","last_unstale":"2024-09-19T04:18:47.726740+0000","last_undegraded":"2024-09-19T04:18:47.726740+0000","last_fullsized":"2024-09-19T04:18:47.726740+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:01.117771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728320+0000","last_change":"2024-09-19T04:18:45.047792+0000","last_active":"2024-09-19T04:18:47.728320+0000","last_peered":"2024-09-19T04:18:47.728320+0000","last_clean":"2024-09-19T04:18:47.728320+0000","last_became_active":"2024-09-19T04:18:45.047357+0000","last_became_peered":"2024-09-19T04:18:45.047357+0000","last_unstale":"2024-09-19T04:18:47.728320+0000","last_undegraded":"2024-09-19T04:18:47.728320+0000","last_fullsized":"2024-09-19T04:18:47.728320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:57.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725043+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:18:47.725043+0000","last_peered":"2024-09-19T04:18:47.725043+0000","last_clean":"2024-09-19T04:18:47.725043+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:18:47.725043+0000","last_undegraded":"2024-09-19T04:18:47.725043+0000","last_fullsized":"2024-09-19T04:18:47.725043+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542214,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1708032,"data_stored":1102774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":521258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1118208,"data_stored":519042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1126400,"data_stored":519202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542214,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":519042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":512350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542215,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1110016,"data_stored":512420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542215,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1724416,"data_stored":1105036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:18:50.140 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:18:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c7ffa640 1 Processor -- start 2024-09-19T04:18:50.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c7ffa640 1 -- start start 2024-09-19T04:18:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c7ffa640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc5c015bec0 0x7fc5c00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c7ffa640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c7ffa640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5c00b3290 con 0x7fc5c00af2a0 2024-09-19T04:18:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c7ffa640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc5c00b3570 con 0x7fc5c015bec0 2024-09-19T04:18:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c7ffa640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c00b3850 con 0x7fc5c015c2a0 2024-09-19T04:18:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5d96640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc5c015bec0 0x7fc5c00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5d96640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc5c015bec0 0x7fc5c00ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46586/0 (socket says 172.21.15.76:46586) 2024-09-19T04:18:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5d96640 1 -- 172.21.15.76:0/2804581975 learned_addr learned my addr 172.21.15.76:0/2804581975 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5595640 1 --2- 172.21.15.76:0/2804581975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5d96640 1 -- 172.21.15.76:0/2804581975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 msgr2=0x7fc5c00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5d96640 1 --2- 172.21.15.76:0/2804581975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5d96640 1 -- 172.21.15.76:0/2804581975 >> v1:172.21.15.116:6789/0 conn(0x7fc5c00af2a0 legacy=0x7fc5c015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:50.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.305+0000 7fc5c5d96640 1 -- 172.21.15.76:0/2804581975 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c00b3b30 con 0x7fc5c015bec0 2024-09-19T04:18:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c5d96640 1 --2- 172.21.15.76:0/2804581975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c00ad200 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009870 tx=0x7fc5bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e3ec825fea955c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c4d94640 1 -- 172.21.15.76:0/2804581975 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0aa610 con 0x7fc5c015bec0 2024-09-19T04:18:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c4d94640 1 -- 172.21.15.76:0/2804581975 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5bc0af070 con 0x7fc5c015bec0 2024-09-19T04:18:50.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c4d94640 1 -- 172.21.15.76:0/2804581975 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0aa9d0 con 0x7fc5c015bec0 2024-09-19T04:18:50.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/2804581975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 msgr2=0x7fc5c00ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/2804581975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c00ad200 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc009870 tx=0x7fc5bc01c4f0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/2804581975 shutdown_connections 2024-09-19T04:18:50.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/2804581975 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/2804581975 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c00ad200 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/2804581975 >> 172.21.15.76:0/2804581975 conn(0x7fc5c00b1f90 msgr2=0x7fc5c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:50.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/2804581975 shutdown_connections 2024-09-19T04:18:50.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.309+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/2804581975 wait complete. 2024-09-19T04:18:50.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 Processor -- start 2024-09-19T04:18:50.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 -- start start 2024-09-19T04:18:50.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5c00af2a0 0x7fc5c009adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c0152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00ad740 con 0x7fc5c015bec0 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc5c00afc90 con 0x7fc5c00af2a0 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc5c0169000 con 0x7fc5c015c2a0 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c0152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c0152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46592/0 (socket says 172.21.15.76:46592) 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 -- 172.21.15.76:0/3335085851 learned_addr learned my addr 172.21.15.76:0/3335085851 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c6597640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5d96640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5c00af2a0 0x7fc5c009adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 -- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 msgr2=0x7fc5c00b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 -- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5c00af2a0 msgr2=0x7fc5c009adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5c00af2a0 0x7fc5c009adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 -- 172.21.15.76:0/3335085851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5c009b390 con 0x7fc5c015bec0 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5d96640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5c00af2a0 0x7fc5c009adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c5595640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c0152d30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c7f0 tx=0x7fc5b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5b67fc640 1 -- 172.21.15.76:0/3335085851 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0015070 con 0x7fc5c015bec0 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5b67fc640 1 -- 172.21.15.76:0/3335085851 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b0007820 con 0x7fc5c015bec0 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5c009b670 con 0x7fc5c015bec0 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5b67fc640 1 -- 172.21.15.76:0/3335085851 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b0007c10 con 0x7fc5c015bec0 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5c009d070 con 0x7fc5c015bec0 2024-09-19T04:18:50.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.313+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5c00ad200 con 0x7fc5c015bec0 2024-09-19T04:18:50.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.317+0000 7fc5b67fc640 1 -- 172.21.15.76:0/3335085851 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc5b0018070 con 0x7fc5c015bec0 2024-09-19T04:18:50.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.317+0000 7fc5b67fc640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5a40678c0 0x7fc5a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.317+0000 7fc5c5d96640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5a40678c0 0x7fc5a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.317+0000 7fc5b67fc640 1 -- 172.21.15.76:0/3335085851 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fc5b0085f00 con 0x7fc5c015bec0 2024-09-19T04:18:50.318 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.317+0000 7fc5c5d96640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5a40678c0 0x7fc5a4069d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0ae000 tx=0x7fc5bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:50.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.321+0000 7fc5b67fc640 1 -- 172.21.15.76:0/3335085851 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc5b00568b0 con 0x7fc5c015bec0 2024-09-19T04:18:50.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.477+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5c009a630 con 0x7fc5a40678c0 2024-09-19T04:18:50.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.493+0000 7fc5b67fc640 1 -- 172.21.15.76:0/3335085851 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252073 (secure 0 0 0) 0x7fc5bc0af040 con 0x7fc5a40678c0 2024-09-19T04:18:50.498 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:50.499 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:18:50.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5a40678c0 msgr2=0x7fc5a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5a40678c0 0x7fc5a4069d80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc0ae000 tx=0x7fc5bc008040 comp rx=0 tx=0).stop 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 msgr2=0x7fc5c0152d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c0152d30 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fc5b000c7f0 tx=0x7fc5b000ccb0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 shutdown_connections 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc5a40678c0 0x7fc5a4069d80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc5c015c2a0 0x7fc5c00b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc5c015bec0 0x7fc5c0152d30 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 --2- 172.21.15.76:0/3335085851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5c00af2a0 0x7fc5c009adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 >> 172.21.15.76:0/3335085851 conn(0x7fc5c00b1f90 msgr2=0x7fc5c0074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 shutdown_connections 2024-09-19T04:18:50.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.497+0000 7fc5c7ffa640 1 -- 172.21.15.76:0/3335085851 wait complete. 2024-09-19T04:18:50.516 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-19T04:18:49.842473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":149,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":255284,"kb_used_data":9912,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754719436,"statfs":{"total":773094113280,"available":772832702464,"internally_reserved":0,"allocated":10149888,"data_stored":5311124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.822903"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098166+0000","last_change":"2024-09-19T04:18:46.097510+0000","last_active":"2024-09-19T04:18:47.098166+0000","last_peered":"2024-09-19T04:18:47.098166+0000","last_clean":"2024-09-19T04:18:47.098166+0000","last_became_active":"2024-09-19T04:18:46.097122+0000","last_became_peered":"2024-09-19T04:18:46.097122+0000","last_unstale":"2024-09-19T04:18:47.098166+0000","last_undegraded":"2024-09-19T04:18:47.098166+0000","last_fullsized":"2024-09-19T04:18:47.098166+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:48:01.024313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100226+0000","last_change":"2024-09-19T04:18:46.105337+0000","last_active":"2024-09-19T04:18:47.100226+0000","last_peered":"2024-09-19T04:18:47.100226+0000","last_clean":"2024-09-19T04:18:47.100226+0000","last_became_active":"2024-09-19T04:18:46.104585+0000","last_became_peered":"2024-09-19T04:18:46.104585+0000","last_unstale":"2024-09-19T04:18:47.100226+0000","last_undegraded":"2024-09-19T04:18:47.100226+0000","last_fullsized":"2024-09-19T04:18:47.100226+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:41:32.235161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098039+0000","last_change":"2024-09-19T04:18:46.111561+0000","last_active":"2024-09-19T04:18:47.098039+0000","last_peered":"2024-09-19T04:18:47.098039+0000","last_clean":"2024-09-19T04:18:47.098039+0000","last_became_active":"2024-09-19T04:18:46.111293+0000","last_became_peered":"2024-09-19T04:18:46.111293+0000","last_unstale":"2024-09-19T04:18:47.098039+0000","last_undegraded":"2024-09-19T04:18:47.098039+0000","last_fullsized":"2024-09-19T04:18:47.098039+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:28:30.291475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727271+0000","last_change":"2024-09-19T04:18:46.112337+0000","last_active":"2024-09-19T04:18:47.727271+0000","last_peered":"2024-09-19T04:18:47.727271+0000","last_clean":"2024-09-19T04:18:47.727271+0000","last_became_active":"2024-09-19T04:18:46.111978+0000","last_became_peered":"2024-09-19T04:18:46.111978+0000","last_unstale":"2024-09-19T04:18:47.727271+0000","last_undegraded":"2024-09-19T04:18:47.727271+0000","last_fullsized":"2024-09-19T04:18:47.727271+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:44.014241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101949+0000","last_change":"2024-09-19T04:18:46.089524+0000","last_active":"2024-09-19T04:18:47.101949+0000","last_peered":"2024-09-19T04:18:47.101949+0000","last_clean":"2024-09-19T04:18:47.101949+0000","last_became_active":"2024-09-19T04:18:46.088754+0000","last_became_peered":"2024-09-19T04:18:46.088754+0000","last_unstale":"2024-09-19T04:18:47.101949+0000","last_undegraded":"2024-09-19T04:18:47.101949+0000","last_fullsized":"2024-09-19T04:18:47.101949+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:45.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728622+0000","last_change":"2024-09-19T04:18:46.092499+0000","last_active":"2024-09-19T04:18:47.728622+0000","last_peered":"2024-09-19T04:18:47.728622+0000","last_clean":"2024-09-19T04:18:47.728622+0000","last_became_active":"2024-09-19T04:18:46.092122+0000","last_became_peered":"2024-09-19T04:18:46.092122+0000","last_unstale":"2024-09-19T04:18:47.728622+0000","last_undegraded":"2024-09-19T04:18:47.728622+0000","last_fullsized":"2024-09-19T04:18:47.728622+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:34:19.113397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725042+0000","last_change":"2024-09-19T04:18:46.102200+0000","last_active":"2024-09-19T04:18:47.725042+0000","last_peered":"2024-09-19T04:18:47.725042+0000","last_clean":"2024-09-19T04:18:47.725042+0000","last_became_active":"2024-09-19T04:18:46.101361+0000","last_became_peered":"2024-09-19T04:18:46.101361+0000","last_unstale":"2024-09-19T04:18:47.725042+0000","last_undegraded":"2024-09-19T04:18:47.725042+0000","last_fullsized":"2024-09-19T04:18:47.725042+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:33:20.844483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.730320+0000","last_change":"2024-09-19T04:18:46.092397+0000","last_active":"2024-09-19T04:18:47.730320+0000","last_peered":"2024-09-19T04:18:47.730320+0000","last_clean":"2024-09-19T04:18:47.730320+0000","last_became_active":"2024-09-19T04:18:46.092124+0000","last_became_peered":"2024-09-19T04:18:46.092124+0000","last_unstale":"2024-09-19T04:18:47.730320+0000","last_undegraded":"2024-09-19T04:18:47.730320+0000","last_fullsized":"2024-09-19T04:18:47.730320+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:16.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102590+0000","last_change":"2024-09-19T04:18:46.081787+0000","last_active":"2024-09-19T04:18:47.102590+0000","last_peered":"2024-09-19T04:18:47.102590+0000","last_clean":"2024-09-19T04:18:47.102590+0000","last_became_active":"2024-09-19T04:18:46.081025+0000","last_became_peered":"2024-09-19T04:18:46.081025+0000","last_unstale":"2024-09-19T04:18:47.102590+0000","last_undegraded":"2024-09-19T04:18:47.102590+0000","last_fullsized":"2024-09-19T04:18:47.102590+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:28:08.034709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727985+0000","last_change":"2024-09-19T04:18:46.104166+0000","last_active":"2024-09-19T04:18:47.727985+0000","last_peered":"2024-09-19T04:18:47.727985+0000","last_clean":"2024-09-19T04:18:47.727985+0000","last_became_active":"2024-09-19T04:18:46.103506+0000","last_became_peered":"2024-09-19T04:18:46.103506+0000","last_unstale":"2024-09-19T04:18:47.727985+0000","last_undegraded":"2024-09-19T04:18:47.727985+0000","last_fullsized":"2024-09-19T04:18:47.727985+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:34:51.573366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101959+0000","last_change":"2024-09-19T04:18:46.097343+0000","last_active":"2024-09-19T04:18:47.101959+0000","last_peered":"2024-09-19T04:18:47.101959+0000","last_clean":"2024-09-19T04:18:47.101959+0000","last_became_active":"2024-09-19T04:18:46.092130+0000","last_became_peered":"2024-09-19T04:18:46.092130+0000","last_unstale":"2024-09-19T04:18:47.101959+0000","last_undegraded":"2024-09-19T04:18:47.101959+0000","last_fullsized":"2024-09-19T04:18:47.101959+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:06.256564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727380+0000","last_change":"2024-09-19T04:18:46.101082+0000","last_active":"2024-09-19T04:18:47.727380+0000","last_peered":"2024-09-19T04:18:47.727380+0000","last_clean":"2024-09-19T04:18:47.727380+0000","last_became_active":"2024-09-19T04:18:46.100683+0000","last_became_peered":"2024-09-19T04:18:46.100683+0000","last_unstale":"2024-09-19T04:18:47.727380+0000","last_undegraded":"2024-09-19T04:18:47.727380+0000","last_fullsized":"2024-09-19T04:18:47.727380+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:32.000061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728479+0000","last_change":"2024-09-19T04:18:46.091532+0000","last_active":"2024-09-19T04:18:47.728479+0000","last_peered":"2024-09-19T04:18:47.728479+0000","last_clean":"2024-09-19T04:18:47.728479+0000","last_became_active":"2024-09-19T04:18:46.090826+0000","last_became_peered":"2024-09-19T04:18:46.090826+0000","last_unstale":"2024-09-19T04:18:47.728479+0000","last_undegraded":"2024-09-19T04:18:47.728479+0000","last_fullsized":"2024-09-19T04:18:47.728479+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:32.959237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099291+0000","last_change":"2024-09-19T04:18:46.105958+0000","last_active":"2024-09-19T04:18:47.099291+0000","last_peered":"2024-09-19T04:18:47.099291+0000","last_clean":"2024-09-19T04:18:47.099291+0000","last_became_active":"2024-09-19T04:18:46.100355+0000","last_became_peered":"2024-09-19T04:18:46.100355+0000","last_unstale":"2024-09-19T04:18:47.099291+0000","last_undegraded":"2024-09-19T04:18:47.099291+0000","last_fullsized":"2024-09-19T04:18:47.099291+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:42:44.014241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729078+0000","last_change":"2024-09-19T04:18:46.099766+0000","last_active":"2024-09-19T04:18:47.729078+0000","last_peered":"2024-09-19T04:18:47.729078+0000","last_clean":"2024-09-19T04:18:47.729078+0000","last_became_active":"2024-09-19T04:18:46.099457+0000","last_became_peered":"2024-09-19T04:18:46.099457+0000","last_unstale":"2024-09-19T04:18:47.729078+0000","last_undegraded":"2024-09-19T04:18:47.729078+0000","last_fullsized":"2024-09-19T04:18:47.729078+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:45.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727272+0000","last_change":"2024-09-19T04:18:46.103105+0000","last_active":"2024-09-19T04:18:47.727272+0000","last_peered":"2024-09-19T04:18:47.727272+0000","last_clean":"2024-09-19T04:18:47.727272+0000","last_became_active":"2024-09-19T04:18:46.102099+0000","last_became_peered":"2024-09-19T04:18:46.102099+0000","last_unstale":"2024-09-19T04:18:47.727272+0000","last_undegraded":"2024-09-19T04:18:47.727272+0000","last_fullsized":"2024-09-19T04:18:47.727272+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:47:47.549289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729035+0000","last_change":"2024-09-19T04:18:46.098621+0000","last_active":"2024-09-19T04:18:47.729035+0000","last_peered":"2024-09-19T04:18:47.729035+0000","last_clean":"2024-09-19T04:18:47.729035+0000","last_became_active":"2024-09-19T04:18:46.098105+0000","last_became_peered":"2024-09-19T04:18:46.098105+0000","last_unstale":"2024-09-19T04:18:47.729035+0000","last_undegraded":"2024-09-19T04:18:47.729035+0000","last_fullsized":"2024-09-19T04:18:47.729035+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:47.769886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.097930+0000","last_change":"2024-09-19T04:18:46.098650+0000","last_active":"2024-09-19T04:18:47.097930+0000","last_peered":"2024-09-19T04:18:47.097930+0000","last_clean":"2024-09-19T04:18:47.097930+0000","last_became_active":"2024-09-19T04:18:46.098316+0000","last_became_peered":"2024-09-19T04:18:46.098316+0000","last_unstale":"2024-09-19T04:18:47.097930+0000","last_undegraded":"2024-09-19T04:18:47.097930+0000","last_fullsized":"2024-09-19T04:18:47.097930+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:18.081156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102399+0000","last_change":"2024-09-19T04:18:46.111648+0000","last_active":"2024-09-19T04:18:47.102399+0000","last_peered":"2024-09-19T04:18:47.102399+0000","last_clean":"2024-09-19T04:18:47.102399+0000","last_became_active":"2024-09-19T04:18:46.108501+0000","last_became_peered":"2024-09-19T04:18:46.108501+0000","last_unstale":"2024-09-19T04:18:47.102399+0000","last_undegraded":"2024-09-19T04:18:47.102399+0000","last_fullsized":"2024-09-19T04:18:47.102399+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:27.775882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727681+0000","last_change":"2024-09-19T04:18:46.101315+0000","last_active":"2024-09-19T04:18:47.727681+0000","last_peered":"2024-09-19T04:18:47.727681+0000","last_clean":"2024-09-19T04:18:47.727681+0000","last_became_active":"2024-09-19T04:18:46.100922+0000","last_became_peered":"2024-09-19T04:18:46.100922+0000","last_unstale":"2024-09-19T04:18:47.727681+0000","last_undegraded":"2024-09-19T04:18:47.727681+0000","last_fullsized":"2024-09-19T04:18:47.727681+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:16.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101232+0000","last_change":"2024-09-19T04:18:46.090088+0000","last_active":"2024-09-19T04:18:47.101232+0000","last_peered":"2024-09-19T04:18:47.101232+0000","last_clean":"2024-09-19T04:18:47.101232+0000","last_became_active":"2024-09-19T04:18:46.089305+0000","last_became_peered":"2024-09-19T04:18:46.089305+0000","last_unstale":"2024-09-19T04:18:47.101232+0000","last_undegraded":"2024-09-19T04:18:47.101232+0000","last_fullsized":"2024-09-19T04:18:47.101232+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:32.959237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099662+0000","last_change":"2024-09-19T04:18:46.096177+0000","last_active":"2024-09-19T04:18:47.099662+0000","last_peered":"2024-09-19T04:18:47.099662+0000","last_clean":"2024-09-19T04:18:47.099662+0000","last_became_active":"2024-09-19T04:18:46.092091+0000","last_became_peered":"2024-09-19T04:18:46.092091+0000","last_unstale":"2024-09-19T04:18:47.099662+0000","last_undegraded":"2024-09-19T04:18:47.099662+0000","last_fullsized":"2024-09-19T04:18:47.099662+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:16.010135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725426+0000","last_change":"2024-09-19T04:18:46.099423+0000","last_active":"2024-09-19T04:18:47.725426+0000","last_peered":"2024-09-19T04:18:47.725426+0000","last_clean":"2024-09-19T04:18:47.725426+0000","last_became_active":"2024-09-19T04:18:46.098728+0000","last_became_peered":"2024-09-19T04:18:46.098728+0000","last_unstale":"2024-09-19T04:18:47.725426+0000","last_undegraded":"2024-09-19T04:18:47.725426+0000","last_fullsized":"2024-09-19T04:18:47.725426+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:08.793346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.097986+0000","last_change":"2024-09-19T04:18:46.098484+0000","last_active":"2024-09-19T04:18:47.097986+0000","last_peered":"2024-09-19T04:18:47.097986+0000","last_clean":"2024-09-19T04:18:47.097986+0000","last_became_active":"2024-09-19T04:18:46.098244+0000","last_became_peered":"2024-09-19T04:18:46.098244+0000","last_unstale":"2024-09-19T04:18:47.097986+0000","last_undegraded":"2024-09-19T04:18:47.097986+0000","last_fullsized":"2024-09-19T04:18:47.097986+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:29.608036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100462+0000","last_change":"2024-09-19T04:18:46.108052+0000","last_active":"2024-09-19T04:18:47.100462+0000","last_peered":"2024-09-19T04:18:47.100462+0000","last_clean":"2024-09-19T04:18:47.100462+0000","last_became_active":"2024-09-19T04:18:46.107296+0000","last_became_peered":"2024-09-19T04:18:46.107296+0000","last_unstale":"2024-09-19T04:18:47.100462+0000","last_undegraded":"2024-09-19T04:18:47.100462+0000","last_fullsized":"2024-09-19T04:18:47.100462+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:28.803233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101636+0000","last_change":"2024-09-19T04:18:46.109379+0000","last_active":"2024-09-19T04:18:47.101636+0000","last_peered":"2024-09-19T04:18:47.101636+0000","last_clean":"2024-09-19T04:18:47.101636+0000","last_became_active":"2024-09-19T04:18:46.106745+0000","last_became_peered":"2024-09-19T04:18:46.106745+0000","last_unstale":"2024-09-19T04:18:47.101636+0000","last_undegraded":"2024-09-19T04:18:47.101636+0000","last_fullsized":"2024-09-19T04:18:47.101636+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:08.793346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726882+0000","last_change":"2024-09-19T04:18:46.111471+0000","last_active":"2024-09-19T04:18:47.726882+0000","last_peered":"2024-09-19T04:18:47.726882+0000","last_clean":"2024-09-19T04:18:47.726882+0000","last_became_active":"2024-09-19T04:18:46.111090+0000","last_became_peered":"2024-09-19T04:18:46.111090+0000","last_unstale":"2024-09-19T04:18:47.726882+0000","last_undegraded":"2024-09-19T04:18:47.726882+0000","last_fullsized":"2024-09-19T04:18:47.726882+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:44:17.187828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728069+0000","last_change":"2024-09-19T04:18:46.099777+0000","last_active":"2024-09-19T04:18:47.728069+0000","last_peered":"2024-09-19T04:18:47.728069+0000","last_clean":"2024-09-19T04:18:47.728069+0000","last_became_active":"2024-09-19T04:18:46.099077+0000","last_became_peered":"2024-09-19T04:18:46.099077+0000","last_unstale":"2024-09-19T04:18:47.728069+0000","last_undegraded":"2024-09-19T04:18:47.728069+0000","last_fullsized":"2024-09-19T04:18:47.728069+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:02.640637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726265+0000","last_change":"2024-09-19T04:18:46.099467+0000","last_active":"2024-09-19T04:18:47.726265+0000","last_peered":"2024-09-19T04:18:47.726265+0000","last_clean":"2024-09-19T04:18:47.726265+0000","last_became_active":"2024-09-19T04:18:46.098753+0000","last_became_peered":"2024-09-19T04:18:46.098753+0000","last_unstale":"2024-09-19T04:18:47.726265+0000","last_undegraded":"2024-09-19T04:18:47.726265+0000","last_fullsized":"2024-09-19T04:18:47.726265+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:31:37.984225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100273+0000","last_change":"2024-09-19T04:18:46.082066+0000","last_active":"2024-09-19T04:18:47.100273+0000","last_peered":"2024-09-19T04:18:47.100273+0000","last_clean":"2024-09-19T04:18:47.100273+0000","last_became_active":"2024-09-19T04:18:46.078672+0000","last_became_peered":"2024-09-19T04:18:46.078672+0000","last_unstale":"2024-09-19T04:18:47.100273+0000","last_undegraded":"2024-09-19T04:18:47.100273+0000","last_fullsized":"2024-09-19T04:18:47.100273+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:40:02.640637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099317+0000","last_change":"2024-09-19T04:18:46.104497+0000","last_active":"2024-09-19T04:18:47.099317+0000","last_peered":"2024-09-19T04:18:47.099317+0000","last_clean":"2024-09-19T04:18:47.099317+0000","last_became_active":"2024-09-19T04:18:46.102334+0000","last_became_peered":"2024-09-19T04:18:46.102334+0000","last_unstale":"2024-09-19T04:18:47.099317+0000","last_undegraded":"2024-09-19T04:18:47.099317+0000","last_fullsized":"2024-09-19T04:18:47.099317+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:39:04.293681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729696+0000","last_change":"2024-09-19T04:18:46.102254+0000","last_active":"2024-09-19T04:18:47.729696+0000","last_peered":"2024-09-19T04:18:47.729696+0000","last_clean":"2024-09-19T04:18:47.729696+0000","last_became_active":"2024-09-19T04:18:46.101967+0000","last_became_peered":"2024-09-19T04:18:46.101967+0000","last_unstale":"2024-09-19T04:18:47.729696+0000","last_undegraded":"2024-09-19T04:18:47.729696+0000","last_fullsized":"2024-09-19T04:18:47.729696+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:11:48.534500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727993+0000","last_change":"2024-09-19T04:18:46.092823+0000","last_active":"2024-09-19T04:18:47.727993+0000","last_peered":"2024-09-19T04:18:47.727993+0000","last_clean":"2024-09-19T04:18:47.727993+0000","last_became_active":"2024-09-19T04:18:46.092231+0000","last_became_peered":"2024-09-19T04:18:46.092231+0000","last_unstale":"2024-09-19T04:18:47.727993+0000","last_undegraded":"2024-09-19T04:18:47.727993+0000","last_fullsized":"2024-09-19T04:18:47.727993+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:32:35.328519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"90'4","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726057+0000","last_change":"2024-09-19T04:18:45.061693+0000","last_active":"2024-09-19T04:18:47.726057+0000","last_peered":"2024-09-19T04:18:47.726057+0000","last_clean":"2024-09-19T04:18:47.726057+0000","last_became_active":"2024-09-19T04:18:45.060982+0000","last_became_peered":"2024-09-19T04:18:45.060982+0000","last_unstale":"2024-09-19T04:18:47.726057+0000","last_undegraded":"2024-09-19T04:18:47.726057+0000","last_fullsized":"2024-09-19T04:18:47.726057+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:41:16.115150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728297+0000","last_change":"2024-09-19T04:18:46.096754+0000","last_active":"2024-09-19T04:18:47.728297+0000","last_peered":"2024-09-19T04:18:47.728297+0000","last_clean":"2024-09-19T04:18:47.728297+0000","last_became_active":"2024-09-19T04:18:46.096455+0000","last_became_peered":"2024-09-19T04:18:46.096455+0000","last_unstale":"2024-09-19T04:18:47.728297+0000","last_undegraded":"2024-09-19T04:18:47.728297+0000","last_fullsized":"2024-09-19T04:18:47.728297+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:02:01.943423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101962+0000","last_change":"2024-09-19T04:18:45.065663+0000","last_active":"2024-09-19T04:18:47.101962+0000","last_peered":"2024-09-19T04:18:47.101962+0000","last_clean":"2024-09-19T04:18:47.101962+0000","last_became_active":"2024-09-19T04:18:45.063514+0000","last_became_peered":"2024-09-19T04:18:45.063514+0000","last_unstale":"2024-09-19T04:18:47.101962+0000","last_undegraded":"2024-09-19T04:18:47.101962+0000","last_fullsized":"2024-09-19T04:18:47.101962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:47.967223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726613+0000","last_change":"2024-09-19T04:18:46.099715+0000","last_active":"2024-09-19T04:18:47.726613+0000","last_peered":"2024-09-19T04:18:47.726613+0000","last_clean":"2024-09-19T04:18:47.726613+0000","last_became_active":"2024-09-19T04:18:46.098947+0000","last_became_peered":"2024-09-19T04:18:46.098947+0000","last_unstale":"2024-09-19T04:18:47.726613+0000","last_undegraded":"2024-09-19T04:18:47.726613+0000","last_fullsized":"2024-09-19T04:18:47.726613+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:51:25.791085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726892+0000","last_change":"2024-09-19T04:18:45.064570+0000","last_active":"2024-09-19T04:18:47.726892+0000","last_peered":"2024-09-19T04:18:47.726892+0000","last_clean":"2024-09-19T04:18:47.726892+0000","last_became_active":"2024-09-19T04:18:45.063857+0000","last_became_peered":"2024-09-19T04:18:45.063857+0000","last_unstale":"2024-09-19T04:18:47.726892+0000","last_undegraded":"2024-09-19T04:18:47.726892+0000","last_fullsized":"2024-09-19T04:18:47.726892+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:17:56.475914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729351+0000","last_change":"2024-09-19T04:18:46.107583+0000","last_active":"2024-09-19T04:18:47.729351+0000","last_peered":"2024-09-19T04:18:47.729351+0000","last_clean":"2024-09-19T04:18:47.729351+0000","last_became_active":"2024-09-19T04:18:46.107272+0000","last_became_peered":"2024-09-19T04:18:46.107272+0000","last_unstale":"2024-09-19T04:18:47.729351+0000","last_undegraded":"2024-09-19T04:18:47.729351+0000","last_fullsized":"2024-09-19T04:18:47.729351+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:33:59.412298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725815+0000","last_change":"2024-09-19T04:18:45.067978+0000","last_active":"2024-09-19T04:18:47.725815+0000","last_peered":"2024-09-19T04:18:47.725815+0000","last_clean":"2024-09-19T04:18:47.725815+0000","last_became_active":"2024-09-19T04:18:45.060466+0000","last_became_peered":"2024-09-19T04:18:45.060466+0000","last_unstale":"2024-09-19T04:18:47.725815+0000","last_undegraded":"2024-09-19T04:18:47.725815+0000","last_fullsized":"2024-09-19T04:18:47.725815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:44:21.243336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727454+0000","last_change":"2024-09-19T04:18:46.099897+0000","last_active":"2024-09-19T04:18:47.727454+0000","last_peered":"2024-09-19T04:18:47.727454+0000","last_clean":"2024-09-19T04:18:47.727454+0000","last_became_active":"2024-09-19T04:18:46.099619+0000","last_became_peered":"2024-09-19T04:18:46.099619+0000","last_unstale":"2024-09-19T04:18:47.727454+0000","last_undegraded":"2024-09-19T04:18:47.727454+0000","last_fullsized":"2024-09-19T04:18:47.727454+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:16.434612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101756+0000","last_change":"2024-09-19T04:18:45.056632+0000","last_active":"2024-09-19T04:18:47.101756+0000","last_peered":"2024-09-19T04:18:47.101756+0000","last_clean":"2024-09-19T04:18:47.101756+0000","last_became_active":"2024-09-19T04:18:45.055905+0000","last_became_peered":"2024-09-19T04:18:45.055905+0000","last_unstale":"2024-09-19T04:18:47.101756+0000","last_undegraded":"2024-09-19T04:18:47.101756+0000","last_fullsized":"2024-09-19T04:18:47.101756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:57.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725908+0000","last_change":"2024-09-19T04:18:46.095507+0000","last_active":"2024-09-19T04:18:47.725908+0000","last_peered":"2024-09-19T04:18:47.725908+0000","last_clean":"2024-09-19T04:18:47.725908+0000","last_became_active":"2024-09-19T04:18:46.094690+0000","last_became_peered":"2024-09-19T04:18:46.094690+0000","last_unstale":"2024-09-19T04:18:47.725908+0000","last_undegraded":"2024-09-19T04:18:47.725908+0000","last_fullsized":"2024-09-19T04:18:47.725908+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:45.529795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.730444+0000","last_change":"2024-09-19T04:18:45.055425+0000","last_active":"2024-09-19T04:18:47.730444+0000","last_peered":"2024-09-19T04:18:47.730444+0000","last_clean":"2024-09-19T04:18:47.730444+0000","last_became_active":"2024-09-19T04:18:45.055023+0000","last_became_peered":"2024-09-19T04:18:45.055023+0000","last_unstale":"2024-09-19T04:18:47.730444+0000","last_undegraded":"2024-09-19T04:18:47.730444+0000","last_fullsized":"2024-09-19T04:18:47.730444+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:21:11.948545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728984+0000","last_change":"2024-09-19T04:18:46.092574+0000","last_active":"2024-09-19T04:18:47.728984+0000","last_peered":"2024-09-19T04:18:47.728984+0000","last_clean":"2024-09-19T04:18:47.728984+0000","last_became_active":"2024-09-19T04:18:46.091536+0000","last_became_peered":"2024-09-19T04:18:46.091536+0000","last_unstale":"2024-09-19T04:18:47.728984+0000","last_undegraded":"2024-09-19T04:18:47.728984+0000","last_fullsized":"2024-09-19T04:18:47.728984+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:10:04.344307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.097868+0000","last_change":"2024-09-19T04:18:45.057250+0000","last_active":"2024-09-19T04:18:47.097868+0000","last_peered":"2024-09-19T04:18:47.097868+0000","last_clean":"2024-09-19T04:18:47.097868+0000","last_became_active":"2024-09-19T04:18:45.056733+0000","last_became_peered":"2024-09-19T04:18:45.056733+0000","last_unstale":"2024-09-19T04:18:47.097868+0000","last_undegraded":"2024-09-19T04:18:47.097868+0000","last_fullsized":"2024-09-19T04:18:47.097868+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:20:27.791923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728849+0000","last_change":"2024-09-19T04:18:46.107825+0000","last_active":"2024-09-19T04:18:47.728849+0000","last_peered":"2024-09-19T04:18:47.728849+0000","last_clean":"2024-09-19T04:18:47.728849+0000","last_became_active":"2024-09-19T04:18:46.107559+0000","last_became_peered":"2024-09-19T04:18:46.107559+0000","last_unstale":"2024-09-19T04:18:47.728849+0000","last_undegraded":"2024-09-19T04:18:47.728849+0000","last_fullsized":"2024-09-19T04:18:47.728849+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:01:08.793346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.724940+0000","last_change":"2024-09-19T04:18:45.068393+0000","last_active":"2024-09-19T04:18:47.724940+0000","last_peered":"2024-09-19T04:18:47.724940+0000","last_clean":"2024-09-19T04:18:47.724940+0000","last_became_active":"2024-09-19T04:18:45.066713+0000","last_became_peered":"2024-09-19T04:18:45.066713+0000","last_unstale":"2024-09-19T04:18:47.724940+0000","last_undegraded":"2024-09-19T04:18:47.724940+0000","last_fullsized":"2024-09-19T04:18:47.724940+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:56:57.653022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101971+0000","last_change":"2024-09-19T04:18:46.109048+0000","last_active":"2024-09-19T04:18:47.101971+0000","last_peered":"2024-09-19T04:18:47.101971+0000","last_clean":"2024-09-19T04:18:47.101971+0000","last_became_active":"2024-09-19T04:18:46.107899+0000","last_became_peered":"2024-09-19T04:18:46.107899+0000","last_unstale":"2024-09-19T04:18:47.101971+0000","last_undegraded":"2024-09-19T04:18:47.101971+0000","last_fullsized":"2024-09-19T04:18:47.101971+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:16.801799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100779+0000","last_change":"2024-09-19T04:18:45.044817+0000","last_active":"2024-09-19T04:18:47.100779+0000","last_peered":"2024-09-19T04:18:47.100779+0000","last_clean":"2024-09-19T04:18:47.100779+0000","last_became_active":"2024-09-19T04:18:45.043707+0000","last_became_peered":"2024-09-19T04:18:45.043707+0000","last_unstale":"2024-09-19T04:18:47.100779+0000","last_undegraded":"2024-09-19T04:18:47.100779+0000","last_fullsized":"2024-09-19T04:18:47.100779+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:44.518485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726745+0000","last_change":"2024-09-19T04:18:46.112976+0000","last_active":"2024-09-19T04:18:47.726745+0000","last_peered":"2024-09-19T04:18:47.726745+0000","last_clean":"2024-09-19T04:18:47.726745+0000","last_became_active":"2024-09-19T04:18:46.112238+0000","last_became_peered":"2024-09-19T04:18:46.112238+0000","last_unstale":"2024-09-19T04:18:47.726745+0000","last_undegraded":"2024-09-19T04:18:47.726745+0000","last_fullsized":"2024-09-19T04:18:47.726745+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:19.159205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098068+0000","last_change":"2024-09-19T04:18:45.053010+0000","last_active":"2024-09-19T04:18:47.098068+0000","last_peered":"2024-09-19T04:18:47.098068+0000","last_clean":"2024-09-19T04:18:47.098068+0000","last_became_active":"2024-09-19T04:18:45.052558+0000","last_became_peered":"2024-09-19T04:18:45.052558+0000","last_unstale":"2024-09-19T04:18:47.098068+0000","last_undegraded":"2024-09-19T04:18:47.098068+0000","last_fullsized":"2024-09-19T04:18:47.098068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:55.151611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098222+0000","last_change":"2024-09-19T04:18:46.099306+0000","last_active":"2024-09-19T04:18:47.098222+0000","last_peered":"2024-09-19T04:18:47.098222+0000","last_clean":"2024-09-19T04:18:47.098222+0000","last_became_active":"2024-09-19T04:18:46.098946+0000","last_became_peered":"2024-09-19T04:18:46.098946+0000","last_unstale":"2024-09-19T04:18:47.098222+0000","last_undegraded":"2024-09-19T04:18:47.098222+0000","last_fullsized":"2024-09-19T04:18:47.098222+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:01.283235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"90'3","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102401+0000","last_change":"2024-09-19T04:18:45.071213+0000","last_active":"2024-09-19T04:18:47.102401+0000","last_peered":"2024-09-19T04:18:47.102401+0000","last_clean":"2024-09-19T04:18:47.102401+0000","last_became_active":"2024-09-19T04:18:45.070552+0000","last_became_peered":"2024-09-19T04:18:45.070552+0000","last_unstale":"2024-09-19T04:18:47.102401+0000","last_undegraded":"2024-09-19T04:18:47.102401+0000","last_fullsized":"2024-09-19T04:18:47.102401+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:16.564686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728976+0000","last_change":"2024-09-19T04:18:46.092781+0000","last_active":"2024-09-19T04:18:47.728976+0000","last_peered":"2024-09-19T04:18:47.728976+0000","last_clean":"2024-09-19T04:18:47.728976+0000","last_became_active":"2024-09-19T04:18:46.090172+0000","last_became_peered":"2024-09-19T04:18:46.090172+0000","last_unstale":"2024-09-19T04:18:47.728976+0000","last_undegraded":"2024-09-19T04:18:47.728976+0000","last_fullsized":"2024-09-19T04:18:47.728976+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:21.590821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"90'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728277+0000","last_change":"2024-09-19T04:18:45.054695+0000","last_active":"2024-09-19T04:18:47.728277+0000","last_peered":"2024-09-19T04:18:47.728277+0000","last_clean":"2024-09-19T04:18:47.728277+0000","last_became_active":"2024-09-19T04:18:45.054264+0000","last_became_peered":"2024-09-19T04:18:45.054264+0000","last_unstale":"2024-09-19T04:18:47.728277+0000","last_undegraded":"2024-09-19T04:18:47.728277+0000","last_fullsized":"2024-09-19T04:18:47.728277+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:17.069845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729837+0000","last_change":"2024-09-19T04:18:46.092871+0000","last_active":"2024-09-19T04:18:47.729837+0000","last_peered":"2024-09-19T04:18:47.729837+0000","last_clean":"2024-09-19T04:18:47.729837+0000","last_became_active":"2024-09-19T04:18:46.092404+0000","last_became_peered":"2024-09-19T04:18:46.092404+0000","last_unstale":"2024-09-19T04:18:47.729837+0000","last_undegraded":"2024-09-19T04:18:47.729837+0000","last_fullsized":"2024-09-19T04:18:47.729837+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:06.256564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726249+0000","last_change":"2024-09-19T04:18:45.062957+0000","last_active":"2024-09-19T04:18:47.726249+0000","last_peered":"2024-09-19T04:18:47.726249+0000","last_clean":"2024-09-19T04:18:47.726249+0000","last_became_active":"2024-09-19T04:18:45.062159+0000","last_became_peered":"2024-09-19T04:18:45.062159+0000","last_unstale":"2024-09-19T04:18:47.726249+0000","last_undegraded":"2024-09-19T04:18:47.726249+0000","last_fullsized":"2024-09-19T04:18:47.726249+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:22.991055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099293+0000","last_change":"2024-09-19T04:18:46.090069+0000","last_active":"2024-09-19T04:18:47.099293+0000","last_peered":"2024-09-19T04:18:47.099293+0000","last_clean":"2024-09-19T04:18:47.099293+0000","last_became_active":"2024-09-19T04:18:46.089282+0000","last_became_peered":"2024-09-19T04:18:46.089282+0000","last_unstale":"2024-09-19T04:18:47.099293+0000","last_undegraded":"2024-09-19T04:18:47.099293+0000","last_fullsized":"2024-09-19T04:18:47.099293+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:00:16.434612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729222+0000","last_change":"2024-09-19T04:18:45.068440+0000","last_active":"2024-09-19T04:18:47.729222+0000","last_peered":"2024-09-19T04:18:47.729222+0000","last_clean":"2024-09-19T04:18:47.729222+0000","last_became_active":"2024-09-19T04:18:45.068087+0000","last_became_peered":"2024-09-19T04:18:45.068087+0000","last_unstale":"2024-09-19T04:18:47.729222+0000","last_undegraded":"2024-09-19T04:18:47.729222+0000","last_fullsized":"2024-09-19T04:18:47.729222+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:42:58.883567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726955+0000","last_change":"2024-09-19T04:18:46.103807+0000","last_active":"2024-09-19T04:18:47.726955+0000","last_peered":"2024-09-19T04:18:47.726955+0000","last_clean":"2024-09-19T04:18:47.726955+0000","last_became_active":"2024-09-19T04:18:46.103512+0000","last_became_peered":"2024-09-19T04:18:46.103512+0000","last_unstale":"2024-09-19T04:18:47.726955+0000","last_undegraded":"2024-09-19T04:18:47.726955+0000","last_fullsized":"2024-09-19T04:18:47.726955+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:10:00.526824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729526+0000","last_change":"2024-09-19T04:18:45.066053+0000","last_active":"2024-09-19T04:18:47.729526+0000","last_peered":"2024-09-19T04:18:47.729526+0000","last_clean":"2024-09-19T04:18:47.729526+0000","last_became_active":"2024-09-19T04:18:45.065679+0000","last_became_peered":"2024-09-19T04:18:45.065679+0000","last_unstale":"2024-09-19T04:18:47.729526+0000","last_undegraded":"2024-09-19T04:18:47.729526+0000","last_fullsized":"2024-09-19T04:18:47.729526+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:45:24.757521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727169+0000","last_change":"2024-09-19T04:18:46.102100+0000","last_active":"2024-09-19T04:18:47.727169+0000","last_peered":"2024-09-19T04:18:47.727169+0000","last_clean":"2024-09-19T04:18:47.727169+0000","last_became_active":"2024-09-19T04:18:46.101819+0000","last_became_peered":"2024-09-19T04:18:46.101819+0000","last_unstale":"2024-09-19T04:18:47.727169+0000","last_undegraded":"2024-09-19T04:18:47.727169+0000","last_fullsized":"2024-09-19T04:18:47.727169+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:07.050105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"90'8","reported_seq":22,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.730122+0000","last_change":"2024-09-19T04:18:45.055284+0000","last_active":"2024-09-19T04:18:47.730122+0000","last_peered":"2024-09-19T04:18:47.730122+0000","last_clean":"2024-09-19T04:18:47.730122+0000","last_became_active":"2024-09-19T04:18:45.054898+0000","last_became_peered":"2024-09-19T04:18:45.054898+0000","last_unstale":"2024-09-19T04:18:47.730122+0000","last_undegraded":"2024-09-19T04:18:47.730122+0000","last_fullsized":"2024-09-19T04:18:47.730122+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:24.060136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728340+0000","last_change":"2024-09-19T04:18:46.092514+0000","last_active":"2024-09-19T04:18:47.728340+0000","last_peered":"2024-09-19T04:18:47.728340+0000","last_clean":"2024-09-19T04:18:47.728340+0000","last_became_active":"2024-09-19T04:18:46.091581+0000","last_became_peered":"2024-09-19T04:18:46.091581+0000","last_unstale":"2024-09-19T04:18:47.728340+0000","last_undegraded":"2024-09-19T04:18:47.728340+0000","last_fullsized":"2024-09-19T04:18:47.728340+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:54.041420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726332+0000","last_change":"2024-09-19T04:18:45.068182+0000","last_active":"2024-09-19T04:18:47.726332+0000","last_peered":"2024-09-19T04:18:47.726332+0000","last_clean":"2024-09-19T04:18:47.726332+0000","last_became_active":"2024-09-19T04:18:45.067149+0000","last_became_peered":"2024-09-19T04:18:45.067149+0000","last_unstale":"2024-09-19T04:18:47.726332+0000","last_undegraded":"2024-09-19T04:18:47.726332+0000","last_fullsized":"2024-09-19T04:18:47.726332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:25:12.559217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725929+0000","last_change":"2024-09-19T04:18:46.084203+0000","last_active":"2024-09-19T04:18:47.725929+0000","last_peered":"2024-09-19T04:18:47.725929+0000","last_clean":"2024-09-19T04:18:47.725929+0000","last_became_active":"2024-09-19T04:18:46.083457+0000","last_became_peered":"2024-09-19T04:18:46.083457+0000","last_unstale":"2024-09-19T04:18:47.725929+0000","last_undegraded":"2024-09-19T04:18:47.725929+0000","last_fullsized":"2024-09-19T04:18:47.725929+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:06.256564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726631+0000","last_change":"2024-09-19T04:18:45.070055+0000","last_active":"2024-09-19T04:18:47.726631+0000","last_peered":"2024-09-19T04:18:47.726631+0000","last_clean":"2024-09-19T04:18:47.726631+0000","last_became_active":"2024-09-19T04:18:45.069313+0000","last_became_peered":"2024-09-19T04:18:45.069313+0000","last_unstale":"2024-09-19T04:18:47.726631+0000","last_undegraded":"2024-09-19T04:18:47.726631+0000","last_fullsized":"2024-09-19T04:18:47.726631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:17:04.342215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.101891+0000","last_change":"2024-09-19T04:18:46.108730+0000","last_active":"2024-09-19T04:18:47.101891+0000","last_peered":"2024-09-19T04:18:47.101891+0000","last_clean":"2024-09-19T04:18:47.101891+0000","last_became_active":"2024-09-19T04:18:46.100297+0000","last_became_peered":"2024-09-19T04:18:46.100297+0000","last_unstale":"2024-09-19T04:18:47.101891+0000","last_undegraded":"2024-09-19T04:18:47.101891+0000","last_fullsized":"2024-09-19T04:18:47.101891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:21.590821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"90'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102612+0000","last_change":"2024-09-19T04:18:45.055509+0000","last_active":"2024-09-19T04:18:47.102612+0000","last_peered":"2024-09-19T04:18:47.102612+0000","last_clean":"2024-09-19T04:18:47.102612+0000","last_became_active":"2024-09-19T04:18:45.054662+0000","last_became_peered":"2024-09-19T04:18:45.054662+0000","last_unstale":"2024-09-19T04:18:47.102612+0000","last_undegraded":"2024-09-19T04:18:47.102612+0000","last_fullsized":"2024-09-19T04:18:47.102612+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:06:43.972021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727701+0000","last_change":"2024-09-19T04:18:46.093493+0000","last_active":"2024-09-19T04:18:47.727701+0000","last_peered":"2024-09-19T04:18:47.727701+0000","last_clean":"2024-09-19T04:18:47.727701+0000","last_became_active":"2024-09-19T04:18:46.093197+0000","last_became_peered":"2024-09-19T04:18:46.093197+0000","last_unstale":"2024-09-19T04:18:47.727701+0000","last_undegraded":"2024-09-19T04:18:47.727701+0000","last_fullsized":"2024-09-19T04:18:47.727701+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:49.190050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099375+0000","last_change":"2024-09-19T04:18:45.055060+0000","last_active":"2024-09-19T04:18:47.099375+0000","last_peered":"2024-09-19T04:18:47.099375+0000","last_clean":"2024-09-19T04:18:47.099375+0000","last_became_active":"2024-09-19T04:18:45.052574+0000","last_became_peered":"2024-09-19T04:18:45.052574+0000","last_unstale":"2024-09-19T04:18:47.099375+0000","last_undegraded":"2024-09-19T04:18:47.099375+0000","last_fullsized":"2024-09-19T04:18:47.099375+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:20.579511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099390+0000","last_change":"2024-09-19T04:18:46.103856+0000","last_active":"2024-09-19T04:18:47.099390+0000","last_peered":"2024-09-19T04:18:47.099390+0000","last_clean":"2024-09-19T04:18:47.099390+0000","last_became_active":"2024-09-19T04:18:46.101379+0000","last_became_peered":"2024-09-19T04:18:46.101379+0000","last_unstale":"2024-09-19T04:18:47.099390+0000","last_undegraded":"2024-09-19T04:18:47.099390+0000","last_fullsized":"2024-09-19T04:18:47.099390+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:53:48.052777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727202+0000","last_change":"2024-09-19T04:18:45.049798+0000","last_active":"2024-09-19T04:18:47.727202+0000","last_peered":"2024-09-19T04:18:47.727202+0000","last_clean":"2024-09-19T04:18:47.727202+0000","last_became_active":"2024-09-19T04:18:45.049428+0000","last_became_peered":"2024-09-19T04:18:45.049428+0000","last_unstale":"2024-09-19T04:18:47.727202+0000","last_undegraded":"2024-09-19T04:18:47.727202+0000","last_fullsized":"2024-09-19T04:18:47.727202+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:55.241517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099922+0000","last_change":"2024-09-19T04:18:46.103821+0000","last_active":"2024-09-19T04:18:47.099922+0000","last_peered":"2024-09-19T04:18:47.099922+0000","last_clean":"2024-09-19T04:18:47.099922+0000","last_became_active":"2024-09-19T04:18:46.101231+0000","last_became_peered":"2024-09-19T04:18:46.101231+0000","last_unstale":"2024-09-19T04:18:47.099922+0000","last_undegraded":"2024-09-19T04:18:47.099922+0000","last_fullsized":"2024-09-19T04:18:47.099922+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:18:57.771748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099624+0000","last_change":"2024-09-19T04:18:45.052821+0000","last_active":"2024-09-19T04:18:47.099624+0000","last_peered":"2024-09-19T04:18:47.099624+0000","last_clean":"2024-09-19T04:18:47.099624+0000","last_became_active":"2024-09-19T04:18:45.052111+0000","last_became_peered":"2024-09-19T04:18:45.052111+0000","last_unstale":"2024-09-19T04:18:47.099624+0000","last_undegraded":"2024-09-19T04:18:47.099624+0000","last_fullsized":"2024-09-19T04:18:47.099624+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:05.245253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727538+0000","last_change":"2024-09-19T04:18:46.097301+0000","last_active":"2024-09-19T04:18:47.727538+0000","last_peered":"2024-09-19T04:18:47.727538+0000","last_clean":"2024-09-19T04:18:47.727538+0000","last_became_active":"2024-09-19T04:18:46.096923+0000","last_became_peered":"2024-09-19T04:18:46.096923+0000","last_unstale":"2024-09-19T04:18:47.727538+0000","last_undegraded":"2024-09-19T04:18:47.727538+0000","last_fullsized":"2024-09-19T04:18:47.727538+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:57:32.294118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726581+0000","last_change":"2024-09-19T04:18:45.048359+0000","last_active":"2024-09-19T04:18:47.726581+0000","last_peered":"2024-09-19T04:18:47.726581+0000","last_clean":"2024-09-19T04:18:47.726581+0000","last_became_active":"2024-09-19T04:18:45.047972+0000","last_became_peered":"2024-09-19T04:18:45.047972+0000","last_unstale":"2024-09-19T04:18:47.726581+0000","last_undegraded":"2024-09-19T04:18:47.726581+0000","last_fullsized":"2024-09-19T04:18:47.726581+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:22:37.891701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725528+0000","last_change":"2024-09-19T04:18:46.095443+0000","last_active":"2024-09-19T04:18:47.725528+0000","last_peered":"2024-09-19T04:18:47.725528+0000","last_clean":"2024-09-19T04:18:47.725528+0000","last_became_active":"2024-09-19T04:18:46.086131+0000","last_became_peered":"2024-09-19T04:18:46.086131+0000","last_unstale":"2024-09-19T04:18:47.725528+0000","last_undegraded":"2024-09-19T04:18:47.725528+0000","last_fullsized":"2024-09-19T04:18:47.725528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:04:21.590821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"90'2","reported_seq":16,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726815+0000","last_change":"2024-09-19T04:18:45.066540+0000","last_active":"2024-09-19T04:18:47.726815+0000","last_peered":"2024-09-19T04:18:47.726815+0000","last_clean":"2024-09-19T04:18:47.726815+0000","last_became_active":"2024-09-19T04:18:45.065648+0000","last_became_peered":"2024-09-19T04:18:45.065648+0000","last_unstale":"2024-09-19T04:18:47.726815+0000","last_undegraded":"2024-09-19T04:18:47.726815+0000","last_fullsized":"2024-09-19T04:18:47.726815+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:49:57.172726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.729417+0000","last_change":"2024-09-19T04:18:46.092324+0000","last_active":"2024-09-19T04:18:47.729417+0000","last_peered":"2024-09-19T04:18:47.729417+0000","last_clean":"2024-09-19T04:18:47.729417+0000","last_became_active":"2024-09-19T04:18:46.091943+0000","last_became_peered":"2024-09-19T04:18:46.091943+0000","last_unstale":"2024-09-19T04:18:47.729417+0000","last_undegraded":"2024-09-19T04:18:47.729417+0000","last_fullsized":"2024-09-19T04:18:47.729417+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:14:51.883485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102037+0000","last_change":"2024-09-19T04:18:45.047755+0000","last_active":"2024-09-19T04:18:47.102037+0000","last_peered":"2024-09-19T04:18:47.102037+0000","last_clean":"2024-09-19T04:18:47.102037+0000","last_became_active":"2024-09-19T04:18:45.046948+0000","last_became_peered":"2024-09-19T04:18:45.046948+0000","last_unstale":"2024-09-19T04:18:47.102037+0000","last_undegraded":"2024-09-19T04:18:47.102037+0000","last_fullsized":"2024-09-19T04:18:47.102037+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:56:24.060136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728731+0000","last_change":"2024-09-19T04:18:46.097027+0000","last_active":"2024-09-19T04:18:47.728731+0000","last_peered":"2024-09-19T04:18:47.728731+0000","last_clean":"2024-09-19T04:18:47.728731+0000","last_became_active":"2024-09-19T04:18:46.096656+0000","last_became_peered":"2024-09-19T04:18:46.096656+0000","last_unstale":"2024-09-19T04:18:47.728731+0000","last_undegraded":"2024-09-19T04:18:47.728731+0000","last_fullsized":"2024-09-19T04:18:47.728731+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:13.087989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.102660+0000","last_change":"2024-09-19T04:18:45.066608+0000","last_active":"2024-09-19T04:18:47.102660+0000","last_peered":"2024-09-19T04:18:47.102660+0000","last_clean":"2024-09-19T04:18:47.102660+0000","last_became_active":"2024-09-19T04:18:45.064427+0000","last_became_peered":"2024-09-19T04:18:45.064427+0000","last_unstale":"2024-09-19T04:18:47.102660+0000","last_undegraded":"2024-09-19T04:18:47.102660+0000","last_fullsized":"2024-09-19T04:18:47.102660+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:08:35.499355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.100654+0000","last_change":"2024-09-19T04:18:46.091808+0000","last_active":"2024-09-19T04:18:47.100654+0000","last_peered":"2024-09-19T04:18:47.100654+0000","last_clean":"2024-09-19T04:18:47.100654+0000","last_became_active":"2024-09-19T04:18:46.091051+0000","last_became_peered":"2024-09-19T04:18:46.091051+0000","last_unstale":"2024-09-19T04:18:47.100654+0000","last_undegraded":"2024-09-19T04:18:47.100654+0000","last_fullsized":"2024-09-19T04:18:47.100654+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:32:56.162921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726082+0000","last_change":"2024-09-19T04:18:45.062064+0000","last_active":"2024-09-19T04:18:47.726082+0000","last_peered":"2024-09-19T04:18:47.726082+0000","last_clean":"2024-09-19T04:18:47.726082+0000","last_became_active":"2024-09-19T04:18:45.059487+0000","last_became_peered":"2024-09-19T04:18:45.059487+0000","last_unstale":"2024-09-19T04:18:47.726082+0000","last_undegraded":"2024-09-19T04:18:47.726082+0000","last_fullsized":"2024-09-19T04:18:47.726082+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:06:27.832132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726319+0000","last_change":"2024-09-19T04:18:46.096522+0000","last_active":"2024-09-19T04:18:47.726319+0000","last_peered":"2024-09-19T04:18:47.726319+0000","last_clean":"2024-09-19T04:18:47.726319+0000","last_became_active":"2024-09-19T04:18:46.096266+0000","last_became_peered":"2024-09-19T04:18:46.096266+0000","last_unstale":"2024-09-19T04:18:47.726319+0000","last_undegraded":"2024-09-19T04:18:47.726319+0000","last_fullsized":"2024-09-19T04:18:47.726319+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:34:07.995511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728218+0000","last_change":"2024-09-19T04:18:45.067236+0000","last_active":"2024-09-19T04:18:47.728218+0000","last_peered":"2024-09-19T04:18:47.728218+0000","last_clean":"2024-09-19T04:18:47.728218+0000","last_became_active":"2024-09-19T04:18:45.066928+0000","last_became_peered":"2024-09-19T04:18:45.066928+0000","last_unstale":"2024-09-19T04:18:47.728218+0000","last_undegraded":"2024-09-19T04:18:47.728218+0000","last_fullsized":"2024-09-19T04:18:47.728218+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:06:28.596726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.098136+0000","last_change":"2024-09-19T04:18:46.103544+0000","last_active":"2024-09-19T04:18:47.098136+0000","last_peered":"2024-09-19T04:18:47.098136+0000","last_clean":"2024-09-19T04:18:47.098136+0000","last_became_active":"2024-09-19T04:18:46.103277+0000","last_became_peered":"2024-09-19T04:18:46.103277+0000","last_unstale":"2024-09-19T04:18:47.098136+0000","last_undegraded":"2024-09-19T04:18:47.098136+0000","last_fullsized":"2024-09-19T04:18:47.098136+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:28:17.077041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.099847+0000","last_change":"2024-09-19T04:18:45.053835+0000","last_active":"2024-09-19T04:18:47.099847+0000","last_peered":"2024-09-19T04:18:47.099847+0000","last_clean":"2024-09-19T04:18:47.099847+0000","last_became_active":"2024-09-19T04:18:45.052396+0000","last_became_peered":"2024-09-19T04:18:45.052396+0000","last_unstale":"2024-09-19T04:18:47.099847+0000","last_undegraded":"2024-09-19T04:18:47.099847+0000","last_fullsized":"2024-09-19T04:18:47.099847+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:13:15.790489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725436+0000","last_change":"2024-09-19T04:18:46.105631+0000","last_active":"2024-09-19T04:18:47.725436+0000","last_peered":"2024-09-19T04:18:47.725436+0000","last_clean":"2024-09-19T04:18:47.725436+0000","last_became_active":"2024-09-19T04:18:46.099065+0000","last_became_peered":"2024-09-19T04:18:46.099065+0000","last_unstale":"2024-09-19T04:18:47.725436+0000","last_undegraded":"2024-09-19T04:18:47.725436+0000","last_fullsized":"2024-09-19T04:18:47.725436+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:32.959237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725774+0000","last_change":"2024-09-19T04:18:45.060688+0000","last_active":"2024-09-19T04:18:47.725774+0000","last_peered":"2024-09-19T04:18:47.725774+0000","last_clean":"2024-09-19T04:18:47.725774+0000","last_became_active":"2024-09-19T04:18:45.059926+0000","last_became_peered":"2024-09-19T04:18:45.059926+0000","last_unstale":"2024-09-19T04:18:47.725774+0000","last_undegraded":"2024-09-19T04:18:47.725774+0000","last_fullsized":"2024-09-19T04:18:47.725774+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:19.186809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.727949+0000","last_change":"2024-09-19T04:18:46.099468+0000","last_active":"2024-09-19T04:18:47.727949+0000","last_peered":"2024-09-19T04:18:47.727949+0000","last_clean":"2024-09-19T04:18:47.727949+0000","last_became_active":"2024-09-19T04:18:46.097621+0000","last_became_peered":"2024-09-19T04:18:46.097621+0000","last_unstale":"2024-09-19T04:18:47.727949+0000","last_undegraded":"2024-09-19T04:18:47.727949+0000","last_fullsized":"2024-09-19T04:18:47.727949+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:08:06.824615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.724888+0000","last_change":"2024-09-19T04:18:45.067388+0000","last_active":"2024-09-19T04:18:47.724888+0000","last_peered":"2024-09-19T04:18:47.724888+0000","last_clean":"2024-09-19T04:18:47.724888+0000","last_became_active":"2024-09-19T04:18:45.065842+0000","last_became_peered":"2024-09-19T04:18:45.065842+0000","last_unstale":"2024-09-19T04:18:47.724888+0000","last_undegraded":"2024-09-19T04:18:47.724888+0000","last_fullsized":"2024-09-19T04:18:47.724888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:23.707349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.726740+0000","last_change":"2024-09-19T04:18:46.098202+0000","last_active":"2024-09-19T04:18:47.726740+0000","last_peered":"2024-09-19T04:18:47.726740+0000","last_clean":"2024-09-19T04:18:47.726740+0000","last_became_active":"2024-09-19T04:18:46.097446+0000","last_became_peered":"2024-09-19T04:18:46.097446+0000","last_unstale":"2024-09-19T04:18:47.726740+0000","last_undegraded":"2024-09-19T04:18:47.726740+0000","last_fullsized":"2024-09-19T04:18:47.726740+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":89,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:45.018355+0000","last_clean_scrub_stamp":"2024-09-19T04:18:45.018355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:31:01.117771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.728320+0000","last_change":"2024-09-19T04:18:45.047792+0000","last_active":"2024-09-19T04:18:47.728320+0000","last_peered":"2024-09-19T04:18:47.728320+0000","last_clean":"2024-09-19T04:18:47.728320+0000","last_became_active":"2024-09-19T04:18:45.047357+0000","last_became_peered":"2024-09-19T04:18:45.047357+0000","last_unstale":"2024-09-19T04:18:47.728320+0000","last_undegraded":"2024-09-19T04:18:47.728320+0000","last_fullsized":"2024-09-19T04:18:47.728320+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:18:44.007045+0000","last_clean_scrub_stamp":"2024-09-19T04:18:44.007045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:37:57.946864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":300,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-09-19T04:18:47.725043+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:18:47.725043+0000","last_peered":"2024-09-19T04:18:47.725043+0000","last_clean":"2024-09-19T04:18:47.725043+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:18:47.725043+0000","last_undegraded":"2024-09-19T04:18:47.725043+0000","last_fullsized":"2024-09-19T04:18:47.725043+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542214,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32860,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338980,"statfs":{"total":96636764160,"available":96603115520,"internally_reserved":0,"allocated":1708032,"data_stored":1102774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":521258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542215,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28128,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343712,"statfs":{"total":96636764160,"available":96607961088,"internally_reserved":0,"allocated":1118208,"data_stored":519042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1126400,"data_stored":519202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542214,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":519042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1105920,"data_stored":512350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542215,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1110016,"data_stored":512420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542215,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1724416,"data_stored":1105036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:18:50.520 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:18:50.520 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T04:18:50.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.685+0000 7f07e1f14640 1 Processor -- start 2024-09-19T04:18:50.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07e1f14640 1 -- start start 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07e1f14640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07dc0cc870 0x7f07dc0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07e1f14640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f07dc0cdfd0 0x7f07dc06de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07e1f14640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07dc0ce8d0 con 0x7f07dc073100 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07e1f14640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f07dc0cebb0 con 0x7f07dc0cdfd0 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07e1f14640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc0cee90 con 0x7f07dc0cc870 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07daffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f07dc073100 0x7f07dc0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:46998/0 (socket says 172.21.15.76:46998) 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07daffd640 1 -- 172.21.15.76:0/1124831525 learned_addr learned my addr 172.21.15.76:0/1124831525 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:50.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07db7fe640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07dc0cc870 0x7f07dc0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4248856622 0 0) 0x7f07dc0ce8d0 con 0x7f07dc073100 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f07c4003530 con 0x7f07dc073100 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07dbfff640 1 --2- 172.21.15.76:0/1124831525 >> v2:172.21.15.116:3300/0 conn(0x7f07dc0cdfd0 0x7f07dc06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3031268223 0 0) 0x7f07c4003530 con 0x7f07dc073100 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07dc0cc870 msgr2=0x7f07dc0cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07dc0cc870 0x7f07dc0cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 >> v2:172.21.15.116:3300/0 conn(0x7f07dc0cdfd0 msgr2=0x7f07dc06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 --2- 172.21.15.76:0/1124831525 >> v2:172.21.15.116:3300/0 conn(0x7f07dc0cdfd0 0x7f07dc06de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc0cf170 con 0x7f07dc073100 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f07cc001680 con 0x7f07dc073100 2024-09-19T04:18:50.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 >> v1:172.21.15.76:6789/0 conn(0x7f07dc073100 legacy=0x7f07dc0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07dc0cc870 0x7f07c40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cebb0 con 0x7f07c40039c0 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cee90 con 0x7f07c4004810 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f07c400b9d0 con 0x7f07dc0cc870 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07dbfff640 1 --2- 172.21.15.76:0/1124831525 >> v2:172.21.15.116:3300/0 conn(0x7f07dc0cdfd0 0x7f07dc06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07db7fe640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07db7fe640 1 -- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07dc0cc870 msgr2=0x7f07c40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:50.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07db7fe640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07dc0cc870 0x7f07c40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07db7fe640 1 -- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c4004810 msgr2=0x7f07c4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:50.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07db7fe640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.689+0000 7f07db7fe640 1 -- 172.21.15.76:0/1124831525 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c4003530 con 0x7f07c40039c0 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07db7fe640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009ae0 tx=0x7f07d001ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df95d0844dbb35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d00b2070 con 0x7f07c40039c0 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d00a5590 con 0x7f07c40039c0 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07da7fc640 1 -- 172.21.15.76:0/1124831525 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d00096b0 con 0x7f07c40039c0 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- 172.21.15.76:0/1124831525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07c40039c0 msgr2=0x7f07c4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 --2- 172.21.15.76:0/1124831525 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07c40039c0 0x7f07c4003de0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009ae0 tx=0x7f07d001ca80 comp rx=0 tx=0).stop 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 -- 172.21.15.76:0/1124831525 reap_dead start 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- 172.21.15.76:0/1124831525 shutdown_connections 2024-09-19T04:18:50.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- 172.21.15.76:0/1124831525 >> 172.21.15.76:0/1124831525 conn(0x7f07dc0b1c80 msgr2=0x7f07dc0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- 172.21.15.76:0/1124831525 shutdown_connections 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- 172.21.15.76:0/1124831525 wait complete. 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 Processor -- start 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- start start 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07c40039c0 0x7f07dc1fbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07dc1400e0 0x7f07dc0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07dc13f8f0 0x7f07dc0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc167180 con 0x7f07dc13f8f0 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cfd80 con 0x7f07dc1400e0 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07e1f14640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc167300 con 0x7f07c40039c0 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07dc13f8f0 0x7f07dc0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07dc13f8f0 0x7f07dc0afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46612/0 (socket says 172.21.15.76:46612) 2024-09-19T04:18:50.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 -- 172.21.15.76:0/3866963034 learned_addr learned my addr 172.21.15.76:0/3866963034 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07db7fe640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07c40039c0 0x7f07dc1fbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07daffd640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07dc1400e0 0x7f07dc0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 -- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07c40039c0 msgr2=0x7f07dc1fbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07c40039c0 0x7f07dc1fbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 -- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07dc1400e0 msgr2=0x7f07dc0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07dc1400e0 0x7f07dc0c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 -- 172.21.15.76:0/3866963034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc15f0b0 con 0x7f07dc13f8f0 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07daffd640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07dc1400e0 0x7f07dc0c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.693+0000 7f07dbfff640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07dc13f8f0 0x7f07dc0afe20 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0b60e0 tx=0x7f07d400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07c3fff640 1 -- 172.21.15.76:0/3866963034 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d401a070 con 0x7f07dc13f8f0 2024-09-19T04:18:50.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07c3fff640 1 -- 172.21.15.76:0/3866963034 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d4007e20 con 0x7f07dc13f8f0 2024-09-19T04:18:50.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc15f390 con 0x7f07dc13f8f0 2024-09-19T04:18:50.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07c3fff640 1 -- 172.21.15.76:0/3866963034 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003be0 con 0x7f07dc13f8f0 2024-09-19T04:18:50.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc1fb000 con 0x7f07dc13f8f0 2024-09-19T04:18:50.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07dc0af8d0 con 0x7f07dc13f8f0 2024-09-19T04:18:50.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07c3fff640 1 -- 172.21.15.76:0/3866963034 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f07d401d070 con 0x7f07dc13f8f0 2024-09-19T04:18:50.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07c3fff640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07c80678c0 0x7f07c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:50.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07c3fff640 1 -- 172.21.15.76:0/3866963034 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f07d4085c80 con 0x7f07dc13f8f0 2024-09-19T04:18:50.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07db7fe640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07c80678c0 0x7f07c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:50.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.697+0000 7f07db7fe640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07c80678c0 0x7f07c8069d80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f07d00af000 tx=0x7f07d00046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:50.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.701+0000 7f07c3fff640 1 -- 172.21.15.76:0/3866963034 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f07d4010070 con 0x7f07dc13f8f0 2024-09-19T04:18:50.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.909+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f07dc0cd830 con 0x7f07dc13f8f0 2024-09-19T04:18:50.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.909+0000 7f07c3fff640 1 -- 172.21.15.76:0/3866963034 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f07d4056630 con 0x7f07dc13f8f0 2024-09-19T04:18:50.912 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-19T04:18:50.912 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:18:50.912 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2024-09-19T04:18:50.912 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2024-09-19T04:18:50.913 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T04:18:50.913 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2024-09-19T04:18:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07c80678c0 msgr2=0x7f07c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07c80678c0 0x7f07c8069d80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f07d00af000 tx=0x7f07d00046d0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07dc13f8f0 msgr2=0x7f07dc0afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07dc13f8f0 0x7f07dc0afe20 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f07dc0b60e0 tx=0x7f07d400d4a0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 shutdown_connections 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f07c80678c0 0x7f07c8069d80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f07dc13f8f0 0x7f07dc0afe20 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07dc1400e0 0x7f07dc0c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 --2- 172.21.15.76:0/3866963034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f07c40039c0 0x7f07dc1fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 >> 172.21.15.76:0/3866963034 conn(0x7f07dc0b1c80 msgr2=0x7f07dc0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 shutdown_connections 2024-09-19T04:18:50.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:50.913+0000 7f07e1f14640 1 -- 172.21.15.76:0/3866963034 wait complete. 2024-09-19T04:18:50.930 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:51.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.041+0000 7f4f7b1c5640 1 Processor -- start 2024-09-19T04:18:51.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.041+0000 7f4f7b1c5640 1 -- start start 2024-09-19T04:18:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f7b1c5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f740b4b20 0x7f4f74141030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f7b1c5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7413fb30 con 0x7f4f74156720 2024-09-19T04:18:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f7b1c5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4f7413fe10 con 0x7f4f74140bd0 2024-09-19T04:18:51.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f7b1c5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4f74141570 con 0x7f4f740b4b20 2024-09-19T04:18:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f78f61640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4f74156720 0x7f4f740b5610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47012/0 (socket says 172.21.15.76:47012) 2024-09-19T04:18:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f78f61640 1 -- 172.21.15.76:0/911211574 learned_addr learned my addr 172.21.15.76:0/911211574 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f79762640 1 --2- 172.21.15.76:0/911211574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f740b4b20 0x7f4f74141030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f737fe640 1 -- 172.21.15.76:0/911211574 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1630513339 0 0) 0x7f4f7413fb30 con 0x7f4f74156720 2024-09-19T04:18:51.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f737fe640 1 -- 172.21.15.76:0/911211574 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f60003530 con 0x7f4f74156720 2024-09-19T04:18:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f737fe640 1 -- 172.21.15.76:0/911211574 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 559965736 0 0) 0x7f4f7413fe10 con 0x7f4f74140bd0 2024-09-19T04:18:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f737fe640 1 -- 172.21.15.76:0/911211574 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4f7413fb30 con 0x7f4f74140bd0 2024-09-19T04:18:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f79762640 1 -- 172.21.15.76:0/911211574 >> v1:172.21.15.116:6789/0 conn(0x7f4f74140bd0 legacy=0x7f4f740b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f79762640 1 -- 172.21.15.76:0/911211574 >> v1:172.21.15.76:6789/0 conn(0x7f4f74156720 legacy=0x7f4f740b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f79762640 1 -- 172.21.15.76:0/911211574 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f741400f0 con 0x7f4f740b4b20 2024-09-19T04:18:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.045+0000 7f4f79762640 1 --2- 172.21.15.76:0/911211574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f740b4b20 0x7f4f74141030 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0097f0 tx=0x7f4f6c01c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200b533a8edda294 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f737fe640 1 -- 172.21.15.76:0/911211574 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0ab5f0 con 0x7f4f740b4b20 2024-09-19T04:18:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f737fe640 1 -- 172.21.15.76:0/911211574 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f6c0b0070 con 0x7f4f740b4b20 2024-09-19T04:18:51.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f737fe640 1 -- 172.21.15.76:0/911211574 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c0ab990 con 0x7f4f740b4b20 2024-09-19T04:18:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/911211574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f740b4b20 msgr2=0x7f4f74141030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/911211574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f740b4b20 0x7f4f74141030 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c0097f0 tx=0x7f4f6c01c4e0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/911211574 shutdown_connections 2024-09-19T04:18:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/911211574 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f740b4b20 0x7f4f74141030 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/911211574 >> 172.21.15.76:0/911211574 conn(0x7f4f7406c0c0 msgr2=0x7f4f7406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/911211574 shutdown_connections 2024-09-19T04:18:51.049 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/911211574 wait complete. 2024-09-19T04:18:51.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 Processor -- start 2024-09-19T04:18:51.050 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- start start 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f740b4b20 0x7f4f74157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f74140bd0 0x7f4f7413c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f74156720 0x7f4f74155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7413f6e0 con 0x7f4f74140bd0 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4f74140740 con 0x7f4f740b4b20 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7413f860 con 0x7f4f74156720 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f74140bd0 0x7f4f7413c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f74140bd0 0x7f4f7413c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46618/0 (socket says 172.21.15.76:46618) 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 -- 172.21.15.76:0/294729627 learned_addr learned my addr 172.21.15.76:0/294729627 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f78f61640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f740b4b20 0x7f4f74157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f79762640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f74156720 0x7f4f74155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 -- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f74156720 msgr2=0x7f4f74155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f74156720 0x7f4f74155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 -- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f740b4b20 msgr2=0x7f4f74157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f740b4b20 0x7f4f74157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 -- 172.21.15.76:0/294729627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f74157b80 con 0x7f4f74140bd0 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f73fff640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f74140bd0 0x7f4f7413c550 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c500 tx=0x7f4f5c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f717fa640 1 -- 172.21.15.76:0/294729627 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c0052d0 con 0x7f4f74140bd0 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f717fa640 1 -- 172.21.15.76:0/294729627 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f5c005490 con 0x7f4f74140bd0 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f740c74c0 con 0x7f4f74140bd0 2024-09-19T04:18:51.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f717fa640 1 -- 172.21.15.76:0/294729627 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c012850 con 0x7f4f74140bd0 2024-09-19T04:18:51.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.049+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f74158c10 con 0x7f4f74140bd0 2024-09-19T04:18:51.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.053+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f74158740 con 0x7f4f74140bd0 2024-09-19T04:18:51.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.053+0000 7f4f717fa640 1 -- 172.21.15.76:0/294729627 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4f5c0129f0 con 0x7f4f74140bd0 2024-09-19T04:18:51.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.053+0000 7f4f717fa640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f540678f0 0x7f4f54069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.053+0000 7f4f78f61640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f540678f0 0x7f4f54069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.053+0000 7f4f717fa640 1 -- 172.21.15.76:0/294729627 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f4f5c085ac0 con 0x7f4f74140bd0 2024-09-19T04:18:51.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.053+0000 7f4f78f61640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f540678f0 0x7f4f54069db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4f64001570 tx=0x7f4f64009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.057+0000 7f4f717fa640 1 -- 172.21.15.76:0/294729627 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4f74157b80 con 0x7f4f74140bd0 2024-09-19T04:18:51.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.241+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f74069170 con 0x7f4f74140bd0 2024-09-19T04:18:51.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.241+0000 7f4f717fa640 1 -- 172.21.15.76:0/294729627 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+5228 (secure 0 0 0) 0x7f4f5c055e30 con 0x7f4f74140bd0 2024-09-19T04:18:51.243 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:51.243 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":84,"btime":"2024-09-19T04:18:49:115048+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":5794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7305,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":84,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:18:48.474617+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-19T04:18:51.244 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 84 2024-09-19T04:18:51.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f540678f0 msgr2=0x7f4f54069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f540678f0 0x7f4f54069db0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4f64001570 tx=0x7f4f64009040 comp rx=0 tx=0).stop 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f74140bd0 msgr2=0x7f4f7413c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f74140bd0 0x7f4f7413c550 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00c500 tx=0x7f4f5c00c9c0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 shutdown_connections 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4f540678f0 0x7f4f54069db0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4f74156720 0x7f4f74155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4f74140bd0 0x7f4f7413c550 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 --2- 172.21.15.76:0/294729627 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f740b4b20 0x7f4f74157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 >> 172.21.15.76:0/294729627 conn(0x7f4f7406c0c0 msgr2=0x7f4f740b5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 shutdown_connections 2024-09-19T04:18:51.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.245+0000 7f4f7b1c5640 1 -- 172.21.15.76:0/294729627 wait complete. 2024-09-19T04:18:51.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 84, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:18:45.404923+0000', 'modified': '2024-09-19T04:18:48.474617+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/2711186256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2711186256}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2711186256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-19T04:18:51.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:18:51.663 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:18:51.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb9a5b3c640 1 Processor -- start 2024-09-19T04:18:51.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb9a5b3c640 1 -- start start 2024-09-19T04:18:51.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb9a5b3c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a00b38e0 0x7fb9a00d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb9a5b3c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9a00d4750 0x7fb9a006d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb9a5b3c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9a00d2b20 con 0x7fb9a015e470 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb9a5b3c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb9a00d3bc0 con 0x7fb9a00d4750 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb9a5b3c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a00d3730 con 0x7fb9a00b38e0 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb99f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb9a015e470 0x7fb9a00d15d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47022/0 (socket says 172.21.15.76:47022) 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1765727359 learned_addr learned my addr 172.21.15.76:0/1765727359 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb99ffff640 1 --2- 172.21.15.76:0/1765727359 >> v2:172.21.15.116:3300/0 conn(0x7fb9a00d4750 0x7fb9a006d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb99effd640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a00b38e0 0x7fb9a00d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1461854207 0 0) 0x7fb9a00d2b20 con 0x7fb9a015e470 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.789+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb98c003530 con 0x7fb9a015e470 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2667644976 0 0) 0x7fb98c003530 con 0x7fb9a015e470 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a00b38e0 msgr2=0x7fb9a00d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a00b38e0 0x7fb9a00d4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 >> v2:172.21.15.116:3300/0 conn(0x7fb9a00d4750 msgr2=0x7fb9a006d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 --2- 172.21.15.76:0/1765727359 >> v2:172.21.15.116:3300/0 conn(0x7fb9a00d4750 0x7fb9a006d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00d2e00 con 0x7fb9a015e470 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb988001680 con 0x7fb9a015e470 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 >> v1:172.21.15.76:6789/0 conn(0x7fb9a015e470 legacy=0x7fb9a00d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb98c0039c0 0x7fb98c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb98c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a00b38e0 0x7fb98c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00d3bc0 con 0x7fb98c004810 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a00d3730 con 0x7fb98c0039c0 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c00ba00 con 0x7fb9a00b38e0 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99effd640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb98c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb98c0039c0 0x7fb98c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99effd640 1 -- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a00b38e0 msgr2=0x7fb98c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99effd640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a00b38e0 0x7fb98c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99effd640 1 -- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb98c0039c0 msgr2=0x7fb98c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99effd640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb98c0039c0 0x7fb98c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99effd640 1 -- 172.21.15.76:0/1765727359 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c003530 con 0x7fb98c004810 2024-09-19T04:18:51.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb98c0039c0 0x7fb98c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:51.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99effd640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb98c004c90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb990009a90 tx=0x7fb99001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa70a9ba8b6f526f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900ad070 con 0x7fb98c004810 2024-09-19T04:18:51.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9900059a0 con 0x7fb98c004810 2024-09-19T04:18:51.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99e7fc640 1 -- 172.21.15.76:0/1765727359 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990009690 con 0x7fb98c004810 2024-09-19T04:18:51.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1765727359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 msgr2=0x7fb98c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb9a5b3c640 1 --2- 172.21.15.76:0/1765727359 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb98c004c90 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fb990009a90 tx=0x7fb99001ca10 comp rx=0 tx=0).stop 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb99ffff640 1 -- 172.21.15.76:0/1765727359 reap_dead start 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1765727359 shutdown_connections 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1765727359 >> 172.21.15.76:0/1765727359 conn(0x7fb9a009d730 msgr2=0x7fb9a008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1765727359 shutdown_connections 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.793+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1765727359 wait complete. 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 Processor -- start 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 -- start start 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb9a015d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a01f8890 0x7fb9a00a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a00a1190 0x7fb9a009e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a006e290 con 0x7fb98c004810 2024-09-19T04:18:51.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a006e110 con 0x7fb9a00a1190 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a006e410 con 0x7fb9a01f8890 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb9a015d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb9a015d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46636/0 (socket says 172.21.15.76:46636) 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1490614710 learned_addr learned my addr 172.21.15.76:0/1490614710 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99effd640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a01f8890 0x7fb9a00a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a01f8890 msgr2=0x7fb9a00a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99ffff640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a00a1190 0x7fb9a009e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a01f8890 0x7fb9a00a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a00a1190 msgr2=0x7fb9a009e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a00a1190 0x7fb9a009e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1490614710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a00a1780 con 0x7fb98c004810 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99effd640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a01f8890 0x7fb9a00a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:51.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb9a015d460 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb98800d8b0 tx=0x7fb98800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb997fff640 1 -- 172.21.15.76:0/1490614710 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988019070 con 0x7fb98c004810 2024-09-19T04:18:51.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb997fff640 1 -- 172.21.15.76:0/1490614710 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9880074e0 con 0x7fb98c004810 2024-09-19T04:18:51.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a0162f00 con 0x7fb98c004810 2024-09-19T04:18:51.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb997fff640 1 -- 172.21.15.76:0/1490614710 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988004a10 con 0x7fb98c004810 2024-09-19T04:18:51.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.797+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01634c0 con 0x7fb98c004810 2024-09-19T04:18:51.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb997fff640 1 -- 172.21.15.76:0/1490614710 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb98801c070 con 0x7fb98c004810 2024-09-19T04:18:51.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb997fff640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb9800678f0 0x7fb980069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99effd640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb9800678f0 0x7fb980069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb997fff640 1 -- 172.21.15.76:0/1490614710 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb988085a70 con 0x7fb98c004810 2024-09-19T04:18:51.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99effd640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb9800678f0 0x7fb980069db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb9900ac000 tx=0x7fb9900046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 Processor -- start 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- start start 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460dadf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460d63d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d64290 0x559460d64670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559460c4bb60 con 0x559460d64290 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559460bb2c70 con 0x559460dae440 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559460b58ed0 con 0x559460dadb20 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d64290 0x559460d64670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d64290 0x559460d64670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46650/0 (socket says 172.21.15.76:46650) 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 -- 172.21.15.76:0/3705172745 learned_addr learned my addr 172.21.15.76:0/3705172745 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99effd640 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460d63d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460dadf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 -- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 msgr2=0x559460dadf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460dadf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 -- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 msgr2=0x559460d63d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460d63d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 -- 172.21.15.76:0/3705172745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559460b6ae50 con 0x559460d64290 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460dadf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb99ffff640 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d64290 0x559460d64670 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b32d0 tx=0x7fb998009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796ac505bbb0ca86 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9957fa640 1 -- 172.21.15.76:0/3705172745 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980155f0 con 0x559460d64290 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9957fa640 1 -- 172.21.15.76:0/3705172745 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99801a070 con 0x559460d64290 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9957fa640 1 -- 172.21.15.76:0/3705172745 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980159f0 con 0x559460d64290 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3705172745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d64290 msgr2=0x559460d64670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d64290 0x559460d64670 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb9a00b32d0 tx=0x7fb998009750 comp rx=0 tx=0).stop 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3705172745 shutdown_connections 2024-09-19T04:18:51.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d64290 0x559460d64670 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460d63d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3705172745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460dadf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3705172745 >> 172.21.15.76:0/3705172745 conn(0x559460d6a600 msgr2=0x559460d62630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3705172745 shutdown_connections 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.801+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3705172745 wait complete. 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- start start 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 Processor -- start 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460d68df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dae440 0x559460d6ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460d69330 0x559460d6f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559460c14b00 con 0x559460dadb20 2024-09-19T04:18:51.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559460b8d360 con 0x559460d69330 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559460b8a2d0 con 0x559460dae440 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 10 client.0 _ll_register_callbacks cb 0x559460d4f820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460d68df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460d68df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46664/0 (socket says 172.21.15.76:46664) 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1089127946 learned_addr learned my addr 172.21.15.76:0/1089127946 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99effd640 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dae440 0x559460d6ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99ffff640 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460d69330 0x559460d6f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dae440 msgr2=0x559460d6ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dae440 0x559460d6ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460d69330 msgr2=0x559460d6f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460d69330 0x559460d6f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 -- 172.21.15.76:0/1089127946 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559460b59120 con 0x559460dadb20 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460d68df0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb98805e330 tx=0x7fb9880556d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87763fe4ab09b6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 1 -- 172.21.15.76:0/1089127946 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988082060 con 0x559460dadb20 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 1 -- 172.21.15.76:0/1089127946 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb988083a40 con 0x559460dadb20 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb99801a070 con 0x559460dadb20 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 1 -- 172.21.15.76:0/1089127946 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb988072050 con 0x559460dadb20 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 10 client.7437 fetch_fsmap learned FSMap version 84 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x559460b52450 con 0x559460dadb20 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 1 -- 172.21.15.76:0/1089127946 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb99801a070 con 0x559460dadb20 2024-09-19T04:18:51.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 1 -- 172.21.15.76:0/1089127946 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7fb988004bb0 con 0x559460dadb20 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 10 client.7437 fetch_fsmap finished waiting for FSMap version 84 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 10 client.7437 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 20 client.7437 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460dbc280 0x559460dbe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 4 client.7437 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559460b4cc00 con 0x559460dbc280 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99effd640 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460dbc280 0x559460dbe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb99effd640 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460dbc280 0x559460dbe670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5108c25167f939c7 server_cookie=e86339aaf334ecf0 in_seq=0 out_seq=0 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 10 client.7437 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 1 -- 172.21.15.76:0/1089127946 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559460b4cc00 con 0x559460dbc280 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 10 client.7437 handle_command_reply: tid=0 2024-09-19T04:18:51.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 client.7437 shutdown 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 msgr2=0x559460d68df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460d68df0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fb98805e330 tx=0x7fb9880556d0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 shutdown_connections 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460dbc280 0x559460dbe670 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460d69330 0x559460d6f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dae440 0x559460d6ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb97e7fc640 0 client.7437 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/1089127946 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460d68df0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 >> 172.21.15.76:0/1089127946 conn(0x559460d64290 msgr2=0x559460d62d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 shutdown_connections 2024-09-19T04:18:51.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/1089127946 wait complete. 2024-09-19T04:18:51.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.805+0000 7fb9a8bf8000 20 client.7437 trim_cache size 0 max 16384 2024-09-19T04:18:51.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 Processor -- start 2024-09-19T04:18:51.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 -- start start 2024-09-19T04:18:51.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460d69330 0x559460daccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460dad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460dbe670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a02038a0 con 0x559460dadb20 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0202c30 con 0x559460dae440 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a0203250 con 0x559460d69330 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460dad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460dad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46666/0 (socket says 172.21.15.76:46666) 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 -- 172.21.15.76:0/3105353657 learned_addr learned my addr 172.21.15.76:0/3105353657 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460d69330 0x559460daccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99ffff640 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460dbe670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 -- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460d69330 msgr2=0x559460daccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460d69330 0x559460daccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 -- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 msgr2=0x559460dbe670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460dbe670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.825+0000 7fb99effd640 1 -- 172.21.15.76:0/3105353657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559460b39fd0 con 0x559460dadb20 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99effd640 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460dad200 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb99001cef0 tx=0x7fb990004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c67bbb8cb0c4d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb994ff9640 1 -- 172.21.15.76:0/3105353657 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900ce070 con 0x559460dadb20 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb994ff9640 1 -- 172.21.15.76:0/3105353657 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990006740 con 0x559460dadb20 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb994ff9640 1 -- 172.21.15.76:0/3105353657 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9900ad040 con 0x559460dadb20 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3105353657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 msgr2=0x559460dad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460dad200 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fb99001cef0 tx=0x7fb990004740 comp rx=0 tx=0).stop 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3105353657 shutdown_connections 2024-09-19T04:18:51.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460dbe670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460dadb20 0x559460dad200 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/3105353657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460d69330 0x559460daccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3105353657 >> 172.21.15.76:0/3105353657 conn(0x559460d64290 msgr2=0x559460d685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3105353657 shutdown_connections 2024-09-19T04:18:51.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/3105353657 wait complete. 2024-09-19T04:18:51.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- start start 2024-09-19T04:18:51.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 Processor -- start 2024-09-19T04:18:51.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d69330 0x559460dc1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460ddc960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460ddcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559460bb1f80 con 0x559460d69330 2024-09-19T04:18:51.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559460b6aca0 con 0x559460dae440 2024-09-19T04:18:51.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559460bb72a0 con 0x559460dadb20 2024-09-19T04:18:51.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 10 client.0 _ll_register_callbacks cb 0x7fb988007710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d69330 0x559460dc1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d69330 0x559460dc1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46676/0 (socket says 172.21.15.76:46676) 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 -- 172.21.15.76:0/33928225 learned_addr learned my addr 172.21.15.76:0/33928225 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 -- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 msgr2=0x559460ddc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99effd640 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460ddc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460ddc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99ffff640 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460ddcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 -- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 msgr2=0x559460ddcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460ddcea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 -- 172.21.15.76:0/33928225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559460c35750 con 0x559460d69330 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99f7fe640 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d69330 0x559460dc1590 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb988001980 tx=0x7fb988083c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad20f5e8695fcd7c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb97e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb97e7fc640 1 -- 172.21.15.76:0/33928225 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9880557c0 con 0x559460d69330 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb97e7fc640 1 -- 172.21.15.76:0/33928225 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fb98805c450 con 0x559460d69330 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb990006740 con 0x559460d69330 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb97e7fc640 1 -- 172.21.15.76:0/33928225 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb988051f20 con 0x559460d69330 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 10 client.7443 fetch_fsmap learned FSMap version 84 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=2+}) -- 0x559460b40a30 con 0x559460d69330 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb97e7fc640 1 -- 172.21.15.76:0/33928225 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb990006740 con 0x559460d69330 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb97e7fc640 1 -- 172.21.15.76:0/33928225 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 84) ==== 3502+0+0 (secure 0 0 0) 0x7fb9880519d0 con 0x559460d69330 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 10 client.7443 fetch_fsmap finished waiting for FSMap version 84 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 10 client.7443 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 20 client.7443 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:51.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460ddd3e0 0x559460e20d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 4 client.7443 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "status"}) -- 0x559460bc03a0 con 0x559460ddd3e0 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.829+0000 7fb99effd640 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460ddd3e0 0x559460e20d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb97e7fc640 10 client.7443 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb99effd640 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460ddd3e0 0x559460e20d70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1fd1b1b38112fb server_cookie=83a1df89e2304ccf in_seq=0 out_seq=0 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb97e7fc640 1 -- 172.21.15.76:0/33928225 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x559460bc03a0 con 0x559460ddd3e0 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb97e7fc640 10 client.7443 handle_command_reply: tid=0 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 client.7443 shutdown 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d69330 msgr2=0x559460dc1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d69330 0x559460dc1590 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fb988001980 tx=0x7fb988083c40 comp rx=0 tx=0).stop 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 shutdown_connections 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x559460ddd3e0 0x559460e20d70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x559460dae440 0x559460ddcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x559460dadb20 0x559460ddc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb97e7fc640 0 client.7443 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 --2- 172.21.15.76:0/33928225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x559460d69330 0x559460dc1590 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 >> 172.21.15.76:0/33928225 conn(0x559460d64290 msgr2=0x559460dabcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 shutdown_connections 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 1 -- 172.21.15.76:0/33928225 wait complete. 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a8bf8000 20 client.7443 trim_cache size 0 max 16384 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:51.840 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7311, 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 5.744815773, 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 84, 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 91, 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 10.819571722999999 2024-09-19T04:18:51.841 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:51.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb9800678f0 msgr2=0x7fb980069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb9800678f0 0x7fb980069db0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb9900ac000 tx=0x7fb9900046d0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 msgr2=0x7fb9a015d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb9a015d460 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb98800d8b0 tx=0x7fb98800dd70 comp rx=0 tx=0).stop 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 shutdown_connections 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb9800678f0 0x7fb980069db0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a00a1190 0x7fb9a009e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb9a01f8890 0x7fb9a00a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 --2- 172.21.15.76:0/1490614710 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb98c004810 0x7fb9a015d460 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 >> 172.21.15.76:0/1490614710 conn(0x7fb9a009d730 msgr2=0x7fb9a015e980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 shutdown_connections 2024-09-19T04:18:51.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.833+0000 7fb9a5b3c640 1 -- 172.21.15.76:0/1490614710 wait complete. 2024-09-19T04:18:51.849 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:18:51.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.957+0000 7f20a4165640 1 Processor -- start 2024-09-19T04:18:51.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.957+0000 7f20a4165640 1 -- start start 2024-09-19T04:18:51.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a4165640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 0x7f209c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a4165640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f209c0cf450 0x7f209c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a4165640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f209c0d0300 con 0x7f209c161a60 2024-09-19T04:18:51.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a4165640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f209c0b0f60 con 0x7f209c0cf450 2024-09-19T04:18:51.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a4165640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f209c0cfd40 con 0x7f209c0d3060 2024-09-19T04:18:51.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1f01640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f209c161a60 0x7f209c0d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47028/0 (socket says 172.21.15.76:47028) 2024-09-19T04:18:51.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1f01640 1 -- 172.21.15.76:0/3554967242 learned_addr learned my addr 172.21.15.76:0/3554967242 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:51.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 0x7f209c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a2702640 1 --2- 172.21.15.76:0/3554967242 >> v2:172.21.15.116:3300/0 conn(0x7f209c0cf450 0x7f209c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1214580620 0 0) 0x7f2098001120 con 0x7f209c161a60 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2088003530 con 0x7f209c161a60 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 711462541 0 0) 0x7f2088003530 con 0x7f209c161a60 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 msgr2=0x7f209c0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 0x7f209c0cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 >> v2:172.21.15.116:3300/0 conn(0x7f209c0cf450 msgr2=0x7f209c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 --2- 172.21.15.76:0/3554967242 >> v2:172.21.15.116:3300/0 conn(0x7f209c0cf450 0x7f209c06d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c0d0020 con 0x7f209c161a60 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2098001850 con 0x7f209c161a60 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 >> v1:172.21.15.76:6789/0 conn(0x7f209c161a60 legacy=0x7f209c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 0x7f20880039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f2088004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2088004d50 0x7f20880051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f209c0b0f60 con 0x7f2088003f40 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f209c0cfd40 con 0x7f2088004d50 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f208800ba20 con 0x7f209c0d3060 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f2088004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a2702640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2088004d50 0x7f20880051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 -- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 msgr2=0x7f20880039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1f01640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 0x7f20880039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 0x7f20880039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 -- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2088004d50 msgr2=0x7f20880051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2088004d50 0x7f20880051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 -- 172.21.15.76:0/3554967242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2088003530 con 0x7f2088003f40 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1f01640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c0d3060 0x7f20880039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:51.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a2702640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2088004d50 0x7f20880051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:51.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a1700640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f2088004810 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f208c009b10 tx=0x7f208c01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6aeb9f0d7e3d9a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.961+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c0b2070 con 0x7f2088003f40 2024-09-19T04:18:51.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f208c0a5610 con 0x7f2088003f40 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a0eff640 1 -- 172.21.15.76:0/3554967242 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c0096b0 con 0x7f2088003f40 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- 172.21.15.76:0/3554967242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 msgr2=0x7f2088004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 --2- 172.21.15.76:0/3554967242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f2088004810 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f208c009b10 tx=0x7f208c01ca40 comp rx=0 tx=0).stop 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a2702640 1 -- 172.21.15.76:0/3554967242 reap_dead start 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- 172.21.15.76:0/3554967242 shutdown_connections 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- 172.21.15.76:0/3554967242 >> 172.21.15.76:0/3554967242 conn(0x7f209c08bf90 msgr2=0x7f209c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- 172.21.15.76:0/3554967242 shutdown_connections 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- 172.21.15.76:0/3554967242 wait complete. 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 Processor -- start 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- start start 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f209c0b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f209c162070 0x7f209c159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c163660 0x7f209c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f209c168f10 con 0x7f2088003f40 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f209c06e610 con 0x7f209c162070 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f209c169090 con 0x7f209c163660 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f209c0b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f209c0b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:46702/0 (socket says 172.21.15.76:46702) 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 -- 172.21.15.76:0/1563680421 learned_addr learned my addr 172.21.15.76:0/1563680421 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a2702640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c163660 0x7f209c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1700640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f209c162070 0x7f209c159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 -- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c163660 msgr2=0x7f209c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c163660 0x7f209c1608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 -- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f209c162070 msgr2=0x7f209c159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f209c162070 0x7f209c159830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 -- 172.21.15.76:0/1563680421 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c1625c0 con 0x7f2088003f40 2024-09-19T04:18:51.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a1f01640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f209c0b2f40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f209800d860 tx=0x7f209800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20927fc640 1 -- 172.21.15.76:0/1563680421 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098019070 con 0x7f2088003f40 2024-09-19T04:18:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20927fc640 1 -- 172.21.15.76:0/1563680421 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20980074e0 con 0x7f2088003f40 2024-09-19T04:18:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c1628a0 con 0x7f2088003f40 2024-09-19T04:18:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20927fc640 1 -- 172.21.15.76:0/1563680421 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2098004a10 con 0x7f2088003f40 2024-09-19T04:18:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.965+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c162e60 con 0x7f2088003f40 2024-09-19T04:18:51.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.969+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c06d6b0 con 0x7f2088003f40 2024-09-19T04:18:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.969+0000 7f20927fc640 1 -- 172.21.15.76:0/1563680421 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f209801c070 con 0x7f2088003f40 2024-09-19T04:18:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.969+0000 7f20927fc640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20800678c0 0x7f2080069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.969+0000 7f20927fc640 1 -- 172.21.15.76:0/1563680421 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f2098085940 con 0x7f2088003f40 2024-09-19T04:18:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.969+0000 7f20a1700640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20800678c0 0x7f2080069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:51.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.969+0000 7f20a1700640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20800678c0 0x7f2080069d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f208c0af000 tx=0x7f208c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:51.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:51.973+0000 7f20927fc640 1 -- 172.21.15.76:0/1563680421 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f209c0d0300 con 0x7f2088003f40 2024-09-19T04:18:52.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.157+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f209c06bcc0 con 0x7f2088003f40 2024-09-19T04:18:52.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.157+0000 7f20927fc640 1 -- 172.21.15.76:0/1563680421 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v84) ==== 53+0+73 (secure 0 0 0) 0x7f2098056250 con 0x7f2088003f40 2024-09-19T04:18:52.158 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20800678c0 msgr2=0x7f2080069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20800678c0 0x7f2080069d80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f208c0af000 tx=0x7f208c008040 comp rx=0 tx=0).stop 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 msgr2=0x7f209c0b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f209c0b2f40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f209800d860 tx=0x7f209800dd20 comp rx=0 tx=0).stop 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 shutdown_connections 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f20800678c0 0x7f2080069d80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f209c163660 0x7f209c1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f209c162070 0x7f209c159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:52.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 --2- 172.21.15.76:0/1563680421 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2088003f40 0x7f209c0b2f40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:52.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 >> 172.21.15.76:0/1563680421 conn(0x7f209c08bf90 msgr2=0x7f209c0b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:52.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 shutdown_connections 2024-09-19T04:18:52.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52.161+0000 7f20a4165640 1 -- 172.21.15.76:0/1563680421 wait complete. 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:18:52.176 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:18:52.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:52.177 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:18:52.183 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42189sec preferred_lft 42189sec 2024-09-19T04:18:52.184 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:52.185 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:18:52.186 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.186 INFO:teuthology.orchestra.run.smithi076.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:18:52.186 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:52.186 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T04:18:52.186 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:18:52.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:52.186 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:18:52.234 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:18:52.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:52.235 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:18:52.284 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:18:52.284 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:18:52.284 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:18:52.284 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:18:52.284 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:18:52.284 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:18:52.284 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:18:52.285 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:18:52.285 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:18:52.285 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:18:52.285 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:18:52.285 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:18:52.285 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:18:52.285 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:18:52.285 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:18:52.285 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:18:52.286 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:18:52.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:18:52.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:18:52.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:52.404 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:18:52.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T04:18:52.452 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:18:52.453 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:18:52.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:52.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:18:52.507 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T04:18:52.507 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:18:52.507 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:52.552 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:18:52.553 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:18:52.553 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:52.600 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:18:52.658 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:18:52.705 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:18:52.705 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:18:52.706 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:18:52.707 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:52.708 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:52.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:52.709 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:18:52.757 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:18:52.757 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T04:18:52.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:18:52.815 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:18:52.861 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:18:52.862 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:52.863 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:52.864 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:52.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:52.865 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:18:52.883 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-09-19T04:18:52.881+0000 7f364191b580 -1 init, newargv = 0x55d0abaf8f80 newargc=15 2024-09-19T04:18:52.883 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[46521]: starting ceph client 2024-09-19T04:18:52.888 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[46521]: starting fuse 2024-09-19T04:18:53.979 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:18:53.982 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:18:53.988 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:18:54.074 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:18:54.121 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:18:54.122 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:18:54.123 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:18:54.124 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:54.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:54.125 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:18:54.173 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:18:54.174 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:18:54.174 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:18:54.175 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:18:54.176 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:18:54.176 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:18:54.176 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:18:54.176 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-09-19T04:18:54.176 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:18:54.298 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.46521.asok 2024-09-19T04:18:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:18:54.310 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46521.asok 2024-09-19T04:18:54.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:54.310 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46521.asok status 2024-09-19T04:18:54.464 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "46521", 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:18:54.465 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7455, 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 7455 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2466961382 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:54.466 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2466961382 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.7455 192.168.0.1:0/2466961382", 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/2466961382", 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 84, 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 91, 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:18:54.467 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:18:54.468 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:54.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:54.478 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:18:54.478 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:54.484 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:18:54.484 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:54.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:54.484 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:18:54.541 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-19T04:18:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde2764640 1 Processor -- start 2024-09-19T04:18:54.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde2764640 1 -- start start 2024-09-19T04:18:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde2764640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efddc15bec0 0x7efddc0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde2764640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efddc0b3290 con 0x7efddc0af470 2024-09-19T04:18:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde2764640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efddc0b3570 con 0x7efddc15c2a0 2024-09-19T04:18:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde2764640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efddc0b3850 con 0x7efddc15bec0 2024-09-19T04:18:54.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde0d01640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efddc0af470 0x7efddc06e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47244/0 (socket says 172.21.15.76:47244) 2024-09-19T04:18:54.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde0d01640 1 -- 172.21.15.76:0/2781016563 learned_addr learned my addr 172.21.15.76:0/2781016563 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:54.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 32236308 0 0) 0x7efddc0b3290 con 0x7efddc0af470 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddbfff640 1 --2- 172.21.15.76:0/2781016563 >> v2:172.21.15.116:3300/0 conn(0x7efddc15bec0 0x7efddc0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efdc8003530 con 0x7efddc0af470 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2879161928 0 0) 0x7efdc8003530 con 0x7efddc0af470 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 >> v2:172.21.15.116:3300/0 conn(0x7efddc15bec0 msgr2=0x7efddc0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 --2- 172.21.15.76:0/2781016563 >> v2:172.21.15.116:3300/0 conn(0x7efddc15bec0 0x7efddc0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 >> v1:172.21.15.116:6789/0 conn(0x7efddc15c2a0 legacy=0x7efddc0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc0b3b30 con 0x7efddc0af470 2024-09-19T04:18:54.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7efdd4001680 con 0x7efddc0af470 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 >> v1:172.21.15.76:6789/0 conn(0x7efddc0af470 legacy=0x7efddc06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efdc8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdc8004810 0x7efdc8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efdc80051d0 0x7efdc800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0b3850 con 0x7efdc8003a10 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efdc800c0d0 con 0x7efdc8004810 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efdc800c2d0 con 0x7efdc80051d0 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddbfff640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efdc8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddb7fe640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdc8004810 0x7efdc8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde0d01640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efdc80051d0 0x7efdc800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddbfff640 1 -- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efdc80051d0 msgr2=0x7efdc800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddbfff640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efdc80051d0 0x7efdc800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddbfff640 1 -- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdc8004810 msgr2=0x7efdc8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddbfff640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efdc8004810 0x7efdc8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efddbfff640 1 -- 172.21.15.76:0/2781016563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdc8003530 con 0x7efdc8003a10 2024-09-19T04:18:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde0d01640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efdc80051d0 0x7efdc800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:54.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.713+0000 7efde0d01640 1 -- 172.21.15.76:0/2781016563 reap_dead start 2024-09-19T04:18:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efddbfff640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efdc8003e30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009fc0 tx=0x7efdcc01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc066efc24b45a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0b2070 con 0x7efdc8003a10 2024-09-19T04:18:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdcc0a5b50 con 0x7efdc8003a10 2024-09-19T04:18:54.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efddaffd640 1 -- 172.21.15.76:0/2781016563 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc009c10 con 0x7efdc8003a10 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 -- 172.21.15.76:0/2781016563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 msgr2=0x7efdc8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efdc8003e30 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7efdcc009fc0 tx=0x7efdcc01cc50 comp rx=0 tx=0).stop 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 -- 172.21.15.76:0/2781016563 shutdown_connections 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 --2- 172.21.15.76:0/2781016563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efdc8003e30 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 -- 172.21.15.76:0/2781016563 >> 172.21.15.76:0/2781016563 conn(0x7efddc0b1f90 msgr2=0x7efddc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 -- 172.21.15.76:0/2781016563 shutdown_connections 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 -- 172.21.15.76:0/2781016563 wait complete. 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 Processor -- start 2024-09-19T04:18:54.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.717+0000 7efde2764640 1 -- start start 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efddc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efddc15c6e0 0x7efddc152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efddc15acc0 0x7efddc14c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0af2b0 con 0x7efdc8003a10 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efddc0af130 con 0x7efddc15c6e0 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efddc0ad610 con 0x7efddc15acc0 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efddc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efddc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48130/0 (socket says 172.21.15.76:48130) 2024-09-19T04:18:54.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 -- 172.21.15.76:0/3450053283 learned_addr learned my addr 172.21.15.76:0/3450053283 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddb7fe640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efddc15c6e0 0x7efddc152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde0d01640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efddc15acc0 0x7efddc14c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 -- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efddc15acc0 msgr2=0x7efddc14c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efddc15acc0 0x7efddc14c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 -- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efddc15c6e0 msgr2=0x7efddc152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efddc15c6e0 0x7efddc152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 -- 172.21.15.76:0/3450053283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efddc151b20 con 0x7efdc8003a10 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde0d01640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efddc15acc0 0x7efddc14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddbfff640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efddc15d5d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7efdcc01c6c0 tx=0x7efdcc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efdd8ff9640 1 -- 172.21.15.76:0/3450053283 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0b2040 con 0x7efdc8003a10 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efdd8ff9640 1 -- 172.21.15.76:0/3450053283 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efdcc0a5950 con 0x7efdc8003a10 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efdd8ff9640 1 -- 172.21.15.76:0/3450053283 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdcc0adc60 con 0x7efdc8003a10 2024-09-19T04:18:54.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efddc151e00 con 0x7efdc8003a10 2024-09-19T04:18:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efddc1523c0 con 0x7efdc8003a10 2024-09-19T04:18:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efddc160470 con 0x7efdc8003a10 2024-09-19T04:18:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efdd8ff9640 1 -- 172.21.15.76:0/3450053283 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efdcc0bf030 con 0x7efdc8003a10 2024-09-19T04:18:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efdd8ff9640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efdbc0678c0 0x7efdbc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:54.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddb7fe640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efdbc0678c0 0x7efdbc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efdd8ff9640 1 -- 172.21.15.76:0/3450053283 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7efdcc120070 con 0x7efdc8003a10 2024-09-19T04:18:54.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.721+0000 7efddb7fe640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efdbc0678c0 0x7efdbc069d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efddc1534a0 tx=0x7efdc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:54.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.725+0000 7efdd8ff9640 1 -- 172.21.15.76:0/3450053283 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7efdcc0c6060 con 0x7efdc8003a10 2024-09-19T04:18:54.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:54.925+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7efddc06fcd0 con 0x7efdc8003a10 2024-09-19T04:18:55.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efdd8ff9640 1 -- 172.21.15.76:0/3450053283 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v85) ==== 105+0+0 (secure 0 0 0) 0x7efdcc016990 con 0x7efdc8003a10 2024-09-19T04:18:55.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efdbc0678c0 msgr2=0x7efdbc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efdbc0678c0 0x7efdbc069d80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efddc1534a0 tx=0x7efdc4009040 comp rx=0 tx=0).stop 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 msgr2=0x7efddc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efddc15d5d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7efdcc01c6c0 tx=0x7efdcc009750 comp rx=0 tx=0).stop 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 shutdown_connections 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efdbc0678c0 0x7efdbc069d80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efddc15acc0 0x7efddc14c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.153+0000 7efde2764640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efddc15c6e0 0x7efddc152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.157+0000 7efde2764640 1 --2- 172.21.15.76:0/3450053283 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efdc8003a10 0x7efddc15d5d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.157+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 >> 172.21.15.76:0/3450053283 conn(0x7efddc0b1f90 msgr2=0x7efddc074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.157+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 shutdown_connections 2024-09-19T04:18:55.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.157+0000 7efde2764640 1 -- 172.21.15.76:0/3450053283 wait complete. 2024-09-19T04:18:55.185 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:55.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ef1c640 1 Processor -- start 2024-09-19T04:18:55.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ef1c640 1 -- start start 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ef1c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f571806ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ef1c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f57180b3aa0 0x7f571806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ef1c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57180cd520 con 0x7f571815c0c0 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ef1c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f57180cd800 con 0x7f57180b3aa0 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ef1c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f57180cdae0 con 0x7f57180b5880 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ccb8640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f571815c0c0 0x7f571815c4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47250/0 (socket says 172.21.15.76:47250) 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571ccb8640 1 -- 172.21.15.76:0/1909028133 learned_addr learned my addr 172.21.15.76:0/1909028133 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:55.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571d4b9640 1 --2- 172.21.15.76:0/1909028133 >> v2:172.21.15.76:3300/0 conn(0x7f57180b3aa0 0x7f571806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f5717fff640 1 --2- 172.21.15.76:0/1909028133 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f571806ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571d4b9640 1 -- 172.21.15.76:0/1909028133 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 msgr2=0x7f571806ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:55.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571d4b9640 1 --2- 172.21.15.76:0/1909028133 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f571806ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571d4b9640 1 -- 172.21.15.76:0/1909028133 >> v1:172.21.15.76:6789/0 conn(0x7f571815c0c0 legacy=0x7f571815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f571d4b9640 1 -- 172.21.15.76:0/1909028133 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180cddc0 con 0x7f57180b3aa0 2024-09-19T04:18:55.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.301+0000 7f5717fff640 1 --2- 172.21.15.76:0/1909028133 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f571806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571d4b9640 1 --2- 172.21.15.76:0/1909028133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571806d210 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5710009dc0 tx=0x7f571001c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea168c3fb44ccb1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f57177fe640 1 -- 172.21.15.76:0/1909028133 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100b2070 con 0x7f57180b3aa0 2024-09-19T04:18:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f57177fe640 1 -- 172.21.15.76:0/1909028133 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57100a5810 con 0x7f57180b3aa0 2024-09-19T04:18:55.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f57177fe640 1 -- 172.21.15.76:0/1909028133 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100a5be0 con 0x7f57180b3aa0 2024-09-19T04:18:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- 172.21.15.76:0/1909028133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 msgr2=0x7f571806d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 --2- 172.21.15.76:0/1909028133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571806d210 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5710009dc0 tx=0x7f571001c7b0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- 172.21.15.76:0/1909028133 shutdown_connections 2024-09-19T04:18:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 --2- 172.21.15.76:0/1909028133 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571806d210 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 --2- 172.21.15.76:0/1909028133 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f571806ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- 172.21.15.76:0/1909028133 >> 172.21.15.76:0/1909028133 conn(0x7f5718072670 msgr2=0x7f5718072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- 172.21.15.76:0/1909028133 shutdown_connections 2024-09-19T04:18:55.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- 172.21.15.76:0/1909028133 wait complete. 2024-09-19T04:18:55.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 Processor -- start 2024-09-19T04:18:55.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- start start 2024-09-19T04:18:55.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f5718158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f571815c0c0 0x7f5718153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f57180cef50 con 0x7f57180b3aa0 2024-09-19T04:18:55.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f57180cedd0 con 0x7f571815c0c0 2024-09-19T04:18:55.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ef1c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f57180cf0d0 con 0x7f57180b5880 2024-09-19T04:18:55.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ccb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ccb8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571814f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48148/0 (socket says 172.21.15.76:48148) 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.305+0000 7f571ccb8640 1 -- 172.21.15.76:0/532644188 learned_addr learned my addr 172.21.15.76:0/532644188 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f5717fff640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f5718158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571d4b9640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f571815c0c0 0x7f5718153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ccb8640 1 -- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 msgr2=0x7f5718158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ccb8640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f5718158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ccb8640 1 -- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f571815c0c0 msgr2=0x7f5718153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ccb8640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f571815c0c0 0x7f5718153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ccb8640 1 -- 172.21.15.76:0/532644188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57180b51e0 con 0x7f57180b3aa0 2024-09-19T04:18:55.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f5717fff640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f5718158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:55.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571d4b9640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f571815c0c0 0x7f5718153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:55.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ccb8640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571814f220 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f570800c780 tx=0x7f570800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:55.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f57157fa640 1 -- 172.21.15.76:0/532644188 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5708018070 con 0x7f57180b3aa0 2024-09-19T04:18:55.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f57157fa640 1 -- 172.21.15.76:0/532644188 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57080072d0 con 0x7f57180b3aa0 2024-09-19T04:18:55.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5718157b30 con 0x7f57180b3aa0 2024-09-19T04:18:55.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f57157fa640 1 -- 172.21.15.76:0/532644188 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57080047f0 con 0x7f57180b3aa0 2024-09-19T04:18:55.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57181580f0 con 0x7f57180b3aa0 2024-09-19T04:18:55.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5718158800 con 0x7f57180b3aa0 2024-09-19T04:18:55.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f57157fa640 1 -- 172.21.15.76:0/532644188 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f570801b070 con 0x7f57180b3aa0 2024-09-19T04:18:55.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f57157fa640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57040678c0 0x7f5704069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:55.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f57157fa640 1 -- 172.21.15.76:0/532644188 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f57080855e0 con 0x7f57180b3aa0 2024-09-19T04:18:55.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f5717fff640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57040678c0 0x7f5704069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:55.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.309+0000 7f5717fff640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57040678c0 0x7f5704069d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f571814bae0 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:55.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.313+0000 7f57157fa640 1 -- 172.21.15.76:0/532644188 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5708056000 con 0x7f57180b3aa0 2024-09-19T04:18:55.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.505+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f571815c4a0 con 0x7f57180b3aa0 2024-09-19T04:18:55.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.505+0000 7f57157fa640 1 -- 172.21.15.76:0/532644188 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) ==== 94+0+5244 (secure 0 0 0) 0x7f5708055900 con 0x7f57180b3aa0 2024-09-19T04:18:55.507 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:55.507 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":86,"btime":"2024-09-19T04:18:55:160092+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":5794,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:18:55.160055+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":7311,"mds_1":7305},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311]},"id":6}]} 2024-09-19T04:18:55.508 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 86 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57040678c0 msgr2=0x7f5704069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57040678c0 0x7f5704069d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f571814bae0 tx=0x7f570c008040 comp rx=0 tx=0).stop 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 msgr2=0x7f571814f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571814f220 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f570800c780 tx=0x7f570800cc40 comp rx=0 tx=0).stop 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 shutdown_connections 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f57040678c0 0x7f5704069d80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f571815c0c0 0x7f5718153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f57180b5880 0x7f5718158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 --2- 172.21.15.76:0/532644188 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f57180b3aa0 0x7f571814f220 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:55.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 >> 172.21.15.76:0/532644188 conn(0x7f5718072670 msgr2=0x7f571814bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:55.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 shutdown_connections 2024-09-19T04:18:55.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:55.509+0000 7f571ef1c640 1 -- 172.21.15.76:0/532644188 wait complete. 2024-09-19T04:18:55.928 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:18:45.404923+0000', 'modified': '2024-09-19T04:18:55.160055+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 7311, 'mds_1': 7305}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7305': {'gid': 7305, 'name': 'c', 'rank': 1, 'incarnation': 86, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.76:6835/987394316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 987394316}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 987394316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7311': {'gid': 7311, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/2711186256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2711186256}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2711186256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [7311]} 2024-09-19T04:18:55.928 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7305:up:creating 2024-09-19T04:18:56.928 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3edcb6d640 1 Processor -- start 2024-09-19T04:18:57.062 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3edcb6d640 1 -- start start 2024-09-19T04:18:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3edcb6d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 0x7f3ed80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3edcb6d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ed815c2a0 0x7f3ed80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3edcb6d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ed80b3290 con 0x7f3ed80af2a0 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3edcb6d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3ed80b3570 con 0x7f3ed815c2a0 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3edcb6d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed80b3850 con 0x7f3ed815bec0 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3ed6d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3ed80af2a0 0x7f3ed815fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47266/0 (socket says 172.21.15.76:47266) 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3ed6d9d640 1 -- 172.21.15.76:0/2430152432 learned_addr learned my addr 172.21.15.76:0/2430152432 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 0x7f3ed80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3ed5d9b640 1 --2- 172.21.15.76:0/2430152432 >> v2:172.21.15.116:3300/0 conn(0x7f3ed815c2a0 0x7f3ed80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 397631104 0 0) 0x7f3ed80b3290 con 0x7f3ed80af2a0 2024-09-19T04:18:57.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.061+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3ec0003530 con 0x7f3ed80af2a0 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1120601051 0 0) 0x7f3ec0003530 con 0x7f3ed80af2a0 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 msgr2=0x7f3ed80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 0x7f3ed80ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 >> v2:172.21.15.116:3300/0 conn(0x7f3ed815c2a0 msgr2=0x7f3ed80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 --2- 172.21.15.76:0/2430152432 >> v2:172.21.15.116:3300/0 conn(0x7f3ed815c2a0 0x7f3ed80aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed80b3b30 con 0x7f3ed80af2a0 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f3ed0001a00 con 0x7f3ed80af2a0 2024-09-19T04:18:57.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 >> v1:172.21.15.76:6789/0 conn(0x7f3ed80af2a0 legacy=0x7f3ed815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ec0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ec0004810 0x7f3ec0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 0x7f3ec00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed80b3570 con 0x7f3ec00039c0 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed80b3850 con 0x7f3ec0004810 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3ec000b9d0 con 0x7f3ed815bec0 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ec0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed6d9d640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 0x7f3ec00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed5d9b640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ec0004810 0x7f3ec0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed659c640 1 -- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 msgr2=0x7f3ec00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed815bec0 0x7f3ec00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed659c640 1 -- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ec0004810 msgr2=0x7f3ec0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ec0004810 0x7f3ec0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed659c640 1 -- 172.21.15.76:0/2430152432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec0003530 con 0x7f3ec00039c0 2024-09-19T04:18:57.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed5d9b640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ec0004810 0x7f3ec0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ec0003de0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009ae0 tx=0x7f3ecc01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b43f955aa3d727b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0b2070 con 0x7f3ec00039c0 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0a5530 con 0x7f3ec00039c0 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed559a640 1 -- 172.21.15.76:0/2430152432 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0096b0 con 0x7f3ec00039c0 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2430152432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 msgr2=0x7f3ec0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3edcb6d640 1 --2- 172.21.15.76:0/2430152432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ec0003de0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009ae0 tx=0x7f3ecc01ca40 comp rx=0 tx=0).stop 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3ed6d9d640 1 -- 172.21.15.76:0/2430152432 reap_dead start 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2430152432 shutdown_connections 2024-09-19T04:18:57.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2430152432 >> 172.21.15.76:0/2430152432 conn(0x7f3ed80b1f90 msgr2=0x7f3ed80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2430152432 shutdown_connections 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.065+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2430152432 wait complete. 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 Processor -- start 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 -- start start 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ed81e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed81ea4c0 0x7f3ed81f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ed81f7110 0x7f3ed8156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed80ad610 con 0x7f3ec00039c0 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3ed80af6b0 con 0x7f3ed81f7110 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3ed80ad790 con 0x7f3ed81ea4c0 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ed81e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ed81e9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48168/0 (socket says 172.21.15.76:48168) 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 -- 172.21.15.76:0/2212603402 learned_addr learned my addr 172.21.15.76:0/2212603402 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed5d9b640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed81ea4c0 0x7f3ed81f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed6d9d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ed81f7110 0x7f3ed8156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 -- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed81ea4c0 msgr2=0x7f3ed81f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed81ea4c0 0x7f3ed81f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 -- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ed81f7110 msgr2=0x7f3ed8156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ed81f7110 0x7f3ed8156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 -- 172.21.15.76:0/2212603402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ed815ba50 con 0x7f3ec00039c0 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed5d9b640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed81ea4c0 0x7f3ed81f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed6d9d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ed81f7110 0x7f3ed8156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ed659c640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ed81e9f80 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009ae0 tx=0x7f3ec0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ec6ffd640 1 -- 172.21.15.76:0/2212603402 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0bc030 con 0x7f3ec00039c0 2024-09-19T04:18:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ec6ffd640 1 -- 172.21.15.76:0/2212603402 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ecc00f000 con 0x7f3ec00039c0 2024-09-19T04:18:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ed8155f40 con 0x7f3ec00039c0 2024-09-19T04:18:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ec6ffd640 1 -- 172.21.15.76:0/2212603402 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc0b2040 con 0x7f3ec00039c0 2024-09-19T04:18:57.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ed8156500 con 0x7f3ec00039c0 2024-09-19T04:18:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ed80aed60 con 0x7f3ec00039c0 2024-09-19T04:18:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.069+0000 7f3ec6ffd640 1 -- 172.21.15.76:0/2212603402 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3ecc0bf070 con 0x7f3ec00039c0 2024-09-19T04:18:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.073+0000 7f3ec6ffd640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3eb80678c0 0x7f3eb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.073+0000 7f3ec6ffd640 1 -- 172.21.15.76:0/2212603402 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f3ecc11f9a0 con 0x7f3ec00039c0 2024-09-19T04:18:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.073+0000 7f3ed5d9b640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3eb80678c0 0x7f3eb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.073+0000 7f3ed5d9b640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3eb80678c0 0x7f3eb8069d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80ae510 tx=0x7f3ec8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.077+0000 7f3ec6ffd640 1 -- 172.21.15.76:0/2212603402 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3ecc016780 con 0x7f3ec00039c0 2024-09-19T04:18:57.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.269+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ed806fcd0 con 0x7f3ec00039c0 2024-09-19T04:18:57.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.269+0000 7f3ec6ffd640 1 -- 172.21.15.76:0/2212603402 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5266 (secure 0 0 0) 0x7f3ecc0f0500 con 0x7f3ec00039c0 2024-09-19T04:18:57.272 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:57.272 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":89,"btime":"2024-09-19T04:18:57:170853+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:18:57.170840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[5794,7305,7311]},"id":6}]} 2024-09-19T04:18:57.272 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 89 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3eb80678c0 msgr2=0x7f3eb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3eb80678c0 0x7f3eb8069d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3ed80ae510 tx=0x7f3ec8008040 comp rx=0 tx=0).stop 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 msgr2=0x7f3ed81e9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ed81e9f80 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009ae0 tx=0x7f3ec0005840 comp rx=0 tx=0).stop 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 shutdown_connections 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3eb80678c0 0x7f3eb8069d80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ed81f7110 0x7f3ed8156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.274 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ed81ea4c0 0x7f3ed81f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 --2- 172.21.15.76:0/2212603402 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ec00039c0 0x7f3ed81e9f80 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 >> 172.21.15.76:0/2212603402 conn(0x7f3ed80b1f90 msgr2=0x7f3ed815b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 shutdown_connections 2024-09-19T04:18:57.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.273+0000 7f3edcb6d640 1 -- 172.21.15.76:0/2212603402 wait complete. 2024-09-19T04:18:57.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:18:45.404923+0000', 'modified': '2024-09-19T04:18:57.170840+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 7311, 'mds_1': 7305, 'mds_2': 5794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5794': {'gid': 5794, 'name': 'b', 'rank': 2, 'incarnation': 88, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6833/2189465635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 2189465635}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 2189465635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7305': {'gid': 7305, 'name': 'c', 'rank': 1, 'incarnation': 86, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6835/987394316', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 987394316}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 987394316}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_7311': {'gid': 7311, 'name': 'a', 'rank': 0, 'incarnation': 80, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/2711186256', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 2711186256}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 2711186256}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7311, 'qdb_cluster': [5794, 7305, 7311]} 2024-09-19T04:18:57.692 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:18:57.692 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-19T04:18:57.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.821+0000 7f514a0bf640 1 Processor -- start 2024-09-19T04:18:57.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.821+0000 7f514a0bf640 1 -- start start 2024-09-19T04:18:57.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.821+0000 7f514a0bf640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f51440d3060 0x7f51440cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.821+0000 7f514a0bf640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f51440cf450 0x7f514406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.821+0000 7f514a0bf640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51440d0300 con 0x7f5144161a60 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.821+0000 7f514a0bf640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f51440b0f60 con 0x7f51440cf450 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.821+0000 7f514a0bf640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f51440cfd40 con 0x7f51440d3060 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f51440cf450 0x7f514406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f51440cf450 0x7f514406d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48172/0 (socket says 172.21.15.76:48172) 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 -- 172.21.15.76:0/1273677486 learned_addr learned my addr 172.21.15.76:0/1273677486 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5142ffd640 1 --2- 172.21.15.76:0/1273677486 >> v2:172.21.15.116:3300/0 conn(0x7f51440d3060 0x7f51440cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 -- 172.21.15.76:0/1273677486 >> v2:172.21.15.116:3300/0 conn(0x7f51440d3060 msgr2=0x7f51440cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 --2- 172.21.15.76:0/1273677486 >> v2:172.21.15.116:3300/0 conn(0x7f51440d3060 0x7f51440cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 -- 172.21.15.76:0/1273677486 >> v1:172.21.15.116:6789/0 conn(0x7f5144161a60 legacy=0x7f51440d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 -- 172.21.15.76:0/1273677486 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51440d0020 con 0x7f51440cf450 2024-09-19T04:18:57.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f5143fff640 1 --2- 172.21.15.76:0/1273677486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51440cf450 0x7f514406d6b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f514408d850 tx=0x7f513c01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbd93c17b6b175e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f51427fc640 1 -- 172.21.15.76:0/1273677486 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c0b0070 con 0x7f51440cf450 2024-09-19T04:18:57.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f51427fc640 1 -- 172.21.15.76:0/1273677486 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c0ab600 con 0x7f51440cf450 2024-09-19T04:18:57.827 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f51427fc640 1 -- 172.21.15.76:0/1273677486 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c0ab9d0 con 0x7f51440cf450 2024-09-19T04:18:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 -- 172.21.15.76:0/1273677486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51440cf450 msgr2=0x7f514406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 --2- 172.21.15.76:0/1273677486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51440cf450 0x7f514406d6b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f514408d850 tx=0x7f513c01c240 comp rx=0 tx=0).stop 2024-09-19T04:18:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 -- 172.21.15.76:0/1273677486 shutdown_connections 2024-09-19T04:18:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 --2- 172.21.15.76:0/1273677486 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f51440cf450 0x7f514406d6b0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 --2- 172.21.15.76:0/1273677486 >> v2:172.21.15.116:3300/0 conn(0x7f51440d3060 0x7f51440cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 -- 172.21.15.76:0/1273677486 >> 172.21.15.76:0/1273677486 conn(0x7f514408bf90 msgr2=0x7f514409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 -- 172.21.15.76:0/1273677486 shutdown_connections 2024-09-19T04:18:57.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.825+0000 7f514a0bf640 1 -- 172.21.15.76:0/1273677486 wait complete. 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 Processor -- start 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 -- start start 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51440cf450 0x7f51440ce020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f51440d3060 0x7f51440b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5144161a60 0x7f51441515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f514406e400 con 0x7f5144161a60 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f51440cfad0 con 0x7f51440cf450 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f514406e580 con 0x7f51440d3060 2024-09-19T04:18:57.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5144161a60 0x7f51441515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5144161a60 0x7f51441515b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48182/0 (socket says 172.21.15.76:48182) 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 -- 172.21.15.76:0/802500505 learned_addr learned my addr 172.21.15.76:0/802500505 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5142ffd640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f51440d3060 0x7f51440b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f51437fe640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51440cf450 0x7f51440ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 -- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f51440d3060 msgr2=0x7f51440b1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f51440d3060 0x7f51440b1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 -- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51440cf450 msgr2=0x7f51440ce020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51440cf450 0x7f51440ce020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 -- 172.21.15.76:0/802500505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5144151fc0 con 0x7f5144161a60 2024-09-19T04:18:57.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f51437fe640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51440cf450 0x7f51440ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f5143fff640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5144161a60 0x7f51441515b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f513c0ae040 tx=0x7f513c018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f512bfff640 1 -- 172.21.15.76:0/802500505 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c0b0040 con 0x7f5144161a60 2024-09-19T04:18:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f512bfff640 1 -- 172.21.15.76:0/802500505 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f513c0b9070 con 0x7f5144161a60 2024-09-19T04:18:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51440b2490 con 0x7f5144161a60 2024-09-19T04:18:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f512bfff640 1 -- 172.21.15.76:0/802500505 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c0b4640 con 0x7f5144161a60 2024-09-19T04:18:57.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.829+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5144150ad0 con 0x7f5144161a60 2024-09-19T04:18:57.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f512bfff640 1 -- 172.21.15.76:0/802500505 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f513c0b47e0 con 0x7f5144161a60 2024-09-19T04:18:57.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f512bfff640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f51300678c0 0x7f5130069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f512bfff640 1 -- 172.21.15.76:0/802500505 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f513c11fb10 con 0x7f5144161a60 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f51300678c0 0x7f5130069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f51300678c0 0x7f5130069d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51440d19b0 tx=0x7f51340073c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 Processor -- start 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 -- start start 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca642f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca5f8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5f9290 0x562eca5f9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562eca4e0b60 con 0x562eca642b20 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562eca447c70 con 0x562eca643440 2024-09-19T04:18:57.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562eca3eded0 con 0x562eca5f9290 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca642f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca642f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48192/0 (socket says 172.21.15.76:48192) 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 -- 172.21.15.76:0/606561481 learned_addr learned my addr 172.21.15.76:0/606561481 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f5142ffd640 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca5f8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f5143fff640 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5f9290 0x562eca5f9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 -- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5f9290 msgr2=0x562eca5f9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5f9290 0x562eca5f9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 -- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 msgr2=0x562eca5f8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca5f8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 -- 172.21.15.76:0/606561481 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eca3ffe50 con 0x562eca642b20 2024-09-19T04:18:57.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51437fe640 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca642f00 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f513400f5e0 tx=0x7f513400f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdd1d6f93fdf255 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51297fa640 1 -- 172.21.15.76:0/606561481 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513400e6c0 con 0x562eca642b20 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51297fa640 1 -- 172.21.15.76:0/606561481 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5134021070 con 0x562eca642b20 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f51297fa640 1 -- 172.21.15.76:0/606561481 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513400ea60 con 0x562eca642b20 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 -- 172.21.15.76:0/606561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 msgr2=0x562eca642f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca642f00 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f513400f5e0 tx=0x7f513400f000 comp rx=0 tx=0).stop 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 -- 172.21.15.76:0/606561481 shutdown_connections 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5f9290 0x562eca5f9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca5f8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 --2- 172.21.15.76:0/606561481 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca642f00 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.833+0000 7f514d17b000 1 -- 172.21.15.76:0/606561481 >> 172.21.15.76:0/606561481 conn(0x562eca5ff600 msgr2=0x562eca5f7630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- 172.21.15.76:0/606561481 shutdown_connections 2024-09-19T04:18:57.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- 172.21.15.76:0/606561481 wait complete. 2024-09-19T04:18:57.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- start start 2024-09-19T04:18:57.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 Processor -- start 2024-09-19T04:18:57.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca5fde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca603cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca5fe390 0x562eca604230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562eca4a9b00 con 0x562eca5fe390 2024-09-19T04:18:57.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562eca422360 con 0x562eca643440 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562eca41f2d0 con 0x562eca642b20 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 10 client.0 _ll_register_callbacks cb 0x562eca5e4820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca5fe390 0x562eca604230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca5fe390 0x562eca604230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48200/0 (socket says 172.21.15.76:48200) 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 -- 172.21.15.76:0/3278125636 learned_addr learned my addr 172.21.15.76:0/3278125636 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51437fe640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca5fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5142ffd640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca603cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 -- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 msgr2=0x562eca5fde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca5fde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 -- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 msgr2=0x562eca603cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca603cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 -- 172.21.15.76:0/3278125636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eca3ee120 con 0x562eca5fe390 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5142ffd640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca603cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f5143fff640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca5fe390 0x562eca604230 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f513c0efac0 tx=0x7f513c0f0580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39068112cfeec649 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51227fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51227fc640 1 -- 172.21.15.76:0/3278125636 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c11b020 con 0x562eca5fe390 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51227fc640 1 -- 172.21.15.76:0/3278125636 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f513c0f0220 con 0x562eca5fe390 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5134021070 con 0x562eca5fe390 2024-09-19T04:18:57.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51227fc640 1 -- 172.21.15.76:0/3278125636 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f513c0f89a0 con 0x562eca5fe390 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 10 client.7488 fetch_fsmap learned FSMap version 89 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x562eca3e7450 con 0x562eca5fe390 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51227fc640 1 -- 172.21.15.76:0/3278125636 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5134021070 con 0x562eca5fe390 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51227fc640 1 -- 172.21.15.76:0/3278125636 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 89) ==== 3542+0+0 (secure 0 0 0) 0x7f513c0f8510 con 0x562eca5fe390 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 10 client.7488 fetch_fsmap finished waiting for FSMap version 89 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 10 client.7488 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 20 client.7488 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca651280 0x562eca653670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 4 client.7488 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562eca3e1c00 con 0x562eca651280 2024-09-19T04:18:57.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51437fe640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca651280 0x562eca653670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51227fc640 10 client.7488 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.837+0000 7f51437fe640 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca651280 0x562eca653670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=368a1d4270e637f7 server_cookie=c6ebc8faa803a471 in_seq=0 out_seq=0 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f51227fc640 1 -- 172.21.15.76:0/3278125636 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562eca3e1c00 con 0x562eca651280 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f51227fc640 10 client.7488 handle_command_reply: tid=0 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 client.7488 shutdown 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca5fe390 msgr2=0x562eca604230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca5fe390 0x562eca604230 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f513c0efac0 tx=0x7f513c0f0580 comp rx=0 tx=0).stop 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 shutdown_connections 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca651280 0x562eca653670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca5fe390 0x562eca604230 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca603cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 --2- 172.21.15.76:0/3278125636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca5fde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 >> 172.21.15.76:0/3278125636 conn(0x562eca5f9290 msgr2=0x562eca5f7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f51227fc640 0 client.7488 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:18:57.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 shutdown_connections 2024-09-19T04:18:57.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 1 -- 172.21.15.76:0/3278125636 wait complete. 2024-09-19T04:18:57.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.841+0000 7f514d17b000 20 client.7488 trim_cache size 0 max 16384 2024-09-19T04:18:57.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 Processor -- start 2024-09-19T04:18:57.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- start start 2024-09-19T04:18:57.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5fe390 0x562eca642470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca6564d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca656a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5144202f60 con 0x562eca642b20 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5144203630 con 0x562eca643440 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5144152b10 con 0x562eca5fe390 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca6564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca6564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48216/0 (socket says 172.21.15.76:48216) 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 -- 172.21.15.76:0/255039997 learned_addr learned my addr 172.21.15.76:0/255039997 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f51437fe640 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5fe390 0x562eca642470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 -- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5fe390 msgr2=0x562eca642470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5fe390 0x562eca642470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 -- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 msgr2=0x562eca656a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5143fff640 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca656a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca656a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 -- 172.21.15.76:0/255039997 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562eca643270 con 0x562eca642b20 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f51437fe640 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5fe390 0x562eca642470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5142ffd640 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca6564d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f513800f940 tx=0x7f513800fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38cc0204d029e33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5128ff9640 1 -- 172.21.15.76:0/255039997 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138011b00 con 0x562eca642b20 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5128ff9640 1 -- 172.21.15.76:0/255039997 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5138011ca0 con 0x562eca642b20 2024-09-19T04:18:57.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f5128ff9640 1 -- 172.21.15.76:0/255039997 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51380156a0 con 0x562eca642b20 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- 172.21.15.76:0/255039997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 msgr2=0x562eca6564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca6564d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f513800f940 tx=0x7f513800fe00 comp rx=0 tx=0).stop 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- 172.21.15.76:0/255039997 shutdown_connections 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca643440 0x562eca656a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca642b20 0x562eca6564d0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- 172.21.15.76:0/255039997 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca5fe390 0x562eca642470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- 172.21.15.76:0/255039997 >> 172.21.15.76:0/255039997 conn(0x562eca5f9290 msgr2=0x562eca5fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- 172.21.15.76:0/255039997 shutdown_connections 2024-09-19T04:18:57.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- 172.21.15.76:0/255039997 wait complete. 2024-09-19T04:18:57.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 -- start start 2024-09-19T04:18:57.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 Processor -- start 2024-09-19T04:18:57.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca5fe390 0x562eca655c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.861+0000 7f514d17b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca671960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca643440 0x562eca671ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562eca446f80 con 0x562eca643440 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562eca3ffca0 con 0x562eca5fe390 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562eca44c2a0 con 0x562eca642b20 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 10 client.0 _ll_register_callbacks cb 0x7f513c0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca643440 0x562eca671ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca643440 0x562eca671ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48230/0 (socket says 172.21.15.76:48230) 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 -- 172.21.15.76:0/2755084342 learned_addr learned my addr 172.21.15.76:0/2755084342 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51437fe640 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca5fe390 0x562eca655c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5142ffd640 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca671960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 -- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 msgr2=0x562eca671960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca671960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 -- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca5fe390 msgr2=0x562eca655c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca5fe390 0x562eca655c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 -- 172.21.15.76:0/2755084342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562eca3cefd0 con 0x562eca643440 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f5143fff640 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca643440 0x562eca671ea0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f513c01c500 tx=0x7f513c11dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33164143a7d51fb6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 1 -- 172.21.15.76:0/2755084342 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c11b040 con 0x562eca643440 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 1 -- 172.21.15.76:0/2755084342 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f513c117420 con 0x562eca643440 2024-09-19T04:18:57.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5138011ca0 con 0x562eca643440 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 1 -- 172.21.15.76:0/2755084342 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f513c0ff050 con 0x562eca643440 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 10 client.7494 fetch_fsmap learned FSMap version 89 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x562eca4ca750 con 0x562eca643440 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 1 -- 172.21.15.76:0/2755084342 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5138011ca0 con 0x562eca643440 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 1 -- 172.21.15.76:0/2755084342 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 89) ==== 3542+0+0 (secure 0 0 0) 0x7f513c0ef210 con 0x562eca643440 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 10 client.7494 fetch_fsmap finished waiting for FSMap version 89 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 10 client.7494 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 20 client.7494 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca6723e0 0x562eca6b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 4 client.7494 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:57.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "status"}) -- 0x562eca4553a0 con 0x562eca6723e0 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51437fe640 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca6723e0 0x562eca6b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 10 client.7494 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51437fe640 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca6723e0 0x562eca6b5a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b328142d4c05c92 server_cookie=1ddbd30a9e8a0276 in_seq=0 out_seq=0 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 1 -- 172.21.15.76:0/2755084342 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x562eca4553a0 con 0x562eca6723e0 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 10 client.7494 handle_command_reply: tid=0 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 client.7494 shutdown 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca643440 msgr2=0x562eca671ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca643440 0x562eca671ea0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f513c01c500 tx=0x7f513c11dd80 comp rx=0 tx=0).stop 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 shutdown_connections 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562eca6723e0 0x562eca6b5a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562eca643440 0x562eca671ea0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f51227fc640 0 client.7494 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562eca642b20 0x562eca671960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 --2- 172.21.15.76:0/2755084342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562eca5fe390 0x562eca655c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 >> 172.21.15.76:0/2755084342 conn(0x562eca5f9290 msgr2=0x562eca5fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 shutdown_connections 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 1 -- 172.21.15.76:0/2755084342 wait complete. 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514d17b000 20 client.7494 trim_cache size 0 max 16384 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:57.875 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 5794, 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 1.6975979450000001, 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 89, 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 91, 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 16.841685512000002 2024-09-19T04:18:57.876 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:57.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f51300678c0 msgr2=0x7f5130069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f51300678c0 0x7f5130069d80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f51440d19b0 tx=0x7f51340073c0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5144161a60 msgr2=0x7f51441515b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5144161a60 0x7f51441515b0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f513c0ae040 tx=0x7f513c018fe0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 shutdown_connections 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f51300678c0 0x7f5130069d80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5144161a60 0x7f51441515b0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f51440d3060 0x7f51440b1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 --2- 172.21.15.76:0/802500505 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51440cf450 0x7f51440ce020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 >> 172.21.15.76:0/802500505 conn(0x7f514408bf90 msgr2=0x7f51440d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 shutdown_connections 2024-09-19T04:18:57.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.865+0000 7f514a0bf640 1 -- 172.21.15.76:0/802500505 wait complete. 2024-09-19T04:18:57.883 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:18:57.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 Processor -- start 2024-09-19T04:18:57.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 -- start start 2024-09-19T04:18:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54dc15bec0 0x7f54dc0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc15c2a0 0x7f54dc0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54dc0af2a0 0x7f54dc15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:57.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f54dc0b3290 con 0x7f54dc0af2a0 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f54dc0b3570 con 0x7f54dc15bec0 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54e1394640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f54dc0b3850 con 0x7f54dc15c2a0 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54dc0af2a0 0x7f54dc15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f54dc0af2a0 0x7f54dc15fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48240/0 (socket says 172.21.15.76:48240) 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 -- 172.21.15.76:0/2592012433 learned_addr learned my addr 172.21.15.76:0/2592012433 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54da7fc640 1 --2- 172.21.15.76:0/2592012433 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc15c2a0 0x7f54dc0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54daffd640 1 --2- 172.21.15.76:0/2592012433 >> v2:172.21.15.116:3300/0 conn(0x7f54dc15bec0 0x7f54dc0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 -- 172.21.15.76:0/2592012433 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc15c2a0 msgr2=0x7f54dc0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 --2- 172.21.15.76:0/2592012433 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc15c2a0 0x7f54dc0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 -- 172.21.15.76:0/2592012433 >> v2:172.21.15.116:3300/0 conn(0x7f54dc15bec0 msgr2=0x7f54dc0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:57.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 --2- 172.21.15.76:0/2592012433 >> v2:172.21.15.116:3300/0 conn(0x7f54dc15bec0 0x7f54dc0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:57.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54db7fe640 1 -- 172.21.15.76:0/2592012433 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc0b3b30 con 0x7f54dc0af2a0 2024-09-19T04:18:57.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.993+0000 7f54da7fc640 1 --2- 172.21.15.76:0/2592012433 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc15c2a0 0x7f54dc0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:18:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54db7fe640 1 --2- 172.21.15.76:0/2592012433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc0af2a0 0x7f54dc15fc60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f54dc071710 tx=0x7f54d401c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92655d9c303644cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54d9ffb640 1 -- 172.21.15.76:0/2592012433 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40a5580 con 0x7f54dc0af2a0 2024-09-19T04:18:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54d9ffb640 1 -- 172.21.15.76:0/2592012433 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d40a5720 con 0x7f54dc0af2a0 2024-09-19T04:18:57.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54d9ffb640 1 -- 172.21.15.76:0/2592012433 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40a5af0 con 0x7f54dc0af2a0 2024-09-19T04:18:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 -- 172.21.15.76:0/2592012433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc0af2a0 msgr2=0x7f54dc15fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 --2- 172.21.15.76:0/2592012433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc0af2a0 0x7f54dc15fc60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f54dc071710 tx=0x7f54d401c700 comp rx=0 tx=0).stop 2024-09-19T04:18:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 -- 172.21.15.76:0/2592012433 shutdown_connections 2024-09-19T04:18:57.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 --2- 172.21.15.76:0/2592012433 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc0af2a0 0x7f54dc15fc60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 --2- 172.21.15.76:0/2592012433 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc15c2a0 0x7f54dc0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 --2- 172.21.15.76:0/2592012433 >> v2:172.21.15.116:3300/0 conn(0x7f54dc15bec0 0x7f54dc0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 -- 172.21.15.76:0/2592012433 >> 172.21.15.76:0/2592012433 conn(0x7f54dc0b1f90 msgr2=0x7f54dc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 -- 172.21.15.76:0/2592012433 shutdown_connections 2024-09-19T04:18:58.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 -- 172.21.15.76:0/2592012433 wait complete. 2024-09-19T04:18:58.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 Processor -- start 2024-09-19T04:18:58.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:57.997+0000 7f54e1394640 1 -- start start 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc0af2a0 0x7f54dc14c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54dc15bec0 0x7f54dc0b0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc15c2a0 0x7f54dc0b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc0ad740 con 0x7f54dc15c2a0 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc0afc90 con 0x7f54dc15bec0 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f54dc06e000 con 0x7f54dc0af2a0 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc15c2a0 0x7f54dc0b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54da7fc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54dc15bec0 0x7f54dc0b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc15c2a0 0x7f54dc0b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48252/0 (socket says 172.21.15.76:48252) 2024-09-19T04:18:58.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 -- 172.21.15.76:0/652150007 learned_addr learned my addr 172.21.15.76:0/652150007 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54daffd640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc0af2a0 0x7f54dc14c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 -- 172.21.15.76:0/652150007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc0af2a0 msgr2=0x7f54dc14c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc0af2a0 0x7f54dc14c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 -- 172.21.15.76:0/652150007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54dc15bec0 msgr2=0x7f54dc0b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54dc15bec0 0x7f54dc0b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 -- 172.21.15.76:0/652150007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54dc150df0 con 0x7f54dc15c2a0 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54db7fe640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc15c2a0 0x7f54dc0b07a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f54d401adb0 tx=0x7f54d40194a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54c37fe640 1 -- 172.21.15.76:0/652150007 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4009510 con 0x7f54dc15c2a0 2024-09-19T04:18:58.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54c37fe640 1 -- 172.21.15.76:0/652150007 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d40be070 con 0x7f54dc15c2a0 2024-09-19T04:18:58.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54c37fe640 1 -- 172.21.15.76:0/652150007 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40b88d0 con 0x7f54dc15c2a0 2024-09-19T04:18:58.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54dc1510d0 con 0x7f54dc15c2a0 2024-09-19T04:18:58.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54dc151690 con 0x7f54dc15c2a0 2024-09-19T04:18:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.001+0000 7f54c37fe640 1 -- 172.21.15.76:0/652150007 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f54d40b8a70 con 0x7f54dc15c2a0 2024-09-19T04:18:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54c37fe640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54c80678c0 0x7f54c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54c37fe640 1 -- 172.21.15.76:0/652150007 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f54d4120cd0 con 0x7f54dc15c2a0 2024-09-19T04:18:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54daffd640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54c80678c0 0x7f54c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54daffd640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54c80678c0 0x7f54c8069d80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f54cc007a60 tx=0x7f54cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 Processor -- start 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- start start 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf05f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cebbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cebc290 0x55917cebc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55917cda3b60 con 0x55917cebc290 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55917cd0ac70 con 0x55917cf05b20 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55917ccb0ed0 con 0x55917cf06440 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cebc290 0x55917cebc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cebc290 0x55917cebc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48264/0 (socket says 172.21.15.76:48264) 2024-09-19T04:18:58.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 -- 172.21.15.76:0/3060836929 learned_addr learned my addr 172.21.15.76:0/3060836929 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54daffd640 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf05f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54da7fc640 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cebbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 -- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 msgr2=0x55917cebbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cebbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 -- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 msgr2=0x55917cf05f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf05f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 -- 172.21.15.76:0/3060836929 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917ccc2e50 con 0x55917cebc290 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54db7fe640 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cebc290 0x55917cebc670 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f54d40ed490 tx=0x7f54d40f0990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39086a37cacdb12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54c0ff9640 1 -- 172.21.15.76:0/3060836929 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40f9b60 con 0x55917cebc290 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54c0ff9640 1 -- 172.21.15.76:0/3060836929 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d40a5680 con 0x55917cebc290 2024-09-19T04:18:58.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54c0ff9640 1 -- 172.21.15.76:0/3060836929 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40ed6a0 con 0x55917cebc290 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- 172.21.15.76:0/3060836929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cebc290 msgr2=0x55917cebc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cebc290 0x55917cebc670 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f54d40ed490 tx=0x7f54d40f0990 comp rx=0 tx=0).stop 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- 172.21.15.76:0/3060836929 shutdown_connections 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cebc290 0x55917cebc670 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cebbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 --2- 172.21.15.76:0/3060836929 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf05f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- 172.21.15.76:0/3060836929 >> 172.21.15.76:0/3060836929 conn(0x55917cec2600 msgr2=0x55917ceba630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- 172.21.15.76:0/3060836929 shutdown_connections 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- 172.21.15.76:0/3060836929 wait complete. 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 -- start start 2024-09-19T04:18:58.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.005+0000 7f54e4450000 1 Processor -- start 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf05b20 0x55917cec0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cec11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cec6cf0 0x55917cec70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55917cd6cb00 con 0x55917cf05b20 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55917cce5360 con 0x55917cec6cf0 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55917cce22d0 con 0x55917cf06440 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 10 client.0 _ll_register_callbacks cb 0x55917cea7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf05b20 0x55917cec0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf05b20 0x55917cec0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48274/0 (socket says 172.21.15.76:48274) 2024-09-19T04:18:58.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 -- 172.21.15.76:0/885386889 learned_addr learned my addr 172.21.15.76:0/885386889 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54db7fe640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cec6cf0 0x55917cec70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54da7fc640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cec11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 -- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 msgr2=0x55917cec11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cec11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 -- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cec6cf0 msgr2=0x55917cec70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cec6cf0 0x55917cec70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 -- 172.21.15.76:0/885386889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917ccb1120 con 0x55917cf05b20 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54da7fc640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cec11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54daffd640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf05b20 0x55917cec0c70 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f54cc011010 tx=0x7f54cc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8fb6f1d228d01f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:58.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 1 -- 172.21.15.76:0/885386889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc01c070 con 0x55917cf05b20 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 1 -- 172.21.15.76:0/885386889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f54cc0149b0 con 0x55917cf05b20 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d40a5680 con 0x55917cf05b20 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 1 -- 172.21.15.76:0/885386889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f54cc02f050 con 0x55917cf05b20 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 10 client.7506 fetch_fsmap learned FSMap version 89 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55917ccaa450 con 0x55917cf05b20 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 1 -- 172.21.15.76:0/885386889 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54d40a5680 con 0x55917cf05b20 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 1 -- 172.21.15.76:0/885386889 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 89) ==== 3542+0+0 (secure 0 0 0) 0x7f54cc032020 con 0x55917cf05b20 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 10 client.7506 fetch_fsmap finished waiting for FSMap version 89 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 10 client.7506 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 20 client.7506 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:58.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf14280 0x55917cf16670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 4 client.7506 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917cca4c00 con 0x55917cf14280 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54da7fc640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf14280 0x55917cf16670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 10 client.7506 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54da7fc640 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf14280 0x55917cf16670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=610775b85d66daf5 server_cookie=3099481d7c36a62d in_seq=0 out_seq=0 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 1 -- 172.21.15.76:0/885386889 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55917cca4c00 con 0x55917cf14280 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 10 client.7506 handle_command_reply: tid=0 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 client.7506 shutdown 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf05b20 msgr2=0x55917cec0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf05b20 0x55917cec0c70 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f54cc011010 tx=0x7f54cc004740 comp rx=0 tx=0).stop 2024-09-19T04:18:58.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 shutdown_connections 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf14280 0x55917cf16670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cec6cf0 0x55917cec70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf06440 0x55917cec11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54b9ffb640 0 client.7506 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 --2- 172.21.15.76:0/885386889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf05b20 0x55917cec0c70 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 >> 172.21.15.76:0/885386889 conn(0x55917cebc290 msgr2=0x55917cebad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 shutdown_connections 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 1 -- 172.21.15.76:0/885386889 wait complete. 2024-09-19T04:18:58.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.009+0000 7f54e4450000 20 client.7506 trim_cache size 0 max 16384 2024-09-19T04:18:58.021 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:18:58.013+0000 7f4997ef7640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:18:58.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 Processor -- start 2024-09-19T04:18:58.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 -- start start 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cec6cf0 0x55917cf04b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf05b20 0x55917cf050d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf06440 0x55917cf16670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc200df0 con 0x55917cec6cf0 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f54dc201480 con 0x55917cf06440 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f54dc1f5880 con 0x55917cf05b20 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cec6cf0 0x55917cf04b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cec6cf0 0x55917cf04b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48284/0 (socket says 172.21.15.76:48284) 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 -- 172.21.15.76:0/40571296 learned_addr learned my addr 172.21.15.76:0/40571296 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54da7fc640 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf05b20 0x55917cf050d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 -- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf05b20 msgr2=0x55917cf050d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf05b20 0x55917cf050d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 -- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf06440 msgr2=0x55917cf16670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf06440 0x55917cf16670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 -- 172.21.15.76:0/40571296 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917cc91fd0 con 0x55917cec6cf0 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54da7fc640 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf05b20 0x55917cf050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54daffd640 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cec6cf0 0x55917cf04b90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f54cc016fd0 tx=0x7f54cc021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebdfad7da4ba6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54bbfff640 1 -- 172.21.15.76:0/40571296 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc038070 con 0x55917cec6cf0 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54bbfff640 1 -- 172.21.15.76:0/40571296 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc01d040 con 0x55917cec6cf0 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.029+0000 7f54bbfff640 1 -- 172.21.15.76:0/40571296 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc004240 con 0x55917cec6cf0 2024-09-19T04:18:58.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/40571296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cec6cf0 msgr2=0x55917cf04b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cec6cf0 0x55917cf04b90 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f54cc016fd0 tx=0x7f54cc021000 comp rx=0 tx=0).stop 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/40571296 shutdown_connections 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf06440 0x55917cf16670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cf05b20 0x55917cf050d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/40571296 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cec6cf0 0x55917cf04b90 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/40571296 >> 172.21.15.76:0/40571296 conn(0x55917cebc290 msgr2=0x55917cebddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/40571296 shutdown_connections 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/40571296 wait complete. 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- start start 2024-09-19T04:18:58.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 Processor -- start 2024-09-19T04:18:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cec6cf0 0x55917cf19640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf78000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf06440 0x55917cf78540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55917cd09f80 con 0x55917cf06440 2024-09-19T04:18:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55917ccc2ca0 con 0x55917cf05b20 2024-09-19T04:18:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55917cd0f2a0 con 0x55917cec6cf0 2024-09-19T04:18:58.039 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 10 client.0 _ll_register_callbacks cb 0x7f54cc0073f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf06440 0x55917cf78540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf06440 0x55917cf78540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48292/0 (socket says 172.21.15.76:48292) 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 -- 172.21.15.76:0/2927193889 learned_addr learned my addr 172.21.15.76:0/2927193889 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54da7fc640 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf78000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54daffd640 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cec6cf0 0x55917cf19640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 -- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cec6cf0 msgr2=0x55917cf19640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cec6cf0 0x55917cf19640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 -- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 msgr2=0x55917cf78000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf78000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 -- 172.21.15.76:0/2927193889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917cd8d750 con 0x55917cf06440 2024-09-19T04:18:58.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54db7fe640 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf06440 0x55917cf78540 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009c90 tx=0x7f54d4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a940d6011a431a17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 1 -- 172.21.15.76:0/2927193889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d40f9b60 con 0x55917cf06440 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 1 -- 172.21.15.76:0/2927193889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f54d40ef330 con 0x55917cf06440 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc01d040 con 0x55917cf06440 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 1 -- 172.21.15.76:0/2927193889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f54d40f1290 con 0x55917cf06440 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 10 client.7512 fetch_fsmap learned FSMap version 89 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55917cc98a30 con 0x55917cf06440 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 1 -- 172.21.15.76:0/2927193889 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54cc01d040 con 0x55917cf06440 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 1 -- 172.21.15.76:0/2927193889 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 89) ==== 3542+0+0 (secure 0 0 0) 0x7f54d40f96d0 con 0x55917cf06440 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 10 client.7512 fetch_fsmap finished waiting for FSMap version 89 2024-09-19T04:18:58.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 10 client.7512 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 20 client.7512 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf34960 0x55917cf34d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 4 client.7512 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "status"}) -- 0x55917cd183a0 con 0x55917cf34960 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54daffd640 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf34960 0x55917cf34d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 10 client.7512 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54daffd640 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf34960 0x55917cf34d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1c97c82bef4ebfb3 server_cookie=d0e8d2b087209f4c in_seq=0 out_seq=0 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 1 -- 172.21.15.76:0/2927193889 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55917cd183a0 con 0x55917cf34960 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 10 client.7512 handle_command_reply: tid=0 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 client.7512 shutdown 2024-09-19T04:18:58.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf06440 msgr2=0x55917cf78540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf06440 0x55917cf78540 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009c90 tx=0x7f54d4009810 comp rx=0 tx=0).stop 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 shutdown_connections 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55917cf34960 0x55917cf34d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55917cf06440 0x55917cf78540 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54b9ffb640 0 client.7512 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55917cf05b20 0x55917cf78000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 --2- 172.21.15.76:0/2927193889 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55917cec6cf0 0x55917cf19640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 >> 172.21.15.76:0/2927193889 conn(0x55917cebc290 msgr2=0x55917cf03b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 shutdown_connections 2024-09-19T04:18:58.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 1 -- 172.21.15.76:0/2927193889 wait complete. 2024-09-19T04:18:58.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.033+0000 7f54e4450000 20 client.7512 trim_cache size 0 max 16384 2024-09-19T04:18:58.044 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:58.044 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:58.044 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-09-19T04:18:58.044 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7305, 2024-09-19T04:18:58.044 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:58.044 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:58.045 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:58.045 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 2.8743052850000002, 2024-09-19T04:18:58.045 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 89, 2024-09-19T04:18:58.045 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 91, 2024-09-19T04:18:58.045 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T04:18:58.045 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 17.030561828 2024-09-19T04:18:58.045 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:58.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54c80678c0 msgr2=0x7f54c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54c80678c0 0x7f54c8069d80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f54cc007a60 tx=0x7f54cc008040 comp rx=0 tx=0).stop 2024-09-19T04:18:58.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc15c2a0 msgr2=0x7f54dc0b07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc15c2a0 0x7f54dc0b07a0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f54d401adb0 tx=0x7f54d40194a0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 shutdown_connections 2024-09-19T04:18:58.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f54c80678c0 0x7f54c8069d80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f54dc15c2a0 0x7f54dc0b07a0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54dc15bec0 0x7f54dc0b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 --2- 172.21.15.76:0/652150007 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f54dc0af2a0 0x7f54dc14c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 >> 172.21.15.76:0/652150007 conn(0x7f54dc0b1f90 msgr2=0x7f54dc06e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 shutdown_connections 2024-09-19T04:18:58.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.037+0000 7f54e1394640 1 -- 172.21.15.76:0/652150007 wait complete. 2024-09-19T04:18:58.052 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:18:58.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.157+0000 7f46416bd640 1 Processor -- start 2024-09-19T04:18:58.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.157+0000 7f46416bd640 1 -- start start 2024-09-19T04:18:58.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.157+0000 7f46416bd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f463c06f010 0x7f463c166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.157+0000 7f46416bd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463c0b3290 con 0x7f463c15c2a0 2024-09-19T04:18:58.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f463c0b3570 con 0x7f463c15bec0 2024-09-19T04:18:58.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f463c0b3850 con 0x7f463c06f010 2024-09-19T04:18:58.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f463b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f463c06f010 0x7f463c166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f463a7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f463c15c2a0 0x7f463c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47278/0 (socket says 172.21.15.76:47278) 2024-09-19T04:18:58.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f463a7fc640 1 -- 172.21.15.76:0/96641321 learned_addr learned my addr 172.21.15.76:0/96641321 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:58.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f4639ffb640 1 -- 172.21.15.76:0/96641321 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3715895181 0 0) 0x7f463c0b3290 con 0x7f463c15c2a0 2024-09-19T04:18:58.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f4639ffb640 1 -- 172.21.15.76:0/96641321 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4624003530 con 0x7f463c15c2a0 2024-09-19T04:18:58.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f463b7fe640 1 -- 172.21.15.76:0/96641321 >> v1:172.21.15.116:6789/0 conn(0x7f463c15bec0 legacy=0x7f463c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f463b7fe640 1 -- 172.21.15.76:0/96641321 >> v1:172.21.15.76:6789/0 conn(0x7f463c15c2a0 legacy=0x7f463c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f463b7fe640 1 -- 172.21.15.76:0/96641321 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c0b3b30 con 0x7f463c06f010 2024-09-19T04:18:58.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f463b7fe640 1 --2- 172.21.15.76:0/96641321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c06f010 0x7f463c166ff0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4634009d90 tx=0x7f463401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4679a3289dd1fed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f4639ffb640 1 -- 172.21.15.76:0/96641321 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46340b2070 con 0x7f463c06f010 2024-09-19T04:18:58.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f4639ffb640 1 -- 172.21.15.76:0/96641321 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46340a5910 con 0x7f463c06f010 2024-09-19T04:18:58.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f4639ffb640 1 -- 172.21.15.76:0/96641321 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46340a5ce0 con 0x7f463c06f010 2024-09-19T04:18:58.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 -- 172.21.15.76:0/96641321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c06f010 msgr2=0x7f463c166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 --2- 172.21.15.76:0/96641321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c06f010 0x7f463c166ff0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f4634009d90 tx=0x7f463401c940 comp rx=0 tx=0).stop 2024-09-19T04:18:58.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 -- 172.21.15.76:0/96641321 shutdown_connections 2024-09-19T04:18:58.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 --2- 172.21.15.76:0/96641321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c06f010 0x7f463c166ff0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 -- 172.21.15.76:0/96641321 >> 172.21.15.76:0/96641321 conn(0x7f463c0b1f90 msgr2=0x7f463c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 -- 172.21.15.76:0/96641321 shutdown_connections 2024-09-19T04:18:58.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.161+0000 7f46416bd640 1 -- 172.21.15.76:0/96641321 wait complete. 2024-09-19T04:18:58.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 Processor -- start 2024-09-19T04:18:58.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 -- start start 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f463c06f010 0x7f463c15d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c15bec0 0x7f463c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f463c15c2a0 0x7f463c14f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f463c169ed0 con 0x7f463c15bec0 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f463c169d50 con 0x7f463c15c2a0 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f463c16a050 con 0x7f463c06f010 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c15bec0 0x7f463c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c15bec0 0x7f463c15d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48300/0 (socket says 172.21.15.76:48300) 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 -- 172.21.15.76:0/4204615577 learned_addr learned my addr 172.21.15.76:0/4204615577 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463affd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f463c06f010 0x7f463c15d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 -- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f463c06f010 msgr2=0x7f463c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f463c06f010 0x7f463c15d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 -- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f463c15c2a0 msgr2=0x7f463c14f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463b7fe640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f463c15c2a0 0x7f463c14f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f463c15c2a0 0x7f463c14f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 -- 172.21.15.76:0/4204615577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463c15dd50 con 0x7f463c15bec0 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463affd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f463c06f010 0x7f463c15d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f463a7fc640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c15bec0 0x7f463c15d0a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f463000c400 tx=0x7f463000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46237fe640 1 -- 172.21.15.76:0/4204615577 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630017070 con 0x7f463c15bec0 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46237fe640 1 -- 172.21.15.76:0/4204615577 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4630005cf0 con 0x7f463c15bec0 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f463c09c890 con 0x7f463c15bec0 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46237fe640 1 -- 172.21.15.76:0/4204615577 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46300072d0 con 0x7f463c15bec0 2024-09-19T04:18:58.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463c1528e0 con 0x7f463c15bec0 2024-09-19T04:18:58.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.165+0000 7f46237fe640 1 -- 172.21.15.76:0/4204615577 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f463001a070 con 0x7f463c15bec0 2024-09-19T04:18:58.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f46237fe640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46280678c0 0x7f4628069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f46237fe640 1 -- 172.21.15.76:0/4204615577 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f4630085240 con 0x7f463c15bec0 2024-09-19T04:18:58.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463affd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46280678c0 0x7f4628069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463affd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46280678c0 0x7f4628069d80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f462c000f70 tx=0x7f462c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 Processor -- start 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- start start 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32a8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a8690 0x5644d325e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d325e6a0 0x5644d325ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644d3141b60 con 0x5644d32a8690 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5644d30a8c70 con 0x5644d32a7d70 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5644d304eed0 con 0x5644d325e6a0 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a8690 0x5644d325e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a8690 0x5644d325e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48310/0 (socket says 172.21.15.76:48310) 2024-09-19T04:18:58.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 -- 172.21.15.76:0/2588006812 learned_addr learned my addr 172.21.15.76:0/2588006812 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463affd640 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 -- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d325e6a0 msgr2=0x5644d325ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463b7fe640 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d325e6a0 0x5644d325ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d325e6a0 0x5644d325ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 -- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 msgr2=0x5644d32a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 -- 172.21.15.76:0/2588006812 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d3060e50 con 0x5644d32a8690 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463affd640 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f463a7fc640 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a8690 0x5644d325e160 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f46300835e0 tx=0x7f4630059960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b172aab27881724e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4620ff9640 1 -- 172.21.15.76:0/2588006812 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46300538b0 con 0x5644d32a8690 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4620ff9640 1 -- 172.21.15.76:0/2588006812 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4630051c20 con 0x5644d32a8690 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4620ff9640 1 -- 172.21.15.76:0/2588006812 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46300076b0 con 0x5644d32a8690 2024-09-19T04:18:58.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- 172.21.15.76:0/2588006812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a8690 msgr2=0x5644d325e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a8690 0x5644d325e160 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f46300835e0 tx=0x7f4630059960 comp rx=0 tx=0).stop 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- 172.21.15.76:0/2588006812 shutdown_connections 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d325e6a0 0x5644d325ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a8690 0x5644d325e160 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 --2- 172.21.15.76:0/2588006812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- 172.21.15.76:0/2588006812 >> 172.21.15.76:0/2588006812 conn(0x5644d3260600 msgr2=0x5644d3258630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- 172.21.15.76:0/2588006812 shutdown_connections 2024-09-19T04:18:58.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- 172.21.15.76:0/2588006812 wait complete. 2024-09-19T04:18:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.169+0000 7f4644779000 1 -- start start 2024-09-19T04:18:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 Processor -- start 2024-09-19T04:18:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d32a8690 0x5644d32ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d3294490 0x5644d3294870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644d310ab00 con 0x5644d3294490 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5644d3083360 con 0x5644d32a7d70 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5644d30802d0 con 0x5644d32a8690 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 10 client.0 _ll_register_callbacks cb 0x5644d3245820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d3294490 0x5644d3294870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d3294490 0x5644d3294870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48326/0 (socket says 172.21.15.76:48326) 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 -- 172.21.15.76:0/3629507112 learned_addr learned my addr 172.21.15.76:0/3629507112 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463affd640 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463a7fc640 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d32a8690 0x5644d32ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 -- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d32a8690 msgr2=0x5644d32ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d32a8690 0x5644d32ae3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 -- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 msgr2=0x5644d32adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32adeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 -- 172.21.15.76:0/3629507112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d304f120 con 0x5644d3294490 2024-09-19T04:18:58.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463b7fe640 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d3294490 0x5644d3294870 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f46340b0040 tx=0x7f463401b180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1895b277b5c42ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 1 -- 172.21.15.76:0/3629507112 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46340bd070 con 0x5644d3294490 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 1 -- 172.21.15.76:0/3629507112 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f46340b8440 con 0x5644d3294490 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4630051c20 con 0x5644d3294490 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 1 -- 172.21.15.76:0/3629507112 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f46340c8050 con 0x5644d3294490 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 10 client.7527 fetch_fsmap learned FSMap version 89 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x5644d3048450 con 0x5644d3294490 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 1 -- 172.21.15.76:0/3629507112 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4630051c20 con 0x5644d3294490 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 1 -- 172.21.15.76:0/3629507112 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 89) ==== 3542+0+0 (secure 0 0 0) 0x7f46340cb020 con 0x5644d3294490 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 10 client.7527 fetch_fsmap finished waiting for FSMap version 89 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 10 client.7527 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 20 client.7527 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:58.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d3295280 0x5644d3295660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 4 client.7527 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644d3042c00 con 0x5644d3295280 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463affd640 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d3295280 0x5644d3295660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f463affd640 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d3295280 0x5644d3295660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e95374a526d3e82a server_cookie=759d8f579fe6f6ae in_seq=0 out_seq=0 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 10 client.7527 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 1 -- 172.21.15.76:0/3629507112 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5644d3042c00 con 0x5644d3295280 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 10 client.7527 handle_command_reply: tid=0 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 client.7527 shutdown 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d3294490 msgr2=0x5644d3294870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d3294490 0x5644d3294870 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f46340b0040 tx=0x7f463401b180 comp rx=0 tx=0).stop 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 shutdown_connections 2024-09-19T04:18:58.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d3295280 0x5644d3295660 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d3294490 0x5644d3294870 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4619ffb640 0 client.7527 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d32a8690 0x5644d32ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 --2- 172.21.15.76:0/3629507112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a7d70 0x5644d32adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 >> 172.21.15.76:0/3629507112 conn(0x5644d325e6a0 msgr2=0x5644d325f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 shutdown_connections 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 1 -- 172.21.15.76:0/3629507112 wait complete. 2024-09-19T04:18:58.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.173+0000 7f4644779000 20 client.7527 trim_cache size 0 max 16384 2024-09-19T04:18:58.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 Processor -- start 2024-09-19T04:18:58.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 -- start start 2024-09-19T04:18:58.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 0x5644d32be260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d32be7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 0x5644d32acca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f463c073b00 con 0x5644d32a7d70 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f463c1ff650 con 0x5644d32a8690 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f463c1f8ef0 con 0x5644d3294490 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.193+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d32be7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d32be7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48328/0 (socket says 172.21.15.76:48328) 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/3184046729 learned_addr learned my addr 172.21.15.76:0/3184046729 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/3184046729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 msgr2=0x5644d32be260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- 172.21.15.76:0/3184046729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 0x5644d32be260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/3184046729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 msgr2=0x5644d32acca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- 172.21.15.76:0/3184046729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 0x5644d32acca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/3184046729 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d3294be0 con 0x5644d32a7d70 2024-09-19T04:18:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- 172.21.15.76:0/3184046729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d32be7a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f46300070a0 tx=0x7f463001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426e2e7a15a32801 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f461bfff640 1 -- 172.21.15.76:0/3184046729 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630080070 con 0x5644d32a7d70 2024-09-19T04:18:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f461bfff640 1 -- 172.21.15.76:0/3184046729 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4630082070 con 0x5644d32a7d70 2024-09-19T04:18:58.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f461bfff640 1 -- 172.21.15.76:0/3184046729 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630053650 con 0x5644d32a7d70 2024-09-19T04:18:58.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/3184046729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 msgr2=0x5644d32be7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- 172.21.15.76:0/3184046729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d32be7a0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f46300070a0 tx=0x7f463001d000 comp rx=0 tx=0).stop 2024-09-19T04:18:58.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/3184046729 shutdown_connections 2024-09-19T04:18:58.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- 172.21.15.76:0/3184046729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 0x5644d32acca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- 172.21.15.76:0/3184046729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d32be7a0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- 172.21.15.76:0/3184046729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 0x5644d32be260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/3184046729 >> 172.21.15.76:0/3184046729 conn(0x5644d325e6a0 msgr2=0x5644d3258ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/3184046729 shutdown_connections 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/3184046729 wait complete. 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- start start 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 Processor -- start 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 0x5644d3320420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d3320960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 0x5644d3320ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5644d30a7f80 con 0x5644d32a7d70 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5644d3060ca0 con 0x5644d32a8690 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5644d30ad2a0 con 0x5644d3294490 2024-09-19T04:18:58.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 10 client.0 _ll_register_callbacks cb 0x7f46340b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d3320960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d3320960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48332/0 (socket says 172.21.15.76:48332) 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/1036854036 learned_addr learned my addr 172.21.15.76:0/1036854036 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/1036854036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 msgr2=0x5644d3320420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 0x5644d3320420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/1036854036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 msgr2=0x5644d3320ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 0x5644d3320ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 -- 172.21.15.76:0/1036854036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d302ffd0 con 0x5644d32a7d70 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463a7fc640 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d3320960 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4630085560 tx=0x7f4630001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6579e7ecb85b76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4619ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4619ffb640 1 -- 172.21.15.76:0/1036854036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4630078020 con 0x5644d32a7d70 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4619ffb640 1 -- 172.21.15.76:0/1036854036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7f4630072030 con 0x5644d32a7d70 2024-09-19T04:18:58.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4630082070 con 0x5644d32a7d70 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4619ffb640 1 -- 172.21.15.76:0/1036854036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f4630080050 con 0x5644d32a7d70 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 10 client.7533 fetch_fsmap learned FSMap version 89 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x5644d312b750 con 0x5644d32a7d70 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4619ffb640 1 -- 172.21.15.76:0/1036854036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4630082070 con 0x5644d32a7d70 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4619ffb640 1 -- 172.21.15.76:0/1036854036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 89) ==== 3542+0+0 (secure 0 0 0) 0x7f4630007470 con 0x5644d32a7d70 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 10 client.7533 fetch_fsmap finished waiting for FSMap version 89 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 10 client.7533 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 20 client.7533 populate_metadata read hostname 'smithi076' 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d32af110 0x5644d32af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 4 client.7533 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "status"}) -- 0x5644d30b63a0 con 0x5644d32af110 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.197+0000 7f463affd640 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d32af110 0x5644d32af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f463affd640 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d32af110 0x5644d32af4f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a04f5a6772358866 server_cookie=e113ecdd41fb47e4 in_seq=0 out_seq=0 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4619ffb640 10 client.7533 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4619ffb640 1 -- 172.21.15.76:0/1036854036 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x5644d30b63a0 con 0x5644d32af110 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4619ffb640 10 client.7533 handle_command_reply: tid=0 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 client.7533 shutdown 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 msgr2=0x5644d3320960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d3320960 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f4630085560 tx=0x7f4630001160 comp rx=0 tx=0).stop 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 shutdown_connections 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5644d32af110 0x5644d32af4f0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5644d32a8690 0x5644d3320ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5644d32a7d70 0x5644d3320960 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 --2- 172.21.15.76:0/1036854036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5644d3294490 0x5644d3320420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4619ffb640 0 client.7533 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:18:58.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 >> 172.21.15.76:0/1036854036 conn(0x5644d325e6a0 msgr2=0x5644d32bd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 shutdown_connections 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 1 -- 172.21.15.76:0/1036854036 wait complete. 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f4644779000 20 client.7533 trim_cache size 0 max 16384 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 7311, 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:18:58.207 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:18:58.208 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:18:58.208 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 12.112904596, 2024-09-19T04:18:58.208 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 89, 2024-09-19T04:18:58.208 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 91, 2024-09-19T04:18:58.208 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 91, 2024-09-19T04:18:58.208 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 17.187661175999999 2024-09-19T04:18:58.208 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46280678c0 msgr2=0x7f4628069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46280678c0 0x7f4628069d80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f462c000f70 tx=0x7f462c009040 comp rx=0 tx=0).stop 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c15bec0 msgr2=0x7f463c15d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c15bec0 0x7f463c15d0a0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f463000c400 tx=0x7f463000c8c0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 shutdown_connections 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f46280678c0 0x7f4628069d80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f463c15c2a0 0x7f463c14f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f463c15bec0 0x7f463c15d0a0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 --2- 172.21.15.76:0/4204615577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f463c06f010 0x7f463c15d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 >> 172.21.15.76:0/4204615577 conn(0x7f463c0b1f90 msgr2=0x7f463c0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 shutdown_connections 2024-09-19T04:18:58.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.201+0000 7f46416bd640 1 -- 172.21.15.76:0/4204615577 wait complete. 2024-09-19T04:18:58.215 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:18:58.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.321+0000 7fcdd882c640 1 Processor -- start 2024-09-19T04:18:58.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.321+0000 7fcdd882c640 1 -- start start 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd882c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcdd415bec0 0x7fcdd40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd882c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcdd415c2a0 0x7fcdd40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd882c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcdd40b3290 con 0x7fcdd40af2a0 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd882c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40b3570 con 0x7fcdd415c2a0 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd882c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcdd40b3850 con 0x7fcdd415bec0 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd1d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcdd415c2a0 0x7fcdd40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd2d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcdd40af2a0 0x7fcdd415fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47294/0 (socket says 172.21.15.76:47294) 2024-09-19T04:18:58.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd1d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcdd415c2a0 0x7fcdd40aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48334/0 (socket says 172.21.15.76:48334) 2024-09-19T04:18:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd2d9d640 1 -- 172.21.15.76:0/4123960262 learned_addr learned my addr 172.21.15.76:0/4123960262 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:18:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd259c640 1 --2- 172.21.15.76:0/4123960262 >> v2:172.21.15.116:3300/0 conn(0x7fcdd415bec0 0x7fcdd40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd1d9b640 1 -- 172.21.15.76:0/4123960262 >> v2:172.21.15.116:3300/0 conn(0x7fcdd415bec0 msgr2=0x7fcdd40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:18:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd1d9b640 1 --2- 172.21.15.76:0/4123960262 >> v2:172.21.15.116:3300/0 conn(0x7fcdd415bec0 0x7fcdd40ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd1d9b640 1 -- 172.21.15.76:0/4123960262 >> v1:172.21.15.76:6789/0 conn(0x7fcdd40af2a0 legacy=0x7fcdd415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd159a640 1 -- 172.21.15.76:0/4123960262 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1865003467 0 0) 0x7fcdd40b3290 con 0x7fcdd40af2a0 2024-09-19T04:18:58.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.325+0000 7fcdd1d9b640 1 -- 172.21.15.76:0/4123960262 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd40b3b30 con 0x7fcdd415c2a0 2024-09-19T04:18:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 --2- 172.21.15.76:0/4123960262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415c2a0 0x7fcdd40aed60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4071710 tx=0x7fcdbc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d0b750bc23558ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd159a640 1 -- 172.21.15.76:0/4123960262 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0ab650 con 0x7fcdd415c2a0 2024-09-19T04:18:58.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd159a640 1 -- 172.21.15.76:0/4123960262 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdbc0b0070 con 0x7fcdd415c2a0 2024-09-19T04:18:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd159a640 1 -- 172.21.15.76:0/4123960262 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0aba10 con 0x7fcdd415c2a0 2024-09-19T04:18:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- 172.21.15.76:0/4123960262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415c2a0 msgr2=0x7fcdd40aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 --2- 172.21.15.76:0/4123960262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415c2a0 0x7fcdd40aed60 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4071710 tx=0x7fcdbc01c450 comp rx=0 tx=0).stop 2024-09-19T04:18:58.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- 172.21.15.76:0/4123960262 shutdown_connections 2024-09-19T04:18:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 --2- 172.21.15.76:0/4123960262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415c2a0 0x7fcdd40aed60 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 --2- 172.21.15.76:0/4123960262 >> v2:172.21.15.116:3300/0 conn(0x7fcdd415bec0 0x7fcdd40ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- 172.21.15.76:0/4123960262 >> 172.21.15.76:0/4123960262 conn(0x7fcdd40b1f90 msgr2=0x7fcdd40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- 172.21.15.76:0/4123960262 shutdown_connections 2024-09-19T04:18:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- 172.21.15.76:0/4123960262 wait complete. 2024-09-19T04:18:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 Processor -- start 2024-09-19T04:18:58.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- start start 2024-09-19T04:18:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdd40af2a0 0x7fcdd414f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415bec0 0x7fcdd4166250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcdd415c2a0 0x7fcdd41f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd40af6d0 con 0x7fcdd415bec0 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcdd40ad740 con 0x7fcdd40af2a0 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd882c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcdd4169000 con 0x7fcdd415c2a0 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415bec0 0x7fcdd4166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415bec0 0x7fcdd4166250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:48350/0 (socket says 172.21.15.76:48350) 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 -- 172.21.15.76:0/3051058398 learned_addr learned my addr 172.21.15.76:0/3051058398 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 -- 172.21.15.76:0/3051058398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcdd415c2a0 msgr2=0x7fcdd41f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcdd415c2a0 0x7fcdd41f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd259c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdd40af2a0 0x7fcdd414f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 -- 172.21.15.76:0/3051058398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdd40af2a0 msgr2=0x7fcdd414f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdd40af2a0 0x7fcdd414f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.329+0000 7fcdd1d9b640 1 -- 172.21.15.76:0/3051058398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdd41651b0 con 0x7fcdd415bec0 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdd1d9b640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415bec0 0x7fcdd4166250 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc019060 tx=0x7fcdbc0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdcaffd640 1 -- 172.21.15.76:0/3051058398 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0abde0 con 0x7fcdd415bec0 2024-09-19T04:18:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdcaffd640 1 -- 172.21.15.76:0/3051058398 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdbc0bf450 con 0x7fcdd415bec0 2024-09-19T04:18:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdcaffd640 1 -- 172.21.15.76:0/3051058398 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdbc0bf840 con 0x7fcdd415bec0 2024-09-19T04:18:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdd4165490 con 0x7fcdd415bec0 2024-09-19T04:18:58.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdd4165a50 con 0x7fcdd415bec0 2024-09-19T04:18:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdd40ad200 con 0x7fcdd415bec0 2024-09-19T04:18:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdcaffd640 1 -- 172.21.15.76:0/3051058398 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcdbc0b43f0 con 0x7fcdd415bec0 2024-09-19T04:18:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdcaffd640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdb40678c0 0x7fcdb4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:18:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdcaffd640 1 -- 172.21.15.76:0/3051058398 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 6101+0+0 (secure 0 0 0) 0x7fcdbc120890 con 0x7fcdd415bec0 2024-09-19T04:18:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdd259c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdb40678c0 0x7fcdb4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:18:58.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.333+0000 7fcdd259c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdb40678c0 0x7fcdb4069d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400b800 tx=0x7fcdc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:18:58.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.337+0000 7fcdcaffd640 1 -- 172.21.15.76:0/3051058398 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcdbc016070 con 0x7fcdd415bec0 2024-09-19T04:18:58.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.533+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdd406fcd0 con 0x7fcdd415bec0 2024-09-19T04:18:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.533+0000 7fcdcaffd640 1 -- 172.21.15.76:0/3051058398 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5266 (secure 0 0 0) 0x7fcdbc0f0c80 con 0x7fcdd415bec0 2024-09-19T04:18:58.536 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:18:58.536 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":89,"btime":"2024-09-19T04:18:57:170853+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:18:57.170840+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[5794,7305,7311]},"id":6}]} 2024-09-19T04:18:58.536 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 89 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdb40678c0 msgr2=0x7fcdb4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdb40678c0 0x7fcdb4069d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcdc400b800 tx=0x7fcdc4008040 comp rx=0 tx=0).stop 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415bec0 msgr2=0x7fcdd4166250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415bec0 0x7fcdd4166250 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc019060 tx=0x7fcdbc0191f0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 shutdown_connections 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcdb40678c0 0x7fcdb4069d80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcdd415c2a0 0x7fcdd41f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcdd415bec0 0x7fcdd4166250 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 --2- 172.21.15.76:0/3051058398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcdd40af2a0 0x7fcdd414f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:18:58.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 >> 172.21.15.76:0/3051058398 conn(0x7fcdd40b1f90 msgr2=0x7fcdd4074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:18:58.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 shutdown_connections 2024-09-19T04:18:58.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:18:58.537+0000 7fcdd882c640 1 -- 172.21.15.76:0/3051058398 wait complete. 2024-09-19T04:18:58.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:58.556 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:18:58.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:58.579 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:18:58.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:58.633 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-19T04:18:58.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:58.683 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-19T04:18:58.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:18:58.732 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-19T04:18:58.785 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:18:58.967 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:18:58.967 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:18:58.967 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:19:00.968 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:19:01.116 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:19:01.117 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:01.117 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:19:03.118 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:19:03.264 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:19:03.265 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:03.265 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:19:05.266 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:19:05.414 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:19:05.415 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:05.415 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:19:07.416 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:19:07.564 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-19T04:19:07.565 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:19:07.565 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:19:07.566 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:19:07.705 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:19:07.705 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:07.705 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:19:09.707 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:19:09.853 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:19:09.857 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:09.857 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:19:11.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:19:12.007 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}}] 2024-09-19T04:19:12.008 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:19:12.008 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:19:12.008 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json get subtrees 2024-09-19T04:19:12.155 INFO:teuthology.orchestra.run.smithi116.stdout:[{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2024-09-19T04:19:12.155 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:12.155 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:19:12.156 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:0 flush journal 2024-09-19T04:19:12.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.285+0000 7fef010da640 1 Processor -- start 2024-09-19T04:19:12.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.285+0000 7fef010da640 1 -- start start 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.285+0000 7fef010da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feefc070020 0x7feefc15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.285+0000 7fef010da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feefc0aede0 con 0x7feefc1520a0 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.285+0000 7fef010da640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feefc15fa10 con 0x7feefc15f1d0 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.285+0000 7fef010da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7feefc152480 con 0x7feefc070020 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feefa59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feefc070020 0x7feefc15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feefad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7feefc1520a0 0x7feefc06f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45286/0 (socket says 172.21.15.76:45286) 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feefad9d640 1 -- 172.21.15.76:0/3354470857 learned_addr learned my addr 172.21.15.76:0/3354470857 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:12.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feefa59c640 1 -- 172.21.15.76:0/3354470857 >> v1:172.21.15.116:6789/0 conn(0x7feefc15f1d0 legacy=0x7feefc0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feefa59c640 1 -- 172.21.15.76:0/3354470857 >> v1:172.21.15.76:6789/0 conn(0x7feefc1520a0 legacy=0x7feefc06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feefa59c640 1 -- 172.21.15.76:0/3354470857 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc074250 con 0x7feefc070020 2024-09-19T04:19:12.289 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feef9d9b640 1 -- 172.21.15.76:0/3354470857 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2509797447 0 0) 0x7feefc0aede0 con 0x7feefc1520a0 2024-09-19T04:19:12.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feefa59c640 1 --2- 172.21.15.76:0/3354470857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc070020 0x7feefc15ec90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7feee40097c0 tx=0x7feee401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a627f4d07d0981 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feef9d9b640 1 -- 172.21.15.76:0/3354470857 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0aeba0 con 0x7feefc070020 2024-09-19T04:19:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feef9d9b640 1 -- 172.21.15.76:0/3354470857 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee40b0070 con 0x7feefc070020 2024-09-19T04:19:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7feef9d9b640 1 -- 172.21.15.76:0/3354470857 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40ab650 con 0x7feefc070020 2024-09-19T04:19:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7fef010da640 1 -- 172.21.15.76:0/3354470857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc070020 msgr2=0x7feefc15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7fef010da640 1 --2- 172.21.15.76:0/3354470857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc070020 0x7feefc15ec90 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7feee40097c0 tx=0x7feee401c430 comp rx=0 tx=0).stop 2024-09-19T04:19:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7fef010da640 1 -- 172.21.15.76:0/3354470857 shutdown_connections 2024-09-19T04:19:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7fef010da640 1 --2- 172.21.15.76:0/3354470857 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc070020 0x7feefc15ec90 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7fef010da640 1 -- 172.21.15.76:0/3354470857 >> 172.21.15.76:0/3354470857 conn(0x7feefc0b1f90 msgr2=0x7feefc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7fef010da640 1 -- 172.21.15.76:0/3354470857 shutdown_connections 2024-09-19T04:19:12.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.289+0000 7fef010da640 1 -- 172.21.15.76:0/3354470857 wait complete. 2024-09-19T04:19:12.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 Processor -- start 2024-09-19T04:19:12.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 -- start start 2024-09-19T04:19:12.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feefc070020 0x7feefc0996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc1520a0 0x7feefc0b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feefc15f1d0 0x7feefc15d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feefc15f680 con 0x7feefc1520a0 2024-09-19T04:19:12.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feefc073d30 con 0x7feefc070020 2024-09-19T04:19:12.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7feefc168f20 con 0x7feefc15f1d0 2024-09-19T04:19:12.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc1520a0 0x7feefc0b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc1520a0 0x7feefc0b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56020/0 (socket says 172.21.15.76:56020) 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 -- 172.21.15.76:0/2975715880 learned_addr learned my addr 172.21.15.76:0/2975715880 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefad9d640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feefc070020 0x7feefc0996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefb59e640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feefc15f1d0 0x7feefc15d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 -- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feefc15f1d0 msgr2=0x7feefc15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feefc15f1d0 0x7feefc15d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 -- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feefc070020 msgr2=0x7feefc0996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feefc070020 0x7feefc0996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 -- 172.21.15.76:0/2975715880 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc14fa50 con 0x7feefc1520a0 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefb59e640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feefc15f1d0 0x7feefc15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feefa59c640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc1520a0 0x7feefc0b1690 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7feee4019040 tx=0x7feee401ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feef37fe640 1 -- 172.21.15.76:0/2975715880 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40bf610 con 0x7feefc1520a0 2024-09-19T04:19:12.296 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feef37fe640 1 -- 172.21.15.76:0/2975715880 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feee40bf7d0 con 0x7feefc1520a0 2024-09-19T04:19:12.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc14d230 con 0x7feefc1520a0 2024-09-19T04:19:12.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7feef37fe640 1 -- 172.21.15.76:0/2975715880 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40bfbc0 con 0x7feefc1520a0 2024-09-19T04:19:12.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.293+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc14d7f0 con 0x7feefc1520a0 2024-09-19T04:19:12.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feef37fe640 1 -- 172.21.15.76:0/2975715880 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feee40bd020 con 0x7feefc1520a0 2024-09-19T04:19:12.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feef37fe640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feedc0678f0 0x7feedc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feef37fe640 1 -- 172.21.15.76:0/2975715880 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7feee40a8030 con 0x7feefc1520a0 2024-09-19T04:19:12.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefad9d640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feedc0678f0 0x7feedc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefad9d640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feedc0678f0 0x7feedc069db0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feeec000f70 tx=0x7feeec007b80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 Processor -- start 2024-09-19T04:19:12.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 -- start start 2024-09-19T04:19:12.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.300 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bb690 0x5598a206e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a206edb0 0x5598a206f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5598a1f58960 con 0x5598a206edb0 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5598a1e9c900 con 0x5598a20bad70 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5598a1ebfd40 con 0x5598a20bb690 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a206edb0 0x5598a206f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a206edb0 0x5598a206f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56022/0 (socket says 172.21.15.76:56022) 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 -- 172.21.15.76:0/3897686729 learned_addr learned my addr 172.21.15.76:0/3897686729 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefad9d640 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefa59c640 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bb690 0x5598a206e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 -- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bb690 msgr2=0x5598a206e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bb690 0x5598a206e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 -- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 msgr2=0x5598a20bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 -- 172.21.15.76:0/3897686729 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a1e66580 con 0x5598a206edb0 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefa59c640 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bb690 0x5598a206e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feefb59e640 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a206edb0 0x5598a206f190 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7feef40045a0 tx=0x7feef400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acb7a023e42a47d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feef0ff9640 1 -- 172.21.15.76:0/3897686729 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4006d60 con 0x5598a206edb0 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feef0ff9640 1 -- 172.21.15.76:0/3897686729 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feef400de40 con 0x5598a206edb0 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7feef0ff9640 1 -- 172.21.15.76:0/3897686729 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef40049f0 con 0x5598a206edb0 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 -- 172.21.15.76:0/3897686729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a206edb0 msgr2=0x5598a206f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.297+0000 7fef04196000 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a206edb0 0x5598a206f190 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7feef40045a0 tx=0x7feef400d520 comp rx=0 tx=0).stop 2024-09-19T04:19:12.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- 172.21.15.76:0/3897686729 shutdown_connections 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a206edb0 0x5598a206f190 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bb690 0x5598a206e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 --2- 172.21.15.76:0/3897686729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- 172.21.15.76:0/3897686729 >> 172.21.15.76:0/3897686729 conn(0x5598a209c040 msgr2=0x5598a209e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- 172.21.15.76:0/3897686729 shutdown_connections 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- 172.21.15.76:0/3897686729 wait complete. 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- start start 2024-09-19T04:19:12.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 Processor -- start 2024-09-19T04:19:12.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a206edb0 0x5598a20c1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20c2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a20bb690 0x5598a20c2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5598a1ebbd00 con 0x5598a20bb690 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5598a1e7aaa0 con 0x5598a20bad70 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5598a1e59340 con 0x5598a206edb0 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 10 client.0 _ll_register_callbacks cb 0x5598a205c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a20bb690 0x5598a20c2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a20bb690 0x5598a20c2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56026/0 (socket says 172.21.15.76:56026) 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 -- 172.21.15.76:0/822762143 learned_addr learned my addr 172.21.15.76:0/822762143 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefad9d640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a206edb0 0x5598a20c1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefa59c640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 -- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a206edb0 msgr2=0x5598a20c1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a206edb0 0x5598a20c1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 -- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 msgr2=0x5598a20c2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20c2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 -- 172.21.15.76:0/822762143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a1e62dc0 con 0x5598a20bb690 2024-09-19T04:19:12.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefa59c640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefb59e640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a20bb690 0x5598a20c2760 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x5598a1e83c90 tx=0x7feef4003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=964f4c1e06e515e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feed9ffb640 1 -- 172.21.15.76:0/822762143 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4007d00 con 0x5598a20bb690 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feed9ffb640 1 -- 172.21.15.76:0/822762143 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7feef4005040 con 0x5598a20bb690 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5598a1e59e10 con 0x5598a20bb690 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feed9ffb640 1 -- 172.21.15.76:0/822762143 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7feef4005240 con 0x5598a20bb690 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 10 client.7557 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5598a1e5f7f0 con 0x5598a20bb690 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feed9ffb640 1 -- 172.21.15.76:0/822762143 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x5598a1e59e10 con 0x5598a20bb690 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feed9ffb640 1 -- 172.21.15.76:0/822762143 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7feef40053e0 con 0x5598a20bb690 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 10 client.7557 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 10 client.7557 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 20 client.7557 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:12.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20ca0b0 0x5598a20cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 4 client.7557 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598a1ecd470 con 0x5598a20ca0b0 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefad9d640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20ca0b0 0x5598a20cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feed9ffb640 10 client.7557 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.301+0000 7feefad9d640 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20ca0b0 0x5598a20cc4a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0d7bdca247e90b0 server_cookie=dd07660fad6123d6 in_seq=0 out_seq=0 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7feed9ffb640 1 -- 172.21.15.76:0/822762143 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5598a1ecd470 con 0x5598a20ca0b0 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7feed9ffb640 10 client.7557 handle_command_reply: tid=0 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 client.7557 shutdown 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a20bb690 msgr2=0x5598a20c2760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a20bb690 0x5598a20c2760 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x5598a1e83c90 tx=0x7feef4003a80 comp rx=0 tx=0).stop 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 shutdown_connections 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20ca0b0 0x5598a20cc4a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a20bb690 0x5598a20c2760 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7feed9ffb640 0 client.7557 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:12.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a20bad70 0x5598a20c2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 --2- 172.21.15.76:0/822762143 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a206edb0 0x5598a20c1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 >> 172.21.15.76:0/822762143 conn(0x5598a209c040 msgr2=0x5598a206df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 shutdown_connections 2024-09-19T04:19:12.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 1 -- 172.21.15.76:0/822762143 wait complete. 2024-09-19T04:19:12.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.305+0000 7fef04196000 20 client.7557 trim_cache size 0 max 16384 2024-09-19T04:19:12.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.317+0000 7fef04196000 1 Processor -- start 2024-09-19T04:19:12.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.317+0000 7fef04196000 1 -- start start 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a2076da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20772e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feefc201bd0 con 0x5598a209c040 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feefc200da0 con 0x5598a206edb0 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7feefc2014a0 con 0x5598a20bad70 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a2076da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a2076da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56034/0 (socket says 172.21.15.76:56034) 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 -- 172.21.15.76:0/3998944731 learned_addr learned my addr 172.21.15.76:0/3998944731 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 -- 172.21.15.76:0/3998944731 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 msgr2=0x5598a20cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 --2- 172.21.15.76:0/3998944731 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 -- 172.21.15.76:0/3998944731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 msgr2=0x5598a20772e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 --2- 172.21.15.76:0/3998944731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20772e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 -- 172.21.15.76:0/3998944731 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a1e7d470 con 0x5598a209c040 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 --2- 172.21.15.76:0/3998944731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a2076da0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7feeec005e80 tx=0x7feeec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14214f26dd4968d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feedbfff640 1 -- 172.21.15.76:0/3998944731 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec031070 con 0x5598a209c040 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feedbfff640 1 -- 172.21.15.76:0/3998944731 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec014930 con 0x5598a209c040 2024-09-19T04:19:12.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feedbfff640 1 -- 172.21.15.76:0/3998944731 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeec028040 con 0x5598a209c040 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- 172.21.15.76:0/3998944731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 msgr2=0x5598a2076da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- 172.21.15.76:0/3998944731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a2076da0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7feeec005e80 tx=0x7feeec012000 comp rx=0 tx=0).stop 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- 172.21.15.76:0/3998944731 shutdown_connections 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- 172.21.15.76:0/3998944731 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- 172.21.15.76:0/3998944731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20772e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- 172.21.15.76:0/3998944731 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a2076da0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- 172.21.15.76:0/3998944731 >> 172.21.15.76:0/3998944731 conn(0x5598a2073770 msgr2=0x5598a20c8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- 172.21.15.76:0/3998944731 shutdown_connections 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- 172.21.15.76:0/3998944731 wait complete. 2024-09-19T04:19:12.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- start start 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 Processor -- start 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20cf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a20ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5598a1ecdfd0 con 0x5598a209c040 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feefc14a850 con 0x5598a206edb0 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5598a1e75ac0 con 0x5598a20bad70 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7fef04196000 10 client.0 _ll_register_callbacks cb 0x7feef4004420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a20ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a20ea790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56048/0 (socket says 172.21.15.76:56048) 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 -- 172.21.15.76:0/3951631078 learned_addr learned my addr 172.21.15.76:0/3951631078 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20cf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefb59e640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 -- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 msgr2=0x5598a20eacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20eacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 -- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 msgr2=0x5598a20cf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20cf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefa59c640 1 -- 172.21.15.76:0/3951631078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a1e53640 con 0x5598a209c040 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefad9d640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.321+0000 7feefb59e640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feefa59c640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a20ea790 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7feee40097c0 tx=0x7feee40f05c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a2838cd64b838d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feed9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feed9ffb640 1 -- 172.21.15.76:0/3951631078 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee40f4db0 con 0x5598a209c040 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feed9ffb640 1 -- 172.21.15.76:0/3951631078 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7feee40f9150 con 0x5598a209c040 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeec014930 con 0x5598a209c040 2024-09-19T04:19:12.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feed9ffb640 1 -- 172.21.15.76:0/3951631078 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7feee40f95e0 con 0x5598a209c040 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 10 client.7563 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5598a1e573b0 con 0x5598a209c040 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feed9ffb640 1 -- 172.21.15.76:0/3951631078 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feeec014930 con 0x5598a209c040 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feed9ffb640 1 -- 172.21.15.76:0/3951631078 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7feee40aa070 con 0x5598a209c040 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 10 client.7563 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 10 client.7563 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 20 client.7563 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20bb690 0x5598a20eb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 4 client.7563 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598a1ec77c0 con 0x5598a20bb690 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feefad9d640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20bb690 0x5598a20eb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feed9ffb640 10 client.7563 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:12.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.325+0000 7feefad9d640 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20bb690 0x5598a20eb210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccc027b183df755 server_cookie=13e15b6a10ec00af in_seq=0 out_seq=0 2024-09-19T04:19:12.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7feed9ffb640 1 -- 172.21.15.76:0/3951631078 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5598a1ec77c0 con 0x5598a20bb690 2024-09-19T04:19:12.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7feed9ffb640 10 client.7563 handle_command_reply: tid=0 2024-09-19T04:19:12.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 client.7563 shutdown 2024-09-19T04:19:12.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 msgr2=0x5598a20ea790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a20ea790 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7feee40097c0 tx=0x7feee40f05c0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 shutdown_connections 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5598a20bb690 0x5598a20eb210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5598a20bad70 0x5598a20eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7feed9ffb640 0 client.7563 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5598a209c040 0x5598a20ea790 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 --2- 172.21.15.76:0/3951631078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5598a206edb0 0x5598a20cf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 >> 172.21.15.76:0/3951631078 conn(0x5598a2073770 msgr2=0x5598a20c8500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 shutdown_connections 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 1 -- 172.21.15.76:0/3951631078 wait complete. 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef04196000 20 client.7563 trim_cache size 0 max 16384 2024-09-19T04:19:12.344 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:12.345 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:19:12.345 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:19:12.345 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:12.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feedc0678f0 msgr2=0x7feedc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feedc0678f0 0x7feedc069db0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feeec000f70 tx=0x7feeec007b80 comp rx=0 tx=0).stop 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc1520a0 msgr2=0x7feefc0b1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc1520a0 0x7feefc0b1690 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7feee4019040 tx=0x7feee401ac70 comp rx=0 tx=0).stop 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 shutdown_connections 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feedc0678f0 0x7feedc069db0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feefc15f1d0 0x7feefc15d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feefc1520a0 0x7feefc0b1690 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 --2- 172.21.15.76:0/2975715880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feefc070020 0x7feefc0996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.341+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 >> 172.21.15.76:0/2975715880 conn(0x7feefc0b1f90 msgr2=0x7feefc14f0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.345+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 shutdown_connections 2024-09-19T04:19:12.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.345+0000 7fef010da640 1 -- 172.21.15.76:0/2975715880 wait complete. 2024-09-19T04:19:12.360 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:1 flush journal 2024-09-19T04:19:12.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eae5b2640 1 Processor -- start 2024-09-19T04:19:12.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eae5b2640 1 -- start start 2024-09-19T04:19:12.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eae5b2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ea80cf620 0x7f8ea8168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eae5b2640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea80d0300 con 0x7f8ea8161a60 2024-09-19T04:19:12.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eae5b2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ea806e6c0 con 0x7f8ea80d3060 2024-09-19T04:19:12.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eae5b2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8ea80cfa90 con 0x7f8ea80cf620 2024-09-19T04:19:12.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea7fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8ea8161a60 0x7f8ea80d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45300/0 (socket says 172.21.15.76:45300) 2024-09-19T04:19:12.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea7fff640 1 -- 172.21.15.76:0/2394999669 learned_addr learned my addr 172.21.15.76:0/2394999669 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:12.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2394999669 >> v2:172.21.15.116:3300/0 conn(0x7f8ea80cf620 0x7f8ea8168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1903679380 0 0) 0x7f8e98001120 con 0x7f8ea8161a60 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e90003530 con 0x7f8ea8161a60 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 20147739 0 0) 0x7f8e90003530 con 0x7f8ea8161a60 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 >> v2:172.21.15.116:3300/0 conn(0x7f8ea80cf620 msgr2=0x7f8ea8168f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 --2- 172.21.15.76:0/2394999669 >> v2:172.21.15.116:3300/0 conn(0x7f8ea80cf620 0x7f8ea8168f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 >> v1:172.21.15.116:6789/0 conn(0x7f8ea80d3060 legacy=0x7f8ea80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea80b0f60 con 0x7f8ea8161a60 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2394999669 >> v2:172.21.15.116:3300/0 conn(0x7f8ea80cf620 0x7f8ea8168f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f8e98001850 con 0x7f8ea8161a60 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 >> v1:172.21.15.76:6789/0 conn(0x7f8ea8161a60 legacy=0x7f8ea80d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900039c0 0x7f8e90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e90004810 0x7f8e90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e900051d0 0x7f8e9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea80cfa90 con 0x7f8e900051d0 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9000c170 con 0x7f8e900039c0 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9000c370 con 0x7f8e90004810 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e900051d0 0x7f8e9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900039c0 0x7f8e90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e90004810 0x7f8e90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e90004810 msgr2=0x7f8e90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8e90004810 0x7f8e90004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900039c0 msgr2=0x7f8e90003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900039c0 0x7f8e90003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2394999669 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e90003530 con 0x7f8e900051d0 2024-09-19T04:19:12.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.477+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900039c0 0x7f8e90003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:12.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2394999669 reap_dead start 2024-09-19T04:19:12.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e900051d0 0x7f8e9000b9d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e350 tx=0x7f8ea001ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15cb16113929bbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0009de0 con 0x7f8e900051d0 2024-09-19T04:19:12.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea00039a0 con 0x7f8e900051d0 2024-09-19T04:19:12.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8ea6ffd640 1 -- 172.21.15.76:0/2394999669 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0003d90 con 0x7f8e900051d0 2024-09-19T04:19:12.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eae5b2640 1 -- 172.21.15.76:0/2394999669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e900051d0 msgr2=0x7f8e9000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e900051d0 0x7f8e9000b9d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e350 tx=0x7f8ea001ce80 comp rx=0 tx=0).stop 2024-09-19T04:19:12.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eae5b2640 1 -- 172.21.15.76:0/2394999669 shutdown_connections 2024-09-19T04:19:12.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/2394999669 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8e900051d0 0x7f8e9000b9d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eae5b2640 1 -- 172.21.15.76:0/2394999669 >> 172.21.15.76:0/2394999669 conn(0x7f8ea808bf90 msgr2=0x7f8ea809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eae5b2640 1 -- 172.21.15.76:0/2394999669 shutdown_connections 2024-09-19T04:19:12.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.481+0000 7f8eae5b2640 1 -- 172.21.15.76:0/2394999669 wait complete. 2024-09-19T04:19:12.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 Processor -- start 2024-09-19T04:19:12.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 -- start start 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900051d0 0x7f8ea80b87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea80b71a0 0x7f8ea815c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea814b5a0 0x7f8ea80d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea806d190 con 0x7f8ea814b5a0 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea806ceb0 con 0x7f8e900051d0 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea80cfc70 con 0x7f8ea80b71a0 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea814b5a0 0x7f8ea80d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea814b5a0 0x7f8ea80d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56058/0 (socket says 172.21.15.76:56058) 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 -- 172.21.15.76:0/3236043877 learned_addr learned my addr 172.21.15.76:0/3236043877 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea80b71a0 0x7f8ea815c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900051d0 0x7f8ea80b87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 -- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea80b71a0 msgr2=0x7f8ea815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea80b71a0 0x7f8ea815c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 -- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900051d0 msgr2=0x7f8ea80b87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900051d0 0x7f8ea80b87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 -- 172.21.15.76:0/3236043877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ea80b8cf0 con 0x7f8ea814b5a0 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900051d0 0x7f8ea80b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea814b5a0 0x7f8ea80d04e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8ea001c8f0 tx=0x7f8ea0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8ea4ff9640 1 -- 172.21.15.76:0/3236043877 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0009de0 con 0x7f8ea814b5a0 2024-09-19T04:19:12.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8ea4ff9640 1 -- 172.21.15.76:0/3236043877 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea00b6440 con 0x7f8ea814b5a0 2024-09-19T04:19:12.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8ea4ff9640 1 -- 172.21.15.76:0/3236043877 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00b6810 con 0x7f8ea814b5a0 2024-09-19T04:19:12.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ea80b7710 con 0x7f8ea814b5a0 2024-09-19T04:19:12.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ea80b7cd0 con 0x7f8ea814b5a0 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.485+0000 7f8ea4ff9640 1 -- 172.21.15.76:0/3236043877 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ea00b69b0 con 0x7f8ea814b5a0 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8ea4ff9640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940678c0 0x7f8e94069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8ea4ff9640 1 -- 172.21.15.76:0/3236043877 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f8ea0120c30 con 0x7f8ea814b5a0 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940678c0 0x7f8e94069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940678c0 0x7f8e94069d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000fc0 tx=0x7f8e98001d60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 Processor -- start 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- start start 2024-09-19T04:19:12.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a44870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a45190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d50392e960 con 0x55d503a44db0 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d503872900 con 0x55d503a90d70 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d503895d40 con 0x55d503a91690 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a45190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a45190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56062/0 (socket says 172.21.15.76:56062) 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2315050343 learned_addr learned my addr 172.21.15.76:0/2315050343 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a44870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 msgr2=0x55d503a44870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a44870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 msgr2=0x55d503a91150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a91150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 -- 172.21.15.76:0/2315050343 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50383c580 con 0x55d503a44db0 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a45190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00f0e00 tx=0x7f8ea00f0890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4fd1255b1c7c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8e8dffb640 1 -- 172.21.15.76:0/2315050343 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00f5220 con 0x55d503a44db0 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8e8dffb640 1 -- 172.21.15.76:0/2315050343 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea00f9a50 con 0x55d503a44db0 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8e8dffb640 1 -- 172.21.15.76:0/2315050343 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00ed4e0 con 0x55d503a44db0 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- 172.21.15.76:0/2315050343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 msgr2=0x55d503a45190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a45190 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f8ea00f0e00 tx=0x7f8ea00f0890 comp rx=0 tx=0).stop 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- 172.21.15.76:0/2315050343 shutdown_connections 2024-09-19T04:19:12.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a45190 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a44870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 --2- 172.21.15.76:0/2315050343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a91150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- 172.21.15.76:0/2315050343 >> 172.21.15.76:0/2315050343 conn(0x55d503a72040 msgr2=0x55d503a74430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- 172.21.15.76:0/2315050343 shutdown_connections 2024-09-19T04:19:12.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.489+0000 7f8eb166e000 1 -- 172.21.15.76:0/2315050343 wait complete. 2024-09-19T04:19:12.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- start start 2024-09-19T04:19:12.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 Processor -- start 2024-09-19T04:19:12.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a4ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a4d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a72420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d503891d00 con 0x55d503a44db0 2024-09-19T04:19:12.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d503850aa0 con 0x55d503a90d70 2024-09-19T04:19:12.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d50382f340 con 0x55d503a91690 2024-09-19T04:19:12.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a4ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 10 client.0 _ll_register_callbacks cb 0x55d503a32760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a4ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56078/0 (socket says 172.21.15.76:56078) 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1641072575 learned_addr learned my addr 172.21.15.76:0/1641072575 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a4d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a72420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 msgr2=0x55d503a72420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a72420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 msgr2=0x55d503a4d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a4d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1641072575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d503838dc0 con 0x55d503a44db0 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a4d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a4ce00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8e98012ec0 tx=0x7f8e98001e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e934a13957dc1ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1641072575 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e98029060 con 0x55d503a44db0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1641072575 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f8e98024ca0 con 0x55d503a44db0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ea00f9a50 con 0x55d503a44db0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1641072575 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8e980155f0 con 0x55d503a44db0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 10 client.7575 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d5038357f0 con 0x55d503a44db0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1641072575 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea00f9a50 con 0x55d503a44db0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1641072575 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f8e98022070 con 0x55d503a44db0 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 10 client.7575 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 10 client.7575 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 20 client.7575 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:12.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a97ad0 0x55d503a97ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 4 client.7575 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50382fe10 con 0x55d503a97ad0 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a97ad0 0x55d503a97ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 10 client.7575 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a97ad0 0x55d503a97ee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ecd51be782e54c7c server_cookie=e51e074c186dbd1f in_seq=0 out_seq=0 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1641072575 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d50382fe10 con 0x55d503a97ad0 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 10 client.7575 handle_command_reply: tid=0 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 client.7575 shutdown 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 msgr2=0x55d503a4ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a4ce00 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f8e98012ec0 tx=0x7f8e98001e20 comp rx=0 tx=0).stop 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 shutdown_connections 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a97ad0 0x55d503a97ee0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a72420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a90d70 0x55d503a4d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8e86ffd640 0 client.7575 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:12.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1641072575 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503a4ce00 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 >> 172.21.15.76:0/1641072575 conn(0x55d503a72040 msgr2=0x55d503a43f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 shutdown_connections 2024-09-19T04:19:12.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 1 -- 172.21.15.76:0/1641072575 wait complete. 2024-09-19T04:19:12.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.493+0000 7f8eb166e000 20 client.7575 trim_cache size 0 max 16384 2024-09-19T04:19:12.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 Processor -- start 2024-09-19T04:19:12.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 -- start start 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a44db0 0x55d503a70d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a90d70 0x55d503a71270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a4bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8201910 con 0x55d503a90d70 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea8200ae0 con 0x55d503a44db0 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8ea82011e0 con 0x55d503a91690 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8ea77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a90d70 0x55d503a71270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8ea77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a90d70 0x55d503a71270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56090/0 (socket says 172.21.15.76:56090) 2024-09-19T04:19:12.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.509+0000 7f8ea77fe640 1 -- 172.21.15.76:0/477288152 learned_addr learned my addr 172.21.15.76:0/477288152 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 -- 172.21.15.76:0/477288152 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 msgr2=0x55d503a4bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/477288152 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a4bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 -- 172.21.15.76:0/477288152 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a44db0 msgr2=0x55d503a70d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/477288152 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a44db0 0x55d503a70d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 -- 172.21.15.76:0/477288152 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d503853470 con 0x55d503a90d70 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/477288152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a90d70 0x55d503a71270 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c001890 tx=0x7f8e9c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87aef00a3d68d79c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e8d7fa640 1 -- 172.21.15.76:0/477288152 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c024070 con 0x55d503a90d70 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e8d7fa640 1 -- 172.21.15.76:0/477288152 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e9c001c70 con 0x55d503a90d70 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e8d7fa640 1 -- 172.21.15.76:0/477288152 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0052f0 con 0x55d503a90d70 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/477288152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a90d70 msgr2=0x55d503a71270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- 172.21.15.76:0/477288152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a90d70 0x55d503a71270 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c001890 tx=0x7f8e9c001510 comp rx=0 tx=0).stop 2024-09-19T04:19:12.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/477288152 shutdown_connections 2024-09-19T04:19:12.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- 172.21.15.76:0/477288152 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a91690 0x55d503a4bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- 172.21.15.76:0/477288152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a90d70 0x55d503a71270 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- 172.21.15.76:0/477288152 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a44db0 0x55d503a70d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/477288152 >> 172.21.15.76:0/477288152 conn(0x55d503a72040 msgr2=0x55d503a8f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/477288152 shutdown_connections 2024-09-19T04:19:12.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/477288152 wait complete. 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- start start 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 Processor -- start 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503b04740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a90d70 0x55d503b04c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a91690 0x55d503a6fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d50389a370 con 0x55d503a44db0 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8ea81ffbd0 con 0x55d503a91690 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d5039520c0 con 0x55d503a90d70 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 10 client.0 _ll_register_callbacks cb 0x7f8e98000ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:12.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503b04740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503b04740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:56098/0 (socket says 172.21.15.76:56098) 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1154813980 learned_addr learned my addr 172.21.15.76:0/1154813980 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a90d70 msgr2=0x55d503b04c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a90d70 0x55d503b04c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eacb4f640 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a91690 0x55d503a6fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a90d70 0x55d503b04c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a91690 msgr2=0x55d503a6fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a91690 0x55d503a6fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 -- 172.21.15.76:0/1154813980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d503829640 con 0x55d503a44db0 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea7fff640 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503b04740 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8e98001dd0 tx=0x7f8e98016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feb52c4c5617066 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1154813980 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9803d060 con 0x55d503a44db0 2024-09-19T04:19:12.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1154813980 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f8e98010040 con 0x55d503a44db0 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9c001c70 con 0x55d503a44db0 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1154813980 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8e9801c020 con 0x55d503a44db0 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 10 client.7581 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d50382d3b0 con 0x55d503a44db0 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1154813980 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e9c001c70 con 0x55d503a44db0 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1154813980 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f8e9802f640 con 0x55d503a44db0 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 10 client.7581 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 10 client.7581 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 20 client.7581 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a70260 0x55d503a70670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 4 client.7581 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5038a3470 con 0x55d503a70260 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.513+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a70260 0x55d503a70670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.517+0000 7f8ea77fe640 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a70260 0x55d503a70670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cc679fde9cc673f9 server_cookie=5bc0c1e428f15e44 in_seq=0 out_seq=0 2024-09-19T04:19:12.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.517+0000 7f8e86ffd640 10 client.7581 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:12.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.525+0000 7f8e86ffd640 1 -- 172.21.15.76:0/1154813980 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5038a3470 con 0x55d503a70260 2024-09-19T04:19:12.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.525+0000 7f8e86ffd640 10 client.7581 handle_command_reply: tid=0 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.525+0000 7f8eb166e000 1 client.7581 shutdown 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.525+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 msgr2=0x55d503b04740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.525+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503b04740 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8e98001dd0 tx=0x7f8e98016000 comp rx=0 tx=0).stop 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 shutdown_connections 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x55d503a70260 0x55d503a70670 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d503a91690 0x55d503a6fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d503a90d70 0x55d503b04c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8e86ffd640 0 client.7581 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 --2- 172.21.15.76:0/1154813980 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d503a44db0 0x55d503b04740 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 >> 172.21.15.76:0/1154813980 conn(0x55d503a72040 msgr2=0x55d503a498c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 shutdown_connections 2024-09-19T04:19:12.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 1 -- 172.21.15.76:0/1154813980 wait complete. 2024-09-19T04:19:12.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eb166e000 20 client.7581 trim_cache size 0 max 16384 2024-09-19T04:19:12.530 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:12.530 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:19:12.530 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:19:12.530 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940678c0 msgr2=0x7f8e94069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940678c0 0x7f8e94069d80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000fc0 tx=0x7f8e98001d60 comp rx=0 tx=0).stop 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea814b5a0 msgr2=0x7f8ea80d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea814b5a0 0x7f8ea80d04e0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8ea001c8f0 tx=0x7f8ea0009750 comp rx=0 tx=0).stop 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 shutdown_connections 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8e940678c0 0x7f8e94069d80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8ea814b5a0 0x7f8ea80d04e0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8ea80b71a0 0x7f8ea815c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 --2- 172.21.15.76:0/3236043877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e900051d0 0x7f8ea80b87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 >> 172.21.15.76:0/3236043877 conn(0x7f8ea808bf90 msgr2=0x7f8ea80b4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 shutdown_connections 2024-09-19T04:19:12.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.529+0000 7f8eae5b2640 1 -- 172.21.15.76:0/3236043877 wait complete. 2024-09-19T04:19:12.546 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.6:2 flush journal 2024-09-19T04:19:12.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 Processor -- start 2024-09-19T04:19:12.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 -- start start 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f067c1587a0 0x7f067c0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f067c0b37d0 0x7f067c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c06e4b0 0x7f067c161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f067c0b3c40 con 0x7f067c0b37d0 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f067c06f870 con 0x7f067c1587a0 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f06841bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f067c06ec60 con 0x7f067c06e4b0 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f067c0b37d0 0x7f067c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f067c0b37d0 0x7f067c06df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43734/0 (socket says 172.21.15.76:43734) 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 -- 172.21.15.76:0/154830410 learned_addr learned my addr 172.21.15.76:0/154830410 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681f58640 1 --2- 172.21.15.76:0/154830410 >> v2:172.21.15.116:3300/0 conn(0x7f067c1587a0 0x7f067c0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0682759640 1 --2- 172.21.15.76:0/154830410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c06e4b0 0x7f067c161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 -- 172.21.15.76:0/154830410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c06e4b0 msgr2=0x7f067c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 --2- 172.21.15.76:0/154830410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c06e4b0 0x7f067c161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 -- 172.21.15.76:0/154830410 >> v2:172.21.15.116:3300/0 conn(0x7f067c1587a0 msgr2=0x7f067c0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 --2- 172.21.15.76:0/154830410 >> v2:172.21.15.116:3300/0 conn(0x7f067c1587a0 0x7f067c0b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681757640 1 -- 172.21.15.76:0/154830410 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c06ef40 con 0x7f067c0b37d0 2024-09-19T04:19:12.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0681f58640 1 --2- 172.21.15.76:0/154830410 >> v2:172.21.15.116:3300/0 conn(0x7f067c1587a0 0x7f067c0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.661+0000 7f0682759640 1 --2- 172.21.15.76:0/154830410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c06e4b0 0x7f067c161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f0681757640 1 --2- 172.21.15.76:0/154830410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c06df70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f067c071710 tx=0x7f067001c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec867309abbee1ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f0680f56640 1 -- 172.21.15.76:0/154830410 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700b0070 con 0x7f067c0b37d0 2024-09-19T04:19:12.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f0680f56640 1 -- 172.21.15.76:0/154830410 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06700ab600 con 0x7f067c0b37d0 2024-09-19T04:19:12.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f0680f56640 1 -- 172.21.15.76:0/154830410 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700ab9d0 con 0x7f067c0b37d0 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 -- 172.21.15.76:0/154830410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 msgr2=0x7f067c06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 --2- 172.21.15.76:0/154830410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c06df70 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f067c071710 tx=0x7f067001c230 comp rx=0 tx=0).stop 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 -- 172.21.15.76:0/154830410 shutdown_connections 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 --2- 172.21.15.76:0/154830410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c06e4b0 0x7f067c161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 --2- 172.21.15.76:0/154830410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c06df70 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 --2- 172.21.15.76:0/154830410 >> v2:172.21.15.116:3300/0 conn(0x7f067c1587a0 0x7f067c0b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 -- 172.21.15.76:0/154830410 >> 172.21.15.76:0/154830410 conn(0x7f067c0b1f90 msgr2=0x7f067c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 -- 172.21.15.76:0/154830410 shutdown_connections 2024-09-19T04:19:12.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.665+0000 7f06841bc640 1 -- 172.21.15.76:0/154830410 wait complete. 2024-09-19T04:19:12.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 Processor -- start 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 -- start start 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f067c06e4b0 0x7f067c138700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c1587a0 0x7f067c154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f067c158b80 con 0x7f067c0b37d0 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f067c072160 con 0x7f067c06e4b0 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f067c153000 con 0x7f067c1587a0 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43742/0 (socket says 172.21.15.76:43742) 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 -- 172.21.15.76:0/119426909 learned_addr learned my addr 172.21.15.76:0/119426909 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681f58640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f067c06e4b0 0x7f067c138700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0682759640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c1587a0 0x7f067c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 -- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c1587a0 msgr2=0x7f067c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c1587a0 0x7f067c154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 -- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f067c06e4b0 msgr2=0x7f067c138700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f067c06e4b0 0x7f067c138700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 -- 172.21.15.76:0/119426909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c153250 con 0x7f067c0b37d0 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681757640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c157ce0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06700ae040 tx=0x7f0670018fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f067a7fc640 1 -- 172.21.15.76:0/119426909 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700b9070 con 0x7f067c0b37d0 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f067a7fc640 1 -- 172.21.15.76:0/119426909 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06700b4600 con 0x7f067c0b37d0 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f067a7fc640 1 -- 172.21.15.76:0/119426909 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06700b0040 con 0x7f067c0b37d0 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c153530 con 0x7f067c0b37d0 2024-09-19T04:19:12.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c153af0 con 0x7f067c0b37d0 2024-09-19T04:19:12.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f067a7fc640 1 -- 172.21.15.76:0/119426909 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f06700bc070 con 0x7f067c0b37d0 2024-09-19T04:19:12.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f067a7fc640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f06680678c0 0x7f0668069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f067a7fc640 1 -- 172.21.15.76:0/119426909 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f067011f010 con 0x7f067c0b37d0 2024-09-19T04:19:12.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.669+0000 7f0681f58640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f06680678c0 0x7f0668069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f06680678c0 0x7f0668069d80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0674008ee0 tx=0x7f0674008430 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 Processor -- start 2024-09-19T04:19:12.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- start start 2024-09-19T04:19:12.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cacdfd70 0x5611cace0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cace0690 0x5611cac93870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac94190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5611cab7d960 con 0x5611cacdfd70 2024-09-19T04:19:12.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5611caac1900 con 0x5611cac93db0 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5611caae4d40 con 0x5611cace0690 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cacdfd70 0x5611cace0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cacdfd70 0x5611cace0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43744/0 (socket says 172.21.15.76:43744) 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 -- 172.21.15.76:0/644033580 learned_addr learned my addr 172.21.15.76:0/644033580 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681757640 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cace0690 0x5611cac93870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0682759640 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac94190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 -- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cace0690 msgr2=0x5611cac93870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cace0690 0x5611cac93870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 -- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 msgr2=0x5611cac94190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac94190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 -- 172.21.15.76:0/644033580 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611caa8b580 con 0x5611cacdfd70 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0682759640 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac94190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0681f58640 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cacdfd70 0x5611cace0150 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0674008fd0 tx=0x7f0674010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31133dd0083203c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f06637fe640 1 -- 172.21.15.76:0/644033580 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674020070 con 0x5611cacdfd70 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f06637fe640 1 -- 172.21.15.76:0/644033580 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067400e9b0 con 0x5611cacdfd70 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f06637fe640 1 -- 172.21.15.76:0/644033580 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f067400ed60 con 0x5611cacdfd70 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- 172.21.15.76:0/644033580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cacdfd70 msgr2=0x5611cace0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cacdfd70 0x5611cace0150 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f0674008fd0 tx=0x7f0674010000 comp rx=0 tx=0).stop 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- 172.21.15.76:0/644033580 shutdown_connections 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac94190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cace0690 0x5611cac93870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 --2- 172.21.15.76:0/644033580 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cacdfd70 0x5611cace0150 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- 172.21.15.76:0/644033580 >> 172.21.15.76:0/644033580 conn(0x5611cacc1040 msgr2=0x5611cacc3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- 172.21.15.76:0/644033580 shutdown_connections 2024-09-19T04:19:12.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- 172.21.15.76:0/644033580 wait complete. 2024-09-19T04:19:12.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 -- start start 2024-09-19T04:19:12.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.673+0000 7f0687278000 1 Processor -- start 2024-09-19T04:19:12.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cace6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cace7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611cace7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5611caae0d00 con 0x5611cac93db0 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5611caa9faa0 con 0x5611cace0690 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5611caa7e340 con 0x5611cacdfd70 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 10 client.0 _ll_register_callbacks cb 0x5611cac81760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cace6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cace6ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43750/0 (socket says 172.21.15.76:43750) 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 -- 172.21.15.76:0/2976929612 learned_addr learned my addr 172.21.15.76:0/2976929612 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 -- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 msgr2=0x5611cace7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681757640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cace7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cace7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 -- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 msgr2=0x5611cace7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0682759640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611cace7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611cace7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 -- 172.21.15.76:0/2976929612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611caa87dc0 con 0x5611cac93db0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0682759640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611cace7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681f58640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cace6ce0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x5611caaa8c90 tx=0x7f0674014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93511b91968fcf36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0660ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0660ff9640 1 -- 172.21.15.76:0/2976929612 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674020050 con 0x5611cac93db0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0660ff9640 1 -- 172.21.15.76:0/2976929612 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f0674030cd0 con 0x5611cac93db0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f067400e9b0 con 0x5611cac93db0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0660ff9640 1 -- 172.21.15.76:0/2976929612 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f0674025660 con 0x5611cac93db0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 10 client.7593 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5611caa847f0 con 0x5611cac93db0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0660ff9640 1 -- 172.21.15.76:0/2976929612 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f067400e9b0 con 0x5611cac93db0 2024-09-19T04:19:12.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0660ff9640 1 -- 172.21.15.76:0/2976929612 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f0674037050 con 0x5611cac93db0 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 10 client.7593 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 10 client.7593 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 20 client.7593 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacef0b0 0x5611cacf14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 4 client.7593 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611caa7ee10 con 0x5611cacef0b0 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681757640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacef0b0 0x5611cacf14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0681757640 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacef0b0 0x5611cacf14a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3e6e403076b822db server_cookie=40435b7b0b49ab12 in_seq=0 out_seq=0 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.677+0000 7f0660ff9640 10 client.7593 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0660ff9640 1 -- 172.21.15.76:0/2976929612 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5611caa7ee10 con 0x5611cacef0b0 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0660ff9640 10 client.7593 handle_command_reply: tid=0 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 client.7593 shutdown 2024-09-19T04:19:12.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 msgr2=0x5611cace6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cace6ce0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x5611caaa8c90 tx=0x7f0674014000 comp rx=0 tx=0).stop 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 shutdown_connections 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacef0b0 0x5611cacf14a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611cace7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cace7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 --2- 172.21.15.76:0/2976929612 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cace6ce0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0660ff9640 0 client.7593 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 >> 172.21.15.76:0/2976929612 conn(0x5611cacc1040 msgr2=0x5611cac92f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 shutdown_connections 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 1 -- 172.21.15.76:0/2976929612 wait complete. 2024-09-19T04:19:12.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.681+0000 7f0687278000 20 client.7593 trim_cache size 0 max 16384 2024-09-19T04:19:12.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 Processor -- start 2024-09-19T04:19:12.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- start start 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac9bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cac9c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cace0690 0x5611cace8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f067c204080 con 0x5611cace0690 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f067c203250 con 0x5611cac93db0 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f067c203950 con 0x5611cacdfd70 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cace0690 0x5611cace8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cace0690 0x5611cace8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43756/0 (socket says 172.21.15.76:43756) 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 -- 172.21.15.76:0/1626545115 learned_addr learned my addr 172.21.15.76:0/1626545115 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0681757640 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cac9c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0681f58640 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac9bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 -- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 msgr2=0x5611cac9c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cac9c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 -- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 msgr2=0x5611cac9bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac9bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 -- 172.21.15.76:0/1626545115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611caaa2470 con 0x5611cace0690 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0681757640 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cac9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0681f58640 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac9bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0682759640 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cace0690 0x5611cace8ab0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f066c00d980 tx=0x7f066c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623b37dab2b4a1aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0662ffd640 1 -- 172.21.15.76:0/1626545115 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c00fd50 con 0x5611cace0690 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0662ffd640 1 -- 172.21.15.76:0/1626545115 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066c012420 con 0x5611cace0690 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0662ffd640 1 -- 172.21.15.76:0/1626545115 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c012810 con 0x5611cace0690 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- 172.21.15.76:0/1626545115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cace0690 msgr2=0x5611cace8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cace0690 0x5611cace8ab0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f066c00d980 tx=0x7f066c00de40 comp rx=0 tx=0).stop 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- 172.21.15.76:0/1626545115 shutdown_connections 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cace0690 0x5611cace8ab0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cac9c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- 172.21.15.76:0/1626545115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cac93db0 0x5611cac9bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- 172.21.15.76:0/1626545115 >> 172.21.15.76:0/1626545115 conn(0x5611cacc1040 msgr2=0x5611cac98820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- 172.21.15.76:0/1626545115 shutdown_connections 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- 172.21.15.76:0/1626545115 wait complete. 2024-09-19T04:19:12.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- start start 2024-09-19T04:19:12.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 Processor -- start 2024-09-19T04:19:12.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cad534b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cad539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611caceb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5611caae9370 con 0x5611cac93db0 2024-09-19T04:19:12.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5611caa7e480 con 0x5611cace0690 2024-09-19T04:19:12.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5611cab24780 con 0x5611cacdfd70 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0687278000 10 client.0 _ll_register_callbacks cb 0x7f067400f670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0681f58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cad534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0681f58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cad534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43770/0 (socket says 172.21.15.76:43770) 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.697+0000 7f0681f58640 1 -- 172.21.15.76:0/2680501256 learned_addr learned my addr 172.21.15.76:0/2680501256 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0682759640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611caceb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681757640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cad539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681f58640 1 -- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 msgr2=0x5611cad539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681f58640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cad539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681f58640 1 -- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 msgr2=0x5611caceb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681f58640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611caceb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681f58640 1 -- 172.21.15.76:0/2680501256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611caa78640 con 0x5611cac93db0 2024-09-19T04:19:12.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0682759640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611caceb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681757640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cad539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681f58640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cad534b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0674008810 tx=0x7f0674004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bcb3f71aa517ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0660ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0660ff9640 1 -- 172.21.15.76:0/2680501256 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0674020040 con 0x5611cac93db0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0660ff9640 1 -- 172.21.15.76:0/2680501256 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f0674025c20 con 0x5611cac93db0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f066c012420 con 0x5611cac93db0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0660ff9640 1 -- 172.21.15.76:0/2680501256 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f0674049050 con 0x5611cac93db0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 10 client.7599 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5611caa7c3b0 con 0x5611cac93db0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0660ff9640 1 -- 172.21.15.76:0/2680501256 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f066c012420 con 0x5611cac93db0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0660ff9640 1 -- 172.21.15.76:0/2680501256 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f067404c020 con 0x5611cac93db0 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 10 client.7599 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 10 client.7599 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-19T04:19:12.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 20 client.7599 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:12.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacebde0 0x5611cacec1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:12.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 4 client.7599 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:19:12.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611caaf2470 con 0x5611cacebde0 2024-09-19T04:19:12.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681757640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacebde0 0x5611cacec1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:12.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0660ff9640 10 client.7599 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:12.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.701+0000 7f0681757640 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacebde0 0x5611cacec1f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=21247207bae371f6 server_cookie=717a44a3de79800d in_seq=0 out_seq=0 2024-09-19T04:19:12.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.713+0000 7f0660ff9640 1 -- 172.21.15.76:0/2680501256 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611caaf2470 con 0x5611cacebde0 2024-09-19T04:19:12.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.713+0000 7f0660ff9640 10 client.7599 handle_command_reply: tid=0 2024-09-19T04:19:12.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.713+0000 7f0687278000 1 client.7599 shutdown 2024-09-19T04:19:12.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.713+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 msgr2=0x5611cad534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.713+0000 7f0687278000 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cad534b0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0674008810 tx=0x7f0674004ac0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 shutdown_connections 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5611cacebde0 0x5611cacec1f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5611cace0690 0x5611caceb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5611cacdfd70 0x5611cad539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0660ff9640 0 client.7599 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 --2- 172.21.15.76:0/2680501256 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5611cac93db0 0x5611cad534b0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 >> 172.21.15.76:0/2680501256 conn(0x5611cacc1040 msgr2=0x5611cac9ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 shutdown_connections 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 1 -- 172.21.15.76:0/2680501256 wait complete. 2024-09-19T04:19:12.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f0687278000 20 client.7599 trim_cache size 0 max 16384 2024-09-19T04:19:12.718 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:12.718 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:19:12.718 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:19:12.718 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f06680678c0 msgr2=0x7f0668069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f06680678c0 0x7f0668069d80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0674008ee0 tx=0x7f0674008430 comp rx=0 tx=0).stop 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 msgr2=0x7f067c157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c157ce0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f06700ae040 tx=0x7f0670018fd0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 shutdown_connections 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f06680678c0 0x7f0668069d80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f067c1587a0 0x7f067c154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f067c0b37d0 0x7f067c157ce0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 --2- 172.21.15.76:0/119426909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f067c06e4b0 0x7f067c138700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:12.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 >> 172.21.15.76:0/119426909 conn(0x7f067c0b1f90 msgr2=0x7f067c0b0660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:12.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 shutdown_connections 2024-09-19T04:19:12.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12.717+0000 7f06841bc640 1 -- 172.21.15.76:0/119426909 wait complete. 2024-09-19T04:19:12.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:12.734 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-19T04:19:12.765 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/ 2024-09-19T04:19:12.765 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4 2024-09-19T04:19:12.766 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5 2024-09-19T04:19:12.766 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6 2024-09-19T04:19:12.766 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-19T04:19:12.766 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-19T04:19:12.766 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:19:12.766 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:19:12.766 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:19:12.766 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:12.766 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-19T04:19:12.767 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:19:12.867 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627778 2024-09-19T04:19:12.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:12.877 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:19:12.877 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:19:12.877 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:19:12.877 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:12.877 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-19T04:19:12.878 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:19:12.980 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627779 2024-09-19T04:19:12.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:12.989 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:19:12.989 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:19:12.989 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:19:12.990 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:12.990 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-19T04:19:12.990 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:19:13.095 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627780 2024-09-19T04:19:13.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:13.104 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:19:13.105 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:19:13.105 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:19:13.105 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:13.105 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-19T04:19:13.105 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:19:13.208 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627781 2024-09-19T04:19:13.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:13.217 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:19:13.217 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:19:13.217 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:19:13.217 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:13.217 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-19T04:19:13.217 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:19:13.319 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627782 2024-09-19T04:19:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:13.329 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:19:13.329 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:19:13.329 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:19:13.330 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:13.330 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-19T04:19:13.330 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:19:13.431 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627783 2024-09-19T04:19:13.434 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:13.441 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-19T04:19:13.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 Processor -- start 2024-09-19T04:19:13.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- start start 2024-09-19T04:19:13.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1a01d0 0x55615f260070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1d2dd0 0x55615f273380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f2738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55615f1d3550 con 0x55615f25dbb0 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55615f0f11b0 con 0x55615f1a01d0 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55615f1d37f0 con 0x55615f1d2dd0 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f2738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f2738c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43800/0 (socket says 172.21.15.76:43800) 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 -- 172.21.15.76:0/594790897 learned_addr learned my addr 172.21.15.76:0/594790897 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddbf4e9640 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1d2dd0 0x55615f273380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddbfcea640 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1a01d0 0x55615f260070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 -- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1d2dd0 msgr2=0x55615f273380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1d2dd0 0x55615f273380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 -- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1a01d0 msgr2=0x55615f260070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1a01d0 0x55615f260070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 -- 172.21.15.76:0/594790897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55615f25d1b0 con 0x55615f25dbb0 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddbfcea640 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1a01d0 0x55615f260070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc04eb640 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f2738c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fddb8002690 tx=0x7fddb8002b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff8af989fc08d46 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddbe4e7640 1 -- 172.21.15.76:0/594790897 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8016070 con 0x55615f25dbb0 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddbe4e7640 1 -- 172.21.15.76:0/594790897 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb800d6b0 con 0x55615f25dbb0 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddbe4e7640 1 -- 172.21.15.76:0/594790897 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb800daa0 con 0x55615f25dbb0 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- 172.21.15.76:0/594790897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 msgr2=0x55615f2738c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f2738c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fddb8002690 tx=0x7fddb8002b50 comp rx=0 tx=0).stop 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- 172.21.15.76:0/594790897 shutdown_connections 2024-09-19T04:19:13.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f2738c0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1d2dd0 0x55615f273380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 --2- 172.21.15.76:0/594790897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1a01d0 0x55615f260070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- 172.21.15.76:0/594790897 >> 172.21.15.76:0/594790897 conn(0x55615f1a0dc0 msgr2=0x55615f27a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- 172.21.15.76:0/594790897 shutdown_connections 2024-09-19T04:19:13.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- 172.21.15.76:0/594790897 wait complete. 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 Processor -- start 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.517+0000 7fddc21d7080 1 -- start start 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1a01d0 0x55615f272a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1d2dd0 0x55615f255bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f256130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55615f25b390 con 0x55615f25dbb0 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55615f25b630 con 0x55615f1d2dd0 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55615f25b8d0 con 0x55615f1a01d0 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f256130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f256130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43802/0 (socket says 172.21.15.76:43802) 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 -- 172.21.15.76:0/1788265236 learned_addr learned my addr 172.21.15.76:0/1788265236 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbfcea640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1a01d0 0x55615f272a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbf4e9640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1d2dd0 0x55615f255bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 -- 172.21.15.76:0/1788265236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1a01d0 msgr2=0x55615f272a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55615f1a01d0 0x55615f272a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 -- 172.21.15.76:0/1788265236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1d2dd0 msgr2=0x55615f255bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55615f1d2dd0 0x55615f255bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 -- 172.21.15.76:0/1788265236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55615f275280 con 0x55615f25dbb0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc04eb640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f256130 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fddb800dac0 tx=0x7fddb8002370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbd4e5640 1 -- 172.21.15.76:0/1788265236 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8021020 con 0x55615f25dbb0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbd4e5640 1 -- 172.21.15.76:0/1788265236 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb8006d50 con 0x55615f25dbb0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55615f275520 con 0x55615f25dbb0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbd4e5640 1 -- 172.21.15.76:0/1788265236 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb8016030 con 0x55615f25dbb0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55615f275a60 con 0x55615f25dbb0 2024-09-19T04:19:13.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbd4e5640 1 -- 172.21.15.76:0/1788265236 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fddb8024020 con 0x55615f25dbb0 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbd4e5640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdda00662a0 0x7fdda0068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbd4e5640 1 -- 172.21.15.76:0/1788265236 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fddb8085fa0 con 0x55615f25dbb0 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbfcea640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdda00662a0 0x7fdda0068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55615f274d70 0x55615f2582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 13.12 13:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x55615f258820 con 0x55615f274d70 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbf4e9640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55615f274d70 0x55615f2582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbfcea640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdda00662a0 0x7fdda0068760 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009750 tx=0x7fddb00059a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.521+0000 7fddbf4e9640 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55615f274d70 0x55615f2582e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddbf4e9640 1 -- 172.21.15.76:0/1788265236 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v93'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fdda80a8070 con 0x55615f274d70 2024-09-19T04:19:13.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55615f274d70 msgr2=0x55615f2582e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55615f274d70 0x55615f2582e0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdda00662a0 msgr2=0x7fdda0068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdda00662a0 0x7fdda0068760 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009750 tx=0x7fddb00059a0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 msgr2=0x55615f256130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 --2- 172.21.15.76:0/1788265236 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55615f25dbb0 0x55615f256130 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fddb800dac0 tx=0x7fddb8002370 comp rx=0 tx=0).stop 2024-09-19T04:19:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc04eb640 1 -- 172.21.15.76:0/1788265236 reap_dead start 2024-09-19T04:19:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 shutdown_connections 2024-09-19T04:19:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 >> 172.21.15.76:0/1788265236 conn(0x55615f1a0dc0 msgr2=0x55615f2609c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 shutdown_connections 2024-09-19T04:19:13.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.525+0000 7fddc21d7080 1 -- 172.21.15.76:0/1788265236 wait complete. 2024-09-19T04:19:13.529 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-19T04:19:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 Processor -- start 2024-09-19T04:19:13.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 -- start start 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f60d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f628230 0x55cf1f6b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f6b61b0 0x55cf1f6b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cf1f6b7570 con 0x55cf1f628230 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cf1f5321b0 con 0x55cf1f611bb0 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d12a2d080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55cf1f6b76f0 con 0x55cf1f6b61b0 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d0fd3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f628230 0x55cf1f6b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d0fd3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f628230 0x55cf1f6b12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43826/0 (socket says 172.21.15.76:43826) 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.605+0000 7f2d0fd3f640 1 -- 172.21.15.76:0/3875424219 learned_addr learned my addr 172.21.15.76:0/3875424219 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10540640 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f60d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f6b61b0 0x55cf1f6b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0fd3f640 1 -- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f6b61b0 msgr2=0x55cf1f6b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0fd3f640 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f6b61b0 0x55cf1f6b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0fd3f640 1 -- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 msgr2=0x55cf1f60d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0fd3f640 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f60d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0fd3f640 1 -- 172.21.15.76:0/3875424219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf1f747fe0 con 0x55cf1f628230 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10540640 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f60d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0fd3f640 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f628230 0x55cf1f6b12a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8001e70 tx=0x7f2cf800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e65ae6ab1bfa48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0ed3d640 1 -- 172.21.15.76:0/3875424219 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8012020 con 0x55cf1f628230 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0ed3d640 1 -- 172.21.15.76:0/3875424219 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2cf800e690 con 0x55cf1f628230 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0ed3d640 1 -- 172.21.15.76:0/3875424219 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800ea80 con 0x55cf1f628230 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- 172.21.15.76:0/3875424219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f628230 msgr2=0x55cf1f6b12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f628230 0x55cf1f6b12a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f2cf8001e70 tx=0x7f2cf800b790 comp rx=0 tx=0).stop 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- 172.21.15.76:0/3875424219 shutdown_connections 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f6b61b0 0x55cf1f6b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f628230 0x55cf1f6b12a0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/3875424219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f60d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- 172.21.15.76:0/3875424219 >> 172.21.15.76:0/3875424219 conn(0x55cf1f60c220 msgr2=0x55cf1f6114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- 172.21.15.76:0/3875424219 shutdown_connections 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- 172.21.15.76:0/3875424219 wait complete. 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 Processor -- start 2024-09-19T04:19:13.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- start start 2024-09-19T04:19:13.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f6ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f628230 0x55cf1f6b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f6b61b0 0x55cf1f6b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55cf1f748520 con 0x55cf1f6b61b0 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55cf1f748280 con 0x55cf1f611bb0 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55cf1f7487c0 con 0x55cf1f628230 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f6b61b0 0x55cf1f6b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f6b61b0 0x55cf1f6b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43832/0 (socket says 172.21.15.76:43832) 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 -- 172.21.15.76:0/1326089883 learned_addr learned my addr 172.21.15.76:0/1326089883 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0fd3f640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f628230 0x55cf1f6b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10540640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f6ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 -- 172.21.15.76:0/1326089883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f628230 msgr2=0x55cf1f6b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55cf1f628230 0x55cf1f6b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 -- 172.21.15.76:0/1326089883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 msgr2=0x55cf1f6ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55cf1f611bb0 0x55cf1f6ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 -- 172.21.15.76:0/1326089883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf1f6a7f10 con 0x55cf1f6b61b0 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d10d41640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f6b61b0 0x55cf1f6b2680 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ad040 tx=0x7f2d00018150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0dd3b640 1 -- 172.21.15.76:0/1326089883 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000ba020 con 0x55cf1f6b61b0 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0dd3b640 1 -- 172.21.15.76:0/1326089883 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d000b5470 con 0x55cf1f6b61b0 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55cf1f6a81b0 con 0x55cf1f6b61b0 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d0dd3b640 1 -- 172.21.15.76:0/1326089883 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d000af040 con 0x55cf1f6b61b0 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.609+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf1f6a86f0 con 0x55cf1f6b61b0 2024-09-19T04:19:13.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d0dd3b640 1 -- 172.21.15.76:0/1326089883 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2d000bd020 con 0x55cf1f6b61b0 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d0dd3b640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2cf00662a0 0x7f2cf0068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d0dd3b640 1 -- 172.21.15.76:0/1326089883 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f2d0011fae0 con 0x55cf1f6b61b0 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d10540640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2cf00662a0 0x7f2cf0068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55cf1f6a7730 0x55cf1f626610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- osd_op(unknown.0.0:1 13.12 13:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x55cf1f626b50 con 0x55cf1f6a7730 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d0fd3f640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55cf1f6a7730 0x55cf1f626610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d10540640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2cf00662a0 0x7f2cf0068760 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800e880 tx=0x7f2d08007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.613+0000 7f2d0fd3f640 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55cf1f6a7730 0x55cf1f626610 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d0fd3f640 1 -- 172.21.15.76:0/1326089883 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v93'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f2cf8008030 con 0x55cf1f6a7730 2024-09-19T04:19:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55cf1f6a7730 msgr2=0x55cf1f626610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x55cf1f6a7730 0x55cf1f626610 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2cf00662a0 msgr2=0x7f2cf0068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2cf00662a0 0x7f2cf0068760 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800e880 tx=0x7f2d08007040 comp rx=0 tx=0).stop 2024-09-19T04:19:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f6b61b0 msgr2=0x55cf1f6b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 --2- 172.21.15.76:0/1326089883 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55cf1f6b61b0 0x55cf1f6b2680 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f2d000ad040 tx=0x7f2d00018150 comp rx=0 tx=0).stop 2024-09-19T04:19:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d10d41640 1 -- 172.21.15.76:0/1326089883 reap_dead start 2024-09-19T04:19:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 shutdown_connections 2024-09-19T04:19:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 >> 172.21.15.76:0/1326089883 conn(0x55cf1f60c220 msgr2=0x55cf1f6114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 shutdown_connections 2024-09-19T04:19:13.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.617+0000 7f2d12a2d080 1 -- 172.21.15.76:0/1326089883 wait complete. 2024-09-19T04:19:13.620 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-19T04:19:13.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.701+0000 7fbedeb88080 1 Processor -- start 2024-09-19T04:19:13.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.701+0000 7fbedeb88080 1 -- start start 2024-09-19T04:19:13.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbec80022e0 0x55da073b76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 0x55da073b6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da074dd210 0x55da074dd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da073bb930 con 0x55da074dd210 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55da072c21b0 con 0x7fbec80022e0 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55da073a09f0 con 0x55da07447da0 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da074dd210 0x55da074dd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da074dd210 0x55da074dd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43864/0 (socket says 172.21.15.76:43864) 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/511073175 learned_addr learned my addr 172.21.15.76:0/511073175 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedbe9a640 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 0x55da073b6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedc69b640 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbec80022e0 0x55da073b76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 msgr2=0x55da073b6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 0x55da073b6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbec80022e0 msgr2=0x55da073b76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbec80022e0 0x55da073b76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/511073175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da0744aaf0 con 0x55da074dd210 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da074dd210 0x55da074dd620 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x55da0735d060 tx=0x7fbed4005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd3c56ffd40c937 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedae98640 1 -- 172.21.15.76:0/511073175 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4013460 con 0x55da074dd210 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedae98640 1 -- 172.21.15.76:0/511073175 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed4013600 con 0x55da074dd210 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedae98640 1 -- 172.21.15.76:0/511073175 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed40139b0 con 0x55da074dd210 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- 172.21.15.76:0/511073175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da074dd210 msgr2=0x55da074dd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da074dd210 0x55da074dd620 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x55da0735d060 tx=0x7fbed4005d90 comp rx=0 tx=0).stop 2024-09-19T04:19:13.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- 172.21.15.76:0/511073175 shutdown_connections 2024-09-19T04:19:13.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55da074dd210 0x55da074dd620 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 0x55da073b6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- 172.21.15.76:0/511073175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbec80022e0 0x55da073b76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- 172.21.15.76:0/511073175 >> 172.21.15.76:0/511073175 conn(0x55da0739c220 msgr2=0x55da073ba130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- 172.21.15.76:0/511073175 shutdown_connections 2024-09-19T04:19:13.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- 172.21.15.76:0/511073175 wait complete. 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 Processor -- start 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- start start 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 0x55da074d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da074dd210 0x55da074dc090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbec80022e0 0x55da074dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55da0744b030 con 0x7fbec80022e0 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55da0744ad90 con 0x55da074dd210 2024-09-19T04:19:13.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedeb88080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55da0744b2d0 con 0x55da07447da0 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbec80022e0 0x55da074dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbec80022e0 0x55da074dc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43878/0 (socket says 172.21.15.76:43878) 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/1132758265 learned_addr learned my addr 172.21.15.76:0/1132758265 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedc69b640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 0x55da074d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 msgr2=0x55da074d82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedbe9a640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da074dd210 0x55da074dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55da07447da0 0x55da074d82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da074dd210 msgr2=0x55da074dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55da074dd210 0x55da074dc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 -- 172.21.15.76:0/1132758265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da074dcc80 con 0x7fbec80022e0 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.705+0000 7fbedce9c640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbec80022e0 0x55da074dc5d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbed4017040 tx=0x7fbed4003340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbed9e96640 1 -- 172.21.15.76:0/1132758265 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4028070 con 0x7fbec80022e0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbed9e96640 1 -- 172.21.15.76:0/1132758265 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbed4005590 con 0x7fbec80022e0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55da074dcf20 con 0x7fbec80022e0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbed9e96640 1 -- 172.21.15.76:0/1132758265 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed4013460 con 0x7fbec80022e0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da0744e2f0 con 0x7fbec80022e0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbed9e96640 1 -- 172.21.15.76:0/1132758265 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbed402b070 con 0x7fbec80022e0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbed9e96640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbebc0662d0 0x7fbebc068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbed9e96640 1 -- 172.21.15.76:0/1132758265 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fbed4085900 con 0x7fbec80022e0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedc69b640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbebc0662d0 0x7fbebc068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedeb88080 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55da074d8800 0x55da0744ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 --> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] -- osd_op(unknown.0.0:1 13.5 13:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x55da0744ef70 con 0x55da074d8800 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedbe9a640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55da074d8800 0x55da0744ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedc69b640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbebc0662d0 0x7fbebc068790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0ad000 tx=0x7fbecc0a6070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.709+0000 7fbedbe9a640 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55da074d8800 0x55da0744ea30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.714 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedbe9a640 1 -- 172.21.15.76:0/1132758265 <== osd.4 v2:172.21.15.116:6800/3265669393 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v93'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbec4009740 con 0x55da074d8800 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55da074d8800 msgr2=0x55da0744ea30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55da074d8800 0x55da0744ea30 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbebc0662d0 msgr2=0x7fbebc068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbebc0662d0 0x7fbebc068790 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbecc0ad000 tx=0x7fbecc0a6070 comp rx=0 tx=0).stop 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbec80022e0 msgr2=0x55da074dc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 --2- 172.21.15.76:0/1132758265 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbec80022e0 0x55da074dc5d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbed4017040 tx=0x7fbed4003340 comp rx=0 tx=0).stop 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedce9c640 1 -- 172.21.15.76:0/1132758265 reap_dead start 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 shutdown_connections 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 >> 172.21.15.76:0/1132758265 conn(0x55da0739c220 msgr2=0x55da073ba130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 shutdown_connections 2024-09-19T04:19:13.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.713+0000 7fbedeb88080 1 -- 172.21.15.76:0/1132758265 wait complete. 2024-09-19T04:19:13.716 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-19T04:19:13.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 Processor -- start 2024-09-19T04:19:13.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- start start 2024-09-19T04:19:13.794 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c3e10 0x55df6e4bba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e4c6130 0x55df6e4bbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e54f8c0 0x55df6e5431d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55df6e3cd1b0 con 0x55df6e4c3e10 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55df6e4aad50 con 0x55df6e54f8c0 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55df6e54e5b0 con 0x55df6e4c6130 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c3e10 0x55df6e4bba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c3e10 0x55df6e4bba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43890/0 (socket says 172.21.15.76:43890) 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 -- 172.21.15.76:0/2926919604 learned_addr learned my addr 172.21.15.76:0/2926919604 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb4808640 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e4c6130 0x55df6e4bbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb580a640 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e54f8c0 0x55df6e5431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 -- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e4c6130 msgr2=0x55df6e4bbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e4c6130 0x55df6e4bbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 -- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e54f8c0 msgr2=0x55df6e5431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e54f8c0 0x55df6e5431d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 -- 172.21.15.76:0/2926919604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6e544ee0 con 0x55df6e4c3e10 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb580a640 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e54f8c0 0x55df6e5431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb5009640 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c3e10 0x55df6e4bba90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdda8001630 tx=0x7fdda8001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81dd9940ef7c9ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddaf7fe640 1 -- 172.21.15.76:0/2926919604 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8013620 con 0x55df6e4c3e10 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddaf7fe640 1 -- 172.21.15.76:0/2926919604 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda80137c0 con 0x55df6e4c3e10 2024-09-19T04:19:13.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddaf7fe640 1 -- 172.21.15.76:0/2926919604 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda8013b90 con 0x55df6e4c3e10 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- 172.21.15.76:0/2926919604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c3e10 msgr2=0x55df6e4bba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c3e10 0x55df6e4bba90 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fdda8001630 tx=0x7fdda8001af0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- 172.21.15.76:0/2926919604 shutdown_connections 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e54f8c0 0x55df6e5431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e4c6130 0x55df6e4bbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 --2- 172.21.15.76:0/2926919604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c3e10 0x55df6e4bba90 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- 172.21.15.76:0/2926919604 >> 172.21.15.76:0/2926919604 conn(0x55df6e4a6940 msgr2=0x55df6e4a9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- 172.21.15.76:0/2926919604 shutdown_connections 2024-09-19T04:19:13.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.793+0000 7fddb74f6080 1 -- 172.21.15.76:0/2926919604 wait complete. 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 Processor -- start 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 -- start start 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e4c3e10 0x55df6e4be490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c6130 0x55df6e5eab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e54f8c0 0x55df6e5eb060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55df6e545420 con 0x55df6e4c6130 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55df6e545180 con 0x55df6e4c3e10 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55df6e4bd3e0 con 0x55df6e54f8c0 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c6130 0x55df6e5eab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c6130 0x55df6e5eab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43896/0 (socket says 172.21.15.76:43896) 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 -- 172.21.15.76:0/4097760488 learned_addr learned my addr 172.21.15.76:0/4097760488 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb5009640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e4c3e10 0x55df6e4be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb580a640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e54f8c0 0x55df6e5eb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 -- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e54f8c0 msgr2=0x55df6e5eb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e54f8c0 0x55df6e5eb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 -- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e4c3e10 msgr2=0x55df6e4be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55df6e4c3e10 0x55df6e4be490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 -- 172.21.15.76:0/4097760488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6e54b320 con 0x55df6e4c6130 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb580a640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55df6e54f8c0 0x55df6e5eb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb4808640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c6130 0x55df6e5eab20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdda001c230 tx=0x7fdda001a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddae7fc640 1 -- 172.21.15.76:0/4097760488 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00bd020 con 0x55df6e4c6130 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddae7fc640 1 -- 172.21.15.76:0/4097760488 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdda00a7e40 con 0x55df6e4c6130 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df6e54b5c0 con 0x55df6e4c6130 2024-09-19T04:19:13.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddae7fc640 1 -- 172.21.15.76:0/4097760488 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda00b4040 con 0x55df6e4c6130 2024-09-19T04:19:13.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6e54bb00 con 0x55df6e4c6130 2024-09-19T04:19:13.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddae7fc640 1 -- 172.21.15.76:0/4097760488 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdda00c0020 con 0x55df6e4c6130 2024-09-19T04:19:13.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.797+0000 7fddae7fc640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd980662a0 0x7fdd98068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddae7fc640 1 -- 172.21.15.76:0/4097760488 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fdda0122a80 con 0x55df6e4c6130 2024-09-19T04:19:13.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb5009640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd980662a0 0x7fdd98068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55df6e4be9d0 0x55df6e54c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 --> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] -- osd_op(unknown.0.0:1 13.9 13:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x55df6e4c1380 con 0x55df6e4be9d0 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb5009640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd980662a0 0x7fdd98068760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdda8006660 tx=0x7fdda8002200 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb580a640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55df6e4be9d0 0x55df6e54c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb580a640 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55df6e4be9d0 0x55df6e54c900 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb580a640 1 -- 172.21.15.76:0/4097760488 <== osd.5 v2:172.21.15.116:6808/3134608166 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v93'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fddb000a890 con 0x55df6e4be9d0 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55df6e4be9d0 msgr2=0x55df6e54c900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x55df6e4be9d0 0x55df6e54c900 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd980662a0 msgr2=0x7fdd98068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdd980662a0 0x7fdd98068760 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdda8006660 tx=0x7fdda8002200 comp rx=0 tx=0).stop 2024-09-19T04:19:13.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c6130 msgr2=0x55df6e5eab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 --2- 172.21.15.76:0/4097760488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55df6e4c6130 0x55df6e5eab20 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdda001c230 tx=0x7fdda001a920 comp rx=0 tx=0).stop 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb580a640 1 -- 172.21.15.76:0/4097760488 reap_dead start 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 shutdown_connections 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 >> 172.21.15.76:0/4097760488 conn(0x55df6e4a6940 msgr2=0x55df6e4bba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 shutdown_connections 2024-09-19T04:19:13.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.801+0000 7fddb74f6080 1 -- 172.21.15.76:0/4097760488 wait complete. 2024-09-19T04:19:13.806 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-19T04:19:13.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 Processor -- start 2024-09-19T04:19:13.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 -- start start 2024-09-19T04:19:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 0x55e7f90b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f909cdd0 0x55e7f9126ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f9127bb0 0x55e7f9140920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e7f9129360 con 0x55e7f909cdd0 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e7f8fbb1b0 con 0x55e7f9127bb0 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692662c080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55e7f91294e0 con 0x55e7f906a3a0 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692393e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f909cdd0 0x55e7f9126ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692393e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f909cdd0 0x55e7f9126ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43898/0 (socket says 172.21.15.76:43898) 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692393e640 1 -- 172.21.15.76:0/2379886593 learned_addr learned my addr 172.21.15.76:0/2379886593 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f692413f640 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 0x55e7f90b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.885+0000 7f6924940640 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f9127bb0 0x55e7f9140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 -- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 msgr2=0x55e7f90b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 0x55e7f90b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 -- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f9127bb0 msgr2=0x55e7f9140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f9127bb0 0x55e7f9140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 -- 172.21.15.76:0/2379886593 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f9125b10 con 0x55e7f909cdd0 2024-09-19T04:19:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f909cdd0 0x55e7f9126ca0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f690c001e40 tx=0x7f690c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307674d9a009c861 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692293c640 1 -- 172.21.15.76:0/2379886593 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c012020 con 0x55e7f909cdd0 2024-09-19T04:19:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692293c640 1 -- 172.21.15.76:0/2379886593 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f690c00e690 con 0x55e7f909cdd0 2024-09-19T04:19:13.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692293c640 1 -- 172.21.15.76:0/2379886593 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690c00eaa0 con 0x55e7f909cdd0 2024-09-19T04:19:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- 172.21.15.76:0/2379886593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f909cdd0 msgr2=0x55e7f9126ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f909cdd0 0x55e7f9126ca0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f690c001e40 tx=0x7f690c00b790 comp rx=0 tx=0).stop 2024-09-19T04:19:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- 172.21.15.76:0/2379886593 shutdown_connections 2024-09-19T04:19:13.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f9127bb0 0x55e7f9140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f909cdd0 0x55e7f9126ca0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 --2- 172.21.15.76:0/2379886593 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 0x55e7f90b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- 172.21.15.76:0/2379886593 >> 172.21.15.76:0/2379886593 conn(0x55e7f906adc0 msgr2=0x55e7f9099510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- 172.21.15.76:0/2379886593 shutdown_connections 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- 172.21.15.76:0/2379886593 wait complete. 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 Processor -- start 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- start start 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 0x55e7f91dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f909cdd0 0x55e7f90ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f9127bb0 0x55e7f9124770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e7f9125db0 con 0x55e7f9127bb0 2024-09-19T04:19:13.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e7f9126050 con 0x55e7f909cdd0 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55e7f90adac0 con 0x55e7f906a3a0 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f9127bb0 0x55e7f9124770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f9127bb0 0x55e7f9124770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43904/0 (socket says 172.21.15.76:43904) 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 -- 172.21.15.76:0/3344330137 learned_addr learned my addr 172.21.15.76:0/3344330137 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692413f640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 0x55e7f91dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f909cdd0 0x55e7f90ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 -- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 msgr2=0x55e7f91dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e7f906a3a0 0x55e7f91dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 -- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f909cdd0 msgr2=0x55e7f90ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f909cdd0 0x55e7f90ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 -- 172.21.15.76:0/3344330137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7f91dc360 con 0x55e7f9127bb0 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692393e640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e7f909cdd0 0x55e7f90ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f6924940640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f9127bb0 0x55e7f9124770 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6914019940 tx=0x7f6914018030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692193a640 1 -- 172.21.15.76:0/3344330137 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69140b9020 con 0x55e7f9127bb0 2024-09-19T04:19:13.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692193a640 1 -- 172.21.15.76:0/3344330137 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f691400a230 con 0x55e7f9127bb0 2024-09-19T04:19:13.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692193a640 1 -- 172.21.15.76:0/3344330137 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69140af040 con 0x55e7f9127bb0 2024-09-19T04:19:13.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e7f91dc600 con 0x55e7f9127bb0 2024-09-19T04:19:13.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.889+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7f91dcb40 con 0x55e7f9127bb0 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692193a640 1 -- 172.21.15.76:0/3344330137 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f69140bc020 con 0x55e7f9127bb0 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692193a640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69080662d0 0x7f6908068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692413f640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69080662d0 0x7f6908068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692193a640 1 -- 172.21.15.76:0/3344330137 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f691411f7a0 con 0x55e7f9127bb0 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692662c080 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55e7f9069cc0 0x55e7f9093d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 --> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] -- osd_op(unknown.0.0:1 13.5 13:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x55e7f9124cb0 con 0x55e7f9069cc0 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692413f640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69080662d0 0x7f6908068790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f691c002940 tx=0x7f691c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692393e640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55e7f9069cc0 0x55e7f9093d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.893+0000 7f692393e640 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55e7f9069cc0 0x55e7f9093d20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692393e640 1 -- 172.21.15.76:0/3344330137 <== osd.4 v2:172.21.15.116:6800/3265669393 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v93'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f690c008030 con 0x55e7f9069cc0 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55e7f9069cc0 msgr2=0x55e7f9093d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55e7f9069cc0 0x55e7f9093d20 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69080662d0 msgr2=0x7f6908068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69080662d0 0x7f6908068790 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f691c002940 tx=0x7f691c007040 comp rx=0 tx=0).stop 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f9127bb0 msgr2=0x55e7f9124770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 --2- 172.21.15.76:0/3344330137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e7f9127bb0 0x55e7f9124770 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f6914019940 tx=0x7f6914018030 comp rx=0 tx=0).stop 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f6924940640 1 -- 172.21.15.76:0/3344330137 reap_dead start 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 shutdown_connections 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 >> 172.21.15.76:0/3344330137 conn(0x55e7f906adc0 msgr2=0x55e7f9140ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 shutdown_connections 2024-09-19T04:19:13.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.897+0000 7f692662c080 1 -- 172.21.15.76:0/3344330137 wait complete. 2024-09-19T04:19:13.901 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-19T04:19:13.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.985+0000 7ff1bc9d7080 1 Processor -- start 2024-09-19T04:19:13.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.985+0000 7ff1bc9d7080 1 -- start start 2024-09-19T04:19:13.988 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 0x55bbab6b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab673d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 0x55bbab6b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bbab5c21b0 con 0x55bbab673920 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbab61b7b0 con 0x55bbab6b2be0 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bbab6b9ee0 con 0x7ff1a8002860 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab673d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab673d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43924/0 (socket says 172.21.15.76:43924) 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 -- 172.21.15.76:0/1010271131 learned_addr learned my addr 172.21.15.76:0/1010271131 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 -- 172.21.15.76:0/1010271131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 msgr2=0x55bbab6b2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 --2- 172.21.15.76:0/1010271131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 0x55bbab6b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 -- 172.21.15.76:0/1010271131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 msgr2=0x55bbab6b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 --2- 172.21.15.76:0/1010271131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 0x55bbab6b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 -- 172.21.15.76:0/1010271131 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab7431b0 con 0x55bbab673920 2024-09-19T04:19:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 --2- 172.21.15.76:0/1010271131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab673d00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac09ac30 tx=0x7ff1ac001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456139104e4efc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b8ce7640 1 -- 172.21.15.76:0/1010271131 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac0bf070 con 0x55bbab673920 2024-09-19T04:19:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b8ce7640 1 -- 172.21.15.76:0/1010271131 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1ac0b4040 con 0x55bbab673920 2024-09-19T04:19:13.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b8ce7640 1 -- 172.21.15.76:0/1010271131 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac0b3070 con 0x55bbab673920 2024-09-19T04:19:13.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/1010271131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 msgr2=0x55bbab673d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/1010271131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab673d00 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac09ac30 tx=0x7ff1ac001a50 comp rx=0 tx=0).stop 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/1010271131 shutdown_connections 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/1010271131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 0x55bbab6b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/1010271131 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab673d00 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/1010271131 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 0x55bbab6b2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/1010271131 >> 172.21.15.76:0/1010271131 conn(0x55bbab69c220 msgr2=0x55bbab6b6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/1010271131 shutdown_connections 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/1010271131 wait complete. 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 Processor -- start 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- start start 2024-09-19T04:19:13.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab7414a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 0x55bbab7ce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 0x55bbab7ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bbab743450 con 0x55bbab673920 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bbab742c40 con 0x55bbab6b2be0 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1bc9d7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bbab6b4600 con 0x7ff1a8002860 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab7414a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab7414a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43934/0 (socket says 172.21.15.76:43934) 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 -- 172.21.15.76:0/2527115067 learned_addr learned my addr 172.21.15.76:0/2527115067 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1baceb640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 0x55bbab7ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 -- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 msgr2=0x55bbab7ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b9ce9640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 0x55bbab7ce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1a8002860 0x55bbab7ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 -- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 msgr2=0x55bbab7ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bbab6b2be0 0x55bbab7ce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 -- 172.21.15.76:0/2527115067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bbab6b48a0 con 0x55bbab673920 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1ba4ea640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab7414a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4002f50 tx=0x7ff1a4008b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b37fe640 1 -- 172.21.15.76:0/2527115067 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4006b70 con 0x55bbab673920 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b37fe640 1 -- 172.21.15.76:0/2527115067 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a401a070 con 0x55bbab673920 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.989+0000 7ff1b37fe640 1 -- 172.21.15.76:0/2527115067 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4015460 con 0x55bbab673920 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bbab6b4b40 con 0x55bbab673920 2024-09-19T04:19:13.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bbab7cb5e0 con 0x55bbab673920 2024-09-19T04:19:13.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1b37fe640 1 -- 172.21.15.76:0/2527115067 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff1a4013030 con 0x55bbab673920 2024-09-19T04:19:13.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1b37fe640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff19c0662d0 0x7ff19c068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1b37fe640 1 -- 172.21.15.76:0/2527115067 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff1a4085220 con 0x55bbab673920 2024-09-19T04:19:13.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1b9ce9640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff19c0662d0 0x7ff19c068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55bbab7ccd30 0x55bbab7d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:13.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- osd_op(unknown.0.0:1 13.1d 13:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93) -- 0x55bbab7d0f80 con 0x55bbab7ccd30 2024-09-19T04:19:13.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1b9ce9640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff19c0662d0 0x7ff19c068790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0bc000 tx=0x7ff1ac0b0040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1baceb640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55bbab7ccd30 0x55bbab7d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:13.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.993+0000 7ff1baceb640 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55bbab7ccd30 0x55bbab7d0a40 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:13.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1baceb640 1 -- 172.21.15.76:0/2527115067 <== osd.7 v2:172.21.15.116:6824/2361616500 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v93'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff1b400e230 con 0x55bbab7ccd30 2024-09-19T04:19:13.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55bbab7ccd30 msgr2=0x55bbab7d0a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x55bbab7ccd30 0x55bbab7d0a40 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:13.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff19c0662d0 msgr2=0x7ff19c068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff19c0662d0 0x7ff19c068790 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac0bc000 tx=0x7ff1ac0b0040 comp rx=0 tx=0).stop 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 msgr2=0x55bbab7414a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 --2- 172.21.15.76:0/2527115067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bbab673920 0x55bbab7414a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4002f50 tx=0x7ff1a4008b20 comp rx=0 tx=0).stop 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1baceb640 1 -- 172.21.15.76:0/2527115067 reap_dead start 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 shutdown_connections 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 >> 172.21.15.76:0/2527115067 conn(0x55bbab69c220 msgr2=0x55bbab69d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 shutdown_connections 2024-09-19T04:19:13.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:13.997+0000 7ff1bc9d7080 1 -- 172.21.15.76:0/2527115067 wait complete. 2024-09-19T04:19:14.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:14.001 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /d1/d2/d3 recursive,force 2024-09-19T04:19:14.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.173+0000 7f425b924640 1 Processor -- start 2024-09-19T04:19:14.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.173+0000 7f425b924640 1 -- start start 2024-09-19T04:19:14.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.173+0000 7f425b924640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 0x7f42540aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.173+0000 7f425b924640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42540b3290 con 0x7f425415bec0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f425b924640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42540b3570 con 0x7f42540af2a0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f425b924640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f42540b3850 con 0x7f425415c2a0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f42596c0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f425415bec0 0x7f42540ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45604/0 (socket says 172.21.15.76:45604) 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f42596c0640 1 -- 172.21.15.76:0/105318096 learned_addr learned my addr 172.21.15.76:0/105318096 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4258ebf640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 0x7f42540aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1374135018 0 0) 0x7f42540b3290 con 0x7f425415bec0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4240003530 con 0x7f425415bec0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4186080871 0 0) 0x7f4240003530 con 0x7f425415bec0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 msgr2=0x7f42540aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 0x7f42540aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 >> v1:172.21.15.116:6789/0 conn(0x7f42540af2a0 legacy=0x7f425406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42540b3b30 con 0x7f425415bec0 2024-09-19T04:19:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f4250001680 con 0x7f425415bec0 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 >> v1:172.21.15.76:6789/0 conn(0x7f425415bec0 legacy=0x7f42540ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 0x7f42400039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4240003f70 0x7f4240004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f42400051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42540b3850 con 0x7f4240004d50 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f424000bab0 con 0x7f4240003f70 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f424000bcb0 con 0x7f425415c2a0 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4259ec1640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f42400051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f42596c0640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 0x7f42400039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4258ebf640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4240003f70 0x7f4240004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4259ec1640 1 -- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 msgr2=0x7f42400039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4259ec1640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425415c2a0 0x7f42400039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4259ec1640 1 -- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4240003f70 msgr2=0x7f4240004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4259ec1640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4240003f70 0x7f4240004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4259ec1640 1 -- 172.21.15.76:0/105318096 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4240003530 con 0x7f4240004d50 2024-09-19T04:19:14.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4258ebf640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4240003f70 0x7f4240004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f4259ec1640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f42400051d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f424c001e70 tx=0x7f424c01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2271c585acf70fb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c001a90 con 0x7f4240004d50 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c003b80 con 0x7f4240004d50 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.177+0000 7f424bfff640 1 -- 172.21.15.76:0/105318096 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c005080 con 0x7f4240004d50 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- 172.21.15.76:0/105318096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 msgr2=0x7f42400051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 --2- 172.21.15.76:0/105318096 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f42400051d0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f424c001e70 tx=0x7f424c01c780 comp rx=0 tx=0).stop 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f4259ec1640 1 -- 172.21.15.76:0/105318096 reap_dead start 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- 172.21.15.76:0/105318096 shutdown_connections 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- 172.21.15.76:0/105318096 >> 172.21.15.76:0/105318096 conn(0x7f42540b1f90 msgr2=0x7f42540b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- 172.21.15.76:0/105318096 shutdown_connections 2024-09-19T04:19:14.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- 172.21.15.76:0/105318096 wait complete. 2024-09-19T04:19:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 Processor -- start 2024-09-19T04:19:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- start start 2024-09-19T04:19:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f4254150570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42540c6d80 0x7f42540c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425409d2e0 0x7f425409d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42540af730 con 0x7f4240004d50 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f42540b40f0 con 0x7f42540c6d80 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f42540afc90 con 0x7f425409d2e0 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f4254150570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f4254150570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43954/0 (socket says 172.21.15.76:43954) 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 -- 172.21.15.76:0/2325769526 learned_addr learned my addr 172.21.15.76:0/2325769526 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f4259ec1640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425409d2e0 0x7f425409d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 -- 172.21.15.76:0/2325769526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425409d2e0 msgr2=0x7f425409d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425409d2e0 0x7f425409d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 -- 172.21.15.76:0/2325769526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42540c6d80 msgr2=0x7f42540c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42540c6d80 0x7f42540c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 -- 172.21.15.76:0/2325769526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42540c7240 con 0x7f4240004d50 2024-09-19T04:19:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f42596c0640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f4254150570 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f425000e8c0 tx=0x7f425000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f4249ffb640 1 -- 172.21.15.76:0/2325769526 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425000c8d0 con 0x7f4240004d50 2024-09-19T04:19:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f4249ffb640 1 -- 172.21.15.76:0/2325769526 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4250018070 con 0x7f4240004d50 2024-09-19T04:19:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42540c7520 con 0x7f4240004d50 2024-09-19T04:19:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f4249ffb640 1 -- 172.21.15.76:0/2325769526 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425000cc90 con 0x7f4240004d50 2024-09-19T04:19:14.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.181+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42540c7ae0 con 0x7f4240004d50 2024-09-19T04:19:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4249ffb640 1 -- 172.21.15.76:0/2325769526 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f425001d460 con 0x7f4240004d50 2024-09-19T04:19:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4249ffb640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42380678c0 0x7f4238069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4258ebf640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42380678c0 0x7f4238069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4249ffb640 1 -- 172.21.15.76:0/2325769526 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f4250085e60 con 0x7f4240004d50 2024-09-19T04:19:14.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4258ebf640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42380678c0 0x7f4238069d80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f42540b1180 tx=0x7f4244008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 Processor -- start 2024-09-19T04:19:14.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 -- start start 2024-09-19T04:19:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c7cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c33c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5597a2a81bd0 con 0x5597a2c33830 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5597a2a272f0 con 0x5597a2c7cb40 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5597a2a7db90 con 0x5597a2c7d460 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c33c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c33c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43970/0 (socket says 172.21.15.76:43970) 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 -- 172.21.15.76:0/1101074948 learned_addr learned my addr 172.21.15.76:0/1101074948 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4258ebf640 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 -- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 msgr2=0x5597a2c332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f42596c0640 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c7cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 -- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 msgr2=0x5597a2c7cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c7cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.185+0000 7f4259ec1640 1 -- 172.21.15.76:0/1101074948 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a2a4db10 con 0x5597a2c33830 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4259ec1640 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c33c10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f424c01d020 tx=0x7f424c0014b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48648de461e66b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f423effd640 1 -- 172.21.15.76:0/1101074948 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c006d60 con 0x5597a2c33830 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f423effd640 1 -- 172.21.15.76:0/1101074948 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c0b5070 con 0x5597a2c33830 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f423effd640 1 -- 172.21.15.76:0/1101074948 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c003ba0 con 0x5597a2c33830 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/1101074948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 msgr2=0x5597a2c33c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c33c10 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f424c01d020 tx=0x7f424c0014b0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/1101074948 shutdown_connections 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c33c10 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1101074948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c7cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/1101074948 >> 172.21.15.76:0/1101074948 conn(0x5597a2c2f7e0 msgr2=0x5597a2c31bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/1101074948 shutdown_connections 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/1101074948 wait complete. 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- start start 2024-09-19T04:19:14.192 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 Processor -- start 2024-09-19T04:19:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c3d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c3de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c3e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5597a2b03700 con 0x5597a2c33830 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5597a2a1b5c0 con 0x5597a2c7cb40 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5597a2a80ee0 con 0x5597a2c7d460 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 10 client.0 _ll_register_callbacks cb 0x5597a2b355a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c3d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c3d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43978/0 (socket says 172.21.15.76:43978) 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 -- 172.21.15.76:0/142577560 learned_addr learned my addr 172.21.15.76:0/142577560 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4259ec1640 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c3e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4258ebf640 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c3de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 -- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 msgr2=0x5597a2c3e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c3e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 -- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 msgr2=0x5597a2c3de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c3de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 -- 172.21.15.76:0/142577560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a2a27500 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f42596c0640 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c3d8f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4250000fc0 tx=0x7f425000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=323e7f20011ed616 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4233fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4233fff640 1 -- 172.21.15.76:0/142577560 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425005ec80 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4233fff640 1 -- 172.21.15.76:0/142577560 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f4250014cd0 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f424c0b5070 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4233fff640 1 -- 172.21.15.76:0/142577560 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f4250052710 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 10 client.7662 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5597a2a21550 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4233fff640 1 -- 172.21.15.76:0/142577560 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c0b5070 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f4233fff640 1 -- 172.21.15.76:0/142577560 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f425005e7f0 con 0x5597a2c33830 2024-09-19T04:19:14.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 10 client.7662 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 10 client.7662 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 20 client.7662 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2c8bbd0 0x5597a2c8dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 4 client.7662 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.189+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597a2a1bd00 con 0x5597a2c8bbd0 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f4258ebf640 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2c8bbd0 0x5597a2c8dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f4258ebf640 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2c8bbd0 0x5597a2c8dfc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1729a68503f2b2 server_cookie=9b9916dad0f51774 in_seq=0 out_seq=0 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f4233fff640 10 client.7662 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f4233fff640 1 -- 172.21.15.76:0/142577560 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5597a2a1bd00 con 0x5597a2c8bbd0 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f4233fff640 10 client.7662 handle_command_reply: tid=0 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 client.7662 shutdown 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 msgr2=0x5597a2c3d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c3d8f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f4250000fc0 tx=0x7f425000cf70 comp rx=0 tx=0).stop 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 shutdown_connections 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2c8bbd0 0x5597a2c8dfc0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7d460 0x5597a2c3e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f4233fff640 0 client.7662 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7cb40 0x5597a2c3de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 --2- 172.21.15.76:0/142577560 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c3d8f0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 >> 172.21.15.76:0/142577560 conn(0x5597a2c2f7e0 msgr2=0x5597a2c323f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 shutdown_connections 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 1 -- 172.21.15.76:0/142577560 wait complete. 2024-09-19T04:19:14.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.193+0000 7f425e9e0000 20 client.7662 trim_cache size 0 max 16384 2024-09-19T04:19:14.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.209+0000 7f425e9e0000 1 Processor -- start 2024-09-19T04:19:14.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.209+0000 7f425e9e0000 1 -- start start 2024-09-19T04:19:14.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c33830 0x5597a2c91200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c91740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c7d460 0x5597a2c31bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f42541ff410 con 0x5597a2c7d460 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f42541fe5e0 con 0x5597a2c33830 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f42541fece0 con 0x5597a2c7cb40 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c7d460 0x5597a2c31bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c7d460 0x5597a2c31bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43984/0 (socket says 172.21.15.76:43984) 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 -- 172.21.15.76:0/1300362471 learned_addr learned my addr 172.21.15.76:0/1300362471 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4258ebf640 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c91740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 -- 172.21.15.76:0/1300362471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 msgr2=0x5597a2c91740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c91740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 -- 172.21.15.76:0/1300362471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c33830 msgr2=0x5597a2c91200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c33830 0x5597a2c91200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 -- 172.21.15.76:0/1300362471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597a2b03b90 con 0x5597a2c7d460 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4259ec1640 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c7d460 0x5597a2c31bd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f424c000f90 tx=0x7f424c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac59498e0275ab06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f423e7fc640 1 -- 172.21.15.76:0/1300362471 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c006d60 con 0x5597a2c7d460 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f423e7fc640 1 -- 172.21.15.76:0/1300362471 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c0b5070 con 0x5597a2c7d460 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f423e7fc640 1 -- 172.21.15.76:0/1300362471 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c003bc0 con 0x5597a2c7d460 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- 172.21.15.76:0/1300362471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c7d460 msgr2=0x5597a2c31bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c7d460 0x5597a2c31bd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f424c000f90 tx=0x7f424c005040 comp rx=0 tx=0).stop 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- 172.21.15.76:0/1300362471 shutdown_connections 2024-09-19T04:19:14.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c7d460 0x5597a2c31bd0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c91740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- 172.21.15.76:0/1300362471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c33830 0x5597a2c91200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- 172.21.15.76:0/1300362471 >> 172.21.15.76:0/1300362471 conn(0x5597a2c2f7e0 msgr2=0x5597a2c37c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- 172.21.15.76:0/1300362471 shutdown_connections 2024-09-19T04:19:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- 172.21.15.76:0/1300362471 wait complete. 2024-09-19T04:19:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- start start 2024-09-19T04:19:14.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 Processor -- start 2024-09-19T04:19:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c90760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c90ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7d460 0x5597a2cec220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5597a2b03700 con 0x5597a2c33830 2024-09-19T04:19:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f42541f27e0 con 0x5597a2c7d460 2024-09-19T04:19:14.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5597a2a4cfa0 con 0x5597a2c7cb40 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 10 client.0 _ll_register_callbacks cb 0x7f4250055a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c90760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c90760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43992/0 (socket says 172.21.15.76:43992) 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 -- 172.21.15.76:0/2870439649 learned_addr learned my addr 172.21.15.76:0/2870439649 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4258ebf640 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c90ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 -- 172.21.15.76:0/2870439649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 msgr2=0x5597a2c90ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c90ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 -- 172.21.15.76:0/2870439649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7d460 msgr2=0x5597a2cec220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7d460 0x5597a2cec220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 -- 172.21.15.76:0/2870439649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597a2adaca0 con 0x5597a2c33830 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f42596c0640 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c90760 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4250001980 tx=0x7f4250084060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adbc8613b143b13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4233fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4233fff640 1 -- 172.21.15.76:0/2870439649 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42500528b0 con 0x5597a2c33830 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4233fff640 1 -- 172.21.15.76:0/2870439649 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f425007daf0 con 0x5597a2c33830 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f424c0b5070 con 0x5597a2c33830 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4233fff640 1 -- 172.21.15.76:0/2870439649 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f425005ec80 con 0x5597a2c33830 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 10 client.7668 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5597a273c4c0 con 0x5597a2c33830 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.213+0000 7f4233fff640 1 -- 172.21.15.76:0/2870439649 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f424c0b5070 con 0x5597a2c33830 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4233fff640 1 -- 172.21.15.76:0/2870439649 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f4250051df0 con 0x5597a2c33830 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 10 client.7668 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 10 client.7668 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 20 client.7668 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2cecc40 0x5597a2ced050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 4 client.7668 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x5597a2a8f300 con 0x5597a2cecc40 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4258ebf640 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2cecc40 0x5597a2ced050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4233fff640 10 client.7668 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4258ebf640 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2cecc40 0x5597a2ced050 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759877dd836abf65 server_cookie=3ec178bca4b6614e in_seq=0 out_seq=0 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4233fff640 1 -- 172.21.15.76:0/2870439649 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5597a2a8f300 con 0x5597a2cecc40 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4233fff640 10 client.7668 handle_command_reply: tid=0 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 client.7668 shutdown 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 msgr2=0x5597a2c90760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c90760 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f4250001980 tx=0x7f4250084060 comp rx=0 tx=0).stop 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 shutdown_connections 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x5597a2cecc40 0x5597a2ced050 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5597a2c7d460 0x5597a2cec220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5597a2c7cb40 0x5597a2c90ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 --2- 172.21.15.76:0/2870439649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5597a2c33830 0x5597a2c90760 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 >> 172.21.15.76:0/2870439649 conn(0x5597a2c2f7e0 msgr2=0x5597a2c384b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4233fff640 0 client.7668 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.223 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f4993eef640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:18:58.573984+0000 1=0+1) n(v3 rc2024-09-19T04:18:58.784529+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7455=pAsLsXsFsx/AsLsXsFsx@9},l=7455 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x561bb63f8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.573984+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.784529+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.573984+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:18:58.784529+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 shutdown_connections 2024-09-19T04:19:14.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 1 -- 172.21.15.76:0/2870439649 wait complete. 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425e9e0000 20 client.7668 trim_cache size 0 max 16384 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "8362a942-ff57-460f-be25-9fdf5af5b05a", 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-09-19T04:19:14.224 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:14.224 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:19:14.217+0000 7efc56243640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:18:58.576784+0000 1=0+1) n(v2 rc2024-09-19T04:18:58.784529+0000 4=0+4) (inest lock) caps={7455=pAsLsXsFs/-@8},l=7455 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55e37ab33080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.576784+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.784529+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.576784+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:18:58.784529+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:19:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42380678c0 msgr2=0x7f4238069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42380678c0 0x7f4238069d80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f42540b1180 tx=0x7f4244008040 comp rx=0 tx=0).stop 2024-09-19T04:19:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 msgr2=0x7f4254150570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f4254150570 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f425000e8c0 tx=0x7f425000ed80 comp rx=0 tx=0).stop 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 shutdown_connections 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42380678c0 0x7f4238069d80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f425409d2e0 0x7f425409d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42540c6d80 0x7f42540c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 --2- 172.21.15.76:0/2325769526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4240004d50 0x7f4254150570 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 >> 172.21.15.76:0/2325769526 conn(0x7f42540b1f90 msgr2=0x7f4254157360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 shutdown_connections 2024-09-19T04:19:14.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f425b924640 1 -- 172.21.15.76:0/2325769526 wait complete. 2024-09-19T04:19:14.226 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:19:14.217+0000 7f499a6fc640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:18:58.575422+0000 1=0+1) n(v4 rc2024-09-19T04:18:58.784529+0000 5=0+5) (inest sync->mix g=1,2) caps={7455=pAsLsXsFs/-@11} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x561bb63f7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.575422+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.784529+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.575422+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:18:58.784529+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:19:14.227 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:19:14.221+0000 7efc5ca50640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:18:58.578146+0000 1=0+1) n(v1 rc2024-09-19T04:18:58.784529+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7455=pAsLsXsFs/-@8} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55e37bb60000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.578146+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.579541+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.578146+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:18:58.784529+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:19:14.227 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-09-19T04:19:14.218+0000 7f8c9ae37640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:18:58.579541+0000 1=0+1) n(v0 rc2024-09-19T04:18:58.579541+0000 2=0+2) (inest lock) (ifile excl) caps={7455=pAsLsXsFsx/-@6},l=7455 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x556a355dd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.579541+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.579541+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.579541+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:18:58.579541+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:19:14.227 INFO:tasks.ceph.mds.b.smithi116.stderr:2024-09-19T04:19:14.218+0000 7f8c9ae37640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={7455=pAsLsXsFs/-@3},l=7455 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x556a355dd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:19:14.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:14.235 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub pause 2024-09-19T04:19:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.349+0000 7f6442b74640 1 Processor -- start 2024-09-19T04:19:14.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.349+0000 7f6442b74640 1 -- start start 2024-09-19T04:19:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6442b74640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 0x7f643c06e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6442b74640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f643c06fef0 0x7f643c15b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6442b74640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f643c15bda0 con 0x7f643c0b3d50 2024-09-19T04:19:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6442b74640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f643c06ee50 con 0x7f643c06fef0 2024-09-19T04:19:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6442b74640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f643c06f130 con 0x7f643c073e60 2024-09-19T04:19:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6433fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f643c06fef0 0x7f643c15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6441111640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f643c0b3d50 0x7f643c0b4160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45610/0 (socket says 172.21.15.76:45610) 2024-09-19T04:19:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6441111640 1 -- 172.21.15.76:0/3160844456 learned_addr learned my addr 172.21.15.76:0/3160844456 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6433fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f643c06fef0 0x7f643c15b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:43996/0 (socket says 172.21.15.76:43996) 2024-09-19T04:19:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6440910640 1 --2- 172.21.15.76:0/3160844456 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 0x7f643c06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6433fff640 1 -- 172.21.15.76:0/3160844456 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 msgr2=0x7f643c06e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6433fff640 1 --2- 172.21.15.76:0/3160844456 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 0x7f643c06e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6433fff640 1 -- 172.21.15.76:0/3160844456 >> v1:172.21.15.76:6789/0 conn(0x7f643c0b3d50 legacy=0x7f643c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6433fff640 1 -- 172.21.15.76:0/3160844456 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c06f410 con 0x7f643c06fef0 2024-09-19T04:19:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f6433fff640 1 --2- 172.21.15.76:0/3160844456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c15b860 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f642c0097c0 tx=0x7f642c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11abfed21f5287c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f64337fe640 1 -- 172.21.15.76:0/3160844456 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b0070 con 0x7f643c06fef0 2024-09-19T04:19:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f64337fe640 1 -- 172.21.15.76:0/3160844456 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c0ab600 con 0x7f643c06fef0 2024-09-19T04:19:14.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.353+0000 7f64337fe640 1 -- 172.21.15.76:0/3160844456 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0ab9d0 con 0x7f643c06fef0 2024-09-19T04:19:14.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- 172.21.15.76:0/3160844456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 msgr2=0x7f643c15b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 --2- 172.21.15.76:0/3160844456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c15b860 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f642c0097c0 tx=0x7f642c01c410 comp rx=0 tx=0).stop 2024-09-19T04:19:14.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- 172.21.15.76:0/3160844456 shutdown_connections 2024-09-19T04:19:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 --2- 172.21.15.76:0/3160844456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c15b860 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 --2- 172.21.15.76:0/3160844456 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 0x7f643c06e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- 172.21.15.76:0/3160844456 >> 172.21.15.76:0/3160844456 conn(0x7f643c0b1f90 msgr2=0x7f643c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- 172.21.15.76:0/3160844456 shutdown_connections 2024-09-19T04:19:14.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- 172.21.15.76:0/3160844456 wait complete. 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 Processor -- start 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- start start 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c09d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 0x7f643c0afee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f643c0b3d50 0x7f643c0b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f643c0729e0 con 0x7f643c06fef0 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f643c072860 con 0x7f643c0b3d50 2024-09-19T04:19:14.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f643c072b60 con 0x7f643c073e60 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c09d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c09d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44002/0 (socket says 172.21.15.76:44002) 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 -- 172.21.15.76:0/1110250475 learned_addr learned my addr 172.21.15.76:0/1110250475 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 -- 172.21.15.76:0/1110250475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 msgr2=0x7f643c0afee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 0x7f643c0afee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 -- 172.21.15.76:0/1110250475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f643c0b3d50 msgr2=0x7f643c0b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f643c0b3d50 0x7f643c0b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 -- 172.21.15.76:0/1110250475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c142530 con 0x7f643c06fef0 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6440910640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c09d5e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f643800d6f0 tx=0x7f643800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f64317fa640 1 -- 172.21.15.76:0/1110250475 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438015070 con 0x7f643c06fef0 2024-09-19T04:19:14.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f64317fa640 1 -- 172.21.15.76:0/1110250475 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6438007940 con 0x7f643c06fef0 2024-09-19T04:19:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c13f8f0 con 0x7f643c06fef0 2024-09-19T04:19:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f64317fa640 1 -- 172.21.15.76:0/1110250475 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643800b040 con 0x7f643c06fef0 2024-09-19T04:19:14.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.357+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c141800 con 0x7f643c06fef0 2024-09-19T04:19:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f64317fa640 1 -- 172.21.15.76:0/1110250475 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6438018070 con 0x7f643c06fef0 2024-09-19T04:19:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f64317fa640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64280678c0 0x7f6428069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f64317fa640 1 -- 172.21.15.76:0/1110250475 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f64380860a0 con 0x7f643c06fef0 2024-09-19T04:19:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6433fff640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64280678c0 0x7f6428069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6433fff640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64280678c0 0x7f6428069d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ad000 tx=0x7f642c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 Processor -- start 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 -- start start 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e09c10 0x55a3b3e09ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3dbd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3dbdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a3b3c0ecc0 con 0x55a3b3dbd780 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a3b3bc9a20 con 0x55a3b3e0a530 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a3b3c0ac80 con 0x55a3b3e09c10 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3dbdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3dbdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44016/0 (socket says 172.21.15.76:44016) 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 -- 172.21.15.76:0/1100613182 learned_addr learned my addr 172.21.15.76:0/1100613182 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6433fff640 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3dbd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6440910640 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e09c10 0x55a3b3e09ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 -- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e09c10 msgr2=0x55a3b3e09ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e09c10 0x55a3b3e09ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 -- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 msgr2=0x55a3b3dbd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3dbd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 -- 172.21.15.76:0/1100613182 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b3bb1d40 con 0x55a3b3dbd780 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6441111640 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3dbdb60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f6434002080 tx=0x7f643400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b33fea29854fb18d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f64227fc640 1 -- 172.21.15.76:0/1100613182 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434017070 con 0x55a3b3dbd780 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f64227fc640 1 -- 172.21.15.76:0/1100613182 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64340072d0 con 0x55a3b3dbd780 2024-09-19T04:19:14.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f64227fc640 1 -- 172.21.15.76:0/1100613182 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340076c0 con 0x55a3b3dbd780 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 -- 172.21.15.76:0/1100613182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 msgr2=0x55a3b3dbdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3dbdb60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f6434002080 tx=0x7f643400c4b0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 -- 172.21.15.76:0/1100613182 shutdown_connections 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3dbdb60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3dbd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 --2- 172.21.15.76:0/1100613182 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e09c10 0x55a3b3e09ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.361+0000 7f6445c30000 1 -- 172.21.15.76:0/1100613182 >> 172.21.15.76:0/1100613182 conn(0x55a3b3dea9b0 msgr2=0x55a3b3decda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- 172.21.15.76:0/1100613182 shutdown_connections 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- 172.21.15.76:0/1100613182 wait complete. 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- start start 2024-09-19T04:19:14.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 Processor -- start 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3dbd780 0x55a3b3e10f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e0a530 0x55a3b3dc3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a3b3ba82c0 con 0x55a3b3e09c10 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a3b3bcd370 con 0x55a3b3dbd780 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a3b3c1cf50 con 0x55a3b3e0a530 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 10 client.0 _ll_register_callbacks cb 0x55a3b3c6fdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e114b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44026/0 (socket says 172.21.15.76:44026) 2024-09-19T04:19:14.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 -- 172.21.15.76:0/3046167025 learned_addr learned my addr 172.21.15.76:0/3046167025 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6441111640 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e0a530 0x55a3b3dc3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6440910640 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3dbd780 0x55a3b3e10f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 -- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e0a530 msgr2=0x55a3b3dc3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e0a530 0x55a3b3dc3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 -- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3dbd780 msgr2=0x55a3b3e10f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3dbd780 0x55a3b3e10f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 -- 172.21.15.76:0/3046167025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b3bae770 con 0x55a3b3e09c10 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6433fff640 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e114b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ae040 tx=0x7f642c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf15b64df8b452 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 1 -- 172.21.15.76:0/3046167025 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0c6060 con 0x55a3b3e09c10 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 1 -- 172.21.15.76:0/3046167025 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f642c004230 con 0x55a3b3e09c10 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64340072d0 con 0x55a3b3e09c10 2024-09-19T04:19:14.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 1 -- 172.21.15.76:0/3046167025 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f642c0cc050 con 0x55a3b3e09c10 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 10 client.7683 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55a3b3bcc3f0 con 0x55a3b3e09c10 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 1 -- 172.21.15.76:0/3046167025 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64340072d0 con 0x55a3b3e09c10 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 1 -- 172.21.15.76:0/3046167025 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f642c0cf020 con 0x55a3b3e09c10 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 10 client.7683 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 10 client.7683 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 20 client.7683 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3dc3e90 0x55a3b3dc42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 4 client.7683 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3b3ba8d90 con 0x55a3b3dc3e90 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6440910640 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3dc3e90 0x55a3b3dc42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6440910640 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3dc3e90 0x55a3b3dc42a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a240b04400f3bbd server_cookie=2e5c28f48f99a707 in_seq=0 out_seq=0 2024-09-19T04:19:14.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 10 client.7683 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 1 -- 172.21.15.76:0/3046167025 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a3b3ba8d90 con 0x55a3b3dc3e90 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f641b7fe640 10 client.7683 handle_command_reply: tid=0 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.365+0000 7f6445c30000 1 client.7683 shutdown 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 msgr2=0x55a3b3e114b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e114b0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ae040 tx=0x7f642c0046d0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 shutdown_connections 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3dc3e90 0x55a3b3dc42a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e0a530 0x55a3b3dc3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e114b0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f641b7fe640 0 client.7683 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 --2- 172.21.15.76:0/3046167025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3dbd780 0x55a3b3e10f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 >> 172.21.15.76:0/3046167025 conn(0x55a3b3dea9b0 msgr2=0x55a3b3dbc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 shutdown_connections 2024-09-19T04:19:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 1 -- 172.21.15.76:0/3046167025 wait complete. 2024-09-19T04:19:14.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.369+0000 7f6445c30000 20 client.7683 trim_cache size 0 max 16384 2024-09-19T04:19:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 Processor -- start 2024-09-19T04:19:14.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 -- start start 2024-09-19T04:19:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3dbd780 0x55a3b3e20310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e20850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e20d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f643c203350 con 0x55a3b3e09c10 2024-09-19T04:19:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f643c203a50 con 0x55a3b3e0a530 2024-09-19T04:19:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f643c202460 con 0x55a3b3dbd780 2024-09-19T04:19:14.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e20850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e20850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44042/0 (socket says 172.21.15.76:44042) 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 -- 172.21.15.76:0/1828322277 learned_addr learned my addr 172.21.15.76:0/1828322277 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 -- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3dbd780 msgr2=0x55a3b3e20310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6440910640 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3dbd780 0x55a3b3e20310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6441111640 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e20d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3dbd780 0x55a3b3e20310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 -- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 msgr2=0x55a3b3e20d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e20d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 -- 172.21.15.76:0/1828322277 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3b3c90320 con 0x55a3b3e09c10 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6433fff640 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e20850 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f642c01a040 tx=0x7f642c0b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45242567fb985595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6421ffb640 1 -- 172.21.15.76:0/1828322277 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b0040 con 0x55a3b3e09c10 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6421ffb640 1 -- 172.21.15.76:0/1828322277 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c0acb20 con 0x55a3b3e09c10 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.385+0000 7f6421ffb640 1 -- 172.21.15.76:0/1828322277 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0c1400 con 0x55a3b3e09c10 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/1828322277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 msgr2=0x55a3b3e20850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e20850 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f642c01a040 tx=0x7f642c0b6000 comp rx=0 tx=0).stop 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/1828322277 shutdown_connections 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e20d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3e09c10 0x55a3b3e20850 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/1828322277 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3dbd780 0x55a3b3e20310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/1828322277 >> 172.21.15.76:0/1828322277 conn(0x55a3b3dea9b0 msgr2=0x55a3b3dbba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/1828322277 shutdown_connections 2024-09-19T04:19:14.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/1828322277 wait complete. 2024-09-19T04:19:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- start start 2024-09-19T04:19:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 Processor -- start 2024-09-19T04:19:14.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3e7a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e16010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e16550 0x55a3b3e16930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55a3b3bb6120 con 0x55a3b3dbd780 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55a3b3c8fe90 con 0x55a3b3e0a530 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55a3b3bb8b90 con 0x55a3b3e16550 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 10 client.0 _ll_register_callbacks cb 0x7f642c0ae070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3e7a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3e7a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44048/0 (socket says 172.21.15.76:44048) 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 -- 172.21.15.76:0/4079584430 learned_addr learned my addr 172.21.15.76:0/4079584430 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6441111640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e16550 0x55a3b3e16930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 -- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e16550 msgr2=0x55a3b3e16930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e16550 0x55a3b3e16930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 -- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 msgr2=0x55a3b3e16010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6433fff640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e16010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e16010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 -- 172.21.15.76:0/4079584430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3b3ba25c0 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6441111640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e16550 0x55a3b3e16930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6440910640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3e7a030 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f64380526b0 tx=0x7f6438013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d089e15bd6309efe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 1 -- 172.21.15.76:0/4079584430 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643801f040 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 1 -- 172.21.15.76:0/4079584430 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f6438007d30 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f642c0acb20 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 1 -- 172.21.15.76:0/4079584430 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f643805eec0 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 10 client.7689 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55a3b3ba6330 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 1 -- 172.21.15.76:0/4079584430 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c0acb20 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 1 -- 172.21.15.76:0/4079584430 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f64380528c0 con 0x55a3b3dbd780 2024-09-19T04:19:14.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 10 client.7689 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 10 client.7689 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 20 client.7689 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3e17780 0x55a3b3e86720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 4 client.7689 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x55a3b3c1c3f0 con 0x55a3b3e17780 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6433fff640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3e17780 0x55a3b3e86720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 10 client.7689 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6433fff640 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3e17780 0x55a3b3e86720 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a939d4a5828ae31d server_cookie=b229cf29651ae270 in_seq=0 out_seq=0 2024-09-19T04:19:14.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 1 -- 172.21.15.76:0/4079584430 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a3b3c1c3f0 con 0x55a3b3e17780 2024-09-19T04:19:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 10 client.7689 handle_command_reply: tid=0 2024-09-19T04:19:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 client.7689 shutdown 2024-09-19T04:19:14.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 msgr2=0x55a3b3e7a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3e7a030 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f64380526b0 tx=0x7f6438013040 comp rx=0 tx=0).stop 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 shutdown_connections 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55a3b3e17780 0x55a3b3e86720 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55a3b3e16550 0x55a3b3e16930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55a3b3e0a530 0x55a3b3e16010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 --2- 172.21.15.76:0/4079584430 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55a3b3dbd780 0x55a3b3e7a030 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 >> 172.21.15.76:0/4079584430 conn(0x55a3b3dea9b0 msgr2=0x55a3b3dbc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f641b7fe640 0 client.7689 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.389+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 shutdown_connections 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6445c30000 1 -- 172.21.15.76:0/4079584430 wait complete. 2024-09-19T04:19:14.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6445c30000 20 client.7689 trim_cache size 0 max 16384 2024-09-19T04:19:14.402 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:14.402 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:19:14.402 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:19:14.402 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64280678c0 msgr2=0x7f6428069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64280678c0 0x7f6428069d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ad000 tx=0x7f642c004740 comp rx=0 tx=0).stop 2024-09-19T04:19:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 msgr2=0x7f643c09d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c09d5e0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f643800d6f0 tx=0x7f643800dbb0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 shutdown_connections 2024-09-19T04:19:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64280678c0 0x7f6428069d80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f643c0b3d50 0x7f643c0b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f643c073e60 0x7f643c0afee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 --2- 172.21.15.76:0/1110250475 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f643c06fef0 0x7f643c09d5e0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 >> 172.21.15.76:0/1110250475 conn(0x7f643c0b1f90 msgr2=0x7f643c15eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 shutdown_connections 2024-09-19T04:19:14.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.393+0000 7f6442b74640 1 -- 172.21.15.76:0/1110250475 wait complete. 2024-09-19T04:19:14.408 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:19:14.408 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-09-19T04:19:14.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3620fc5640 1 Processor -- start 2024-09-19T04:19:14.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3620fc5640 1 -- start start 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3620fc5640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f361c15bec0 0x7f361c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3620fc5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f361c15c2a0 0x7f361c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3620fc5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f361c0b3290 con 0x7f361c0af2a0 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3620fc5640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f361c0b3570 con 0x7f361c15bec0 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3620fc5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f361c0b3850 con 0x7f361c15c2a0 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f361a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f361c15bec0 0x7f361c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f361a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f361c15bec0 0x7f361c0ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44054/0 (socket says 172.21.15.76:44054) 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f361a59c640 1 -- 172.21.15.76:0/3198266120 learned_addr learned my addr 172.21.15.76:0/3198266120 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f3613fff640 1 --2- 172.21.15.76:0/3198266120 >> v2:172.21.15.116:3300/0 conn(0x7f361c15c2a0 0x7f361c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f361a59c640 1 -- 172.21.15.76:0/3198266120 >> v2:172.21.15.116:3300/0 conn(0x7f361c15c2a0 msgr2=0x7f361c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f361a59c640 1 --2- 172.21.15.76:0/3198266120 >> v2:172.21.15.116:3300/0 conn(0x7f361c15c2a0 0x7f361c0aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f361a59c640 1 -- 172.21.15.76:0/3198266120 >> v1:172.21.15.76:6789/0 conn(0x7f361c0af2a0 legacy=0x7f361c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.521+0000 7f361a59c640 1 -- 172.21.15.76:0/3198266120 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c0b3b30 con 0x7f361c15bec0 2024-09-19T04:19:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f361a59c640 1 --2- 172.21.15.76:0/3198266120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c0ad200 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f361c0ae510 tx=0x7f360401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8607423374fbd269 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3619d9b640 1 -- 172.21.15.76:0/3198266120 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040aa650 con 0x7f361c15bec0 2024-09-19T04:19:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3619d9b640 1 -- 172.21.15.76:0/3198266120 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36040af070 con 0x7f361c15bec0 2024-09-19T04:19:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3619d9b640 1 -- 172.21.15.76:0/3198266120 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040aaa10 con 0x7f361c15bec0 2024-09-19T04:19:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198266120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 msgr2=0x7f361c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198266120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c0ad200 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f361c0ae510 tx=0x7f360401c4f0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198266120 shutdown_connections 2024-09-19T04:19:14.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198266120 >> v2:172.21.15.116:3300/0 conn(0x7f361c15c2a0 0x7f361c0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198266120 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c0ad200 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198266120 >> 172.21.15.76:0/3198266120 conn(0x7f361c0b1f90 msgr2=0x7f361c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198266120 shutdown_connections 2024-09-19T04:19:14.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198266120 wait complete. 2024-09-19T04:19:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 Processor -- start 2024-09-19T04:19:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- start start 2024-09-19T04:19:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f361c0af2a0 0x7f361c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f361c15c2a0 0x7f361c09ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0af6b0 con 0x7f361c15bec0 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f361c0ad740 con 0x7f361c0af2a0 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3620fc5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f361c169000 con 0x7f361c15c2a0 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44060/0 (socket says 172.21.15.76:44060) 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 -- 172.21.15.76:0/3198857911 learned_addr learned my addr 172.21.15.76:0/3198857911 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 -- 172.21.15.76:0/3198857911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f361c15c2a0 msgr2=0x7f361c09ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f361c15c2a0 0x7f361c09ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 -- 172.21.15.76:0/3198857911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f361c0af2a0 msgr2=0x7f361c152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f361c0af2a0 0x7f361c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 -- 172.21.15.76:0/3198857911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c15d0a0 con 0x7f361c15bec0 2024-09-19T04:19:14.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3613fff640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c152790 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c610 tx=0x7f360c00cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.525+0000 7f3612ffd640 1 -- 172.21.15.76:0/3198857911 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c015070 con 0x7f361c15bec0 2024-09-19T04:19:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3612ffd640 1 -- 172.21.15.76:0/3198857911 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c004220 con 0x7f361c15bec0 2024-09-19T04:19:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c15d340 con 0x7f361c15bec0 2024-09-19T04:19:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3612ffd640 1 -- 172.21.15.76:0/3198857911 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c003620 con 0x7f361c15bec0 2024-09-19T04:19:14.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c15d900 con 0x7f361c15bec0 2024-09-19T04:19:14.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3612ffd640 1 -- 172.21.15.76:0/3198857911 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f360c0037c0 con 0x7f361c15bec0 2024-09-19T04:19:14.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3612ffd640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f35fc0678f0 0x7f35fc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3612ffd640 1 -- 172.21.15.76:0/3198857911 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f360c085bf0 con 0x7f361c15bec0 2024-09-19T04:19:14.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f361a59c640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f35fc0678f0 0x7f35fc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 Processor -- start 2024-09-19T04:19:14.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f361a59c640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f35fc0678f0 0x7f35fc069db0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f36040ae000 tx=0x7f3604004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 -- start start 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f9f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21f52c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 0x555e21f535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555e21e38960 con 0x555e21f9faf0 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555e21d7c900 con 0x555e21f531c0 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555e21d9fd40 con 0x555e21f9f1d0 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21f52c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21f52c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44072/0 (socket says 172.21.15.76:44072) 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 -- 172.21.15.76:0/229487377 learned_addr learned my addr 172.21.15.76:0/229487377 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 -- 172.21.15.76:0/229487377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 msgr2=0x555e21f9f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f9f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 -- 172.21.15.76:0/229487377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 msgr2=0x555e21f535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f361ad9d640 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 0x555e21f535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 0x555e21f535a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 -- 172.21.15.76:0/229487377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e21d46580 con 0x555e21f9faf0 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.529+0000 7f3613fff640 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21f52c80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f360c083ec0 tx=0x7f360c055ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece69691f9b80157 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35fbfff640 1 -- 172.21.15.76:0/229487377 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c082070 con 0x555e21f9faf0 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35fbfff640 1 -- 172.21.15.76:0/229487377 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c054130 con 0x555e21f9faf0 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35fbfff640 1 -- 172.21.15.76:0/229487377 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c01e040 con 0x555e21f9faf0 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/229487377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 msgr2=0x555e21f52c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21f52c80 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f360c083ec0 tx=0x7f360c055ef0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/229487377 shutdown_connections 2024-09-19T04:19:14.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 0x555e21f535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21f52c80 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- 172.21.15.76:0/229487377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f9f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/229487377 >> 172.21.15.76:0/229487377 conn(0x555e21f804a0 msgr2=0x555e21f82890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/229487377 shutdown_connections 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/229487377 wait complete. 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- start start 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 Processor -- start 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f531c0 0x555e21f840b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f845f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e21f84b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x555e21d9bd00 con 0x555e21f531c0 2024-09-19T04:19:14.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555e21d5aaa0 con 0x555e21f9faf0 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555e21d39340 con 0x555e21f9f1d0 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 10 client.0 _ll_register_callbacks cb 0x555e21f3c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f531c0 0x555e21f840b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f531c0 0x555e21f840b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44082/0 (socket says 172.21.15.76:44082) 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 -- 172.21.15.76:0/1845340325 learned_addr learned my addr 172.21.15.76:0/1845340325 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361ad9d640 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e21f84b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3613fff640 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f845f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 -- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 msgr2=0x555e21f845f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f845f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 -- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 msgr2=0x555e21f84b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e21f84b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 -- 172.21.15.76:0/1845340325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e21d42dc0 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f361a59c640 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f531c0 0x555e21f840b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3604097fc0 tx=0x7f360401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b154dea5b1bf3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35f97fa640 1 -- 172.21.15.76:0/1845340325 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36040c6060 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35f97fa640 1 -- 172.21.15.76:0/1845340325 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f36040b3af0 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f360c054130 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35f97fa640 1 -- 172.21.15.76:0/1845340325 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f36040b3cf0 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 10 client.7704 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x555e21d3f7f0 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35f97fa640 1 -- 172.21.15.76:0/1845340325 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f360c054130 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35f97fa640 1 -- 172.21.15.76:0/1845340325 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f36040c0070 con 0x555e21f531c0 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 10 client.7704 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 10 client.7704 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-09-19T04:19:14.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 20 client.7704 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e21fae470 0x555e21fb0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 4 client.7704 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e21d39e10 con 0x555e21fae470 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3613fff640 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e21fae470 0x555e21fb0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f3613fff640 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e21fae470 0x555e21fb0860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b1f4c95333ca6eb8 server_cookie=59c4f548bc74a4c8 in_seq=0 out_seq=0 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.533+0000 7f35f97fa640 10 client.7704 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f35f97fa640 1 -- 172.21.15.76:0/1845340325 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555e21d39e10 con 0x555e21fae470 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f35f97fa640 10 client.7704 handle_command_reply: tid=0 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 client.7704 shutdown 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f531c0 msgr2=0x555e21f840b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f531c0 0x555e21f840b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f3604097fc0 tx=0x7f360401ad30 comp rx=0 tx=0).stop 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 shutdown_connections 2024-09-19T04:19:14.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e21fae470 0x555e21fb0860 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e21f84b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f845f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f35f97fa640 0 client.7704 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 --2- 172.21.15.76:0/1845340325 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f531c0 0x555e21f840b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 >> 172.21.15.76:0/1845340325 conn(0x555e21f804a0 msgr2=0x555e21f52490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 shutdown_connections 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 1 -- 172.21.15.76:0/1845340325 wait complete. 2024-09-19T04:19:14.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.537+0000 7f3624081000 20 client.7704 trim_cache size 0 max 16384 2024-09-19T04:19:14.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 Processor -- start 2024-09-19T04:19:14.554 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 -- start start 2024-09-19T04:19:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 0x555e21f9d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f9ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21fb38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f361c200060 con 0x555e21f9faf0 2024-09-19T04:19:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f361c073e90 con 0x555e21f531c0 2024-09-19T04:19:14.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f361c1ff930 con 0x555e21f9f1d0 2024-09-19T04:19:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21fb38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21fb38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44098/0 (socket says 172.21.15.76:44098) 2024-09-19T04:19:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 -- 172.21.15.76:0/590684936 learned_addr learned my addr 172.21.15.76:0/590684936 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f3613fff640 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f9ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 -- 172.21.15.76:0/590684936 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 msgr2=0x555e21f9ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f9ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 -- 172.21.15.76:0/590684936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 msgr2=0x555e21f9d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 0x555e21f9d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 -- 172.21.15.76:0/590684936 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e21d5d470 con 0x555e21f9faf0 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f361ad9d640 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21fb38a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3614001b40 tx=0x7f3614002000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bd6d9aa1d96fca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.553+0000 7f35fb7fe640 1 -- 172.21.15.76:0/590684936 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361400ebb0 con 0x555e21f9faf0 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35fb7fe640 1 -- 172.21.15.76:0/590684936 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361400c070 con 0x555e21f9faf0 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35fb7fe640 1 -- 172.21.15.76:0/590684936 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3614003980 con 0x555e21f9faf0 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/590684936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 msgr2=0x555e21fb38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21fb38a0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f3614001b40 tx=0x7f3614002000 comp rx=0 tx=0).stop 2024-09-19T04:19:14.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/590684936 shutdown_connections 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9faf0 0x555e21fb38a0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f9f1d0 0x555e21f9ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- 172.21.15.76:0/590684936 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f531c0 0x555e21f9d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/590684936 >> 172.21.15.76:0/590684936 conn(0x555e21f804a0 msgr2=0x555e21f52490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/590684936 shutdown_connections 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/590684936 wait complete. 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- start start 2024-09-19T04:19:14.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 Processor -- start 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f531c0 0x555e22013a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9f1d0 0x555e22013f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e220144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f361c15f370 con 0x555e21f9f1d0 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x555e21d67400 con 0x555e21f9faf0 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x555e21da4370 con 0x555e21f531c0 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 10 client.0 _ll_register_callbacks cb 0x7f36040096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9f1d0 0x555e22013f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9f1d0 0x555e22013f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44110/0 (socket says 172.21.15.76:44110) 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 -- 172.21.15.76:0/50914836 learned_addr learned my addr 172.21.15.76:0/50914836 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 -- 172.21.15.76:0/50914836 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f531c0 msgr2=0x555e22013a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f531c0 0x555e22013a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 -- 172.21.15.76:0/50914836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 msgr2=0x555e220144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f361ad9d640 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e220144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e220144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 -- 172.21.15.76:0/50914836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e21d33640 con 0x555e21f9f1d0 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3613fff640 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9f1d0 0x555e22013f90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c5e0 tx=0x7f360c01b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db275a2e7cad76b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 1 -- 172.21.15.76:0/50914836 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c071040 con 0x555e21f9f1d0 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 1 -- 172.21.15.76:0/50914836 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f360c0743a0 con 0x555e21f9f1d0 2024-09-19T04:19:14.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f361400c070 con 0x555e21f9f1d0 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 1 -- 172.21.15.76:0/50914836 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f360c082040 con 0x555e21f9f1d0 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 10 client.7710 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x555e21d373b0 con 0x555e21f9f1d0 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 1 -- 172.21.15.76:0/50914836 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f361400c070 con 0x555e21f9f1d0 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 1 -- 172.21.15.76:0/50914836 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f360c0524a0 con 0x555e21f9f1d0 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 10 client.7710 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 10 client.7710 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 20 client.7710 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e22014a10 0x555e22014e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 4 client.7710 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555e21dad470 con 0x555e22014a10 2024-09-19T04:19:14.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f361a59c640 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e22014a10 0x555e22014e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 10 client.7710 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f361a59c640 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e22014a10 0x555e22014e20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=184ef1266c38c337 server_cookie=e933d953f1368fa0 in_seq=0 out_seq=0 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 1 -- 172.21.15.76:0/50914836 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x555e21dad470 con 0x555e22014a10 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f35f97fa640 10 client.7710 handle_command_reply: tid=0 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.557+0000 7f3624081000 1 client.7710 shutdown 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9f1d0 msgr2=0x555e22013f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9f1d0 0x555e22013f90 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c5e0 tx=0x7f360c01b070 comp rx=0 tx=0).stop 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 shutdown_connections 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x555e22014a10 0x555e22014e20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x555e21f9faf0 0x555e220144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x555e21f9f1d0 0x555e22013f90 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f35f97fa640 0 client.7710 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 --2- 172.21.15.76:0/50914836 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x555e21f531c0 0x555e22013a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 >> 172.21.15.76:0/50914836 conn(0x555e21f804a0 msgr2=0x555e21f9d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 shutdown_connections 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 1 -- 172.21.15.76:0/50914836 wait complete. 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3624081000 20 client.7710 trim_cache size 0 max 16384 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:14.568 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "8362a942-ff57-460f-be25-9fdf5af5b05a": { 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "8362a942-ff57-460f-be25-9fdf5af5b05a", 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:14.569 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f35fc0678f0 msgr2=0x7f35fc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f35fc0678f0 0x7f35fc069db0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f36040ae000 tx=0x7f3604004740 comp rx=0 tx=0).stop 2024-09-19T04:19:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 msgr2=0x7f361c152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c152790 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c610 tx=0x7f360c00cad0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 shutdown_connections 2024-09-19T04:19:14.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f35fc0678f0 0x7f35fc069db0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f361c15c2a0 0x7f361c09ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f361c15bec0 0x7f361c152790 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 --2- 172.21.15.76:0/3198857911 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f361c0af2a0 0x7f361c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 >> 172.21.15.76:0/3198857911 conn(0x7f361c0b1f90 msgr2=0x7f361c1597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 shutdown_connections 2024-09-19T04:19:14.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.561+0000 7f3620fc5640 1 -- 172.21.15.76:0/3198857911 wait complete. 2024-09-19T04:19:14.576 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:19:14.576 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:19:14.576 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-09-19T04:19:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d65f7640 1 Processor -- start 2024-09-19T04:19:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d65f7640 1 -- start start 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d65f7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d65f7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f38d00af2a0 0x7f38d006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d65f7640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38d00b3290 con 0x7f38d015bec0 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d65f7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f38d00b3570 con 0x7f38d00af2a0 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d65f7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f38d00b3850 con 0x7f38d015c2a0 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d4b94640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f38d00af2a0 0x7f38d006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38cffff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f38d015bec0 0x7f38d00ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45628/0 (socket says 172.21.15.76:45628) 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d4b94640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f38d00af2a0 0x7f38d006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44120/0 (socket says 172.21.15.76:44120) 2024-09-19T04:19:14.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38cffff640 1 -- 172.21.15.76:0/1604148964 learned_addr learned my addr 172.21.15.76:0/1604148964 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/1604148964 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d4b94640 1 -- 172.21.15.76:0/1604148964 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 msgr2=0x7f38d00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.685+0000 7f38d4b94640 1 --2- 172.21.15.76:0/1604148964 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d4b94640 1 -- 172.21.15.76:0/1604148964 >> v1:172.21.15.76:6789/0 conn(0x7f38d015bec0 legacy=0x7f38d00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d4b94640 1 -- 172.21.15.76:0/1604148964 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d00b3b30 con 0x7f38d00af2a0 2024-09-19T04:19:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38ceffd640 1 -- 172.21.15.76:0/1604148964 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1647855402 0 0) 0x7f38d00b3290 con 0x7f38d015bec0 2024-09-19T04:19:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/1604148964 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d4b94640 1 --2- 172.21.15.76:0/1604148964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d00af2a0 0x7f38d006e000 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097c0 tx=0x7f38c001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e99957b6fe6dd1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38ceffd640 1 -- 172.21.15.76:0/1604148964 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00a7070 con 0x7f38d00af2a0 2024-09-19T04:19:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38ceffd640 1 -- 172.21.15.76:0/1604148964 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38c00b0070 con 0x7f38d00af2a0 2024-09-19T04:19:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38ceffd640 1 -- 172.21.15.76:0/1604148964 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00ab890 con 0x7f38d00af2a0 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 -- 172.21.15.76:0/1604148964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d00af2a0 msgr2=0x7f38d006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 --2- 172.21.15.76:0/1604148964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d00af2a0 0x7f38d006e000 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097c0 tx=0x7f38c001c470 comp rx=0 tx=0).stop 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 -- 172.21.15.76:0/1604148964 shutdown_connections 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 --2- 172.21.15.76:0/1604148964 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d00af2a0 0x7f38d006e000 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 --2- 172.21.15.76:0/1604148964 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 -- 172.21.15.76:0/1604148964 >> 172.21.15.76:0/1604148964 conn(0x7f38d00b1f90 msgr2=0x7f38d00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 -- 172.21.15.76:0/1604148964 shutdown_connections 2024-09-19T04:19:14.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.689+0000 7f38d65f7640 1 -- 172.21.15.76:0/1604148964 wait complete. 2024-09-19T04:19:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 Processor -- start 2024-09-19T04:19:14.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 -- start start 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d00af2a0 0x7f38d015d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d015bec0 0x7f38d01505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d01553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f38d006ed50 con 0x7f38d015bec0 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d00afc90 con 0x7f38d00af2a0 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f38d006eed0 con 0x7f38d015c2a0 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d015bec0 0x7f38d01505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d015bec0 0x7f38d01505c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44128/0 (socket says 172.21.15.76:44128) 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 -- 172.21.15.76:0/603095270 learned_addr learned my addr 172.21.15.76:0/603095270 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d4b94640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d01553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cffff640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d00af2a0 0x7f38d015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 -- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 msgr2=0x7f38d01553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d01553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 -- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d00af2a0 msgr2=0x7f38d015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d00af2a0 0x7f38d015d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 -- 172.21.15.76:0/603095270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d0154390 con 0x7f38d015bec0 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d015bec0 0x7f38d01505c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f38b800d800 tx=0x7f38b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38ccff9640 1 -- 172.21.15.76:0/603095270 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b800b6c0 con 0x7f38d015bec0 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38ccff9640 1 -- 172.21.15.76:0/603095270 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b8019070 con 0x7f38d015bec0 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d0154630 con 0x7f38d015bec0 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38ccff9640 1 -- 172.21.15.76:0/603095270 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b800baa0 con 0x7f38d015bec0 2024-09-19T04:19:14.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d0154bf0 con 0x7f38d015bec0 2024-09-19T04:19:14.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.693+0000 7f38ccff9640 1 -- 172.21.15.76:0/603095270 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f38b800bc40 con 0x7f38d015bec0 2024-09-19T04:19:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38ccff9640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b00678c0 0x7f38b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38ccff9640 1 -- 172.21.15.76:0/603095270 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f38b80862d0 con 0x7f38d015bec0 2024-09-19T04:19:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cffff640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b00678c0 0x7f38b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cffff640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b00678c0 0x7f38b0069d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38c80012f0 tx=0x7f38c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 Processor -- start 2024-09-19T04:19:14.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- start start 2024-09-19T04:19:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x55653017e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x556530131870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x556530132190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55653001b960 con 0x55653017e690 2024-09-19T04:19:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55652ff5f900 con 0x556530131db0 2024-09-19T04:19:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55652ff82d40 con 0x55653017dd70 2024-09-19T04:19:14.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x556530131870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x556530131870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44142/0 (socket says 172.21.15.76:44142) 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4123392314 learned_addr learned my addr 172.21.15.76:0/4123392314 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d4b94640 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x556530132190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cffff640 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x55653017e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 msgr2=0x55653017e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x55653017e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 msgr2=0x556530132190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x556530132190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4123392314 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55652ff29580 con 0x55653017e690 2024-09-19T04:19:14.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cffff640 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x55653017e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x556530131870 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f38b8004780 tx=0x7f38b8056c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d23ef642fe5bb39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38c5ffb640 1 -- 172.21.15.76:0/4123392314 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b806e070 con 0x55653017e690 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38c5ffb640 1 -- 172.21.15.76:0/4123392314 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b80860c0 con 0x55653017e690 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38c5ffb640 1 -- 172.21.15.76:0/4123392314 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b805f0e0 con 0x55653017e690 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- 172.21.15.76:0/4123392314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 msgr2=0x556530131870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x556530131870 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f38b8004780 tx=0x7f38b8056c00 comp rx=0 tx=0).stop 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- 172.21.15.76:0/4123392314 shutdown_connections 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x556530132190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x556530131870 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4123392314 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x55653017e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- 172.21.15.76:0/4123392314 >> 172.21.15.76:0/4123392314 conn(0x55653015f040 msgr2=0x556530161430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- 172.21.15.76:0/4123392314 shutdown_connections 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.697+0000 7f38d96b3000 1 -- 172.21.15.76:0/4123392314 wait complete. 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 -- start start 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 Processor -- start 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556530131db0 0x556530139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530139a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55653017e690 0x55653015f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55652ff7ed00 con 0x55653017dd70 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55652ff3daa0 con 0x55653017e690 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55652ff1c340 con 0x556530131db0 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 10 client.0 _ll_register_callbacks cb 0x55653011f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530139a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530139a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44144/0 (socket says 172.21.15.76:44144) 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4105239360 learned_addr learned my addr 172.21.15.76:0/4105239360 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cffff640 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556530131db0 0x556530139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d4b94640 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55653017e690 0x55653015f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556530131db0 msgr2=0x556530139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556530131db0 0x556530139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55653017e690 msgr2=0x55653015f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55653017e690 0x55653015f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 -- 172.21.15.76:0/4105239360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55652ff25dc0 con 0x55653017dd70 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530139a90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f38b8056db0 tx=0x7f38b8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99785e5575136e2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38aeffd640 1 -- 172.21.15.76:0/4105239360 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b806e040 con 0x55653017dd70 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38aeffd640 1 -- 172.21.15.76:0/4105239360 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f38b8052b70 con 0x55653017dd70 2024-09-19T04:19:14.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b80860c0 con 0x55653017dd70 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38aeffd640 1 -- 172.21.15.76:0/4105239360 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f38b805ec50 con 0x55653017dd70 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 10 client.7725 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55652ff227f0 con 0x55653017dd70 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38aeffd640 1 -- 172.21.15.76:0/4105239360 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b80860c0 con 0x55653017dd70 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38aeffd640 1 -- 172.21.15.76:0/4105239360 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f38b8013070 con 0x55653017dd70 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 10 client.7725 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 10 client.7725 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 20 client.7725 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x55653015fe30 0x55653018f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 4 client.7725 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55652ff1ce10 con 0x55653015fe30 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cffff640 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x55653015fe30 0x55653018f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38cffff640 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x55653015fe30 0x55653018f020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1636895ed40faae6 server_cookie=af8818a561f69195 in_seq=0 out_seq=0 2024-09-19T04:19:14.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.701+0000 7f38aeffd640 10 client.7725 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38aeffd640 1 -- 172.21.15.76:0/4105239360 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55652ff1ce10 con 0x55653015fe30 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38aeffd640 10 client.7725 handle_command_reply: tid=0 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 client.7725 shutdown 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 msgr2=0x556530139a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530139a90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f38b8056db0 tx=0x7f38b8000bd0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 shutdown_connections 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x55653015fe30 0x55653018f020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55653017e690 0x55653015f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38aeffd640 0 client.7725 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530139a90 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 --2- 172.21.15.76:0/4105239360 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x556530131db0 0x556530139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 >> 172.21.15.76:0/4105239360 conn(0x55653015f040 msgr2=0x556530130f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 shutdown_connections 2024-09-19T04:19:14.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 1 -- 172.21.15.76:0/4105239360 wait complete. 2024-09-19T04:19:14.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.705+0000 7f38d96b3000 20 client.7725 trim_cache size 0 max 16384 2024-09-19T04:19:14.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 Processor -- start 2024-09-19T04:19:14.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 -- start start 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x556530192580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530191420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017e690 0x556530191960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f38d02019a0 con 0x55653017dd70 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f38d0200b70 con 0x556530131db0 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f38d0201270 con 0x55653017e690 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530191420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530191420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44150/0 (socket says 172.21.15.76:44150) 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 -- 172.21.15.76:0/2567181756 learned_addr learned my addr 172.21.15.76:0/2567181756 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 -- 172.21.15.76:0/2567181756 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017e690 msgr2=0x556530191960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/2567181756 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017e690 0x556530191960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 -- 172.21.15.76:0/2567181756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 msgr2=0x556530192580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/2567181756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x556530192580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 -- 172.21.15.76:0/2567181756 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55652ff40470 con 0x55653017dd70 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/2567181756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530191420 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f38b8054a50 tx=0x7f38b807f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa2453d8c56ac1cb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38c57fa640 1 -- 172.21.15.76:0/2567181756 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b80860b0 con 0x55653017dd70 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38c57fa640 1 -- 172.21.15.76:0/2567181756 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b805ee90 con 0x55653017dd70 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.721+0000 7f38c57fa640 1 -- 172.21.15.76:0/2567181756 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8055950 con 0x55653017dd70 2024-09-19T04:19:14.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/2567181756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 msgr2=0x556530191420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- 172.21.15.76:0/2567181756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530191420 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f38b8054a50 tx=0x7f38b807f000 comp rx=0 tx=0).stop 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/2567181756 shutdown_connections 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- 172.21.15.76:0/2567181756 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017e690 0x556530191960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- 172.21.15.76:0/2567181756 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017dd70 0x556530191420 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- 172.21.15.76:0/2567181756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x556530192580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/2567181756 >> 172.21.15.76:0/2567181756 conn(0x55653015f040 msgr2=0x55653017c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/2567181756 shutdown_connections 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/2567181756 wait complete. 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- start start 2024-09-19T04:19:14.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 Processor -- start 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x5565301395e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x556530139b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x5565301f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55652ff7ed00 con 0x55653017e690 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55652ff87370 con 0x556530131db0 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55653013d3f0 con 0x55653017dd70 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 10 client.0 _ll_register_callbacks cb 0x7f38b807df50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x5565301f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x5565301f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44162/0 (socket says 172.21.15.76:44162) 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 -- 172.21.15.76:0/3454818462 learned_addr learned my addr 172.21.15.76:0/3454818462 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38cf7fe640 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x556530139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38cffff640 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x5565301395e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 -- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 msgr2=0x556530139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x556530139b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 -- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 msgr2=0x5565301395e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x5565301395e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 -- 172.21.15.76:0/3454818462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55652ff16640 con 0x55653017e690 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d4b94640 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x5565301f5530 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097c0 tx=0x7f38c001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26e60020f0b8acb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38aeffd640 1 -- 172.21.15.76:0/3454818462 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c00c2060 con 0x55653017e690 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38aeffd640 1 -- 172.21.15.76:0/3454818462 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f38c00abad0 con 0x55653017e690 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38b805ee90 con 0x55653017e690 2024-09-19T04:19:14.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38aeffd640 1 -- 172.21.15.76:0/3454818462 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f38c00c8050 con 0x55653017e690 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 10 client.7731 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55652ff1a3b0 con 0x55653017e690 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38aeffd640 1 -- 172.21.15.76:0/3454818462 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b805ee90 con 0x55653017e690 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38aeffd640 1 -- 172.21.15.76:0/3454818462 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f38c00cb020 con 0x55653017e690 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 10 client.7731 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 10 client.7731 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 20 client.7731 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5565301f1eb0 0x5565301f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 4 client.7731 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55652ff90470 con 0x5565301f1eb0 2024-09-19T04:19:14.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38cffff640 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5565301f1eb0 0x5565301f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38aeffd640 10 client.7731 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.725+0000 7f38cffff640 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5565301f1eb0 0x5565301f22c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c93e2efcfe1be86 server_cookie=b0bb3dc3e6bde921 in_seq=0 out_seq=0 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38aeffd640 1 -- 172.21.15.76:0/3454818462 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55652ff90470 con 0x5565301f1eb0 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38aeffd640 10 client.7731 handle_command_reply: tid=0 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 client.7731 shutdown 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 msgr2=0x5565301f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x5565301f5530 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f38c00097c0 tx=0x7f38c001acb0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 shutdown_connections 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x5565301f1eb0 0x5565301f22c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55653017e690 0x5565301f5530 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38aeffd640 0 client.7731 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:14.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55653017dd70 0x556530139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 --2- 172.21.15.76:0/3454818462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x556530131db0 0x5565301395e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 >> 172.21.15.76:0/3454818462 conn(0x55653015f040 msgr2=0x55653017cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 shutdown_connections 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 1 -- 172.21.15.76:0/3454818462 wait complete. 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d96b3000 20 client.7731 trim_cache size 0 max 16384 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:19:14.735 INFO:teuthology.orchestra.run.smithi076.stdout: "8362a942-ff57-460f-be25-9fdf5af5b05a": { 2024-09-19T04:19:14.736 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:19:14.736 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "8362a942-ff57-460f-be25-9fdf5af5b05a", 2024-09-19T04:19:14.736 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:19:14.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:14.736 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:14.736 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b00678c0 msgr2=0x7f38b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b00678c0 0x7f38b0069d80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38c80012f0 tx=0x7f38c8009040 comp rx=0 tx=0).stop 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d015bec0 msgr2=0x7f38d01505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d015bec0 0x7f38d01505c0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f38b800d800 tx=0x7f38b800dcc0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 shutdown_connections 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f38b00678c0 0x7f38b0069d80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f38d015c2a0 0x7f38d01553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f38d015bec0 0x7f38d01505c0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 --2- 172.21.15.76:0/603095270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f38d00af2a0 0x7f38d015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 >> 172.21.15.76:0/603095270 conn(0x7f38d00b1f90 msgr2=0x7f38d00afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 shutdown_connections 2024-09-19T04:19:14.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.729+0000 7f38d65f7640 1 -- 172.21.15.76:0/603095270 wait complete. 2024-09-19T04:19:14.744 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:19:14.744 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:19:14.744 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-19T04:19:14.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 Processor -- start 2024-09-19T04:19:14.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 -- start start 2024-09-19T04:19:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c1520a0 0x7f175c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f175c06fe50 0x7f175c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f175c15f1d0 0x7f175c071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f175c152480 con 0x7f175c06fe50 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f175c070250 con 0x7f175c15f1d0 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f1761c16640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f175c0aede0 con 0x7f175c1520a0 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f175affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f175c06fe50 0x7f175c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f175affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f175c06fe50 0x7f175c15ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44166/0 (socket says 172.21.15.76:44166) 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.849+0000 7f175affd640 1 -- 172.21.15.76:0/2182063431 learned_addr learned my addr 172.21.15.76:0/2182063431 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175b7fe640 1 --2- 172.21.15.76:0/2182063431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c1520a0 0x7f175c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175affd640 1 -- 172.21.15.76:0/2182063431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c1520a0 msgr2=0x7f175c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175affd640 1 --2- 172.21.15.76:0/2182063431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c1520a0 0x7f175c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175affd640 1 -- 172.21.15.76:0/2182063431 >> v2:172.21.15.116:3300/0 conn(0x7f175c15f1d0 msgr2=0x7f175c071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175bfff640 1 --2- 172.21.15.76:0/2182063431 >> v2:172.21.15.116:3300/0 conn(0x7f175c15f1d0 0x7f175c071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175affd640 1 --2- 172.21.15.76:0/2182063431 >> v2:172.21.15.116:3300/0 conn(0x7f175c15f1d0 0x7f175c071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175affd640 1 -- 172.21.15.76:0/2182063431 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c15fa10 con 0x7f175c06fe50 2024-09-19T04:19:14.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175b7fe640 1 --2- 172.21.15.76:0/2182063431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c1520a0 0x7f175c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175affd640 1 --2- 172.21.15.76:0/2182063431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c06fe50 0x7f175c15ec90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f175c06e250 tx=0x7f175001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdf7adeb27d2800 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175a7fc640 1 -- 172.21.15.76:0/2182063431 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500b0070 con 0x7f175c06fe50 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175a7fc640 1 -- 172.21.15.76:0/2182063431 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17500ab600 con 0x7f175c06fe50 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f175a7fc640 1 -- 172.21.15.76:0/2182063431 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500ab9d0 con 0x7f175c06fe50 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 -- 172.21.15.76:0/2182063431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c06fe50 msgr2=0x7f175c15ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 --2- 172.21.15.76:0/2182063431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c06fe50 0x7f175c15ec90 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f175c06e250 tx=0x7f175001c240 comp rx=0 tx=0).stop 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 -- 172.21.15.76:0/2182063431 shutdown_connections 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 --2- 172.21.15.76:0/2182063431 >> v2:172.21.15.116:3300/0 conn(0x7f175c15f1d0 0x7f175c071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 --2- 172.21.15.76:0/2182063431 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c06fe50 0x7f175c15ec90 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 --2- 172.21.15.76:0/2182063431 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c1520a0 0x7f175c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 -- 172.21.15.76:0/2182063431 >> 172.21.15.76:0/2182063431 conn(0x7f175c0b1f90 msgr2=0x7f175c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 -- 172.21.15.76:0/2182063431 shutdown_connections 2024-09-19T04:19:14.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 -- 172.21.15.76:0/2182063431 wait complete. 2024-09-19T04:19:14.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.853+0000 7f1761c16640 1 Processor -- start 2024-09-19T04:19:14.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 -- start start 2024-09-19T04:19:14.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c06fe50 0x7f175c14afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c1520a0 0x7f175c098050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f175c15f1d0 0x7f175c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f175c073d30 con 0x7f175c1520a0 2024-09-19T04:19:14.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f175c073a50 con 0x7f175c15f1d0 2024-09-19T04:19:14.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f175c15f5e0 con 0x7f175c06fe50 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c1520a0 0x7f175c098050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c1520a0 0x7f175c098050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44182/0 (socket says 172.21.15.76:44182) 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 -- 172.21.15.76:0/1066601072 learned_addr learned my addr 172.21.15.76:0/1066601072 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175bfff640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f175c15f1d0 0x7f175c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 -- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c06fe50 msgr2=0x7f175c14afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175b7fe640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c06fe50 0x7f175c14afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c06fe50 0x7f175c14afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 -- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f175c15f1d0 msgr2=0x7f175c151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f175c15f1d0 0x7f175c151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 -- 172.21.15.76:0/1066601072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175c0cdbf0 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175affd640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c1520a0 0x7f175c098050 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1750000c10 tx=0x7f1750018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1743fff640 1 -- 172.21.15.76:0/1066601072 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500bb420 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1743fff640 1 -- 172.21.15.76:0/1066601072 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17500b6030 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175c0cded0 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1743fff640 1 -- 172.21.15.76:0/1066601072 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500bb7c0 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f175c14bb20 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1743fff640 1 -- 172.21.15.76:0/1066601072 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f17500b23f0 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1743fff640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17480678c0 0x7f1748069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f1743fff640 1 -- 172.21.15.76:0/1066601072 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f17501217d0 con 0x7f175c1520a0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.857+0000 7f175b7fe640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17480678c0 0x7f1748069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175b7fe640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17480678c0 0x7f1748069d80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f174c00b820 tx=0x7f174c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 Processor -- start 2024-09-19T04:19:14.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- start start 2024-09-19T04:19:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddefd70 0x55f7cddf0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cdda3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cdda4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7cdc8d960 con 0x55f7cddf0690 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f7cdbd1900 con 0x55f7cddefd70 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f7cdbf4d40 con 0x55f7cdda3db0 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cdda3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cdda3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44190/0 (socket says 172.21.15.76:44190) 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 -- 172.21.15.76:0/2672721591 learned_addr learned my addr 172.21.15.76:0/2672721591 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175b7fe640 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddefd70 0x55f7cddf0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 -- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 msgr2=0x55f7cdda4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175bfff640 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cdda4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cdda4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 -- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddefd70 msgr2=0x55f7cddf0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddefd70 0x55f7cddf0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 -- 172.21.15.76:0/2672721591 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7cdb9b580 con 0x55f7cddf0690 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f175affd640 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cdda3870 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f17500f1a30 tx=0x7f1750009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306ae7123325b62a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f17417fa640 1 -- 172.21.15.76:0/2672721591 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500c0070 con 0x55f7cddf0690 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f17417fa640 1 -- 172.21.15.76:0/2672721591 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17500f5e50 con 0x55f7cddf0690 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f17417fa640 1 -- 172.21.15.76:0/2672721591 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17500fa680 con 0x55f7cddf0690 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- 172.21.15.76:0/2672721591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 msgr2=0x55f7cdda3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cdda3870 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f17500f1a30 tx=0x7f1750009240 comp rx=0 tx=0).stop 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- 172.21.15.76:0/2672721591 shutdown_connections 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cdda4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cdda3870 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- 172.21.15.76:0/2672721591 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddefd70 0x55f7cddf0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- 172.21.15.76:0/2672721591 >> 172.21.15.76:0/2672721591 conn(0x55f7cddd1040 msgr2=0x55f7cddd3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- 172.21.15.76:0/2672721591 shutdown_connections 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- 172.21.15.76:0/2672721591 wait complete. 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- start start 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 Processor -- start 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cdda3db0 0x55f7cddf6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cddefd70 0x55f7cddf7010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cddf7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7cdbf0d00 con 0x55f7cddf0690 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f7cdbafaa0 con 0x55f7cdda3db0 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.861+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f7cdb8e340 con 0x55f7cddefd70 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 10 client.0 _ll_register_callbacks cb 0x55f7cdd91760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cddf7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cddf7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44202/0 (socket says 172.21.15.76:44202) 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 -- 172.21.15.76:0/258575999 learned_addr learned my addr 172.21.15.76:0/258575999 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 -- 172.21.15.76:0/258575999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cddefd70 msgr2=0x55f7cddf7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cddefd70 0x55f7cddf7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 -- 172.21.15.76:0/258575999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cdda3db0 msgr2=0x55f7cddf6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cdda3db0 0x55f7cddf6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 -- 172.21.15.76:0/258575999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7cdb97dc0 con 0x55f7cddf0690 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175bfff640 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cddf7550 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f175400e7e0 tx=0x7f175400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4a60e493159f41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 1 -- 172.21.15.76:0/258575999 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175401b060 con 0x55f7cddf0690 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 1 -- 172.21.15.76:0/258575999 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f175400cd00 con 0x55f7cddf0690 2024-09-19T04:19:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17500f5e50 con 0x55f7cddf0690 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 1 -- 172.21.15.76:0/258575999 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f17540039a0 con 0x55f7cddf0690 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 10 client.7743 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f7cdb947f0 con 0x55f7cddf0690 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 1 -- 172.21.15.76:0/258575999 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17500f5e50 con 0x55f7cddf0690 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 1 -- 172.21.15.76:0/258575999 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f1754024080 con 0x55f7cddf0690 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 10 client.7743 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 10 client.7743 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 20 client.7743 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddfeea0 0x55f7cde01290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 4 client.7743 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7cdb8ee10 con 0x55f7cddfeea0 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175b7fe640 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddfeea0 0x55f7cde01290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f175b7fe640 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddfeea0 0x55f7cde01290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3ae91bfa51751f server_cookie=4cc8b34e65c6558d in_seq=0 out_seq=0 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 10 client.7743 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 1 -- 172.21.15.76:0/258575999 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f7cdb8ee10 con 0x55f7cddfeea0 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 10 client.7743 handle_command_reply: tid=0 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 client.7743 shutdown 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 msgr2=0x55f7cddf7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cddf7550 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f175400e7e0 tx=0x7f175400eca0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 shutdown_connections 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddfeea0 0x55f7cde01290 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddf0690 0x55f7cddf7550 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cddefd70 0x55f7cddf7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f173a7fc640 0 client.7743 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 --2- 172.21.15.76:0/258575999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cdda3db0 0x55f7cddf6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 >> 172.21.15.76:0/258575999 conn(0x55f7cddd1040 msgr2=0x55f7cdda2f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 shutdown_connections 2024-09-19T04:19:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 1 -- 172.21.15.76:0/258575999 wait complete. 2024-09-19T04:19:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.865+0000 7f1764cd2000 20 client.7743 trim_cache size 0 max 16384 2024-09-19T04:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 Processor -- start 2024-09-19T04:19:14.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 -- start start 2024-09-19T04:19:14.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde641f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde64730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde64c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1ffbd0 con 0x55f7cddefd70 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f175c1feda0 con 0x55f7cddd1040 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f175c1ff4a0 con 0x55f7cdda3db0 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f175bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde64c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f175bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde64c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44210/0 (socket says 172.21.15.76:44210) 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.881+0000 7f175bfff640 1 -- 172.21.15.76:0/983167577 learned_addr learned my addr 172.21.15.76:0/983167577 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 -- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 msgr2=0x55f7cde64730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175affd640 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde64730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175b7fe640 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde641f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde64730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 -- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 msgr2=0x55f7cde641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde641f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 -- 172.21.15.76:0/983167577 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7cdbb2470 con 0x55f7cddefd70 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175affd640 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde64730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde64c70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f175400cb80 tx=0x7f1754011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e324be8577dab5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1740ff9640 1 -- 172.21.15.76:0/983167577 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175401b030 con 0x55f7cddefd70 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1740ff9640 1 -- 172.21.15.76:0/983167577 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175400e270 con 0x55f7cddefd70 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1740ff9640 1 -- 172.21.15.76:0/983167577 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175402b5c0 con 0x55f7cddefd70 2024-09-19T04:19:14.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/983167577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 msgr2=0x55f7cde64c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde64c70 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f175400cb80 tx=0x7f1754011000 comp rx=0 tx=0).stop 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/983167577 shutdown_connections 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde64c70 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde64730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- 172.21.15.76:0/983167577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde641f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/983167577 >> 172.21.15.76:0/983167577 conn(0x55f7cdda8770 msgr2=0x55f7cde03690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/983167577 shutdown_connections 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/983167577 wait complete. 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- start start 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 Processor -- start 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde68a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde68f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde238d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f7cdb9af20 con 0x55f7cddefd70 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f175c06f090 con 0x55f7cddd1040 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f7cdbb33f0 con 0x55f7cdda3db0 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 10 client.0 _ll_register_callbacks cb 0x7f17540096a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde238d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde238d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44216/0 (socket says 172.21.15.76:44216) 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 -- 172.21.15.76:0/600403132 learned_addr learned my addr 172.21.15.76:0/600403132 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175b7fe640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde68a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175affd640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde68f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 -- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 msgr2=0x55f7cde68a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde68a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 -- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 msgr2=0x55f7cde68f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:14.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde68f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 -- 172.21.15.76:0/600403132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7cdb88640 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175affd640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde68f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175bfff640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde238d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f175400e7e0 tx=0x7f1754004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10df0340ff2186bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f173a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f173a7fc640 1 -- 172.21.15.76:0/600403132 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f175401b070 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f173a7fc640 1 -- 172.21.15.76:0/600403132 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f175400e070 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f175400e270 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f173a7fc640 1 -- 172.21.15.76:0/600403132 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f175403d050 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 10 client.7749 fetch_fsmap learned FSMap version 91 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f7cdb8c3b0 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f173a7fc640 1 -- 172.21.15.76:0/600403132 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f175400e270 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f173a7fc640 1 -- 172.21.15.76:0/600403132 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 91) ==== 3554+0+0 (secure 0 0 0) 0x7f1754040020 con 0x55f7cddefd70 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 10 client.7749 fetch_fsmap finished waiting for FSMap version 91 2024-09-19T04:19:14.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 10 client.7749 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 20 client.7749 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddf0690 0x55f7cde23e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 4 client.7749 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f7cdc02470 con 0x55f7cddf0690 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.885+0000 7f175b7fe640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddf0690 0x55f7cde23e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f175b7fe640 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddf0690 0x55f7cde23e10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a049d3411c740565 server_cookie=d36a8c8722b91d5f in_seq=0 out_seq=0 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f173a7fc640 10 client.7749 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f173a7fc640 1 -- 172.21.15.76:0/600403132 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55f7cdc02470 con 0x55f7cddf0690 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f173a7fc640 10 client.7749 handle_command_reply: tid=0 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 client.7749 shutdown 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 msgr2=0x55f7cde238d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde238d0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f175400e7e0 tx=0x7f1754004a50 comp rx=0 tx=0).stop 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 shutdown_connections 2024-09-19T04:19:14.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55f7cddf0690 0x55f7cde23e10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f7cddefd70 0x55f7cde238d0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f7cddd1040 0x55f7cde68f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 --2- 172.21.15.76:0/600403132 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f7cdda3db0 0x55f7cde68a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f173a7fc640 0 client.7749 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 >> 172.21.15.76:0/600403132 conn(0x55f7cdda8770 msgr2=0x55f7cddfd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 shutdown_connections 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 1 -- 172.21.15.76:0/600403132 wait complete. 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1764cd2000 20 client.7749 trim_cache size 0 max 16384 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:19:14.897 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:19:14.898 INFO:teuthology.orchestra.run.smithi076.stdout: "8362a942-ff57-460f-be25-9fdf5af5b05a": { 2024-09-19T04:19:14.898 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:19:14.898 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "8362a942-ff57-460f-be25-9fdf5af5b05a", 2024-09-19T04:19:14.898 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:19:14.898 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:14.898 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:14.898 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17480678c0 msgr2=0x7f1748069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17480678c0 0x7f1748069d80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f174c00b820 tx=0x7f174c008040 comp rx=0 tx=0).stop 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c1520a0 msgr2=0x7f175c098050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c1520a0 0x7f175c098050 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1750000c10 tx=0x7f1750018fe0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 shutdown_connections 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17480678c0 0x7f1748069d80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f175c15f1d0 0x7f175c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f175c1520a0 0x7f175c098050 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 --2- 172.21.15.76:0/1066601072 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f175c06fe50 0x7f175c14afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 >> 172.21.15.76:0/1066601072 conn(0x7f175c0b1f90 msgr2=0x7f175c15c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:14.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 shutdown_connections 2024-09-19T04:19:14.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:14.889+0000 7f1761c16640 1 -- 172.21.15.76:0/1066601072 wait complete. 2024-09-19T04:19:14.903 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:19:14.903 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.009+0000 7fd0922b3640 1 Processor -- start 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.009+0000 7fd0922b3640 1 -- start start 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd0922b3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd08c1520a0 0x7fd08c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd0922b3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd08c15f3a0 0x7fd08c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd0922b3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd08c070250 con 0x7fd08c06fe50 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd0922b3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd08c152480 con 0x7fd08c15f3a0 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd0922b3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd08c0aede0 con 0x7fd08c1520a0 2024-09-19T04:19:15.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd08c15f3a0 0x7fd08c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd08c15f3a0 0x7fd08c0adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44228/0 (socket says 172.21.15.76:44228) 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 -- 172.21.15.76:0/1024086800 learned_addr learned my addr 172.21.15.76:0/1024086800 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd08b7fe640 1 --1- 172.21.15.76:0/1024086800 >> v1:172.21.15.76:6789/0 conn(0x7fd08c06fe50 0x7fd08c15ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45640/0 (socket says 172.21.15.76:45640) 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd08bfff640 1 --2- 172.21.15.76:0/1024086800 >> v2:172.21.15.116:3300/0 conn(0x7fd08c1520a0 0x7fd08c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 -- 172.21.15.76:0/1024086800 >> v2:172.21.15.116:3300/0 conn(0x7fd08c1520a0 msgr2=0x7fd08c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 --2- 172.21.15.76:0/1024086800 >> v2:172.21.15.116:3300/0 conn(0x7fd08c1520a0 0x7fd08c06f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 -- 172.21.15.76:0/1024086800 >> v1:172.21.15.76:6789/0 conn(0x7fd08c06fe50 legacy=0x7fd08c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd08affd640 1 -- 172.21.15.76:0/1024086800 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2474915493 0 0) 0x7fd08c070250 con 0x7fd08c06fe50 2024-09-19T04:19:15.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 -- 172.21.15.76:0/1024086800 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c15fa10 con 0x7fd08c15f3a0 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd090850640 1 --2- 172.21.15.76:0/1024086800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c15f3a0 0x7fd08c0adc80 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd08c071710 tx=0x7fd08401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165c4f5d334c221e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd08affd640 1 -- 172.21.15.76:0/1024086800 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0840a57a0 con 0x7fd08c15f3a0 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd08affd640 1 -- 172.21.15.76:0/1024086800 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0840a5940 con 0x7fd08c15f3a0 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.013+0000 7fd08affd640 1 -- 172.21.15.76:0/1024086800 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0840a5d10 con 0x7fd08c15f3a0 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- 172.21.15.76:0/1024086800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c15f3a0 msgr2=0x7fd08c0adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 --2- 172.21.15.76:0/1024086800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c15f3a0 0x7fd08c0adc80 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd08c071710 tx=0x7fd08401c9f0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- 172.21.15.76:0/1024086800 shutdown_connections 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 --2- 172.21.15.76:0/1024086800 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c15f3a0 0x7fd08c0adc80 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 --2- 172.21.15.76:0/1024086800 >> v2:172.21.15.116:3300/0 conn(0x7fd08c1520a0 0x7fd08c06f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- 172.21.15.76:0/1024086800 >> 172.21.15.76:0/1024086800 conn(0x7fd08c0b1f90 msgr2=0x7fd08c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:15.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- 172.21.15.76:0/1024086800 shutdown_connections 2024-09-19T04:19:15.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- 172.21.15.76:0/1024086800 wait complete. 2024-09-19T04:19:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 Processor -- start 2024-09-19T04:19:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- start start 2024-09-19T04:19:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c06fe50 0x7fd08c0ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd08c1520a0 0x7fd08c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd08c15f3a0 0x7fd08c0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c073d30 con 0x7fd08c06fe50 2024-09-19T04:19:15.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c073a50 con 0x7fd08c15f3a0 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd08c06e800 con 0x7fd08c1520a0 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c06fe50 0x7fd08c0ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c06fe50 0x7fd08c0ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44234/0 (socket says 172.21.15.76:44234) 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 -- 172.21.15.76:0/3536500488 learned_addr learned my addr 172.21.15.76:0/3536500488 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 -- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd08c1520a0 msgr2=0x7fd08c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08b7fe640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd08c1520a0 0x7fd08c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd090850640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd08c15f3a0 0x7fd08c0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd08c1520a0 0x7fd08c0cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 -- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd08c15f3a0 msgr2=0x7fd08c0cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd08c15f3a0 0x7fd08c0cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 -- 172.21.15.76:0/3536500488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c164b80 con 0x7fd08c06fe50 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd08bfff640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c06fe50 0x7fd08c0ccdd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00d800 tx=0x7fd07c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd088ff9640 1 -- 172.21.15.76:0/3536500488 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00b6c0 con 0x7fd08c06fe50 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd088ff9640 1 -- 172.21.15.76:0/3536500488 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd07c019070 con 0x7fd08c06fe50 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c164e60 con 0x7fd08c06fe50 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd088ff9640 1 -- 172.21.15.76:0/3536500488 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00ba60 con 0x7fd08c06fe50 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.017+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c165420 con 0x7fd08c06fe50 2024-09-19T04:19:15.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.021+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08c0adc80 con 0x7fd08c06fe50 2024-09-19T04:19:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.021+0000 7fd088ff9640 1 -- 172.21.15.76:0/3536500488 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd07c00bc00 con 0x7fd08c06fe50 2024-09-19T04:19:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.021+0000 7fd088ff9640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0780678f0 0x7fd078069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.021+0000 7fd08b7fe640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0780678f0 0x7fd078069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.021+0000 7fd088ff9640 1 -- 172.21.15.76:0/3536500488 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fd07c086370 con 0x7fd08c06fe50 2024-09-19T04:19:15.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.021+0000 7fd08b7fe640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0780678f0 0x7fd078069db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd08c14b750 tx=0x7fd080009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.025+0000 7fd088ff9640 1 -- 172.21.15.76:0/3536500488 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd07c056c20 con 0x7fd08c06fe50 2024-09-19T04:19:15.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.217+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd08c06e540 con 0x7fd08c06fe50 2024-09-19T04:19:15.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.217+0000 7fd088ff9640 1 -- 172.21.15.76:0/3536500488 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5270 (secure 0 0 0) 0x7fd07c056520 con 0x7fd08c06fe50 2024-09-19T04:19:15.219 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:15.219 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"btime":"2024-09-19T04:19:11:952624+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:11.010206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[5794,7305,7311]},"id":6}]} 2024-09-19T04:19:15.219 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 91 2024-09-19T04:19:15.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0780678f0 msgr2=0x7fd078069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0780678f0 0x7fd078069db0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd08c14b750 tx=0x7fd080009040 comp rx=0 tx=0).stop 2024-09-19T04:19:15.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c06fe50 msgr2=0x7fd08c0ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c06fe50 0x7fd08c0ccdd0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00d800 tx=0x7fd07c00dcc0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 shutdown_connections 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0780678f0 0x7fd078069db0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd08c15f3a0 0x7fd08c0cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd08c1520a0 0x7fd08c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 --2- 172.21.15.76:0/3536500488 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd08c06fe50 0x7fd08c0ccdd0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 >> 172.21.15.76:0/3536500488 conn(0x7fd08c0b1f90 msgr2=0x7fd08c06ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 shutdown_connections 2024-09-19T04:19:15.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.221+0000 7fd0922b3640 1 -- 172.21.15.76:0/3536500488 wait complete. 2024-09-19T04:19:15.236 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:15.237 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:15.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.349+0000 7f3ae8b88640 1 Processor -- start 2024-09-19T04:19:15.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae8b88640 1 -- start start 2024-09-19T04:19:15.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae8b88640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3ae406d3e0 0x7f3ae40b5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae8b88640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3ae40b50c0 0x7f3ae40b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae8b88640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ae40ae780 con 0x7f3ae4158a90 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae8b88640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae40b57a0 con 0x7f3ae406d3e0 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae8b88640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3ae415e960 con 0x7f3ae40b50c0 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae259c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3ae4158a90 0x7f3ae406ccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45648/0 (socket says 172.21.15.76:45648) 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae1d9b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3ae406d3e0 0x7f3ae40b5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae259c640 1 -- 172.21.15.76:0/3105277246 learned_addr learned my addr 172.21.15.76:0/3105277246 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae1d9b640 1 -- 172.21.15.76:0/3105277246 >> v2:172.21.15.116:3300/0 conn(0x7f3ae40b50c0 msgr2=0x7f3ae40b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae2d9d640 1 --2- 172.21.15.76:0/3105277246 >> v2:172.21.15.116:3300/0 conn(0x7f3ae40b50c0 0x7f3ae40b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae1d9b640 1 --2- 172.21.15.76:0/3105277246 >> v2:172.21.15.116:3300/0 conn(0x7f3ae40b50c0 0x7f3ae40b3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae1d9b640 1 -- 172.21.15.76:0/3105277246 >> v1:172.21.15.76:6789/0 conn(0x7f3ae4158a90 legacy=0x7f3ae406ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae159a640 1 -- 172.21.15.76:0/3105277246 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2154720171 0 0) 0x7f3ad4001120 con 0x7f3ae4158a90 2024-09-19T04:19:15.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae1d9b640 1 -- 172.21.15.76:0/3105277246 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae415ec40 con 0x7f3ae406d3e0 2024-09-19T04:19:15.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae2d9d640 1 --2- 172.21.15.76:0/3105277246 >> v2:172.21.15.116:3300/0 conn(0x7f3ae40b50c0 0x7f3ae40b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:15.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae1d9b640 1 --2- 172.21.15.76:0/3105277246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae40b5be0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4073a10 tx=0x7f3acc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d470de7c2a7d2e3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae159a640 1 -- 172.21.15.76:0/3105277246 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0b2070 con 0x7f3ae406d3e0 2024-09-19T04:19:15.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae159a640 1 -- 172.21.15.76:0/3105277246 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3acc0a9070 con 0x7f3ae406d3e0 2024-09-19T04:19:15.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.353+0000 7f3ae159a640 1 -- 172.21.15.76:0/3105277246 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc0ad880 con 0x7f3ae406d3e0 2024-09-19T04:19:15.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- 172.21.15.76:0/3105277246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 msgr2=0x7f3ae40b5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/3105277246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae40b5be0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4073a10 tx=0x7f3acc01c450 comp rx=0 tx=0).stop 2024-09-19T04:19:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- 172.21.15.76:0/3105277246 shutdown_connections 2024-09-19T04:19:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/3105277246 >> v2:172.21.15.116:3300/0 conn(0x7f3ae40b50c0 0x7f3ae40b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/3105277246 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae40b5be0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- 172.21.15.76:0/3105277246 >> 172.21.15.76:0/3105277246 conn(0x7f3ae4072670 msgr2=0x7f3ae4072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- 172.21.15.76:0/3105277246 shutdown_connections 2024-09-19T04:19:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- 172.21.15.76:0/3105277246 wait complete. 2024-09-19T04:19:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 Processor -- start 2024-09-19T04:19:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- start start 2024-09-19T04:19:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae4146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ae40b50c0 0x7f3ae415a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ae4158a90 0x7f3ae40b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae416afb0 con 0x7f3ae406d3e0 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3ae415f720 con 0x7f3ae40b50c0 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3ae416b130 con 0x7f3ae4158a90 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae4146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae4146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44252/0 (socket says 172.21.15.76:44252) 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 -- 172.21.15.76:0/1459211282 learned_addr learned my addr 172.21.15.76:0/1459211282 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae2d9d640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ae4158a90 0x7f3ae40b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae1d9b640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ae40b50c0 0x7f3ae415a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 -- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ae4158a90 msgr2=0x7f3ae40b0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ae4158a90 0x7f3ae40b0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 -- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ae40b50c0 msgr2=0x7f3ae415a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ae40b50c0 0x7f3ae415a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 -- 172.21.15.76:0/1459211282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ae414ce10 con 0x7f3ae406d3e0 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae1d9b640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ae40b50c0 0x7f3ae415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae259c640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae4146e50 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c4b0 tx=0x7f3ad400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3adaffd640 1 -- 172.21.15.76:0/1459211282 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4018070 con 0x7f3ae406d3e0 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3adaffd640 1 -- 172.21.15.76:0/1459211282 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ad4007500 con 0x7f3ae406d3e0 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3adaffd640 1 -- 172.21.15.76:0/1459211282 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4004a30 con 0x7f3ae406d3e0 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ae414d0f0 con 0x7f3ae406d3e0 2024-09-19T04:19:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.357+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ae414d6b0 con 0x7f3ae406d3e0 2024-09-19T04:19:15.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.361+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ae40b5be0 con 0x7f3ae406d3e0 2024-09-19T04:19:15.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.361+0000 7f3adaffd640 1 -- 172.21.15.76:0/1459211282 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3ad401b070 con 0x7f3ae406d3e0 2024-09-19T04:19:15.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.361+0000 7f3adaffd640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ac40678c0 0x7f3ac4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:15.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.361+0000 7f3adaffd640 1 -- 172.21.15.76:0/1459211282 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f3ad4085160 con 0x7f3ae406d3e0 2024-09-19T04:19:15.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.361+0000 7f3ae1d9b640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ac40678c0 0x7f3ac4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:15.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.361+0000 7f3ae1d9b640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ac40678c0 0x7f3ac4069d80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0af000 tx=0x7f3acc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:15.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.365+0000 7f3adaffd640 1 -- 172.21.15.76:0/1459211282 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3ae40ae780 con 0x7f3ae406d3e0 2024-09-19T04:19:15.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.569+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f3ae406c990 con 0x7f3ae406d3e0 2024-09-19T04:19:15.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.569+0000 7f3adaffd640 1 -- 172.21.15.76:0/1459211282 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1382 (secure 0 0 0) 0x7f3ad4055aa0 con 0x7f3ae406d3e0 2024-09-19T04:19:15.570 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:15.570 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":374,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":694895,"bytes_used":261849088,"bytes_avail":772832264192,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":14203,"read_op_per_sec":0,"write_op_per_sec":11},"fsmap":{"epoch":91,"btime":"2024-09-19T04:19:11:952624+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-19T04:18:33.838048+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:15.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ac40678c0 msgr2=0x7f3ac4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ac40678c0 0x7f3ac4069d80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f3acc0af000 tx=0x7f3acc008040 comp rx=0 tx=0).stop 2024-09-19T04:19:15.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 msgr2=0x7f3ae4146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:15.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae4146e50 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c4b0 tx=0x7f3ad400c970 comp rx=0 tx=0).stop 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 shutdown_connections 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3ac40678c0 0x7f3ac4069d80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3ae4158a90 0x7f3ae40b0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ae40b50c0 0x7f3ae415a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 --2- 172.21.15.76:0/1459211282 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3ae406d3e0 0x7f3ae4146e50 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 >> 172.21.15.76:0/1459211282 conn(0x7f3ae4072670 msgr2=0x7f3ae415c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 shutdown_connections 2024-09-19T04:19:15.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:15.573+0000 7f3ae8b88640 1 -- 172.21.15.76:0/1459211282 wait complete. 2024-09-19T04:19:15.588 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-19T04:19:16.589 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:16.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.717+0000 7f1505352640 1 Processor -- start 2024-09-19T04:19:16.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.717+0000 7f1505352640 1 -- start start 2024-09-19T04:19:16.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f1505352640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f150015e470 0x7f15000d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f1505352640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 0x7f15000d4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f1505352640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15000d2b20 con 0x7f15000d4750 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f1505352640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f15000d3bc0 con 0x7f150015e470 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f1505352640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f15000d3730 con 0x7f15000b38e0 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f15000d4750 0x7f1500161e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45658/0 (socket says 172.21.15.76:45658) 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 -- 172.21.15.76:0/2052327615 learned_addr learned my addr 172.21.15.76:0/2052327615 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fe7fc640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 0x7f15000d4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1244760220 0 0) 0x7f15000d2b20 con 0x7f15000d4750 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14feffd640 1 --2- 172.21.15.76:0/2052327615 >> v2:172.21.15.116:3300/0 conn(0x7f150015e470 0x7f15000d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14e8003530 con 0x7f15000d4750 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 165154067 0 0) 0x7f14e8003530 con 0x7f15000d4750 2024-09-19T04:19:16.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 msgr2=0x7f15000d4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 0x7f15000d4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 >> v2:172.21.15.116:3300/0 conn(0x7f150015e470 msgr2=0x7f15000d15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 --2- 172.21.15.76:0/2052327615 >> v2:172.21.15.116:3300/0 conn(0x7f150015e470 0x7f15000d15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15000d2e00 con 0x7f15000d4750 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f14f8001a00 con 0x7f15000d4750 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 >> v1:172.21.15.76:6789/0 conn(0x7f15000d4750 legacy=0x7f1500161e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 0x7f14e80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14e8003f40 0x7f14e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14e8004d50 0x7f14e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15000d3bc0 con 0x7f14e8004d50 2024-09-19T04:19:16.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15000d3730 con 0x7f14e8003f40 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f14e800ba20 con 0x7f15000b38e0 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14e8004d50 0x7f14e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14fe7fc640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14e8003f40 0x7f14e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14feffd640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 0x7f14e80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 -- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 msgr2=0x7f14e80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 0x7f14e80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 -- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14e8003f40 msgr2=0x7f14e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14e8003f40 0x7f14e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14ff7fe640 1 -- 172.21.15.76:0/2052327615 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e8003530 con 0x7f14e8004d50 2024-09-19T04:19:16.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.721+0000 7f14feffd640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b38e0 0x7f14e80039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f14ff7fe640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14e8004d50 0x7f14e80051d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f14f800a8a0 tx=0x7f14f801fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376122a636cff3bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f800a4c0 con 0x7f14e8004d50 2024-09-19T04:19:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f80aa710 con 0x7f14e8004d50 2024-09-19T04:19:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f14fdffb640 1 -- 172.21.15.76:0/2052327615 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f80aab00 con 0x7f14e8004d50 2024-09-19T04:19:16.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- 172.21.15.76:0/2052327615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14e8004d50 msgr2=0x7f14e80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 --2- 172.21.15.76:0/2052327615 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14e8004d50 0x7f14e80051d0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f14f800a8a0 tx=0x7f14f801fae0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f14ff7fe640 1 -- 172.21.15.76:0/2052327615 reap_dead start 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- 172.21.15.76:0/2052327615 shutdown_connections 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- 172.21.15.76:0/2052327615 >> 172.21.15.76:0/2052327615 conn(0x7f150009d730 msgr2=0x7f150008bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- 172.21.15.76:0/2052327615 shutdown_connections 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- 172.21.15.76:0/2052327615 wait complete. 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 Processor -- start 2024-09-19T04:19:16.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- start start 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1500154940 0x7f1500157140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1500157680 0x7f15000b83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b8900 0x7f15000b8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15000d1b10 con 0x7f1500157680 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15000d33c0 con 0x7f1500154940 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.725+0000 7f1505352640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f150006e1c0 con 0x7f15000b8900 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1500157680 0x7f15000b83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1500157680 0x7f15000b83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44270/0 (socket says 172.21.15.76:44270) 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 -- 172.21.15.76:0/2964780825 learned_addr learned my addr 172.21.15.76:0/2964780825 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:16.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14ff7fe640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b8900 0x7f15000b8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14feffd640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1500154940 0x7f1500157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 -- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b8900 msgr2=0x7f15000b8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b8900 0x7f15000b8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 -- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1500154940 msgr2=0x7f1500157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1500154940 0x7f1500157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 -- 172.21.15.76:0/2964780825 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150015a7f0 con 0x7f1500157680 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14fe7fc640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1500157680 0x7f15000b83c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f14f40115f0 tx=0x7f14f4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14e77fe640 1 -- 172.21.15.76:0/2964780825 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4015870 con 0x7f1500157680 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14e77fe640 1 -- 172.21.15.76:0/2964780825 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14f4015a10 con 0x7f1500157680 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14e77fe640 1 -- 172.21.15.76:0/2964780825 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4015e00 con 0x7f1500157680 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f150015aad0 con 0x7f1500157680 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f150015b090 con 0x7f1500157680 2024-09-19T04:19:16.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15000d2a80 con 0x7f1500157680 2024-09-19T04:19:16.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14e77fe640 1 -- 172.21.15.76:0/2964780825 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f14f401d400 con 0x7f1500157680 2024-09-19T04:19:16.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14e77fe640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14ec0678c0 0x7f14ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:16.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14e77fe640 1 -- 172.21.15.76:0/2964780825 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f14f40866b0 con 0x7f1500157680 2024-09-19T04:19:16.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14feffd640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14ec0678c0 0x7f14ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:16.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.729+0000 7f14feffd640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14ec0678c0 0x7f14ec069d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009b40 tx=0x7f14f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:16.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.733+0000 7f14e77fe640 1 -- 172.21.15.76:0/2964780825 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f14f4019030 con 0x7f1500157680 2024-09-19T04:19:16.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15000d2420 con 0x7f1500157680 2024-09-19T04:19:16.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f14e77fe640 1 -- 172.21.15.76:0/2964780825 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5270 (secure 0 0 0) 0x7f14f4057080 con 0x7f1500157680 2024-09-19T04:19:16.929 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:16.929 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":91,"btime":"2024-09-19T04:19:11:952624+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:11.010206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[5794,7305,7311]},"id":6}]} 2024-09-19T04:19:16.929 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 91 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14ec0678c0 msgr2=0x7f14ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14ec0678c0 0x7f14ec069d80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f14f0009b40 tx=0x7f14f0008040 comp rx=0 tx=0).stop 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1500157680 msgr2=0x7f15000b83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1500157680 0x7f15000b83c0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f14f40115f0 tx=0x7f14f4011ab0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 shutdown_connections 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14ec0678c0 0x7f14ec069d80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15000b8900 0x7f15000b8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1500157680 0x7f15000b83c0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 --2- 172.21.15.76:0/2964780825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1500154940 0x7f1500157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:16.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 >> 172.21.15.76:0/2964780825 conn(0x7f150009d730 msgr2=0x7f15000b3d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:16.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 shutdown_connections 2024-09-19T04:19:16.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:16.929+0000 7f1505352640 1 -- 172.21.15.76:0/2964780825 wait complete. 2024-09-19T04:19:16.947 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:16.947 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:16.976 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:19:16.977+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:19:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.061+0000 7f1ea7567640 1 Processor -- start 2024-09-19T04:19:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.061+0000 7f1ea7567640 1 -- start start 2024-09-19T04:19:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.061+0000 7f1ea7567640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ea01520a0 0x7f1ea006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.061+0000 7f1ea7567640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ea0070250 con 0x7f1ea006fe50 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1ea0152480 con 0x7f1ea01520a0 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea00aede0 con 0x7f1ea015f3a0 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ea01520a0 0x7f1ea006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1ea01520a0 0x7f1ea006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44274/0 (socket says 172.21.15.76:44274) 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 -- 172.21.15.76:0/3783503455 learned_addr learned my addr 172.21.15.76:0/3783503455 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5b04640 1 --2- 172.21.15.76:0/3783503455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 -- 172.21.15.76:0/3783503455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 msgr2=0x7f1ea00adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 --2- 172.21.15.76:0/3783503455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea00adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 -- 172.21.15.76:0/3783503455 >> v1:172.21.15.116:6789/0 conn(0x7f1ea006fe50 legacy=0x7f1ea015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:17.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 -- 172.21.15.76:0/3783503455 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea015fa10 con 0x7f1ea01520a0 2024-09-19T04:19:17.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea5303640 1 --2- 172.21.15.76:0/3783503455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea006f910 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009e50 tx=0x7f1e9801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e00f828534498169 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:17.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1e9ffff640 1 -- 172.21.15.76:0/3783503455 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980a5760 con 0x7f1ea01520a0 2024-09-19T04:19:17.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1e9ffff640 1 -- 172.21.15.76:0/3783503455 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e980a5900 con 0x7f1ea01520a0 2024-09-19T04:19:17.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1e9ffff640 1 -- 172.21.15.76:0/3783503455 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e980a5cd0 con 0x7f1ea01520a0 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- 172.21.15.76:0/3783503455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 msgr2=0x7f1ea006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3783503455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea006f910 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1e98009e50 tx=0x7f1e9801c9f0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- 172.21.15.76:0/3783503455 shutdown_connections 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3783503455 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3783503455 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea006f910 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- 172.21.15.76:0/3783503455 >> 172.21.15.76:0/3783503455 conn(0x7f1ea00b1f90 msgr2=0x7f1ea00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- 172.21.15.76:0/3783503455 shutdown_connections 2024-09-19T04:19:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.065+0000 7f1ea7567640 1 -- 172.21.15.76:0/3783503455 wait complete. 2024-09-19T04:19:17.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 Processor -- start 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 -- start start 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea006fe50 0x7f1ea01f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea0145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea01494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0073d30 con 0x7f1ea01520a0 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea0073a50 con 0x7f1ea006fe50 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea0167f10 con 0x7f1ea015f3a0 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea0145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea0145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44276/0 (socket says 172.21.15.76:44276) 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 -- 172.21.15.76:0/3880968034 learned_addr learned my addr 172.21.15.76:0/3880968034 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea5303640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea006fe50 0x7f1ea01f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea5b04640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea01494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 -- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 msgr2=0x7f1ea01494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea01494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 -- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea006fe50 msgr2=0x7f1ea01f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea006fe50 0x7f1ea01f7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 -- 172.21.15.76:0/3880968034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea014b810 con 0x7f1ea01520a0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea5b04640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea01494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea4b02640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea0145460 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400b440 tx=0x7f1e9400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1e9dffb640 1 -- 172.21.15.76:0/3880968034 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94015020 con 0x7f1ea01520a0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1e9dffb640 1 -- 172.21.15.76:0/3880968034 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e94011890 con 0x7f1ea01520a0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea014baf0 con 0x7f1ea01520a0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1e9dffb640 1 -- 172.21.15.76:0/3880968034 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e94011c80 con 0x7f1ea01520a0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea014c0b0 con 0x7f1ea01520a0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea006e540 con 0x7f1ea01520a0 2024-09-19T04:19:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1e9dffb640 1 -- 172.21.15.76:0/3880968034 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1e94018020 con 0x7f1ea01520a0 2024-09-19T04:19:17.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.069+0000 7f1e9dffb640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:17.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.073+0000 7f1e9dffb640 1 -- 172.21.15.76:0/3880968034 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f1e94085cf0 con 0x7f1ea01520a0 2024-09-19T04:19:17.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.073+0000 7f1ea5303640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:17.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.073+0000 7f1ea5303640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e8c0678c0 0x7f1e8c069d80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1e980ae000 tx=0x7f1e980a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:17.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.077+0000 7f1e9dffb640 1 -- 172.21.15.76:0/3880968034 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1e9408a070 con 0x7f1ea01520a0 2024-09-19T04:19:17.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.301+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f1ea006f340 con 0x7f1ea01520a0 2024-09-19T04:19:17.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1e9dffb640 1 -- 172.21.15.76:0/3880968034 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1498 (secure 0 0 0) 0x7f1e9401f030 con 0x7f1ea01520a0 2024-09-19T04:19:17.304 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:17.304 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":376,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":694895,"bytes_used":261849088,"bytes_avail":772832264192,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":9340,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":92,"btime":"2024-09-19T04:19:16:978898+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-09-19T04:18:33.838048+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:17.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e8c0678c0 msgr2=0x7f1e8c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:17.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e8c0678c0 0x7f1e8c069d80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1e980ae000 tx=0x7f1e980a7070 comp rx=0 tx=0).stop 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 msgr2=0x7f1ea0145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea0145460 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400b440 tx=0x7f1e9400b900 comp rx=0 tx=0).stop 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 shutdown_connections 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1e8c0678c0 0x7f1e8c069d80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1ea015f3a0 0x7f1ea01494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ea01520a0 0x7f1ea0145460 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 --2- 172.21.15.76:0/3880968034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ea006fe50 0x7f1ea01f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 >> 172.21.15.76:0/3880968034 conn(0x7f1ea00b1f90 msgr2=0x7f1ea00b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 shutdown_connections 2024-09-19T04:19:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:17.305+0000 7f1ea7567640 1 -- 172.21.15.76:0/3880968034 wait complete. 2024-09-19T04:19:17.323 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-19T04:19:18.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:18.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.453+0000 7f596c510640 1 Processor -- start 2024-09-19T04:19:18.455 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.453+0000 7f596c510640 1 -- start start 2024-09-19T04:19:18.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f596c510640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f596406fef0 0x7f596406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f596c510640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f596406ee50 con 0x7f59640b3d50 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f596c510640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f596406f130 con 0x7f5964073e60 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f596c510640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f596406f410 con 0x7f596406fef0 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f5969aab640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f596406fef0 0x7f596406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f596aaad640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f59640b3d50 0x7f5964072610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45660/0 (socket says 172.21.15.76:45660) 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f596aaad640 1 -- 172.21.15.76:0/3819096116 learned_addr learned my addr 172.21.15.76:0/3819096116 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f59692aa640 1 -- 172.21.15.76:0/3819096116 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1778019235 0 0) 0x7f596406ee50 con 0x7f59640b3d50 2024-09-19T04:19:18.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f59692aa640 1 -- 172.21.15.76:0/3819096116 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5950003530 con 0x7f59640b3d50 2024-09-19T04:19:18.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f59692aa640 1 -- 172.21.15.76:0/3819096116 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1425769657 0 0) 0x7f5950003530 con 0x7f59640b3d50 2024-09-19T04:19:18.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f5969aab640 1 -- 172.21.15.76:0/3819096116 >> v1:172.21.15.116:6789/0 conn(0x7f5964073e60 legacy=0x7f596415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f5969aab640 1 -- 172.21.15.76:0/3819096116 >> v1:172.21.15.76:6789/0 conn(0x7f59640b3d50 legacy=0x7f5964072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f5969aab640 1 -- 172.21.15.76:0/3819096116 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596406f6f0 con 0x7f596406fef0 2024-09-19T04:19:18.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f5969aab640 1 --2- 172.21.15.76:0/3819096116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596406fef0 0x7f596406e800 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f595400a960 tx=0x7f595401d700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2aed6c69e7573f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f59692aa640 1 -- 172.21.15.76:0/3819096116 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c001a00 con 0x7f596406fef0 2024-09-19T04:19:18.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f59692aa640 1 -- 172.21.15.76:0/3819096116 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59540a9450 con 0x7f596406fef0 2024-09-19T04:19:18.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.457+0000 7f59692aa640 1 -- 172.21.15.76:0/3819096116 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595400b040 con 0x7f596406fef0 2024-09-19T04:19:18.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- 172.21.15.76:0/3819096116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596406fef0 msgr2=0x7f596406e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 --2- 172.21.15.76:0/3819096116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596406fef0 0x7f596406e800 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f595400a960 tx=0x7f595401d700 comp rx=0 tx=0).stop 2024-09-19T04:19:18.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- 172.21.15.76:0/3819096116 shutdown_connections 2024-09-19T04:19:18.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 --2- 172.21.15.76:0/3819096116 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f596406fef0 0x7f596406e800 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- 172.21.15.76:0/3819096116 >> 172.21.15.76:0/3819096116 conn(0x7f59640b1f90 msgr2=0x7f59640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:18.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- 172.21.15.76:0/3819096116 shutdown_connections 2024-09-19T04:19:18.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- 172.21.15.76:0/3819096116 wait complete. 2024-09-19T04:19:18.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 Processor -- start 2024-09-19T04:19:18.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- start start 2024-09-19T04:19:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f596406fef0 0x7f5964153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5964073e60 0x7f596414d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59640b3d50 0x7f5964155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59640730f0 con 0x7f59640b3d50 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5964072f70 con 0x7f596406fef0 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5964073270 con 0x7f5964073e60 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59640b3d50 0x7f5964155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59640b3d50 0x7f5964155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44286/0 (socket says 172.21.15.76:44286) 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 -- 172.21.15.76:0/2580327677 learned_addr learned my addr 172.21.15.76:0/2580327677 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596a2ac640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f596406fef0 0x7f5964153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f5969aab640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5964073e60 0x7f596414d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 -- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5964073e60 msgr2=0x7f596414d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5964073e60 0x7f596414d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 -- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f596406fef0 msgr2=0x7f5964153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f596406fef0 0x7f5964153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 -- 172.21.15.76:0/2580327677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59641533c0 con 0x7f59640b3d50 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596a2ac640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f596406fef0 0x7f5964153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:18.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f5969aab640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5964073e60 0x7f596414d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:18.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596aaad640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59640b3d50 0x7f5964155430 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f595c0018a0 tx=0x7f595c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f595affd640 1 -- 172.21.15.76:0/2580327677 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c012400 con 0x7f59640b3d50 2024-09-19T04:19:18.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f595affd640 1 -- 172.21.15.76:0/2580327677 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f595c0125c0 con 0x7f59640b3d50 2024-09-19T04:19:18.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5964155970 con 0x7f59640b3d50 2024-09-19T04:19:18.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f595affd640 1 -- 172.21.15.76:0/2580327677 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0129b0 con 0x7f59640b3d50 2024-09-19T04:19:18.465 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.461+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596415aa30 con 0x7f59640b3d50 2024-09-19T04:19:18.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.465+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596406e800 con 0x7f59640b3d50 2024-09-19T04:19:18.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.465+0000 7f595affd640 1 -- 172.21.15.76:0/2580327677 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f595c012b50 con 0x7f59640b3d50 2024-09-19T04:19:18.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.465+0000 7f595affd640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59480678c0 0x7f5948069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.465+0000 7f596a2ac640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59480678c0 0x7f5948069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.465+0000 7f595affd640 1 -- 172.21.15.76:0/2580327677 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f595c0867b0 con 0x7f59640b3d50 2024-09-19T04:19:18.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.465+0000 7f596a2ac640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59480678c0 0x7f5948069d80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5960000f70 tx=0x7f5960009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.470 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.469+0000 7f595affd640 1 -- 172.21.15.76:0/2580327677 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f595c057050 con 0x7f59640b3d50 2024-09-19T04:19:18.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.661+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59640723f0 con 0x7f59640b3d50 2024-09-19T04:19:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.661+0000 7f595affd640 1 -- 172.21.15.76:0/2580327677 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5273 (secure 0 0 0) 0x7f595c056950 con 0x7f59640b3d50 2024-09-19T04:19:18.663 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:18.663 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":92,"btime":"2024-09-19T04:19:16:978898+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:16.196325+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:18.663 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 92 2024-09-19T04:19:18.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59480678c0 msgr2=0x7f5948069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59480678c0 0x7f5948069d80 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5960000f70 tx=0x7f5960009040 comp rx=0 tx=0).stop 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59640b3d50 msgr2=0x7f5964155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59640b3d50 0x7f5964155430 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f595c0018a0 tx=0x7f595c00a430 comp rx=0 tx=0).stop 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 shutdown_connections 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59480678c0 0x7f5948069d80 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59640b3d50 0x7f5964155430 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5964073e60 0x7f596414d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 --2- 172.21.15.76:0/2580327677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f596406fef0 0x7f5964153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 >> 172.21.15.76:0/2580327677 conn(0x7f59640b1f90 msgr2=0x7f59640b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 shutdown_connections 2024-09-19T04:19:18.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.665+0000 7f596c510640 1 -- 172.21.15.76:0/2580327677 wait complete. 2024-09-19T04:19:18.681 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:18.681 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:18.786 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2ef4d58640 1 Processor -- start 2024-09-19T04:19:18.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2ef4d58640 1 -- start start 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2ef4d58640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2ef00cf620 0x7f2ef0168f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2ef4d58640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef00d0300 con 0x7f2ef0161a60 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2ef4d58640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ef006e6c0 con 0x7f2ef00d3060 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2ef4d58640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2ef00cfa90 con 0x7f2ef00cf620 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2eeed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2ef00cf620 0x7f2ef0168f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2eeed9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2ef00cf620 0x7f2ef0168f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44300/0 (socket says 172.21.15.76:44300) 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2eeed9d640 1 -- 172.21.15.76:0/894114866 learned_addr learned my addr 172.21.15.76:0/894114866 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.785+0000 7f2eee59c640 1 --1- 172.21.15.76:0/894114866 >> v1:172.21.15.76:6789/0 conn(0x7f2ef0161a60 0x7f2ef00d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45664/0 (socket says 172.21.15.76:45664) 2024-09-19T04:19:18.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2eeed9d640 1 -- 172.21.15.76:0/894114866 >> v1:172.21.15.116:6789/0 conn(0x7f2ef00d3060 legacy=0x7f2ef00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:18.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2eeed9d640 1 -- 172.21.15.76:0/894114866 >> v1:172.21.15.76:6789/0 conn(0x7f2ef0161a60 legacy=0x7f2ef00d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2eeed9d640 1 -- 172.21.15.76:0/894114866 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef00b0f60 con 0x7f2ef00cf620 2024-09-19T04:19:18.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2eeed9d640 1 --2- 172.21.15.76:0/894114866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef0168f10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009870 tx=0x7f2ee801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92d51f8f80f8c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2eed59a640 1 -- 172.21.15.76:0/894114866 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80ab660 con 0x7f2ef00cf620 2024-09-19T04:19:18.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2eed59a640 1 -- 172.21.15.76:0/894114866 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee80b0070 con 0x7f2ef00cf620 2024-09-19T04:19:18.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2eed59a640 1 -- 172.21.15.76:0/894114866 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee80aba20 con 0x7f2ef00cf620 2024-09-19T04:19:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2ef4d58640 1 -- 172.21.15.76:0/894114866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 msgr2=0x7f2ef0168f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/894114866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef0168f10 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009870 tx=0x7f2ee801c460 comp rx=0 tx=0).stop 2024-09-19T04:19:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2ef4d58640 1 -- 172.21.15.76:0/894114866 shutdown_connections 2024-09-19T04:19:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/894114866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef0168f10 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2ef4d58640 1 -- 172.21.15.76:0/894114866 >> 172.21.15.76:0/894114866 conn(0x7f2ef008bf90 msgr2=0x7f2ef009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2ef4d58640 1 -- 172.21.15.76:0/894114866 shutdown_connections 2024-09-19T04:19:18.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.789+0000 7f2ef4d58640 1 -- 172.21.15.76:0/894114866 wait complete. 2024-09-19T04:19:18.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 Processor -- start 2024-09-19T04:19:18.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 -- start start 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef00d3060 0x7f2ef00b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef0161a60 0x7f2ef015a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef016bc70 con 0x7f2ef00cf620 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef006d190 con 0x7f2ef00d3060 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef016bdf0 con 0x7f2ef0161a60 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef015c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44304/0 (socket says 172.21.15.76:44304) 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 -- 172.21.15.76:0/3918780674 learned_addr learned my addr 172.21.15.76:0/3918780674 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eeed9d640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef0161a60 0x7f2ef015a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eedd9b640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef00d3060 0x7f2ef00b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 -- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef0161a60 msgr2=0x7f2ef015a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef0161a60 0x7f2ef015a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 -- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef00d3060 msgr2=0x7f2ef00b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef00d3060 0x7f2ef00b72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 -- 172.21.15.76:0/3918780674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef00b7830 con 0x7f2ef00cf620 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eeed9d640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef0161a60 0x7f2ef015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2eee59c640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef015c770 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c3a0 tx=0x7f2ee000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ee6ffd640 1 -- 172.21.15.76:0/3918780674 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee00074e0 con 0x7f2ef00cf620 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ee6ffd640 1 -- 172.21.15.76:0/3918780674 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ee0007680 con 0x7f2ef00cf620 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ee6ffd640 1 -- 172.21.15.76:0/3918780674 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0004a30 con 0x7f2ef00cf620 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef015ce70 con 0x7f2ef00cf620 2024-09-19T04:19:18.796 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef01fa9d0 con 0x7f2ef00cf620 2024-09-19T04:19:18.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef00cef10 con 0x7f2ef00cf620 2024-09-19T04:19:18.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.793+0000 7f2ee6ffd640 1 -- 172.21.15.76:0/3918780674 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ee001a020 con 0x7f2ef00cf620 2024-09-19T04:19:18.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.797+0000 7f2ee6ffd640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ed00678c0 0x7f2ed0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:18.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.797+0000 7f2ee6ffd640 1 -- 172.21.15.76:0/3918780674 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f2ee0085590 con 0x7f2ef00cf620 2024-09-19T04:19:18.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.797+0000 7f2eedd9b640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ed00678c0 0x7f2ed0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:18.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.797+0000 7f2eedd9b640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ed00678c0 0x7f2ed0069d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ce7c0 tx=0x7f2ed8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:18.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:18.801+0000 7f2ee6ffd640 1 -- 172.21.15.76:0/3918780674 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2ef00b7830 con 0x7f2ef00cf620 2024-09-19T04:19:19.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.017+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f2ef00d2a80 con 0x7f2ef00cf620 2024-09-19T04:19:19.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ee6ffd640 1 -- 172.21.15.76:0/3918780674 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1926 (secure 0 0 0) 0x7f2ef00d0300 con 0x7f2ef00cf620 2024-09-19T04:19:19.020 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:19.020 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":378,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":695469,"bytes_used":261898240,"bytes_avail":772832215040,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7842,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":93,"btime":"2024-09-19T04:19:18:992120+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-09-19T04:19:17.851396+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ed00678c0 msgr2=0x7f2ed0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ed00678c0 0x7f2ed0069d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00ce7c0 tx=0x7f2ed8009040 comp rx=0 tx=0).stop 2024-09-19T04:19:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 msgr2=0x7f2ef015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef015c770 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000c3a0 tx=0x7f2ee000c860 comp rx=0 tx=0).stop 2024-09-19T04:19:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 shutdown_connections 2024-09-19T04:19:19.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ed00678c0 0x7f2ed0069d80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:19.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ef0161a60 0x7f2ef015a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:19.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ef00d3060 0x7f2ef00b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:19.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 --2- 172.21.15.76:0/3918780674 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ef00cf620 0x7f2ef015c770 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:19.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 >> 172.21.15.76:0/3918780674 conn(0x7f2ef008bf90 msgr2=0x7f2ef00b5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:19.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 shutdown_connections 2024-09-19T04:19:19.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:19.021+0000 7f2ef4d58640 1 -- 172.21.15.76:0/3918780674 wait complete. 2024-09-19T04:19:19.038 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:19:20.039 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:20.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e4a9fe640 1 Processor -- start 2024-09-19T04:19:20.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e4a9fe640 1 -- start start 2024-09-19T04:19:20.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e4a9fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e44073e60 0x7f2e4406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e4a9fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e440b3d50 0x7f2e440b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e4a9fe640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e440b4670 con 0x7f2e4406fef0 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e4a9fe640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4406ee50 con 0x7f2e44073e60 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e4a9fe640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2e4406f130 con 0x7f2e440b3d50 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e44073e60 0x7f2e4406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e44073e60 0x7f2e4406e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44316/0 (socket says 172.21.15.76:44316) 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 -- 172.21.15.76:0/1308111992 learned_addr learned my addr 172.21.15.76:0/1308111992 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 -- 172.21.15.76:0/1308111992 >> v2:172.21.15.116:3300/0 conn(0x7f2e440b3d50 msgr2=0x7f2e440b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 --2- 172.21.15.76:0/1308111992 >> v2:172.21.15.116:3300/0 conn(0x7f2e440b3d50 0x7f2e440b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 -- 172.21.15.76:0/1308111992 >> v1:172.21.15.116:6789/0 conn(0x7f2e4406fef0 legacy=0x7f2e4415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 -- 172.21.15.76:0/1308111992 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4406f410 con 0x7f2e44073e60 2024-09-19T04:19:20.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.169+0000 7f2e43fff640 1 --2- 172.21.15.76:0/1308111992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e44073e60 0x7f2e4406e800 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2e440b2d50 tx=0x7f2e3401e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34198b1b2191bf8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e42ffd640 1 -- 172.21.15.76:0/1308111992 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340ac5f0 con 0x7f2e44073e60 2024-09-19T04:19:20.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e42ffd640 1 -- 172.21.15.76:0/1308111992 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e340b1070 con 0x7f2e44073e60 2024-09-19T04:19:20.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e42ffd640 1 -- 172.21.15.76:0/1308111992 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e340ac9b0 con 0x7f2e44073e60 2024-09-19T04:19:20.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/1308111992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e44073e60 msgr2=0x7f2e4406e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/1308111992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e44073e60 0x7f2e4406e800 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2e440b2d50 tx=0x7f2e3401e670 comp rx=0 tx=0).stop 2024-09-19T04:19:20.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/1308111992 shutdown_connections 2024-09-19T04:19:20.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/1308111992 >> v2:172.21.15.116:3300/0 conn(0x7f2e440b3d50 0x7f2e440b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/1308111992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e44073e60 0x7f2e4406e800 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/1308111992 >> 172.21.15.76:0/1308111992 conn(0x7f2e440b1f90 msgr2=0x7f2e440b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/1308111992 shutdown_connections 2024-09-19T04:19:20.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/1308111992 wait complete. 2024-09-19T04:19:20.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 Processor -- start 2024-09-19T04:19:20.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- start start 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4406fef0 0x7f2e4414d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e44073e60 0x7f2e440b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e440b3d50 0x7f2e440d0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e440729a0 con 0x7f2e440b3d50 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44072820 con 0x7f2e4406fef0 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2e44072b20 con 0x7f2e44073e60 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e440b3d50 0x7f2e440d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e440b3d50 0x7f2e440d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44322/0 (socket says 172.21.15.76:44322) 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 -- 172.21.15.76:0/3492438715 learned_addr learned my addr 172.21.15.76:0/3492438715 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e437fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e44073e60 0x7f2e440b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e43fff640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4406fef0 0x7f2e4414d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 -- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e44073e60 msgr2=0x7f2e440b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e44073e60 0x7f2e440b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 -- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4406fef0 msgr2=0x7f2e4414d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4406fef0 0x7f2e4414d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 -- 172.21.15.76:0/3492438715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e44153790 con 0x7f2e440b3d50 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e437fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e44073e60 0x7f2e440b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e43fff640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4406fef0 0x7f2e4414d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e48f9b640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e440b3d50 0x7f2e440d0560 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00b970 tx=0x7f2e3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e40ff9640 1 -- 172.21.15.76:0/3492438715 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c019070 con 0x7f2e440b3d50 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e40ff9640 1 -- 172.21.15.76:0/3492438715 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e3c00cb60 con 0x7f2e440b3d50 2024-09-19T04:19:20.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e44153a70 con 0x7f2e440b3d50 2024-09-19T04:19:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e40ff9640 1 -- 172.21.15.76:0/3492438715 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c014440 con 0x7f2e440b3d50 2024-09-19T04:19:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.173+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e44154030 con 0x7f2e440b3d50 2024-09-19T04:19:20.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.177+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4406e800 con 0x7f2e440b3d50 2024-09-19T04:19:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.177+0000 7f2e40ff9640 1 -- 172.21.15.76:0/3492438715 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e3c01c070 con 0x7f2e440b3d50 2024-09-19T04:19:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.177+0000 7f2e40ff9640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678f0 0x7f2e30069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.177+0000 7f2e40ff9640 1 -- 172.21.15.76:0/3492438715 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f2e3c085b00 con 0x7f2e440b3d50 2024-09-19T04:19:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.177+0000 7f2e43fff640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678f0 0x7f2e30069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.177+0000 7f2e43fff640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678f0 0x7f2e30069db0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2e340b0000 tx=0x7f2e34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.181+0000 7f2e40ff9640 1 -- 172.21.15.76:0/3492438715 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2e3c0564d0 con 0x7f2e440b3d50 2024-09-19T04:19:20.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.377+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e440723f0 con 0x7f2e440b3d50 2024-09-19T04:19:20.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.377+0000 7f2e40ff9640 1 -- 172.21.15.76:0/3492438715 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7f2e3c055dd0 con 0x7f2e440b3d50 2024-09-19T04:19:20.379 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:20.379 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:20.379 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678f0 msgr2=0x7f2e30069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678f0 0x7f2e30069db0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2e340b0000 tx=0x7f2e34008040 comp rx=0 tx=0).stop 2024-09-19T04:19:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e440b3d50 msgr2=0x7f2e440d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e440b3d50 0x7f2e440d0560 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00b970 tx=0x7f2e3c00be30 comp rx=0 tx=0).stop 2024-09-19T04:19:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 shutdown_connections 2024-09-19T04:19:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e300678f0 0x7f2e30069db0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e440b3d50 0x7f2e440d0560 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e44073e60 0x7f2e440b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 --2- 172.21.15.76:0/3492438715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4406fef0 0x7f2e4414d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 >> 172.21.15.76:0/3492438715 conn(0x7f2e440b1f90 msgr2=0x7f2e4415b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 shutdown_connections 2024-09-19T04:19:20.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.381+0000 7f2e4a9fe640 1 -- 172.21.15.76:0/3492438715 wait complete. 2024-09-19T04:19:20.398 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:20.398 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:20.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.513+0000 7f257667d640 1 Processor -- start 2024-09-19T04:19:20.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.513+0000 7f257667d640 1 -- start start 2024-09-19T04:19:20.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f257667d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25701609f0 0x7f25700d0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f257667d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f25700d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f257667d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25700ce030 con 0x7f2570145960 2024-09-19T04:19:20.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f257667d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f25700ce310 con 0x7f25701609f0 2024-09-19T04:19:20.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f257667d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f25700ce5f0 con 0x7f25700cd8e0 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f25677fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f25700d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f256ffff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f25701609f0 0x7f25700d0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f25677fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f25700d09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:51052/0 (socket says 172.21.15.76:51052) 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f2574c1a640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2570145960 0x7f25700cf240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:53972/0 (socket says 172.21.15.76:53972) 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f25677fe640 1 -- 172.21.15.76:0/159761194 learned_addr learned my addr 172.21.15.76:0/159761194 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f25677fe640 1 -- 172.21.15.76:0/159761194 >> v2:172.21.15.116:3300/0 conn(0x7f25701609f0 msgr2=0x7f25700d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f256f7fe640 1 -- 172.21.15.76:0/159761194 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 950222421 0 0) 0x7f25700ce030 con 0x7f2570145960 2024-09-19T04:19:20.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f25677fe640 1 --2- 172.21.15.76:0/159761194 >> v2:172.21.15.116:3300/0 conn(0x7f25701609f0 0x7f25700d0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f25677fe640 1 -- 172.21.15.76:0/159761194 >> v1:172.21.15.116:6789/0 conn(0x7f2570145960 legacy=0x7f25700cf240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.517+0000 7f25677fe640 1 -- 172.21.15.76:0/159761194 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25700cdd20 con 0x7f25700cd8e0 2024-09-19T04:19:20.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f25677fe640 1 --2- 172.21.15.76:0/159761194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f25700d09a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2558009870 tx=0x7f255801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9a444d938bb0594 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f256f7fe640 1 -- 172.21.15.76:0/159761194 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580aa650 con 0x7f25700cd8e0 2024-09-19T04:19:20.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f256f7fe640 1 -- 172.21.15.76:0/159761194 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25580af070 con 0x7f25700cd8e0 2024-09-19T04:19:20.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f256f7fe640 1 -- 172.21.15.76:0/159761194 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580aaa10 con 0x7f25700cd8e0 2024-09-19T04:19:20.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- 172.21.15.76:0/159761194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 msgr2=0x7f25700d09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 --2- 172.21.15.76:0/159761194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f25700d09a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2558009870 tx=0x7f255801c4d0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- 172.21.15.76:0/159761194 shutdown_connections 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 --2- 172.21.15.76:0/159761194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f25700d09a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 --2- 172.21.15.76:0/159761194 >> v2:172.21.15.116:3300/0 conn(0x7f25701609f0 0x7f25700d0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- 172.21.15.76:0/159761194 >> 172.21.15.76:0/159761194 conn(0x7f257009aa00 msgr2=0x7f25700b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- 172.21.15.76:0/159761194 shutdown_connections 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- 172.21.15.76:0/159761194 wait complete. 2024-09-19T04:19:20.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 Processor -- start 2024-09-19T04:19:20.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- start start 2024-09-19T04:19:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f2570143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2570145960 0x7f257015ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25701609f0 0x7f25700c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2570169450 con 0x7f2570145960 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f25701692d0 con 0x7f25701609f0 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f257667d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f25701695d0 con 0x7f25700cd8e0 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f25677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2570145960 0x7f257015ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f25677fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2570145960 0x7f257015ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44332/0 (socket says 172.21.15.76:44332) 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.521+0000 7f25677fe640 1 -- 172.21.15.76:0/3626255064 learned_addr learned my addr 172.21.15.76:0/3626255064 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256ffff640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f2570143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f2574c1a640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25701609f0 0x7f25700c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f25677fe640 1 -- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 msgr2=0x7f2570143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f25677fe640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f2570143700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f25677fe640 1 -- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25701609f0 msgr2=0x7f25700c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f25677fe640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25701609f0 0x7f25700c7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f25677fe640 1 -- 172.21.15.76:0/3626255064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25701660c0 con 0x7f2570145960 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f25677fe640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2570145960 0x7f257015ee20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f25580ac040 tx=0x7f255801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256d7fa640 1 -- 172.21.15.76:0/3626255064 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580bd5f0 con 0x7f2570145960 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256d7fa640 1 -- 172.21.15.76:0/3626255064 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25580a6040 con 0x7f2570145960 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256d7fa640 1 -- 172.21.15.76:0/3626255064 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580bd9d0 con 0x7f2570145960 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2570143d90 con 0x7f2570145960 2024-09-19T04:19:20.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2570144350 con 0x7f2570145960 2024-09-19T04:19:20.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25700c9a20 con 0x7f2570145960 2024-09-19T04:19:20.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256d7fa640 1 -- 172.21.15.76:0/3626255064 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f25580c6440 con 0x7f2570145960 2024-09-19T04:19:20.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256d7fa640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f25500678c0 0x7f2550069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256ffff640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f25500678c0 0x7f2550069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256d7fa640 1 -- 172.21.15.76:0/3626255064 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f25581204e0 con 0x7f2570145960 2024-09-19T04:19:20.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.525+0000 7f256ffff640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f25500678c0 0x7f2550069d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2560009af0 tx=0x7f2560008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.529+0000 7f256d7fa640 1 -- 172.21.15.76:0/3626255064 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2558016150 con 0x7f2570145960 2024-09-19T04:19:20.742 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.741+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f25700d1120 con 0x7f2570145960 2024-09-19T04:19:20.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.741+0000 7f256d7fa640 1 -- 172.21.15.76:0/3626255064 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2008 (secure 0 0 0) 0x7f25580aae20 con 0x7f2570145960 2024-09-19T04:19:20.743 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:20.743 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":379,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":695469,"bytes_used":261898240,"bytes_avail":772832215040,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7124,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-09-19T04:19:19.851943+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:20.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f25500678c0 msgr2=0x7f2550069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f25500678c0 0x7f2550069d80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2560009af0 tx=0x7f2560008040 comp rx=0 tx=0).stop 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2570145960 msgr2=0x7f257015ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2570145960 0x7f257015ee20 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f25580ac040 tx=0x7f255801ad10 comp rx=0 tx=0).stop 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 shutdown_connections 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f25500678c0 0x7f2550069d80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25701609f0 0x7f25700c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2570145960 0x7f257015ee20 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 --2- 172.21.15.76:0/3626255064 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f25700cd8e0 0x7f2570143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 >> 172.21.15.76:0/3626255064 conn(0x7f257009aa00 msgr2=0x7f257009b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 shutdown_connections 2024-09-19T04:19:20.747 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.745+0000 7f257667d640 1 -- 172.21.15.76:0/3626255064 wait complete. 2024-09-19T04:19:20.761 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:19:20.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:20.762 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub resume 2024-09-19T04:19:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.881+0000 7fe8dbaae640 1 Processor -- start 2024-09-19T04:19:20.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.881+0000 7fe8dbaae640 1 -- start start 2024-09-19T04:19:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8dbaae640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe8d415c2a0 0x7fe8d40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8dbaae640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d40af2a0 0x7fe8d406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8dbaae640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8d40b3290 con 0x7fe8d415bec0 2024-09-19T04:19:20.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8dbaae640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe8d40b3570 con 0x7fe8d415c2a0 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8dbaae640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d40b3850 con 0x7fe8d40af2a0 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d984a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe8d415bec0 0x7fe8d40ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45674/0 (socket says 172.21.15.76:45674) 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d984a640 1 -- 172.21.15.76:0/1440971506 learned_addr learned my addr 172.21.15.76:0/1440971506 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d9049640 1 --2- 172.21.15.76:0/1440971506 >> v2:172.21.15.76:3300/0 conn(0x7fe8d415c2a0 0x7fe8d40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8da04b640 1 --2- 172.21.15.76:0/1440971506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d40af2a0 0x7fe8d406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d8848640 1 -- 172.21.15.76:0/1440971506 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1638464728 0 0) 0x7fe8d40b3290 con 0x7fe8d415bec0 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d8848640 1 -- 172.21.15.76:0/1440971506 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe8c0003530 con 0x7fe8d415bec0 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d9049640 1 -- 172.21.15.76:0/1440971506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d40af2a0 msgr2=0x7fe8d406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d9049640 1 --2- 172.21.15.76:0/1440971506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d40af2a0 0x7fe8d406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d9049640 1 -- 172.21.15.76:0/1440971506 >> v1:172.21.15.76:6789/0 conn(0x7fe8d415bec0 legacy=0x7fe8d40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.885+0000 7fe8d9049640 1 -- 172.21.15.76:0/1440971506 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d40b3b30 con 0x7fe8d415c2a0 2024-09-19T04:19:20.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8d9049640 1 --2- 172.21.15.76:0/1440971506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d415c2a0 0x7fe8d40aed60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40097c0 tx=0x7fe8c401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30df314c56f7c76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8d8848640 1 -- 172.21.15.76:0/1440971506 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40a7070 con 0x7fe8d415c2a0 2024-09-19T04:19:20.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8d8848640 1 -- 172.21.15.76:0/1440971506 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8c40b0070 con 0x7fe8d415c2a0 2024-09-19T04:19:20.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8d8848640 1 -- 172.21.15.76:0/1440971506 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40ab870 con 0x7fe8d415c2a0 2024-09-19T04:19:20.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- 172.21.15.76:0/1440971506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d415c2a0 msgr2=0x7fe8d40aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/1440971506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d415c2a0 0x7fe8d40aed60 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40097c0 tx=0x7fe8c401c470 comp rx=0 tx=0).stop 2024-09-19T04:19:20.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- 172.21.15.76:0/1440971506 shutdown_connections 2024-09-19T04:19:20.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/1440971506 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d40af2a0 0x7fe8d406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/1440971506 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d415c2a0 0x7fe8d40aed60 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- 172.21.15.76:0/1440971506 >> 172.21.15.76:0/1440971506 conn(0x7fe8d40b1f90 msgr2=0x7fe8d40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- 172.21.15.76:0/1440971506 shutdown_connections 2024-09-19T04:19:20.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- 172.21.15.76:0/1440971506 wait complete. 2024-09-19T04:19:20.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 Processor -- start 2024-09-19T04:19:20.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- start start 2024-09-19T04:19:20.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d40af2a0 0x7fe8d415dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d415bec0 0x7fe8d415c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8d415c2a0 0x7fe8d409a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d406ed50 con 0x7fe8d40af2a0 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d40af6b0 con 0x7fe8d415c2a0 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8dbaae640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d406eed0 con 0x7fe8d415bec0 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8d984a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d40af2a0 0x7fe8d415dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8d984a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d40af2a0 0x7fe8d415dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44352/0 (socket says 172.21.15.76:44352) 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.889+0000 7fe8d984a640 1 -- 172.21.15.76:0/568104686 learned_addr learned my addr 172.21.15.76:0/568104686 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d984a640 1 -- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d415bec0 msgr2=0x7fe8d415c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d9049640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d415bec0 0x7fe8d415c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8da04b640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8d415c2a0 0x7fe8d409a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d984a640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d415bec0 0x7fe8d415c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d984a640 1 -- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8d415c2a0 msgr2=0x7fe8d409a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d984a640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8d415c2a0 0x7fe8d409a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d984a640 1 -- 172.21.15.76:0/568104686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d415ce40 con 0x7fe8d40af2a0 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8da04b640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8d415c2a0 0x7fe8d409a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d984a640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d40af2a0 0x7fe8d415dee0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c480 tx=0x7fe8d000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8ca7fc640 1 -- 172.21.15.76:0/568104686 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0017070 con 0x7fe8d40af2a0 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8ca7fc640 1 -- 172.21.15.76:0/568104686 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0005cf0 con 0x7fe8d40af2a0 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d415d120 con 0x7fe8d40af2a0 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8ca7fc640 1 -- 172.21.15.76:0/568104686 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0007290 con 0x7fe8d40af2a0 2024-09-19T04:19:20.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d415d6e0 con 0x7fe8d40af2a0 2024-09-19T04:19:20.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8ca7fc640 1 -- 172.21.15.76:0/568104686 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe8d001a070 con 0x7fe8d40af2a0 2024-09-19T04:19:20.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8ca7fc640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8b80678c0 0x7fe8b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8ca7fc640 1 -- 172.21.15.76:0/568104686 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fe8d0085250 con 0x7fe8d40af2a0 2024-09-19T04:19:20.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d9049640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8b80678c0 0x7fe8b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8d9049640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8b80678c0 0x7fe8b8069d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40af000 tx=0x7fe8c40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8deb6a000 1 Processor -- start 2024-09-19T04:19:20.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.893+0000 7fe8deb6a000 1 -- start start 2024-09-19T04:19:20.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a43150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a1a43690 0x55b0a19f6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a19f6db0 0x55b0a19f7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b0a18e0960 con 0x55b0a1a42d70 2024-09-19T04:19:20.898 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b0a1824900 con 0x55b0a1a43690 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b0a1847d40 con 0x55b0a19f6db0 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a43150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a43150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44360/0 (socket says 172.21.15.76:44360) 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 -- 172.21.15.76:0/2317177264 learned_addr learned my addr 172.21.15.76:0/2317177264 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a1a43690 0x55b0a19f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8da04b640 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a19f6db0 0x55b0a19f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 -- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a19f6db0 msgr2=0x55b0a19f7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a19f6db0 0x55b0a19f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 -- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a1a43690 msgr2=0x55b0a19f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a1a43690 0x55b0a19f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 -- 172.21.15.76:0/2317177264 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a17ee580 con 0x55b0a1a42d70 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a1a43690 0x55b0a19f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:20.899 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a43150 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0007fd0 tx=0x7fe8d0055660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e520d2ca08d3ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8b77fe640 1 -- 172.21.15.76:0/2317177264 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0053820 con 0x55b0a1a42d70 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8b77fe640 1 -- 172.21.15.76:0/2317177264 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0051b90 con 0x55b0a1a42d70 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8b77fe640 1 -- 172.21.15.76:0/2317177264 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d005deb0 con 0x55b0a1a42d70 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2317177264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 msgr2=0x55b0a1a43150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a43150 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0007fd0 tx=0x7fe8d0055660 comp rx=0 tx=0).stop 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2317177264 shutdown_connections 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a19f6db0 0x55b0a19f7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a1a43690 0x55b0a19f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2317177264 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a43150 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2317177264 >> 172.21.15.76:0/2317177264 conn(0x55b0a1a24040 msgr2=0x55b0a1a26430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2317177264 shutdown_connections 2024-09-19T04:19:20.900 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2317177264 wait complete. 2024-09-19T04:19:20.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- start start 2024-09-19T04:19:20.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 Processor -- start 2024-09-19T04:19:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a49ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a4a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1a4a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b0a1843d00 con 0x55b0a1a42d70 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55b0a1802aa0 con 0x55b0a19f6db0 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b0a17e1340 con 0x55b0a1a43690 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8deb6a000 10 client.0 _ll_register_callbacks cb 0x55b0a19e4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a4a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a4a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44366/0 (socket says 172.21.15.76:44366) 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 -- 172.21.15.76:0/3418090348 learned_addr learned my addr 172.21.15.76:0/3418090348 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d984a640 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a49ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8da04b640 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1a4a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 -- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 msgr2=0x55b0a1a4a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1a4a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 -- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 msgr2=0x55b0a1a49ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a49ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 -- 172.21.15.76:0/3418090348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a17eadc0 con 0x55b0a1a42d70 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8d9049640 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a4a220 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe8c401cd10 tx=0x7fe8c401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c03d4635eca701a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.897+0000 7fe8b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/3418090348 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40cd060 con 0x55b0a1a42d70 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/3418090348 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fe8c40c0b60 con 0x55b0a1a42d70 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8d0051b90 con 0x55b0a1a42d70 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/3418090348 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fe8c40c0d60 con 0x55b0a1a42d70 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 10 client.7821 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55b0a17e77f0 con 0x55b0a1a42d70 2024-09-19T04:19:20.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/3418090348 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d0051b90 con 0x55b0a1a42d70 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/3418090348 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7fe8c40c7030 con 0x55b0a1a42d70 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 10 client.7821 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 10 client.7821 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 20 client.7821 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1a520b0 0x55b0a1a544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 4 client.7821 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0a17e1e10 con 0x55b0a1a520b0 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8d984a640 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1a520b0 0x55b0a1a544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8d984a640 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1a520b0 0x55b0a1a544a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b09b6192921e3b server_cookie=23927be7bdbfef34 in_seq=0 out_seq=0 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 10 client.7821 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/3418090348 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0a17e1e10 con 0x55b0a1a520b0 2024-09-19T04:19:20.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 10 client.7821 handle_command_reply: tid=0 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 client.7821 shutdown 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 msgr2=0x55b0a1a4a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a4a220 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fe8c401cd10 tx=0x7fe8c401acb0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 shutdown_connections 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1a520b0 0x55b0a1a544a0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1a4a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1a4a220 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8b4ff9640 0 client.7821 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/3418090348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a49ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 >> 172.21.15.76:0/3418090348 conn(0x55b0a1a24040 msgr2=0x55b0a19f5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 shutdown_connections 2024-09-19T04:19:20.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 1 -- 172.21.15.76:0/3418090348 wait complete. 2024-09-19T04:19:20.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.901+0000 7fe8deb6a000 20 client.7821 trim_cache size 0 max 16384 2024-09-19T04:19:20.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.917+0000 7fe8deb6a000 1 Processor -- start 2024-09-19T04:19:20.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.917+0000 7fe8deb6a000 1 -- start start 2024-09-19T04:19:20.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a19fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a42d70 0x55b0a19ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a43690 0x55b0a1a4cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4203da0 con 0x55b0a1a43690 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d4202f40 con 0x55b0a19f6db0 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe8d4203670 con 0x55b0a1a42d70 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a43690 0x55b0a1a4cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a43690 0x55b0a1a4cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44382/0 (socket says 172.21.15.76:44382) 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 -- 172.21.15.76:0/2391864588 learned_addr learned my addr 172.21.15.76:0/2391864588 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 -- 172.21.15.76:0/2391864588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a42d70 msgr2=0x55b0a19ff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 --2- 172.21.15.76:0/2391864588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a42d70 0x55b0a19ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 -- 172.21.15.76:0/2391864588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 msgr2=0x55b0a19fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 --2- 172.21.15.76:0/2391864588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a19fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 -- 172.21.15.76:0/2391864588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a1805470 con 0x55b0a1a43690 2024-09-19T04:19:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 --2- 172.21.15.76:0/2391864588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a43690 0x55b0a1a4cd00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc007460 tx=0x7fe8cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc17396f811e1afa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8b6ffd640 1 -- 172.21.15.76:0/2391864588 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc019070 con 0x55b0a1a43690 2024-09-19T04:19:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8b6ffd640 1 -- 172.21.15.76:0/2391864588 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc015820 con 0x55b0a1a43690 2024-09-19T04:19:20.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8b6ffd640 1 -- 172.21.15.76:0/2391864588 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc015c10 con 0x55b0a1a43690 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2391864588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a43690 msgr2=0x55b0a1a4cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2391864588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a43690 0x55b0a1a4cd00 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc007460 tx=0x7fe8cc009750 comp rx=0 tx=0).stop 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2391864588 shutdown_connections 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2391864588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a43690 0x55b0a1a4cd00 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2391864588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a42d70 0x55b0a19ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2391864588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a19fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2391864588 >> 172.21.15.76:0/2391864588 conn(0x55b0a1a24040 msgr2=0x55b0a1a49aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2391864588 shutdown_connections 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2391864588 wait complete. 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- start start 2024-09-19T04:19:20.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 Processor -- start 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a4fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1ab6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1ab6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55b0a1a023f0 con 0x55b0a1a42d70 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d409c930 con 0x55b0a19f6db0 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55b0a17e1480 con 0x55b0a1a43690 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 10 client.0 _ll_register_callbacks cb 0x7fe8c0003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1ab6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1ab6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44390/0 (socket says 172.21.15.76:44390) 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 -- 172.21.15.76:0/2853040643 learned_addr learned my addr 172.21.15.76:0/2853040643 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d984a640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a4fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8da04b640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1ab6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 -- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 msgr2=0x55b0a1ab6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1ab6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 -- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 msgr2=0x55b0a1a4fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a4fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 -- 172.21.15.76:0/2853040643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a17db640 con 0x55b0a1a42d70 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d984a640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a4fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8d9049640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1ab6810 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40a4ff0 tx=0x7fe8c4019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53236182a3215319 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/2853040643 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c40dc060 con 0x55b0a1a42d70 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/2853040643 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fe8c40bec70 con 0x55b0a1a42d70 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8cc015820 con 0x55b0a1a42d70 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/2853040643 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fe8c40e2050 con 0x55b0a1a42d70 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 10 client.7827 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:20.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.921+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x55b0a17df3b0 con 0x55b0a1a42d70 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/2853040643 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8cc015820 con 0x55b0a1a42d70 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/2853040643 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7fe8c40e5020 con 0x55b0a1a42d70 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 10 client.7827 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 10 client.7827 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 20 client.7827 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1ab7290 0x55b0a1ab3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 4 client.7827 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x55b0a1855470 con 0x55b0a1ab7290 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8d984a640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1ab7290 0x55b0a1ab3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8b4ff9640 10 client.7827 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8d984a640 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1ab7290 0x55b0a1ab3190 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d48bb3686d33c037 server_cookie=707fa675d7163bc0 in_seq=0 out_seq=0 2024-09-19T04:19:20.930 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8b4ff9640 1 -- 172.21.15.76:0/2853040643 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b0a1855470 con 0x55b0a1ab7290 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8b4ff9640 10 client.7827 handle_command_reply: tid=0 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 client.7827 shutdown 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 msgr2=0x55b0a1ab6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1ab6810 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40a4ff0 tx=0x7fe8c4019210 comp rx=0 tx=0).stop 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 shutdown_connections 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x55b0a1ab7290 0x55b0a1ab3190 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55b0a1a43690 0x55b0a1ab6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55b0a1a42d70 0x55b0a1ab6810 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8b4ff9640 0 client.7827 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 --2- 172.21.15.76:0/2853040643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b0a19f6db0 0x55b0a1a4fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 >> 172.21.15.76:0/2853040643 conn(0x55b0a1a24040 msgr2=0x55b0a1a4a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 shutdown_connections 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 1 -- 172.21.15.76:0/2853040643 wait complete. 2024-09-19T04:19:20.931 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8deb6a000 20 client.7827 trim_cache size 0 max 16384 2024-09-19T04:19:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:20.932 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:19:20.932 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8b80678c0 msgr2=0x7fe8b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8b80678c0 0x7fe8b8069d80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40af000 tx=0x7fe8c40a7000 comp rx=0 tx=0).stop 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d40af2a0 msgr2=0x7fe8d415dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d40af2a0 0x7fe8d415dee0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000c480 tx=0x7fe8d000c940 comp rx=0 tx=0).stop 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 shutdown_connections 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8b80678c0 0x7fe8b8069d80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8d415c2a0 0x7fe8d409a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.933 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8d415bec0 0x7fe8d415c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 --2- 172.21.15.76:0/568104686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8d40af2a0 0x7fe8d415dee0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 >> 172.21.15.76:0/568104686 conn(0x7fe8d40b1f90 msgr2=0x7fe8d40adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 shutdown_connections 2024-09-19T04:19:20.934 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:20.925+0000 7fe8dbaae640 1 -- 172.21.15.76:0/568104686 wait complete. 2024-09-19T04:19:20.941 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:19:20.941 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:1 scrub status 2024-09-19T04:19:21.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.053+0000 7ff530c49640 1 Processor -- start 2024-09-19T04:19:21.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.053+0000 7ff530c49640 1 -- start start 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff530c49640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff52c161a60 0x7ff52c0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff530c49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff530c49640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff52c0cf450 0x7ff52c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff530c49640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0d32b0 con 0x7ff52c0cf450 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff530c49640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff52c0d0300 con 0x7ff52c161a60 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff530c49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff52c0b0f60 con 0x7ff52c0d2e90 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff52c0cf450 0x7ff52c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff52c0cf450 0x7ff52c0d0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44398/0 (socket says 172.21.15.76:44398) 2024-09-19T04:19:21.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 -- 172.21.15.76:0/1967681152 learned_addr learned my addr 172.21.15.76:0/1967681152 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 -- 172.21.15.76:0/1967681152 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 msgr2=0x7ff52c0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/1967681152 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52a59c640 1 --2- 172.21.15.76:0/1967681152 >> v2:172.21.15.116:3300/0 conn(0x7ff52c161a60 0x7ff52c0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 -- 172.21.15.76:0/1967681152 >> v2:172.21.15.116:3300/0 conn(0x7ff52c161a60 msgr2=0x7ff52c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/1967681152 >> v2:172.21.15.116:3300/0 conn(0x7ff52c161a60 0x7ff52c0d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 -- 172.21.15.76:0/1967681152 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c0cfd40 con 0x7ff52c0cf450 2024-09-19T04:19:21.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/1967681152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0d0a10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff52c08e010 tx=0x7ff52401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa38911226b599a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff529d9b640 1 -- 172.21.15.76:0/1967681152 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240a57b0 con 0x7ff52c0cf450 2024-09-19T04:19:21.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff529d9b640 1 -- 172.21.15.76:0/1967681152 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5240a5950 con 0x7ff52c0cf450 2024-09-19T04:19:21.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.057+0000 7ff529d9b640 1 -- 172.21.15.76:0/1967681152 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240a5d20 con 0x7ff52c0cf450 2024-09-19T04:19:21.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- 172.21.15.76:0/1967681152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 msgr2=0x7ff52c0d0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 --2- 172.21.15.76:0/1967681152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0d0a10 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7ff52c08e010 tx=0x7ff52401c930 comp rx=0 tx=0).stop 2024-09-19T04:19:21.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- 172.21.15.76:0/1967681152 shutdown_connections 2024-09-19T04:19:21.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 --2- 172.21.15.76:0/1967681152 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0d0a10 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 --2- 172.21.15.76:0/1967681152 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 --2- 172.21.15.76:0/1967681152 >> v2:172.21.15.116:3300/0 conn(0x7ff52c161a60 0x7ff52c0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- 172.21.15.76:0/1967681152 >> 172.21.15.76:0/1967681152 conn(0x7ff52c08bf90 msgr2=0x7ff52c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- 172.21.15.76:0/1967681152 shutdown_connections 2024-09-19T04:19:21.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- 172.21.15.76:0/1967681152 wait complete. 2024-09-19T04:19:21.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 Processor -- start 2024-09-19T04:19:21.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- start start 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff52c161a60 0x7ff52c158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0cfa60 con 0x7ff52c0cf450 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c0cf8e0 con 0x7ff52c161a60 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff52c06d6b0 con 0x7ff52c0d2e90 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0cab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44402/0 (socket says 172.21.15.76:44402) 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 -- 172.21.15.76:0/2840784809 learned_addr learned my addr 172.21.15.76:0/2840784809 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff523fff640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 -- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 msgr2=0x7ff52c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 -- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff52c161a60 msgr2=0x7ff52c158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff52c161a60 0x7ff52c158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff52c161a60 0x7ff52c158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 -- 172.21.15.76:0/2840784809 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff52c1576f0 con 0x7ff52c0cf450 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff523fff640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff52a59c640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0cab80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c7f0 tx=0x7ff51c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff522ffd640 1 -- 172.21.15.76:0/2840784809 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c015070 con 0x7ff52c0cf450 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff522ffd640 1 -- 172.21.15.76:0/2840784809 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff51c007820 con 0x7ff52c0cf450 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff522ffd640 1 -- 172.21.15.76:0/2840784809 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c007c10 con 0x7ff52c0cf450 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff52c1579d0 con 0x7ff52c0cf450 2024-09-19T04:19:21.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.061+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff52c157f90 con 0x7ff52c0cf450 2024-09-19T04:19:21.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff522ffd640 1 -- 172.21.15.76:0/2840784809 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff51c018070 con 0x7ff52c0cf450 2024-09-19T04:19:21.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff522ffd640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff50c0678c0 0x7ff50c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff523fff640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff50c0678c0 0x7ff50c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff522ffd640 1 -- 172.21.15.76:0/2840784809 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff51c085fc0 con 0x7ff52c0cf450 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff523fff640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff50c0678c0 0x7ff50c069d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff52c1599a0 tx=0x7ff514008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 Processor -- start 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 -- start start 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446dd70 0x5560c446e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446e690 0x5560c4421870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c4421db0 0x5560c4422190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5560c430b960 con 0x5560c4421db0 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5560c424f900 con 0x5560c446dd70 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5560c4272d40 con 0x5560c446e690 2024-09-19T04:19:21.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c4421db0 0x5560c4422190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c4421db0 0x5560c4422190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44414/0 (socket says 172.21.15.76:44414) 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 -- 172.21.15.76:0/2598017995 learned_addr learned my addr 172.21.15.76:0/2598017995 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff523fff640 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446e690 0x5560c4421870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52a59c640 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446dd70 0x5560c446e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 -- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446e690 msgr2=0x5560c4421870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446e690 0x5560c4421870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 -- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446dd70 msgr2=0x5560c446e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446dd70 0x5560c446e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 -- 172.21.15.76:0/2598017995 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c4219580 con 0x5560c4421db0 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52a59c640 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446dd70 0x5560c446e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c4421db0 0x5560c4422190 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff52401afe0 tx=0x7ff524004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=983908132fa0b7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff50bfff640 1 -- 172.21.15.76:0/2598017995 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240be070 con 0x5560c4421db0 2024-09-19T04:19:21.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff50bfff640 1 -- 172.21.15.76:0/2598017995 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5240a57b0 con 0x5560c4421db0 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.065+0000 7ff50bfff640 1 -- 172.21.15.76:0/2598017995 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240a5b60 con 0x5560c4421db0 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/2598017995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c4421db0 msgr2=0x5560c4422190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c4421db0 0x5560c4422190 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff52401afe0 tx=0x7ff524004ac0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/2598017995 shutdown_connections 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c4421db0 0x5560c4422190 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446e690 0x5560c4421870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- 172.21.15.76:0/2598017995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446dd70 0x5560c446e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/2598017995 >> 172.21.15.76:0/2598017995 conn(0x5560c444f040 msgr2=0x5560c4451430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/2598017995 shutdown_connections 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/2598017995 wait complete. 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- start start 2024-09-19T04:19:21.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 Processor -- start 2024-09-19T04:19:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 0x5560c4474ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c4475220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4475760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5560c426ed00 con 0x5560c446e690 2024-09-19T04:19:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5560c422daa0 con 0x5560c4421db0 2024-09-19T04:19:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5560c420c340 con 0x5560c446dd70 2024-09-19T04:19:21.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 10 client.0 _ll_register_callbacks cb 0x5560c440f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4475760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4475760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44422/0 (socket says 172.21.15.76:44422) 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 -- 172.21.15.76:0/4061389737 learned_addr learned my addr 172.21.15.76:0/4061389737 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff523fff640 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c4475220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 -- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 msgr2=0x5560c4475220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c4475220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 -- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 msgr2=0x5560c4474ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 0x5560c4474ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 -- 172.21.15.76:0/4061389737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c4215dc0 con 0x5560c446e690 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff523fff640 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c4475220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4475760 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff5240b9040 tx=0x7ff52401b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13508897cb2d9967 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff5097fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff5097fa640 1 -- 172.21.15.76:0/4061389737 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240c6020 con 0x5560c446e690 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff5097fa640 1 -- 172.21.15.76:0/4061389737 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff5240c3ce0 con 0x5560c446e690 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5240a57b0 con 0x5560c446e690 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff5097fa640 1 -- 172.21.15.76:0/4061389737 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff5240be040 con 0x5560c446e690 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 10 client.7839 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x5560c42127f0 con 0x5560c446e690 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff5097fa640 1 -- 172.21.15.76:0/4061389737 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5240a57b0 con 0x5560c446e690 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff5097fa640 1 -- 172.21.15.76:0/4061389737 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7ff5240c1070 con 0x5560c446e690 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 10 client.7839 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 10 client.7839 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 20 client.7839 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:21.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447d0b0 0x5560c447f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 4 client.7839 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560c420ce10 con 0x5560c447d0b0 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52a59c640 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447d0b0 0x5560c447f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff5097fa640 10 client.7839 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.069+0000 7ff52a59c640 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447d0b0 0x5560c447f4a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d272ffbfae205436 server_cookie=e22b64e4952fd1ae in_seq=0 out_seq=0 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff5097fa640 1 -- 172.21.15.76:0/4061389737 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5560c420ce10 con 0x5560c447d0b0 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff5097fa640 10 client.7839 handle_command_reply: tid=0 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 client.7839 shutdown 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 msgr2=0x5560c4475760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4475760 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff5240b9040 tx=0x7ff52401b170 comp rx=0 tx=0).stop 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 shutdown_connections 2024-09-19T04:19:21.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447d0b0 0x5560c447f4a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4475760 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c4475220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff5097fa640 0 client.7839 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 --2- 172.21.15.76:0/4061389737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 0x5560c4474ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 >> 172.21.15.76:0/4061389737 conn(0x5560c444f040 msgr2=0x5560c4420f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 shutdown_connections 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 1 -- 172.21.15.76:0/4061389737 wait complete. 2024-09-19T04:19:21.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.073+0000 7ff533d05000 20 client.7839 trim_cache size 0 max 16384 2024-09-19T04:19:21.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 Processor -- start 2024-09-19T04:19:21.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- start start 2024-09-19T04:19:21.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c4421db0 0x5560c4429bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446dd70 0x5560c442a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446e690 0x5560c447ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c2042d0 con 0x5560c446dd70 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff52c203470 con 0x5560c446e690 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff52c203ba0 con 0x5560c4421db0 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446dd70 0x5560c442a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446dd70 0x5560c442a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44426/0 (socket says 172.21.15.76:44426) 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 -- 172.21.15.76:0/1542370850 learned_addr learned my addr 172.21.15.76:0/1542370850 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 -- 172.21.15.76:0/1542370850 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c4421db0 msgr2=0x5560c4429bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 --2- 172.21.15.76:0/1542370850 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c4421db0 0x5560c4429bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 -- 172.21.15.76:0/1542370850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446e690 msgr2=0x5560c447ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 --2- 172.21.15.76:0/1542370850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446e690 0x5560c447ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 -- 172.21.15.76:0/1542370850 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560c4230470 con 0x5560c446dd70 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff523fff640 1 --2- 172.21.15.76:0/1542370850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446dd70 0x5560c442a130 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff51400cac0 tx=0x7ff514012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c620283539e21a09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff50b7fe640 1 -- 172.21.15.76:0/1542370850 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51401c040 con 0x5560c446dd70 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff50b7fe640 1 -- 172.21.15.76:0/1542370850 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff514025070 con 0x5560c446dd70 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff50b7fe640 1 -- 172.21.15.76:0/1542370850 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff514013d40 con 0x5560c446dd70 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- 172.21.15.76:0/1542370850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446dd70 msgr2=0x5560c442a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 --2- 172.21.15.76:0/1542370850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446dd70 0x5560c442a130 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff51400cac0 tx=0x7ff514012000 comp rx=0 tx=0).stop 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- 172.21.15.76:0/1542370850 shutdown_connections 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 --2- 172.21.15.76:0/1542370850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c446e690 0x5560c447ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 --2- 172.21.15.76:0/1542370850 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446dd70 0x5560c442a130 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 --2- 172.21.15.76:0/1542370850 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c4421db0 0x5560c4429bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- 172.21.15.76:0/1542370850 >> 172.21.15.76:0/1542370850 conn(0x5560c444f040 msgr2=0x5560c4474aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- 172.21.15.76:0/1542370850 shutdown_connections 2024-09-19T04:19:21.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.089+0000 7ff533d05000 1 -- 172.21.15.76:0/1542370850 wait complete. 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- start start 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 Processor -- start 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 0x5560c44e12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c44e17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4479e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5560c42313f0 con 0x5560c446e690 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5560c442d3f0 con 0x5560c4421db0 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5560c432f0c0 con 0x5560c446dd70 2024-09-19T04:19:21.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 10 client.0 _ll_register_callbacks cb 0x7ff5240bc070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4479e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4479e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44438/0 (socket says 172.21.15.76:44438) 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 -- 172.21.15.76:0/3322729779 learned_addr learned my addr 172.21.15.76:0/3322729779 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52a59c640 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 0x5560c44e12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff523fff640 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c44e17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 -- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 msgr2=0x5560c44e17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c44e17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 -- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 msgr2=0x5560c44e12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 0x5560c44e12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 -- 172.21.15.76:0/3322729779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560c4206640 con 0x5560c446e690 2024-09-19T04:19:21.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52ad9d640 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4479e50 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff52401cea0 tx=0x7ff5240196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510a3acca90cfe56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 1 -- 172.21.15.76:0/3322729779 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5240d3030 con 0x5560c446e690 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 1 -- 172.21.15.76:0/3322729779 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff524009210 con 0x5560c446e690 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff514025070 con 0x5560c446e690 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 1 -- 172.21.15.76:0/3322729779 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff5240dc050 con 0x5560c446e690 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 10 client.7845 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x5560c420a3b0 con 0x5560c446e690 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 1 -- 172.21.15.76:0/3322729779 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff514025070 con 0x5560c446e690 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 1 -- 172.21.15.76:0/3322729779 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7ff5240df020 con 0x5560c446e690 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 10 client.7845 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:21.100 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 10 client.7845 resolve_mds: resolved 6:1 to role '6:1' aka daemon mds.c 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 20 client.7845 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447a390 0x5560c447a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 4 client.7845 mds_command: new command op to 7305 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 --> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5560c4280470 con 0x5560c447a390 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52a59c640 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447a390 0x5560c447a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 10 client.7845 ms_handle_connect on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff52a59c640 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447a390 0x5560c447a7a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8bce8ff99d64a592 server_cookie=6cc0d97b9f0ac7ff in_seq=0 out_seq=0 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 1 -- 172.21.15.76:0/3322729779 <== mds.1 v2:172.21.15.76:6834/987394316 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5560c4280470 con 0x5560c447a390 2024-09-19T04:19:21.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 10 client.7845 handle_command_reply: tid=0 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 client.7845 shutdown 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 msgr2=0x5560c4479e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4479e50 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7ff52401cea0 tx=0x7ff5240196d0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 shutdown_connections 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:6834/987394316,v1:172.21.15.76:6835/987394316] conn(0x5560c447a390 0x5560c447a7a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5560c446e690 0x5560c4479e50 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5560c446dd70 0x5560c44e17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 --2- 172.21.15.76:0/3322729779 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5560c4421db0 0x5560c44e12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 >> 172.21.15.76:0/3322729779 conn(0x5560c444f040 msgr2=0x5560c446d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff5097fa640 0 client.7845 ms_handle_reset on v2:172.21.15.76:6834/987394316 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 shutdown_connections 2024-09-19T04:19:21.102 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 1 -- 172.21.15.76:0/3322729779 wait complete. 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff533d05000 20 client.7845 trim_cache size 0 max 16384 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: "8362a942-ff57-460f-be25-9fdf5af5b05a": { 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "8362a942-ff57-460f-be25-9fdf5af5b05a", 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:21.103 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:21.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff50c0678c0 msgr2=0x7ff50c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff50c0678c0 0x7ff50c069d80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff52c1599a0 tx=0x7ff514008040 comp rx=0 tx=0).stop 2024-09-19T04:19:21.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 msgr2=0x7ff52c0cab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0cab80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c7f0 tx=0x7ff51c00ccb0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 shutdown_connections 2024-09-19T04:19:21.104 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff50c0678c0 0x7ff50c069d80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff52c161a60 0x7ff52c158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff52c0d2e90 0x7ff52c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 --2- 172.21.15.76:0/2840784809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff52c0cf450 0x7ff52c0cab80 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 >> 172.21.15.76:0/2840784809 conn(0x7ff52c08bf90 msgr2=0x7ff52c0b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 shutdown_connections 2024-09-19T04:19:21.105 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.093+0000 7ff530c49640 1 -- 172.21.15.76:0/2840784809 wait complete. 2024-09-19T04:19:21.110 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:19:21.110 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:19:21.111 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:2 scrub status 2024-09-19T04:19:21.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1d2499640 1 Processor -- start 2024-09-19T04:19:21.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1d2499640 1 -- start start 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1d2499640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff1cc1630c0 0x7ff1cc06f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1d2499640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc163bb0 con 0x7ff1cc06df70 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1d2499640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1cc06e470 con 0x7ff1cc0b3290 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1d2499640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff1cc06e750 con 0x7ff1cc1630c0 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1cb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff1cc06df70 0x7ff1cc1634a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45688/0 (socket says 172.21.15.76:45688) 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/4038750166 learned_addr learned my addr 172.21.15.76:0/4038750166 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1cbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff1cc1630c0 0x7ff1cc06f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1caffd640 1 -- 172.21.15.76:0/4038750166 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 604068212 0 0) 0x7ff1cc163bb0 con 0x7ff1cc06df70 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1caffd640 1 -- 172.21.15.76:0/4038750166 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1b0003530 con 0x7ff1cc06df70 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1cbfff640 1 -- 172.21.15.76:0/4038750166 >> v1:172.21.15.116:6789/0 conn(0x7ff1cc0b3290 legacy=0x7ff1cc0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1cbfff640 1 -- 172.21.15.76:0/4038750166 >> v1:172.21.15.76:6789/0 conn(0x7ff1cc06df70 legacy=0x7ff1cc1634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.224 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.221+0000 7ff1cbfff640 1 -- 172.21.15.76:0/4038750166 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc06ea30 con 0x7ff1cc1630c0 2024-09-19T04:19:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/4038750166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc1630c0 0x7ff1cc06f510 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0097c0 tx=0x7ff1bc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32816a8352eacb9e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1caffd640 1 -- 172.21.15.76:0/4038750166 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0b0070 con 0x7ff1cc1630c0 2024-09-19T04:19:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1caffd640 1 -- 172.21.15.76:0/4038750166 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc004230 con 0x7ff1cc1630c0 2024-09-19T04:19:21.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1caffd640 1 -- 172.21.15.76:0/4038750166 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc003620 con 0x7ff1cc1630c0 2024-09-19T04:19:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 -- 172.21.15.76:0/4038750166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc1630c0 msgr2=0x7ff1cc06f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 --2- 172.21.15.76:0/4038750166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc1630c0 0x7ff1cc06f510 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0097c0 tx=0x7ff1bc01c460 comp rx=0 tx=0).stop 2024-09-19T04:19:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 -- 172.21.15.76:0/4038750166 shutdown_connections 2024-09-19T04:19:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 --2- 172.21.15.76:0/4038750166 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc1630c0 0x7ff1cc06f510 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 -- 172.21.15.76:0/4038750166 >> 172.21.15.76:0/4038750166 conn(0x7ff1cc0b1f90 msgr2=0x7ff1cc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 -- 172.21.15.76:0/4038750166 shutdown_connections 2024-09-19T04:19:21.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 -- 172.21.15.76:0/4038750166 wait complete. 2024-09-19T04:19:21.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 Processor -- start 2024-09-19T04:19:21.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 -- start start 2024-09-19T04:19:21.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.225+0000 7ff1d2499640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc06df70 0x7ff1cc09a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d2499640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1cc0b3290 0x7ff1cc156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d2499640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1cc1630c0 0x7ff1cc0cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d2499640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc06fbd0 con 0x7ff1cc06df70 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d2499640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc06fa50 con 0x7ff1cc1630c0 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d2499640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc170e10 con 0x7ff1cc0b3290 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc06df70 0x7ff1cc09a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc06df70 0x7ff1cc09a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44456/0 (socket says 172.21.15.76:44456) 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 -- 172.21.15.76:0/1469445594 learned_addr learned my addr 172.21.15.76:0/1469445594 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1cc0b3290 0x7ff1cc156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1cc1630c0 0x7ff1cc0cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 -- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1cc0b3290 msgr2=0x7ff1cc156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1cc0b3290 0x7ff1cc156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 -- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1cc1630c0 msgr2=0x7ff1cc0cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1cc1630c0 0x7ff1cc0cdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 -- 172.21.15.76:0/1469445594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1cc09d0e0 con 0x7ff1cc06df70 2024-09-19T04:19:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc06df70 0x7ff1cc09a380 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc019070 tx=0x7ff1bc019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1469445594 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0b0040 con 0x7ff1cc06df70 2024-09-19T04:19:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1469445594 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1bc0bc070 con 0x7ff1cc06df70 2024-09-19T04:19:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1cc09b3e0 con 0x7ff1cc06df70 2024-09-19T04:19:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1469445594 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc003e10 con 0x7ff1cc06df70 2024-09-19T04:19:21.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1cc1593a0 con 0x7ff1cc06df70 2024-09-19T04:19:21.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1469445594 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff1bc0b3070 con 0x7ff1cc06df70 2024-09-19T04:19:21.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1c8ff9640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1c8ff9640 1 -- 172.21.15.76:0/1469445594 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff1bc11fc80 con 0x7ff1cc06df70 2024-09-19T04:19:21.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0cfd80 tx=0x7ff1b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d5555000 1 Processor -- start 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.229+0000 7ff1d5555000 1 -- start start 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 0x562c71152150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71152690 0x562c71105870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71106190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c70fef960 con 0x562c71105db0 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562c70f33900 con 0x562c71151d70 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562c70f56d40 con 0x562c71152690 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71106190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44464/0 (socket says 172.21.15.76:44464) 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 -- 172.21.15.76:0/2774656326 learned_addr learned my addr 172.21.15.76:0/2774656326 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71152690 0x562c71105870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 -- 172.21.15.76:0/2774656326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71152690 msgr2=0x562c71105870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71152690 0x562c71105870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 -- 172.21.15.76:0/2774656326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 msgr2=0x562c71152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 0x562c71152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 -- 172.21.15.76:0/2774656326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c70efd580 con 0x562c71105db0 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71106190 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40044d0 tx=0x7ff1c400d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc68cd1917432aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1c1ffb640 1 -- 172.21.15.76:0/2774656326 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4016030 con 0x562c71105db0 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1c1ffb640 1 -- 172.21.15.76:0/2774656326 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c400db90 con 0x562c71105db0 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1c1ffb640 1 -- 172.21.15.76:0/2774656326 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004990 con 0x562c71105db0 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- 172.21.15.76:0/2774656326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 msgr2=0x562c71106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71106190 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40044d0 tx=0x7ff1c400d150 comp rx=0 tx=0).stop 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- 172.21.15.76:0/2774656326 shutdown_connections 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71106190 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71152690 0x562c71105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2774656326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 0x562c71152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- 172.21.15.76:0/2774656326 >> 172.21.15.76:0/2774656326 conn(0x562c71133040 msgr2=0x562c71135430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- 172.21.15.76:0/2774656326 shutdown_connections 2024-09-19T04:19:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- 172.21.15.76:0/2774656326 wait complete. 2024-09-19T04:19:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- start start 2024-09-19T04:19:21.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 Processor -- start 2024-09-19T04:19:21.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71151d70 0x562c71159220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71152690 0x562c71159760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c70f52d00 con 0x562c71105db0 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562c70f11aa0 con 0x562c71152690 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562c70ef0340 con 0x562c71151d70 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d5555000 10 client.0 _ll_register_callbacks cb 0x562c710f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44468/0 (socket says 172.21.15.76:44468) 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 -- 172.21.15.76:0/496180922 learned_addr learned my addr 172.21.15.76:0/496180922 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71152690 0x562c71159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71151d70 0x562c71159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 -- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71151d70 msgr2=0x562c71159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71151d70 0x562c71159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 -- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71152690 msgr2=0x562c71159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71152690 0x562c71159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cbfff640 1 -- 172.21.15.76:0/496180922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c70ef9dc0 con 0x562c71105db0 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71152690 0x562c71159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.233+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71151d70 0x562c71159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71158ce0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0ae040 tx=0x7ff1bc0ef840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fadeeef098cb66a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/496180922 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0f8770 con 0x562c71105db0 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/496180922 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff1bc0ef270 con 0x562c71105db0 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c400db90 con 0x562c71105db0 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/496180922 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff1bc0efe20 con 0x562c71105db0 2024-09-19T04:19:21.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 10 client.7860 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x562c70ef67f0 con 0x562c71105db0 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/496180922 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c400db90 con 0x562c71105db0 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/496180922 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7ff1bc10e120 con 0x562c71105db0 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 10 client.7860 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 10 client.7860 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 20 client.7860 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c711610b0 0x562c711634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 4 client.7860 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c70ef0e10 con 0x562c711610b0 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c711610b0 0x562c711634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 10 client.7860 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:21.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c711610b0 0x562c711634a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=70f11da607e04014 server_cookie=bbbc737fb398be0 in_seq=0 out_seq=0 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/496180922 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c70ef0e10 con 0x562c711610b0 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 10 client.7860 handle_command_reply: tid=0 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 client.7860 shutdown 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 msgr2=0x562c71158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71158ce0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0ae040 tx=0x7ff1bc0ef840 comp rx=0 tx=0).stop 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 shutdown_connections 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c711610b0 0x562c711634a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71152690 0x562c71159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71151d70 0x562c71159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1a6ffd640 0 client.7860 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 --2- 172.21.15.76:0/496180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71105db0 0x562c71158ce0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 >> 172.21.15.76:0/496180922 conn(0x562c71133040 msgr2=0x562c71104f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 shutdown_connections 2024-09-19T04:19:21.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 1 -- 172.21.15.76:0/496180922 wait complete. 2024-09-19T04:19:21.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.237+0000 7ff1d5555000 20 client.7860 trim_cache size 0 max 16384 2024-09-19T04:19:21.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 Processor -- start 2024-09-19T04:19:21.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- start start 2024-09-19T04:19:21.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71133040 0x562c7110e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 0x562c7115f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71151d70 0x562c7115f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc208c70 con 0x562c71151d70 2024-09-19T04:19:21.258 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1cc207e10 con 0x562c71133040 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1cc208540 con 0x562c71105db0 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71151d70 0x562c7115f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71151d70 0x562c7115f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44482/0 (socket says 172.21.15.76:44482) 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 -- 172.21.15.76:0/1362343764 learned_addr learned my addr 172.21.15.76:0/1362343764 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 -- 172.21.15.76:0/1362343764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 msgr2=0x562c7115f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/1362343764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 0x562c7115f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 -- 172.21.15.76:0/1362343764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71133040 msgr2=0x562c7110e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/1362343764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71133040 0x562c7110e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 -- 172.21.15.76:0/1362343764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c70f14470 con 0x562c71151d70 2024-09-19T04:19:21.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d0a36640 1 --2- 172.21.15.76:0/1362343764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71151d70 0x562c7115f720 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x562c70f1ac90 tx=0x7ff1c4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8b1768d931e96b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1c17fa640 1 -- 172.21.15.76:0/1362343764 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c400de90 con 0x562c71151d70 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1c17fa640 1 -- 172.21.15.76:0/1362343764 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4004990 con 0x562c71151d70 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1c17fa640 1 -- 172.21.15.76:0/1362343764 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4004d60 con 0x562c71151d70 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- 172.21.15.76:0/1362343764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71151d70 msgr2=0x562c7115f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- 172.21.15.76:0/1362343764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71151d70 0x562c7115f720 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x562c70f1ac90 tx=0x7ff1c4012000 comp rx=0 tx=0).stop 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- 172.21.15.76:0/1362343764 shutdown_connections 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- 172.21.15.76:0/1362343764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71151d70 0x562c7115f720 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- 172.21.15.76:0/1362343764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 0x562c7115f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- 172.21.15.76:0/1362343764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71133040 0x562c7110e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- 172.21.15.76:0/1362343764 >> 172.21.15.76:0/1362343764 conn(0x562c7110a890 msgr2=0x562c7115ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- 172.21.15.76:0/1362343764 shutdown_connections 2024-09-19T04:19:21.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- 172.21.15.76:0/1362343764 wait complete. 2024-09-19T04:19:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- start start 2024-09-19T04:19:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 Processor -- start 2024-09-19T04:19:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 0x562c711659c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71133040 0x562c711c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 0x562c711c6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562c70f5b370 con 0x562c71133040 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562c70f157c0 con 0x562c71151d70 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562c70f96780 con 0x562c71105db0 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.257+0000 7ff1d5555000 10 client.0 _ll_register_callbacks cb 0x7ff1bc006c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71133040 0x562c711c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71133040 0x562c711c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44494/0 (socket says 172.21.15.76:44494) 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/2108848255 learned_addr learned my addr 172.21.15.76:0/2108848255 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 msgr2=0x562c711659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 0x562c711659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 0x562c711659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 msgr2=0x562c711c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 0x562c711c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 -- 172.21.15.76:0/2108848255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c70eea640 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cb7fe640 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71133040 0x562c711c5ed0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005d10 tx=0x7ff1b8000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884e73750bc4962e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/2108848255 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b801d040 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/2108848255 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff1b802dcb0 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1c4004990 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/2108848255 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7ff1b8005ad0 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 10 client.7866 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x562c70eee3b0 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/2108848255 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c4004990 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/2108848255 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7ff1b8020070 con 0x562c71133040 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 10 client.7866 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:21.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 10 client.7866 resolve_mds: resolved 6:2 to role '6:2' aka daemon mds.b 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 20 client.7866 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c71152690 0x562c711c6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 4 client.7866 mds_command: new command op to 5794 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 --> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562c70f64470 con 0x562c71152690 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c71152690 0x562c711c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 10 client.7866 ms_handle_connect on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1cbfff640 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c71152690 0x562c711c6950 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=11753fc20b8507a3 server_cookie=fe47009655d705f7 in_seq=0 out_seq=0 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 1 -- 172.21.15.76:0/2108848255 <== mds.2 v2:172.21.15.116:6832/2189465635 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x562c70f64470 con 0x562c71152690 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 10 client.7866 handle_command_reply: tid=0 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 client.7866 shutdown 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71133040 msgr2=0x562c711c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71133040 0x562c711c5ed0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005d10 tx=0x7ff1b8000bd0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 shutdown_connections 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:6832/2189465635,v1:172.21.15.116:6833/2189465635] conn(0x562c71152690 0x562c711c6950 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562c71151d70 0x562c711c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562c71133040 0x562c711c5ed0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1a6ffd640 0 client.7866 ms_handle_reset on v2:172.21.15.116:6832/2189465635 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 --2- 172.21.15.76:0/2108848255 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562c71105db0 0x562c711659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 >> 172.21.15.76:0/2108848255 conn(0x562c7110a890 msgr2=0x562c7115ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 shutdown_connections 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 1 -- 172.21.15.76:0/2108848255 wait complete. 2024-09-19T04:19:21.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d5555000 20 client.7866 trim_cache size 0 max 16384 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: "8362a942-ff57-460f-be25-9fdf5af5b05a": { 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "8362a942-ff57-460f-be25-9fdf5af5b05a", 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:21.269 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 msgr2=0x7ff1b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc0cfd80 tx=0x7ff1b8009040 comp rx=0 tx=0).stop 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc06df70 msgr2=0x7ff1cc09a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc06df70 0x7ff1cc09a380 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc019070 tx=0x7ff1bc019200 comp rx=0 tx=0).stop 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 shutdown_connections 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1cc1630c0 0x7ff1cc0cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1cc0b3290 0x7ff1cc156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 --2- 172.21.15.76:0/1469445594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1cc06df70 0x7ff1cc09a380 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 >> 172.21.15.76:0/1469445594 conn(0x7ff1cc0b1f90 msgr2=0x7ff1cc1643e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 shutdown_connections 2024-09-19T04:19:21.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.261+0000 7ff1d2499640 1 -- 172.21.15.76:0/1469445594 wait complete. 2024-09-19T04:19:21.278 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:19:21.278 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:19:21.278 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-19T04:19:21.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5cc302640 1 Processor -- start 2024-09-19T04:19:21.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5cc302640 1 -- start start 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5cc302640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 0x7fb5c40d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5cc302640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c40d32b0 con 0x7fb5c40cf620 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5cc302640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5c40cfad0 con 0x7fb5c40d2e90 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5cc302640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c40d0300 con 0x7fb5c4161a60 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca89f640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fb5c40cf620 0x7fb5c406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45702/0 (socket says 172.21.15.76:45702) 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca89f640 1 -- 172.21.15.76:0/3734452974 learned_addr learned my addr 172.21.15.76:0/3734452974 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 0x7fb5c40d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1264063686 0 0) 0x7fb5c40d32b0 con 0x7fb5c40cf620 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5b0003530 con 0x7fb5c40cf620 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2047787024 0 0) 0x7fb5b0003530 con 0x7fb5c40cf620 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 msgr2=0x7fb5c40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 0x7fb5c40d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 >> v1:172.21.15.116:6789/0 conn(0x7fb5c40d2e90 legacy=0x7fb5c40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c40b0f60 con 0x7fb5c40cf620 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fb5bc001a00 con 0x7fb5c40cf620 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 >> v1:172.21.15.76:6789/0 conn(0x7fb5c40cf620 legacy=0x7fb5c406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 0x7fb5b0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5b0004d50 0x7fb5b00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c40d0300 con 0x7fb5b00039f0 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb5b000bb00 con 0x7fb5b0004d50 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb5b000bd00 con 0x7fb5c4161a60 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca89f640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5b0004d50 0x7fb5b00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5c989d640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 0x7fb5b0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca09e640 1 -- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 msgr2=0x7fb5b0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c4161a60 0x7fb5b0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca09e640 1 -- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5b0004d50 msgr2=0x7fb5b00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5b0004d50 0x7fb5b00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca09e640 1 -- 172.21.15.76:0/3734452974 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5b0003530 con 0x7fb5b00039f0 2024-09-19T04:19:21.386 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.381+0000 7fb5ca89f640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5b0004d50 0x7fb5b00051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:21.389 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e10 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009ae0 tx=0x7fb5c001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29cfef837dac34b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00096b0 con 0x7fb5b00039f0 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c00a6470 con 0x7fb5b00039f0 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5c909c640 1 -- 172.21.15.76:0/3734452974 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a6860 con 0x7fb5b00039f0 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 -- 172.21.15.76:0/3734452974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 msgr2=0x7fb5b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 --2- 172.21.15.76:0/3734452974 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5b0003e10 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009ae0 tx=0x7fb5c001c980 comp rx=0 tx=0).stop 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5ca89f640 1 -- 172.21.15.76:0/3734452974 reap_dead start 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 -- 172.21.15.76:0/3734452974 shutdown_connections 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 -- 172.21.15.76:0/3734452974 >> 172.21.15.76:0/3734452974 conn(0x7fb5c408bf90 msgr2=0x7fb5c409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 -- 172.21.15.76:0/3734452974 shutdown_connections 2024-09-19T04:19:21.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 -- 172.21.15.76:0/3734452974 wait complete. 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 Processor -- start 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.385+0000 7fb5cc302640 1 -- start start 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5c415cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c415fd30 0x7fb5c40b8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c40b4400 0x7fb5c40b4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4169f20 con 0x7fb5b00039f0 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c406d190 con 0x7fb5c40b4400 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c416a0a0 con 0x7fb5c415fd30 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5c415cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5c415cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44512/0 (socket says 172.21.15.76:44512) 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2855748464 learned_addr learned my addr 172.21.15.76:0/2855748464 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5c989d640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c415fd30 0x7fb5c40b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c415fd30 msgr2=0x7fb5c40b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c415fd30 0x7fb5c40b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c40b4400 msgr2=0x7fb5c40b4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca89f640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c40b4400 0x7fb5c40b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c40b4400 0x7fb5c40b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2855748464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5c415d0c0 con 0x7fb5b00039f0 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5c415cb80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009ae0 tx=0x7fb5b0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5baffd640 1 -- 172.21.15.76:0/2855748464 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00c0070 con 0x7fb5b00039f0 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5baffd640 1 -- 172.21.15.76:0/2855748464 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c000f000 con 0x7fb5b00039f0 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5c40b8e30 con 0x7fb5b00039f0 2024-09-19T04:19:21.392 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5baffd640 1 -- 172.21.15.76:0/2855748464 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a7450 con 0x7fb5b00039f0 2024-09-19T04:19:21.393 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5c40b93f0 con 0x7fb5b00039f0 2024-09-19T04:19:21.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5baffd640 1 -- 172.21.15.76:0/2855748464 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb5c00c3070 con 0x7fb5b00039f0 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5baffd640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5baffd640 1 -- 172.21.15.76:0/2855748464 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fb5c011fbc0 con 0x7fb5b00039f0 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5c989d640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.389+0000 7fb5c989d640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb5c41604e0 tx=0x7fb5b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 Processor -- start 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- start start 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 0x561d213f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f4690 0x561d213a7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213a7db0 0x561d213a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561d21291960 con 0x561d213f4690 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561d211d5900 con 0x561d213f3d70 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561d211f8d40 con 0x561d213a7db0 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f4690 0x561d213a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f4690 0x561d213a7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44522/0 (socket says 172.21.15.76:44522) 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 -- 172.21.15.76:0/1593452820 learned_addr learned my addr 172.21.15.76:0/1593452820 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 -- 172.21.15.76:0/1593452820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213a7db0 msgr2=0x561d213a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 --2- 172.21.15.76:0/1593452820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213a7db0 0x561d213a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 -- 172.21.15.76:0/1593452820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 msgr2=0x561d213f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 --2- 172.21.15.76:0/1593452820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 0x561d213f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 -- 172.21.15.76:0/1593452820 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d2119f580 con 0x561d213f4690 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 --2- 172.21.15.76:0/1593452820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f4690 0x561d213a7870 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40051d0 tx=0x7fb5b4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a807ed41a0b5d2fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5a7fff640 1 -- 172.21.15.76:0/1593452820 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4026070 con 0x561d213f4690 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5a7fff640 1 -- 172.21.15.76:0/1593452820 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b400fe60 con 0x561d213f4690 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5a7fff640 1 -- 172.21.15.76:0/1593452820 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4003a10 con 0x561d213f4690 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- 172.21.15.76:0/1593452820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f4690 msgr2=0x561d213a7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/1593452820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f4690 0x561d213a7870 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fb5b40051d0 tx=0x7fb5b4012000 comp rx=0 tx=0).stop 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- 172.21.15.76:0/1593452820 shutdown_connections 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/1593452820 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213a7db0 0x561d213a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/1593452820 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f4690 0x561d213a7870 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/1593452820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 0x561d213f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- 172.21.15.76:0/1593452820 >> 172.21.15.76:0/1593452820 conn(0x561d213d5040 msgr2=0x561d213d7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- 172.21.15.76:0/1593452820 shutdown_connections 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- 172.21.15.76:0/1593452820 wait complete. 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- start start 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 Processor -- start 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d213face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f3d70 0x561d213fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f4690 0x561d213fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561d211f4d00 con 0x561d213a7db0 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561d211b3aa0 con 0x561d213f4690 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561d21192340 con 0x561d213f3d70 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5cf3be000 10 client.0 _ll_register_callbacks cb 0x561d21395760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5ca09e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d213face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5ca09e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d213face0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44526/0 (socket says 172.21.15.76:44526) 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2511368377 learned_addr learned my addr 172.21.15.76:0/2511368377 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5c989d640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f3d70 0x561d213fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.393+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f3d70 msgr2=0x561d213fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5ca89f640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f4690 0x561d213fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f3d70 0x561d213fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f4690 msgr2=0x561d213fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f4690 0x561d213fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2511368377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2119bdc0 con 0x561d213a7db0 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5c989d640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f3d70 0x561d213fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d213face0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00ec260 tx=0x7fb5c0009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98ddf213c34b80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 1 -- 172.21.15.76:0/2511368377 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c010d060 con 0x561d213a7db0 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 1 -- 172.21.15.76:0/2511368377 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fb5c00ee100 con 0x561d213a7db0 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5b400fe60 con 0x561d213a7db0 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 1 -- 172.21.15.76:0/2511368377 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fb5c00f41b0 con 0x561d213a7db0 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 10 client.7878 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:21.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x561d211987f0 con 0x561d213a7db0 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 1 -- 172.21.15.76:0/2511368377 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b400fe60 con 0x561d213a7db0 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 1 -- 172.21.15.76:0/2511368377 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7fb5c010e250 con 0x561d213a7db0 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 10 client.7878 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 10 client.7878 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 20 client.7878 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214030b0 0x561d214054a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 4 client.7878 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d21192e10 con 0x561d214030b0 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5c989d640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214030b0 0x561d214054a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 10 client.7878 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5c989d640 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214030b0 0x561d214054a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc8621a46205bc6 server_cookie=6531bd66b44de32c in_seq=0 out_seq=0 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 1 -- 172.21.15.76:0/2511368377 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d21192e10 con 0x561d214030b0 2024-09-19T04:19:21.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 10 client.7878 handle_command_reply: tid=0 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 client.7878 shutdown 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 msgr2=0x561d213face0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d213face0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00ec260 tx=0x7fb5c0009770 comp rx=0 tx=0).stop 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 shutdown_connections 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214030b0 0x561d214054a0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f4690 0x561d213fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f3d70 0x561d213fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2511368377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d213face0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5a57fa640 0 client.7878 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 >> 172.21.15.76:0/2511368377 conn(0x561d213d5040 msgr2=0x561d213a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 shutdown_connections 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2511368377 wait complete. 2024-09-19T04:19:21.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.397+0000 7fb5cf3be000 20 client.7878 trim_cache size 0 max 16384 2024-09-19T04:19:21.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 Processor -- start 2024-09-19T04:19:21.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- start start 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d21401360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 0x561d213afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 0x561d213b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4204f70 con 0x561d213a7db0 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c4204110 con 0x561d213f3d70 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb5c4204840 con 0x561d213f4690 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d21401360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d21401360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44528/0 (socket says 172.21.15.76:44528) 2024-09-19T04:19:21.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2977887036 learned_addr learned my addr 172.21.15.76:0/2977887036 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2977887036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 msgr2=0x561d213b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2977887036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 0x561d213b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2977887036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 msgr2=0x561d213afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2977887036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 0x561d213afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 -- 172.21.15.76:0/2977887036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d211b6470 con 0x561d213a7db0 2024-09-19T04:19:21.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/2977887036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d21401360 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb5c010e5c0 tx=0x7fb5c00bc000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa429b7c6cd57a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5a77fe640 1 -- 172.21.15.76:0/2977887036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00ef940 con 0x561d213a7db0 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5a77fe640 1 -- 172.21.15.76:0/2977887036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c00ee100 con 0x561d213a7db0 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5a77fe640 1 -- 172.21.15.76:0/2977887036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c010e250 con 0x561d213a7db0 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2977887036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 msgr2=0x561d21401360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2977887036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d21401360 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb5c010e5c0 tx=0x7fb5c00bc000 comp rx=0 tx=0).stop 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2977887036 shutdown_connections 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2977887036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 0x561d213b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2977887036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213f3d70 0x561d213afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/2977887036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213a7db0 0x561d21401360 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2977887036 >> 172.21.15.76:0/2977887036 conn(0x561d213d5040 msgr2=0x561d213faaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2977887036 shutdown_connections 2024-09-19T04:19:21.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- 172.21.15.76:0/2977887036 wait complete. 2024-09-19T04:19:21.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- start start 2024-09-19T04:19:21.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 Processor -- start 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213a7db0 0x561d21400700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f3d70 0x561d21463730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 0x561d21463c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561d212b50c0 con 0x561d213f3d70 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb5c40c8c30 con 0x561d213a7db0 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561d211f8050 con 0x561d213f4690 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5cf3be000 10 client.0 _ll_register_callbacks cb 0x7fb5c00069d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5c989d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f3d70 0x561d21463730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5c989d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f3d70 0x561d21463730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44536/0 (socket says 172.21.15.76:44536) 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.417+0000 7fb5c989d640 1 -- 172.21.15.76:0/3739068157 learned_addr learned my addr 172.21.15.76:0/3739068157 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5ca89f640 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 0x561d21463c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5c989d640 1 -- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 msgr2=0x561d21463c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5c989d640 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 0x561d21463c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213a7db0 0x561d21400700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5c989d640 1 -- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213a7db0 msgr2=0x561d21400700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5c989d640 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213a7db0 0x561d21400700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5c989d640 1 -- 172.21.15.76:0/3739068157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d2118c640 con 0x561d213f3d70 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5c989d640 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f3d70 0x561d21463730 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c10 tx=0x7fb5b4001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e4e39848a296cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 1 -- 172.21.15.76:0/3739068157 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b4026030 con 0x561d213f3d70 2024-09-19T04:19:21.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 1 -- 172.21.15.76:0/3739068157 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fb5b402e440 con 0x561d213f3d70 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5c00ee100 con 0x561d213f3d70 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 1 -- 172.21.15.76:0/3739068157 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=94) ==== 24+0+0 (secure 0 0 0) 0x7fb5b403f050 con 0x561d213f3d70 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 10 client.7884 fetch_fsmap learned FSMap version 94 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=94,monmap=2+}) -- 0x561d211903b0 con 0x561d213f3d70 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 1 -- 172.21.15.76:0/3739068157 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c00ee100 con 0x561d213f3d70 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 1 -- 172.21.15.76:0/3739068157 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 94) ==== 3558+0+0 (secure 0 0 0) 0x7fb5b4042020 con 0x561d213f3d70 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 10 client.7884 fetch_fsmap finished waiting for FSMap version 94 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 10 client.7884 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 20 client.7884 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214641b0 0x561d214645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.428 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 4 client.7884 mds_command: new command op to 7311 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 --> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561d21206470 con 0x561d214641b0 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214641b0 0x561d214645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5ca09e640 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214641b0 0x561d214645c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9f83afa67f29b92 server_cookie=4caaf90e26df7d49 in_seq=0 out_seq=0 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 10 client.7884 ms_handle_connect on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 1 -- 172.21.15.76:0/3739068157 <== mds.0 v2:172.21.15.76:6836/2711186256 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x561d21206470 con 0x561d214641b0 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 10 client.7884 handle_command_reply: tid=0 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 client.7884 shutdown 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f3d70 msgr2=0x561d21463730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f3d70 0x561d21463730 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb5b4000c10 tx=0x7fb5b4001130 comp rx=0 tx=0).stop 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 shutdown_connections 2024-09-19T04:19:21.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:6836/2711186256,v1:172.21.15.76:6837/2711186256] conn(0x561d214641b0 0x561d214645c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561d213f4690 0x561d21463c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561d213f3d70 0x561d21463730 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5a57fa640 0 client.7884 ms_handle_reset on v2:172.21.15.76:6836/2711186256 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 --2- 172.21.15.76:0/3739068157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561d213a7db0 0x561d21400700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 >> 172.21.15.76:0/3739068157 conn(0x561d213d5040 msgr2=0x561d213aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 shutdown_connections 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 1 -- 172.21.15.76:0/3739068157 wait complete. 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cf3be000 20 client.7884 trim_cache size 0 max 16384 2024-09-19T04:19:21.430 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: "8362a942-ff57-460f-be25-9fdf5af5b05a": { 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "8362a942-ff57-460f-be25-9fdf5af5b05a", 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:21.431 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:21.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 msgr2=0x7fb5a8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb5c41604e0 tx=0x7fb5b4009040 comp rx=0 tx=0).stop 2024-09-19T04:19:21.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 msgr2=0x7fb5c415cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5c415cb80 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009ae0 tx=0x7fb5b0005840 comp rx=0 tx=0).stop 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 shutdown_connections 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb5a80678c0 0x7fb5a8069d80 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5c40b4400 0x7fb5c40b4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb5c415fd30 0x7fb5c40b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 --2- 172.21.15.76:0/2855748464 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb5b00039f0 0x7fb5c415cb80 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 >> 172.21.15.76:0/2855748464 conn(0x7fb5c408bf90 msgr2=0x7fb5c40cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 shutdown_connections 2024-09-19T04:19:21.433 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.421+0000 7fb5cc302640 1 -- 172.21.15.76:0/2855748464 wait complete. 2024-09-19T04:19:21.437 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:19:21.437 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:21.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b817d640 1 Processor -- start 2024-09-19T04:19:21.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b817d640 1 -- start start 2024-09-19T04:19:21.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b817d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b015e5f0 0x7f15b006d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b817d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15b00ce9d0 0x7f15b00cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b817d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f15b006d9c0 con 0x7f15b00cf920 2024-09-19T04:19:21.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b817d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f15b006dca0 con 0x7f15b00ce9d0 2024-09-19T04:19:21.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b817d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f15b006df80 con 0x7f15b015e5f0 2024-09-19T04:19:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5f19640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b015e5f0 0x7f15b006d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5f19640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b015e5f0 0x7f15b006d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:51282/0 (socket says 172.21.15.76:51282) 2024-09-19T04:19:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5f19640 1 -- 172.21.15.76:0/3057471313 learned_addr learned my addr 172.21.15.76:0/3057471313 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5718640 1 --2- 172.21.15.76:0/3057471313 >> v2:172.21.15.116:3300/0 conn(0x7f15b00ce9d0 0x7f15b00cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5f19640 1 -- 172.21.15.76:0/3057471313 >> v2:172.21.15.116:3300/0 conn(0x7f15b00ce9d0 msgr2=0x7f15b00cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5f19640 1 --2- 172.21.15.76:0/3057471313 >> v2:172.21.15.116:3300/0 conn(0x7f15b00ce9d0 0x7f15b00cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5f19640 1 -- 172.21.15.76:0/3057471313 >> v1:172.21.15.116:6789/0 conn(0x7f15b00cf920 legacy=0x7f15b00d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.537+0000 7f15b5f19640 1 -- 172.21.15.76:0/3057471313 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b006e260 con 0x7f15b015e5f0 2024-09-19T04:19:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b5f19640 1 --2- 172.21.15.76:0/3057471313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b015e5f0 0x7f15b006d090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009870 tx=0x7f15ac01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fef8aafaf373dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b4f17640 1 -- 172.21.15.76:0/3057471313 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0b2070 con 0x7f15b015e5f0 2024-09-19T04:19:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b4f17640 1 -- 172.21.15.76:0/3057471313 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15ac0ad640 con 0x7f15b015e5f0 2024-09-19T04:19:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b4f17640 1 -- 172.21.15.76:0/3057471313 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac0ada10 con 0x7f15b015e5f0 2024-09-19T04:19:21.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- 172.21.15.76:0/3057471313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b015e5f0 msgr2=0x7f15b006d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 --2- 172.21.15.76:0/3057471313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b015e5f0 0x7f15b006d090 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f15ac009870 tx=0x7f15ac01c560 comp rx=0 tx=0).stop 2024-09-19T04:19:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- 172.21.15.76:0/3057471313 shutdown_connections 2024-09-19T04:19:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 --2- 172.21.15.76:0/3057471313 >> v2:172.21.15.116:3300/0 conn(0x7f15b00ce9d0 0x7f15b00cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 --2- 172.21.15.76:0/3057471313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b015e5f0 0x7f15b006d090 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- 172.21.15.76:0/3057471313 >> 172.21.15.76:0/3057471313 conn(0x7f15b0074030 msgr2=0x7f15b009d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- 172.21.15.76:0/3057471313 shutdown_connections 2024-09-19T04:19:21.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- 172.21.15.76:0/3057471313 wait complete. 2024-09-19T04:19:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 Processor -- start 2024-09-19T04:19:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- start start 2024-09-19T04:19:21.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00cf920 0x7f15b01569d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b00ce1c0 0x7f15b00b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b0154b00 0x7f15b0154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f15b016b1b0 con 0x7f15b0154b00 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f15b00cfd50 con 0x7f15b00cf920 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b817d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f15b016b330 con 0x7f15b00ce1c0 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b0154b00 0x7f15b0154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b0154b00 0x7f15b0154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44542/0 (socket says 172.21.15.76:44542) 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 -- 172.21.15.76:0/2110894339 learned_addr learned my addr 172.21.15.76:0/2110894339 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b5718640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b00ce1c0 0x7f15b00b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b5f19640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00cf920 0x7f15b01569d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 -- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b00ce1c0 msgr2=0x7f15b00b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b00ce1c0 0x7f15b00b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 -- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00cf920 msgr2=0x7f15b01569d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00cf920 0x7f15b01569d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 -- 172.21.15.76:0/2110894339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b01554c0 con 0x7f15b0154b00 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.541+0000 7f15b671a640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b0154b00 0x7f15b0154f80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f15a800c6c0 tx=0x7f15a800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15a67fc640 1 -- 172.21.15.76:0/2110894339 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a8003b80 con 0x7f15b0154b00 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15a67fc640 1 -- 172.21.15.76:0/2110894339 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15a8003d20 con 0x7f15b0154b00 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15a67fc640 1 -- 172.21.15.76:0/2110894339 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a80052f0 con 0x7f15b0154b00 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b0156f10 con 0x7f15b0154b00 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b00cb5f0 con 0x7f15b0154b00 2024-09-19T04:19:21.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b0158040 con 0x7f15b0154b00 2024-09-19T04:19:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15a67fc640 1 -- 172.21.15.76:0/2110894339 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f15a8005490 con 0x7f15b0154b00 2024-09-19T04:19:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15a67fc640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15940678f0 0x7f1594069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15a67fc640 1 -- 172.21.15.76:0/2110894339 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f15a8085c20 con 0x7f15b0154b00 2024-09-19T04:19:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15b5f19640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15940678f0 0x7f1594069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.547 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.545+0000 7f15b5f19640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15940678f0 0x7f1594069db0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0af000 tx=0x7f15ac0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.550 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.549+0000 7f15a67fc640 1 -- 172.21.15.76:0/2110894339 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f15a80565f0 con 0x7f15b0154b00 2024-09-19T04:19:21.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.729+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15b00d11a0 con 0x7f15b0154b00 2024-09-19T04:19:21.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15a67fc640 1 -- 172.21.15.76:0/2110894339 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7f15a8055ef0 con 0x7f15b0154b00 2024-09-19T04:19:21.733 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:21.733 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:21.733 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:21.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15940678f0 msgr2=0x7f1594069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15940678f0 0x7f1594069db0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0af000 tx=0x7f15ac0a7070 comp rx=0 tx=0).stop 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b0154b00 msgr2=0x7f15b0154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b0154b00 0x7f15b0154f80 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f15a800c6c0 tx=0x7f15a800cb80 comp rx=0 tx=0).stop 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 shutdown_connections 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15940678f0 0x7f1594069db0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f15b0154b00 0x7f15b0154f80 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f15b00ce1c0 0x7f15b00b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 --2- 172.21.15.76:0/2110894339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00cf920 0x7f15b01569d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 >> 172.21.15.76:0/2110894339 conn(0x7f15b00ce9d0 msgr2=0x7f15b00b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 shutdown_connections 2024-09-19T04:19:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.733+0000 7f15b817d640 1 -- 172.21.15.76:0/2110894339 wait complete. 2024-09-19T04:19:21.750 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:21.750 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4679a29640 1 Processor -- start 2024-09-19T04:19:21.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4679a29640 1 -- start start 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4679a29640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f467415c2a0 0x7f46740aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4679a29640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46740b3290 con 0x7f467415bec0 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4679a29640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46740b3570 con 0x7f46740af2a0 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4679a29640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f46740b3850 con 0x7f467415c2a0 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f466bfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f467415c2a0 0x7f46740aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4672ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f467415bec0 0x7f46740ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45712/0 (socket says 172.21.15.76:45712) 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f4672ffd640 1 -- 172.21.15.76:0/2611289142 learned_addr learned my addr 172.21.15.76:0/2611289142 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.865+0000 7f46727fc640 1 -- 172.21.15.76:0/2611289142 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3356992470 0 0) 0x7f46740b3290 con 0x7f467415bec0 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f46727fc640 1 -- 172.21.15.76:0/2611289142 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4660003530 con 0x7f467415bec0 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f466bfff640 1 -- 172.21.15.76:0/2611289142 >> v1:172.21.15.116:6789/0 conn(0x7f46740af2a0 legacy=0x7f467406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f466bfff640 1 -- 172.21.15.76:0/2611289142 >> v1:172.21.15.76:6789/0 conn(0x7f467415bec0 legacy=0x7f46740ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f466bfff640 1 -- 172.21.15.76:0/2611289142 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46740b3b30 con 0x7f467415c2a0 2024-09-19T04:19:21.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f466bfff640 1 --2- 172.21.15.76:0/2611289142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467415c2a0 0x7f46740aed60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f465c0097c0 tx=0x7f465c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90324237e66c2ba4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f46727fc640 1 -- 172.21.15.76:0/2611289142 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0ab430 con 0x7f467415c2a0 2024-09-19T04:19:21.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f46727fc640 1 -- 172.21.15.76:0/2611289142 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f465c0b0070 con 0x7f467415c2a0 2024-09-19T04:19:21.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f46727fc640 1 -- 172.21.15.76:0/2611289142 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0ab7f0 con 0x7f467415c2a0 2024-09-19T04:19:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f4679a29640 1 -- 172.21.15.76:0/2611289142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467415c2a0 msgr2=0x7f46740aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f4679a29640 1 --2- 172.21.15.76:0/2611289142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467415c2a0 0x7f46740aed60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f465c0097c0 tx=0x7f465c01c430 comp rx=0 tx=0).stop 2024-09-19T04:19:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f4679a29640 1 -- 172.21.15.76:0/2611289142 shutdown_connections 2024-09-19T04:19:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f4679a29640 1 --2- 172.21.15.76:0/2611289142 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f467415c2a0 0x7f46740aed60 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f4679a29640 1 -- 172.21.15.76:0/2611289142 >> 172.21.15.76:0/2611289142 conn(0x7f46740b1f90 msgr2=0x7f46740b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f4679a29640 1 -- 172.21.15.76:0/2611289142 shutdown_connections 2024-09-19T04:19:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.869+0000 7f4679a29640 1 -- 172.21.15.76:0/2611289142 wait complete. 2024-09-19T04:19:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 Processor -- start 2024-09-19T04:19:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 -- start start 2024-09-19T04:19:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46740af2a0 0x7f467415efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f467415bec0 0x7f4674146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f467415c2a0 0x7f467414e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f467406ef20 con 0x7f46740af2a0 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f46740af710 con 0x7f467415bec0 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f467406f0a0 con 0x7f467415c2a0 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46740af2a0 0x7f467415efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46740af2a0 0x7f467415efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:44572/0 (socket says 172.21.15.76:44572) 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 -- 172.21.15.76:0/2616678561 learned_addr learned my addr 172.21.15.76:0/2616678561 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 -- 172.21.15.76:0/2616678561 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f467415c2a0 msgr2=0x7f467414e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f467415c2a0 0x7f467414e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 -- 172.21.15.76:0/2616678561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f467415bec0 msgr2=0x7f4674146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f467415bec0 0x7f4674146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 -- 172.21.15.76:0/2616678561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467415d160 con 0x7f46740af2a0 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4672ffd640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46740af2a0 0x7f467415efd0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f466400c450 tx=0x7f466400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4670ff9640 1 -- 172.21.15.76:0/2616678561 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664017070 con 0x7f46740af2a0 2024-09-19T04:19:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4670ff9640 1 -- 172.21.15.76:0/2616678561 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4664005cf0 con 0x7f46740af2a0 2024-09-19T04:19:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4670ff9640 1 -- 172.21.15.76:0/2616678561 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4664007290 con 0x7f46740af2a0 2024-09-19T04:19:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467415d400 con 0x7f46740af2a0 2024-09-19T04:19:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467415f5b0 con 0x7f46740af2a0 2024-09-19T04:19:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.873+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46740aed60 con 0x7f46740af2a0 2024-09-19T04:19:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.877+0000 7f4670ff9640 1 -- 172.21.15.76:0/2616678561 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f466401a070 con 0x7f46740af2a0 2024-09-19T04:19:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.877+0000 7f4670ff9640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4654067910 0x7f4654069dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.877+0000 7f4670ff9640 1 -- 172.21.15.76:0/2616678561 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f4664085200 con 0x7f46740af2a0 2024-09-19T04:19:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.877+0000 7f466bfff640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4654067910 0x7f4654069dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.877+0000 7f466bfff640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4654067910 0x7f4654069dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f465c0af000 tx=0x7f465c004740 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:21.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:21.881+0000 7f4670ff9640 1 -- 172.21.15.76:0/2616678561 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4664055c20 con 0x7f46740af2a0 2024-09-19T04:19:22.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.089+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f46740af9b0 con 0x7f46740af2a0 2024-09-19T04:19:22.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.089+0000 7f4670ff9640 1 -- 172.21.15.76:0/2616678561 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2008 (secure 0 0 0) 0x7f4664055520 con 0x7f46740af2a0 2024-09-19T04:19:22.091 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:22.091 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":381,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":695469,"bytes_used":261898240,"bytes_avail":772832215040,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6483,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-09-19T04:19:19.851943+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:22.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4654067910 msgr2=0x7f4654069dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:22.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4654067910 0x7f4654069dd0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f465c0af000 tx=0x7f465c004740 comp rx=0 tx=0).stop 2024-09-19T04:19:22.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46740af2a0 msgr2=0x7f467415efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:22.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46740af2a0 0x7f467415efd0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f466400c450 tx=0x7f466400c910 comp rx=0 tx=0).stop 2024-09-19T04:19:22.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 shutdown_connections 2024-09-19T04:19:22.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4654067910 0x7f4654069dd0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f467415c2a0 0x7f467414e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f467415bec0 0x7f4674146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 --2- 172.21.15.76:0/2616678561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f46740af2a0 0x7f467415efd0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:22.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 >> 172.21.15.76:0/2616678561 conn(0x7f46740b1f90 msgr2=0x7f46740b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:22.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 shutdown_connections 2024-09-19T04:19:22.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:22.093+0000 7f4679a29640 1 -- 172.21.15.76:0/2616678561 wait complete. 2024-09-19T04:19:22.109 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:19:23.110 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:23.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda6b57c640 1 Processor -- start 2024-09-19T04:19:23.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda6b57c640 1 -- start start 2024-09-19T04:19:23.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda6b57c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda6415c2a0 0x7fda640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda6b57c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda640af2a0 0x7fda6406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda6b57c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda640b3290 con 0x7fda6415bec0 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda6b57c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fda640b3570 con 0x7fda6415c2a0 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda6b57c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fda640b3850 con 0x7fda640af2a0 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda69318640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fda6415bec0 0x7fda640ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50844/0 (socket says 172.21.15.76:50844) 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda68b17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fda6415c2a0 0x7fda640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda69318640 1 -- 172.21.15.76:0/3474495713 learned_addr learned my addr 172.21.15.76:0/3474495713 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda5bfff640 1 -- 172.21.15.76:0/3474495713 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2938291724 0 0) 0x7fda640b3290 con 0x7fda6415bec0 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda5bfff640 1 -- 172.21.15.76:0/3474495713 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fda50003530 con 0x7fda6415bec0 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda69b19640 1 --2- 172.21.15.76:0/3474495713 >> v2:172.21.15.116:3300/0 conn(0x7fda640af2a0 0x7fda6406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda68b17640 1 -- 172.21.15.76:0/3474495713 >> v2:172.21.15.116:3300/0 conn(0x7fda640af2a0 msgr2=0x7fda6406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda68b17640 1 --2- 172.21.15.76:0/3474495713 >> v2:172.21.15.116:3300/0 conn(0x7fda640af2a0 0x7fda6406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda68b17640 1 -- 172.21.15.76:0/3474495713 >> v1:172.21.15.76:6789/0 conn(0x7fda6415bec0 legacy=0x7fda640ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda68b17640 1 -- 172.21.15.76:0/3474495713 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda640b3b30 con 0x7fda6415c2a0 2024-09-19T04:19:23.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.240+0000 7fda68b17640 1 --2- 172.21.15.76:0/3474495713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda640aed60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fda640b2fb0 tx=0x7fda6001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975553ba68850a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:23.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda5bfff640 1 -- 172.21.15.76:0/3474495713 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda600ab650 con 0x7fda6415c2a0 2024-09-19T04:19:23.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda5bfff640 1 -- 172.21.15.76:0/3474495713 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda600b0070 con 0x7fda6415c2a0 2024-09-19T04:19:23.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda5bfff640 1 -- 172.21.15.76:0/3474495713 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda600ab9f0 con 0x7fda6415c2a0 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- 172.21.15.76:0/3474495713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 msgr2=0x7fda640aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 --2- 172.21.15.76:0/3474495713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda640aed60 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fda640b2fb0 tx=0x7fda6001c450 comp rx=0 tx=0).stop 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- 172.21.15.76:0/3474495713 shutdown_connections 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 --2- 172.21.15.76:0/3474495713 >> v2:172.21.15.116:3300/0 conn(0x7fda640af2a0 0x7fda6406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 --2- 172.21.15.76:0/3474495713 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda640aed60 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- 172.21.15.76:0/3474495713 >> 172.21.15.76:0/3474495713 conn(0x7fda640b1f90 msgr2=0x7fda640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- 172.21.15.76:0/3474495713 shutdown_connections 2024-09-19T04:19:23.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- 172.21.15.76:0/3474495713 wait complete. 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 Processor -- start 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- start start 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda640af2a0 0x7fda6414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda6415bec0 0x7fda64157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda64157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fda6406ed50 con 0x7fda6415c2a0 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fda640af6b0 con 0x7fda640af2a0 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fda6406eed0 con 0x7fda6415bec0 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda64157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda64157ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34524/0 (socket says 172.21.15.76:34524) 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 -- 172.21.15.76:0/901536685 learned_addr learned my addr 172.21.15.76:0/901536685 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:23.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda68b17640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda6415bec0 0x7fda64157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 -- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda6415bec0 msgr2=0x7fda64157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69318640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda640af2a0 0x7fda6414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda6415bec0 0x7fda64157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 -- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda640af2a0 msgr2=0x7fda6414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda640af2a0 0x7fda6414c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 -- 172.21.15.76:0/901536685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6415f480 con 0x7fda6415c2a0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda68b17640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda6415bec0 0x7fda64157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda69b19640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda64157ba0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007e70 tx=0x7fda5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda59ffb640 1 -- 172.21.15.76:0/901536685 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c014620 con 0x7fda6415c2a0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda59ffb640 1 -- 172.21.15.76:0/901536685 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda5c019070 con 0x7fda6415c2a0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6415d830 con 0x7fda6415c2a0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda59ffb640 1 -- 172.21.15.76:0/901536685 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c014a20 con 0x7fda6415c2a0 2024-09-19T04:19:23.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.244+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6415ddb0 con 0x7fda6415c2a0 2024-09-19T04:19:23.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.248+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6406f1d0 con 0x7fda6415c2a0 2024-09-19T04:19:23.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.248+0000 7fda59ffb640 1 -- 172.21.15.76:0/901536685 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fda5c014bc0 con 0x7fda6415c2a0 2024-09-19T04:19:23.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.248+0000 7fda59ffb640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda480678c0 0x7fda48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.248+0000 7fda59ffb640 1 -- 172.21.15.76:0/901536685 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fda5c087580 con 0x7fda6415c2a0 2024-09-19T04:19:23.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.248+0000 7fda69318640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda480678c0 0x7fda48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.248+0000 7fda69318640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda480678c0 0x7fda48069d80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fda54007fd0 tx=0x7fda54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:23.253 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.252+0000 7fda59ffb640 1 -- 172.21.15.76:0/901536685 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fda5c058030 con 0x7fda6415c2a0 2024-09-19T04:19:23.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.440+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda6414ce70 con 0x7fda6415c2a0 2024-09-19T04:19:23.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.440+0000 7fda59ffb640 1 -- 172.21.15.76:0/901536685 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7fda5c057930 con 0x7fda6415c2a0 2024-09-19T04:19:23.441 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:23.441 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:23.442 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.440+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda480678c0 msgr2=0x7fda48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.440+0000 7fda6b57c640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda480678c0 0x7fda48069d80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fda54007fd0 tx=0x7fda54009040 comp rx=0 tx=0).stop 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.440+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 msgr2=0x7fda64157ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.440+0000 7fda6b57c640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda64157ba0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fda5c007e70 tx=0x7fda5c00e4b0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 shutdown_connections 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fda480678c0 0x7fda48069d80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fda6415c2a0 0x7fda64157ba0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fda6415bec0 0x7fda64157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.444 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 --2- 172.21.15.76:0/901536685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda640af2a0 0x7fda6414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 >> 172.21.15.76:0/901536685 conn(0x7fda640b1f90 msgr2=0x7fda640b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:23.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 shutdown_connections 2024-09-19T04:19:23.445 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.444+0000 7fda6b57c640 1 -- 172.21.15.76:0/901536685 wait complete. 2024-09-19T04:19:23.459 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:23.459 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:23.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1b6b3640 1 Processor -- start 2024-09-19T04:19:23.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1b6b3640 1 -- start start 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1b6b3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf141587a0 0x7fbf140b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1b6b3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf1406e680 0x7fbf14072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1b6b3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf1406ec60 con 0x7fbf140b37d0 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1b6b3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fbf140b3c40 con 0x7fbf1406e680 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1b6b3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbf1406f870 con 0x7fbf141587a0 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf19c50640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fbf1406e680 0x7fbf14072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf18c4e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbf140b37d0 0x7fbf1406df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50860/0 (socket says 172.21.15.76:50860) 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf18c4e640 1 -- 172.21.15.76:0/528132815 learned_addr learned my addr 172.21.15.76:0/528132815 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf19c50640 1 --2- 172.21.15.76:0/528132815 >> v2:172.21.15.76:3300/0 conn(0x7fbf1406e680 0x7fbf14072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34532/0 (socket says 172.21.15.76:34532) 2024-09-19T04:19:23.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf1944f640 1 --2- 172.21.15.76:0/528132815 >> v2:172.21.15.116:3300/0 conn(0x7fbf141587a0 0x7fbf140b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf19c50640 1 -- 172.21.15.76:0/528132815 >> v2:172.21.15.116:3300/0 conn(0x7fbf141587a0 msgr2=0x7fbf140b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:23.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf19c50640 1 --2- 172.21.15.76:0/528132815 >> v2:172.21.15.116:3300/0 conn(0x7fbf141587a0 0x7fbf140b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf19c50640 1 -- 172.21.15.76:0/528132815 >> v1:172.21.15.76:6789/0 conn(0x7fbf140b37d0 legacy=0x7fbf1406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.568+0000 7fbf19c50640 1 -- 172.21.15.76:0/528132815 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf1406ef40 con 0x7fbf1406e680 2024-09-19T04:19:23.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf19c50640 1 --2- 172.21.15.76:0/528132815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf1406e680 0x7fbf14072160 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbf14071710 tx=0x7fbf0c01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721415c240f28efc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:23.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf07fff640 1 -- 172.21.15.76:0/528132815 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0b2070 con 0x7fbf1406e680 2024-09-19T04:19:23.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf07fff640 1 -- 172.21.15.76:0/528132815 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c0a59c0 con 0x7fbf1406e680 2024-09-19T04:19:23.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf07fff640 1 -- 172.21.15.76:0/528132815 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0a5d90 con 0x7fbf1406e680 2024-09-19T04:19:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/528132815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf1406e680 msgr2=0x7fbf14072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/528132815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf1406e680 0x7fbf14072160 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbf14071710 tx=0x7fbf0c01c900 comp rx=0 tx=0).stop 2024-09-19T04:19:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/528132815 shutdown_connections 2024-09-19T04:19:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/528132815 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf1406e680 0x7fbf14072160 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/528132815 >> v2:172.21.15.116:3300/0 conn(0x7fbf141587a0 0x7fbf140b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/528132815 >> 172.21.15.76:0/528132815 conn(0x7fbf140b1f90 msgr2=0x7fbf140b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:23.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/528132815 shutdown_connections 2024-09-19T04:19:23.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/528132815 wait complete. 2024-09-19T04:19:23.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 Processor -- start 2024-09-19T04:19:23.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.572+0000 7fbf1b6b3640 1 -- start start 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf1406e680 0x7fbf14154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf140b37d0 0x7fbf1409df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf140b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14073030 con 0x7fbf141587a0 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbf14072eb0 con 0x7fbf1406e680 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbf140731b0 con 0x7fbf140b37d0 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf140b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.577 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf140b0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34538/0 (socket says 172.21.15.76:34538) 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 -- 172.21.15.76:0/3317661631 learned_addr learned my addr 172.21.15.76:0/3317661631 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf18c4e640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf140b37d0 0x7fbf1409df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1944f640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf1406e680 0x7fbf14154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 -- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf140b37d0 msgr2=0x7fbf1409df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf140b37d0 0x7fbf1409df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 -- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf1406e680 msgr2=0x7fbf14154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf1406e680 0x7fbf14154450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 -- 172.21.15.76:0/3317661631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf14154a30 con 0x7fbf141587a0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf19c50640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf140b0600 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c019510 tx=0x7fbf0c0196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf05ffb640 1 -- 172.21.15.76:0/3317661631 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0b2050 con 0x7fbf141587a0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf05ffb640 1 -- 172.21.15.76:0/3317661631 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf0c0be070 con 0x7fbf141587a0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf05ffb640 1 -- 172.21.15.76:0/3317661631 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0adb10 con 0x7fbf141587a0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf14154d10 con 0x7fbf141587a0 2024-09-19T04:19:23.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf141552d0 con 0x7fbf141587a0 2024-09-19T04:19:23.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf14070080 con 0x7fbf141587a0 2024-09-19T04:19:23.579 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf05ffb640 1 -- 172.21.15.76:0/3317661631 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbf0c0adcb0 con 0x7fbf141587a0 2024-09-19T04:19:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf05ffb640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbef80678c0 0x7fbef8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf05ffb640 1 -- 172.21.15.76:0/3317661631 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fbf0c1212f0 con 0x7fbf141587a0 2024-09-19T04:19:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1944f640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbef80678c0 0x7fbef8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:23.580 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.576+0000 7fbf1944f640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbef80678c0 0x7fbef8069d80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf100096f0 tx=0x7fbf10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:23.583 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.580+0000 7fbf05ffb640 1 -- 172.21.15.76:0/3317661631 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbf0c016520 con 0x7fbf141587a0 2024-09-19T04:19:23.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.800+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbf140aebe0 con 0x7fbf141587a0 2024-09-19T04:19:23.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.800+0000 7fbf05ffb640 1 -- 172.21.15.76:0/3317661631 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2008 (secure 0 0 0) 0x7fbf0c0f1c40 con 0x7fbf141587a0 2024-09-19T04:19:23.802 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:23.802 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":382,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":695469,"bytes_used":261898240,"bytes_avail":772832215040,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6483,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-09-19T04:19:19.851943+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.800+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbef80678c0 msgr2=0x7fbef8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.800+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbef80678c0 0x7fbef8069d80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbf100096f0 tx=0x7fbf10008040 comp rx=0 tx=0).stop 2024-09-19T04:19:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.800+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 msgr2=0x7fbf140b0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.800+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf140b0600 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c019510 tx=0x7fbf0c0196a0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 shutdown_connections 2024-09-19T04:19:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbef80678c0 0x7fbef8069d80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbf141587a0 0x7fbf140b0600 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbf140b37d0 0x7fbf1409df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 --2- 172.21.15.76:0/3317661631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf1406e680 0x7fbf14154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:23.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 >> 172.21.15.76:0/3317661631 conn(0x7fbf140b1f90 msgr2=0x7fbf14161550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:23.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 shutdown_connections 2024-09-19T04:19:23.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:23.804+0000 7fbf1b6b3640 1 -- 172.21.15.76:0/3317661631 wait complete. 2024-09-19T04:19:23.819 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'b': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:19:24.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:19:24.104 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:19:24.820 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:24.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 Processor -- start 2024-09-19T04:19:24.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 -- start start 2024-09-19T04:19:24.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3fc815bec0 0x7f3fc80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815c2a0 0x7f3fc80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3fc80af2a0 0x7f3fc815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc80b3290 con 0x7f3fc815bec0 2024-09-19T04:19:24.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3fc80b3570 con 0x7f3fc80af2a0 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fccf86640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc80b3850 con 0x7f3fc815c2a0 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3fc815bec0 0x7f3fc80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3fc815bec0 0x7f3fc80ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34546/0 (socket says 172.21.15.76:34546) 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 -- 172.21.15.76:0/797936394 learned_addr learned my addr 172.21.15.76:0/797936394 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 -- 172.21.15.76:0/797936394 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815c2a0 msgr2=0x7f3fc80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 --2- 172.21.15.76:0/797936394 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815c2a0 0x7f3fc80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 -- 172.21.15.76:0/797936394 >> v2:172.21.15.116:3300/0 conn(0x7f3fc80af2a0 msgr2=0x7f3fc815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 --2- 172.21.15.76:0/797936394 >> v2:172.21.15.116:3300/0 conn(0x7f3fc80af2a0 0x7f3fc815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 -- 172.21.15.76:0/797936394 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc80b3b30 con 0x7f3fc815bec0 2024-09-19T04:19:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.948+0000 7f3fc659c640 1 --2- 172.21.15.76:0/797936394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815bec0 0x7f3fc80ad200 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3fc806dc90 tx=0x7f3fb801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a48ad85ca0be4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc559a640 1 -- 172.21.15.76:0/797936394 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80b0070 con 0x7f3fc815bec0 2024-09-19T04:19:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc559a640 1 -- 172.21.15.76:0/797936394 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fb80ab600 con 0x7f3fc815bec0 2024-09-19T04:19:24.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc559a640 1 -- 172.21.15.76:0/797936394 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80ab9d0 con 0x7f3fc815bec0 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- 172.21.15.76:0/797936394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815bec0 msgr2=0x7f3fc80ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 --2- 172.21.15.76:0/797936394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815bec0 0x7f3fc80ad200 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f3fc806dc90 tx=0x7f3fb801c230 comp rx=0 tx=0).stop 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- 172.21.15.76:0/797936394 shutdown_connections 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 --2- 172.21.15.76:0/797936394 >> v2:172.21.15.116:3300/0 conn(0x7f3fc80af2a0 0x7f3fc815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 --2- 172.21.15.76:0/797936394 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815c2a0 0x7f3fc80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 --2- 172.21.15.76:0/797936394 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815bec0 0x7f3fc80ad200 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- 172.21.15.76:0/797936394 >> 172.21.15.76:0/797936394 conn(0x7f3fc80b1f90 msgr2=0x7f3fc80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- 172.21.15.76:0/797936394 shutdown_connections 2024-09-19T04:19:24.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- 172.21.15.76:0/797936394 wait complete. 2024-09-19T04:19:24.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 Processor -- start 2024-09-19T04:19:24.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- start start 2024-09-19T04:19:24.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fc80af2a0 0x7f3fc8156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815bec0 0x7f3fc814eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815c2a0 0x7f3fc815e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc80ad740 con 0x7f3fc815c2a0 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc80af710 con 0x7f3fc80af2a0 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3fc806e000 con 0x7f3fc815bec0 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815c2a0 0x7f3fc815e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815c2a0 0x7f3fc815e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34562/0 (socket says 172.21.15.76:34562) 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 -- 172.21.15.76:0/1600805499 learned_addr learned my addr 172.21.15.76:0/1600805499 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc5d9b640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815bec0 0x7f3fc814eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 -- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815bec0 msgr2=0x7f3fc814eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc659c640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fc80af2a0 0x7f3fc8156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815bec0 0x7f3fc814eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 -- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fc80af2a0 msgr2=0x7f3fc8156d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fc80af2a0 0x7f3fc8156d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 -- 172.21.15.76:0/1600805499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc8157280 con 0x7f3fc815c2a0 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc5d9b640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815bec0 0x7f3fc814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc659c640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fc80af2a0 0x7f3fc8156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fc6d9d640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815c2a0 0x7f3fc815e8e0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000b970 tx=0x7f3fc000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fb6ffd640 1 -- 172.21.15.76:0/1600805499 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0019070 con 0x7f3fc815c2a0 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fb6ffd640 1 -- 172.21.15.76:0/1600805499 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fc000cb60 con 0x7f3fc815c2a0 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fb6ffd640 1 -- 172.21.15.76:0/1600805499 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0014440 con 0x7f3fc815c2a0 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc8159a40 con 0x7f3fc815c2a0 2024-09-19T04:19:24.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.952+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc815a000 con 0x7f3fc815c2a0 2024-09-19T04:19:24.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.956+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc80ad200 con 0x7f3fc815c2a0 2024-09-19T04:19:24.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.956+0000 7f3fb6ffd640 1 -- 172.21.15.76:0/1600805499 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3fc001c070 con 0x7f3fc815c2a0 2024-09-19T04:19:24.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.956+0000 7f3fb6ffd640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fa80678c0 0x7f3fa8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:24.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.956+0000 7f3fb6ffd640 1 -- 172.21.15.76:0/1600805499 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f3fc0085bd0 con 0x7f3fc815c2a0 2024-09-19T04:19:24.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.956+0000 7f3fc659c640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fa80678c0 0x7f3fa8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:24.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.956+0000 7f3fc659c640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fa80678c0 0x7f3fa8069d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80ad000 tx=0x7f3fb8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:24.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:24.960+0000 7f3fb6ffd640 1 -- 172.21.15.76:0/1600805499 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3fc0056510 con 0x7f3fc815c2a0 2024-09-19T04:19:25.148 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fc80cf670 con 0x7f3fc815c2a0 2024-09-19T04:19:25.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fb6ffd640 1 -- 172.21.15.76:0/1600805499 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7f3fc0055e10 con 0x7f3fc815c2a0 2024-09-19T04:19:25.149 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:25.149 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:25.149 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:25.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fa80678c0 msgr2=0x7f3fa8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fa80678c0 0x7f3fa8069d80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3fb80ad000 tx=0x7f3fb8008040 comp rx=0 tx=0).stop 2024-09-19T04:19:25.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815c2a0 msgr2=0x7f3fc815e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815c2a0 0x7f3fc815e8e0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000b970 tx=0x7f3fc000be30 comp rx=0 tx=0).stop 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 shutdown_connections 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3fa80678c0 0x7f3fa8069d80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3fc815c2a0 0x7f3fc815e8e0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3fc815bec0 0x7f3fc814eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 --2- 172.21.15.76:0/1600805499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3fc80af2a0 0x7f3fc8156d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 >> 172.21.15.76:0/1600805499 conn(0x7f3fc80b1f90 msgr2=0x7f3fc80cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 shutdown_connections 2024-09-19T04:19:25.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.148+0000 7f3fccf86640 1 -- 172.21.15.76:0/1600805499 wait complete. 2024-09-19T04:19:25.167 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:25.167 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:25.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.280+0000 7f763f86a640 1 Processor -- start 2024-09-19T04:19:25.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.280+0000 7f763f86a640 1 -- start start 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763f86a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 0x7f76380b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763f86a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f76380b37d0 0x7f763806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763f86a640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f763806ec60 con 0x7f763806e4b0 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763f86a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f763806f870 con 0x7f76380b37d0 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763f86a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f76380b3c10 con 0x7f76381587a0 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763de07640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f763806e4b0 0x7f7638161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50862/0 (socket says 172.21.15.76:50862) 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763de07640 1 -- 172.21.15.76:0/3790872363 learned_addr learned my addr 172.21.15.76:0/3790872363 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:25.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763d606640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 0x7f76380b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3350935322 0 0) 0x7f763806ec60 con 0x7f763806e4b0 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 --2- 172.21.15.76:0/3790872363 >> v2:172.21.15.116:3300/0 conn(0x7f76380b37d0 0x7f763806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7620003530 con 0x7f763806e4b0 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1317857301 0 0) 0x7f7620003530 con 0x7f763806e4b0 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 msgr2=0x7f76380b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 0x7f76380b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 >> v2:172.21.15.116:3300/0 conn(0x7f76380b37d0 msgr2=0x7f763806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 --2- 172.21.15.76:0/3790872363 >> v2:172.21.15.116:3300/0 conn(0x7f76380b37d0 0x7f763806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763806ef40 con 0x7f763806e4b0 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f7628001680 con 0x7f763806e4b0 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 --2- 172.21.15.76:0/3790872363 >> v2:172.21.15.116:3300/0 conn(0x7f76380b37d0 0x7f763806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 >> v1:172.21.15.76:6789/0 conn(0x7f763806e4b0 legacy=0x7f7638161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 0x7f76200039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7620003f40 0x7f7620004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620004d50 0x7f76200051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f763806f870 con 0x7f7620003f40 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f76380b3c10 con 0x7f7620004d50 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f762000ba20 con 0x7f76381587a0 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7620003f40 0x7f7620004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763d606640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 0x7f76200039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 -- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 msgr2=0x7f76200039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381587a0 0x7f76200039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 -- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620004d50 msgr2=0x7f76200051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620004d50 0x7f76200051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.288 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 -- 172.21.15.76:0/3790872363 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7620003530 con 0x7f7620003f40 2024-09-19T04:19:25.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.284+0000 7f763ce05640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7620003f40 0x7f7620004810 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f76300073e0 tx=0x7f763001cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13028ad9cbff1b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:25.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300b2070 con 0x7f7620003f40 2024-09-19T04:19:25.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76300a5e40 con 0x7f7620003f40 2024-09-19T04:19:25.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f7637fff640 1 -- 172.21.15.76:0/3790872363 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7630009c80 con 0x7f7620003f40 2024-09-19T04:19:25.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- 172.21.15.76:0/3790872363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7620003f40 msgr2=0x7f7620004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 --2- 172.21.15.76:0/3790872363 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7620003f40 0x7f7620004810 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f76300073e0 tx=0x7f763001cf40 comp rx=0 tx=0).stop 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763de07640 1 -- 172.21.15.76:0/3790872363 reap_dead start 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- 172.21.15.76:0/3790872363 shutdown_connections 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- 172.21.15.76:0/3790872363 >> 172.21.15.76:0/3790872363 conn(0x7f76380b1f90 msgr2=0x7f76380b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- 172.21.15.76:0/3790872363 shutdown_connections 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- 172.21.15.76:0/3790872363 wait complete. 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 Processor -- start 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- start start 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620003f40 0x7f763809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7638157930 0x7f7638142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381430d0 0x7f763815ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f763806e920 con 0x7f7638157930 2024-09-19T04:19:25.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f763806f500 con 0x7f7620003f40 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7638158b80 con 0x7f76381430d0 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7638157930 0x7f7638142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7638157930 0x7f7638142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34568/0 (socket says 172.21.15.76:34568) 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 -- 172.21.15.76:0/333239649 learned_addr learned my addr 172.21.15.76:0/333239649 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763de07640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381430d0 0x7f763815ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763d606640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620003f40 0x7f763809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 -- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381430d0 msgr2=0x7f763815ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381430d0 0x7f763815ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 -- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620003f40 msgr2=0x7f763809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620003f40 0x7f763809d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 -- 172.21.15.76:0/333239649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f763809c740 con 0x7f7638157930 2024-09-19T04:19:25.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763de07640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381430d0 0x7f763815ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:25.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763ce05640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7638157930 0x7f7638142b90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f763001c9b0 tx=0x7f7620005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:25.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f7635ffb640 1 -- 172.21.15.76:0/333239649 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300bc070 con 0x7f7638157930 2024-09-19T04:19:25.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f7635ffb640 1 -- 172.21.15.76:0/333239649 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76300adae0 con 0x7f7638157930 2024-09-19T04:19:25.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f7635ffb640 1 -- 172.21.15.76:0/333239649 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300b2040 con 0x7f7638157930 2024-09-19T04:19:25.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f763809ca20 con 0x7f7638157930 2024-09-19T04:19:25.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.288+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f763809cfe0 con 0x7f7638157930 2024-09-19T04:19:25.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.292+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76381434b0 con 0x7f7638157930 2024-09-19T04:19:25.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.292+0000 7f7635ffb640 1 -- 172.21.15.76:0/333239649 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f76300bf070 con 0x7f7638157930 2024-09-19T04:19:25.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.292+0000 7f7635ffb640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f76240678f0 0x7f7624069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:25.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.292+0000 7f7635ffb640 1 -- 172.21.15.76:0/333239649 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f7630120240 con 0x7f7638157930 2024-09-19T04:19:25.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.292+0000 7f763d606640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f76240678f0 0x7f7624069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:25.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.292+0000 7f763d606640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f76240678f0 0x7f7624069db0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f76381479e0 tx=0x7f762c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:25.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.296+0000 7f7635ffb640 1 -- 172.21.15.76:0/333239649 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7630016c50 con 0x7f7638157930 2024-09-19T04:19:25.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.520+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f7638070080 con 0x7f7638157930 2024-09-19T04:19:25.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.520+0000 7f7635ffb640 1 -- 172.21.15.76:0/333239649 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2026 (secure 0 0 0) 0x7f76300f0b80 con 0x7f7638157930 2024-09-19T04:19:25.523 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:25.524 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":384,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700801,"bytes_used":261906432,"bytes_avail":772832206848,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7165,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-09-19T04:19:23.853477+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f76240678f0 msgr2=0x7f7624069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f76240678f0 0x7f7624069db0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f76381479e0 tx=0x7f762c008040 comp rx=0 tx=0).stop 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7638157930 msgr2=0x7f7638142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7638157930 0x7f7638142b90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f763001c9b0 tx=0x7f7620005840 comp rx=0 tx=0).stop 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 shutdown_connections 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f76240678f0 0x7f7624069db0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f76381430d0 0x7f763815ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7638157930 0x7f7638142b90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 --2- 172.21.15.76:0/333239649 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7620003f40 0x7f763809d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:25.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 >> 172.21.15.76:0/333239649 conn(0x7f76380b1f90 msgr2=0x7f76380aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:25.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 shutdown_connections 2024-09-19T04:19:25.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:25.524+0000 7f763f86a640 1 -- 172.21.15.76:0/333239649 wait complete. 2024-09-19T04:19:25.542 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-19T04:19:25.784 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:19:25.784+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:19:26.543 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:26.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff925333640 1 Processor -- start 2024-09-19T04:19:26.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff925333640 1 -- start start 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff925333640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff92006df70 0x7ff9201634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff925333640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff9200b3290 0x7ff9200b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff925333640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff92006e470 con 0x7ff9201630c0 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff925333640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff92006fc20 con 0x7ff92006df70 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff925333640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff92006e750 con 0x7ff9200b3290 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff92006df70 0x7ff9201634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff92006df70 0x7ff9201634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34584/0 (socket says 172.21.15.76:34584) 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 -- 172.21.15.76:0/855180922 learned_addr learned my addr 172.21.15.76:0/855180922 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91effd640 1 --1- 172.21.15.76:0/855180922 >> v1:172.21.15.76:6789/0 conn(0x7ff9201630c0 0x7ff92006f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50876/0 (socket says 172.21.15.76:50876) 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 -- 172.21.15.76:0/855180922 >> v2:172.21.15.116:3300/0 conn(0x7ff9200b3290 msgr2=0x7ff9200b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91f7fe640 1 --2- 172.21.15.76:0/855180922 >> v2:172.21.15.116:3300/0 conn(0x7ff9200b3290 0x7ff9200b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 --2- 172.21.15.76:0/855180922 >> v2:172.21.15.116:3300/0 conn(0x7ff9200b3290 0x7ff9200b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 -- 172.21.15.76:0/855180922 >> v1:172.21.15.76:6789/0 conn(0x7ff9201630c0 legacy=0x7ff92006f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91dffb640 1 -- 172.21.15.76:0/855180922 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2759749101 0 0) 0x7ff92006e470 con 0x7ff9201630c0 2024-09-19T04:19:26.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 -- 172.21.15.76:0/855180922 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff92006ea30 con 0x7ff92006df70 2024-09-19T04:19:26.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91f7fe640 1 --2- 172.21.15.76:0/855180922 >> v2:172.21.15.116:3300/0 conn(0x7ff9200b3290 0x7ff9200b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:26.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.672+0000 7ff91e7fc640 1 --2- 172.21.15.76:0/855180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff9201634a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff9200af090 tx=0x7ff91401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4832d64462584df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:26.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91dffb640 1 -- 172.21.15.76:0/855180922 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140b0070 con 0x7ff92006df70 2024-09-19T04:19:26.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91dffb640 1 -- 172.21.15.76:0/855180922 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9140ab600 con 0x7ff92006df70 2024-09-19T04:19:26.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91dffb640 1 -- 172.21.15.76:0/855180922 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9140ab9d0 con 0x7ff92006df70 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- 172.21.15.76:0/855180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 msgr2=0x7ff9201634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 --2- 172.21.15.76:0/855180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff9201634a0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff9200af090 tx=0x7ff91401c520 comp rx=0 tx=0).stop 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- 172.21.15.76:0/855180922 shutdown_connections 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 --2- 172.21.15.76:0/855180922 >> v2:172.21.15.116:3300/0 conn(0x7ff9200b3290 0x7ff9200b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 --2- 172.21.15.76:0/855180922 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff9201634a0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- 172.21.15.76:0/855180922 >> 172.21.15.76:0/855180922 conn(0x7ff9200b1f90 msgr2=0x7ff9200b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- 172.21.15.76:0/855180922 shutdown_connections 2024-09-19T04:19:26.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- 172.21.15.76:0/855180922 wait complete. 2024-09-19T04:19:26.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 Processor -- start 2024-09-19T04:19:26.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- start start 2024-09-19T04:19:26.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff920095260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9200b3290 0x7ff92009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9201630c0 0x7ff92009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff92016f770 con 0x7ff92006df70 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff92016f5f0 con 0x7ff9200b3290 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff92016f8f0 con 0x7ff9201630c0 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff920095260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff920095260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34588/0 (socket says 172.21.15.76:34588) 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 -- 172.21.15.76:0/745714254 learned_addr learned my addr 172.21.15.76:0/745714254 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 -- 172.21.15.76:0/745714254 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9201630c0 msgr2=0x7ff92009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9201630c0 0x7ff92009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 -- 172.21.15.76:0/745714254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9200b3290 msgr2=0x7ff92009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9200b3290 0x7ff92009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 -- 172.21.15.76:0/745714254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff920158490 con 0x7ff92006df70 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff91effd640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff920095260 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7ff91000c480 tx=0x7ff91000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff9077fe640 1 -- 172.21.15.76:0/745714254 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910018070 con 0x7ff92006df70 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff9077fe640 1 -- 172.21.15.76:0/745714254 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff910007500 con 0x7ff92006df70 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff920158770 con 0x7ff92006df70 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff9077fe640 1 -- 172.21.15.76:0/745714254 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff910004a30 con 0x7ff92006df70 2024-09-19T04:19:26.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.676+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff920158d30 con 0x7ff92006df70 2024-09-19T04:19:26.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.680+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9201634a0 con 0x7ff92006df70 2024-09-19T04:19:26.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.680+0000 7ff9077fe640 1 -- 172.21.15.76:0/745714254 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff91001b070 con 0x7ff92006df70 2024-09-19T04:19:26.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.680+0000 7ff9077fe640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff90c0678c0 0x7ff90c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:26.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.680+0000 7ff9077fe640 1 -- 172.21.15.76:0/745714254 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff910085190 con 0x7ff92006df70 2024-09-19T04:19:26.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.680+0000 7ff91e7fc640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff90c0678c0 0x7ff90c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:26.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.680+0000 7ff91e7fc640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff90c0678c0 0x7ff90c069d80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ad000 tx=0x7ff914008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:26.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.684+0000 7ff9077fe640 1 -- 172.21.15.76:0/745714254 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff910055b40 con 0x7ff92006df70 2024-09-19T04:19:26.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.876+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff920070080 con 0x7ff92006df70 2024-09-19T04:19:26.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff9077fe640 1 -- 172.21.15.76:0/745714254 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7ff910055440 con 0x7ff92006df70 2024-09-19T04:19:26.880 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:26.880 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:26.881 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:26.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff90c0678c0 msgr2=0x7ff90c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff90c0678c0 0x7ff90c069d80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ad000 tx=0x7ff914008040 comp rx=0 tx=0).stop 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 msgr2=0x7ff920095260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff920095260 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7ff91000c480 tx=0x7ff91000c940 comp rx=0 tx=0).stop 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 shutdown_connections 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff90c0678c0 0x7ff90c069d80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9201630c0 0x7ff92009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9200b3290 0x7ff92009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 --2- 172.21.15.76:0/745714254 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff92006df70 0x7ff920095260 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 >> 172.21.15.76:0/745714254 conn(0x7ff9200b1f90 msgr2=0x7ff9201651c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 shutdown_connections 2024-09-19T04:19:26.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:26.880+0000 7ff925333640 1 -- 172.21.15.76:0/745714254 wait complete. 2024-09-19T04:19:26.897 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:26.898 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:27.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f90900640 1 Processor -- start 2024-09-19T04:19:27.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f90900640 1 -- start start 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f90900640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f8c15bec0 0x7f5f8c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f90900640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0af470 0x7f5f8c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f90900640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f8c0b3290 con 0x7f5f8c15c2a0 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f90900640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5f8c0b3570 con 0x7f5f8c15bec0 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f90900640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8c0b3850 con 0x7f5f8c0af470 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f89d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5f8c15c2a0 0x7f5f8c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50878/0 (socket says 172.21.15.76:50878) 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f89d9b640 1 -- 172.21.15.76:0/4181004636 learned_addr learned my addr 172.21.15.76:0/4181004636 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.000+0000 7f5f8a59c640 1 --2- 172.21.15.76:0/4181004636 >> v2:172.21.15.76:3300/0 conn(0x7f5f8c15bec0 0x7f5f8c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:27.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8ad9d640 1 --2- 172.21.15.76:0/4181004636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0af470 0x7f5f8c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8a59c640 1 -- 172.21.15.76:0/4181004636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0af470 msgr2=0x7f5f8c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8a59c640 1 --2- 172.21.15.76:0/4181004636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0af470 0x7f5f8c06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8a59c640 1 -- 172.21.15.76:0/4181004636 >> v1:172.21.15.76:6789/0 conn(0x7f5f8c15c2a0 legacy=0x7f5f8c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8a59c640 1 -- 172.21.15.76:0/4181004636 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c0b3b30 con 0x7f5f8c15bec0 2024-09-19T04:19:27.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8959a640 1 -- 172.21.15.76:0/4181004636 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 370497843 0 0) 0x7f5f8c0b3290 con 0x7f5f8c15c2a0 2024-09-19T04:19:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8a59c640 1 --2- 172.21.15.76:0/4181004636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c0ad200 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0097c0 tx=0x7f5f7c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3dd3e4d74f37e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8959a640 1 -- 172.21.15.76:0/4181004636 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0b1070 con 0x7f5f8c15bec0 2024-09-19T04:19:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8959a640 1 -- 172.21.15.76:0/4181004636 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f7c0a8050 con 0x7f5f8c15bec0 2024-09-19T04:19:27.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f8959a640 1 -- 172.21.15.76:0/4181004636 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f7c0ac860 con 0x7f5f8c15bec0 2024-09-19T04:19:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 -- 172.21.15.76:0/4181004636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 msgr2=0x7f5f8c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 --2- 172.21.15.76:0/4181004636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c0ad200 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0097c0 tx=0x7f5f7c01c3e0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 -- 172.21.15.76:0/4181004636 shutdown_connections 2024-09-19T04:19:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 --2- 172.21.15.76:0/4181004636 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c0af470 0x7f5f8c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 --2- 172.21.15.76:0/4181004636 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c0ad200 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 -- 172.21.15.76:0/4181004636 >> 172.21.15.76:0/4181004636 conn(0x7f5f8c0b1f90 msgr2=0x7f5f8c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:27.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 -- 172.21.15.76:0/4181004636 shutdown_connections 2024-09-19T04:19:27.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.004+0000 7f5f90900640 1 -- 172.21.15.76:0/4181004636 wait complete. 2024-09-19T04:19:27.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 Processor -- start 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 -- start start 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c0af470 0x7f5f8c1fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c15c2a0 0x7f5f8c1f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c06ed50 con 0x7f5f8c15bec0 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5f8c0ad740 con 0x7f5f8c0af470 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8c06eed0 con 0x7f5f8c15c2a0 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c1571d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34608/0 (socket says 172.21.15.76:34608) 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 -- 172.21.15.76:0/4282130648 learned_addr learned my addr 172.21.15.76:0/4282130648 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:27.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 -- 172.21.15.76:0/4282130648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c15c2a0 msgr2=0x7f5f8c1f6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f8a59c640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c0af470 0x7f5f8c1fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c15c2a0 0x7f5f8c1f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 -- 172.21.15.76:0/4282130648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c0af470 msgr2=0x7f5f8c1fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c0af470 0x7f5f8c1fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 -- 172.21.15.76:0/4282130648 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f8c073df0 con 0x7f5f8c15bec0 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f89d9b640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c1571d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400c450 tx=0x7f5f7400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f82ffd640 1 -- 172.21.15.76:0/4282130648 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f74018070 con 0x7f5f8c15bec0 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f82ffd640 1 -- 172.21.15.76:0/4282130648 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f740074e0 con 0x7f5f8c15bec0 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f8c1fa120 con 0x7f5f8c15bec0 2024-09-19T04:19:27.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f82ffd640 1 -- 172.21.15.76:0/4282130648 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f740049f0 con 0x7f5f8c15bec0 2024-09-19T04:19:27.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f8c1fa6e0 con 0x7f5f8c15bec0 2024-09-19T04:19:27.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f8c1f76f0 con 0x7f5f8c15bec0 2024-09-19T04:19:27.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f82ffd640 1 -- 172.21.15.76:0/4282130648 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5f7401b070 con 0x7f5f8c15bec0 2024-09-19T04:19:27.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f82ffd640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f6c0678c0 0x7f5f6c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:27.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f82ffd640 1 -- 172.21.15.76:0/4282130648 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f5f74085230 con 0x7f5f8c15bec0 2024-09-19T04:19:27.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.008+0000 7f5f8a59c640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f6c0678c0 0x7f5f6c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:27.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.012+0000 7f5f8a59c640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f6c0678c0 0x7f5f6c069d80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0ae000 tx=0x7f5f7c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:27.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.012+0000 7f5f82ffd640 1 -- 172.21.15.76:0/4282130648 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5f8c1f76f0 con 0x7f5f8c15bec0 2024-09-19T04:19:27.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.220+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f5f8c06e000 con 0x7f5f8c15bec0 2024-09-19T04:19:27.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.220+0000 7f5f82ffd640 1 -- 172.21.15.76:0/4282130648 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2006 (secure 0 0 0) 0x7f5f8c073df0 con 0x7f5f8c15bec0 2024-09-19T04:19:27.223 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:27.223 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":386,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700801,"bytes_used":261906432,"bytes_avail":772832206848,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-09-19T04:19:23.853477+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:27.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f6c0678c0 msgr2=0x7f5f6c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:27.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f6c0678c0 0x7f5f6c069d80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c0ae000 tx=0x7f5f7c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 msgr2=0x7f5f8c1571d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c1571d0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5f7400c450 tx=0x7f5f7400c910 comp rx=0 tx=0).stop 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 shutdown_connections 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5f6c0678c0 0x7f5f6c069d80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f8c15c2a0 0x7f5f8c1f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f8c15bec0 0x7f5f8c1571d0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 --2- 172.21.15.76:0/4282130648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f8c0af470 0x7f5f8c1fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 >> 172.21.15.76:0/4282130648 conn(0x7f5f8c0b1f90 msgr2=0x7f5f8c159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 shutdown_connections 2024-09-19T04:19:27.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:27.224+0000 7f5f90900640 1 -- 172.21.15.76:0/4282130648 wait complete. 2024-09-19T04:19:27.241 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-19T04:19:28.241 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:28.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 Processor -- start 2024-09-19T04:19:28.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 -- start start 2024-09-19T04:19:28.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fde0815bec0 0x7fde080ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815c2a0 0x7fde080aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fde080af2a0 0x7fde0815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fde080b3290 con 0x7fde0815bec0 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fde080b3570 con 0x7fde080af2a0 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0fd38640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fde080b3850 con 0x7fde0815c2a0 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fde0815bec0 0x7fde080ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fde0815bec0 0x7fde080ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34620/0 (socket says 172.21.15.76:34620) 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 -- 172.21.15.76:0/1087650508 learned_addr learned my addr 172.21.15.76:0/1087650508 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0e2d5640 1 --2- 172.21.15.76:0/1087650508 >> v2:172.21.15.116:3300/0 conn(0x7fde080af2a0 0x7fde0815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0d2d3640 1 --2- 172.21.15.76:0/1087650508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815c2a0 0x7fde080aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 -- 172.21.15.76:0/1087650508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815c2a0 msgr2=0x7fde080aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:28.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 --2- 172.21.15.76:0/1087650508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815c2a0 0x7fde080aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 -- 172.21.15.76:0/1087650508 >> v2:172.21.15.116:3300/0 conn(0x7fde080af2a0 msgr2=0x7fde0815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:28.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 --2- 172.21.15.76:0/1087650508 >> v2:172.21.15.116:3300/0 conn(0x7fde080af2a0 0x7fde0815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.375 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.372+0000 7fde0dad4640 1 -- 172.21.15.76:0/1087650508 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde080b3b30 con 0x7fde0815bec0 2024-09-19T04:19:28.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0dad4640 1 --2- 172.21.15.76:0/1087650508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815bec0 0x7fde080ad200 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fde0806dc90 tx=0x7fde0401c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63e1db19729c5a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0cad2640 1 -- 172.21.15.76:0/1087650508 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde040b0070 con 0x7fde0815bec0 2024-09-19T04:19:28.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0cad2640 1 -- 172.21.15.76:0/1087650508 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde040ab600 con 0x7fde0815bec0 2024-09-19T04:19:28.376 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0cad2640 1 -- 172.21.15.76:0/1087650508 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde040ab9d0 con 0x7fde0815bec0 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- 172.21.15.76:0/1087650508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815bec0 msgr2=0x7fde080ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 --2- 172.21.15.76:0/1087650508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815bec0 0x7fde080ad200 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fde0806dc90 tx=0x7fde0401c230 comp rx=0 tx=0).stop 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- 172.21.15.76:0/1087650508 shutdown_connections 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 --2- 172.21.15.76:0/1087650508 >> v2:172.21.15.116:3300/0 conn(0x7fde080af2a0 0x7fde0815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 --2- 172.21.15.76:0/1087650508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815c2a0 0x7fde080aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 --2- 172.21.15.76:0/1087650508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815bec0 0x7fde080ad200 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- 172.21.15.76:0/1087650508 >> 172.21.15.76:0/1087650508 conn(0x7fde080b1f90 msgr2=0x7fde080b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:28.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- 172.21.15.76:0/1087650508 shutdown_connections 2024-09-19T04:19:28.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- 172.21.15.76:0/1087650508 wait complete. 2024-09-19T04:19:28.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 Processor -- start 2024-09-19T04:19:28.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- start start 2024-09-19T04:19:28.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde080af2a0 0x7fde08071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815bec0 0x7fde080b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815c2a0 0x7fde08152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fde080af6b0 con 0x7fde0815c2a0 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fde080ad740 con 0x7fde080af2a0 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fde0806e000 con 0x7fde0815bec0 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815c2a0 0x7fde08152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815c2a0 0x7fde08152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34632/0 (socket says 172.21.15.76:34632) 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 -- 172.21.15.76:0/2049680917 learned_addr learned my addr 172.21.15.76:0/2049680917 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0dad4640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde080af2a0 0x7fde08071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0d2d3640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815bec0 0x7fde080b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 -- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815bec0 msgr2=0x7fde080b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815bec0 0x7fde080b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 -- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde080af2a0 msgr2=0x7fde08071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde080af2a0 0x7fde08071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 -- 172.21.15.76:0/2049680917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde08072610 con 0x7fde0815c2a0 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0d2d3640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815bec0 0x7fde080b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0e2d5640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815c2a0 0x7fde08152c30 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fde00007e40 tx=0x7fde0000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fddfe7fc640 1 -- 172.21.15.76:0/2049680917 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00014620 con 0x7fde0815c2a0 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fddfe7fc640 1 -- 172.21.15.76:0/2049680917 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde00019070 con 0x7fde0815c2a0 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde080728f0 con 0x7fde0815c2a0 2024-09-19T04:19:28.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fddfe7fc640 1 -- 172.21.15.76:0/2049680917 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00014a20 con 0x7fde0815c2a0 2024-09-19T04:19:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.376+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde08073190 con 0x7fde0815c2a0 2024-09-19T04:19:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.380+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde08072ce0 con 0x7fde0815c2a0 2024-09-19T04:19:28.383 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.380+0000 7fddfe7fc640 1 -- 172.21.15.76:0/2049680917 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fde00014bc0 con 0x7fde0815c2a0 2024-09-19T04:19:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.380+0000 7fddfe7fc640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddec0678c0 0x7fddec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.380+0000 7fddfe7fc640 1 -- 172.21.15.76:0/2049680917 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fde00086770 con 0x7fde0815c2a0 2024-09-19T04:19:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.380+0000 7fde0dad4640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddec0678c0 0x7fddec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.380+0000 7fde0dad4640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddec0678c0 0x7fddec069d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fde040ad000 tx=0x7fde040a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.384+0000 7fddfe7fc640 1 -- 172.21.15.76:0/2049680917 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fde00057140 con 0x7fde0815c2a0 2024-09-19T04:19:28.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.564+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde080ad200 con 0x7fde0815c2a0 2024-09-19T04:19:28.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.564+0000 7fddfe7fc640 1 -- 172.21.15.76:0/2049680917 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7fde00056a40 con 0x7fde0815c2a0 2024-09-19T04:19:28.567 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:28.567 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:28.567 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddec0678c0 msgr2=0x7fddec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddec0678c0 0x7fddec069d80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fde040ad000 tx=0x7fde040a6000 comp rx=0 tx=0).stop 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815c2a0 msgr2=0x7fde08152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815c2a0 0x7fde08152c30 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fde00007e40 tx=0x7fde0000e4b0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 shutdown_connections 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddec0678c0 0x7fddec069d80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fde0815c2a0 0x7fde08152c30 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fde0815bec0 0x7fde080b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 --2- 172.21.15.76:0/2049680917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fde080af2a0 0x7fde08071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 >> 172.21.15.76:0/2049680917 conn(0x7fde080b1f90 msgr2=0x7fde08165080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:28.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 shutdown_connections 2024-09-19T04:19:28.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.568+0000 7fde0fd38640 1 -- 172.21.15.76:0/2049680917 wait complete. 2024-09-19T04:19:28.584 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:28.584 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:28.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 Processor -- start 2024-09-19T04:19:28.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 -- start start 2024-09-19T04:19:28.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d1815bec0 0x7f6d180ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d1815c2a0 0x7f6d180aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d180af2a0 0x7f6d1815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6d180b3290 con 0x7f6d1815c2a0 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6d180b3570 con 0x7f6d1815bec0 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d1d304640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6d180b3850 con 0x7f6d180af2a0 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d1815c2a0 0x7f6d180aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d1815c2a0 0x7f6d180aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34642/0 (socket says 172.21.15.76:34642) 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 -- 172.21.15.76:0/2056209957 learned_addr learned my addr 172.21.15.76:0/2056209957 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d177fe640 1 --2- 172.21.15.76:0/2056209957 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d180af2a0 0x7f6d1815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d16ffd640 1 --2- 172.21.15.76:0/2056209957 >> v2:172.21.15.116:3300/0 conn(0x7f6d1815bec0 0x7f6d180ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 -- 172.21.15.76:0/2056209957 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d180af2a0 msgr2=0x7f6d1815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 --2- 172.21.15.76:0/2056209957 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d180af2a0 0x7f6d1815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 -- 172.21.15.76:0/2056209957 >> v2:172.21.15.116:3300/0 conn(0x7f6d1815bec0 msgr2=0x7f6d180ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:28.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 --2- 172.21.15.76:0/2056209957 >> v2:172.21.15.116:3300/0 conn(0x7f6d1815bec0 0x7f6d180ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d167fc640 1 -- 172.21.15.76:0/2056209957 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d180b3b30 con 0x7f6d1815c2a0 2024-09-19T04:19:28.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.688+0000 7f6d16ffd640 1 --2- 172.21.15.76:0/2056209957 >> v2:172.21.15.116:3300/0 conn(0x7f6d1815bec0 0x7f6d180ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:28.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d167fc640 1 --2- 172.21.15.76:0/2056209957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d1815c2a0 0x7f6d180aed60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f6d18071710 tx=0x7f6d0801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b682f0a1c6b82b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d15ffb640 1 -- 172.21.15.76:0/2056209957 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d080b0070 con 0x7f6d1815c2a0 2024-09-19T04:19:28.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d15ffb640 1 -- 172.21.15.76:0/2056209957 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d080ab600 con 0x7f6d1815c2a0 2024-09-19T04:19:28.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d15ffb640 1 -- 172.21.15.76:0/2056209957 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d080ab9d0 con 0x7f6d1815c2a0 2024-09-19T04:19:28.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 -- 172.21.15.76:0/2056209957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d1815c2a0 msgr2=0x7f6d180aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 --2- 172.21.15.76:0/2056209957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d1815c2a0 0x7f6d180aed60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f6d18071710 tx=0x7f6d0801c230 comp rx=0 tx=0).stop 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 -- 172.21.15.76:0/2056209957 shutdown_connections 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 --2- 172.21.15.76:0/2056209957 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d180af2a0 0x7f6d1815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 --2- 172.21.15.76:0/2056209957 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d1815c2a0 0x7f6d180aed60 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 --2- 172.21.15.76:0/2056209957 >> v2:172.21.15.116:3300/0 conn(0x7f6d1815bec0 0x7f6d180ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 -- 172.21.15.76:0/2056209957 >> 172.21.15.76:0/2056209957 conn(0x7f6d180b1f90 msgr2=0x7f6d180b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 -- 172.21.15.76:0/2056209957 shutdown_connections 2024-09-19T04:19:28.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 -- 172.21.15.76:0/2056209957 wait complete. 2024-09-19T04:19:28.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.692+0000 7f6d1d304640 1 Processor -- start 2024-09-19T04:19:28.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 -- start start 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d180af2a0 0x7f6d18157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1815bec0 0x7f6d18148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d1815c2a0 0x7f6d181542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6d180af6b0 con 0x7f6d180af2a0 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6d180ad740 con 0x7f6d1815bec0 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6d1806e000 con 0x7f6d1815c2a0 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d180af2a0 0x7f6d18157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d180af2a0 0x7f6d18157340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34646/0 (socket says 172.21.15.76:34646) 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 -- 172.21.15.76:0/1606235897 learned_addr learned my addr 172.21.15.76:0/1606235897 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d167fc640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1815bec0 0x7f6d18148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d177fe640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d1815c2a0 0x7f6d181542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 -- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d1815c2a0 msgr2=0x7f6d181542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d1815c2a0 0x7f6d181542f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 -- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1815bec0 msgr2=0x7f6d18148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1815bec0 0x7f6d18148840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 -- 172.21.15.76:0/1606235897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1814a890 con 0x7f6d180af2a0 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d16ffd640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d180af2a0 0x7f6d18157340 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c7f0 tx=0x7f6d0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6cff7fe640 1 -- 172.21.15.76:0/1606235897 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c007a30 con 0x7f6d180af2a0 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6cff7fe640 1 -- 172.21.15.76:0/1606235897 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d0c007bd0 con 0x7f6d180af2a0 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d18157880 con 0x7f6d180af2a0 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6cff7fe640 1 -- 172.21.15.76:0/1606235897 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0113f0 con 0x7f6d180af2a0 2024-09-19T04:19:28.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d18149710 con 0x7f6d180af2a0 2024-09-19T04:19:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d180ad200 con 0x7f6d180af2a0 2024-09-19T04:19:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6cff7fe640 1 -- 172.21.15.76:0/1606235897 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d0c011590 con 0x7f6d180af2a0 2024-09-19T04:19:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6cff7fe640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d040678c0 0x7f6d04069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6cff7fe640 1 -- 172.21.15.76:0/1606235897 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f6d0c085fd0 con 0x7f6d180af2a0 2024-09-19T04:19:28.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.696+0000 7f6d167fc640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d040678c0 0x7f6d04069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:28.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.700+0000 7f6d167fc640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d040678c0 0x7f6d04069d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6d080ad000 tx=0x7f6d080a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:28.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.700+0000 7f6cff7fe640 1 -- 172.21.15.76:0/1606235897 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6d1814a890 con 0x7f6d180af2a0 2024-09-19T04:19:28.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.912+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f6d1806fcd0 con 0x7f6d180af2a0 2024-09-19T04:19:28.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6cff7fe640 1 -- 172.21.15.76:0/1606235897 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2006 (secure 0 0 0) 0x7f6d0c015020 con 0x7f6d180af2a0 2024-09-19T04:19:28.916 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:28.916 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":387,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700801,"bytes_used":261906432,"bytes_avail":772832206848,"bytes_total":773094113280,"write_bytes_sec":938,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-09-19T04:19:23.853477+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:28.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d040678c0 msgr2=0x7f6d04069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d040678c0 0x7f6d04069d80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6d080ad000 tx=0x7f6d080a6000 comp rx=0 tx=0).stop 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d180af2a0 msgr2=0x7f6d18157340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d180af2a0 0x7f6d18157340 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c00c7f0 tx=0x7f6d0c00ccb0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 shutdown_connections 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d040678c0 0x7f6d04069d80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d1815c2a0 0x7f6d181542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1815bec0 0x7f6d18148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 --2- 172.21.15.76:0/1606235897 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d180af2a0 0x7f6d18157340 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 >> 172.21.15.76:0/1606235897 conn(0x7f6d180b1f90 msgr2=0x7f6d180cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 shutdown_connections 2024-09-19T04:19:28.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:28.916+0000 7f6d1d304640 1 -- 172.21.15.76:0/1606235897 wait complete. 2024-09-19T04:19:28.933 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'idle+waiting paths [/d1/d2/d3]', 'b': 'idle+waiting paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-19T04:19:29.934 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:30.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.064+0000 7f9afaa3d640 1 Processor -- start 2024-09-19T04:19:30.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.064+0000 7f9afaa3d640 1 -- start start 2024-09-19T04:19:30.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af406d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9af409b110 0x7f9af40b6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9af406dd30 con 0x7f9af406d890 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f9af406e5d0 con 0x7f9af409b110 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9af40b5ca0 con 0x7f9af40b3790 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9af409b110 0x7f9af40b6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f9af409b110 0x7f9af40b6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34650/0 (socket says 172.21.15.76:34650) 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 -- 172.21.15.76:0/4174443288 learned_addr learned my addr 172.21.15.76:0/4174443288 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af3fff640 1 --2- 172.21.15.76:0/4174443288 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 -- 172.21.15.76:0/4174443288 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 msgr2=0x7f9af406d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 --2- 172.21.15.76:0/4174443288 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af406d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 -- 172.21.15.76:0/4174443288 >> v1:172.21.15.116:6789/0 conn(0x7f9af406d890 legacy=0x7f9af409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 -- 172.21.15.76:0/4174443288 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af406e010 con 0x7f9af409b110 2024-09-19T04:19:30.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af8fda640 1 --2- 172.21.15.76:0/4174443288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40b6420 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009870 tx=0x7f9ae401c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae224e75e437b6c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af2ffd640 1 -- 172.21.15.76:0/4174443288 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40a95f0 con 0x7f9af409b110 2024-09-19T04:19:30.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af2ffd640 1 -- 172.21.15.76:0/4174443288 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae40a9790 con 0x7f9af409b110 2024-09-19T04:19:30.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9af2ffd640 1 -- 172.21.15.76:0/4174443288 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40a9b60 con 0x7f9af409b110 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- 172.21.15.76:0/4174443288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 msgr2=0x7f9af40b6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/4174443288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40b6420 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009870 tx=0x7f9ae401c530 comp rx=0 tx=0).stop 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- 172.21.15.76:0/4174443288 shutdown_connections 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/4174443288 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40b6420 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/4174443288 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af406d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- 172.21.15.76:0/4174443288 >> 172.21.15.76:0/4174443288 conn(0x7f9af40b78d0 msgr2=0x7f9af40b3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- 172.21.15.76:0/4174443288 shutdown_connections 2024-09-19T04:19:30.072 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.068+0000 7f9afaa3d640 1 -- 172.21.15.76:0/4174443288 wait complete. 2024-09-19T04:19:30.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 Processor -- start 2024-09-19T04:19:30.073 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 -- start start 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9af406d890 0x7f9af4147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40cc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af414fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9af415f7e0 con 0x7f9af409b110 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9af415f660 con 0x7f9af406d890 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9af40b7170 con 0x7f9af40b3790 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34664/0 (socket says 172.21.15.76:34664) 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 -- 172.21.15.76:0/1174394423 learned_addr learned my addr 172.21.15.76:0/1174394423 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:30.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af8fda640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af414fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af3fff640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9af406d890 0x7f9af4147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 -- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 msgr2=0x7f9af414fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af414fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 -- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9af406d890 msgr2=0x7f9af4147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9af406d890 0x7f9af4147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 -- 172.21.15.76:0/1174394423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af40cb690 con 0x7f9af409b110 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af8fda640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af37fe640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40cc730 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c310 tx=0x7f9adc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af0ff9640 1 -- 172.21.15.76:0/1174394423 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc0052d0 con 0x7f9af409b110 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af0ff9640 1 -- 172.21.15.76:0/1174394423 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9adc005470 con 0x7f9af409b110 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af40cb970 con 0x7f9af409b110 2024-09-19T04:19:30.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9af0ff9640 1 -- 172.21.15.76:0/1174394423 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9adc007260 con 0x7f9af409b110 2024-09-19T04:19:30.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af40cbf30 con 0x7f9af409b110 2024-09-19T04:19:30.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.072+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af4068ce0 con 0x7f9af409b110 2024-09-19T04:19:30.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.076+0000 7f9af0ff9640 1 -- 172.21.15.76:0/1174394423 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9adc007400 con 0x7f9af409b110 2024-09-19T04:19:30.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.076+0000 7f9af0ff9640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad40678c0 0x7f9ad4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.076+0000 7f9af0ff9640 1 -- 172.21.15.76:0/1174394423 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f9adc085260 con 0x7f9af409b110 2024-09-19T04:19:30.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.076+0000 7f9af3fff640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad40678c0 0x7f9ad4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.076+0000 7f9af3fff640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad40678c0 0x7f9ad4069d80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9aec007330 tx=0x7f9aec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.080+0000 7f9af0ff9640 1 -- 172.21.15.76:0/1174394423 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9adc055c30 con 0x7f9af409b110 2024-09-19T04:19:30.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.272+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9af40b75b0 con 0x7f9af409b110 2024-09-19T04:19:30.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9af0ff9640 1 -- 172.21.15.76:0/1174394423 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7f9adc055530 con 0x7f9af409b110 2024-09-19T04:19:30.276 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:30.276 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:30.277 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:30.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad40678c0 msgr2=0x7f9ad4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad40678c0 0x7f9ad4069d80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9aec007330 tx=0x7f9aec00b040 comp rx=0 tx=0).stop 2024-09-19T04:19:30.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 msgr2=0x7f9af40cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40cc730 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c310 tx=0x7f9adc00c7d0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 shutdown_connections 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ad40678c0 0x7f9ad4069d80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9af40b3790 0x7f9af414fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9af409b110 0x7f9af40cc730 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 --2- 172.21.15.76:0/1174394423 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9af406d890 0x7f9af4147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 >> 172.21.15.76:0/1174394423 conn(0x7f9af40b78d0 msgr2=0x7f9af409aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 shutdown_connections 2024-09-19T04:19:30.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.276+0000 7f9afaa3d640 1 -- 172.21.15.76:0/1174394423 wait complete. 2024-09-19T04:19:30.294 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'b'] 2024-09-19T04:19:30.294 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:19:30.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.408+0000 7ff5fe6af640 1 Processor -- start 2024-09-19T04:19:30.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.408+0000 7ff5fe6af640 1 -- start start 2024-09-19T04:19:30.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.408+0000 7ff5fe6af640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5f8073e60 0x7ff5f806e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.408+0000 7ff5fe6af640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5f806fef0 0x7ff5f815b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.408+0000 7ff5fe6af640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5f815bda0 con 0x7ff5f80b3d50 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.408+0000 7ff5fe6af640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f806ee50 con 0x7ff5f806fef0 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.408+0000 7ff5fe6af640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff5f806f130 con 0x7ff5f8073e60 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5f806fef0 0x7ff5f815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5f806fef0 0x7ff5f815b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34678/0 (socket says 172.21.15.76:34678) 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 -- 172.21.15.76:0/694472050 learned_addr learned my addr 172.21.15.76:0/694472050 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fcc4c640 1 --1- 172.21.15.76:0/694472050 >> v1:172.21.15.76:6789/0 conn(0x7ff5f80b3d50 0x7ff5f80b4170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50886/0 (socket says 172.21.15.76:50886) 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f7fff640 1 --2- 172.21.15.76:0/694472050 >> v2:172.21.15.116:3300/0 conn(0x7ff5f8073e60 0x7ff5f806e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 -- 172.21.15.76:0/694472050 >> v2:172.21.15.116:3300/0 conn(0x7ff5f8073e60 msgr2=0x7ff5f806e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 --2- 172.21.15.76:0/694472050 >> v2:172.21.15.116:3300/0 conn(0x7ff5f8073e60 0x7ff5f806e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 -- 172.21.15.76:0/694472050 >> v1:172.21.15.76:6789/0 conn(0x7ff5f80b3d50 legacy=0x7ff5f80b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 -- 172.21.15.76:0/694472050 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f806f410 con 0x7ff5f806fef0 2024-09-19T04:19:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f6ffd640 1 -- 172.21.15.76:0/694472050 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2090958613 0 0) 0x7ff5f815bda0 con 0x7ff5f80b3d50 2024-09-19T04:19:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f77fe640 1 --2- 172.21.15.76:0/694472050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f806fef0 0x7ff5f815b860 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8070340 tx=0x7ff5ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30832dd4a65fb1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f6ffd640 1 -- 172.21.15.76:0/694472050 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0b0070 con 0x7ff5f806fef0 2024-09-19T04:19:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f6ffd640 1 -- 172.21.15.76:0/694472050 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec0ab600 con 0x7ff5f806fef0 2024-09-19T04:19:30.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5f6ffd640 1 -- 172.21.15.76:0/694472050 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0ab9d0 con 0x7ff5f806fef0 2024-09-19T04:19:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 -- 172.21.15.76:0/694472050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f806fef0 msgr2=0x7ff5f815b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/694472050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f806fef0 0x7ff5f815b860 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8070340 tx=0x7ff5ec01c490 comp rx=0 tx=0).stop 2024-09-19T04:19:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 -- 172.21.15.76:0/694472050 shutdown_connections 2024-09-19T04:19:30.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/694472050 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f806fef0 0x7ff5f815b860 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/694472050 >> v2:172.21.15.116:3300/0 conn(0x7ff5f8073e60 0x7ff5f806e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 -- 172.21.15.76:0/694472050 >> 172.21.15.76:0/694472050 conn(0x7ff5f80b1f90 msgr2=0x7ff5f80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 -- 172.21.15.76:0/694472050 shutdown_connections 2024-09-19T04:19:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.412+0000 7ff5fe6af640 1 -- 172.21.15.76:0/694472050 wait complete. 2024-09-19T04:19:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 Processor -- start 2024-09-19T04:19:30.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 -- start start 2024-09-19T04:19:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f806fef0 0x7ff5f8142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f8073e60 0x7ff5f80ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5f80b3d50 0x7ff5f80ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f80729a0 con 0x7ff5f8073e60 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5f8072820 con 0x7ff5f806fef0 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff5f8072b20 con 0x7ff5f80b3d50 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f8073e60 0x7ff5f80ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f8073e60 0x7ff5f80ae380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34690/0 (socket says 172.21.15.76:34690) 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 -- 172.21.15.76:0/3258206679 learned_addr learned my addr 172.21.15.76:0/3258206679 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fcc4c640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5f80b3d50 0x7ff5f80ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 -- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5f80b3d50 msgr2=0x7ff5f80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f7fff640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f806fef0 0x7ff5f8142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5f80b3d50 0x7ff5f80ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 -- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f806fef0 msgr2=0x7ff5f8142420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f806fef0 0x7ff5f8142420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 -- 172.21.15.76:0/3258206679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5f81555b0 con 0x7ff5f8073e60 2024-09-19T04:19:30.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f77fe640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f8073e60 0x7ff5f80ae380 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0ae040 tx=0x7ff5ec01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f4ff9640 1 -- 172.21.15.76:0/3258206679 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0b0040 con 0x7ff5f8073e60 2024-09-19T04:19:30.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f4ff9640 1 -- 172.21.15.76:0/3258206679 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5ec004030 con 0x7ff5f8073e60 2024-09-19T04:19:30.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5f8155890 con 0x7ff5f8073e60 2024-09-19T04:19:30.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f4ff9640 1 -- 172.21.15.76:0/3258206679 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec0043e0 con 0x7ff5f8073e60 2024-09-19T04:19:30.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5f8155e50 con 0x7ff5f8073e60 2024-09-19T04:19:30.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5f8156180 con 0x7ff5f8073e60 2024-09-19T04:19:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.416+0000 7ff5f4ff9640 1 -- 172.21.15.76:0/3258206679 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff5ec003620 con 0x7ff5f8073e60 2024-09-19T04:19:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.420+0000 7ff5f4ff9640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5e40678c0 0x7ff5e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.420+0000 7ff5f4ff9640 1 -- 172.21.15.76:0/3258206679 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff5ec120160 con 0x7ff5f8073e60 2024-09-19T04:19:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.420+0000 7ff5f7fff640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5e40678c0 0x7ff5e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.420+0000 7ff5f7fff640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5e40678c0 0x7ff5e4069d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80af670 tx=0x7ff5e800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.424+0000 7ff5f4ff9640 1 -- 172.21.15.76:0/3258206679 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff5ec016110 con 0x7ff5f8073e60 2024-09-19T04:19:30.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff5f806e800 con 0x7ff5f8073e60 2024-09-19T04:19:30.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5f4ff9640 1 -- 172.21.15.76:0/3258206679 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1912 (secure 0 0 0) 0x7ff5ec0f0a80 con 0x7ff5f8073e60 2024-09-19T04:19:30.641 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:30.641 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":389,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":93,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700801,"bytes_used":261906432,"bytes_avail":772832206848,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","id":6,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":6,"rank":2,"name":"b","status":"up:active","gid":5794},{"filesystem_id":6,"rank":1,"name":"c","status":"up:active","gid":7305},{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7311}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-09-19T04:19:29.855585+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5e40678c0 msgr2=0x7ff5e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5e40678c0 0x7ff5e4069d80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff5f80af670 tx=0x7ff5e800b040 comp rx=0 tx=0).stop 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f8073e60 msgr2=0x7ff5f80ae380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f8073e60 0x7ff5f80ae380 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0ae040 tx=0x7ff5ec01acd0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 shutdown_connections 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5e40678c0 0x7ff5e4069d80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5f80b3d50 0x7ff5f80ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5f8073e60 0x7ff5f80ae380 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.640+0000 7ff5fe6af640 1 --2- 172.21.15.76:0/3258206679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5f806fef0 0x7ff5f8142420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.644+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 >> 172.21.15.76:0/3258206679 conn(0x7ff5f80b1f90 msgr2=0x7ff5f80b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.644+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 shutdown_connections 2024-09-19T04:19:30.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.644+0000 7ff5fe6af640 1 -- 172.21.15.76:0/3258206679 wait complete. 2024-09-19T04:19:30.659 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-19T04:19:30.659 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:19:30.659 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:19:30.721 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:19:30.810 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:19:30.811 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:19:30.811 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:19:30.811 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:30.817 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:19:30.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b40c56640 1 Processor -- start 2024-09-19T04:19:30.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b40c56640 1 -- start start 2024-09-19T04:19:30.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b40c56640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0b3280 0x7f6b3c0cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b40c56640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6b3c15d0d0 0x7f6b3c0b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b40c56640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b3c0b3ba0 con 0x7f6b3c0cf1e0 2024-09-19T04:19:30.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b40c56640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3c0ce140 con 0x7f6b3c0b3280 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b40c56640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6b3c0ce420 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b3a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0b3280 0x7f6b3c0cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b3a59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6b3c0b3280 0x7f6b3c0cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34694/0 (socket says 172.21.15.76:34694) 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b3a59c640 1 -- 172.21.15.76:0/3521276177 learned_addr learned my addr 172.21.15.76:0/3521276177 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b39d9b640 1 --2- 172.21.15.76:0/3521276177 >> v2:172.21.15.116:3300/0 conn(0x7f6b3c15d0d0 0x7f6b3c0b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b3a59c640 1 -- 172.21.15.76:0/3521276177 >> v2:172.21.15.116:3300/0 conn(0x7f6b3c15d0d0 msgr2=0x7f6b3c0b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b3a59c640 1 --2- 172.21.15.76:0/3521276177 >> v2:172.21.15.116:3300/0 conn(0x7f6b3c15d0d0 0x7f6b3c0b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b3a59c640 1 -- 172.21.15.76:0/3521276177 >> v1:172.21.15.116:6789/0 conn(0x7f6b3c0cf1e0 legacy=0x7f6b3c15d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:30.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.980+0000 7f6b3a59c640 1 -- 172.21.15.76:0/3521276177 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c0ce700 con 0x7f6b3c0b3280 2024-09-19T04:19:30.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b3a59c640 1 --2- 172.21.15.76:0/3521276177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c0b3280 0x7f6b3c0cdb00 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0d0790 tx=0x7f6b3401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add5e2e3b2f0bfc3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b3959a640 1 -- 172.21.15.76:0/3521276177 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b340a57b0 con 0x7f6b3c0b3280 2024-09-19T04:19:30.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b3959a640 1 -- 172.21.15.76:0/3521276177 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b340a5950 con 0x7f6b3c0b3280 2024-09-19T04:19:30.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b3959a640 1 -- 172.21.15.76:0/3521276177 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b340a5d20 con 0x7f6b3c0b3280 2024-09-19T04:19:30.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 -- 172.21.15.76:0/3521276177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c0b3280 msgr2=0x7f6b3c0cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 --2- 172.21.15.76:0/3521276177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c0b3280 0x7f6b3c0cdb00 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c0d0790 tx=0x7f6b3401c8f0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 -- 172.21.15.76:0/3521276177 shutdown_connections 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 --2- 172.21.15.76:0/3521276177 >> v2:172.21.15.116:3300/0 conn(0x7f6b3c15d0d0 0x7f6b3c0b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 --2- 172.21.15.76:0/3521276177 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c0b3280 0x7f6b3c0cdb00 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 -- 172.21.15.76:0/3521276177 >> 172.21.15.76:0/3521276177 conn(0x7f6b3c0b6870 msgr2=0x7f6b3c0b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 -- 172.21.15.76:0/3521276177 shutdown_connections 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 -- 172.21.15.76:0/3521276177 wait complete. 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.984+0000 7f6b40c56640 1 Processor -- start 2024-09-19T04:19:30.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 -- start start 2024-09-19T04:19:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b3c0b3280 0x7f6b3c156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b3c0cf1e0 0x7f6b3c1659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c15d0d0 0x7f6b3c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c15d4e0 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c159ee0 con 0x7f6b3c0cf1e0 2024-09-19T04:19:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3c156fb0 con 0x7f6b3c0b3280 2024-09-19T04:19:30.989 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c15d0d0 0x7f6b3c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c15d0d0 0x7f6b3c15a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34710/0 (socket says 172.21.15.76:34710) 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 -- 172.21.15.76:0/4279857353 learned_addr learned my addr 172.21.15.76:0/4279857353 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 -- 172.21.15.76:0/4279857353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b3c0b3280 msgr2=0x7f6b3c156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b39d9b640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b3c0cf1e0 0x7f6b3c1659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b3c0b3280 0x7f6b3c156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 -- 172.21.15.76:0/4279857353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b3c0cf1e0 msgr2=0x7f6b3c1659c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b3c0cf1e0 0x7f6b3c1659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 -- 172.21.15.76:0/4279857353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c165f00 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3ad9d640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c15d0d0 0x7f6b3c15a8f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00b440 tx=0x7f6b2c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b2affd640 1 -- 172.21.15.76:0/4279857353 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c015020 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b2affd640 1 -- 172.21.15.76:0/4279857353 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b2c011850 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c1573b0 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b2affd640 1 -- 172.21.15.76:0/4279857353 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c011c40 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c1634f0 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c0733c0 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b2affd640 1 -- 172.21.15.76:0/4279857353 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6b2c018020 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b2affd640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6b1c0678c0 0x7f6b1c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:30.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b2affd640 1 -- 172.21.15.76:0/4279857353 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f6b2c085c80 con 0x7f6b3c15d0d0 2024-09-19T04:19:30.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3a59c640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6b1c0678c0 0x7f6b1c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:30.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.988+0000 7f6b3a59c640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6b1c0678c0 0x7f6b1c069d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6b340ae000 tx=0x7f6b34004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:30.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:30.992+0000 7f6b2affd640 1 -- 172.21.15.76:0/4279857353 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6b2c01f020 con 0x7f6b3c15d0d0 2024-09-19T04:19:31.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6b3c06d980 con 0x7f6b3c15d0d0 2024-09-19T04:19:31.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b2affd640 1 -- 172.21.15.76:0/4279857353 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v93) ==== 80+0+0 (secure 0 0 0) 0x7f6b2c056650 con 0x7f6b3c15d0d0 2024-09-19T04:19:31.173 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6b1c0678c0 msgr2=0x7f6b1c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6b1c0678c0 0x7f6b1c069d80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6b340ae000 tx=0x7f6b34004ac0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c15d0d0 msgr2=0x7f6b3c15a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c15d0d0 0x7f6b3c15a8f0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c00b440 tx=0x7f6b2c00b900 comp rx=0 tx=0).stop 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 shutdown_connections 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6b1c0678c0 0x7f6b1c069d80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6b3c15d0d0 0x7f6b3c15a8f0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b3c0cf1e0 0x7f6b3c1659c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 --2- 172.21.15.76:0/4279857353 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6b3c0b3280 0x7f6b3c156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 >> 172.21.15.76:0/4279857353 conn(0x7f6b3c0b6870 msgr2=0x7f6b3c0b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:31.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 shutdown_connections 2024-09-19T04:19:31.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.172+0000 7f6b40c56640 1 -- 172.21.15.76:0/4279857353 wait complete. 2024-09-19T04:19:31.191 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:19:31.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:31.191 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:31.202 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[46521]: fuse finished with error 0 and tester_r 0 2024-09-19T04:19:31.203 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:19:31.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:31.203 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:19:31.327 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:19:31.328 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:19:31.328 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:31.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:19:31.335 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:19:31.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:31.335 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T04:19:31.385 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T04:19:31.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:19:31.386 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:19:31.386 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:19:31.386 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:31.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:19:31.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:19:31.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:31.392 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T04:19:31.439 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T04:19:31.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:19:31.441 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:19:31.441 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:19:31.441 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:31.446 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:19:31.447 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:19:31.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:31.447 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T04:19:31.493 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T04:19:31.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:19:31.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:31.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.624+0000 7fbaf24e8640 1 Processor -- start 2024-09-19T04:19:31.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.624+0000 7fbaf24e8640 1 -- start start 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf24e8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec15bec0 0x7fbaec0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf24e8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbaec0af470 0x7fbaec06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf24e8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec0b3290 con 0x7fbaec15c2a0 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf24e8640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbaec0b3570 con 0x7fbaec0af470 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf24e8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec0b3850 con 0x7fbaec15bec0 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbaec15c2a0 0x7fbaec0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50894/0 (socket says 172.21.15.76:50894) 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeb7fe640 1 -- 172.21.15.76:0/1900282978 learned_addr learned my addr 172.21.15.76:0/1900282978 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133779913 0 0) 0x7fbaec0b3290 con 0x7fbaec15c2a0 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf0a85640 1 --2- 172.21.15.76:0/1900282978 >> v2:172.21.15.116:3300/0 conn(0x7fbaec0af470 0x7fbaec06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaebfff640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec15bec0 0x7fbaec0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbad4003530 con 0x7fbaec15c2a0 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 98337760 0 0) 0x7fbad4003530 con 0x7fbaec15c2a0 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec15bec0 msgr2=0x7fbaec0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec15bec0 0x7fbaec0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 >> v2:172.21.15.116:3300/0 conn(0x7fbaec0af470 msgr2=0x7fbaec06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 --2- 172.21.15.76:0/1900282978 >> v2:172.21.15.116:3300/0 conn(0x7fbaec0af470 0x7fbaec06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec0b3b30 con 0x7fbaec15c2a0 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fbae0001680 con 0x7fbaec15c2a0 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 >> v1:172.21.15.76:6789/0 conn(0x7fbaec15c2a0 legacy=0x7fbaec0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad40039c0 0x7fbad4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec15bec0 0x7fbad4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad4004d50 0x7fbad40051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0b3570 con 0x7fbad40039c0 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0b3850 con 0x7fbad4004d50 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbad400b9d0 con 0x7fbaec15bec0 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaebfff640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad40039c0 0x7fbad4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf0a85640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad4004d50 0x7fbad40051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaebfff640 1 -- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec15bec0 msgr2=0x7fbad4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaebfff640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec15bec0 0x7fbad4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaebfff640 1 -- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad4004d50 msgr2=0x7fbad40051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaebfff640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad4004d50 0x7fbad40051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaebfff640 1 -- 172.21.15.76:0/1900282978 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad4003530 con 0x7fbad40039c0 2024-09-19T04:19:31.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.628+0000 7fbaf0a85640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbad4004d50 0x7fbad40051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaebfff640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad40039c0 0x7fbad4003de0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009ae0 tx=0x7fbadc01ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d062399cffe3649f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0b2070 con 0x7fbad40039c0 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbadc0a5650 con 0x7fbad40039c0 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaeaffd640 1 -- 172.21.15.76:0/1900282978 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbadc0096b0 con 0x7fbad40039c0 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- 172.21.15.76:0/1900282978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad40039c0 msgr2=0x7fbad4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 --2- 172.21.15.76:0/1900282978 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbad40039c0 0x7fbad4003de0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fbadc009ae0 tx=0x7fbadc01ca80 comp rx=0 tx=0).stop 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 -- 172.21.15.76:0/1900282978 reap_dead start 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- 172.21.15.76:0/1900282978 shutdown_connections 2024-09-19T04:19:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- 172.21.15.76:0/1900282978 >> 172.21.15.76:0/1900282978 conn(0x7fbaec0b1f90 msgr2=0x7fbaec0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- 172.21.15.76:0/1900282978 shutdown_connections 2024-09-19T04:19:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- 172.21.15.76:0/1900282978 wait complete. 2024-09-19T04:19:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 Processor -- start 2024-09-19T04:19:31.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- start start 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaec1551a0 0x7fbaec09dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec09bad0 0x7fbaec15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec15e700 0x7fbaec15eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0ad790 con 0x7fbaec15e700 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0ad610 con 0x7fbaec1551a0 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf24e8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec06ed50 con 0x7fbaec09bad0 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec15e700 0x7fbaec15eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec15e700 0x7fbaec15eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34726/0 (socket says 172.21.15.76:34726) 2024-09-19T04:19:31.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 -- 172.21.15.76:0/2737940144 learned_addr learned my addr 172.21.15.76:0/2737940144 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaebfff640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaec1551a0 0x7fbaec09dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaeb7fe640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec09bad0 0x7fbaec15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 -- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec09bad0 msgr2=0x7fbaec15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec09bad0 0x7fbaec15e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 -- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaec1551a0 msgr2=0x7fbaec09dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaec1551a0 0x7fbaec09dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 -- 172.21.15.76:0/2737940144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec09cc10 con 0x7fbaec15e700 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaeb7fe640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec09bad0 0x7fbaec15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.632+0000 7fbaf0a85640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec15e700 0x7fbaec15eb80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0b5950 tx=0x7fbae400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbae8ff9640 1 -- 172.21.15.76:0/2737940144 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4016620 con 0x7fbaec15e700 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbae8ff9640 1 -- 172.21.15.76:0/2737940144 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae401b070 con 0x7fbaec15e700 2024-09-19T04:19:31.638 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec09c000 con 0x7fbaec15e700 2024-09-19T04:19:31.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbae8ff9640 1 -- 172.21.15.76:0/2737940144 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40169e0 con 0x7fbaec15e700 2024-09-19T04:19:31.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec09c5c0 con 0x7fbaec15e700 2024-09-19T04:19:31.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec06f130 con 0x7fbaec15e700 2024-09-19T04:19:31.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbae8ff9640 1 -- 172.21.15.76:0/2737940144 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbae4016b80 con 0x7fbaec15e700 2024-09-19T04:19:31.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbae8ff9640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbad80678c0 0x7fbad8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.639 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbae8ff9640 1 -- 172.21.15.76:0/2737940144 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7fbae4086a40 con 0x7fbaec15e700 2024-09-19T04:19:31.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbaebfff640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbad80678c0 0x7fbad8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.636+0000 7fbaebfff640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbad80678c0 0x7fbad8069d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0af000 tx=0x7fbadc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:31.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.640+0000 7fbae8ff9640 1 -- 172.21.15.76:0/2737940144 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbae40122d0 con 0x7fbaec15e700 2024-09-19T04:19:31.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.836+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaec0b2430 con 0x7fbaec15e700 2024-09-19T04:19:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.836+0000 7fbae8ff9640 1 -- 172.21.15.76:0/2737940144 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7fbae4057380 con 0x7fbaec15e700 2024-09-19T04:19:31.838 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:31.838 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:31.838 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:31.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbad80678c0 msgr2=0x7fbad8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbad80678c0 0x7fbad8069d80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0af000 tx=0x7fbadc008040 comp rx=0 tx=0).stop 2024-09-19T04:19:31.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec15e700 msgr2=0x7fbaec15eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec15e700 0x7fbaec15eb80 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fbaec0b5950 tx=0x7fbae400f520 comp rx=0 tx=0).stop 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 shutdown_connections 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbad80678c0 0x7fbad8069d80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaec15e700 0x7fbaec15eb80 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaec09bad0 0x7fbaec15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 --2- 172.21.15.76:0/2737940144 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaec1551a0 0x7fbaec09dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 >> 172.21.15.76:0/2737940144 conn(0x7fbaec0b1f90 msgr2=0x7fbaec15a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 shutdown_connections 2024-09-19T04:19:31.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.840+0000 7fbaf24e8640 1 -- 172.21.15.76:0/2737940144 wait complete. 2024-09-19T04:19:31.858 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:31.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f81029d0640 1 Processor -- start 2024-09-19T04:19:31.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f81029d0640 1 -- start start 2024-09-19T04:19:31.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f81029d0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80fc073170 0x7f80fc06e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f81029d0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80fc06ec60 con 0x7f80fc0b39a0 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f81029d0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80fc06ef40 con 0x7f80fc1587a0 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f81029d0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f80fc06f870 con 0x7f80fc073170 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f8100f6d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80fc073170 0x7f80fc06e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f8100f6d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f80fc073170 0x7f80fc06e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34742/0 (socket says 172.21.15.76:34742) 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f8100f6d640 1 -- 172.21.15.76:0/264215651 learned_addr learned my addr 172.21.15.76:0/264215651 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f80fb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f80fc0b39a0 0x7f80fc06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50896/0 (socket says 172.21.15.76:50896) 2024-09-19T04:19:31.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f8100f6d640 1 -- 172.21.15.76:0/264215651 >> v1:172.21.15.116:6789/0 conn(0x7f80fc1587a0 legacy=0x7f80fc0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f8100f6d640 1 -- 172.21.15.76:0/264215651 >> v1:172.21.15.76:6789/0 conn(0x7f80fc0b39a0 legacy=0x7f80fc06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f80faffd640 1 -- 172.21.15.76:0/264215651 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902598269 0 0) 0x7f80fc06ec60 con 0x7f80fc0b39a0 2024-09-19T04:19:31.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.972+0000 7f8100f6d640 1 -- 172.21.15.76:0/264215651 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc06f220 con 0x7f80fc073170 2024-09-19T04:19:31.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f8100f6d640 1 --2- 172.21.15.76:0/264215651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc073170 0x7f80fc06e680 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f80fc06f9f0 tx=0x7f80f401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bba71cdec21ada4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:31.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80faffd640 1 -- 172.21.15.76:0/264215651 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40ab430 con 0x7f80fc073170 2024-09-19T04:19:31.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80faffd640 1 -- 172.21.15.76:0/264215651 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80f40b0070 con 0x7f80fc073170 2024-09-19T04:19:31.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80faffd640 1 -- 172.21.15.76:0/264215651 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40ab7f0 con 0x7f80fc073170 2024-09-19T04:19:31.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- 172.21.15.76:0/264215651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc073170 msgr2=0x7f80fc06e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 --2- 172.21.15.76:0/264215651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc073170 0x7f80fc06e680 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f80fc06f9f0 tx=0x7f80f401c510 comp rx=0 tx=0).stop 2024-09-19T04:19:31.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- 172.21.15.76:0/264215651 shutdown_connections 2024-09-19T04:19:31.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 --2- 172.21.15.76:0/264215651 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc073170 0x7f80fc06e680 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- 172.21.15.76:0/264215651 >> 172.21.15.76:0/264215651 conn(0x7f80fc0b1f90 msgr2=0x7f80fc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:31.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- 172.21.15.76:0/264215651 shutdown_connections 2024-09-19T04:19:31.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- 172.21.15.76:0/264215651 wait complete. 2024-09-19T04:19:31.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 Processor -- start 2024-09-19T04:19:31.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- start start 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f80fc073170 0x7f80fc146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc0b39a0 0x7f80fc157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc1587a0 0x7f80fc14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc16bd90 con 0x7f80fc0b39a0 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f80fc16bc10 con 0x7f80fc1587a0 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f81029d0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f80fc16bf10 con 0x7f80fc073170 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc0b39a0 0x7f80fc157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80fb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc0b39a0 0x7f80fc157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34758/0 (socket says 172.21.15.76:34758) 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80fb7fe640 1 -- 172.21.15.76:0/3444323763 learned_addr learned my addr 172.21.15.76:0/3444323763 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80fb7fe640 1 -- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f80fc073170 msgr2=0x7f80fc146f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f8100f6d640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc1587a0 0x7f80fc14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.976+0000 7f80fbfff640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f80fc073170 0x7f80fc146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80fb7fe640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f80fc073170 0x7f80fc146f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80fb7fe640 1 -- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc1587a0 msgr2=0x7f80fc14fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80fb7fe640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc1587a0 0x7f80fc14fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80fb7fe640 1 -- 172.21.15.76:0/3444323763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80fc160b70 con 0x7f80fc0b39a0 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f8100f6d640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc1587a0 0x7f80fc14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80fb7fe640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc0b39a0 0x7f80fc157cd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c4d0 tx=0x7f80ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80f8ff9640 1 -- 172.21.15.76:0/3444323763 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec018070 con 0x7f80fc0b39a0 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80f8ff9640 1 -- 172.21.15.76:0/3444323763 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80ec0074e0 con 0x7f80fc0b39a0 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80fc160e10 con 0x7f80fc0b39a0 2024-09-19T04:19:31.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80f8ff9640 1 -- 172.21.15.76:0/3444323763 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80ec004a30 con 0x7f80fc0b39a0 2024-09-19T04:19:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80fc14f7f0 con 0x7f80fc0b39a0 2024-09-19T04:19:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80fc0b3290 con 0x7f80fc0b39a0 2024-09-19T04:19:31.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80f8ff9640 1 -- 172.21.15.76:0/3444323763 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f80ec01b070 con 0x7f80fc0b39a0 2024-09-19T04:19:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80f8ff9640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f80dc0678c0 0x7f80dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80f8ff9640 1 -- 172.21.15.76:0/3444323763 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f80ec0851a0 con 0x7f80fc0b39a0 2024-09-19T04:19:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80fbfff640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f80dc0678c0 0x7f80dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:31.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.980+0000 7f80fbfff640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f80dc0678c0 0x7f80dc069d80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f80fc147cd0 tx=0x7f80e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:31.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:31.984+0000 7f80f8ff9640 1 -- 172.21.15.76:0/3444323763 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f80ec055ae0 con 0x7f80fc0b39a0 2024-09-19T04:19:32.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.180+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80fc070080 con 0x7f80fc0b39a0 2024-09-19T04:19:32.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f80f8ff9640 1 -- 172.21.15.76:0/3444323763 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7f80ec0553e0 con 0x7f80fc0b39a0 2024-09-19T04:19:32.185 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:32.185 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:32.185 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:32.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f80dc0678c0 msgr2=0x7f80dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f80dc0678c0 0x7f80dc069d80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f80fc147cd0 tx=0x7f80e4009040 comp rx=0 tx=0).stop 2024-09-19T04:19:32.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc0b39a0 msgr2=0x7f80fc157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc0b39a0 0x7f80fc157cd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00c4d0 tx=0x7f80ec00c990 comp rx=0 tx=0).stop 2024-09-19T04:19:32.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 shutdown_connections 2024-09-19T04:19:32.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f80dc0678c0 0x7f80dc069d80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc1587a0 0x7f80fc14fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f80fc0b39a0 0x7f80fc157cd0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 --2- 172.21.15.76:0/3444323763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f80fc073170 0x7f80fc146f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 >> 172.21.15.76:0/3444323763 conn(0x7f80fc0b1f90 msgr2=0x7f80fc0d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 shutdown_connections 2024-09-19T04:19:32.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.184+0000 7f81029d0640 1 -- 172.21.15.76:0/3444323763 wait complete. 2024-09-19T04:19:32.204 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:19:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16096fb640 1 Processor -- start 2024-09-19T04:19:32.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16096fb640 1 -- start start 2024-09-19T04:19:32.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16096fb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f160415c2a0 0x7f16040aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16096fb640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f16040af2a0 0x7f160406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16096fb640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16040b3290 con 0x7f160415bec0 2024-09-19T04:19:32.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16096fb640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f16040b3570 con 0x7f16040af2a0 2024-09-19T04:19:32.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16096fb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f16040b3850 con 0x7f160415c2a0 2024-09-19T04:19:32.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16037fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f16040af2a0 0x7f160406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16037fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f16040af2a0 0x7f160406e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34760/0 (socket says 172.21.15.76:34760) 2024-09-19T04:19:32.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16037fe640 1 -- 172.21.15.76:0/4213570469 learned_addr learned my addr 172.21.15.76:0/4213570469 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:32.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f1602ffd640 1 --1- 172.21.15.76:0/4213570469 >> v1:172.21.15.76:6789/0 conn(0x7f160415bec0 0x7f16040ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:50904/0 (socket says 172.21.15.76:50904) 2024-09-19T04:19:32.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f15fbfff640 1 --2- 172.21.15.76:0/4213570469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f160415c2a0 0x7f16040aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16037fe640 1 -- 172.21.15.76:0/4213570469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f160415c2a0 msgr2=0x7f16040aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:32.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16037fe640 1 --2- 172.21.15.76:0/4213570469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f160415c2a0 0x7f16040aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16037fe640 1 -- 172.21.15.76:0/4213570469 >> v1:172.21.15.76:6789/0 conn(0x7f160415bec0 legacy=0x7f16040ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16037fe640 1 -- 172.21.15.76:0/4213570469 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16040b3b30 con 0x7f16040af2a0 2024-09-19T04:19:32.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.324+0000 7f16027fc640 1 -- 172.21.15.76:0/4213570469 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4179794992 0 0) 0x7f16040b3290 con 0x7f160415bec0 2024-09-19T04:19:32.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 --2- 172.21.15.76:0/4213570469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16040af2a0 0x7f160406e000 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e50 tx=0x7f15fc01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5974daaaa7ae18ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16027fc640 1 -- 172.21.15.76:0/4213570469 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0a58d0 con 0x7f16040af2a0 2024-09-19T04:19:32.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16027fc640 1 -- 172.21.15.76:0/4213570469 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc0a5a70 con 0x7f16040af2a0 2024-09-19T04:19:32.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16027fc640 1 -- 172.21.15.76:0/4213570469 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0a5e40 con 0x7f16040af2a0 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- 172.21.15.76:0/4213570469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16040af2a0 msgr2=0x7f160406e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 --2- 172.21.15.76:0/4213570469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16040af2a0 0x7f160406e000 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f15fc009e50 tx=0x7f15fc01ca00 comp rx=0 tx=0).stop 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- 172.21.15.76:0/4213570469 shutdown_connections 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 --2- 172.21.15.76:0/4213570469 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f16040af2a0 0x7f160406e000 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 --2- 172.21.15.76:0/4213570469 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f160415c2a0 0x7f16040aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- 172.21.15.76:0/4213570469 >> 172.21.15.76:0/4213570469 conn(0x7f16040b1f90 msgr2=0x7f16040b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- 172.21.15.76:0/4213570469 shutdown_connections 2024-09-19T04:19:32.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- 172.21.15.76:0/4213570469 wait complete. 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 Processor -- start 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- start start 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f16040af2a0 0x7f16040730b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160415bec0 0x7f1604072800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f160415c2a0 0x7f16041571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f160406ed50 con 0x7f160415c2a0 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f16040afc90 con 0x7f160415bec0 2024-09-19T04:19:32.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f160406eed0 con 0x7f16040af2a0 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f160415c2a0 0x7f16041571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f160415c2a0 0x7f16041571f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:34766/0 (socket says 172.21.15.76:34766) 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 -- 172.21.15.76:0/3764449497 learned_addr learned my addr 172.21.15.76:0/3764449497 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 -- 172.21.15.76:0/3764449497 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f16040af2a0 msgr2=0x7f16040730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f15fbfff640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160415bec0 0x7f1604072800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f16040af2a0 0x7f16040730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 -- 172.21.15.76:0/3764449497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160415bec0 msgr2=0x7f1604072800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160415bec0 0x7f1604072800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 -- 172.21.15.76:0/3764449497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1604073d00 con 0x7f160415c2a0 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16037fe640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f160415c2a0 0x7f16041571f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0af040 tx=0x7f15fc0197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f15fb7fe640 1 -- 172.21.15.76:0/3764449497 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0bc030 con 0x7f160415c2a0 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f15fb7fe640 1 -- 172.21.15.76:0/3764449497 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc007c20 con 0x7f160415c2a0 2024-09-19T04:19:32.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1604072040 con 0x7f160415c2a0 2024-09-19T04:19:32.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f15fb7fe640 1 -- 172.21.15.76:0/3764449497 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc0a58d0 con 0x7f160415c2a0 2024-09-19T04:19:32.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.328+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1604073740 con 0x7f160415c2a0 2024-09-19T04:19:32.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.332+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16040cfa20 con 0x7f160415c2a0 2024-09-19T04:19:32.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.332+0000 7f15fb7fe640 1 -- 172.21.15.76:0/3764449497 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f15fc0bf070 con 0x7f160415c2a0 2024-09-19T04:19:32.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.332+0000 7f15fb7fe640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15e40678c0 0x7f15e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.332+0000 7f1602ffd640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15e40678c0 0x7f15e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.332+0000 7f15fb7fe640 1 -- 172.21.15.76:0/3764449497 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f15fc120e70 con 0x7f160415c2a0 2024-09-19T04:19:32.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.332+0000 7f1602ffd640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15e40678c0 0x7f15e4069d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f15f40011f0 tx=0x7f15f4007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.336+0000 7f15fb7fe640 1 -- 172.21.15.76:0/3764449497 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f15fc016620 con 0x7f160415c2a0 2024-09-19T04:19:32.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.488+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f160406e000 con 0x7f160415c2a0 2024-09-19T04:19:32.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.488+0000 7f15fb7fe640 1 -- 172.21.15.76:0/3764449497 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17163 (secure 0 0 0) 0x7f15fc0c03b0 con 0x7f160415c2a0 2024-09-19T04:19:32.490 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:32.490 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":93,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:19:08.925712+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:43.054193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:44.388097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:19:32.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15e40678c0 msgr2=0x7f15e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15e40678c0 0x7f15e4069d80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f15f40011f0 tx=0x7f15f4007930 comp rx=0 tx=0).stop 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f160415c2a0 msgr2=0x7f16041571f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f160415c2a0 0x7f16041571f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f15fc0af040 tx=0x7f15fc0197a0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 shutdown_connections 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f15e40678c0 0x7f15e4069d80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f160415c2a0 0x7f16041571f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f160415bec0 0x7f1604072800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 --2- 172.21.15.76:0/3764449497 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f16040af2a0 0x7f16040730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 >> 172.21.15.76:0/3764449497 conn(0x7f16040b1f90 msgr2=0x7f16040cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 shutdown_connections 2024-09-19T04:19:32.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.492+0000 7f16096fb640 1 -- 172.21.15.76:0/3764449497 wait complete. 2024-09-19T04:19:32.508 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:19:32.508 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:32.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87f6481640 1 Processor -- start 2024-09-19T04:19:32.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87f6481640 1 -- start start 2024-09-19T04:19:32.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87f6481640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f87f015bec0 0x7f87f00ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87f6481640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f00b3290 con 0x7f87f00af470 2024-09-19T04:19:32.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87f6481640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f00b3570 con 0x7f87f015c2a0 2024-09-19T04:19:32.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87f6481640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f87f00b3850 con 0x7f87f015bec0 2024-09-19T04:19:32.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87effff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f87f015bec0 0x7f87f00ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87effff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f87f015bec0 0x7f87f00ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51796/0 (socket says 172.21.15.76:51796) 2024-09-19T04:19:32.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87effff640 1 -- 172.21.15.76:0/2205604046 learned_addr learned my addr 172.21.15.76:0/2205604046 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:32.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87effff640 1 -- 172.21.15.76:0/2205604046 >> v1:172.21.15.116:6789/0 conn(0x7f87f015c2a0 legacy=0x7f87f00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:32.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87effff640 1 -- 172.21.15.76:0/2205604046 >> v1:172.21.15.76:6789/0 conn(0x7f87f00af470 legacy=0x7f87f006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87eeffd640 1 -- 172.21.15.76:0/2205604046 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1364054315 0 0) 0x7f87f00b3290 con 0x7f87f00af470 2024-09-19T04:19:32.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.624+0000 7f87effff640 1 -- 172.21.15.76:0/2205604046 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f00b3b30 con 0x7f87f015bec0 2024-09-19T04:19:32.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 --2- 172.21.15.76:0/2205604046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f015bec0 0x7f87f00ad200 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009870 tx=0x7f87e401c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82c1255ca2218f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87eeffd640 1 -- 172.21.15.76:0/2205604046 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e40aa650 con 0x7f87f015bec0 2024-09-19T04:19:32.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87eeffd640 1 -- 172.21.15.76:0/2205604046 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e40af070 con 0x7f87f015bec0 2024-09-19T04:19:32.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87eeffd640 1 -- 172.21.15.76:0/2205604046 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e40aaa10 con 0x7f87f015bec0 2024-09-19T04:19:32.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- 172.21.15.76:0/2205604046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f015bec0 msgr2=0x7f87f00ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 --2- 172.21.15.76:0/2205604046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f015bec0 0x7f87f00ad200 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009870 tx=0x7f87e401c4f0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- 172.21.15.76:0/2205604046 shutdown_connections 2024-09-19T04:19:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 --2- 172.21.15.76:0/2205604046 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f015bec0 0x7f87f00ad200 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- 172.21.15.76:0/2205604046 >> 172.21.15.76:0/2205604046 conn(0x7f87f00b1f90 msgr2=0x7f87f00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- 172.21.15.76:0/2205604046 shutdown_connections 2024-09-19T04:19:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- 172.21.15.76:0/2205604046 wait complete. 2024-09-19T04:19:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 Processor -- start 2024-09-19T04:19:32.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- start start 2024-09-19T04:19:32.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f00af470 0x7f87f0146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87f015bec0 0x7f87f014fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87f015c2a0 0x7f87f00afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f87f006ef20 con 0x7f87f00af470 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f87f00ad740 con 0x7f87f015bec0 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f87f006f0a0 con 0x7f87f015c2a0 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f00af470 0x7f87f0146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f00af470 0x7f87f0146d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51802/0 (socket says 172.21.15.76:51802) 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 -- 172.21.15.76:0/598862228 learned_addr learned my addr 172.21.15.76:0/598862228 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87ef7fe640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87f015bec0 0x7f87f014fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 -- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87f015c2a0 msgr2=0x7f87f00afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87f015c2a0 0x7f87f00afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 -- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87f015bec0 msgr2=0x7f87f014fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87f015bec0 0x7f87f014fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 -- 172.21.15.76:0/598862228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f0150370 con 0x7f87f00af470 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87ef7fe640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87f015bec0 0x7f87f014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87effff640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f00af470 0x7f87f0146d10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009870 tx=0x7f87e401ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87ecff9640 1 -- 172.21.15.76:0/598862228 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e40b9070 con 0x7f87f00af470 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87ecff9640 1 -- 172.21.15.76:0/598862228 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e40a6040 con 0x7f87f00af470 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87ecff9640 1 -- 172.21.15.76:0/598862228 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e4004020 con 0x7f87f00af470 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f014e9c0 con 0x7f87f00af470 2024-09-19T04:19:32.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.628+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f014d720 con 0x7f87f00af470 2024-09-19T04:19:32.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.632+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f00ad200 con 0x7f87f00af470 2024-09-19T04:19:32.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.632+0000 7f87ecff9640 1 -- 172.21.15.76:0/598862228 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f87e40041c0 con 0x7f87f00af470 2024-09-19T04:19:32.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.632+0000 7f87ecff9640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87dc0678c0 0x7f87dc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.632+0000 7f87ef7fe640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87dc0678c0 0x7f87dc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.632+0000 7f87ecff9640 1 -- 172.21.15.76:0/598862228 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f87e41202c0 con 0x7f87f00af470 2024-09-19T04:19:32.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.632+0000 7f87ef7fe640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87dc0678c0 0x7f87dc069d80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f87f00b56e0 tx=0x7f87e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.636+0000 7f87ecff9640 1 -- 172.21.15.76:0/598862228 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f87e40c2060 con 0x7f87f00af470 2024-09-19T04:19:32.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.828+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87f006fcd0 con 0x7f87f00af470 2024-09-19T04:19:32.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.828+0000 7f87ecff9640 1 -- 172.21.15.76:0/598862228 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7f87f0150370 con 0x7f87f00af470 2024-09-19T04:19:32.830 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:32.830 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:32.830 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87dc0678c0 msgr2=0x7f87dc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87dc0678c0 0x7f87dc069d80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f87f00b56e0 tx=0x7f87e0008040 comp rx=0 tx=0).stop 2024-09-19T04:19:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f00af470 msgr2=0x7f87f0146d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f00af470 0x7f87f0146d10 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f87e4009870 tx=0x7f87e401ad30 comp rx=0 tx=0).stop 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 shutdown_connections 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87dc0678c0 0x7f87dc069d80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87f015c2a0 0x7f87f00afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87f015bec0 0x7f87f014fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 --2- 172.21.15.76:0/598862228 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87f00af470 0x7f87f0146d10 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 >> 172.21.15.76:0/598862228 conn(0x7f87f00b1f90 msgr2=0x7f87f01597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 shutdown_connections 2024-09-19T04:19:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.832+0000 7f87f6481640 1 -- 172.21.15.76:0/598862228 wait complete. 2024-09-19T04:19:32.848 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:32.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.952+0000 7f5cd1e49640 1 Processor -- start 2024-09-19T04:19:32.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.952+0000 7f5cd1e49640 1 -- start start 2024-09-19T04:19:32.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cd1e49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc141d20 0x7f5ccc166cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cd1e49640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc142300 con 0x7f5ccc141230 2024-09-19T04:19:32.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cd1e49640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ccc1425e0 con 0x7f5ccc071ef0 2024-09-19T04:19:32.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cd1e49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ccc0d1160 con 0x7f5ccc141d20 2024-09-19T04:19:32.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccb7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f5ccc141230 0x7f5ccc0717e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36518/0 (socket says 172.21.15.76:36518) 2024-09-19T04:19:32.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccb7fe640 1 -- 172.21.15.76:0/3699699396 learned_addr learned my addr 172.21.15.76:0/3699699396 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:32.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccbfff640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc141d20 0x7f5ccc166cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1982224938 0 0) 0x7f5ccc142300 con 0x7f5ccc141230 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cac003530 con 0x7f5ccc141230 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3784252862 0 0) 0x7f5cac003530 con 0x7f5ccc141230 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc141d20 msgr2=0x7f5ccc166cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc141d20 0x7f5ccc166cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 >> v1:172.21.15.116:6789/0 conn(0x7f5ccc071ef0 legacy=0x7f5ccc141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc0d1440 con 0x7f5ccc141230 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5cbc001680 con 0x7f5ccc141230 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 >> v1:172.21.15.76:6789/0 conn(0x7f5ccc141230 legacy=0x7f5ccc0717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cac0039f0 0x7f5cac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc141d20 0x7f5cac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0d1160 con 0x7f5cac004810 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac00ba80 con 0x7f5cac0039f0 2024-09-19T04:19:32.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5cac00bc80 con 0x7f5ccc141d20 2024-09-19T04:19:32.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5ccb7fe640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cac0039f0 0x7f5cac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cc3fff640 1 -- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc141d20 msgr2=0x7f5cac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:32.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc141d20 0x7f5cac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cc3fff640 1 -- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cac0039f0 msgr2=0x7f5cac003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:32.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cac0039f0 0x7f5cac003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.956+0000 7f5cc3fff640 1 -- 172.21.15.76:0/3699699396 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac003530 con 0x7f5cac004810 2024-09-19T04:19:32.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4001750 tx=0x7f5cb401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8519ebdabd32576 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40013a0 con 0x7f5cac004810 2024-09-19T04:19:32.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cb40052f0 con 0x7f5cac004810 2024-09-19T04:19:32.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccaffd640 1 -- 172.21.15.76:0/3699699396 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4003840 con 0x7f5cac004810 2024-09-19T04:19:32.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- 172.21.15.76:0/3699699396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 msgr2=0x7f5cac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 --2- 172.21.15.76:0/3699699396 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5cac004c90 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4001750 tx=0x7f5cb401c3c0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccbfff640 1 -- 172.21.15.76:0/3699699396 reap_dead start 2024-09-19T04:19:32.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- 172.21.15.76:0/3699699396 shutdown_connections 2024-09-19T04:19:32.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- 172.21.15.76:0/3699699396 >> 172.21.15.76:0/3699699396 conn(0x7f5ccc06c0c0 msgr2=0x7f5ccc06c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:32.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- 172.21.15.76:0/3699699396 shutdown_connections 2024-09-19T04:19:32.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- 172.21.15.76:0/3699699396 wait complete. 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 Processor -- start 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- start start 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5ccc1f1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1582a0 0x7f5ccc158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc15a6f0 0x7f5ccc15ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0d0f20 con 0x7f5cac004810 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ccc0d0da0 con 0x7f5ccc1582a0 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cd1e49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ccc0d2970 con 0x7f5ccc15a6f0 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5ccc1f1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5ccc1f1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51818/0 (socket says 172.21.15.76:51818) 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 -- 172.21.15.76:0/2302287067 learned_addr learned my addr 172.21.15.76:0/2302287067 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1582a0 0x7f5ccc158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccbfff640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc15a6f0 0x7f5ccc15ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 -- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc15a6f0 msgr2=0x7f5ccc15ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc15a6f0 0x7f5ccc15ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 -- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1582a0 msgr2=0x7f5ccc158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1582a0 0x7f5ccc158720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 -- 172.21.15.76:0/2302287067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ccc158dc0 con 0x7f5cac004810 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1582a0 0x7f5ccc158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.960+0000 7f5ccb7fe640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5ccc1f1410 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc011870 tx=0x7f5cbc011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc8ff9640 1 -- 172.21.15.76:0/2302287067 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc010040 con 0x7f5cac004810 2024-09-19T04:19:32.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc8ff9640 1 -- 172.21.15.76:0/2302287067 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cbc01a070 con 0x7f5cac004810 2024-09-19T04:19:32.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ccc1590a0 con 0x7f5cac004810 2024-09-19T04:19:32.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc8ff9640 1 -- 172.21.15.76:0/2302287067 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00fda0 con 0x7f5cac004810 2024-09-19T04:19:32.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ccc159660 con 0x7f5cac004810 2024-09-19T04:19:32.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ccc141610 con 0x7f5cac004810 2024-09-19T04:19:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc8ff9640 1 -- 172.21.15.76:0/2302287067 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5cbc01f460 con 0x7f5cac004810 2024-09-19T04:19:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc8ff9640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5cb00678c0 0x7f5cb0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc8ff9640 1 -- 172.21.15.76:0/2302287067 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f5cbc089050 con 0x7f5cac004810 2024-09-19T04:19:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5cb00678c0 0x7f5cb0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:32.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.964+0000 7f5cc3fff640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5cb00678c0 0x7f5cb0069d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0057f0 tx=0x7f5cb40011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:32.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:32.968+0000 7f5cc8ff9640 1 -- 172.21.15.76:0/2302287067 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5cbc059a20 con 0x7f5cac004810 2024-09-19T04:19:33.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.164+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ccc1f1c10 con 0x7f5cac004810 2024-09-19T04:19:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cc8ff9640 1 -- 172.21.15.76:0/2302287067 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+5275 (secure 0 0 0) 0x7f5cbc059320 con 0x7f5cac004810 2024-09-19T04:19:33.169 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:33.169 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":94,"btime":"2024-09-19T04:19:20:003228+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:18:45.404923+0000","modified":"2024-09-19T04:19:19.183583+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":7311,"mds_1":7305,"mds_2":5794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5794":{"gid":5794,"name":"b","rank":2,"incarnation":88,"state":"up:active","state_seq":10,"addr":"172.21.15.116:6833/2189465635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2189465635},{"type":"v1","addr":"172.21.15.116:6833","nonce":2189465635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7305":{"gid":7305,"name":"c","rank":1,"incarnation":86,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/987394316","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":987394316},{"type":"v1","addr":"172.21.15.76:6835","nonce":987394316}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_7311":{"gid":7311,"name":"a","rank":0,"incarnation":80,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/2711186256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2711186256},{"type":"v1","addr":"172.21.15.76:6837","nonce":2711186256}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7311,"qdb_cluster":[7311,7305,5794]},"id":6}]} 2024-09-19T04:19:33.169 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 94 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5cb00678c0 msgr2=0x7f5cb0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5cb00678c0 0x7f5cb0069d80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0057f0 tx=0x7f5cb40011a0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 msgr2=0x7f5ccc1f1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5ccc1f1410 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc011870 tx=0x7f5cbc011d30 comp rx=0 tx=0).stop 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 shutdown_connections 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5cb00678c0 0x7f5cb0069d80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ccc15a6f0 0x7f5ccc15ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ccc1582a0 0x7f5ccc158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 --2- 172.21.15.76:0/2302287067 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5cac004810 0x7f5ccc1f1410 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 >> 172.21.15.76:0/2302287067 conn(0x7f5ccc06c0c0 msgr2=0x7f5ccc13f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:33.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 shutdown_connections 2024-09-19T04:19:33.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.168+0000 7f5cd1e49640 1 -- 172.21.15.76:0/2302287067 wait complete. 2024-09-19T04:19:33.187 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:19:33.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.300+0000 7ff4cf998640 1 Processor -- start 2024-09-19T04:19:33.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.300+0000 7ff4cf998640 1 -- start start 2024-09-19T04:19:33.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cf998640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff4c815bec0 0x7ff4c80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cf998640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c80b3290 con 0x7ff4c80af470 2024-09-19T04:19:33.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cf998640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4c80b3570 con 0x7ff4c815c2a0 2024-09-19T04:19:33.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cf998640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff4c80b3850 con 0x7ff4c815bec0 2024-09-19T04:19:33.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cdf35640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff4c80af470 0x7ff4c806e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36532/0 (socket says 172.21.15.76:36532) 2024-09-19T04:19:33.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cdf35640 1 -- 172.21.15.76:0/3144421919 learned_addr learned my addr 172.21.15.76:0/3144421919 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:33.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3233535441 0 0) 0x7ff4c80b3290 con 0x7ff4c80af470 2024-09-19T04:19:33.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b4003530 con 0x7ff4c80af470 2024-09-19T04:19:33.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3144421919 >> v2:172.21.15.116:3300/0 conn(0x7ff4c815bec0 0x7ff4c80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4089486347 0 0) 0x7ff4b4003530 con 0x7ff4c80af470 2024-09-19T04:19:33.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 >> v2:172.21.15.116:3300/0 conn(0x7ff4c815bec0 msgr2=0x7ff4c80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 --2- 172.21.15.76:0/3144421919 >> v2:172.21.15.116:3300/0 conn(0x7ff4c815bec0 0x7ff4c80ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 >> v1:172.21.15.116:6789/0 conn(0x7ff4c815c2a0 legacy=0x7ff4c80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c80b3b30 con 0x7ff4c80af470 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7ff4c0001a00 con 0x7ff4c80af470 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3144421919 >> v2:172.21.15.116:3300/0 conn(0x7ff4c815bec0 0x7ff4c80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 >> v1:172.21.15.76:6789/0 conn(0x7ff4c80af470 legacy=0x7ff4c806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4b40039c0 0x7ff4b4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4004810 0x7ff4b4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40051d0 0x7ff4b400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80b3850 con 0x7ff4b4004810 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b400c120 con 0x7ff4b40051d0 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b400c320 con 0x7ff4b40039c0 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4ccf33640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4004810 0x7ff4b4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4b40039c0 0x7ff4b4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cdf35640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40051d0 0x7ff4b400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4ccf33640 1 -- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4b40039c0 msgr2=0x7ff4b4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4ccf33640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4b40039c0 0x7ff4b4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4ccf33640 1 -- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40051d0 msgr2=0x7ff4b400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4ccf33640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40051d0 0x7ff4b400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4ccf33640 1 -- 172.21.15.76:0/3144421919 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b4003530 con 0x7ff4b4004810 2024-09-19T04:19:33.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4b40039c0 0x7ff4b4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:33.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cdf35640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b40051d0 0x7ff4b400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:33.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4cdf35640 1 -- 172.21.15.76:0/3144421919 reap_dead start 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4ccf33640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4004810 0x7ff4b4004c90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8001750 tx=0x7ff4b801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c1a61f2bff9062 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b80013a0 con 0x7ff4b4004810 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4b80052f0 con 0x7ff4b4004810 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.304+0000 7ff4bffff640 1 -- 172.21.15.76:0/3144421919 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8003840 con 0x7ff4b4004810 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- 172.21.15.76:0/3144421919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4004810 msgr2=0x7ff4b4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4004810 0x7ff4b4004c90 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7ff4b8001750 tx=0x7ff4b801c3c0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- 172.21.15.76:0/3144421919 shutdown_connections 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3144421919 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4b4004810 0x7ff4b4004c90 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- 172.21.15.76:0/3144421919 >> 172.21.15.76:0/3144421919 conn(0x7ff4c80b1f90 msgr2=0x7ff4c80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:33.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- 172.21.15.76:0/3144421919 shutdown_connections 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- 172.21.15.76:0/3144421919 wait complete. 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 Processor -- start 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- start start 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b4004810 0x7ff4c815e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4c8150330 0x7ff4c81f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4c815ee00 0x7ff4c815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80afc90 con 0x7ff4c815ee00 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff4c80b40f0 con 0x7ff4b4004810 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff4c80aed60 con 0x7ff4c8150330 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4c815ee00 0x7ff4c815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4c815ee00 0x7ff4c815f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51838/0 (socket says 172.21.15.76:51838) 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 -- 172.21.15.76:0/3579997953 learned_addr learned my addr 172.21.15.76:0/3579997953 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4ccf33640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4c8150330 0x7ff4c81f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b4004810 0x7ff4c815e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 -- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4c8150330 msgr2=0x7ff4c81f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4c8150330 0x7ff4c81f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 -- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b4004810 msgr2=0x7ff4c815e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b4004810 0x7ff4c815e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 -- 172.21.15.76:0/3579997953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4c81f3840 con 0x7ff4c815ee00 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b4004810 0x7ff4c815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:33.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cdf35640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4c815ee00 0x7ff4c815f290 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b5950 tx=0x7ff4c0010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4bdffb640 1 -- 172.21.15.76:0/3579997953 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c001a070 con 0x7ff4c815ee00 2024-09-19T04:19:33.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4bdffb640 1 -- 172.21.15.76:0/3579997953 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c0011040 con 0x7ff4c815ee00 2024-09-19T04:19:33.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4c81507f0 con 0x7ff4c815ee00 2024-09-19T04:19:33.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4bdffb640 1 -- 172.21.15.76:0/3579997953 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000ed20 con 0x7ff4c815ee00 2024-09-19T04:19:33.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.308+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4c8150db0 con 0x7ff4c815ee00 2024-09-19T04:19:33.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.312+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4c80af0f0 con 0x7ff4c815ee00 2024-09-19T04:19:33.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.312+0000 7ff4bdffb640 1 -- 172.21.15.76:0/3579997953 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff4c001d070 con 0x7ff4c815ee00 2024-09-19T04:19:33.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.312+0000 7ff4bdffb640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4ac0678c0 0x7ff4ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.312+0000 7ff4bdffb640 1 -- 172.21.15.76:0/3579997953 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7ff4c00885a0 con 0x7ff4c815ee00 2024-09-19T04:19:33.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.312+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4ac0678c0 0x7ff4ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.312+0000 7ff4cd734640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4ac0678c0 0x7ff4ac069d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80512d0 tx=0x7ff4c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.316+0000 7ff4bdffb640 1 -- 172.21.15.76:0/3579997953 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff4c81f3840 con 0x7ff4c815ee00 2024-09-19T04:19:33.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.464+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4c806fcd0 con 0x7ff4c815ee00 2024-09-19T04:19:33.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4bdffb640 1 -- 172.21.15.76:0/3579997953 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+17163 (secure 0 0 0) 0x7ff4c0058ed0 con 0x7ff4c815ee00 2024-09-19T04:19:33.469 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:33.469 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":93,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:19:08.925712+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:18:43.054193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-09-19T04:18:44.388097+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":7}]},{"pgid":"14.1e","mappings":[{"from":1,"to":2}]},{"pgid":"14.36","mappings":[{"from":1,"to":7}]},{"pgid":"14.3a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:19:33.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4ac0678c0 msgr2=0x7ff4ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4ac0678c0 0x7ff4ac069d80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80512d0 tx=0x7ff4c4008040 comp rx=0 tx=0).stop 2024-09-19T04:19:33.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4c815ee00 msgr2=0x7ff4c815f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4c815ee00 0x7ff4c815f290 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7ff4c80b5950 tx=0x7ff4c0010c30 comp rx=0 tx=0).stop 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 shutdown_connections 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4ac0678c0 0x7ff4ac069d80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff4c815ee00 0x7ff4c815f290 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff4c8150330 0x7ff4c81f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 --2- 172.21.15.76:0/3579997953 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4b4004810 0x7ff4c815e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 >> 172.21.15.76:0/3579997953 conn(0x7ff4c80b1f90 msgr2=0x7ff4c815a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 shutdown_connections 2024-09-19T04:19:33.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.468+0000 7ff4cf998640 1 -- 172.21.15.76:0/3579997953 wait complete. 2024-09-19T04:19:33.487 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T04:19:33.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5666d43640 1 Processor -- start 2024-09-19T04:19:33.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5666d43640 1 -- start start 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5666d43640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f566015c2a0 0x7f56600aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5666d43640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56600af2a0 0x7f566006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5666d43640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56600b3290 con 0x7f566015bec0 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5666d43640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f56600b3570 con 0x7f56600af2a0 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5666d43640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f56600b3850 con 0x7f566015c2a0 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5664adf640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f566015bec0 0x7f56600ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36534/0 (socket says 172.21.15.76:36534) 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f5664adf640 1 -- 172.21.15.76:0/3795053555 learned_addr learned my addr 172.21.15.76:0/3795053555 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565ffff640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f566015c2a0 0x7f56600aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.604 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4219130773 0 0) 0x7f56600b3290 con 0x7f566015bec0 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5648003530 con 0x7f566015bec0 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f56652e0640 1 --2- 172.21.15.76:0/3795053555 >> v2:172.21.15.116:3300/0 conn(0x7f56600af2a0 0x7f566006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3370765342 0 0) 0x7f5648003530 con 0x7f566015bec0 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f566015c2a0 msgr2=0x7f56600aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f566015c2a0 0x7f56600aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 >> v2:172.21.15.116:3300/0 conn(0x7f56600af2a0 msgr2=0x7f566006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 --2- 172.21.15.76:0/3795053555 >> v2:172.21.15.116:3300/0 conn(0x7f56600af2a0 0x7f566006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56600b3b30 con 0x7f566015bec0 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f5654001680 con 0x7f566015bec0 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 >> v1:172.21.15.76:6789/0 conn(0x7f566015bec0 legacy=0x7f56600ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f566015c2a0 0x7f56480039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5648003f40 0x7f5648004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.605 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5648004d50 0x7f56480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f56600b3570 con 0x7f5648003f40 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56600b3850 con 0x7f5648004d50 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f564800b9d0 con 0x7f566015c2a0 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565ffff640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5648003f40 0x7f5648004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f56652e0640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5648004d50 0x7f56480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565ffff640 1 -- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f566015c2a0 msgr2=0x7f56480039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565ffff640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f566015c2a0 0x7f56480039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.600+0000 7f565ffff640 1 -- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5648004d50 msgr2=0x7f56480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f565ffff640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5648004d50 0x7f56480051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f565ffff640 1 -- 172.21.15.76:0/3795053555 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5648003530 con 0x7f5648003f40 2024-09-19T04:19:33.606 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f56652e0640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5648004d50 0x7f56480051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f565ffff640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5648003f40 0x7f5648004810 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5650009ae0 tx=0x7f565001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506e85064e06207b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500b2070 con 0x7f5648003f40 2024-09-19T04:19:33.607 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500a56a0 con 0x7f5648003f40 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f565f7fe640 1 -- 172.21.15.76:0/3795053555 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500096b0 con 0x7f5648003f40 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f5666d43640 1 -- 172.21.15.76:0/3795053555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5648003f40 msgr2=0x7f5648004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f5666d43640 1 --2- 172.21.15.76:0/3795053555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5648003f40 0x7f5648004810 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5650009ae0 tx=0x7f565001ca50 comp rx=0 tx=0).stop 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f56652e0640 1 -- 172.21.15.76:0/3795053555 reap_dead start 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f5666d43640 1 -- 172.21.15.76:0/3795053555 shutdown_connections 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f5666d43640 1 -- 172.21.15.76:0/3795053555 >> 172.21.15.76:0/3795053555 conn(0x7f56600b1f90 msgr2=0x7f56600b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f5666d43640 1 -- 172.21.15.76:0/3795053555 shutdown_connections 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.604+0000 7f5666d43640 1 -- 172.21.15.76:0/3795053555 wait complete. 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 Processor -- start 2024-09-19T04:19:33.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 -- start start 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5648003f40 0x7f566015f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5660151b90 0x7f5660153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f566014d970 0x7f566015cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f56600af730 con 0x7f5660151b90 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f56600b40f0 con 0x7f566014d970 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f56600afc90 con 0x7f5648003f40 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5660151b90 0x7f5660153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5660151b90 0x7f5660153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51850/0 (socket says 172.21.15.76:51850) 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 -- 172.21.15.76:0/1801956578 learned_addr learned my addr 172.21.15.76:0/1801956578 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f56652e0640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f566014d970 0x7f566015cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5664adf640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5648003f40 0x7f566015f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 -- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5648003f40 msgr2=0x7f566015f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5648003f40 0x7f566015f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 -- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f566014d970 msgr2=0x7f566015cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f566014d970 0x7f566015cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 -- 172.21.15.76:0/1801956578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5660153960 con 0x7f5660151b90 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565ffff640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5660151b90 0x7f5660153420 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f565001d2f0 tx=0x7f5648005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565d7fa640 1 -- 172.21.15.76:0/1801956578 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500ba030 con 0x7f5660151b90 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565d7fa640 1 -- 172.21.15.76:0/1801956578 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56500a5e50 con 0x7f5660151b90 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5660152380 con 0x7f5660151b90 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565d7fa640 1 -- 172.21.15.76:0/1801956578 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56500b2040 con 0x7f5660151b90 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5660152940 con 0x7f5660151b90 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56600aed60 con 0x7f5660151b90 2024-09-19T04:19:33.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565d7fa640 1 -- 172.21.15.76:0/1801956578 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5650004020 con 0x7f5660151b90 2024-09-19T04:19:33.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565d7fa640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f564c0678c0 0x7f564c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f565d7fa640 1 -- 172.21.15.76:0/1801956578 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 6217+0+0 (secure 0 0 0) 0x7f565011f460 con 0x7f5660151b90 2024-09-19T04:19:33.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5664adf640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f564c0678c0 0x7f564c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.608+0000 7f5664adf640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f564c0678c0 0x7f564c069d80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5654000c10 tx=0x7f5654001580 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.612+0000 7f565d7fa640 1 -- 172.21.15.76:0/1801956578 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f56500c4060 con 0x7f5660151b90 2024-09-19T04:19:33.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.796+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f566006fcd0 con 0x7f5660151b90 2024-09-19T04:19:33.798 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:19:33.796+0000 7fa674384640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:19:33.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.800+0000 7f565d7fa640 1 -- 172.21.15.76:0/1801956578 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) ==== 161+0+0 (secure 0 0 0) 0x7f56500167c0 con 0x7f5660151b90 2024-09-19T04:19:33.801 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.800+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f564c0678c0 msgr2=0x7f564c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.800+0000 7f5666d43640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f564c0678c0 0x7f564c069d80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f5654000c10 tx=0x7f5654001580 comp rx=0 tx=0).stop 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.800+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5660151b90 msgr2=0x7f5660153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.800+0000 7f5666d43640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5660151b90 0x7f5660153420 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f565001d2f0 tx=0x7f5648005840 comp rx=0 tx=0).stop 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 shutdown_connections 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f564c0678c0 0x7f564c069d80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f566014d970 0x7f566015cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5660151b90 0x7f5660153420 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 --2- 172.21.15.76:0/1801956578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5648003f40 0x7f566015f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 >> 172.21.15.76:0/1801956578 conn(0x7f56600b1f90 msgr2=0x7f56601597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:33.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 shutdown_connections 2024-09-19T04:19:33.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.808+0000 7f5666d43640 1 -- 172.21.15.76:0/1801956578 wait complete. 2024-09-19T04:19:33.817 INFO:tasks.ceph.mds.a.smithi076.stderr: -2464> 2024-09-19T04:19:14.217+0000 7f4993eef640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:18:58.573984+0000 1=0+1) n(v3 rc2024-09-19T04:18:58.784529+0000 6=0+6) (inest mix dirty) (ifile excl) caps={7455=pAsLsXsFsx/AsLsXsFsx@9},l=7455 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x561bb63f8680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.573984+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.784529+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.573984+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:18:58.784529+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:19:33.817 INFO:tasks.ceph.mds.a.smithi076.stderr: -2342> 2024-09-19T04:19:14.217+0000 7f499a6fc640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:18:58.575422+0000 1=0+1) n(v4 rc2024-09-19T04:18:58.784529+0000 5=0+5) (inest sync->mix g=1,2) caps={7455=pAsLsXsFs/-@11} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x561bb63f7600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v18>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.575422+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.784529+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.575422+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:18:58.784529+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:19:33.821 INFO:tasks.ceph.mds.b.smithi116.stderr: -1033> 2024-09-19T04:19:14.218+0000 7f8c9ae37640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:18:58.579541+0000 1=0+1) n(v0 rc2024-09-19T04:18:58.579541+0000 2=0+2) (inest lock) (ifile excl) caps={7455=pAsLsXsFsx/-@6},l=7455 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x556a355dd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.579541+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.579541+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.579541+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:18:58.579541+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:19:33.822 INFO:tasks.ceph.mds.b.smithi116.stderr: -1006> 2024-09-19T04:19:14.218+0000 7f8c9ae37640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={7455=pAsLsXsFs/-@3},l=7455 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x556a355dd600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:19:33.825 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:19:33.826 INFO:tasks.ceph.mds.c.smithi076.stderr: -1142> 2024-09-19T04:19:14.217+0000 7efc56243640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:18:58.576784+0000 1=0+1) n(v2 rc2024-09-19T04:18:58.784529+0000 4=0+4) (inest lock) caps={7455=pAsLsXsFs/-@8},l=7455 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55e37ab33080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.576784+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.784529+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.576784+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:18:58.784529+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:19:33.827 INFO:tasks.ceph.mds.c.smithi076.stderr: -996> 2024-09-19T04:19:14.221+0000 7efc5ca50640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:18:58.578146+0000 1=0+1) n(v1 rc2024-09-19T04:18:58.784529+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={7455=pAsLsXsFs/-@8} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x55e37bb60000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v16>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:18:58.578146+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:18:58.579541+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:18:58.578146+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:18:58.784529+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:19:33.848 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:19:33.854 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:19:33.878 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:19:33.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e9ab31640 1 Processor -- start 2024-09-19T04:19:33.944 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e9ab31640 1 -- start start 2024-09-19T04:19:33.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e9ab31640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e940d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e9ab31640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e940d2e90 0x7f5e940cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.945 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e9ab31640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e940d32b0 con 0x7f5e940cf450 2024-09-19T04:19:33.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e9ab31640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5e940d0300 con 0x7f5e940d2e90 2024-09-19T04:19:33.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e9ab31640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5e9406e6c0 con 0x7f5e94161a60 2024-09-19T04:19:33.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e940d2e90 0x7f5e940cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5e940d2e90 0x7f5e940cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51912/0 (socket says 172.21.15.76:51912) 2024-09-19T04:19:33.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 -- 172.21.15.76:0/2483868427 learned_addr learned my addr 172.21.15.76:0/2483868427 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:33.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e988cd640 1 --2- 172.21.15.76:0/2483868427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e940d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 -- 172.21.15.76:0/2483868427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 msgr2=0x7f5e940d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:33.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 --2- 172.21.15.76:0/2483868427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e940d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 -- 172.21.15.76:0/2483868427 >> v1:172.21.15.76:6789/0 conn(0x7f5e940cf450 legacy=0x7f5e940d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 -- 172.21.15.76:0/2483868427 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e940b0f60 con 0x7f5e940d2e90 2024-09-19T04:19:33.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e93fff640 1 --2- 172.21.15.76:0/2483868427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940d2e90 0x7f5e940cef10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009870 tx=0x7f5e8001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d4d0aba44e63c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.944+0000 7f5e937fe640 1 -- 172.21.15.76:0/2483868427 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800aa600 con 0x7f5e940d2e90 2024-09-19T04:19:33.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e937fe640 1 -- 172.21.15.76:0/2483868427 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e800af070 con 0x7f5e940d2e90 2024-09-19T04:19:33.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e937fe640 1 -- 172.21.15.76:0/2483868427 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e800aa9c0 con 0x7f5e940d2e90 2024-09-19T04:19:33.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- 172.21.15.76:0/2483868427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940d2e90 msgr2=0x7f5e940cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/2483868427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940d2e90 0x7f5e940cef10 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f5e80009870 tx=0x7f5e8001c4a0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- 172.21.15.76:0/2483868427 shutdown_connections 2024-09-19T04:19:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/2483868427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940d2e90 0x7f5e940cef10 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/2483868427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e940d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- 172.21.15.76:0/2483868427 >> 172.21.15.76:0/2483868427 conn(0x7f5e9408bf90 msgr2=0x7f5e9409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:33.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- 172.21.15.76:0/2483868427 shutdown_connections 2024-09-19T04:19:33.951 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- 172.21.15.76:0/2483868427 wait complete. 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 Processor -- start 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- start start 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940cf450 0x7f5e9414f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e940d2e90 0x7f5e94150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e94151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5e9406d190 con 0x7f5e940cf450 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5e940d1930 con 0x7f5e940d2e90 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5e940cf880 con 0x7f5e94161a60 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940cf450 0x7f5e9414f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940cf450 0x7f5e9414f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51914/0 (socket says 172.21.15.76:51914) 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 -- 172.21.15.76:0/1732137563 learned_addr learned my addr 172.21.15.76:0/1732137563 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e990ce640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e94151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e93fff640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e940d2e90 0x7f5e94150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 -- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 msgr2=0x7f5e94151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e94151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 -- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e940d2e90 msgr2=0x7f5e94150dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e940d2e90 0x7f5e94150dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 -- 172.21.15.76:0/1732137563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e94152c40 con 0x7f5e940cf450 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e988cd640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940cf450 0x7f5e9414f610 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800d5f0 tx=0x7f5e8800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e917fa640 1 -- 172.21.15.76:0/1732137563 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880125f0 con 0x7f5e940cf450 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e917fa640 1 -- 172.21.15.76:0/1732137563 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e88019070 con 0x7f5e940cf450 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e94152f20 con 0x7f5e940cf450 2024-09-19T04:19:33.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e917fa640 1 -- 172.21.15.76:0/1732137563 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e880129b0 con 0x7f5e940cf450 2024-09-19T04:19:33.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.948+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e941534e0 con 0x7f5e940cf450 2024-09-19T04:19:33.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.952+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e940cef10 con 0x7f5e940cf450 2024-09-19T04:19:33.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.952+0000 7f5e917fa640 1 -- 172.21.15.76:0/1732137563 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5e88012b50 con 0x7f5e940cf450 2024-09-19T04:19:33.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.952+0000 7f5e917fa640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e780678f0 0x7f5e78069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:33.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.952+0000 7f5e917fa640 1 -- 172.21.15.76:0/1732137563 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6475+0+0 (secure 0 0 0) 0x7f5e88086380 con 0x7f5e940cf450 2024-09-19T04:19:33.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.952+0000 7f5e93fff640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e780678f0 0x7f5e78069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:33.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.952+0000 7f5e93fff640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e780678f0 0x7f5e78069db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5e800ae000 tx=0x7f5e800a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:33.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:33.956+0000 7f5e917fa640 1 -- 172.21.15.76:0/1732137563 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5e88056bc0 con 0x7f5e940cf450 2024-09-19T04:19:34.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.160+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5e94150ba0 con 0x7f5e940cf450 2024-09-19T04:19:34.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.808+0000 7f5e917fa640 1 -- 172.21.15.76:0/1732137563 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) ==== 104+0+0 (secure 0 0 0) 0x7f5e880564c0 con 0x7f5e940cf450 2024-09-19T04:19:34.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e780678f0 msgr2=0x7f5e78069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:34.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e780678f0 0x7f5e78069db0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f5e800ae000 tx=0x7f5e800a7000 comp rx=0 tx=0).stop 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940cf450 msgr2=0x7f5e9414f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940cf450 0x7f5e9414f610 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800d5f0 tx=0x7f5e8800dab0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 shutdown_connections 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5e780678f0 0x7f5e78069db0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5e94161a60 0x7f5e94151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5e940d2e90 0x7f5e94150dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 --2- 172.21.15.76:0/1732137563 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5e940cf450 0x7f5e9414f610 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 >> 172.21.15.76:0/1732137563 conn(0x7f5e9408bf90 msgr2=0x7f5e94150090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 shutdown_connections 2024-09-19T04:19:34.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:34.812+0000 7f5e9ab31640 1 -- 172.21.15.76:0/1732137563 wait complete. 2024-09-19T04:19:34.833 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:19:35.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43aaa49640 1 Processor -- start 2024-09-19T04:19:35.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43aaa49640 1 -- start start 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43aaa49640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f43a40d3060 0x7f43a40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43aaa49640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a40d0300 con 0x7f43a40cf450 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43aaa49640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43a40b0f60 con 0x7f43a4161a60 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43aaa49640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f43a40cfd40 con 0x7f43a40d3060 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a8fe6640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f43a40cf450 0x7f43a406d6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36566/0 (socket says 172.21.15.76:36566) 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a8fe6640 1 -- 172.21.15.76:0/1209747530 learned_addr learned my addr 172.21.15.76:0/1209747530 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a37fe640 1 --2- 172.21.15.76:0/1209747530 >> v2:172.21.15.116:3300/0 conn(0x7f43a40d3060 0x7f43a40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2430830832 0 0) 0x7f439c001120 con 0x7f43a40cf450 2024-09-19T04:19:35.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4390003530 con 0x7f43a40cf450 2024-09-19T04:19:35.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2977946701 0 0) 0x7f4390003530 con 0x7f43a40cf450 2024-09-19T04:19:35.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 >> v2:172.21.15.116:3300/0 conn(0x7f43a40d3060 msgr2=0x7f43a40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:35.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 --2- 172.21.15.76:0/1209747530 >> v2:172.21.15.116:3300/0 conn(0x7f43a40d3060 0x7f43a40cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:35.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 >> v1:172.21.15.116:6789/0 conn(0x7f43a4161a60 legacy=0x7f43a40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a40d0020 con 0x7f43a40cf450 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f439c001850 con 0x7f43a40cf450 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 >> v1:172.21.15.76:6789/0 conn(0x7f43a40cf450 legacy=0x7f43a406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43900039c0 0x7f4390003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f4390004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43900051d0 0x7f439000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f43a40cfd40 con 0x7f4390004810 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f439000c170 con 0x7f43900051d0 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f439000c370 con 0x7f43900039c0 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a37fe640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f4390004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a8fe6640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43900051d0 0x7f439000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a3fff640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43900039c0 0x7f4390003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a37fe640 1 -- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43900039c0 msgr2=0x7f4390003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:35.013 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a37fe640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43900039c0 0x7f4390003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:35.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a37fe640 1 -- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43900051d0 msgr2=0x7f439000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:35.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a37fe640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43900051d0 0x7f439000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:35.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a8fe6640 1 -- 172.21.15.76:0/1209747530 reap_dead start 2024-09-19T04:19:35.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a37fe640 1 -- 172.21.15.76:0/1209747530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4390003530 con 0x7f4390004810 2024-09-19T04:19:35.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.008+0000 7f43a3fff640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43900039c0 0x7f4390003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43a37fe640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f4390004c90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f438c009fa0 tx=0x7f438c01cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd335d93f9c4bfa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c0b2070 con 0x7f4390004810 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f438c0a5cc0 con 0x7f4390004810 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43a2ffd640 1 -- 172.21.15.76:0/1209747530 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c009bf0 con 0x7f4390004810 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 -- 172.21.15.76:0/1209747530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 msgr2=0x7f4390004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f4390004c90 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f438c009fa0 tx=0x7f438c01cc10 comp rx=0 tx=0).stop 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 -- 172.21.15.76:0/1209747530 shutdown_connections 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 --2- 172.21.15.76:0/1209747530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f4390004c90 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:35.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 -- 172.21.15.76:0/1209747530 >> 172.21.15.76:0/1209747530 conn(0x7f43a408bf90 msgr2=0x7f43a409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:35.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 -- 172.21.15.76:0/1209747530 shutdown_connections 2024-09-19T04:19:35.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 -- 172.21.15.76:0/1209747530 wait complete. 2024-09-19T04:19:35.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 Processor -- start 2024-09-19T04:19:35.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.012+0000 7f43aaa49640 1 -- start start 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f43a4154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43a4155470 0x7f43a409ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a409ef80 0x7f43a40cd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f43a40d2ce0 con 0x7f4390004810 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f43a40d2b60 con 0x7f43a409ef80 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f43a40d2e60 con 0x7f43a4155470 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f43a4154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f43a4154f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51928/0 (socket says 172.21.15.76:51928) 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 -- 172.21.15.76:0/3011349790 learned_addr learned my addr 172.21.15.76:0/3011349790 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a8fe6640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a409ef80 0x7f43a40cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a37fe640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43a4155470 0x7f43a409ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 -- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43a4155470 msgr2=0x7f43a409ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43a4155470 0x7f43a409ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:35.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 -- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a409ef80 msgr2=0x7f43a40cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a409ef80 0x7f43a40cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 -- 172.21.15.76:0/3011349790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43a40a1270 con 0x7f4390004810 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a3fff640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f43a4154f30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f439400d4a0 tx=0x7f439400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a0ff9640 1 -- 172.21.15.76:0/3011349790 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394005280 con 0x7f4390004810 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a0ff9640 1 -- 172.21.15.76:0/3011349790 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4394005440 con 0x7f4390004810 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a0ff9640 1 -- 172.21.15.76:0/3011349790 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394007260 con 0x7f4390004810 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43a40a1550 con 0x7f4390004810 2024-09-19T04:19:35.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43a40a1b10 con 0x7f4390004810 2024-09-19T04:19:35.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43a40d1220 con 0x7f4390004810 2024-09-19T04:19:35.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a0ff9640 1 -- 172.21.15.76:0/3011349790 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4394019020 con 0x7f4390004810 2024-09-19T04:19:35.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a0ff9640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43840678f0 0x7f4384069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:35.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a0ff9640 1 -- 172.21.15.76:0/3011349790 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 6475+0+0 (secure 0 0 0) 0x7f43940856e0 con 0x7f4390004810 2024-09-19T04:19:35.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.016+0000 7f43a37fe640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43840678f0 0x7f4384069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:35.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.020+0000 7f43a37fe640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43840678f0 0x7f4384069db0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f438c0af000 tx=0x7f438c004ac0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:35.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.024+0000 7f43a0ff9640 1 -- 172.21.15.76:0/3011349790 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4394055f10 con 0x7f4390004810 2024-09-19T04:19:35.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:35.188+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f43a415efb0 con 0x7f4390004810 2024-09-19T04:19:36.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.072+0000 7f43a0ff9640 1 -- 172.21.15.76:0/3011349790 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7f4394055810 con 0x7f4390004810 2024-09-19T04:19:36.075 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:19:36.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43840678f0 msgr2=0x7f4384069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43840678f0 0x7f4384069db0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f438c0af000 tx=0x7f438c004ac0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 msgr2=0x7f43a4154f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f43a4154f30 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f439400d4a0 tx=0x7f439400d960 comp rx=0 tx=0).stop 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 shutdown_connections 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43840678f0 0x7f4384069db0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43a409ef80 0x7f43a40cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43a4155470 0x7f43a409ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 --2- 172.21.15.76:0/3011349790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4390004810 0x7f43a4154f30 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 >> 172.21.15.76:0/3011349790 conn(0x7f43a408bf90 msgr2=0x7f43a40b13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 shutdown_connections 2024-09-19T04:19:36.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.076+0000 7f43aaa49640 1 -- 172.21.15.76:0/3011349790 wait complete. 2024-09-19T04:19:36.101 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:19:36.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.224+0000 7f849311e640 1 Processor -- start 2024-09-19T04:19:36.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.224+0000 7f849311e640 1 -- start start 2024-09-19T04:19:36.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.224+0000 7f849311e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f848c15c2a0 0x7f848c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.224+0000 7f849311e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0af2a0 0x7f848c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.224+0000 7f849311e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f848c0b3290 con 0x7f848c15bec0 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.224+0000 7f849311e640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f848c0b3570 con 0x7f848c15c2a0 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.224+0000 7f849311e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f848c0b3850 con 0x7f848c0af2a0 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f8490eba640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f848c15bec0 0x7f848c0ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36576/0 (socket says 172.21.15.76:36576) 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f8490eba640 1 -- 172.21.15.76:0/1637888863 learned_addr learned my addr 172.21.15.76:0/1637888863 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0af2a0 0x7f848c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848bfff640 1 --2- 172.21.15.76:0/1637888863 >> v2:172.21.15.116:3300/0 conn(0x7f848c15c2a0 0x7f848c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2400134323 0 0) 0x7f848c0b3290 con 0x7f848c15bec0 2024-09-19T04:19:36.228 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8474003530 con 0x7f848c15bec0 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3237376809 0 0) 0x7f8474003530 con 0x7f848c15bec0 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0af2a0 msgr2=0x7f848c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0af2a0 0x7f848c06e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 >> v2:172.21.15.116:3300/0 conn(0x7f848c15c2a0 msgr2=0x7f848c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 --2- 172.21.15.76:0/1637888863 >> v2:172.21.15.116:3300/0 conn(0x7f848c15c2a0 0x7f848c0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c0b3b30 con 0x7f848c15bec0 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f8480001680 con 0x7f848c15bec0 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 >> v1:172.21.15.76:6789/0 conn(0x7f848c15bec0 legacy=0x7f848c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84740039c0 0x7f8474003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0af2a0 0x7f8474004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8474004d50 0x7f84740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f848c0b3570 con 0x7f8474004d50 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f848c0b3850 con 0x7f84740039c0 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f847400b9d0 con 0x7f848c0af2a0 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8474004d50 0x7f84740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f8490eba640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84740039c0 0x7f8474003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 -- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0af2a0 msgr2=0x7f8474004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0af2a0 0x7f8474004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 -- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84740039c0 msgr2=0x7f8474003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84740039c0 0x7f8474003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 -- 172.21.15.76:0/1637888863 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8474003530 con 0x7f8474004d50 2024-09-19T04:19:36.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f84916bb640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8474004d50 0x7f84740051d0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8484009fc0 tx=0x7f848401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece77beeef5ab0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84840b2070 con 0x7f8474004d50 2024-09-19T04:19:36.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84840a5cd0 con 0x7f8474004d50 2024-09-19T04:19:36.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.228+0000 7f848b7fe640 1 -- 172.21.15.76:0/1637888863 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484009c10 con 0x7f8474004d50 2024-09-19T04:19:36.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1637888863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8474004d50 msgr2=0x7f84740051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 --2- 172.21.15.76:0/1637888863 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8474004d50 0x7f84740051d0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f8484009fc0 tx=0x7f848401ccc0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 -- 172.21.15.76:0/1637888863 reap_dead start 2024-09-19T04:19:36.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1637888863 shutdown_connections 2024-09-19T04:19:36.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1637888863 >> 172.21.15.76:0/1637888863 conn(0x7f848c0b1f90 msgr2=0x7f848c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:36.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1637888863 shutdown_connections 2024-09-19T04:19:36.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1637888863 wait complete. 2024-09-19T04:19:36.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 Processor -- start 2024-09-19T04:19:36.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- start start 2024-09-19T04:19:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0cec80 0x7f848c1e1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f848c1e13f0 0x7f848c0cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c0d0150 0x7f848c071f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f848c0af730 con 0x7f848c0d0150 2024-09-19T04:19:36.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f848c0b40f0 con 0x7f848c1e13f0 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f848c0afc90 con 0x7f848c0cec80 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c0d0150 0x7f848c071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c0d0150 0x7f848c071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51936/0 (socket says 172.21.15.76:51936) 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 -- 172.21.15.76:0/1530493999 learned_addr learned my addr 172.21.15.76:0/1530493999 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f848bfff640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f848c1e13f0 0x7f848c0cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 -- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0cec80 msgr2=0x7f848c1e1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f8490eba640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0cec80 0x7f848c1e1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0cec80 0x7f848c1e1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 -- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f848c1e13f0 msgr2=0x7f848c0cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f848c1e13f0 0x7f848c0cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 -- 172.21.15.76:0/1530493999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f848c151930 con 0x7f848c0d0150 2024-09-19T04:19:36.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f848bfff640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f848c1e13f0 0x7f848c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84916bb640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c0d0150 0x7f848c071f10 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8484002240 tx=0x7f8484009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84897fa640 1 -- 172.21.15.76:0/1530493999 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84840bd070 con 0x7f848c0d0150 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84897fa640 1 -- 172.21.15.76:0/1530493999 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84840a5ad0 con 0x7f848c0d0150 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f848c151c10 con 0x7f848c0d0150 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f84897fa640 1 -- 172.21.15.76:0/1530493999 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84840b2040 con 0x7f848c0d0150 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f848c1521d0 con 0x7f848c0d0150 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.232+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f848c0aed60 con 0x7f848c0d0150 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.236+0000 7f84897fa640 1 -- 172.21.15.76:0/1530493999 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f84840c0070 con 0x7f848c0d0150 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.236+0000 7f84897fa640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84780678c0 0x7f8478069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.236+0000 7f8490eba640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84780678c0 0x7f8478069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:36.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.236+0000 7f84897fa640 1 -- 172.21.15.76:0/1530493999 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 6037+0+0 (secure 0 0 0) 0x7f848411fbd0 con 0x7f848c0d0150 2024-09-19T04:19:36.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.236+0000 7f8490eba640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84780678c0 0x7f8478069d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8480000f70 tx=0x7f8480009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:36.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.240+0000 7f84897fa640 1 -- 172.21.15.76:0/1530493999 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f84840169d0 con 0x7f848c0d0150 2024-09-19T04:19:36.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:36.404+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f848c06fcd0 con 0x7f848c0d0150 2024-09-19T04:19:37.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.084+0000 7f84897fa640 1 -- 172.21.15.76:0/1530493999 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7f84840efee0 con 0x7f848c0d0150 2024-09-19T04:19:37.084 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-09-19T04:19:37.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.084+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84780678c0 msgr2=0x7f8478069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.084+0000 7f849311e640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84780678c0 0x7f8478069d80 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8480000f70 tx=0x7f8480009040 comp rx=0 tx=0).stop 2024-09-19T04:19:37.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.084+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c0d0150 msgr2=0x7f848c071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.084+0000 7f849311e640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c0d0150 0x7f848c071f10 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f8484002240 tx=0x7f8484009750 comp rx=0 tx=0).stop 2024-09-19T04:19:37.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 shutdown_connections 2024-09-19T04:19:37.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84780678c0 0x7f8478069d80 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f848c0d0150 0x7f848c071f10 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f848c1e13f0 0x7f848c0cfc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 --2- 172.21.15.76:0/1530493999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f848c0cec80 0x7f848c1e1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 >> 172.21.15.76:0/1530493999 conn(0x7f848c0b1f90 msgr2=0x7f848c0afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:37.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 shutdown_connections 2024-09-19T04:19:37.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.088+0000 7f849311e640 1 -- 172.21.15.76:0/1530493999 wait complete. 2024-09-19T04:19:37.111 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T04:19:37.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.236+0000 7fd0420a3640 1 Processor -- start 2024-09-19T04:19:37.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.236+0000 7fd0420a3640 1 -- start start 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd0420a3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd03c1627d0 0x7fd03c09aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd0420a3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd03c0b40c0 0x7fd03c163f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd0420a3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd03c073740 con 0x7fd03c06bcc0 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd0420a3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd03c0b4630 con 0x7fd03c0b40c0 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd0420a3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd03c0b6050 con 0x7fd03c1627d0 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd03c0b40c0 0x7fd03c163f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd03c0b40c0 0x7fd03c163f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51942/0 (socket says 172.21.15.76:51942) 2024-09-19T04:19:37.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 -- 172.21.15.76:0/1052844570 learned_addr learned my addr 172.21.15.76:0/1052844570 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:37.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 -- 172.21.15.76:0/1052844570 >> v2:172.21.15.116:3300/0 conn(0x7fd03c1627d0 msgr2=0x7fd03c09aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:37.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 --2- 172.21.15.76:0/1052844570 >> v2:172.21.15.116:3300/0 conn(0x7fd03c1627d0 0x7fd03c09aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 -- 172.21.15.76:0/1052844570 >> v1:172.21.15.116:6789/0 conn(0x7fd03c06bcc0 legacy=0x7fd03c166850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 -- 172.21.15.76:0/1052844570 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c0b6330 con 0x7fd03c0b40c0 2024-09-19T04:19:37.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03affd640 1 --2- 172.21.15.76:0/1052844570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c163f90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd03c073030 tx=0x7fd02c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842702042950f441 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:37.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03a7fc640 1 -- 172.21.15.76:0/1052844570 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0b0070 con 0x7fd03c0b40c0 2024-09-19T04:19:37.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03a7fc640 1 -- 172.21.15.76:0/1052844570 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02c0ab600 con 0x7fd03c0b40c0 2024-09-19T04:19:37.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd03a7fc640 1 -- 172.21.15.76:0/1052844570 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0ab9d0 con 0x7fd03c0b40c0 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd0420a3640 1 -- 172.21.15.76:0/1052844570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 msgr2=0x7fd03c163f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.240+0000 7fd0420a3640 1 --2- 172.21.15.76:0/1052844570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c163f90 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fd03c073030 tx=0x7fd02c01c520 comp rx=0 tx=0).stop 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- 172.21.15.76:0/1052844570 shutdown_connections 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 --2- 172.21.15.76:0/1052844570 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c163f90 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 --2- 172.21.15.76:0/1052844570 >> v2:172.21.15.116:3300/0 conn(0x7fd03c1627d0 0x7fd03c09aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- 172.21.15.76:0/1052844570 >> 172.21.15.76:0/1052844570 conn(0x7fd03c0b6800 msgr2=0x7fd03c0b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- 172.21.15.76:0/1052844570 shutdown_connections 2024-09-19T04:19:37.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- 172.21.15.76:0/1052844570 wait complete. 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 Processor -- start 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- start start 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd03c06bcc0 0x7fd03c14df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd03c1627d0 0x7fd03c0afd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c163d40 con 0x7fd03c0b40c0 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c163bc0 con 0x7fd03c06bcc0 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c170020 con 0x7fd03c1627d0 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51954/0 (socket says 172.21.15.76:51954) 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 -- 172.21.15.76:0/3273971736 learned_addr learned my addr 172.21.15.76:0/3273971736 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:37.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03bfff640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd03c1627d0 0x7fd03c0afd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03b7fe640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd03c06bcc0 0x7fd03c14df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 -- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd03c1627d0 msgr2=0x7fd03c0afd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd03c1627d0 0x7fd03c0afd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 -- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd03c06bcc0 msgr2=0x7fd03c14df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd03c06bcc0 0x7fd03c14df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 -- 172.21.15.76:0/3273971736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c0b04d0 con 0x7fd03c0b40c0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd03affd640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c158d50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd02c01ca00 tx=0x7fd02c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd023fff640 1 -- 172.21.15.76:0/3273971736 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0ba020 con 0x7fd03c0b40c0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd023fff640 1 -- 172.21.15.76:0/3273971736 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd02c0b6620 con 0x7fd03c0b40c0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c152ae0 con 0x7fd03c0b40c0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd023fff640 1 -- 172.21.15.76:0/3273971736 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c0b0040 con 0x7fd03c0b40c0 2024-09-19T04:19:37.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c1657b0 con 0x7fd03c0b40c0 2024-09-19T04:19:37.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.244+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03c163f90 con 0x7fd03c0b40c0 2024-09-19T04:19:37.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.248+0000 7fd023fff640 1 -- 172.21.15.76:0/3273971736 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd02c0bd020 con 0x7fd03c0b40c0 2024-09-19T04:19:37.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.248+0000 7fd023fff640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0280678c0 0x7fd028069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.248+0000 7fd023fff640 1 -- 172.21.15.76:0/3273971736 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7fd02c1202c0 con 0x7fd03c0b40c0 2024-09-19T04:19:37.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.248+0000 7fd03b7fe640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0280678c0 0x7fd028069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.248+0000 7fd03b7fe640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0280678c0 0x7fd028069d80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd03000a790 tx=0x7fd030008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:37.252 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.252+0000 7fd023fff640 1 -- 172.21.15.76:0/3273971736 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd02c0c4030 con 0x7fd03c0b40c0 2024-09-19T04:19:37.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.412+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fd03c06e7c0 con 0x7fd03c0b40c0 2024-09-19T04:19:37.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd023fff640 1 -- 172.21.15.76:0/3273971736 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fd02c016110 con 0x7fd03c0b40c0 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0280678c0 msgr2=0x7fd028069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0280678c0 0x7fd028069d80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd03000a790 tx=0x7fd030008040 comp rx=0 tx=0).stop 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 msgr2=0x7fd03c158d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c158d50 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fd02c01ca00 tx=0x7fd02c01ad60 comp rx=0 tx=0).stop 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 shutdown_connections 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd0280678c0 0x7fd028069d80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd03c1627d0 0x7fd03c0afd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd03c0b40c0 0x7fd03c158d50 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 --2- 172.21.15.76:0/3273971736 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd03c06bcc0 0x7fd03c14df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 >> 172.21.15.76:0/3273971736 conn(0x7fd03c0b6800 msgr2=0x7fd03c166850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 shutdown_connections 2024-09-19T04:19:37.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.420+0000 7fd0420a3640 1 -- 172.21.15.76:0/3273971736 wait complete. 2024-09-19T04:19:37.440 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0' 2024-09-19T04:19:37.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d8384640 1 Processor -- start 2024-09-19T04:19:37.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d8384640 1 -- start start 2024-09-19T04:19:37.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d8384640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1d015c2a0 0x7ff1d00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d8384640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d00b3290 con 0x7ff1d00af2a0 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d8384640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1d00b3570 con 0x7ff1d015bec0 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d8384640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff1d00b3850 con 0x7ff1d015c2a0 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6921640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff1d00af2a0 0x7ff1d006e000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36578/0 (socket says 172.21.15.76:36578) 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6921640 1 -- 172.21.15.76:0/113793349 learned_addr learned my addr 172.21.15.76:0/113793349 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d591f640 1 --2- 172.21.15.76:0/113793349 >> v2:172.21.15.116:3300/0 conn(0x7ff1d015c2a0 0x7ff1d00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3049256868 0 0) 0x7ff1d00b3290 con 0x7ff1d00af2a0 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff1bc003530 con 0x7ff1d00af2a0 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3418187484 0 0) 0x7ff1bc003530 con 0x7ff1d00af2a0 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 >> v2:172.21.15.116:3300/0 conn(0x7ff1d015c2a0 msgr2=0x7ff1d00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 --2- 172.21.15.76:0/113793349 >> v2:172.21.15.116:3300/0 conn(0x7ff1d015c2a0 0x7ff1d00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 >> v1:172.21.15.116:6789/0 conn(0x7ff1d015bec0 legacy=0x7ff1d00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d00b3b30 con 0x7ff1d00af2a0 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7ff1c8001a00 con 0x7ff1d00af2a0 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 >> v1:172.21.15.76:6789/0 conn(0x7ff1d00af2a0 legacy=0x7ff1d006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1bc0039c0 0x7ff1bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc004810 0x7ff1bc004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1bc0051d0 0x7ff1bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00b3850 con 0x7ff1bc0039c0 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc00c070 con 0x7ff1bc004810 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1bc00c270 con 0x7ff1bc0051d0 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6120640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1bc0039c0 0x7ff1bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6921640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1bc0051d0 0x7ff1bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d591f640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc004810 0x7ff1bc004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6120640 1 -- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1bc0051d0 msgr2=0x7ff1bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:37.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6120640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1bc0051d0 0x7ff1bc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6120640 1 -- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc004810 msgr2=0x7ff1bc004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:37.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6120640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc004810 0x7ff1bc004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6120640 1 -- 172.21.15.76:0/113793349 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc003530 con 0x7ff1bc0039c0 2024-09-19T04:19:37.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.564+0000 7ff1d6921640 1 -- 172.21.15.76:0/113793349 reap_dead start 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d6120640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1bc0039c0 0x7ff1bc003de0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc001780 tx=0x7ff1cc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27443978c321f1e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0013a0 con 0x7ff1bc0039c0 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1cc0052f0 con 0x7ff1bc0039c0 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d511e640 1 -- 172.21.15.76:0/113793349 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc003840 con 0x7ff1bc0039c0 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- 172.21.15.76:0/113793349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1bc0039c0 msgr2=0x7ff1bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1bc0039c0 0x7ff1bc003de0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc001780 tx=0x7ff1cc01c3d0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- 172.21.15.76:0/113793349 shutdown_connections 2024-09-19T04:19:37.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 --2- 172.21.15.76:0/113793349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1bc0039c0 0x7ff1bc003de0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- 172.21.15.76:0/113793349 >> 172.21.15.76:0/113793349 conn(0x7ff1d00b1f90 msgr2=0x7ff1d00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:37.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- 172.21.15.76:0/113793349 shutdown_connections 2024-09-19T04:19:37.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- 172.21.15.76:0/113793349 wait complete. 2024-09-19T04:19:37.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 Processor -- start 2024-09-19T04:19:37.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- start start 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc0039c0 0x7ff1d014d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1d015c6e0 0x7ff1d015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1d0152460 0x7ff1d01528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00af680 con 0x7ff1d0152460 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d00ad610 con 0x7ff1bc0039c0 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d8384640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d0071f10 con 0x7ff1d015c6e0 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d6921640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1d0152460 0x7ff1d01528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d6921640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1d0152460 0x7ff1d01528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51966/0 (socket says 172.21.15.76:51966) 2024-09-19T04:19:37.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.568+0000 7ff1d6921640 1 -- 172.21.15.76:0/832587344 learned_addr learned my addr 172.21.15.76:0/832587344 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d591f640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1d015c6e0 0x7ff1d015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6120640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc0039c0 0x7ff1d014d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6921640 1 -- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1d015c6e0 msgr2=0x7ff1d015cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6921640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1d015c6e0 0x7ff1d015cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6921640 1 -- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc0039c0 msgr2=0x7ff1d014d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6921640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc0039c0 0x7ff1d014d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6921640 1 -- 172.21.15.76:0/832587344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d0152e20 con 0x7ff1d0152460 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6921640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1d0152460 0x7ff1d01528e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7ff1d015c680 tx=0x7ff1c8007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1c6ffd640 1 -- 172.21.15.76:0/832587344 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800eaa0 con 0x7ff1d0152460 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1c6ffd640 1 -- 172.21.15.76:0/832587344 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1c800ec80 con 0x7ff1d0152460 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d0072160 con 0x7ff1d0152460 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1c6ffd640 1 -- 172.21.15.76:0/832587344 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8014430 con 0x7ff1d0152460 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d0072720 con 0x7ff1d0152460 2024-09-19T04:19:37.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1d0160470 con 0x7ff1d0152460 2024-09-19T04:19:37.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1c6ffd640 1 -- 172.21.15.76:0/832587344 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff1c80145d0 con 0x7ff1d0152460 2024-09-19T04:19:37.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1c6ffd640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:37.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6120640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:37.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1c6ffd640 1 -- 172.21.15.76:0/832587344 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7ff1c80858e0 con 0x7ff1d0152460 2024-09-19T04:19:37.575 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.572+0000 7ff1d6120640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc001750 tx=0x7ff1cc0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:37.578 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.576+0000 7ff1c6ffd640 1 -- 172.21.15.76:0/832587344 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff1c80564e0 con 0x7ff1d0152460 2024-09-19T04:19:37.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:37.800+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]} v 0) -- 0x7ff1d006fcd0 con 0x7ff1d0152460 2024-09-19T04:19:38.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.084+0000 7ff1c6ffd640 1 -- 172.21.15.76:0/832587344 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_mds0"]}]=0 v319) ==== 154+0+0 (secure 0 0 0) 0x7ff1c8055de0 con 0x7ff1d0152460 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 msgr2=0x7ff1b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc001750 tx=0x7ff1cc0011a0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1d0152460 msgr2=0x7ff1d01528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1d0152460 0x7ff1d01528e0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7ff1d015c680 tx=0x7ff1c8007bc0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 shutdown_connections 2024-09-19T04:19:38.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1b40678c0 0x7ff1b4069d80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff1d0152460 0x7ff1d01528e0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff1d015c6e0 0x7ff1d015cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 --2- 172.21.15.76:0/832587344 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1bc0039c0 0x7ff1d014d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 >> 172.21.15.76:0/832587344 conn(0x7ff1d00b1f90 msgr2=0x7ff1d00b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 shutdown_connections 2024-09-19T04:19:38.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.088+0000 7ff1d8384640 1 -- 172.21.15.76:0/832587344 wait complete. 2024-09-19T04:19:38.109 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-19T04:19:38.109 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) 2024-09-19T04:19:38.109 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:19:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f23739e8640 1 Processor -- start 2024-09-19T04:19:38.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f23739e8640 1 -- start start 2024-09-19T04:19:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f23739e8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 0x7f236c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f23739e8640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f236c15c2a0 0x7f236c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f23739e8640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f236c0b3290 con 0x7f236c0af2a0 2024-09-19T04:19:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f23739e8640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f236c0b3570 con 0x7f236c15c2a0 2024-09-19T04:19:38.235 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f23739e8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f236c0b3850 con 0x7f236c15bec0 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371f85640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f236c0af2a0 0x7f236c15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36586/0 (socket says 172.21.15.76:36586) 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371f85640 1 -- 172.21.15.76:0/2566378107 learned_addr learned my addr 172.21.15.76:0/2566378107 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371784640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 0x7f236c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2370f83640 1 --2- 172.21.15.76:0/2566378107 >> v2:172.21.15.116:3300/0 conn(0x7f236c15c2a0 0x7f236c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2568962477 0 0) 0x7f236c0b3290 con 0x7f236c0af2a0 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2358003530 con 0x7f236c0af2a0 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3730439034 0 0) 0x7f2358003530 con 0x7f236c0af2a0 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 msgr2=0x7f236c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 0x7f236c0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 >> v2:172.21.15.116:3300/0 conn(0x7f236c15c2a0 msgr2=0x7f236c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.236 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 --2- 172.21.15.76:0/2566378107 >> v2:172.21.15.116:3300/0 conn(0x7f236c15c2a0 0x7f236c0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c0b3b30 con 0x7f236c0af2a0 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2364001a00 con 0x7f236c0af2a0 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 >> v1:172.21.15.76:6789/0 conn(0x7f236c0af2a0 legacy=0x7f236c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23580039c0 0x7f2358003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 0x7f2358004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2358004d50 0x7f23580051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f236c0b3570 con 0x7f23580039c0 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f236c0b3850 con 0x7f2358004d50 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f235800b9d0 con 0x7f236c15bec0 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371784640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23580039c0 0x7f2358003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2370f83640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 0x7f2358004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371f85640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2358004d50 0x7f23580051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371784640 1 -- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 msgr2=0x7f2358004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371784640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f236c15bec0 0x7f2358004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371784640 1 -- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2358004d50 msgr2=0x7f23580051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371784640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2358004d50 0x7f23580051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371784640 1 -- 172.21.15.76:0/2566378107 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2358003530 con 0x7f23580039c0 2024-09-19T04:19:38.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.232+0000 7f2371f85640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2358004d50 0x7f23580051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f2371784640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23580039c0 0x7f2358003de0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2368009ae0 tx=0x7f236801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdee36a86cffacc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23680b2070 con 0x7f23580039c0 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23680a5610 con 0x7f23580039c0 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f2363fff640 1 -- 172.21.15.76:0/2566378107 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23680096b0 con 0x7f23580039c0 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 -- 172.21.15.76:0/2566378107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23580039c0 msgr2=0x7f2358003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 --2- 172.21.15.76:0/2566378107 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f23580039c0 0x7f2358003de0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f2368009ae0 tx=0x7f236801ca40 comp rx=0 tx=0).stop 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f2371f85640 1 -- 172.21.15.76:0/2566378107 reap_dead start 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 -- 172.21.15.76:0/2566378107 shutdown_connections 2024-09-19T04:19:38.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 -- 172.21.15.76:0/2566378107 >> 172.21.15.76:0/2566378107 conn(0x7f236c0b1f90 msgr2=0x7f236c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 -- 172.21.15.76:0/2566378107 shutdown_connections 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 -- 172.21.15.76:0/2566378107 wait complete. 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 Processor -- start 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.236+0000 7f23739e8640 1 -- start start 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23580039c0 0x7f236c156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c158900 0x7f236c14c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f236c14ebb0 0x7f236c14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f236c0ad610 con 0x7f236c14ebb0 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f236c0af6d0 con 0x7f236c158900 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f236c0ad790 con 0x7f23580039c0 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f236c14ebb0 0x7f236c14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f236c14ebb0 0x7f236c14f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51984/0 (socket says 172.21.15.76:51984) 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 -- 172.21.15.76:0/4086240088 learned_addr learned my addr 172.21.15.76:0/4086240088 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371784640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23580039c0 0x7f236c156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2370f83640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c158900 0x7f236c14c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 -- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23580039c0 msgr2=0x7f236c156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23580039c0 0x7f236c156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 -- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c158900 msgr2=0x7f236c14c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c158900 0x7f236c14c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 -- 172.21.15.76:0/4086240088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c158d80 con 0x7f236c14ebb0 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371784640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23580039c0 0x7f236c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2370f83640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c158900 0x7f236c14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371f85640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f236c14ebb0 0x7f236c14f060 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f236c071880 tx=0x7f236400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2361ffb640 1 -- 172.21.15.76:0/4086240088 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364004d70 con 0x7f236c14ebb0 2024-09-19T04:19:38.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2361ffb640 1 -- 172.21.15.76:0/4086240088 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f236400bb20 con 0x7f236c14ebb0 2024-09-19T04:19:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c157030 con 0x7f236c14ebb0 2024-09-19T04:19:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2361ffb640 1 -- 172.21.15.76:0/4086240088 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364007110 con 0x7f236c14ebb0 2024-09-19T04:19:38.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1575f0 con 0x7f236c14ebb0 2024-09-19T04:19:38.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f236c0aed60 con 0x7f236c14ebb0 2024-09-19T04:19:38.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2361ffb640 1 -- 172.21.15.76:0/4086240088 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f236401b020 con 0x7f236c14ebb0 2024-09-19T04:19:38.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2361ffb640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23500678c0 0x7f2350069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2361ffb640 1 -- 172.21.15.76:0/4086240088 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f2364085cc0 con 0x7f236c14ebb0 2024-09-19T04:19:38.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371784640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23500678c0 0x7f2350069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.245 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.240+0000 7f2371784640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23500678c0 0x7f2350069d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f23680af000 tx=0x7f23680046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.244+0000 7f2361ffb640 1 -- 172.21.15.76:0/4086240088 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2364056950 con 0x7f236c14ebb0 2024-09-19T04:19:38.410 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.408+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f236c06fcd0 con 0x7f236c14ebb0 2024-09-19T04:19:38.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f2361ffb640 1 -- 172.21.15.76:0/4086240088 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+13590 (secure 0 0 0) 0x7f2364056250 con 0x7f236c14ebb0 2024-09-19T04:19:38.412 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:38.413 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":96,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:19:37.077208+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.116:6833/2189465635":"2024-09-20T04:19:33.800616+0000","172.21.15.116:6832/2189465635":"2024-09-20T04:19:33.800616+0000","172.21.15.76:6837/2711186256":"2024-09-20T04:19:33.800683+0000","172.21.15.76:6835/987394316":"2024-09-20T04:19:33.800653+0000","172.21.15.76:6836/2711186256":"2024-09-20T04:19:33.800683+0000","172.21.15.76:6834/987394316":"2024-09-20T04:19:33.800653+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23500678c0 msgr2=0x7f2350069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23500678c0 0x7f2350069d80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f23680af000 tx=0x7f23680046d0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f236c14ebb0 msgr2=0x7f236c14f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f236c14ebb0 0x7f236c14f060 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f236c071880 tx=0x7f236400b4a0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 shutdown_connections 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f23500678c0 0x7f2350069d80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f236c14ebb0 0x7f236c14f060 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f236c158900 0x7f236c14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 --2- 172.21.15.76:0/4086240088 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f23580039c0 0x7f236c156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 >> 172.21.15.76:0/4086240088 conn(0x7f236c0b1f90 msgr2=0x7f236c0b15e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 shutdown_connections 2024-09-19T04:19:38.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.412+0000 7f23739e8640 1 -- 172.21.15.76:0/4086240088 wait complete. 2024-09-19T04:19:38.432 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T04:13:13.812655+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T04:19:38.433 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T04:19:38.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.556+0000 7f292ce3f640 1 Processor -- start 2024-09-19T04:19:38.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.556+0000 7f292ce3f640 1 -- start start 2024-09-19T04:19:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.556+0000 7f292ce3f640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29280b3330 0x7f29280b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.556+0000 7f292ce3f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29280b51b0 0x7f29280b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.556+0000 7f292ce3f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29280b3c50 con 0x7f292815b570 2024-09-19T04:19:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.556+0000 7f292ce3f640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f29280d0b80 con 0x7f29280b3330 2024-09-19T04:19:38.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.556+0000 7f292ce3f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f29280d0e60 con 0x7f29280b51b0 2024-09-19T04:19:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f2925d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29280b51b0 0x7f29280b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292659c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29280b3330 0x7f29280b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f2925d9b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29280b51b0 0x7f29280b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:38104/0 (socket says 172.21.15.76:38104) 2024-09-19T04:19:38.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f2926d9d640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f292815b570 0x7f29280cfde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:39820/0 (socket says 172.21.15.76:39820) 2024-09-19T04:19:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f2925d9b640 1 -- 172.21.15.76:0/752842367 learned_addr learned my addr 172.21.15.76:0/752842367 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292659c640 1 -- 172.21.15.76:0/752842367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29280b51b0 msgr2=0x7f29280b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292659c640 1 --2- 172.21.15.76:0/752842367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29280b51b0 0x7f29280b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292659c640 1 -- 172.21.15.76:0/752842367 >> v1:172.21.15.116:6789/0 conn(0x7f292815b570 legacy=0x7f29280cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292659c640 1 -- 172.21.15.76:0/752842367 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29280b46d0 con 0x7f29280b3330 2024-09-19T04:19:38.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292659c640 1 --2- 172.21.15.76:0/752842367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b3330 0x7f29280b3710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d90 tx=0x7f291801c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffaee21e6646d0b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292559a640 1 -- 172.21.15.76:0/752842367 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29180b2070 con 0x7f29280b3330 2024-09-19T04:19:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292559a640 1 -- 172.21.15.76:0/752842367 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29180a57a0 con 0x7f29280b3330 2024-09-19T04:19:38.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.560+0000 7f292559a640 1 -- 172.21.15.76:0/752842367 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29180a5b70 con 0x7f29280b3330 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- 172.21.15.76:0/752842367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b3330 msgr2=0x7f29280b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 --2- 172.21.15.76:0/752842367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b3330 0x7f29280b3710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f2918009d90 tx=0x7f291801c8e0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- 172.21.15.76:0/752842367 shutdown_connections 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 --2- 172.21.15.76:0/752842367 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f29280b51b0 0x7f29280b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 --2- 172.21.15.76:0/752842367 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b3330 0x7f29280b3710 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- 172.21.15.76:0/752842367 >> 172.21.15.76:0/752842367 conn(0x7f29280b6870 msgr2=0x7f29280b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- 172.21.15.76:0/752842367 shutdown_connections 2024-09-19T04:19:38.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- 172.21.15.76:0/752842367 wait complete. 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 Processor -- start 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- start start 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29280b3330 0x7f292815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b51b0 0x7f292813c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f292815b570 0x7f2928138b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2928144e80 con 0x7f29280b3330 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2928144d00 con 0x7f29280b51b0 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2928168350 con 0x7f292815b570 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29280b3330 0x7f292815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29280b3330 0x7f292815a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52000/0 (socket says 172.21.15.76:52000) 2024-09-19T04:19:38.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 -- 172.21.15.76:0/3467514168 learned_addr learned my addr 172.21.15.76:0/3467514168 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f2926d9d640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f292815b570 0x7f2928138b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f2925d9b640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b51b0 0x7f292813c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 -- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f292815b570 msgr2=0x7f2928138b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f292815b570 0x7f2928138b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 -- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b51b0 msgr2=0x7f292813c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b51b0 0x7f292813c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 -- 172.21.15.76:0/3467514168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292813b050 con 0x7f29280b3330 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f2925d9b640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b51b0 0x7f292813c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292659c640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29280b3330 0x7f292815a250 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f29180b0040 tx=0x7f2918019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f2916ffd640 1 -- 172.21.15.76:0/3467514168 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29180b2050 con 0x7f29280b3330 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f2916ffd640 1 -- 172.21.15.76:0/3467514168 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29180be070 con 0x7f29280b3330 2024-09-19T04:19:38.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f2916ffd640 1 -- 172.21.15.76:0/3467514168 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29180ad850 con 0x7f29280b3330 2024-09-19T04:19:38.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292813b330 con 0x7f29280b3330 2024-09-19T04:19:38.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.564+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292813b8f0 con 0x7f29280b3330 2024-09-19T04:19:38.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.568+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29280b3710 con 0x7f29280b3330 2024-09-19T04:19:38.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.568+0000 7f2916ffd640 1 -- 172.21.15.76:0/3467514168 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f29180ad9f0 con 0x7f29280b3330 2024-09-19T04:19:38.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.568+0000 7f2916ffd640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080678c0 0x7f2908069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.568+0000 7f2916ffd640 1 -- 172.21.15.76:0/3467514168 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f2918120b10 con 0x7f29280b3330 2024-09-19T04:19:38.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.568+0000 7f2925d9b640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080678c0 0x7f2908069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.568+0000 7f2925d9b640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080678c0 0x7f2908069d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f29281416d0 tx=0x7f291c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.573 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.572+0000 7f2916ffd640 1 -- 172.21.15.76:0/3467514168 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2918016560 con 0x7f29280b3330 2024-09-19T04:19:38.733 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.732+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2928073200 con 0x7f29280b3330 2024-09-19T04:19:38.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.732+0000 7f2916ffd640 1 -- 172.21.15.76:0/3467514168 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7f29180f1860 con 0x7f29280b3330 2024-09-19T04:19:38.734 INFO:teuthology.orchestra.run.smithi076.stdout:pg_num: 1 2024-09-19T04:19:38.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080678c0 msgr2=0x7f2908069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080678c0 0x7f2908069d80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f29281416d0 tx=0x7f291c008040 comp rx=0 tx=0).stop 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29280b3330 msgr2=0x7f292815a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29280b3330 0x7f292815a250 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f29180b0040 tx=0x7f2918019680 comp rx=0 tx=0).stop 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 shutdown_connections 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f29080678c0 0x7f2908069d80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f292815b570 0x7f2928138b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29280b51b0 0x7f292813c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 --2- 172.21.15.76:0/3467514168 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f29280b3330 0x7f292815a250 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 >> 172.21.15.76:0/3467514168 conn(0x7f29280b6870 msgr2=0x7f2928141840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 shutdown_connections 2024-09-19T04:19:38.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.736+0000 7f292ce3f640 1 -- 172.21.15.76:0/3467514168 wait complete. 2024-09-19T04:19:38.752 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-09-19T04:19:38.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d2d706640 1 Processor -- start 2024-09-19T04:19:38.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d2d706640 1 -- start start 2024-09-19T04:19:38.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d2d706640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6d2806f010 0x7f6d28166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d2d706640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d280b3290 con 0x7f6d2815bec0 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d2d706640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d280b3570 con 0x7f6d2815c2a0 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d2d706640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6d280b3850 con 0x7f6d2806f010 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d26ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6d2815bec0 0x7f6d280ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36590/0 (socket says 172.21.15.76:36590) 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d26ffd640 1 -- 172.21.15.76:0/1129372194 learned_addr learned my addr 172.21.15.76:0/1129372194 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d277fe640 1 --2- 172.21.15.76:0/1129372194 >> v2:172.21.15.76:3300/0 conn(0x7f6d2806f010 0x7f6d28166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.868+0000 7f6d25ffb640 1 -- 172.21.15.76:0/1129372194 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 48818520 0 0) 0x7f6d280b3290 con 0x7f6d2815bec0 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d25ffb640 1 -- 172.21.15.76:0/1129372194 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d14003530 con 0x7f6d2815bec0 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d277fe640 1 -- 172.21.15.76:0/1129372194 >> v1:172.21.15.116:6789/0 conn(0x7f6d2815c2a0 legacy=0x7f6d280aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d277fe640 1 -- 172.21.15.76:0/1129372194 >> v1:172.21.15.76:6789/0 conn(0x7f6d2815bec0 legacy=0x7f6d280ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d277fe640 1 -- 172.21.15.76:0/1129372194 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d280b3b30 con 0x7f6d2806f010 2024-09-19T04:19:38.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d277fe640 1 --2- 172.21.15.76:0/1129372194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2806f010 0x7f6d28166ff0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097c0 tx=0x7f6d2001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a74f6095fb854d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d25ffb640 1 -- 172.21.15.76:0/1129372194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200b2070 con 0x7f6d2806f010 2024-09-19T04:19:38.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d25ffb640 1 -- 172.21.15.76:0/1129372194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d200ad680 con 0x7f6d2806f010 2024-09-19T04:19:38.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d25ffb640 1 -- 172.21.15.76:0/1129372194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200ada50 con 0x7f6d2806f010 2024-09-19T04:19:38.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d2d706640 1 -- 172.21.15.76:0/1129372194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2806f010 msgr2=0x7f6d28166ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1129372194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2806f010 0x7f6d28166ff0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f6d200097c0 tx=0x7f6d2001c470 comp rx=0 tx=0).stop 2024-09-19T04:19:38.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d2d706640 1 -- 172.21.15.76:0/1129372194 shutdown_connections 2024-09-19T04:19:38.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1129372194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2806f010 0x7f6d28166ff0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d2d706640 1 -- 172.21.15.76:0/1129372194 >> 172.21.15.76:0/1129372194 conn(0x7f6d280b1f90 msgr2=0x7f6d280b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:38.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d2d706640 1 -- 172.21.15.76:0/1129372194 shutdown_connections 2024-09-19T04:19:38.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.872+0000 7f6d2d706640 1 -- 172.21.15.76:0/1129372194 wait complete. 2024-09-19T04:19:38.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 Processor -- start 2024-09-19T04:19:38.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 -- start start 2024-09-19T04:19:38.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d2806f010 0x7f6d2815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d2815bec0 0x7f6d28152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2815c2a0 0x7f6d28155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28169ed0 con 0x7f6d2815c2a0 2024-09-19T04:19:38.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6d28169d50 con 0x7f6d2815bec0 2024-09-19T04:19:38.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2816a050 con 0x7f6d2806f010 2024-09-19T04:19:38.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2815c2a0 0x7f6d28155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2815c2a0 0x7f6d28155070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52014/0 (socket says 172.21.15.76:52014) 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 -- 172.21.15.76:0/1030422924 learned_addr learned my addr 172.21.15.76:0/1030422924 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d26ffd640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d2806f010 0x7f6d2815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d267fc640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d2815bec0 0x7f6d28152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 -- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d2806f010 msgr2=0x7f6d2815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d2806f010 0x7f6d2815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 -- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d2815bec0 msgr2=0x7f6d28152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d2815bec0 0x7f6d28152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 -- 172.21.15.76:0/1030422924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d28152690 con 0x7f6d2815c2a0 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d277fe640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2815c2a0 0x7f6d28155070 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6d20019080 tx=0x7f6d2001acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d1f7fe640 1 -- 172.21.15.76:0/1030422924 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200b2040 con 0x7f6d2815c2a0 2024-09-19T04:19:38.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d1f7fe640 1 -- 172.21.15.76:0/1030422924 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d200bd070 con 0x7f6d2815c2a0 2024-09-19T04:19:38.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d281f5160 con 0x7f6d2815c2a0 2024-09-19T04:19:38.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d1f7fe640 1 -- 172.21.15.76:0/1030422924 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d200add70 con 0x7f6d2815c2a0 2024-09-19T04:19:38.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2815c9c0 con 0x7f6d2815c2a0 2024-09-19T04:19:38.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d280b3290 con 0x7f6d2815c2a0 2024-09-19T04:19:38.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.876+0000 7f6d1f7fe640 1 -- 172.21.15.76:0/1030422924 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6d200c2460 con 0x7f6d2815c2a0 2024-09-19T04:19:38.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.880+0000 7f6d1f7fe640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d080678c0 0x7f6d08069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:38.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.880+0000 7f6d1f7fe640 1 -- 172.21.15.76:0/1030422924 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f6d2011fe30 con 0x7f6d2815c2a0 2024-09-19T04:19:38.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.880+0000 7f6d26ffd640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d080678c0 0x7f6d08069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:38.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.880+0000 7f6d26ffd640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d080678c0 0x7f6d08069d80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6d18001570 tx=0x7f6d18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:38.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:38.884+0000 7f6d1f7fe640 1 -- 172.21.15.76:0/1030422924 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6d20016090 con 0x7f6d2815c2a0 2024-09-19T04:19:39.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:39.096+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7f6d2806fcd0 con 0x7f6d2815c2a0 2024-09-19T04:19:40.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.088+0000 7f6d1f7fe640 1 -- 172.21.15.76:0/1030422924 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v321) ==== 168+0+0 (secure 0 0 0) 0x7f6d200f0a40 con 0x7f6d2815c2a0 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d080678c0 msgr2=0x7f6d08069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d080678c0 0x7f6d08069d80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f6d18001570 tx=0x7f6d18009040 comp rx=0 tx=0).stop 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2815c2a0 msgr2=0x7f6d28155070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2815c2a0 0x7f6d28155070 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6d20019080 tx=0x7f6d2001acb0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 shutdown_connections 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6d080678c0 0x7f6d08069d80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6d2815c2a0 0x7f6d28155070 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d2815bec0 0x7f6d28152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 --2- 172.21.15.76:0/1030422924 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6d2806f010 0x7f6d2815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 >> 172.21.15.76:0/1030422924 conn(0x7f6d280b1f90 msgr2=0x7f6d281580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:40.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 shutdown_connections 2024-09-19T04:19:40.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.092+0000 7f6d2d706640 1 -- 172.21.15.76:0/1030422924 wait complete. 2024-09-19T04:19:40.114 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-19T04:19:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bd1c2c640 1 Processor -- start 2024-09-19T04:19:40.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bd1c2c640 1 -- start start 2024-09-19T04:19:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bd1c2c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bd1c2c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7bcc0af470 0x7f7bcc06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bd1c2c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bcc0b3290 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bd1c2c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7bcc0b3570 con 0x7f7bcc0af470 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bd1c2c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7bcc0b3850 con 0x7f7bcc15bec0 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bcbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7bcc0af470 0x7f7bcc06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bcbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7bcc0af470 0x7f7bcc06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52016/0 (socket says 172.21.15.76:52016) 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bcbfff640 1 -- 172.21.15.76:0/1536184809 learned_addr learned my addr 172.21.15.76:0/1536184809 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bcb7fe640 1 --2- 172.21.15.76:0/1536184809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bcbfff640 1 -- 172.21.15.76:0/1536184809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 msgr2=0x7f7bcc0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bcbfff640 1 --2- 172.21.15.76:0/1536184809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.236+0000 7f7bcbfff640 1 -- 172.21.15.76:0/1536184809 >> v1:172.21.15.76:6789/0 conn(0x7f7bcc15c2a0 legacy=0x7f7bcc0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bcbfff640 1 -- 172.21.15.76:0/1536184809 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc0b3b30 con 0x7f7bcc0af470 2024-09-19T04:19:40.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bcbfff640 1 --2- 172.21.15.76:0/1536184809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc0af470 0x7f7bcc06e000 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009e50 tx=0x7f7bc401ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373ac18d0807d5b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bca7fc640 1 -- 172.21.15.76:0/1536184809 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40a58d0 con 0x7f7bcc0af470 2024-09-19T04:19:40.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bca7fc640 1 -- 172.21.15.76:0/1536184809 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc40a5a70 con 0x7f7bcc0af470 2024-09-19T04:19:40.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bca7fc640 1 -- 172.21.15.76:0/1536184809 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40a5e40 con 0x7f7bcc0af470 2024-09-19T04:19:40.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/1536184809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc0af470 msgr2=0x7f7bcc06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/1536184809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc0af470 0x7f7bcc06e000 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009e50 tx=0x7f7bc401ca00 comp rx=0 tx=0).stop 2024-09-19T04:19:40.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/1536184809 shutdown_connections 2024-09-19T04:19:40.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/1536184809 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc0af470 0x7f7bcc06e000 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/1536184809 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/1536184809 >> 172.21.15.76:0/1536184809 conn(0x7f7bcc0b1f90 msgr2=0x7f7bcc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:40.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/1536184809 shutdown_connections 2024-09-19T04:19:40.244 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.240+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/1536184809 wait complete. 2024-09-19T04:19:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 Processor -- start 2024-09-19T04:19:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 -- start start 2024-09-19T04:19:40.246 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bcc0af470 0x7f7bcc155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc15c2a0 0x7f7bcc14da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc06ed50 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc0ad740 con 0x7f7bcc0af470 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7bcc06eed0 con 0x7f7bcc15bec0 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc15c2a0 0x7f7bcc14da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc15c2a0 0x7f7bcc14da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52018/0 (socket says 172.21.15.76:52018) 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 -- 172.21.15.76:0/2678275284 learned_addr learned my addr 172.21.15.76:0/2678275284 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcaffd640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.247 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcb7fe640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bcc0af470 0x7f7bcc155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 -- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 msgr2=0x7f7bcc150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 -- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bcc0af470 msgr2=0x7f7bcc155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bcc0af470 0x7f7bcc155f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 -- 172.21.15.76:0/2678275284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc153950 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bcbfff640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc15c2a0 0x7f7bcc14da50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7bc401a630 tx=0x7f7bc40197a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bb3fff640 1 -- 172.21.15.76:0/2678275284 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40bc030 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bb3fff640 1 -- 172.21.15.76:0/2678275284 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc4007c20 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc153670 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.248 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bb3fff640 1 -- 172.21.15.76:0/2678275284 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40a58d0 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc14d530 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bcc06f220 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.249 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.244+0000 7f7bb3fff640 1 -- 172.21.15.76:0/2678275284 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7bc40bf070 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.248+0000 7f7bb3fff640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7bb80678f0 0x7f7bb8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.248+0000 7f7bb3fff640 1 -- 172.21.15.76:0/2678275284 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f7bc4120980 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.248+0000 7f7bcb7fe640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7bb80678f0 0x7f7bb8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.250 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.248+0000 7f7bcb7fe640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7bb80678f0 0x7f7bb8069db0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00b9d0 tx=0x7f7bbc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.251 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.252+0000 7f7bb3fff640 1 -- 172.21.15.76:0/2678275284 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7bc40c7020 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.400+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7bcc06fcd0 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.404+0000 7f7bb3fff640 1 -- 172.21.15.76:0/2678275284 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7f7bc4016620 con 0x7f7bcc15c2a0 2024-09-19T04:19:40.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7bb80678f0 msgr2=0x7f7bb8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7bb80678f0 0x7f7bb8069db0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00b9d0 tx=0x7f7bbc008040 comp rx=0 tx=0).stop 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc15c2a0 msgr2=0x7f7bcc14da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc15c2a0 0x7f7bcc14da50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f7bc401a630 tx=0x7f7bc40197a0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 shutdown_connections 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7bb80678f0 0x7f7bb8069db0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7bcc15c2a0 0x7f7bcc14da50 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7bcc15bec0 0x7f7bcc150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 --2- 172.21.15.76:0/2678275284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7bcc0af470 0x7f7bcc155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 >> 172.21.15.76:0/2678275284 conn(0x7f7bcc0b1f90 msgr2=0x7f7bcc1597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 shutdown_connections 2024-09-19T04:19:40.409 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.408+0000 7f7bd1c2c640 1 -- 172.21.15.76:0/2678275284 wait complete. 2024-09-19T04:19:40.425 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:19:40.425 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:19:40.489 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:19:40.555 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:19:40.555 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:19:40.555 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:40.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:19:40.561 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:19:40.561 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:40.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:19:40.567 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:19:40.567 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:40.608 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:19:40.608 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:19:40.608 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:19:40.613 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:19:40.613 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-19T04:19:40.614 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:40.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa105112640 1 Processor -- start 2024-09-19T04:19:40.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa105112640 1 -- start start 2024-09-19T04:19:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa105112640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 0x7fa1000b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.773 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa105112640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10006ec60 con 0x7fa10006e680 2024-09-19T04:19:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa105112640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa10006ef40 con 0x7fa1000b37d0 2024-09-19T04:19:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa105112640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa10006f870 con 0x7fa1001587a0 2024-09-19T04:19:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0ff59e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa10006e680 0x7fa100072160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36608/0 (socket says 172.21.15.76:36608) 2024-09-19T04:19:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0ff59e640 1 -- 172.21.15.76:0/2578459432 learned_addr learned my addr 172.21.15.76:0/2578459432 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 0x7fa1000b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2236967862 0 0) 0x7fa10006ec60 con 0x7fa10006e680 2024-09-19T04:19:40.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0e8003530 con 0x7fa10006e680 2024-09-19T04:19:40.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 364281796 0 0) 0x7fa0e8003530 con 0x7fa10006e680 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 msgr2=0x7fa1000b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 0x7fa1000b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 >> v1:172.21.15.116:6789/0 conn(0x7fa1000b37d0 legacy=0x7fa10006df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa10006f220 con 0x7fa10006e680 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fa0f8001a00 con 0x7fa10006e680 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 >> v1:172.21.15.76:6789/0 conn(0x7fa10006e680 legacy=0x7fa100072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e80039f0 0x7fa0e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 0x7fa0e80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa10006f870 con 0x7fa0e80039f0 2024-09-19T04:19:40.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e800ba80 con 0x7fa0e8004810 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa0e800bc80 con 0x7fa1001587a0 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e80039f0 0x7fa0e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0ff59e640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 0x7fa0e80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fe59c640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fed9d640 1 -- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 msgr2=0x7fa0e80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa1001587a0 0x7fa0e80051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fed9d640 1 -- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e8004810 msgr2=0x7fa0e8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fed9d640 1 -- 172.21.15.76:0/2578459432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e8003530 con 0x7fa0e80039f0 2024-09-19T04:19:40.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.772+0000 7fa0fe59c640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e8004810 0x7fa0e8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:40.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e80039f0 0x7fa0e8003e10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009ab0 tx=0x7fa0f401ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3440ddaacc2f4d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40096b0 con 0x7fa0e80039f0 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f40a6410 con 0x7fa0e80039f0 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa0fdd9b640 1 -- 172.21.15.76:0/2578459432 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40a6800 con 0x7fa0e80039f0 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa105112640 1 -- 172.21.15.76:0/2578459432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e80039f0 msgr2=0x7fa0e8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa105112640 1 --2- 172.21.15.76:0/2578459432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa0e80039f0 0x7fa0e8003e10 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009ab0 tx=0x7fa0f401ca30 comp rx=0 tx=0).stop 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa0ff59e640 1 -- 172.21.15.76:0/2578459432 reap_dead start 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa105112640 1 -- 172.21.15.76:0/2578459432 shutdown_connections 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa105112640 1 -- 172.21.15.76:0/2578459432 >> 172.21.15.76:0/2578459432 conn(0x7fa1000b1f90 msgr2=0x7fa1000b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa105112640 1 -- 172.21.15.76:0/2578459432 shutdown_connections 2024-09-19T04:19:40.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa105112640 1 -- 172.21.15.76:0/2578459432 wait complete. 2024-09-19T04:19:40.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.776+0000 7fa105112640 1 Processor -- start 2024-09-19T04:19:40.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 -- start start 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0e80039f0 0x7fa1000ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1001567f0 0x7fa100157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa100157aa0 0x7fa1001fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa10016a1f0 con 0x7fa1001567f0 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa100073310 con 0x7fa100157aa0 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa10016a370 con 0x7fa0e80039f0 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1001567f0 0x7fa100157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1001567f0 0x7fa100157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52036/0 (socket says 172.21.15.76:52036) 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 -- 172.21.15.76:0/1275487432 learned_addr learned my addr 172.21.15.76:0/1275487432 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0ff59e640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa100157aa0 0x7fa1001fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0e80039f0 0x7fa1000ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 -- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0e80039f0 msgr2=0x7fa1000ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0e80039f0 0x7fa1000ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 -- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa100157aa0 msgr2=0x7fa1001fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa100157aa0 0x7fa1001fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 -- 172.21.15.76:0/1275487432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa100169040 con 0x7fa1001567f0 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0e80039f0 0x7fa1000ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0ff59e640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa100157aa0 0x7fa1001fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:40.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fe59c640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1001567f0 0x7fa100157560 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d7a0 tx=0x7fa0f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0e77fe640 1 -- 172.21.15.76:0/1275487432 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0019070 con 0x7fa1001567f0 2024-09-19T04:19:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0e77fe640 1 -- 172.21.15.76:0/1275487432 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0005270 con 0x7fa1001567f0 2024-09-19T04:19:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1001663d0 con 0x7fa1001567f0 2024-09-19T04:19:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0e77fe640 1 -- 172.21.15.76:0/1275487432 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0007030 con 0x7fa1001567f0 2024-09-19T04:19:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa100166990 con 0x7fa1001567f0 2024-09-19T04:19:40.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa100072160 con 0x7fa1001567f0 2024-09-19T04:19:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0e77fe640 1 -- 172.21.15.76:0/1275487432 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa0f001c070 con 0x7fa1001567f0 2024-09-19T04:19:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0e77fe640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0ec0678c0 0x7fa0ec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0e77fe640 1 -- 172.21.15.76:0/1275487432 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7fa0f0085430 con 0x7fa1001567f0 2024-09-19T04:19:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.780+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0ec0678c0 0x7fa0ec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:40.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.784+0000 7fa0fed9d640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0ec0678c0 0x7fa0ec069d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40ad000 tx=0x7fa0f40046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:40.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.788+0000 7fa0e77fe640 1 -- 172.21.15.76:0/1275487432 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa0f0055f90 con 0x7fa1001567f0 2024-09-19T04:19:40.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.968+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa100157e80 con 0x7fa1001567f0 2024-09-19T04:19:40.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.968+0000 7fa0e77fe640 1 -- 172.21.15.76:0/1275487432 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) ==== 94+0+3914 (secure 0 0 0) 0x7fa0f0055890 con 0x7fa1001567f0 2024-09-19T04:19:40.970 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:40.970 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":96,"btime":"2024-09-19T04:19:34:804153+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5507,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/3949234525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3949234525},{"type":"v1","addr":"172.21.15.116:6835","nonce":3949234525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":6061,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/3741339910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3741339910},{"type":"v1","addr":"172.21.15.116:6833","nonce":3741339910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8034,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/189820766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":189820766},{"type":"v1","addr":"172.21.15.76:6835","nonce":189820766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8040,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/2236489055","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":2236489055},{"type":"v1","addr":"172.21.15.76:6837","nonce":2236489055}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96}],"filesystems":[]} 2024-09-19T04:19:40.971 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 96 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0ec0678c0 msgr2=0x7fa0ec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0ec0678c0 0x7fa0ec069d80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40ad000 tx=0x7fa0f40046d0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1001567f0 msgr2=0x7fa100157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1001567f0 0x7fa100157560 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000d7a0 tx=0x7fa0f000dc60 comp rx=0 tx=0).stop 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 shutdown_connections 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa0ec0678c0 0x7fa0ec069d80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa100157aa0 0x7fa1001fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa1001567f0 0x7fa100157560 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 --2- 172.21.15.76:0/1275487432 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa0e80039f0 0x7fa1000ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 >> 172.21.15.76:0/1275487432 conn(0x7fa1000b1f90 msgr2=0x7fa1000b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:40.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 shutdown_connections 2024-09-19T04:19:40.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:40.972+0000 7fa105112640 1 -- 172.21.15.76:0/1275487432 wait complete. 2024-09-19T04:19:40.988 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-19T04:19:40.988 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-19T04:19:40.988 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T04:19:40.988 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:19:40.989 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-19T04:19:40.989 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-19T04:19:40.989 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T04:19:40.989 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:19:40.989 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-19T04:19:40.989 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-19T04:19:40.990 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T04:19:40.990 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:19:40.990 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-19T04:19:40.990 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-19T04:19:40.990 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T04:19:40.990 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:19:40.997 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T04:19:40.997 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-19T04:19:41.000 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T04:19:41.000 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-19T04:19:41.004 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T04:19:41.004 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-19T04:19:41.005 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T04:19:41.005 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-19T04:19:41.048 INFO:tasks.ceph.mds.a:Started 2024-09-19T04:19:41.048 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:41.049 INFO:tasks.ceph.mds.c:Started 2024-09-19T04:19:41.050 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:41.050 INFO:tasks.ceph.mds.d:Started 2024-09-19T04:19:41.050 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:41.051 INFO:tasks.ceph.mds.b:Started 2024-09-19T04:19:41.051 DEBUG:teuthology.parallel:result is None 2024-09-19T04:19:41.051 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-19T04:19:41.107 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:19:41.117 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:19:41.123 INFO:tasks.ceph.mds.b.smithi116.stdout:starting mds.b at 2024-09-19T04:19:41.127 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:19:41.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bca673640 1 Processor -- start 2024-09-19T04:19:41.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bca673640 1 -- start start 2024-09-19T04:19:41.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bca673640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8bc41520a0 0x7f8bc406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bca673640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8bc406fe50 0x7f8bc415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bca673640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bc4070250 con 0x7f8bc415f1d0 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bca673640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc4152480 con 0x7f8bc41520a0 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bca673640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8bc40aede0 con 0x7f8bc406fe50 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8bc41520a0 0x7f8bc406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc3fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8bc41520a0 0x7f8bc406f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52108/0 (socket says 172.21.15.76:52108) 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc3fff640 1 -- 172.21.15.76:0/2353098321 learned_addr learned my addr 172.21.15.76:0/2353098321 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc37fe640 1 --2- 172.21.15.76:0/2353098321 >> v2:172.21.15.116:3300/0 conn(0x7f8bc406fe50 0x7f8bc415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc3fff640 1 -- 172.21.15.76:0/2353098321 >> v2:172.21.15.116:3300/0 conn(0x7f8bc406fe50 msgr2=0x7f8bc415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc3fff640 1 --2- 172.21.15.76:0/2353098321 >> v2:172.21.15.116:3300/0 conn(0x7f8bc406fe50 0x7f8bc415ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc3fff640 1 -- 172.21.15.76:0/2353098321 >> v1:172.21.15.116:6789/0 conn(0x7f8bc415f1d0 legacy=0x7f8bc4071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:41.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.164+0000 7f8bc3fff640 1 -- 172.21.15.76:0/2353098321 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc415fa10 con 0x7f8bc41520a0 2024-09-19T04:19:41.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc3fff640 1 --2- 172.21.15.76:0/2353098321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc41520a0 0x7f8bc406f910 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406e510 tx=0x7f8bbc01eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f81432780d6d4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc2ffd640 1 -- 172.21.15.76:0/2353098321 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0b4070 con 0x7f8bc41520a0 2024-09-19T04:19:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc2ffd640 1 -- 172.21.15.76:0/2353098321 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bbc0a7b80 con 0x7f8bc41520a0 2024-09-19T04:19:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc2ffd640 1 -- 172.21.15.76:0/2353098321 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc0af440 con 0x7f8bc41520a0 2024-09-19T04:19:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- 172.21.15.76:0/2353098321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc41520a0 msgr2=0x7f8bc406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 --2- 172.21.15.76:0/2353098321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc41520a0 0x7f8bc406f910 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406e510 tx=0x7f8bbc01eb90 comp rx=0 tx=0).stop 2024-09-19T04:19:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- 172.21.15.76:0/2353098321 shutdown_connections 2024-09-19T04:19:41.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 --2- 172.21.15.76:0/2353098321 >> v2:172.21.15.116:3300/0 conn(0x7f8bc406fe50 0x7f8bc415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 --2- 172.21.15.76:0/2353098321 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc41520a0 0x7f8bc406f910 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- 172.21.15.76:0/2353098321 >> 172.21.15.76:0/2353098321 conn(0x7f8bc40b1f90 msgr2=0x7f8bc40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- 172.21.15.76:0/2353098321 shutdown_connections 2024-09-19T04:19:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- 172.21.15.76:0/2353098321 wait complete. 2024-09-19T04:19:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 Processor -- start 2024-09-19T04:19:41.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- start start 2024-09-19T04:19:41.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bc406fe50 0x7f8bc4149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bc41520a0 0x7f8bc414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc415f1d0 0x7f8bc40982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4073d30 con 0x7f8bc415f1d0 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc4073a50 con 0x7f8bc41520a0 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8bc415f660 con 0x7f8bc406fe50 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc415f1d0 0x7f8bc40982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc415f1d0 0x7f8bc40982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52110/0 (socket says 172.21.15.76:52110) 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 -- 172.21.15.76:0/3244228042 learned_addr learned my addr 172.21.15.76:0/3244228042 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc3fff640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bc406fe50 0x7f8bc4149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc37fe640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bc41520a0 0x7f8bc414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 -- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bc406fe50 msgr2=0x7f8bc4149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bc406fe50 0x7f8bc4149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 -- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bc41520a0 msgr2=0x7f8bc414a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bc41520a0 0x7f8bc414a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 -- 172.21.15.76:0/3244228042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc4155170 con 0x7f8bc415f1d0 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc8c10640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc415f1d0 0x7f8bc40982f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c780 tx=0x7f8bb400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc0ff9640 1 -- 172.21.15.76:0/3244228042 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4004b70 con 0x7f8bc415f1d0 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc0ff9640 1 -- 172.21.15.76:0/3244228042 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bb4004d10 con 0x7f8bc415f1d0 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc4151be0 con 0x7f8bc415f1d0 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bc0ff9640 1 -- 172.21.15.76:0/3244228042 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400aa90 con 0x7f8bc415f1d0 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.168+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc414a790 con 0x7f8bc415f1d0 2024-09-19T04:19:41.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.172+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bc415ec90 con 0x7f8bc415f1d0 2024-09-19T04:19:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.172+0000 7f8bc0ff9640 1 -- 172.21.15.76:0/3244228042 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8bb400ac30 con 0x7f8bc415f1d0 2024-09-19T04:19:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.172+0000 7f8bc0ff9640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bb00678f0 0x7f8bb0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.172+0000 7f8bc3fff640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bb00678f0 0x7f8bb0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.172+0000 7f8bc0ff9640 1 -- 172.21.15.76:0/3244228042 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5507+0+0 (secure 0 0 0) 0x7f8bb4085cd0 con 0x7f8bc415f1d0 2024-09-19T04:19:41.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.172+0000 7f8bc3fff640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bb00678f0 0x7f8bb0069db0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b1000 tx=0x7f8bbc0a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:41.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.176+0000 7f8bc0ff9640 1 -- 172.21.15.76:0/3244228042 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8bb408b020 con 0x7f8bc415f1d0 2024-09-19T04:19:41.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:41.348+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8bc406e540 con 0x7f8bc415f1d0 2024-09-19T04:19:42.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.108+0000 7f8bc0ff9640 1 -- 172.21.15.76:0/3244228042 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v98) ==== 97+0+0 (secure 0 0 0) 0x7f8bb4020070 con 0x7f8bc415f1d0 2024-09-19T04:19:42.108 INFO:teuthology.orchestra.run.smithi076.stderr: removed all blocklist entries 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.108+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bb00678f0 msgr2=0x7f8bb0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.108+0000 7f8bca673640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bb00678f0 0x7f8bb0069db0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc0b1000 tx=0x7f8bbc0a9070 comp rx=0 tx=0).stop 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc415f1d0 msgr2=0x7f8bc40982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc415f1d0 0x7f8bc40982f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c780 tx=0x7f8bb400cc40 comp rx=0 tx=0).stop 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 shutdown_connections 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bb00678f0 0x7f8bb0069db0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bc415f1d0 0x7f8bc40982f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bc41520a0 0x7f8bc414a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 --2- 172.21.15.76:0/3244228042 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8bc406fe50 0x7f8bc4149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 >> 172.21.15.76:0/3244228042 conn(0x7f8bc40b1f90 msgr2=0x7f8bc4071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 shutdown_connections 2024-09-19T04:19:42.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.112+0000 7f8bca673640 1 -- 172.21.15.76:0/3244228042 wait complete. 2024-09-19T04:19:42.134 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-19T04:19:42.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f264d8fa640 1 Processor -- start 2024-09-19T04:19:42.256 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f264d8fa640 1 -- start start 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f264d8fa640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2648070020 0x7f264815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f264d8fa640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26480aede0 con 0x7f26481520a0 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f264d8fa640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f264815fa10 con 0x7f264815f1d0 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f264d8fa640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2648152480 con 0x7f2648070020 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2646ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f26481520a0 0x7f264806f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:36632/0 (socket says 172.21.15.76:36632) 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2646ffd640 1 -- 172.21.15.76:0/864818747 learned_addr learned my addr 172.21.15.76:0/864818747 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 984757400 0 0) 0x7f26480aede0 con 0x7f26481520a0 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f262c003530 con 0x7f26481520a0 2024-09-19T04:19:42.259 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26467fc640 1 --2- 172.21.15.76:0/864818747 >> v2:172.21.15.116:3300/0 conn(0x7f2648070020 0x7f264815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 553394989 0 0) 0x7f262c003530 con 0x7f26481520a0 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 >> v2:172.21.15.116:3300/0 conn(0x7f2648070020 msgr2=0x7f264815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 --2- 172.21.15.76:0/864818747 >> v2:172.21.15.116:3300/0 conn(0x7f2648070020 0x7f264815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 >> v1:172.21.15.116:6789/0 conn(0x7f264815f1d0 legacy=0x7f26480adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648074250 con 0x7f26481520a0 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2638001680 con 0x7f26481520a0 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 >> v1:172.21.15.76:6789/0 conn(0x7f26481520a0 legacy=0x7f264806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c0039c0 0x7f262c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f262c004810 0x7f262c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.260 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f262c0051d0 0x7f262c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00c040 con 0x7f262c004810 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f262c00c1f0 con 0x7f262c0051d0 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f262c00c3f0 con 0x7f262c0039c0 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26467fc640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f262c004810 0x7f262c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f2646ffd640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c0039c0 0x7f262c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26467fc640 1 -- 172.21.15.76:0/864818747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c0039c0 msgr2=0x7f262c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26467fc640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c0039c0 0x7f262c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26467fc640 1 -- 172.21.15.76:0/864818747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f262c0051d0 msgr2=0x7f262c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26467fc640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f262c0051d0 0x7f262c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26477fe640 1 -- 172.21.15.76:0/864818747 reap_dead start 2024-09-19T04:19:42.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.256+0000 7f26467fc640 1 -- 172.21.15.76:0/864818747 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f262c003530 con 0x7f262c004810 2024-09-19T04:19:42.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f26467fc640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f262c004810 0x7f262c004c90 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2634009ab0 tx=0x7f263401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7417f4ebd1abaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340b2070 con 0x7f262c004810 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26340a5660 con 0x7f262c004810 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f2645ffb640 1 -- 172.21.15.76:0/864818747 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340096b0 con 0x7f262c004810 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- 172.21.15.76:0/864818747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f262c004810 msgr2=0x7f262c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f262c004810 0x7f262c004c90 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f2634009ab0 tx=0x7f263401ca70 comp rx=0 tx=0).stop 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- 172.21.15.76:0/864818747 shutdown_connections 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 --2- 172.21.15.76:0/864818747 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f262c004810 0x7f262c004c90 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- 172.21.15.76:0/864818747 >> 172.21.15.76:0/864818747 conn(0x7f26480b1f90 msgr2=0x7f26480b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- 172.21.15.76:0/864818747 shutdown_connections 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- 172.21.15.76:0/864818747 wait complete. 2024-09-19T04:19:42.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 Processor -- start 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- start start 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c004810 0x7f264815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264814a190 0x7f26480cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f264816ab40 0x7f26480cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2648073a50 con 0x7f264816ab40 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2648073770 con 0x7f264814a190 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f264d8fa640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2648073d30 con 0x7f262c004810 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f26477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f264816ab40 0x7f26480cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f26477fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f264816ab40 0x7f26480cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52128/0 (socket says 172.21.15.76:52128) 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f26477fe640 1 -- 172.21.15.76:0/310033803 learned_addr learned my addr 172.21.15.76:0/310033803 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.260+0000 7f2646ffd640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c004810 0x7f264815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f26467fc640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264814a190 0x7f26480cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f26477fe640 1 -- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c004810 msgr2=0x7f264815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f26477fe640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c004810 0x7f264815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f26477fe640 1 -- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264814a190 msgr2=0x7f26480cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f26477fe640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264814a190 0x7f26480cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f26477fe640 1 -- 172.21.15.76:0/310033803 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2648169180 con 0x7f264816ab40 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f26477fe640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f264816ab40 0x7f26480cd1f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f264815f5b0 tx=0x7f264000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f263ffff640 1 -- 172.21.15.76:0/310033803 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640017070 con 0x7f264816ab40 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f263ffff640 1 -- 172.21.15.76:0/310033803 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26400070d0 con 0x7f264816ab40 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f263ffff640 1 -- 172.21.15.76:0/310033803 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400074c0 con 0x7f264816ab40 2024-09-19T04:19:42.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2648169460 con 0x7f264816ab40 2024-09-19T04:19:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2648169a20 con 0x7f264816ab40 2024-09-19T04:19:42.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2648068e50 con 0x7f264816ab40 2024-09-19T04:19:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f263ffff640 1 -- 172.21.15.76:0/310033803 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f264001a070 con 0x7f264816ab40 2024-09-19T04:19:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f263ffff640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26300678c0 0x7f2630069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f263ffff640 1 -- 172.21.15.76:0/310033803 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f2640084cd0 con 0x7f264816ab40 2024-09-19T04:19:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f2646ffd640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26300678c0 0x7f2630069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.269 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.264+0000 7f2646ffd640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26300678c0 0x7f2630069d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26380017f0 tx=0x7f2638001660 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.268+0000 7f263ffff640 1 -- 172.21.15.76:0/310033803 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2640055a60 con 0x7f264816ab40 2024-09-19T04:19:42.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.472+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f26480aede0 con 0x7f264816ab40 2024-09-19T04:19:42.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f263ffff640 1 -- 172.21.15.76:0/310033803 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7f2640055360 con 0x7f264816ab40 2024-09-19T04:19:42.476 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:42.476 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:42.476 INFO:teuthology.orchestra.run.smithi076.stdout: "auth_dump": [ 2024-09-19T04:19:42.476 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.476 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.a", 2024-09-19T04:19:42.476 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmA0pDBxAAOYuXdlqeDKmpv/gwuQax0w==", 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.b", 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm+yl2FRAA1uF8YehlxKuEYRikF7QyPA==", 2024-09-19T04:19:42.477 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.c", 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6eWjDxAAC9T/JFbkca8D3HYgQkT/Dw==", 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.478 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mds.d", 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmZCiSHRAAEuMS2PUmi5FM/cGiLYcgIA==", 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:19:42.479 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.0", 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCspOtmSHpRLxAAAER4qxUkPDVkImxYzXmQxQ==", 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.480 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.1", 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCwpOtmXnovEBAAExHevoBqDYCv0I3HSQzc1w==", 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.481 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.2", 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCzpOtm8ra2IBAAu/f4FOCWrbDplBFnJ+jhBQ==", 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.3", 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC2pOtmpcG8MBAAndk1KBJJE9OCiTfs7berBw==", 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.482 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.4", 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQC8pOtm3cCWOBAAoxn8B/9BX3FwTEwHrag5QA==", 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.483 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.5", 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDApOtm4LQJCRAAXyGWmo0dKRc3fUcZMpNGLw==", 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.6", 2024-09-19T04:19:42.484 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDDpOtm86VbGRAAsxwCEkg2lR0P49u2h2FVRg==", 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "osd.7", 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQDGpOtmH19JJhAAqXXB8r7Ur/7pEBctfzqnbA==", 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow profile osd", 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile osd", 2024-09-19T04:19:42.485 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.0", 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg==", 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow r", 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.486 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.1", 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmWZ75KhAApzvNNjLa6M9FH/v2EE8kYg==", 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.2", 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmo7NpKBAA7LflsVMkEUUN0+R2RdSRIQ==", 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.487 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.3", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtm6RfxLxAAFgfwUWfVXfQ7Y7i5/Gk3sA==", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow r", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow rw", 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow rwx" 2024-09-19T04:19:42.488 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "client.admin", 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCopOtmGhUPLBAA9AkjLm1xF81Mjf1IYlN6Fg==", 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: "mgr": "allow *", 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow *", 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.489 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.x", 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmcFzkAxAATLtn350RnZJyw/a2JiKP2A==", 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "entity": "mgr.y", 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "key": "AQCppOtmveFYARAAAuT3HEDatnUyz4U3NAUIRg==", 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "caps": { 2024-09-19T04:19:42.490 INFO:teuthology.orchestra.run.smithi076.stdout: "mds": "allow *", 2024-09-19T04:19:42.491 INFO:teuthology.orchestra.run.smithi076.stdout: "mon": "allow profile mgr", 2024-09-19T04:19:42.491 INFO:teuthology.orchestra.run.smithi076.stdout: "osd": "allow *" 2024-09-19T04:19:42.491 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.491 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.491 INFO:teuthology.orchestra.run.smithi076.stdout: ] 2024-09-19T04:19:42.491 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26300678c0 msgr2=0x7f2630069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26300678c0 0x7f2630069d80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f26380017f0 tx=0x7f2638001660 comp rx=0 tx=0).stop 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f264816ab40 msgr2=0x7f26480cd1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f264816ab40 0x7f26480cd1f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f264815f5b0 tx=0x7f264000c890 comp rx=0 tx=0).stop 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 shutdown_connections 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26300678c0 0x7f2630069d80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f264816ab40 0x7f26480cd1f0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f264814a190 0x7f26480cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 --2- 172.21.15.76:0/310033803 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f262c004810 0x7f264815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 >> 172.21.15.76:0/310033803 conn(0x7f26480b1f90 msgr2=0x7f2648151920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:42.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 shutdown_connections 2024-09-19T04:19:42.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.476+0000 7f264d8fa640 1 -- 172.21.15.76:0/310033803 wait complete. 2024-09-19T04:19:42.494 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-19T04:19:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fdb44f640 1 Processor -- start 2024-09-19T04:19:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fdb44f640 1 -- start start 2024-09-19T04:19:42.608 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fdb44f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7fd415f3a0 0x7f7fd4165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fdb44f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd40aede0 con 0x7f7fd41520a0 2024-09-19T04:19:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fdb44f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7fd415fa10 con 0x7f7fd4070020 2024-09-19T04:19:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fdb44f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7fd4152480 con 0x7f7fd415f3a0 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd91eb640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7fd41520a0 0x7f7fd406f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54908/0 (socket says 172.21.15.76:54908) 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd91eb640 1 -- 172.21.15.76:0/3608409968 learned_addr learned my addr 172.21.15.76:0/3608409968 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd99ec640 1 --2- 172.21.15.76:0/3608409968 >> v2:172.21.15.76:3300/0 conn(0x7f7fd415f3a0 0x7f7fd4165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd3fff640 1 -- 172.21.15.76:0/3608409968 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375302440 0 0) 0x7f7fd40aede0 con 0x7f7fd41520a0 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd3fff640 1 -- 172.21.15.76:0/3608409968 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7fbc003530 con 0x7f7fd41520a0 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd99ec640 1 -- 172.21.15.76:0/3608409968 >> v1:172.21.15.116:6789/0 conn(0x7f7fd4070020 legacy=0x7f7fd415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd99ec640 1 -- 172.21.15.76:0/3608409968 >> v1:172.21.15.76:6789/0 conn(0x7f7fd41520a0 legacy=0x7f7fd406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.608+0000 7f7fd99ec640 1 -- 172.21.15.76:0/3608409968 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd4074250 con 0x7f7fd415f3a0 2024-09-19T04:19:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd99ec640 1 --2- 172.21.15.76:0/3608409968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd415f3a0 0x7f7fd4165f00 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40097c0 tx=0x7f7fc401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40afa5cd149f1d35 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd3fff640 1 -- 172.21.15.76:0/3608409968 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40aa600 con 0x7f7fd415f3a0 2024-09-19T04:19:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd3fff640 1 -- 172.21.15.76:0/3608409968 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc40af070 con 0x7f7fd415f3a0 2024-09-19T04:19:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd3fff640 1 -- 172.21.15.76:0/3608409968 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40aa9c0 con 0x7f7fd415f3a0 2024-09-19T04:19:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3608409968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd415f3a0 msgr2=0x7f7fd4165f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3608409968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd415f3a0 0x7f7fd4165f00 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f7fc40097c0 tx=0x7f7fc401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3608409968 shutdown_connections 2024-09-19T04:19:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3608409968 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd415f3a0 0x7f7fd4165f00 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3608409968 >> 172.21.15.76:0/3608409968 conn(0x7f7fd40b1f90 msgr2=0x7f7fd40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3608409968 shutdown_connections 2024-09-19T04:19:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3608409968 wait complete. 2024-09-19T04:19:42.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 Processor -- start 2024-09-19T04:19:42.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- start start 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fd4070020 0x7f7fd4145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd41520a0 0x7f7fd40afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fd415f3a0 0x7f7fd40b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd406e800 con 0x7f7fd41520a0 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7fd4073d30 con 0x7f7fd415f3a0 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7fd406e980 con 0x7f7fd4070020 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd41520a0 0x7f7fd40afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd41520a0 0x7f7fd40afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58896/0 (socket says 172.21.15.76:58896) 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 -- 172.21.15.76:0/3135300891 learned_addr learned my addr 172.21.15.76:0/3135300891 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:42.616 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 -- 172.21.15.76:0/3135300891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fd4070020 msgr2=0x7f7fd4145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fd4070020 0x7f7fd4145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 -- 172.21.15.76:0/3135300891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fd415f3a0 msgr2=0x7f7fd40b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fd415f3a0 0x7f7fd40b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 -- 172.21.15.76:0/3135300891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fd4144390 con 0x7f7fd41520a0 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd89ea640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd41520a0 0x7f7fd40afe00 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c340 tx=0x7f7fc800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd1ffb640 1 -- 172.21.15.76:0/3135300891 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80052d0 con 0x7f7fd41520a0 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd1ffb640 1 -- 172.21.15.76:0/3135300891 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fc8005470 con 0x7f7fd41520a0 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fd415acb0 con 0x7f7fd41520a0 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fd1ffb640 1 -- 172.21.15.76:0/3135300891 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8007260 con 0x7f7fd41520a0 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.612+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fd415a9d0 con 0x7f7fd41520a0 2024-09-19T04:19:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.616+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fd4068e50 con 0x7f7fd41520a0 2024-09-19T04:19:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.616+0000 7f7fd1ffb640 1 -- 172.21.15.76:0/3135300891 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7fc8007400 con 0x7f7fd41520a0 2024-09-19T04:19:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.616+0000 7f7fd1ffb640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7fc00678c0 0x7f7fc0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.616+0000 7f7fd91eb640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7fc00678c0 0x7f7fc0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.616+0000 7f7fd1ffb640 1 -- 172.21.15.76:0/3135300891 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f7fc8084f50 con 0x7f7fd41520a0 2024-09-19T04:19:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.616+0000 7f7fd91eb640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7fc00678c0 0x7f7fc0069d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0020b0 tx=0x7f7fcc0016a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.620+0000 7f7fd1ffb640 1 -- 172.21.15.76:0/3135300891 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7fc8055c50 con 0x7f7fd41520a0 2024-09-19T04:19:42.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7fd406f3a0 con 0x7f7fd41520a0 2024-09-19T04:19:42.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fd1ffb640 1 -- 172.21.15.76:0/3135300891 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) ==== 84+0+65 (secure 0 0 0) 0x7f7fc8055550 con 0x7f7fd41520a0 2024-09-19T04:19:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:42.772 INFO:teuthology.orchestra.run.smithi076.stdout:[ 2024-09-19T04:19:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: { 2024-09-19T04:19:42.772 INFO:teuthology.orchestra.run.smithi076.stdout: "poolnum": 1, 2024-09-19T04:19:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: "poolname": ".mgr" 2024-09-19T04:19:42.773 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:42.773 INFO:teuthology.orchestra.run.smithi076.stdout:] 2024-09-19T04:19:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7fc00678c0 msgr2=0x7f7fc0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7fc00678c0 0x7f7fc0069d80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc0020b0 tx=0x7f7fcc0016a0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd41520a0 msgr2=0x7f7fd40afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd41520a0 0x7f7fd40afe00 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800c340 tx=0x7f7fc800c800 comp rx=0 tx=0).stop 2024-09-19T04:19:42.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 shutdown_connections 2024-09-19T04:19:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7fc00678c0 0x7f7fc0069d80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7fd415f3a0 0x7f7fd40b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7fd41520a0 0x7f7fd40afe00 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 --2- 172.21.15.76:0/3135300891 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7fd4070020 0x7f7fd4145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 >> 172.21.15.76:0/3135300891 conn(0x7f7fd40b1f90 msgr2=0x7f7fd40b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 shutdown_connections 2024-09-19T04:19:42.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.772+0000 7f7fdb44f640 1 -- 172.21.15.76:0/3135300891 wait complete. 2024-09-19T04:19:42.792 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T04:19:42.793 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T04:19:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8208ef1640 1 Processor -- start 2024-09-19T04:19:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8208ef1640 1 -- start start 2024-09-19T04:19:42.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8208ef1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82041520a0 0x7f820406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8208ef1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f820415f3a0 0x7f82040adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8208ef1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8204070250 con 0x7f820406fe50 2024-09-19T04:19:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8208ef1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8204152480 con 0x7f820415f3a0 2024-09-19T04:19:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8208ef1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f82040aede0 con 0x7f82041520a0 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8202d9d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f820415f3a0 0x7f82040adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f820259c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82041520a0 0x7f820406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8201d9b640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f820406fe50 0x7f820415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:58502/0 (socket says 172.21.15.76:58502) 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.916+0000 7f8201d9b640 1 -- 172.21.15.76:0/3560973359 learned_addr learned my addr 172.21.15.76:0/3560973359 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820259c640 1 -- 172.21.15.76:0/3560973359 >> v2:172.21.15.116:3300/0 conn(0x7f820415f3a0 msgr2=0x7f82040adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820259c640 1 --2- 172.21.15.76:0/3560973359 >> v2:172.21.15.116:3300/0 conn(0x7f820415f3a0 0x7f82040adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820259c640 1 -- 172.21.15.76:0/3560973359 >> v1:172.21.15.116:6789/0 conn(0x7f820406fe50 legacy=0x7f820415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820259c640 1 -- 172.21.15.76:0/3560973359 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820415fa10 con 0x7f82041520a0 2024-09-19T04:19:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820159a640 1 -- 172.21.15.76:0/3560973359 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 963581707 0 0) 0x7f8204070250 con 0x7f820406fe50 2024-09-19T04:19:42.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8202d9d640 1 --2- 172.21.15.76:0/3560973359 >> v2:172.21.15.116:3300/0 conn(0x7f820415f3a0 0x7f82040adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820259c640 1 --2- 172.21.15.76:0/3560973359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82041520a0 0x7f820406f910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81f40097c0 tx=0x7f81f401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58ef0e86898a6833 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820159a640 1 -- 172.21.15.76:0/3560973359 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40aa600 con 0x7f82041520a0 2024-09-19T04:19:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820159a640 1 -- 172.21.15.76:0/3560973359 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f40af070 con 0x7f82041520a0 2024-09-19T04:19:42.922 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f820159a640 1 -- 172.21.15.76:0/3560973359 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f40aa9c0 con 0x7f82041520a0 2024-09-19T04:19:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 -- 172.21.15.76:0/3560973359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82041520a0 msgr2=0x7f820406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 --2- 172.21.15.76:0/3560973359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82041520a0 0x7f820406f910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f81f40097c0 tx=0x7f81f401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 -- 172.21.15.76:0/3560973359 shutdown_connections 2024-09-19T04:19:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 --2- 172.21.15.76:0/3560973359 >> v2:172.21.15.116:3300/0 conn(0x7f820415f3a0 0x7f82040adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 --2- 172.21.15.76:0/3560973359 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f82041520a0 0x7f820406f910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.923 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 -- 172.21.15.76:0/3560973359 >> 172.21.15.76:0/3560973359 conn(0x7f82040b1f90 msgr2=0x7f82040b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:42.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 -- 172.21.15.76:0/3560973359 shutdown_connections 2024-09-19T04:19:42.924 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.920+0000 7f8208ef1640 1 -- 172.21.15.76:0/3560973359 wait complete. 2024-09-19T04:19:42.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 Processor -- start 2024-09-19T04:19:42.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 -- start start 2024-09-19T04:19:42.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f820406fe50 0x7f82040999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82041520a0 0x7f820414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f820415f3a0 0x7f820415b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8204073d30 con 0x7f82041520a0 2024-09-19T04:19:42.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8204073a50 con 0x7f820406fe50 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8204167f10 con 0x7f820415f3a0 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82041520a0 0x7f820414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82041520a0 0x7f820414d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58908/0 (socket says 172.21.15.76:58908) 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 -- 172.21.15.76:0/4269860552 learned_addr learned my addr 172.21.15.76:0/4269860552 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f820259c640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f820406fe50 0x7f82040999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8202d9d640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f820415f3a0 0x7f820415b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 -- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f820415f3a0 msgr2=0x7f820415b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f820415f3a0 0x7f820415b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 -- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f820406fe50 msgr2=0x7f82040999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f820406fe50 0x7f82040999b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:42.927 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 -- 172.21.15.76:0/4269860552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820414daf0 con 0x7f82041520a0 2024-09-19T04:19:42.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f820259c640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f820406fe50 0x7f82040999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:42.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8201d9b640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82041520a0 0x7f820414d0a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f81f800c450 tx=0x7f81f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f81f2ffd640 1 -- 172.21.15.76:0/4269860552 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f8018070 con 0x7f82041520a0 2024-09-19T04:19:42.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f81f2ffd640 1 -- 172.21.15.76:0/4269860552 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81f80074e0 con 0x7f82041520a0 2024-09-19T04:19:42.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820414ddd0 con 0x7f82041520a0 2024-09-19T04:19:42.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f81f2ffd640 1 -- 172.21.15.76:0/4269860552 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f80049f0 con 0x7f82041520a0 2024-09-19T04:19:42.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820414e390 con 0x7f82041520a0 2024-09-19T04:19:42.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820406f910 con 0x7f82041520a0 2024-09-19T04:19:42.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.924+0000 7f81f2ffd640 1 -- 172.21.15.76:0/4269860552 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f81f801b070 con 0x7f82041520a0 2024-09-19T04:19:42.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.928+0000 7f81f2ffd640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81e40678c0 0x7f81e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:42.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.928+0000 7f81f2ffd640 1 -- 172.21.15.76:0/4269860552 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5249+0+0 (secure 0 0 0) 0x7f81f8084e30 con 0x7f82041520a0 2024-09-19T04:19:42.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.928+0000 7f820259c640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81e40678c0 0x7f81e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:42.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.928+0000 7f820259c640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81e40678c0 0x7f81e4069d80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f81f40ae000 tx=0x7f81f40a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:42.932 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:42.932+0000 7f81f2ffd640 1 -- 172.21.15.76:0/4269860552 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f81f8055b30 con 0x7f82041520a0 2024-09-19T04:19:43.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.092+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f820406e540 con 0x7f82041520a0 2024-09-19T04:19:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.180+0000 7f81f2ffd640 1 -- 172.21.15.76:0/4269860552 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v101) ==== 138+0+0 (secure 0 0 0) 0x7f81f8055430 con 0x7f82041520a0 2024-09-19T04:19:43.181 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' created 2024-09-19T04:19:43.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81e40678c0 msgr2=0x7f81e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81e40678c0 0x7f81e4069d80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f81f40ae000 tx=0x7f81f40a7000 comp rx=0 tx=0).stop 2024-09-19T04:19:43.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82041520a0 msgr2=0x7f820414d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82041520a0 0x7f820414d0a0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f81f800c450 tx=0x7f81f800c910 comp rx=0 tx=0).stop 2024-09-19T04:19:43.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 shutdown_connections 2024-09-19T04:19:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f81e40678c0 0x7f81e4069d80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f820415f3a0 0x7f820415b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f82041520a0 0x7f820414d0a0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 --2- 172.21.15.76:0/4269860552 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f820406fe50 0x7f82040999b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 >> 172.21.15.76:0/4269860552 conn(0x7f82040b1f90 msgr2=0x7f820406e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 shutdown_connections 2024-09-19T04:19:43.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.184+0000 7f8208ef1640 1 -- 172.21.15.76:0/4269860552 wait complete. 2024-09-19T04:19:43.210 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T04:19:43.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b32a22640 1 Processor -- start 2024-09-19T04:19:43.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b32a22640 1 -- start start 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b32a22640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b2c073100 0x7f3b2c0cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b32a22640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3b2c0cdfd0 0x7f3b2c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b32a22640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b2c0ce8d0 con 0x7f3b2c0cc870 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b32a22640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2c0cebb0 con 0x7f3b2c073100 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b32a22640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f3b2c0cee90 con 0x7f3b2c0cdfd0 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b2b7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3b2c073100 0x7f3b2c0cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.328+0000 7f3b2bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3b2c0cc870 0x7f3b2c0ccc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54916/0 (socket says 172.21.15.76:54916) 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2bfff640 1 -- 172.21.15.76:0/1462012121 learned_addr learned my addr 172.21.15.76:0/1462012121 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b30fbf640 1 --2- 172.21.15.76:0/1462012121 >> v2:172.21.15.116:3300/0 conn(0x7f3b2c0cdfd0 0x7f3b2c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2b7fe640 1 -- 172.21.15.76:0/1462012121 >> v2:172.21.15.116:3300/0 conn(0x7f3b2c0cdfd0 msgr2=0x7f3b2c06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:43.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2b7fe640 1 --2- 172.21.15.76:0/1462012121 >> v2:172.21.15.116:3300/0 conn(0x7f3b2c0cdfd0 0x7f3b2c06de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2b7fe640 1 -- 172.21.15.76:0/1462012121 >> v1:172.21.15.76:6789/0 conn(0x7f3b2c0cc870 legacy=0x7f3b2c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2affd640 1 -- 172.21.15.76:0/1462012121 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209396925 0 0) 0x7f3b2c0ce8d0 con 0x7f3b2c0cc870 2024-09-19T04:19:43.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2b7fe640 1 -- 172.21.15.76:0/1462012121 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c0cf170 con 0x7f3b2c073100 2024-09-19T04:19:43.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2b7fe640 1 --2- 172.21.15.76:0/1462012121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c073100 0x7f3b2c0cf970 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c071d30 tx=0x7f3b1401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56148ccde5ff7c71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2affd640 1 -- 172.21.15.76:0/1462012121 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b140a7070 con 0x7f3b2c073100 2024-09-19T04:19:43.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2affd640 1 -- 172.21.15.76:0/1462012121 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b140b0070 con 0x7f3b2c073100 2024-09-19T04:19:43.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b2affd640 1 -- 172.21.15.76:0/1462012121 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b140ab890 con 0x7f3b2c073100 2024-09-19T04:19:43.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b32a22640 1 -- 172.21.15.76:0/1462012121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c073100 msgr2=0x7f3b2c0cf970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b32a22640 1 --2- 172.21.15.76:0/1462012121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c073100 0x7f3b2c0cf970 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c071d30 tx=0x7f3b1401c450 comp rx=0 tx=0).stop 2024-09-19T04:19:43.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b32a22640 1 -- 172.21.15.76:0/1462012121 shutdown_connections 2024-09-19T04:19:43.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b32a22640 1 --2- 172.21.15.76:0/1462012121 >> v2:172.21.15.116:3300/0 conn(0x7f3b2c0cdfd0 0x7f3b2c06de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b32a22640 1 --2- 172.21.15.76:0/1462012121 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c073100 0x7f3b2c0cf970 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.332+0000 7f3b32a22640 1 -- 172.21.15.76:0/1462012121 >> 172.21.15.76:0/1462012121 conn(0x7f3b2c0b1c80 msgr2=0x7f3b2c0b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- 172.21.15.76:0/1462012121 shutdown_connections 2024-09-19T04:19:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- 172.21.15.76:0/1462012121 wait complete. 2024-09-19T04:19:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 Processor -- start 2024-09-19T04:19:43.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- start start 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b2c073100 0x7f3b2c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c0cc870 0x7f3b2c13fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b2c0cdfd0 0x7f3b2c144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c169520 con 0x7f3b2c0cc870 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c1693a0 con 0x7f3b2c073100 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2c1696a0 con 0x7f3b2c0cdfd0 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c0cc870 0x7f3b2c13fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c0cc870 0x7f3b2c13fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58928/0 (socket says 172.21.15.76:58928) 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 -- 172.21.15.76:0/970687637 learned_addr learned my addr 172.21.15.76:0/970687637 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 -- 172.21.15.76:0/970687637 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b2c0cdfd0 msgr2=0x7f3b2c144dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b2c0cdfd0 0x7f3b2c144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 -- 172.21.15.76:0/970687637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b2c073100 msgr2=0x7f3b2c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b2c073100 0x7f3b2c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 -- 172.21.15.76:0/970687637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c142c60 con 0x7f3b2c0cc870 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b2b7fe640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c0cc870 0x7f3b2c13fe40 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3b1401a080 tx=0x7f3b1401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b28ff9640 1 -- 172.21.15.76:0/970687637 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14004220 con 0x7f3b2c0cc870 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b28ff9640 1 -- 172.21.15.76:0/970687637 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b140043c0 con 0x7f3b2c0cc870 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c140380 con 0x7f3b2c0cc870 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b28ff9640 1 -- 172.21.15.76:0/970687637 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14003820 con 0x7f3b2c0cc870 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1426a0 con 0x7f3b2c0cc870 2024-09-19T04:19:43.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.336+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c0cf970 con 0x7f3b2c0cc870 2024-09-19T04:19:43.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.340+0000 7f3b28ff9640 1 -- 172.21.15.76:0/970687637 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3b140039c0 con 0x7f3b2c0cc870 2024-09-19T04:19:43.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.340+0000 7f3b28ff9640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b0c0678c0 0x7f3b0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:43.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.340+0000 7f3b2bfff640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b0c0678c0 0x7f3b0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:43.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.340+0000 7f3b28ff9640 1 -- 172.21.15.76:0/970687637 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5619+0+0 (secure 0 0 0) 0x7f3b141203e0 con 0x7f3b2c0cc870 2024-09-19T04:19:43.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.340+0000 7f3b2bfff640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b0c0678c0 0x7f3b0c069d80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c001570 tx=0x7f3b1c007cb0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:43.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.344+0000 7f3b28ff9640 1 -- 172.21.15.76:0/970687637 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3b14016070 con 0x7f3b2c0cc870 2024-09-19T04:19:43.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:43.508+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f3b2c0af8d0 con 0x7f3b2c0cc870 2024-09-19T04:19:44.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.188+0000 7f3b28ff9640 1 -- 172.21.15.76:0/970687637 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v102) ==== 170+0+0 (secure 0 0 0) 0x7f3b140f1090 con 0x7f3b2c0cc870 2024-09-19T04:19:44.191 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' created 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b0c0678c0 msgr2=0x7f3b0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b0c0678c0 0x7f3b0c069d80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c001570 tx=0x7f3b1c007cb0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c0cc870 msgr2=0x7f3b2c13fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c0cc870 0x7f3b2c13fe40 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f3b1401a080 tx=0x7f3b1401ac90 comp rx=0 tx=0).stop 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 shutdown_connections 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3b0c0678c0 0x7f3b0c069d80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3b2c0cdfd0 0x7f3b2c144dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3b2c0cc870 0x7f3b2c13fe40 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 --2- 172.21.15.76:0/970687637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b2c073100 0x7f3b2c15f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 >> 172.21.15.76:0/970687637 conn(0x7f3b2c0b1c80 msgr2=0x7f3b2c15ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:44.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.192+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 shutdown_connections 2024-09-19T04:19:44.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.196+0000 7f3b32a22640 1 -- 172.21.15.76:0/970687637 wait complete. 2024-09-19T04:19:44.235 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T04:19:44.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b33bc640 1 Processor -- start 2024-09-19T04:19:44.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b33bc640 1 -- start start 2024-09-19T04:19:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b33bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f87ac1625b0 0x7f87ac169000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b33bc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac0afbb0 con 0x7f87ac067e30 2024-09-19T04:19:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b33bc640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87ac0af310 con 0x7f87ac15e260 2024-09-19T04:19:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b33bc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f87ac0b0700 con 0x7f87ac1625b0 2024-09-19T04:19:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b1959640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f87ac1625b0 0x7f87ac169000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b0957640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f87ac067e30 0x7f87ac161ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54920/0 (socket says 172.21.15.76:54920) 2024-09-19T04:19:44.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b0957640 1 -- 172.21.15.76:0/3926796758 learned_addr learned my addr 172.21.15.76:0/3926796758 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b1959640 1 -- 172.21.15.76:0/3926796758 >> v1:172.21.15.116:6789/0 conn(0x7f87ac15e260 legacy=0x7f87ac0afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b1959640 1 -- 172.21.15.76:0/3926796758 >> v1:172.21.15.76:6789/0 conn(0x7f87ac067e30 legacy=0x7f87ac161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87b1959640 1 -- 172.21.15.76:0/3926796758 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac0af5f0 con 0x7f87ac1625b0 2024-09-19T04:19:44.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.352+0000 7f87a3fff640 1 -- 172.21.15.76:0/3926796758 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 907973296 0 0) 0x7f87ac0afbb0 con 0x7f87ac067e30 2024-09-19T04:19:44.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b1959640 1 --2- 172.21.15.76:0/3926796758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac1625b0 0x7f87ac169000 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009d90 tx=0x7f87a401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbb099d2200387b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:44.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87a3fff640 1 -- 172.21.15.76:0/3926796758 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a40b2070 con 0x7f87ac1625b0 2024-09-19T04:19:44.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87a3fff640 1 -- 172.21.15.76:0/3926796758 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87a40a5a10 con 0x7f87ac1625b0 2024-09-19T04:19:44.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87a3fff640 1 -- 172.21.15.76:0/3926796758 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a40a5de0 con 0x7f87ac1625b0 2024-09-19T04:19:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- 172.21.15.76:0/3926796758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac1625b0 msgr2=0x7f87ac169000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 --2- 172.21.15.76:0/3926796758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac1625b0 0x7f87ac169000 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f87a4009d90 tx=0x7f87a401c930 comp rx=0 tx=0).stop 2024-09-19T04:19:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- 172.21.15.76:0/3926796758 shutdown_connections 2024-09-19T04:19:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 --2- 172.21.15.76:0/3926796758 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac1625b0 0x7f87ac169000 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- 172.21.15.76:0/3926796758 >> 172.21.15.76:0/3926796758 conn(0x7f87ac073580 msgr2=0x7f87ac06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- 172.21.15.76:0/3926796758 shutdown_connections 2024-09-19T04:19:44.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- 172.21.15.76:0/3926796758 wait complete. 2024-09-19T04:19:44.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 Processor -- start 2024-09-19T04:19:44.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- start start 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87ac067e30 0x7f87ac0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac15e260 0x7f87ac0ce280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87ac1625b0 0x7f87ac154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac16c0f0 con 0x7f87ac15e260 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac16bf70 con 0x7f87ac1625b0 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b33bc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f87ac16c270 con 0x7f87ac067e30 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac15e260 0x7f87ac0ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac15e260 0x7f87ac0ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58946/0 (socket says 172.21.15.76:58946) 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 -- 172.21.15.76:0/4175840821 learned_addr learned my addr 172.21.15.76:0/4175840821 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 -- 172.21.15.76:0/4175840821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87ac067e30 msgr2=0x7f87ac0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87ac067e30 0x7f87ac0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 -- 172.21.15.76:0/4175840821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87ac1625b0 msgr2=0x7f87ac154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b1959640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87ac1625b0 0x7f87ac154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87ac1625b0 0x7f87ac154a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 -- 172.21.15.76:0/4175840821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac0cffa0 con 0x7f87ac15e260 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.356+0000 7f87b0957640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac15e260 0x7f87ac0ce280 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f879c0011f0 tx=0x7f879c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87a1ffb640 1 -- 172.21.15.76:0/4175840821 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c018070 con 0x7f87ac15e260 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87a1ffb640 1 -- 172.21.15.76:0/4175840821 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879c0074e0 con 0x7f87ac15e260 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac0d0210 con 0x7f87ac15e260 2024-09-19T04:19:44.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87a1ffb640 1 -- 172.21.15.76:0/4175840821 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c004a10 con 0x7f87ac15e260 2024-09-19T04:19:44.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac0d04f0 con 0x7f87ac15e260 2024-09-19T04:19:44.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87ac154330 con 0x7f87ac15e260 2024-09-19T04:19:44.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87a1ffb640 1 -- 172.21.15.76:0/4175840821 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f879c01b070 con 0x7f87ac15e260 2024-09-19T04:19:44.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87a1ffb640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87900678c0 0x7f8790069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:44.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87a1ffb640 1 -- 172.21.15.76:0/4175840821 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 6001+0+0 (secure 0 0 0) 0x7f879c085070 con 0x7f87ac15e260 2024-09-19T04:19:44.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87b1158640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87900678c0 0x7f8790069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:44.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.360+0000 7f87b1158640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87900678c0 0x7f8790069d80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0b6330 tx=0x7f87a80079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:44.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.364+0000 7f87a1ffb640 1 -- 172.21.15.76:0/4175840821 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f879c055b10 con 0x7f87ac15e260 2024-09-19T04:19:44.548 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:44.548+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f87ac161ea0 con 0x7f87ac15e260 2024-09-19T04:19:45.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.192+0000 7f87a1ffb640 1 -- 172.21.15.76:0/4175840821 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v99) ==== 174+0+0 (secure 0 0 0) 0x7f879c055410 con 0x7f87ac15e260 2024-09-19T04:19:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:new fs with metadata pool 15 and data pool 16 2024-09-19T04:19:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87900678c0 msgr2=0x7f8790069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87900678c0 0x7f8790069d80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0b6330 tx=0x7f87a80079c0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac15e260 msgr2=0x7f87ac0ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac15e260 0x7f87ac0ce280 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f879c0011f0 tx=0x7f879c00c9b0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 shutdown_connections 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f87900678c0 0x7f8790069d80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87ac1625b0 0x7f87ac154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f87ac15e260 0x7f87ac0ce280 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 --2- 172.21.15.76:0/4175840821 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f87ac067e30 0x7f87ac0cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.196+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 >> 172.21.15.76:0/4175840821 conn(0x7f87ac073580 msgr2=0x7f87ac156330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.200+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 shutdown_connections 2024-09-19T04:19:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.200+0000 7f87b33bc640 1 -- 172.21.15.76:0/4175840821 wait complete. 2024-09-19T04:19:45.225 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:19:45.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8ee7c640 1 Processor -- start 2024-09-19T04:19:45.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8ee7c640 1 -- start start 2024-09-19T04:19:45.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8ee7c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e88161a60 0x7f2e880d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8ee7c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880cf620 0x7f2e8806d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8ee7c640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e880d32b0 con 0x7f2e880d2e90 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8ee7c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f2e880d0300 con 0x7f2e88161a60 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8ee7c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2e880cfab0 con 0x7f2e880cf620 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8cc18640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e88161a60 0x7f2e880d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8cc18640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2e88161a60 0x7f2e880d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58962/0 (socket says 172.21.15.76:58962) 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8cc18640 1 -- 172.21.15.76:0/4253404743 learned_addr learned my addr 172.21.15.76:0/4253404743 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8d419640 1 --2- 172.21.15.76:0/4253404743 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880cf620 0x7f2e8806d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8cc18640 1 -- 172.21.15.76:0/4253404743 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880cf620 msgr2=0x7f2e8806d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8cc18640 1 --2- 172.21.15.76:0/4253404743 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880cf620 0x7f2e8806d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8cc18640 1 -- 172.21.15.76:0/4253404743 >> v1:172.21.15.116:6789/0 conn(0x7f2e880d2e90 legacy=0x7f2e880cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.340+0000 7f2e8cc18640 1 -- 172.21.15.76:0/4253404743 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e880b0f60 con 0x7f2e88161a60 2024-09-19T04:19:45.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8cc18640 1 --2- 172.21.15.76:0/4253404743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880d2950 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009870 tx=0x7f2e7801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275e893fc18fd689 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e877fe640 1 -- 172.21.15.76:0/4253404743 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780ab620 con 0x7f2e88161a60 2024-09-19T04:19:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e877fe640 1 -- 172.21.15.76:0/4253404743 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e780b0070 con 0x7f2e88161a60 2024-09-19T04:19:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e877fe640 1 -- 172.21.15.76:0/4253404743 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e780ab9e0 con 0x7f2e88161a60 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/4253404743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 msgr2=0x7f2e880d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/4253404743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880d2950 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f2e78009870 tx=0x7f2e7801c460 comp rx=0 tx=0).stop 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/4253404743 shutdown_connections 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/4253404743 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880cf620 0x7f2e8806d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/4253404743 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880d2950 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/4253404743 >> 172.21.15.76:0/4253404743 conn(0x7f2e8808bf90 msgr2=0x7f2e8809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/4253404743 shutdown_connections 2024-09-19T04:19:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/4253404743 wait complete. 2024-09-19T04:19:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 Processor -- start 2024-09-19T04:19:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.344+0000 7f2e8ee7c640 1 -- start start 2024-09-19T04:19:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e880cf620 0x7f2e880b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880d2e90 0x7f2e880b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8806e400 con 0x7f2e88161a60 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2e8806d190 con 0x7f2e880cf620 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2e8806e580 con 0x7f2e880d2e90 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58964/0 (socket says 172.21.15.76:58964) 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 -- 172.21.15.76:0/2085550757 learned_addr learned my addr 172.21.15.76:0/2085550757 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e87fff640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880d2e90 0x7f2e880b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8cc18640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e880cf620 0x7f2e880b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 -- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880d2e90 msgr2=0x7f2e880b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880d2e90 0x7f2e880b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 -- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e880cf620 msgr2=0x7f2e880b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e880cf620 0x7f2e880b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 -- 172.21.15.76:0/2085550757 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e880ce1a0 con 0x7f2e88161a60 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8d419640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880cba00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000cc30 tx=0x7f2e8000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e857fa640 1 -- 172.21.15.76:0/2085550757 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8000ce40 con 0x7f2e88161a60 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e857fa640 1 -- 172.21.15.76:0/2085550757 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e800039a0 con 0x7f2e88161a60 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e857fa640 1 -- 172.21.15.76:0/2085550757 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e80003d50 con 0x7f2e88161a60 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e880cd100 con 0x7f2e88161a60 2024-09-19T04:19:45.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e880cd6c0 con 0x7f2e88161a60 2024-09-19T04:19:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e8806d6b0 con 0x7f2e88161a60 2024-09-19T04:19:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e857fa640 1 -- 172.21.15.76:0/2085550757 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2e80003f30 con 0x7f2e88161a60 2024-09-19T04:19:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e857fa640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e740678f0 0x7f2e74069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e857fa640 1 -- 172.21.15.76:0/2085550757 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f2e80086300 con 0x7f2e88161a60 2024-09-19T04:19:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8cc18640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e740678f0 0x7f2e74069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.348+0000 7f2e8cc18640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e740678f0 0x7f2e74069db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e780af000 tx=0x7f2e780a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.352+0000 7f2e857fa640 1 -- 172.21.15.76:0/2085550757 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2e80014030 con 0x7f2e88161a60 2024-09-19T04:19:45.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.520+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2e880d1950 con 0x7f2e88161a60 2024-09-19T04:19:45.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.520+0000 7f2e857fa640 1 -- 172.21.15.76:0/2085550757 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16981 (secure 0 0 0) 0x7f2e80056c20 con 0x7f2e88161a60 2024-09-19T04:19:45.523 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:45.523 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":103,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:19:44.552119+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:19:43.130613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T04:19:43.548160+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e740678f0 msgr2=0x7f2e74069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e740678f0 0x7f2e74069db0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2e780af000 tx=0x7f2e780a7000 comp rx=0 tx=0).stop 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 msgr2=0x7f2e880cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880cba00 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f2e8000cc30 tx=0x7f2e8000a5c0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 shutdown_connections 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2e740678f0 0x7f2e74069db0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2e88161a60 0x7f2e880cba00 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2e880d2e90 0x7f2e880b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 --2- 172.21.15.76:0/2085550757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e880cf620 0x7f2e880b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 >> 172.21.15.76:0/2085550757 conn(0x7f2e8808bf90 msgr2=0x7f2e8815e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:45.526 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 shutdown_connections 2024-09-19T04:19:45.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.524+0000 7f2e8ee7c640 1 -- 172.21.15.76:0/2085550757 wait complete. 2024-09-19T04:19:45.542 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:19:45.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74e53e7640 1 Processor -- start 2024-09-19T04:19:45.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74e53e7640 1 -- start start 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74e53e7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f74e00d3060 0x7f74e00cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74e53e7640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74e00cf450 0x7f74e006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74e53e7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74e00d0300 con 0x7f74e0161a60 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74e53e7640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f74e00b0f60 con 0x7f74e00cf450 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74e53e7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f74e00cfd40 con 0x7f74e00d3060 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74df7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74e00cf450 0x7f74e006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74df7fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74e00cf450 0x7f74e006d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58976/0 (socket says 172.21.15.76:58976) 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74df7fe640 1 -- 172.21.15.76:0/1877495889 learned_addr learned my addr 172.21.15.76:0/1877495889 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:45.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74de7fc640 1 --2- 172.21.15.76:0/1877495889 >> v2:172.21.15.116:3300/0 conn(0x7f74e00d3060 0x7f74e00cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74df7fe640 1 -- 172.21.15.76:0/1877495889 >> v2:172.21.15.116:3300/0 conn(0x7f74e00d3060 msgr2=0x7f74e00cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74df7fe640 1 --2- 172.21.15.76:0/1877495889 >> v2:172.21.15.116:3300/0 conn(0x7f74e00d3060 0x7f74e00cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74df7fe640 1 -- 172.21.15.76:0/1877495889 >> v1:172.21.15.116:6789/0 conn(0x7f74e0161a60 legacy=0x7f74e00d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:45.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.652+0000 7f74df7fe640 1 -- 172.21.15.76:0/1877495889 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e00d0020 con 0x7f74e00cf450 2024-09-19T04:19:45.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74df7fe640 1 --2- 172.21.15.76:0/1877495889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00cf450 0x7f74e006d6b0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f74e008d850 tx=0x7f74d001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e97c392e10b6ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74ddffb640 1 -- 172.21.15.76:0/1877495889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00b0070 con 0x7f74e00cf450 2024-09-19T04:19:45.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74ddffb640 1 -- 172.21.15.76:0/1877495889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d00ab600 con 0x7f74e00cf450 2024-09-19T04:19:45.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74ddffb640 1 -- 172.21.15.76:0/1877495889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d00ab9d0 con 0x7f74e00cf450 2024-09-19T04:19:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- 172.21.15.76:0/1877495889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00cf450 msgr2=0x7f74e006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 --2- 172.21.15.76:0/1877495889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00cf450 0x7f74e006d6b0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f74e008d850 tx=0x7f74d001c240 comp rx=0 tx=0).stop 2024-09-19T04:19:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- 172.21.15.76:0/1877495889 shutdown_connections 2024-09-19T04:19:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 --2- 172.21.15.76:0/1877495889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00cf450 0x7f74e006d6b0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 --2- 172.21.15.76:0/1877495889 >> v2:172.21.15.116:3300/0 conn(0x7f74e00d3060 0x7f74e00cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- 172.21.15.76:0/1877495889 >> 172.21.15.76:0/1877495889 conn(0x7f74e008bf90 msgr2=0x7f74e009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- 172.21.15.76:0/1877495889 shutdown_connections 2024-09-19T04:19:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- 172.21.15.76:0/1877495889 wait complete. 2024-09-19T04:19:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 Processor -- start 2024-09-19T04:19:45.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- start start 2024-09-19T04:19:45.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74e00cf450 0x7f74e00b9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00d3060 0x7f74e00ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74e0161a60 0x7f74e0152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74e006e400 con 0x7f74e00d3060 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f74e00cfad0 con 0x7f74e00cf450 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74e53e7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f74e006e580 con 0x7f74e0161a60 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74de7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00d3060 0x7f74e00ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74de7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00d3060 0x7f74e00ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58988/0 (socket says 172.21.15.76:58988) 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.656+0000 7f74de7fc640 1 -- 172.21.15.76:0/3948058685 learned_addr learned my addr 172.21.15.76:0/3948058685 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74deffd640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74e00cf450 0x7f74e00b9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74df7fe640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74e0161a60 0x7f74e0152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74de7fc640 1 -- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74e0161a60 msgr2=0x7f74e0152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74de7fc640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74e0161a60 0x7f74e0152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74de7fc640 1 -- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74e00cf450 msgr2=0x7f74e00b9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74de7fc640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74e00cf450 0x7f74e00b9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74de7fc640 1 -- 172.21.15.76:0/3948058685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e00c8260 con 0x7f74e00d3060 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74deffd640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74e00cf450 0x7f74e00b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74de7fc640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00d3060 0x7f74e00ca6e0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f74c800d620 tx=0x7f74c800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74d77fe640 1 -- 172.21.15.76:0/3948058685 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800b620 con 0x7f74e00d3060 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74d77fe640 1 -- 172.21.15.76:0/3948058685 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74c8019070 con 0x7f74e00d3060 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e00c8540 con 0x7f74e00d3060 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74d77fe640 1 -- 172.21.15.76:0/3948058685 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c800ba20 con 0x7f74e00d3060 2024-09-19T04:19:45.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e00c8de0 con 0x7f74e00d3060 2024-09-19T04:19:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e006d6b0 con 0x7f74e00d3060 2024-09-19T04:19:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74d77fe640 1 -- 172.21.15.76:0/3948058685 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f74c800bbc0 con 0x7f74e00d3060 2024-09-19T04:19:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74d77fe640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c00678c0 0x7f74c0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74d77fe640 1 -- 172.21.15.76:0/3948058685 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7f74c80862c0 con 0x7f74e00d3060 2024-09-19T04:19:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74deffd640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c00678c0 0x7f74c0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.660+0000 7f74deffd640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c00678c0 0x7f74c0069d80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74e015a9a0 tx=0x7f74d8009570 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.664+0000 7f74d77fe640 1 -- 172.21.15.76:0/3948058685 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f74c8056c70 con 0x7f74e00d3060 2024-09-19T04:19:45.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.828+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74e00b1330 con 0x7f74e00d3060 2024-09-19T04:19:45.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.832+0000 7f74d77fe640 1 -- 172.21.15.76:0/3948058685 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16981 (secure 0 0 0) 0x7f74c80860e0 con 0x7f74e00d3060 2024-09-19T04:19:45.834 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:45.834 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":103,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:19:44.552119+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:19:43.130613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T04:19:43.548160+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:19:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c00678c0 msgr2=0x7f74c0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c00678c0 0x7f74c0069d80 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f74e015a9a0 tx=0x7f74d8009570 comp rx=0 tx=0).stop 2024-09-19T04:19:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00d3060 msgr2=0x7f74e00ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00d3060 0x7f74e00ca6e0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f74c800d620 tx=0x7f74c800dae0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 shutdown_connections 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c00678c0 0x7f74c0069d80 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74e0161a60 0x7f74e0152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74e00d3060 0x7f74e00ca6e0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 --2- 172.21.15.76:0/3948058685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74e00cf450 0x7f74e00b9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 >> 172.21.15.76:0/3948058685 conn(0x7f74e008bf90 msgr2=0x7f74e015d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 shutdown_connections 2024-09-19T04:19:45.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.836+0000 7f74e53e7640 1 -- 172.21.15.76:0/3948058685 wait complete. 2024-09-19T04:19:45.853 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-19T04:19:45.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26cf53640 1 Processor -- start 2024-09-19T04:19:45.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26cf53640 1 -- start start 2024-09-19T04:19:45.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26cf53640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd26815c2a0 0x7fd2680aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26cf53640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd2680af2a0 0x7fd26806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26cf53640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2680b3290 con 0x7fd26815bec0 2024-09-19T04:19:45.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26cf53640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd2680b3570 con 0x7fd26815c2a0 2024-09-19T04:19:45.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26cf53640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd2680b3850 con 0x7fd2680af2a0 2024-09-19T04:19:45.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd266d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd2680af2a0 0x7fd26806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd265d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd26815c2a0 0x7fd2680aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26659c640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd26815bec0 0x7fd2680ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:58542/0 (socket says 172.21.15.76:58542) 2024-09-19T04:19:45.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd26659c640 1 -- 172.21.15.76:0/1068324020 learned_addr learned my addr 172.21.15.76:0/1068324020 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:45.974 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd266d9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd2680af2a0 0x7fd26806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:60240/0 (socket says 172.21.15.76:60240) 2024-09-19T04:19:45.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd265d9b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd26815c2a0 0x7fd2680aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:47240/0 (socket says 172.21.15.76:47240) 2024-09-19T04:19:45.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd266d9d640 1 -- 172.21.15.76:0/1068324020 >> v2:172.21.15.116:3300/0 conn(0x7fd26815c2a0 msgr2=0x7fd2680aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:45.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd266d9d640 1 --2- 172.21.15.76:0/1068324020 >> v2:172.21.15.116:3300/0 conn(0x7fd26815c2a0 0x7fd2680aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd266d9d640 1 -- 172.21.15.76:0/1068324020 >> v1:172.21.15.116:6789/0 conn(0x7fd26815bec0 legacy=0x7fd2680ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd266d9d640 1 -- 172.21.15.76:0/1068324020 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2680b3b30 con 0x7fd2680af2a0 2024-09-19T04:19:45.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.972+0000 7fd265d9b640 1 --2- 172.21.15.76:0/1068324020 >> v2:172.21.15.116:3300/0 conn(0x7fd26815c2a0 0x7fd2680aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:19:45.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd266d9d640 1 --2- 172.21.15.76:0/1068324020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd2680af2a0 0x7fd26806e000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd260009d90 tx=0x7fd26001c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5fd2aaca807633a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26559a640 1 -- 172.21.15.76:0/1068324020 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600b2070 con 0x7fd2680af2a0 2024-09-19T04:19:45.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26559a640 1 -- 172.21.15.76:0/1068324020 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2600a5a00 con 0x7fd2680af2a0 2024-09-19T04:19:45.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26559a640 1 -- 172.21.15.76:0/1068324020 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2600a5dd0 con 0x7fd2680af2a0 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 -- 172.21.15.76:0/1068324020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd2680af2a0 msgr2=0x7fd26806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 --2- 172.21.15.76:0/1068324020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd2680af2a0 0x7fd26806e000 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd260009d90 tx=0x7fd26001c940 comp rx=0 tx=0).stop 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 -- 172.21.15.76:0/1068324020 shutdown_connections 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 --2- 172.21.15.76:0/1068324020 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd2680af2a0 0x7fd26806e000 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 --2- 172.21.15.76:0/1068324020 >> v2:172.21.15.116:3300/0 conn(0x7fd26815c2a0 0x7fd2680aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 -- 172.21.15.76:0/1068324020 >> 172.21.15.76:0/1068324020 conn(0x7fd2680b1f90 msgr2=0x7fd2680b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 -- 172.21.15.76:0/1068324020 shutdown_connections 2024-09-19T04:19:45.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 -- 172.21.15.76:0/1068324020 wait complete. 2024-09-19T04:19:45.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 Processor -- start 2024-09-19T04:19:45.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.976+0000 7fd26cf53640 1 -- start start 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2680af2a0 0x7fd26814b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd26815bec0 0x7fd2681552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd26815c2a0 0x7fd268155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd26806ed50 con 0x7fd2680af2a0 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680afc90 con 0x7fd26815c2a0 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd26806eed0 con 0x7fd26815bec0 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2680af2a0 0x7fd26814b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2680af2a0 0x7fd26814b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:58992/0 (socket says 172.21.15.76:58992) 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 -- 172.21.15.76:0/3832699764 learned_addr learned my addr 172.21.15.76:0/3832699764 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd266d9d640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd26815c2a0 0x7fd268155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 -- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd26815bec0 msgr2=0x7fd2681552e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd265d9b640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd26815bec0 0x7fd2681552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd26815bec0 0x7fd2681552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 -- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd26815c2a0 msgr2=0x7fd268155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd26815c2a0 0x7fd268155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 -- 172.21.15.76:0/3832699764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26814d280 con 0x7fd2680af2a0 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd266d9d640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd26815c2a0 0x7fd268155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26659c640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2680af2a0 0x7fd26814b800 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c420 tx=0x7fd25800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd256ffd640 1 -- 172.21.15.76:0/3832699764 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258017070 con 0x7fd2680af2a0 2024-09-19T04:19:45.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd256ffd640 1 -- 172.21.15.76:0/3832699764 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd258005cf0 con 0x7fd2680af2a0 2024-09-19T04:19:45.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26814d520 con 0x7fd2680af2a0 2024-09-19T04:19:45.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd256ffd640 1 -- 172.21.15.76:0/3832699764 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2580072f0 con 0x7fd2680af2a0 2024-09-19T04:19:45.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26814dae0 con 0x7fd2680af2a0 2024-09-19T04:19:45.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26806e000 con 0x7fd2680af2a0 2024-09-19T04:19:45.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd256ffd640 1 -- 172.21.15.76:0/3832699764 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd25801a070 con 0x7fd2680af2a0 2024-09-19T04:19:45.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd256ffd640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd2480678c0 0x7fd248069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:45.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd256ffd640 1 -- 172.21.15.76:0/3832699764 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 6101+0+0 (secure 0 0 0) 0x7fd258085220 con 0x7fd2680af2a0 2024-09-19T04:19:45.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd265d9b640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd2480678c0 0x7fd248069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:45.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.980+0000 7fd265d9b640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd2480678c0 0x7fd248069d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd26814e5a0 tx=0x7fd25c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:45.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:45.984+0000 7fd256ffd640 1 -- 172.21.15.76:0/3832699764 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd258055bd0 con 0x7fd2680af2a0 2024-09-19T04:19:46.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:46.172+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd2680ae300 con 0x7fd2680af2a0 2024-09-19T04:19:47.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.148+0000 7fd256ffd640 1 -- 172.21.15.76:0/3832699764 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) ==== 118+0+0 (secure 0 0 0) 0x7fd2580554d0 con 0x7fd2680af2a0 2024-09-19T04:19:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd2480678c0 msgr2=0x7fd248069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd2480678c0 0x7fd248069d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd26814e5a0 tx=0x7fd25c008040 comp rx=0 tx=0).stop 2024-09-19T04:19:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2680af2a0 msgr2=0x7fd26814b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2680af2a0 0x7fd26814b800 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c420 tx=0x7fd25800c8e0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 shutdown_connections 2024-09-19T04:19:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd2480678c0 0x7fd248069d80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd26815c2a0 0x7fd268155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd26815bec0 0x7fd2681552e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 --2- 172.21.15.76:0/3832699764 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd2680af2a0 0x7fd26814b800 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 >> 172.21.15.76:0/3832699764 conn(0x7fd2680b1f90 msgr2=0x7fd2680aff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 shutdown_connections 2024-09-19T04:19:47.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.152+0000 7fd26cf53640 1 -- 172.21.15.76:0/3832699764 wait complete. 2024-09-19T04:19:47.174 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-19T04:19:47.175 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-19T04:19:47.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff844491640 1 Processor -- start 2024-09-19T04:19:47.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff844491640 1 -- start start 2024-09-19T04:19:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff844491640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff83c15c2a0 0x7ff83c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff844491640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff844491640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83c0b3290 con 0x7ff83c15bec0 2024-09-19T04:19:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff844491640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff83c0b3570 con 0x7ff83c15c2a0 2024-09-19T04:19:47.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff844491640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c0b3850 con 0x7ff83c0af2a0 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff84222d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff83c15bec0 0x7ff83c0ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54932/0 (socket says 172.21.15.76:54932) 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff84222d640 1 -- 172.21.15.76:0/3217352531 learned_addr learned my addr 172.21.15.76:0/3217352531 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff841a2c640 1 --2- 172.21.15.76:0/3217352531 >> v2:172.21.15.76:3300/0 conn(0x7ff83c15c2a0 0x7ff83c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff84122b640 1 -- 172.21.15.76:0/3217352531 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2251454248 0 0) 0x7ff83c0b3290 con 0x7ff83c15bec0 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff842a2e640 1 --2- 172.21.15.76:0/3217352531 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff84122b640 1 -- 172.21.15.76:0/3217352531 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff828003530 con 0x7ff83c15bec0 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff841a2c640 1 -- 172.21.15.76:0/3217352531 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 msgr2=0x7ff83c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff841a2c640 1 --2- 172.21.15.76:0/3217352531 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c06e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff841a2c640 1 -- 172.21.15.76:0/3217352531 >> v1:172.21.15.76:6789/0 conn(0x7ff83c15bec0 legacy=0x7ff83c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.304+0000 7ff841a2c640 1 -- 172.21.15.76:0/3217352531 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c0b3b30 con 0x7ff83c15c2a0 2024-09-19T04:19:47.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 --2- 172.21.15.76:0/3217352531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15c2a0 0x7ff83c0aed60 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0097c0 tx=0x7ff82c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f232004f18a04e67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff84122b640 1 -- 172.21.15.76:0/3217352531 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0a7070 con 0x7ff83c15c2a0 2024-09-19T04:19:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff84122b640 1 -- 172.21.15.76:0/3217352531 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c0b0070 con 0x7ff83c15c2a0 2024-09-19T04:19:47.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff84122b640 1 -- 172.21.15.76:0/3217352531 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0ab870 con 0x7ff83c15c2a0 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- 172.21.15.76:0/3217352531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15c2a0 msgr2=0x7ff83c0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 --2- 172.21.15.76:0/3217352531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15c2a0 0x7ff83c0aed60 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0097c0 tx=0x7ff82c01c470 comp rx=0 tx=0).stop 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- 172.21.15.76:0/3217352531 shutdown_connections 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 --2- 172.21.15.76:0/3217352531 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 --2- 172.21.15.76:0/3217352531 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15c2a0 0x7ff83c0aed60 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- 172.21.15.76:0/3217352531 >> 172.21.15.76:0/3217352531 conn(0x7ff83c0b1f90 msgr2=0x7ff83c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- 172.21.15.76:0/3217352531 shutdown_connections 2024-09-19T04:19:47.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- 172.21.15.76:0/3217352531 wait complete. 2024-09-19T04:19:47.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 Processor -- start 2024-09-19T04:19:47.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- start start 2024-09-19T04:19:47.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c14c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:47.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15bec0 0x7ff83c150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c15c2a0 0x7ff83c150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c06ed50 con 0x7ff83c15bec0 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0af6b0 con 0x7ff83c15c2a0 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff844491640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c06eed0 con 0x7ff83c0af2a0 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15bec0 0x7ff83c150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15bec0 0x7ff83c150430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59002/0 (socket says 172.21.15.76:59002) 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 -- 172.21.15.76:0/265469408 learned_addr learned my addr 172.21.15.76:0/265469408 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff84222d640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c14c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff842a2e640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c15c2a0 0x7ff83c150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 -- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 msgr2=0x7ff83c14c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c14c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.312 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 -- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c15c2a0 msgr2=0x7ff83c150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c15c2a0 0x7ff83c150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 -- 172.21.15.76:0/265469408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c14c890 con 0x7ff83c15bec0 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff84222d640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c14c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.308+0000 7ff841a2c640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15bec0 0x7ff83c150430 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff82c01a0a0 tx=0x7ff82c019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff832ffd640 1 -- 172.21.15.76:0/265469408 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c003b30 con 0x7ff83c15bec0 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff832ffd640 1 -- 172.21.15.76:0/265469408 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c003cd0 con 0x7ff83c15bec0 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff832ffd640 1 -- 172.21.15.76:0/265469408 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0c0680 con 0x7ff83c15bec0 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c15f260 con 0x7ff83c15bec0 2024-09-19T04:19:47.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c1f5da0 con 0x7ff83c15bec0 2024-09-19T04:19:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c06e000 con 0x7ff83c15bec0 2024-09-19T04:19:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff832ffd640 1 -- 172.21.15.76:0/265469408 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff82c0ab470 con 0x7ff83c15bec0 2024-09-19T04:19:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff832ffd640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8200678c0 0x7ff820069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff832ffd640 1 -- 172.21.15.76:0/265469408 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7ff82c120c90 con 0x7ff83c15bec0 2024-09-19T04:19:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff84222d640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8200678c0 0x7ff820069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:47.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.312+0000 7ff84222d640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8200678c0 0x7ff820069d80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff838005890 tx=0x7ff838009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:47.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.316+0000 7ff832ffd640 1 -- 172.21.15.76:0/265469408 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff82c016090 con 0x7ff83c15bec0 2024-09-19T04:19:47.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:47.516+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff83c06fcd0 con 0x7ff83c15bec0 2024-09-19T04:19:48.148 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:19:48.148+0000 7f3e47568640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T04:19:48.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.156+0000 7ff832ffd640 1 -- 172.21.15.76:0/265469408 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v102) ==== 122+0+0 (secure 0 0 0) 0x7ff82c0f16d0 con 0x7ff83c15bec0 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8200678c0 msgr2=0x7ff820069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8200678c0 0x7ff820069d80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff838005890 tx=0x7ff838009040 comp rx=0 tx=0).stop 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15bec0 msgr2=0x7ff83c150430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15bec0 0x7ff83c150430 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff82c01a0a0 tx=0x7ff82c019210 comp rx=0 tx=0).stop 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 shutdown_connections 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff8200678c0 0x7ff820069d80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c15c2a0 0x7ff83c150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff83c15bec0 0x7ff83c150430 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 --2- 172.21.15.76:0/265469408 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff83c0af2a0 0x7ff83c14c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 >> 172.21.15.76:0/265469408 conn(0x7ff83c0b1f90 msgr2=0x7ff83c15b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 shutdown_connections 2024-09-19T04:19:48.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.160+0000 7ff844491640 1 -- 172.21.15.76:0/265469408 wait complete. 2024-09-19T04:19:48.181 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-19T04:19:48.181 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0167393640 1 Processor -- start 2024-09-19T04:19:48.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0167393640 1 -- start start 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0167393640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 0x7f0160165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0167393640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01600aede0 con 0x7f0160070020 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0167393640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f016015fa10 con 0x7f01601520a0 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0167393640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0160152480 con 0x7f016015f3a0 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f016492e640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0160070020 0x7f016015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54940/0 (socket says 172.21.15.76:54940) 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f016492e640 1 -- 172.21.15.76:0/3789269407 learned_addr learned my addr 172.21.15.76:0/3789269407 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0165930640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 0x7f0160165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3057604758 0 0) 0x7f01600aede0 con 0x7f0160070020 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f014c003530 con 0x7f0160070020 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3661317820 0 0) 0x7f014c003530 con 0x7f0160070020 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 msgr2=0x7f0160165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 0x7f0160165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 >> v1:172.21.15.116:6789/0 conn(0x7f01601520a0 legacy=0x7f016006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0160074250 con 0x7f0160070020 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f0150001680 con 0x7f0160070020 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 >> v1:172.21.15.76:6789/0 conn(0x7f0160070020 legacy=0x7f016015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 0x7f014c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f014c003f30 0x7f014c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c004d50 0x7f014c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f014c00ba00 con 0x7f014c004d50 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f014c00bbb0 con 0x7f014c003f30 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f014c00bdb0 con 0x7f016015f3a0 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0165930640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c004d50 0x7f014c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f016492e640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f014c003f30 0x7f014c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f016512f640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 0x7f014c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0165930640 1 -- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 msgr2=0x7f014c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0165930640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 0x7f014c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0165930640 1 -- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f014c003f30 msgr2=0x7f014c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0165930640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f014c003f30 0x7f014c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f0165930640 1 -- 172.21.15.76:0/3789269407 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c003530 con 0x7f014c004d50 2024-09-19T04:19:48.309 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.304+0000 7f016512f640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f016015f3a0 0x7f014c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:48.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0165930640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c004d50 0x7f014c0051d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0158009fc0 tx=0x7f015801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4662ae95d7de4682 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01580b2070 con 0x7f014c004d50 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01580a5b80 con 0x7f014c004d50 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0157fff640 1 -- 172.21.15.76:0/3789269407 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158009c10 con 0x7f014c004d50 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 -- 172.21.15.76:0/3789269407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c004d50 msgr2=0x7f014c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 --2- 172.21.15.76:0/3789269407 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f014c004d50 0x7f014c0051d0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f0158009fc0 tx=0x7f015801cc80 comp rx=0 tx=0).stop 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0165930640 1 -- 172.21.15.76:0/3789269407 reap_dead start 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 -- 172.21.15.76:0/3789269407 shutdown_connections 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 -- 172.21.15.76:0/3789269407 >> 172.21.15.76:0/3789269407 conn(0x7f01600b1f90 msgr2=0x7f01600b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 -- 172.21.15.76:0/3789269407 shutdown_connections 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 -- 172.21.15.76:0/3789269407 wait complete. 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 Processor -- start 2024-09-19T04:19:48.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.308+0000 7f0167393640 1 -- start start 2024-09-19T04:19:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0160152910 0x7f0160152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f016015e070 0x7f0160154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01601552a0 0x7f01601497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0160073a50 con 0x7f016015e070 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0160073770 con 0x7f01601552a0 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0160073d30 con 0x7f0160152910 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f016015e070 0x7f0160154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f016015e070 0x7f0160154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59024/0 (socket says 172.21.15.76:59024) 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 -- 172.21.15.76:0/4065619334 learned_addr learned my addr 172.21.15.76:0/4065619334 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016512f640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0160152910 0x7f0160152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0165930640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01601552a0 0x7f01601497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 -- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0160152910 msgr2=0x7f0160152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0160152910 0x7f0160152cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 -- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01601552a0 msgr2=0x7f01601497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01601552a0 0x7f01601497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 -- 172.21.15.76:0/4065619334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0160153470 con 0x7f016015e070 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016492e640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f016015e070 0x7f0160154d60 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0150011810 tx=0x7f0150011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0155ffb640 1 -- 172.21.15.76:0/4065619334 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150010040 con 0x7f016015e070 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0155ffb640 1 -- 172.21.15.76:0/4065619334 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f015001a070 con 0x7f016015e070 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0160153750 con 0x7f016015e070 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0155ffb640 1 -- 172.21.15.76:0/4065619334 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015000fd20 con 0x7f016015e070 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0160145330 con 0x7f016015e070 2024-09-19T04:19:48.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0160068e50 con 0x7f016015e070 2024-09-19T04:19:48.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0155ffb640 1 -- 172.21.15.76:0/4065619334 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f015001f460 con 0x7f016015e070 2024-09-19T04:19:48.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0155ffb640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01440678f0 0x7f0144069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f0155ffb640 1 -- 172.21.15.76:0/4065619334 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f0150088f50 con 0x7f016015e070 2024-09-19T04:19:48.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016512f640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01440678f0 0x7f0144069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.316 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.312+0000 7f016512f640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01440678f0 0x7f0144069db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f015c001890 tx=0x7f015c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.320+0000 7f0155ffb640 1 -- 172.21.15.76:0/4065619334 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0150059990 con 0x7f016015e070 2024-09-19T04:19:48.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.512+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f016006e540 con 0x7f016015e070 2024-09-19T04:19:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.512+0000 7f0155ffb640 1 -- 172.21.15.76:0/4065619334 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5230 (secure 0 0 0) 0x7f0150059290 con 0x7f016015e070 2024-09-19T04:19:48.515 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:48.515 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":102,"btime":"2024-09-19T04:19:48:158361+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6127,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:19:47.518980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127]},"id":7}]} 2024-09-19T04:19:48.515 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 102 2024-09-19T04:19:48.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01440678f0 msgr2=0x7f0144069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01440678f0 0x7f0144069db0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f015c001890 tx=0x7f015c009040 comp rx=0 tx=0).stop 2024-09-19T04:19:48.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f016015e070 msgr2=0x7f0160154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f016015e070 0x7f0160154d60 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f0150011810 tx=0x7f0150011cd0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 shutdown_connections 2024-09-19T04:19:48.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f01440678f0 0x7f0144069db0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01601552a0 0x7f01601497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f016015e070 0x7f0160154d60 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 --2- 172.21.15.76:0/4065619334 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0160152910 0x7f0160152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 >> 172.21.15.76:0/4065619334 conn(0x7f01600b1f90 msgr2=0x7f01600b4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 shutdown_connections 2024-09-19T04:19:48.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.516+0000 7f0167393640 1 -- 172.21.15.76:0/4065619334 wait complete. 2024-09-19T04:19:48.533 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:19:48.647 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.644+0000 7f59187f3640 1 Processor -- start 2024-09-19T04:19:48.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f59187f3640 1 -- start start 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f59187f3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5910070020 0x7f591015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f59187f3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f591015f1d0 0x7f59100adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f59187f3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59100aede0 con 0x7f59101520a0 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f59187f3640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5910152480 con 0x7f5910070020 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f59187f3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f591015fa10 con 0x7f591015f1d0 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5910070020 0x7f591015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5910070020 0x7f591015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59030/0 (socket says 172.21.15.76:59030) 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 -- 172.21.15.76:0/1750722645 learned_addr learned my addr 172.21.15.76:0/1750722645 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:48.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 -- 172.21.15.76:0/1750722645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f591015f1d0 msgr2=0x7f59100adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5916d90640 1 --2- 172.21.15.76:0/1750722645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f591015f1d0 0x7f59100adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 --2- 172.21.15.76:0/1750722645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f591015f1d0 0x7f59100adc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 -- 172.21.15.76:0/1750722645 >> v1:172.21.15.116:6789/0 conn(0x7f59101520a0 legacy=0x7f591006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 -- 172.21.15.76:0/1750722645 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5910074250 con 0x7f5910070020 2024-09-19T04:19:48.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f5915d8e640 1 --2- 172.21.15.76:0/1750722645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f591015ec90 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5904009e50 tx=0x7f590401c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee452b5845599f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f591558d640 1 -- 172.21.15.76:0/1750722645 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59040a5840 con 0x7f5910070020 2024-09-19T04:19:48.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f591558d640 1 -- 172.21.15.76:0/1750722645 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59040a59e0 con 0x7f5910070020 2024-09-19T04:19:48.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.648+0000 7f591558d640 1 -- 172.21.15.76:0/1750722645 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59040a5db0 con 0x7f5910070020 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- 172.21.15.76:0/1750722645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 msgr2=0x7f591015ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 --2- 172.21.15.76:0/1750722645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f591015ec90 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5904009e50 tx=0x7f590401c9c0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- 172.21.15.76:0/1750722645 shutdown_connections 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 --2- 172.21.15.76:0/1750722645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f591015f1d0 0x7f59100adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 --2- 172.21.15.76:0/1750722645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f591015ec90 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- 172.21.15.76:0/1750722645 >> 172.21.15.76:0/1750722645 conn(0x7f59100b1f90 msgr2=0x7f59100b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- 172.21.15.76:0/1750722645 shutdown_connections 2024-09-19T04:19:48.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- 172.21.15.76:0/1750722645 wait complete. 2024-09-19T04:19:48.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 Processor -- start 2024-09-19T04:19:48.653 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- start start 2024-09-19T04:19:48.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f5910099ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59101520a0 0x7f591014a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f591015f1d0 0x7f591014fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f591015f600 con 0x7f5910070020 2024-09-19T04:19:48.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5910073d30 con 0x7f591015f1d0 2024-09-19T04:19:48.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f591015f780 con 0x7f59101520a0 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f5910099ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f5910099ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59042/0 (socket says 172.21.15.76:59042) 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 -- 172.21.15.76:0/3983522972 learned_addr learned my addr 172.21.15.76:0/3983522972 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f5916d90640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f591015f1d0 0x7f591014fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f5915d8e640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59101520a0 0x7f591014a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 -- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59101520a0 msgr2=0x7f591014a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59101520a0 0x7f591014a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 -- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f591015f1d0 msgr2=0x7f591014fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f591015f1d0 0x7f591014fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 -- 172.21.15.76:0/3983522972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5910150720 con 0x7f5910070020 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f5915d8e640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59101520a0 0x7f591014a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:48.656 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f5916d90640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f591015f1d0 0x7f591014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:48.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f591658f640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f5910099ad0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f590000b410 tx=0x7f590000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f590effd640 1 -- 172.21.15.76:0/3983522972 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900015030 con 0x7f5910070020 2024-09-19T04:19:48.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f590effd640 1 -- 172.21.15.76:0/3983522972 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5900004020 con 0x7f5910070020 2024-09-19T04:19:48.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591014ebb0 con 0x7f5910070020 2024-09-19T04:19:48.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f590effd640 1 -- 172.21.15.76:0/3983522972 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900004450 con 0x7f5910070020 2024-09-19T04:19:48.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.652+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f591014ee50 con 0x7f5910070020 2024-09-19T04:19:48.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.656+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591015ec90 con 0x7f5910070020 2024-09-19T04:19:48.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.656+0000 7f590effd640 1 -- 172.21.15.76:0/3983522972 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5900018070 con 0x7f5910070020 2024-09-19T04:19:48.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.656+0000 7f590effd640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58fc0678f0 0x7f58fc069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.656+0000 7f590effd640 1 -- 172.21.15.76:0/3983522972 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5900085980 con 0x7f5910070020 2024-09-19T04:19:48.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.656+0000 7f5915d8e640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58fc0678f0 0x7f58fc069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.656+0000 7f5915d8e640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58fc0678f0 0x7f58fc069db0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f59040ae000 tx=0x7f59040a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.660+0000 7f590effd640 1 -- 172.21.15.76:0/3983522972 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f59000563c0 con 0x7f5910070020 2024-09-19T04:19:48.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.808+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f591006e540 con 0x7f5910070020 2024-09-19T04:19:48.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.812+0000 7f590effd640 1 -- 172.21.15.76:0/3983522972 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16968 (secure 0 0 0) 0x7f5900055cc0 con 0x7f5910070020 2024-09-19T04:19:48.814 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:48.814 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":104,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:19:46.137223+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:19:43.130613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T04:19:43.548160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58fc0678f0 msgr2=0x7f58fc069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58fc0678f0 0x7f58fc069db0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f59040ae000 tx=0x7f59040a7070 comp rx=0 tx=0).stop 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 msgr2=0x7f5910099ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f5910099ad0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f590000b410 tx=0x7f590000b8d0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 shutdown_connections 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58fc0678f0 0x7f58fc069db0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f591015f1d0 0x7f591014fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59101520a0 0x7f591014a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 --2- 172.21.15.76:0/3983522972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5910070020 0x7f5910099ad0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 >> 172.21.15.76:0/3983522972 conn(0x7f59100b1f90 msgr2=0x7f5910151590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 shutdown_connections 2024-09-19T04:19:48.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.816+0000 7f59187f3640 1 -- 172.21.15.76:0/3983522972 wait complete. 2024-09-19T04:19:48.831 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:19:48.831 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:19:48.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.944+0000 7f0e23a5d640 1 Processor -- start 2024-09-19T04:19:48.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.944+0000 7f0e23a5d640 1 -- start start 2024-09-19T04:19:48.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e23a5d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0e1c15bec0 0x7f0e1c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e23a5d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0e1c15c2a0 0x7f0e1c0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e23a5d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1c0b3290 con 0x7f0e1c0af2a0 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e23a5d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0b3570 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e23a5d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0e1c0b3850 con 0x7f0e1c15bec0 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e20ff8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f0e1c15c2a0 0x7f0e1c0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e21ffa640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0e1c0af2a0 0x7f0e1c15fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54948/0 (socket says 172.21.15.76:54948) 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e21ffa640 1 -- 172.21.15.76:0/2783665314 learned_addr learned my addr 172.21.15.76:0/2783665314 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e217f9640 1 --2- 172.21.15.76:0/2783665314 >> v2:172.21.15.116:3300/0 conn(0x7f0e1c15bec0 0x7f0e1c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e20ff8640 1 -- 172.21.15.76:0/2783665314 >> v2:172.21.15.116:3300/0 conn(0x7f0e1c15bec0 msgr2=0x7f0e1c0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e20ff8640 1 --2- 172.21.15.76:0/2783665314 >> v2:172.21.15.116:3300/0 conn(0x7f0e1c15bec0 0x7f0e1c0ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e20ff8640 1 -- 172.21.15.76:0/2783665314 >> v1:172.21.15.76:6789/0 conn(0x7f0e1c0af2a0 legacy=0x7f0e1c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e20ff8640 1 -- 172.21.15.76:0/2783665314 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0b3b30 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e13fff640 1 -- 172.21.15.76:0/2783665314 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3930144406 0 0) 0x7f0e1c0b3290 con 0x7f0e1c0af2a0 2024-09-19T04:19:48.950 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e217f9640 1 --2- 172.21.15.76:0/2783665314 >> v2:172.21.15.116:3300/0 conn(0x7f0e1c15bec0 0x7f0e1c0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e20ff8640 1 --2- 172.21.15.76:0/2783665314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aed60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c071710 tx=0x7f0e0c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985b911ae5bb2aa1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e13fff640 1 -- 172.21.15.76:0/2783665314 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0ab650 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e13fff640 1 -- 172.21.15.76:0/2783665314 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0c0b0070 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.948+0000 7f0e13fff640 1 -- 172.21.15.76:0/2783665314 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c0aba10 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2783665314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 msgr2=0x7f0e1c0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2783665314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aed60 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c071710 tx=0x7f0e0c01c450 comp rx=0 tx=0).stop 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2783665314 shutdown_connections 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2783665314 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aed60 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2783665314 >> v2:172.21.15.116:3300/0 conn(0x7f0e1c15bec0 0x7f0e1c0ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2783665314 >> 172.21.15.76:0/2783665314 conn(0x7f0e1c0b1f90 msgr2=0x7f0e1c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:48.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2783665314 shutdown_connections 2024-09-19T04:19:48.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2783665314 wait complete. 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 Processor -- start 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- start start 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0e1c0af2a0 0x7f0e1c0b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e1c15bec0 0x7f0e1c14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0af6d0 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1c0ad740 con 0x7f0e1c15bec0 2024-09-19T04:19:48.955 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1c169000 con 0x7f0e1c0af2a0 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59070/0 (socket says 172.21.15.76:59070) 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 -- 172.21.15.76:0/2435307679 learned_addr learned my addr 172.21.15.76:0/2435307679 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e20ff8640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e1c15bec0 0x7f0e1c14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e217f9640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0e1c0af2a0 0x7f0e1c0b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 -- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0e1c0af2a0 msgr2=0x7f0e1c0b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0e1c0af2a0 0x7f0e1c0b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 -- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e1c15bec0 msgr2=0x7f0e1c14c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e1c15bec0 0x7f0e1c14c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 -- 172.21.15.76:0/2435307679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1c0b1700 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e21ffa640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aff90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c7a0 tx=0x7f0e1400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e11ffb640 1 -- 172.21.15.76:0/2435307679 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e14018070 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e11ffb640 1 -- 172.21.15.76:0/2435307679 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e1400a640 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1c156a80 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e11ffb640 1 -- 172.21.15.76:0/2435307679 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400aa70 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.952+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1c157040 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.956+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e1c0ad200 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.956+0000 7f0e11ffb640 1 -- 172.21.15.76:0/2435307679 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0e1400ac10 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.956+0000 7f0e11ffb640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0e000678c0 0x7f0e00069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:48.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.956+0000 7f0e11ffb640 1 -- 172.21.15.76:0/2435307679 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f0e140854a0 con 0x7f0e1c15c2a0 2024-09-19T04:19:48.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.956+0000 7f0e217f9640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0e000678c0 0x7f0e00069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:48.958 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.956+0000 7f0e217f9640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0e000678c0 0x7f0e00069d80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0b4840 tx=0x7f0e18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:48.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:48.960+0000 7f0e11ffb640 1 -- 172.21.15.76:0/2435307679 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0e14055e60 con 0x7f0e1c15c2a0 2024-09-19T04:19:49.120 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.120+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e1c06fcd0 con 0x7f0e000678c0 2024-09-19T04:19:49.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.132+0000 7f0e11ffb640 1 -- 172.21.15.76:0/2435307679 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7f0e1c06fcd0 con 0x7f0e000678c0 2024-09-19T04:19:49.135 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:49.136 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:19:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0e000678c0 msgr2=0x7f0e00069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0e000678c0 0x7f0e00069d80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c0b4840 tx=0x7f0e18008040 comp rx=0 tx=0).stop 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 msgr2=0x7f0e1c0aff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aff90 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400c7a0 tx=0x7f0e1400cc60 comp rx=0 tx=0).stop 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 shutdown_connections 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0e000678c0 0x7f0e00069d80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0e1c15c2a0 0x7f0e1c0aff90 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e1c15bec0 0x7f0e1c14c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 --2- 172.21.15.76:0/2435307679 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0e1c0af2a0 0x7f0e1c0b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 >> 172.21.15.76:0/2435307679 conn(0x7f0e1c0b1f90 msgr2=0x7f0e1c165de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 shutdown_connections 2024-09-19T04:19:49.138 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.136+0000 7f0e23a5d640 1 -- 172.21.15.76:0/2435307679 wait complete. 2024-09-19T04:19:49.154 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-19T04:19:47.860147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260772,"kb_used_data":11248,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713948,"statfs":{"total":773094113280,"available":772827082752,"internally_reserved":0,"allocated":11517952,"data_stored":5978796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.673437"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143948+0000","last_change":"2024-09-19T04:19:45.179577+0000","last_active":"2024-09-19T04:19:46.143948+0000","last_peered":"2024-09-19T04:19:46.143948+0000","last_clean":"2024-09-19T04:19:46.143948+0000","last_became_active":"2024-09-19T04:19:45.178771+0000","last_became_peered":"2024-09-19T04:19:45.178771+0000","last_unstale":"2024-09-19T04:19:46.143948+0000","last_undegraded":"2024-09-19T04:19:46.143948+0000","last_fullsized":"2024-09-19T04:19:46.143948+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:33.078579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148040+0000","last_change":"2024-09-19T04:19:45.175091+0000","last_active":"2024-09-19T04:19:47.148040+0000","last_peered":"2024-09-19T04:19:47.148040+0000","last_clean":"2024-09-19T04:19:47.148040+0000","last_became_active":"2024-09-19T04:19:45.169137+0000","last_became_peered":"2024-09-19T04:19:45.169137+0000","last_unstale":"2024-09-19T04:19:47.148040+0000","last_undegraded":"2024-09-19T04:19:47.148040+0000","last_fullsized":"2024-09-19T04:19:47.148040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:32.290167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143032+0000","last_change":"2024-09-19T04:19:45.163308+0000","last_active":"2024-09-19T04:19:46.143032+0000","last_peered":"2024-09-19T04:19:46.143032+0000","last_clean":"2024-09-19T04:19:46.143032+0000","last_became_active":"2024-09-19T04:19:45.162577+0000","last_became_peered":"2024-09-19T04:19:45.162577+0000","last_unstale":"2024-09-19T04:19:46.143032+0000","last_undegraded":"2024-09-19T04:19:46.143032+0000","last_fullsized":"2024-09-19T04:19:46.143032+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:37.050316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149320+0000","last_change":"2024-09-19T04:19:45.171520+0000","last_active":"2024-09-19T04:19:47.149320+0000","last_peered":"2024-09-19T04:19:47.149320+0000","last_clean":"2024-09-19T04:19:47.149320+0000","last_became_active":"2024-09-19T04:19:45.169744+0000","last_became_peered":"2024-09-19T04:19:45.169744+0000","last_unstale":"2024-09-19T04:19:47.149320+0000","last_undegraded":"2024-09-19T04:19:47.149320+0000","last_fullsized":"2024-09-19T04:19:47.149320+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:45.851905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148901+0000","last_change":"2024-09-19T04:19:45.185748+0000","last_active":"2024-09-19T04:19:47.148901+0000","last_peered":"2024-09-19T04:19:47.148901+0000","last_clean":"2024-09-19T04:19:47.148901+0000","last_became_active":"2024-09-19T04:19:45.184526+0000","last_became_peered":"2024-09-19T04:19:45.184526+0000","last_unstale":"2024-09-19T04:19:47.148901+0000","last_undegraded":"2024-09-19T04:19:47.148901+0000","last_fullsized":"2024-09-19T04:19:47.148901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:12.253975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147099+0000","last_change":"2024-09-19T04:19:45.192067+0000","last_active":"2024-09-19T04:19:47.147099+0000","last_peered":"2024-09-19T04:19:47.147099+0000","last_clean":"2024-09-19T04:19:47.147099+0000","last_became_active":"2024-09-19T04:19:45.191498+0000","last_became_peered":"2024-09-19T04:19:45.191498+0000","last_unstale":"2024-09-19T04:19:47.147099+0000","last_undegraded":"2024-09-19T04:19:47.147099+0000","last_fullsized":"2024-09-19T04:19:47.147099+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147543+0000","last_change":"2024-09-19T04:19:45.194119+0000","last_active":"2024-09-19T04:19:47.147543+0000","last_peered":"2024-09-19T04:19:47.147543+0000","last_clean":"2024-09-19T04:19:47.147543+0000","last_became_active":"2024-09-19T04:19:45.193726+0000","last_became_peered":"2024-09-19T04:19:45.193726+0000","last_unstale":"2024-09-19T04:19:47.147543+0000","last_undegraded":"2024-09-19T04:19:47.147543+0000","last_fullsized":"2024-09-19T04:19:47.147543+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:51.049472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.582830+0000","last_change":"2024-09-19T04:19:45.200906+0000","last_active":"2024-09-19T04:19:46.582830+0000","last_peered":"2024-09-19T04:19:46.582830+0000","last_clean":"2024-09-19T04:19:46.582830+0000","last_became_active":"2024-09-19T04:19:45.200591+0000","last_became_peered":"2024-09-19T04:19:45.200591+0000","last_unstale":"2024-09-19T04:19:46.582830+0000","last_undegraded":"2024-09-19T04:19:46.582830+0000","last_fullsized":"2024-09-19T04:19:46.582830+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:23.168352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147799+0000","last_change":"2024-09-19T04:19:45.179957+0000","last_active":"2024-09-19T04:19:47.147799+0000","last_peered":"2024-09-19T04:19:47.147799+0000","last_clean":"2024-09-19T04:19:47.147799+0000","last_became_active":"2024-09-19T04:19:45.179221+0000","last_became_peered":"2024-09-19T04:19:45.179221+0000","last_unstale":"2024-09-19T04:19:47.147799+0000","last_undegraded":"2024-09-19T04:19:47.147799+0000","last_fullsized":"2024-09-19T04:19:47.147799+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:12:47.700486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148480+0000","last_change":"2024-09-19T04:19:45.189940+0000","last_active":"2024-09-19T04:19:47.148480+0000","last_peered":"2024-09-19T04:19:47.148480+0000","last_clean":"2024-09-19T04:19:47.148480+0000","last_became_active":"2024-09-19T04:19:45.184874+0000","last_became_peered":"2024-09-19T04:19:45.184874+0000","last_unstale":"2024-09-19T04:19:47.148480+0000","last_undegraded":"2024-09-19T04:19:47.148480+0000","last_fullsized":"2024-09-19T04:19:47.148480+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:53.207406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145358+0000","last_change":"2024-09-19T04:19:45.163864+0000","last_active":"2024-09-19T04:19:46.145358+0000","last_peered":"2024-09-19T04:19:46.145358+0000","last_clean":"2024-09-19T04:19:46.145358+0000","last_became_active":"2024-09-19T04:19:45.163057+0000","last_became_peered":"2024-09-19T04:19:45.163057+0000","last_unstale":"2024-09-19T04:19:46.145358+0000","last_undegraded":"2024-09-19T04:19:46.145358+0000","last_fullsized":"2024-09-19T04:19:46.145358+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:13.454807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145680+0000","last_change":"2024-09-19T04:19:45.178304+0000","last_active":"2024-09-19T04:19:46.145680+0000","last_peered":"2024-09-19T04:19:46.145680+0000","last_clean":"2024-09-19T04:19:46.145680+0000","last_became_active":"2024-09-19T04:19:45.177598+0000","last_became_peered":"2024-09-19T04:19:45.177598+0000","last_unstale":"2024-09-19T04:19:46.145680+0000","last_undegraded":"2024-09-19T04:19:46.145680+0000","last_fullsized":"2024-09-19T04:19:46.145680+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:58.578284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148944+0000","last_change":"2024-09-19T04:19:45.169051+0000","last_active":"2024-09-19T04:19:47.148944+0000","last_peered":"2024-09-19T04:19:47.148944+0000","last_clean":"2024-09-19T04:19:47.148944+0000","last_became_active":"2024-09-19T04:19:45.168267+0000","last_became_peered":"2024-09-19T04:19:45.168267+0000","last_unstale":"2024-09-19T04:19:47.148944+0000","last_undegraded":"2024-09-19T04:19:47.148944+0000","last_fullsized":"2024-09-19T04:19:47.148944+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:48.296104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309011+0000","last_change":"2024-09-19T04:19:45.178221+0000","last_active":"2024-09-19T04:19:46.309011+0000","last_peered":"2024-09-19T04:19:46.309011+0000","last_clean":"2024-09-19T04:19:46.309011+0000","last_became_active":"2024-09-19T04:19:45.177453+0000","last_became_peered":"2024-09-19T04:19:45.177453+0000","last_unstale":"2024-09-19T04:19:46.309011+0000","last_undegraded":"2024-09-19T04:19:46.309011+0000","last_fullsized":"2024-09-19T04:19:46.309011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:57.866440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309007+0000","last_change":"2024-09-19T04:19:45.185362+0000","last_active":"2024-09-19T04:19:46.309007+0000","last_peered":"2024-09-19T04:19:46.309007+0000","last_clean":"2024-09-19T04:19:46.309007+0000","last_became_active":"2024-09-19T04:19:45.184564+0000","last_became_peered":"2024-09-19T04:19:45.184564+0000","last_unstale":"2024-09-19T04:19:46.309007+0000","last_undegraded":"2024-09-19T04:19:46.309007+0000","last_fullsized":"2024-09-19T04:19:46.309007+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:04.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309009+0000","last_change":"2024-09-19T04:19:45.170439+0000","last_active":"2024-09-19T04:19:46.309009+0000","last_peered":"2024-09-19T04:19:46.309009+0000","last_clean":"2024-09-19T04:19:46.309009+0000","last_became_active":"2024-09-19T04:19:45.169639+0000","last_became_peered":"2024-09-19T04:19:45.169639+0000","last_unstale":"2024-09-19T04:19:46.309009+0000","last_undegraded":"2024-09-19T04:19:46.309009+0000","last_fullsized":"2024-09-19T04:19:46.309009+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:12.953139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144130+0000","last_change":"2024-09-19T04:19:45.168627+0000","last_active":"2024-09-19T04:19:46.144130+0000","last_peered":"2024-09-19T04:19:46.144130+0000","last_clean":"2024-09-19T04:19:46.144130+0000","last_became_active":"2024-09-19T04:19:45.167855+0000","last_became_peered":"2024-09-19T04:19:45.167855+0000","last_unstale":"2024-09-19T04:19:46.144130+0000","last_undegraded":"2024-09-19T04:19:46.144130+0000","last_fullsized":"2024-09-19T04:19:46.144130+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:05:02.245083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148208+0000","last_change":"2024-09-19T04:19:45.192276+0000","last_active":"2024-09-19T04:19:47.148208+0000","last_peered":"2024-09-19T04:19:47.148208+0000","last_clean":"2024-09-19T04:19:47.148208+0000","last_became_active":"2024-09-19T04:19:45.191610+0000","last_became_peered":"2024-09-19T04:19:45.191610+0000","last_unstale":"2024-09-19T04:19:47.148208+0000","last_undegraded":"2024-09-19T04:19:47.148208+0000","last_fullsized":"2024-09-19T04:19:47.148208+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:58.727906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143180+0000","last_change":"2024-09-19T04:19:45.179524+0000","last_active":"2024-09-19T04:19:46.143180+0000","last_peered":"2024-09-19T04:19:46.143180+0000","last_clean":"2024-09-19T04:19:46.143180+0000","last_became_active":"2024-09-19T04:19:45.178987+0000","last_became_peered":"2024-09-19T04:19:45.178987+0000","last_unstale":"2024-09-19T04:19:46.143180+0000","last_undegraded":"2024-09-19T04:19:46.143180+0000","last_fullsized":"2024-09-19T04:19:46.143180+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:44.531383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145750+0000","last_change":"2024-09-19T04:19:45.168512+0000","last_active":"2024-09-19T04:19:46.145750+0000","last_peered":"2024-09-19T04:19:46.145750+0000","last_clean":"2024-09-19T04:19:46.145750+0000","last_became_active":"2024-09-19T04:19:45.167737+0000","last_became_peered":"2024-09-19T04:19:45.167737+0000","last_unstale":"2024-09-19T04:19:46.145750+0000","last_undegraded":"2024-09-19T04:19:46.145750+0000","last_fullsized":"2024-09-19T04:19:46.145750+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:51.049472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583701+0000","last_change":"2024-09-19T04:19:45.176746+0000","last_active":"2024-09-19T04:19:46.583701+0000","last_peered":"2024-09-19T04:19:46.583701+0000","last_clean":"2024-09-19T04:19:46.583701+0000","last_became_active":"2024-09-19T04:19:45.175869+0000","last_became_peered":"2024-09-19T04:19:45.175869+0000","last_unstale":"2024-09-19T04:19:46.583701+0000","last_undegraded":"2024-09-19T04:19:46.583701+0000","last_fullsized":"2024-09-19T04:19:46.583701+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:11.483700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148761+0000","last_change":"2024-09-19T04:19:45.190525+0000","last_active":"2024-09-19T04:19:47.148761+0000","last_peered":"2024-09-19T04:19:47.148761+0000","last_clean":"2024-09-19T04:19:47.148761+0000","last_became_active":"2024-09-19T04:19:45.184256+0000","last_became_peered":"2024-09-19T04:19:45.184256+0000","last_unstale":"2024-09-19T04:19:47.148761+0000","last_undegraded":"2024-09-19T04:19:47.148761+0000","last_fullsized":"2024-09-19T04:19:47.148761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:18.279383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144384+0000","last_change":"2024-09-19T04:19:45.167039+0000","last_active":"2024-09-19T04:19:46.144384+0000","last_peered":"2024-09-19T04:19:46.144384+0000","last_clean":"2024-09-19T04:19:46.144384+0000","last_became_active":"2024-09-19T04:19:45.166242+0000","last_became_peered":"2024-09-19T04:19:45.166242+0000","last_unstale":"2024-09-19T04:19:46.144384+0000","last_undegraded":"2024-09-19T04:19:46.144384+0000","last_fullsized":"2024-09-19T04:19:46.144384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:04.971293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147689+0000","last_change":"2024-09-19T04:19:45.184944+0000","last_active":"2024-09-19T04:19:47.147689+0000","last_peered":"2024-09-19T04:19:47.147689+0000","last_clean":"2024-09-19T04:19:47.147689+0000","last_became_active":"2024-09-19T04:19:45.183888+0000","last_became_peered":"2024-09-19T04:19:45.183888+0000","last_unstale":"2024-09-19T04:19:47.147689+0000","last_undegraded":"2024-09-19T04:19:47.147689+0000","last_fullsized":"2024-09-19T04:19:47.147689+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:34.494505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144747+0000","last_change":"2024-09-19T04:19:45.169001+0000","last_active":"2024-09-19T04:19:46.144747+0000","last_peered":"2024-09-19T04:19:46.144747+0000","last_clean":"2024-09-19T04:19:46.144747+0000","last_became_active":"2024-09-19T04:19:45.168286+0000","last_became_peered":"2024-09-19T04:19:45.168286+0000","last_unstale":"2024-09-19T04:19:46.144747+0000","last_undegraded":"2024-09-19T04:19:46.144747+0000","last_fullsized":"2024-09-19T04:19:46.144747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145050+0000","last_change":"2024-09-19T04:19:45.184020+0000","last_active":"2024-09-19T04:19:46.145050+0000","last_peered":"2024-09-19T04:19:46.145050+0000","last_clean":"2024-09-19T04:19:46.145050+0000","last_became_active":"2024-09-19T04:19:45.183293+0000","last_became_peered":"2024-09-19T04:19:45.183293+0000","last_unstale":"2024-09-19T04:19:46.145050+0000","last_undegraded":"2024-09-19T04:19:46.145050+0000","last_fullsized":"2024-09-19T04:19:46.145050+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145517+0000","last_change":"2024-09-19T04:19:45.173612+0000","last_active":"2024-09-19T04:19:46.145517+0000","last_peered":"2024-09-19T04:19:46.145517+0000","last_clean":"2024-09-19T04:19:46.145517+0000","last_became_active":"2024-09-19T04:19:45.172851+0000","last_became_peered":"2024-09-19T04:19:45.172851+0000","last_unstale":"2024-09-19T04:19:46.145517+0000","last_undegraded":"2024-09-19T04:19:46.145517+0000","last_fullsized":"2024-09-19T04:19:46.145517+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:24.112252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147845+0000","last_change":"2024-09-19T04:19:45.166979+0000","last_active":"2024-09-19T04:19:47.147845+0000","last_peered":"2024-09-19T04:19:47.147845+0000","last_clean":"2024-09-19T04:19:47.147845+0000","last_became_active":"2024-09-19T04:19:45.166219+0000","last_became_peered":"2024-09-19T04:19:45.166219+0000","last_unstale":"2024-09-19T04:19:47.147845+0000","last_undegraded":"2024-09-19T04:19:47.147845+0000","last_fullsized":"2024-09-19T04:19:47.147845+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:13.454807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583946+0000","last_change":"2024-09-19T04:19:45.172717+0000","last_active":"2024-09-19T04:19:46.583946+0000","last_peered":"2024-09-19T04:19:46.583946+0000","last_clean":"2024-09-19T04:19:46.583946+0000","last_became_active":"2024-09-19T04:19:45.171590+0000","last_became_peered":"2024-09-19T04:19:45.171590+0000","last_unstale":"2024-09-19T04:19:46.583946+0000","last_undegraded":"2024-09-19T04:19:46.583946+0000","last_fullsized":"2024-09-19T04:19:46.583946+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:20.010469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143759+0000","last_change":"2024-09-19T04:19:45.175929+0000","last_active":"2024-09-19T04:19:46.143759+0000","last_peered":"2024-09-19T04:19:46.143759+0000","last_clean":"2024-09-19T04:19:46.143759+0000","last_became_active":"2024-09-19T04:19:45.175243+0000","last_became_peered":"2024-09-19T04:19:45.175243+0000","last_unstale":"2024-09-19T04:19:46.143759+0000","last_undegraded":"2024-09-19T04:19:46.143759+0000","last_fullsized":"2024-09-19T04:19:46.143759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:18.279383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.146996+0000","last_change":"2024-09-19T04:19:45.169266+0000","last_active":"2024-09-19T04:19:47.146996+0000","last_peered":"2024-09-19T04:19:47.146996+0000","last_clean":"2024-09-19T04:19:47.146996+0000","last_became_active":"2024-09-19T04:19:45.168439+0000","last_became_peered":"2024-09-19T04:19:45.168439+0000","last_unstale":"2024-09-19T04:19:47.146996+0000","last_undegraded":"2024-09-19T04:19:47.146996+0000","last_fullsized":"2024-09-19T04:19:47.146996+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583072+0000","last_change":"2024-09-19T04:19:45.164808+0000","last_active":"2024-09-19T04:19:46.583072+0000","last_peered":"2024-09-19T04:19:46.583072+0000","last_clean":"2024-09-19T04:19:46.583072+0000","last_became_active":"2024-09-19T04:19:45.163828+0000","last_became_peered":"2024-09-19T04:19:45.163828+0000","last_unstale":"2024-09-19T04:19:46.583072+0000","last_undegraded":"2024-09-19T04:19:46.583072+0000","last_fullsized":"2024-09-19T04:19:46.583072+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:56.653211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143466+0000","last_change":"2024-09-19T04:19:45.180066+0000","last_active":"2024-09-19T04:19:46.143466+0000","last_peered":"2024-09-19T04:19:46.143466+0000","last_clean":"2024-09-19T04:19:46.143466+0000","last_became_active":"2024-09-19T04:19:45.179314+0000","last_became_peered":"2024-09-19T04:19:45.179314+0000","last_unstale":"2024-09-19T04:19:46.143466+0000","last_undegraded":"2024-09-19T04:19:46.143466+0000","last_fullsized":"2024-09-19T04:19:46.143466+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.657775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149783+0000","last_change":"2024-09-19T04:19:44.231697+0000","last_active":"2024-09-19T04:19:47.149783+0000","last_peered":"2024-09-19T04:19:47.149783+0000","last_clean":"2024-09-19T04:19:47.149783+0000","last_became_active":"2024-09-19T04:19:44.231076+0000","last_became_peered":"2024-09-19T04:19:44.231076+0000","last_unstale":"2024-09-19T04:19:47.149783+0000","last_undegraded":"2024-09-19T04:19:47.149783+0000","last_fullsized":"2024-09-19T04:19:47.149783+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:12:46.696922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583010+0000","last_change":"2024-09-19T04:19:45.193367+0000","last_active":"2024-09-19T04:19:46.583010+0000","last_peered":"2024-09-19T04:19:46.583010+0000","last_clean":"2024-09-19T04:19:46.583010+0000","last_became_active":"2024-09-19T04:19:45.192916+0000","last_became_peered":"2024-09-19T04:19:45.192916+0000","last_unstale":"2024-09-19T04:19:46.583010+0000","last_undegraded":"2024-09-19T04:19:46.583010+0000","last_fullsized":"2024-09-19T04:19:46.583010+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:57.350022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583650+0000","last_change":"2024-09-19T04:19:44.213081+0000","last_active":"2024-09-19T04:19:46.583650+0000","last_peered":"2024-09-19T04:19:46.583650+0000","last_clean":"2024-09-19T04:19:46.583650+0000","last_became_active":"2024-09-19T04:19:44.212554+0000","last_became_peered":"2024-09-19T04:19:44.212554+0000","last_unstale":"2024-09-19T04:19:46.583650+0000","last_undegraded":"2024-09-19T04:19:46.583650+0000","last_fullsized":"2024-09-19T04:19:46.583650+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:16.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144160+0000","last_change":"2024-09-19T04:19:45.198183+0000","last_active":"2024-09-19T04:19:46.144160+0000","last_peered":"2024-09-19T04:19:46.144160+0000","last_clean":"2024-09-19T04:19:46.144160+0000","last_became_active":"2024-09-19T04:19:45.197436+0000","last_became_peered":"2024-09-19T04:19:45.197436+0000","last_unstale":"2024-09-19T04:19:46.144160+0000","last_undegraded":"2024-09-19T04:19:46.144160+0000","last_fullsized":"2024-09-19T04:19:46.144160+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:12.253975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144840+0000","last_change":"2024-09-19T04:19:44.232018+0000","last_active":"2024-09-19T04:19:46.144840+0000","last_peered":"2024-09-19T04:19:46.144840+0000","last_clean":"2024-09-19T04:19:46.144840+0000","last_became_active":"2024-09-19T04:19:44.226498+0000","last_became_peered":"2024-09-19T04:19:44.226498+0000","last_unstale":"2024-09-19T04:19:46.144840+0000","last_undegraded":"2024-09-19T04:19:46.144840+0000","last_fullsized":"2024-09-19T04:19:46.144840+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:30.396563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145023+0000","last_change":"2024-09-19T04:19:45.163758+0000","last_active":"2024-09-19T04:19:46.145023+0000","last_peered":"2024-09-19T04:19:46.145023+0000","last_clean":"2024-09-19T04:19:46.145023+0000","last_became_active":"2024-09-19T04:19:45.162990+0000","last_became_peered":"2024-09-19T04:19:45.162990+0000","last_unstale":"2024-09-19T04:19:46.145023+0000","last_undegraded":"2024-09-19T04:19:46.145023+0000","last_fullsized":"2024-09-19T04:19:46.145023+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:51.049472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148427+0000","last_change":"2024-09-19T04:19:44.226570+0000","last_active":"2024-09-19T04:19:47.148427+0000","last_peered":"2024-09-19T04:19:47.148427+0000","last_clean":"2024-09-19T04:19:47.148427+0000","last_became_active":"2024-09-19T04:19:44.226215+0000","last_became_peered":"2024-09-19T04:19:44.226215+0000","last_unstale":"2024-09-19T04:19:47.148427+0000","last_undegraded":"2024-09-19T04:19:47.148427+0000","last_fullsized":"2024-09-19T04:19:47.148427+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.434191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149726+0000","last_change":"2024-09-19T04:19:45.199474+0000","last_active":"2024-09-19T04:19:47.149726+0000","last_peered":"2024-09-19T04:19:47.149726+0000","last_clean":"2024-09-19T04:19:47.149726+0000","last_became_active":"2024-09-19T04:19:45.198928+0000","last_became_peered":"2024-09-19T04:19:45.198928+0000","last_unstale":"2024-09-19T04:19:47.149726+0000","last_undegraded":"2024-09-19T04:19:47.149726+0000","last_fullsized":"2024-09-19T04:19:47.149726+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:55.032656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309273+0000","last_change":"2024-09-19T04:19:44.222531+0000","last_active":"2024-09-19T04:19:46.309273+0000","last_peered":"2024-09-19T04:19:46.309273+0000","last_clean":"2024-09-19T04:19:46.309273+0000","last_became_active":"2024-09-19T04:19:44.218242+0000","last_became_peered":"2024-09-19T04:19:44.218242+0000","last_unstale":"2024-09-19T04:19:46.309273+0000","last_undegraded":"2024-09-19T04:19:46.309273+0000","last_fullsized":"2024-09-19T04:19:46.309273+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:04.937333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143477+0000","last_change":"2024-09-19T04:19:45.180374+0000","last_active":"2024-09-19T04:19:46.143477+0000","last_peered":"2024-09-19T04:19:46.143477+0000","last_clean":"2024-09-19T04:19:46.143477+0000","last_became_active":"2024-09-19T04:19:45.179795+0000","last_became_peered":"2024-09-19T04:19:45.179795+0000","last_unstale":"2024-09-19T04:19:46.143477+0000","last_undegraded":"2024-09-19T04:19:46.143477+0000","last_fullsized":"2024-09-19T04:19:46.143477+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:30.929127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144496+0000","last_change":"2024-09-19T04:19:44.241686+0000","last_active":"2024-09-19T04:19:46.144496+0000","last_peered":"2024-09-19T04:19:46.144496+0000","last_clean":"2024-09-19T04:19:46.144496+0000","last_became_active":"2024-09-19T04:19:44.241051+0000","last_became_peered":"2024-09-19T04:19:44.241051+0000","last_unstale":"2024-09-19T04:19:46.144496+0000","last_undegraded":"2024-09-19T04:19:46.144496+0000","last_fullsized":"2024-09-19T04:19:46.144496+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:48.538899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148294+0000","last_change":"2024-09-19T04:19:45.175667+0000","last_active":"2024-09-19T04:19:47.148294+0000","last_peered":"2024-09-19T04:19:47.148294+0000","last_clean":"2024-09-19T04:19:47.148294+0000","last_became_active":"2024-09-19T04:19:45.168151+0000","last_became_peered":"2024-09-19T04:19:45.168151+0000","last_unstale":"2024-09-19T04:19:47.148294+0000","last_undegraded":"2024-09-19T04:19:47.148294+0000","last_fullsized":"2024-09-19T04:19:47.148294+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:56.170958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143136+0000","last_change":"2024-09-19T04:19:44.225516+0000","last_active":"2024-09-19T04:19:46.143136+0000","last_peered":"2024-09-19T04:19:46.143136+0000","last_clean":"2024-09-19T04:19:46.143136+0000","last_became_active":"2024-09-19T04:19:44.224853+0000","last_became_peered":"2024-09-19T04:19:44.224853+0000","last_unstale":"2024-09-19T04:19:46.143136+0000","last_undegraded":"2024-09-19T04:19:46.143136+0000","last_fullsized":"2024-09-19T04:19:46.143136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:44.458855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149613+0000","last_change":"2024-09-19T04:19:45.178996+0000","last_active":"2024-09-19T04:19:47.149613+0000","last_peered":"2024-09-19T04:19:47.149613+0000","last_clean":"2024-09-19T04:19:47.149613+0000","last_became_active":"2024-09-19T04:19:45.178210+0000","last_became_peered":"2024-09-19T04:19:45.178210+0000","last_unstale":"2024-09-19T04:19:47.149613+0000","last_undegraded":"2024-09-19T04:19:47.149613+0000","last_fullsized":"2024-09-19T04:19:47.149613+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:26:52.242923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148880+0000","last_change":"2024-09-19T04:19:44.212254+0000","last_active":"2024-09-19T04:19:47.148880+0000","last_peered":"2024-09-19T04:19:47.148880+0000","last_clean":"2024-09-19T04:19:47.148880+0000","last_became_active":"2024-09-19T04:19:44.211798+0000","last_became_peered":"2024-09-19T04:19:44.211798+0000","last_unstale":"2024-09-19T04:19:47.148880+0000","last_undegraded":"2024-09-19T04:19:47.148880+0000","last_fullsized":"2024-09-19T04:19:47.148880+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:57.574720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143084+0000","last_change":"2024-09-19T04:19:45.170015+0000","last_active":"2024-09-19T04:19:46.143084+0000","last_peered":"2024-09-19T04:19:46.143084+0000","last_clean":"2024-09-19T04:19:46.143084+0000","last_became_active":"2024-09-19T04:19:45.160059+0000","last_became_peered":"2024-09-19T04:19:45.160059+0000","last_unstale":"2024-09-19T04:19:46.143084+0000","last_undegraded":"2024-09-19T04:19:46.143084+0000","last_fullsized":"2024-09-19T04:19:46.143084+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:38.829936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145125+0000","last_change":"2024-09-19T04:19:44.232439+0000","last_active":"2024-09-19T04:19:46.145125+0000","last_peered":"2024-09-19T04:19:46.145125+0000","last_clean":"2024-09-19T04:19:46.145125+0000","last_became_active":"2024-09-19T04:19:44.231570+0000","last_became_peered":"2024-09-19T04:19:44.231570+0000","last_unstale":"2024-09-19T04:19:46.145125+0000","last_undegraded":"2024-09-19T04:19:46.145125+0000","last_fullsized":"2024-09-19T04:19:46.145125+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:01:07.475209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148081+0000","last_change":"2024-09-19T04:19:45.186053+0000","last_active":"2024-09-19T04:19:47.148081+0000","last_peered":"2024-09-19T04:19:47.148081+0000","last_clean":"2024-09-19T04:19:47.148081+0000","last_became_active":"2024-09-19T04:19:45.183824+0000","last_became_peered":"2024-09-19T04:19:45.183824+0000","last_unstale":"2024-09-19T04:19:47.148081+0000","last_undegraded":"2024-09-19T04:19:47.148081+0000","last_fullsized":"2024-09-19T04:19:47.148081+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:01.109409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143658+0000","last_change":"2024-09-19T04:19:44.233959+0000","last_active":"2024-09-19T04:19:46.143658+0000","last_peered":"2024-09-19T04:19:46.143658+0000","last_clean":"2024-09-19T04:19:46.143658+0000","last_became_active":"2024-09-19T04:19:44.233633+0000","last_became_peered":"2024-09-19T04:19:44.233633+0000","last_unstale":"2024-09-19T04:19:46.143658+0000","last_undegraded":"2024-09-19T04:19:46.143658+0000","last_fullsized":"2024-09-19T04:19:46.143658+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:16.273621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583501+0000","last_change":"2024-09-19T04:19:45.161029+0000","last_active":"2024-09-19T04:19:46.583501+0000","last_peered":"2024-09-19T04:19:46.583501+0000","last_clean":"2024-09-19T04:19:46.583501+0000","last_became_active":"2024-09-19T04:19:45.160312+0000","last_became_peered":"2024-09-19T04:19:45.160312+0000","last_unstale":"2024-09-19T04:19:46.583501+0000","last_undegraded":"2024-09-19T04:19:46.583501+0000","last_fullsized":"2024-09-19T04:19:46.583501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:14.593374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309376+0000","last_change":"2024-09-19T04:19:44.217666+0000","last_active":"2024-09-19T04:19:46.309376+0000","last_peered":"2024-09-19T04:19:46.309376+0000","last_clean":"2024-09-19T04:19:46.309376+0000","last_became_active":"2024-09-19T04:19:44.216889+0000","last_became_peered":"2024-09-19T04:19:44.216889+0000","last_unstale":"2024-09-19T04:19:46.309376+0000","last_undegraded":"2024-09-19T04:19:46.309376+0000","last_fullsized":"2024-09-19T04:19:46.309376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:17.543539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149633+0000","last_change":"2024-09-19T04:19:45.190070+0000","last_active":"2024-09-19T04:19:47.149633+0000","last_peered":"2024-09-19T04:19:47.149633+0000","last_clean":"2024-09-19T04:19:47.149633+0000","last_became_active":"2024-09-19T04:19:45.185072+0000","last_became_peered":"2024-09-19T04:19:45.185072+0000","last_unstale":"2024-09-19T04:19:47.149633+0000","last_undegraded":"2024-09-19T04:19:47.149633+0000","last_fullsized":"2024-09-19T04:19:47.149633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147537+0000","last_change":"2024-09-19T04:19:44.230464+0000","last_active":"2024-09-19T04:19:47.147537+0000","last_peered":"2024-09-19T04:19:47.147537+0000","last_clean":"2024-09-19T04:19:47.147537+0000","last_became_active":"2024-09-19T04:19:44.229641+0000","last_became_peered":"2024-09-19T04:19:44.229641+0000","last_unstale":"2024-09-19T04:19:47.147537+0000","last_undegraded":"2024-09-19T04:19:47.147537+0000","last_fullsized":"2024-09-19T04:19:47.147537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:47.639016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144520+0000","last_change":"2024-09-19T04:19:45.171897+0000","last_active":"2024-09-19T04:19:46.144520+0000","last_peered":"2024-09-19T04:19:46.144520+0000","last_clean":"2024-09-19T04:19:46.144520+0000","last_became_active":"2024-09-19T04:19:45.171180+0000","last_became_peered":"2024-09-19T04:19:45.171180+0000","last_unstale":"2024-09-19T04:19:46.144520+0000","last_undegraded":"2024-09-19T04:19:46.144520+0000","last_fullsized":"2024-09-19T04:19:46.144520+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:58.578284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149925+0000","last_change":"2024-09-19T04:19:44.215853+0000","last_active":"2024-09-19T04:19:47.149925+0000","last_peered":"2024-09-19T04:19:47.149925+0000","last_clean":"2024-09-19T04:19:47.149925+0000","last_became_active":"2024-09-19T04:19:44.215492+0000","last_became_peered":"2024-09-19T04:19:44.215492+0000","last_unstale":"2024-09-19T04:19:47.149925+0000","last_undegraded":"2024-09-19T04:19:47.149925+0000","last_fullsized":"2024-09-19T04:19:47.149925+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:41.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147891+0000","last_change":"2024-09-19T04:19:45.168677+0000","last_active":"2024-09-19T04:19:47.147891+0000","last_peered":"2024-09-19T04:19:47.147891+0000","last_clean":"2024-09-19T04:19:47.147891+0000","last_became_active":"2024-09-19T04:19:45.167897+0000","last_became_peered":"2024-09-19T04:19:45.167897+0000","last_unstale":"2024-09-19T04:19:47.147891+0000","last_undegraded":"2024-09-19T04:19:47.147891+0000","last_fullsized":"2024-09-19T04:19:47.147891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:24.112252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149527+0000","last_change":"2024-09-19T04:19:44.218497+0000","last_active":"2024-09-19T04:19:47.149527+0000","last_peered":"2024-09-19T04:19:47.149527+0000","last_clean":"2024-09-19T04:19:47.149527+0000","last_became_active":"2024-09-19T04:19:44.218114+0000","last_became_peered":"2024-09-19T04:19:44.218114+0000","last_unstale":"2024-09-19T04:19:47.149527+0000","last_undegraded":"2024-09-19T04:19:47.149527+0000","last_fullsized":"2024-09-19T04:19:47.149527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:45.932308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.584147+0000","last_change":"2024-09-19T04:19:45.167687+0000","last_active":"2024-09-19T04:19:46.584147+0000","last_peered":"2024-09-19T04:19:46.584147+0000","last_clean":"2024-09-19T04:19:46.584147+0000","last_became_active":"2024-09-19T04:19:45.166759+0000","last_became_peered":"2024-09-19T04:19:45.166759+0000","last_unstale":"2024-09-19T04:19:46.584147+0000","last_undegraded":"2024-09-19T04:19:46.584147+0000","last_fullsized":"2024-09-19T04:19:46.584147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:37.150211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583541+0000","last_change":"2024-09-19T04:19:44.227994+0000","last_active":"2024-09-19T04:19:46.583541+0000","last_peered":"2024-09-19T04:19:46.583541+0000","last_clean":"2024-09-19T04:19:46.583541+0000","last_became_active":"2024-09-19T04:19:44.227521+0000","last_became_peered":"2024-09-19T04:19:44.227521+0000","last_unstale":"2024-09-19T04:19:46.583541+0000","last_undegraded":"2024-09-19T04:19:46.583541+0000","last_fullsized":"2024-09-19T04:19:46.583541+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:03.300792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144312+0000","last_change":"2024-09-19T04:19:45.169434+0000","last_active":"2024-09-19T04:19:46.144312+0000","last_peered":"2024-09-19T04:19:46.144312+0000","last_clean":"2024-09-19T04:19:46.144312+0000","last_became_active":"2024-09-19T04:19:45.168222+0000","last_became_peered":"2024-09-19T04:19:45.168222+0000","last_unstale":"2024-09-19T04:19:46.144312+0000","last_undegraded":"2024-09-19T04:19:46.144312+0000","last_fullsized":"2024-09-19T04:19:46.144312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:53.342842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":22,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.142788+0000","last_change":"2024-09-19T04:19:44.232483+0000","last_active":"2024-09-19T04:19:46.142788+0000","last_peered":"2024-09-19T04:19:46.142788+0000","last_clean":"2024-09-19T04:19:46.142788+0000","last_became_active":"2024-09-19T04:19:44.226270+0000","last_became_peered":"2024-09-19T04:19:44.226270+0000","last_unstale":"2024-09-19T04:19:46.142788+0000","last_undegraded":"2024-09-19T04:19:46.142788+0000","last_fullsized":"2024-09-19T04:19:46.142788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:41.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583258+0000","last_change":"2024-09-19T04:19:45.172864+0000","last_active":"2024-09-19T04:19:46.583258+0000","last_peered":"2024-09-19T04:19:46.583258+0000","last_clean":"2024-09-19T04:19:46.583258+0000","last_became_active":"2024-09-19T04:19:45.172135+0000","last_became_peered":"2024-09-19T04:19:45.172135+0000","last_unstale":"2024-09-19T04:19:46.583258+0000","last_undegraded":"2024-09-19T04:19:46.583258+0000","last_fullsized":"2024-09-19T04:19:46.583258+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:21.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148091+0000","last_change":"2024-09-19T04:19:44.239785+0000","last_active":"2024-09-19T04:19:47.148091+0000","last_peered":"2024-09-19T04:19:47.148091+0000","last_clean":"2024-09-19T04:19:47.148091+0000","last_became_active":"2024-09-19T04:19:44.239052+0000","last_became_peered":"2024-09-19T04:19:44.239052+0000","last_unstale":"2024-09-19T04:19:47.148091+0000","last_undegraded":"2024-09-19T04:19:47.148091+0000","last_fullsized":"2024-09-19T04:19:47.148091+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.434191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147157+0000","last_change":"2024-09-19T04:19:45.169150+0000","last_active":"2024-09-19T04:19:47.147157+0000","last_peered":"2024-09-19T04:19:47.147157+0000","last_clean":"2024-09-19T04:19:47.147157+0000","last_became_active":"2024-09-19T04:19:45.168374+0000","last_became_peered":"2024-09-19T04:19:45.168374+0000","last_unstale":"2024-09-19T04:19:47.147157+0000","last_undegraded":"2024-09-19T04:19:47.147157+0000","last_fullsized":"2024-09-19T04:19:47.147157+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:58.578284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309563+0000","last_change":"2024-09-19T04:19:44.214954+0000","last_active":"2024-09-19T04:19:46.309563+0000","last_peered":"2024-09-19T04:19:46.309563+0000","last_clean":"2024-09-19T04:19:46.309563+0000","last_became_active":"2024-09-19T04:19:44.214233+0000","last_became_peered":"2024-09-19T04:19:44.214233+0000","last_unstale":"2024-09-19T04:19:46.309563+0000","last_undegraded":"2024-09-19T04:19:46.309563+0000","last_fullsized":"2024-09-19T04:19:46.309563+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:39.430232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149303+0000","last_change":"2024-09-19T04:19:45.190240+0000","last_active":"2024-09-19T04:19:47.149303+0000","last_peered":"2024-09-19T04:19:47.149303+0000","last_clean":"2024-09-19T04:19:47.149303+0000","last_became_active":"2024-09-19T04:19:45.185691+0000","last_became_peered":"2024-09-19T04:19:45.185691+0000","last_unstale":"2024-09-19T04:19:47.149303+0000","last_undegraded":"2024-09-19T04:19:47.149303+0000","last_fullsized":"2024-09-19T04:19:47.149303+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144077+0000","last_change":"2024-09-19T04:19:44.231274+0000","last_active":"2024-09-19T04:19:46.144077+0000","last_peered":"2024-09-19T04:19:46.144077+0000","last_clean":"2024-09-19T04:19:46.144077+0000","last_became_active":"2024-09-19T04:19:44.230495+0000","last_became_peered":"2024-09-19T04:19:44.230495+0000","last_unstale":"2024-09-19T04:19:46.144077+0000","last_undegraded":"2024-09-19T04:19:46.144077+0000","last_fullsized":"2024-09-19T04:19:46.144077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:45.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143637+0000","last_change":"2024-09-19T04:19:45.167264+0000","last_active":"2024-09-19T04:19:46.143637+0000","last_peered":"2024-09-19T04:19:46.143637+0000","last_clean":"2024-09-19T04:19:46.143637+0000","last_became_active":"2024-09-19T04:19:45.166457+0000","last_became_peered":"2024-09-19T04:19:45.166457+0000","last_unstale":"2024-09-19T04:19:46.143637+0000","last_undegraded":"2024-09-19T04:19:46.143637+0000","last_fullsized":"2024-09-19T04:19:46.143637+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:51.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583959+0000","last_change":"2024-09-19T04:19:44.208558+0000","last_active":"2024-09-19T04:19:46.583959+0000","last_peered":"2024-09-19T04:19:46.583959+0000","last_clean":"2024-09-19T04:19:46.583959+0000","last_became_active":"2024-09-19T04:19:44.208143+0000","last_became_peered":"2024-09-19T04:19:44.208143+0000","last_unstale":"2024-09-19T04:19:46.583959+0000","last_undegraded":"2024-09-19T04:19:46.583959+0000","last_fullsized":"2024-09-19T04:19:46.583959+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:24.355737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143332+0000","last_change":"2024-09-19T04:19:45.196062+0000","last_active":"2024-09-19T04:19:46.143332+0000","last_peered":"2024-09-19T04:19:46.143332+0000","last_clean":"2024-09-19T04:19:46.143332+0000","last_became_active":"2024-09-19T04:19:45.195515+0000","last_became_peered":"2024-09-19T04:19:45.195515+0000","last_unstale":"2024-09-19T04:19:46.143332+0000","last_undegraded":"2024-09-19T04:19:46.143332+0000","last_fullsized":"2024-09-19T04:19:46.143332+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:51.346757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.582875+0000","last_change":"2024-09-19T04:19:44.218216+0000","last_active":"2024-09-19T04:19:46.582875+0000","last_peered":"2024-09-19T04:19:46.582875+0000","last_clean":"2024-09-19T04:19:46.582875+0000","last_became_active":"2024-09-19T04:19:44.217207+0000","last_became_peered":"2024-09-19T04:19:44.217207+0000","last_unstale":"2024-09-19T04:19:46.582875+0000","last_undegraded":"2024-09-19T04:19:46.582875+0000","last_fullsized":"2024-09-19T04:19:46.582875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:29.851224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.150111+0000","last_change":"2024-09-19T04:19:45.187264+0000","last_active":"2024-09-19T04:19:47.150111+0000","last_peered":"2024-09-19T04:19:47.150111+0000","last_clean":"2024-09-19T04:19:47.150111+0000","last_became_active":"2024-09-19T04:19:45.186733+0000","last_became_peered":"2024-09-19T04:19:45.186733+0000","last_unstale":"2024-09-19T04:19:47.150111+0000","last_undegraded":"2024-09-19T04:19:47.150111+0000","last_fullsized":"2024-09-19T04:19:47.150111+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:58:12.105319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147553+0000","last_change":"2024-09-19T04:19:44.225860+0000","last_active":"2024-09-19T04:19:47.147553+0000","last_peered":"2024-09-19T04:19:47.147553+0000","last_clean":"2024-09-19T04:19:47.147553+0000","last_became_active":"2024-09-19T04:19:44.225195+0000","last_became_peered":"2024-09-19T04:19:44.225195+0000","last_unstale":"2024-09-19T04:19:47.147553+0000","last_undegraded":"2024-09-19T04:19:47.147553+0000","last_fullsized":"2024-09-19T04:19:47.147553+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:41.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583649+0000","last_change":"2024-09-19T04:19:45.195754+0000","last_active":"2024-09-19T04:19:46.583649+0000","last_peered":"2024-09-19T04:19:46.583649+0000","last_clean":"2024-09-19T04:19:46.583649+0000","last_became_active":"2024-09-19T04:19:45.195351+0000","last_became_peered":"2024-09-19T04:19:45.195351+0000","last_unstale":"2024-09-19T04:19:46.583649+0000","last_undegraded":"2024-09-19T04:19:46.583649+0000","last_fullsized":"2024-09-19T04:19:46.583649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:02:11.906613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583353+0000","last_change":"2024-09-19T04:19:44.212879+0000","last_active":"2024-09-19T04:19:46.583353+0000","last_peered":"2024-09-19T04:19:46.583353+0000","last_clean":"2024-09-19T04:19:46.583353+0000","last_became_active":"2024-09-19T04:19:44.212515+0000","last_became_peered":"2024-09-19T04:19:44.212515+0000","last_unstale":"2024-09-19T04:19:46.583353+0000","last_undegraded":"2024-09-19T04:19:46.583353+0000","last_fullsized":"2024-09-19T04:19:46.583353+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:51.882953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143045+0000","last_change":"2024-09-19T04:19:45.173519+0000","last_active":"2024-09-19T04:19:46.143045+0000","last_peered":"2024-09-19T04:19:46.143045+0000","last_clean":"2024-09-19T04:19:46.143045+0000","last_became_active":"2024-09-19T04:19:45.172808+0000","last_became_peered":"2024-09-19T04:19:45.172808+0000","last_unstale":"2024-09-19T04:19:46.143045+0000","last_undegraded":"2024-09-19T04:19:46.143045+0000","last_fullsized":"2024-09-19T04:19:46.143045+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.683069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148230+0000","last_change":"2024-09-19T04:19:44.214785+0000","last_active":"2024-09-19T04:19:47.148230+0000","last_peered":"2024-09-19T04:19:47.148230+0000","last_clean":"2024-09-19T04:19:47.148230+0000","last_became_active":"2024-09-19T04:19:44.214401+0000","last_became_peered":"2024-09-19T04:19:44.214401+0000","last_unstale":"2024-09-19T04:19:47.148230+0000","last_undegraded":"2024-09-19T04:19:47.148230+0000","last_fullsized":"2024-09-19T04:19:47.148230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:44.458855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.142814+0000","last_change":"2024-09-19T04:19:45.178122+0000","last_active":"2024-09-19T04:19:46.142814+0000","last_peered":"2024-09-19T04:19:46.142814+0000","last_clean":"2024-09-19T04:19:46.142814+0000","last_became_active":"2024-09-19T04:19:45.177383+0000","last_became_peered":"2024-09-19T04:19:45.177383+0000","last_unstale":"2024-09-19T04:19:46.142814+0000","last_undegraded":"2024-09-19T04:19:46.142814+0000","last_fullsized":"2024-09-19T04:19:46.142814+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:53.207406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.207152+0000","last_change":"2024-09-19T04:19:44.223941+0000","last_active":"2024-09-19T04:19:46.207152+0000","last_peered":"2024-09-19T04:19:46.207152+0000","last_clean":"2024-09-19T04:19:46.207152+0000","last_became_active":"2024-09-19T04:19:44.223600+0000","last_became_peered":"2024-09-19T04:19:44.223600+0000","last_unstale":"2024-09-19T04:19:46.207152+0000","last_undegraded":"2024-09-19T04:19:46.207152+0000","last_fullsized":"2024-09-19T04:19:46.207152+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:31.282568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149290+0000","last_change":"2024-09-19T04:19:45.185218+0000","last_active":"2024-09-19T04:19:47.149290+0000","last_peered":"2024-09-19T04:19:47.149290+0000","last_clean":"2024-09-19T04:19:47.149290+0000","last_became_active":"2024-09-19T04:19:45.184686+0000","last_became_peered":"2024-09-19T04:19:45.184686+0000","last_unstale":"2024-09-19T04:19:47.149290+0000","last_undegraded":"2024-09-19T04:19:47.149290+0000","last_fullsized":"2024-09-19T04:19:47.149290+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:14.992786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147530+0000","last_change":"2024-09-19T04:19:44.223965+0000","last_active":"2024-09-19T04:19:47.147530+0000","last_peered":"2024-09-19T04:19:47.147530+0000","last_clean":"2024-09-19T04:19:47.147530+0000","last_became_active":"2024-09-19T04:19:44.223266+0000","last_became_peered":"2024-09-19T04:19:44.223266+0000","last_unstale":"2024-09-19T04:19:47.147530+0000","last_undegraded":"2024-09-19T04:19:47.147530+0000","last_fullsized":"2024-09-19T04:19:47.147530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:44.458855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145044+0000","last_change":"2024-09-19T04:19:45.199988+0000","last_active":"2024-09-19T04:19:46.145044+0000","last_peered":"2024-09-19T04:19:46.145044+0000","last_clean":"2024-09-19T04:19:46.145044+0000","last_became_active":"2024-09-19T04:19:45.199288+0000","last_became_peered":"2024-09-19T04:19:45.199288+0000","last_unstale":"2024-09-19T04:19:46.145044+0000","last_undegraded":"2024-09-19T04:19:46.145044+0000","last_fullsized":"2024-09-19T04:19:46.145044+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:12:47.700486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148715+0000","last_change":"2024-09-19T04:19:44.231683+0000","last_active":"2024-09-19T04:19:47.148715+0000","last_peered":"2024-09-19T04:19:47.148715+0000","last_clean":"2024-09-19T04:19:47.148715+0000","last_became_active":"2024-09-19T04:19:44.231251+0000","last_became_peered":"2024-09-19T04:19:44.231251+0000","last_unstale":"2024-09-19T04:19:47.148715+0000","last_undegraded":"2024-09-19T04:19:47.148715+0000","last_fullsized":"2024-09-19T04:19:47.148715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:47.639016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583263+0000","last_change":"2024-09-19T04:19:45.201980+0000","last_active":"2024-09-19T04:19:46.583263+0000","last_peered":"2024-09-19T04:19:46.583263+0000","last_clean":"2024-09-19T04:19:46.583263+0000","last_became_active":"2024-09-19T04:19:45.201738+0000","last_became_peered":"2024-09-19T04:19:45.201738+0000","last_unstale":"2024-09-19T04:19:46.583263+0000","last_undegraded":"2024-09-19T04:19:46.583263+0000","last_fullsized":"2024-09-19T04:19:46.583263+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:18:04.519511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144118+0000","last_change":"2024-09-19T04:19:44.231967+0000","last_active":"2024-09-19T04:19:46.144118+0000","last_peered":"2024-09-19T04:19:46.144118+0000","last_clean":"2024-09-19T04:19:46.144118+0000","last_became_active":"2024-09-19T04:19:44.231185+0000","last_became_peered":"2024-09-19T04:19:44.231185+0000","last_unstale":"2024-09-19T04:19:46.144118+0000","last_undegraded":"2024-09-19T04:19:46.144118+0000","last_fullsized":"2024-09-19T04:19:46.144118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:47.639016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583820+0000","last_change":"2024-09-19T04:19:45.202821+0000","last_active":"2024-09-19T04:19:46.583820+0000","last_peered":"2024-09-19T04:19:46.583820+0000","last_clean":"2024-09-19T04:19:46.583820+0000","last_became_active":"2024-09-19T04:19:45.202375+0000","last_became_peered":"2024-09-19T04:19:45.202375+0000","last_unstale":"2024-09-19T04:19:46.583820+0000","last_undegraded":"2024-09-19T04:19:46.583820+0000","last_fullsized":"2024-09-19T04:19:46.583820+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:38.286645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583167+0000","last_change":"2024-09-19T04:19:44.227834+0000","last_active":"2024-09-19T04:19:46.583167+0000","last_peered":"2024-09-19T04:19:46.583167+0000","last_clean":"2024-09-19T04:19:46.583167+0000","last_became_active":"2024-09-19T04:19:44.227529+0000","last_became_peered":"2024-09-19T04:19:44.227529+0000","last_unstale":"2024-09-19T04:19:46.583167+0000","last_undegraded":"2024-09-19T04:19:46.583167+0000","last_fullsized":"2024-09-19T04:19:46.583167+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:23.454086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309655+0000","last_change":"2024-09-19T04:19:45.175959+0000","last_active":"2024-09-19T04:19:46.309655+0000","last_peered":"2024-09-19T04:19:46.309655+0000","last_clean":"2024-09-19T04:19:46.309655+0000","last_became_active":"2024-09-19T04:19:45.174859+0000","last_became_peered":"2024-09-19T04:19:45.174859+0000","last_unstale":"2024-09-19T04:19:46.309655+0000","last_undegraded":"2024-09-19T04:19:46.309655+0000","last_fullsized":"2024-09-19T04:19:46.309655+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:41.220850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147797+0000","last_change":"2024-09-19T04:19:44.225348+0000","last_active":"2024-09-19T04:19:47.147797+0000","last_peered":"2024-09-19T04:19:47.147797+0000","last_clean":"2024-09-19T04:19:47.147797+0000","last_became_active":"2024-09-19T04:19:44.224288+0000","last_became_peered":"2024-09-19T04:19:44.224288+0000","last_unstale":"2024-09-19T04:19:47.147797+0000","last_undegraded":"2024-09-19T04:19:47.147797+0000","last_fullsized":"2024-09-19T04:19:47.147797+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:30.396563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144487+0000","last_change":"2024-09-19T04:19:45.163616+0000","last_active":"2024-09-19T04:19:46.144487+0000","last_peered":"2024-09-19T04:19:46.144487+0000","last_clean":"2024-09-19T04:19:46.144487+0000","last_became_active":"2024-09-19T04:19:45.162812+0000","last_became_peered":"2024-09-19T04:19:45.162812+0000","last_unstale":"2024-09-19T04:19:46.144487+0000","last_undegraded":"2024-09-19T04:19:46.144487+0000","last_fullsized":"2024-09-19T04:19:46.144487+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583521+0000","last_change":"2024-09-19T04:19:44.227839+0000","last_active":"2024-09-19T04:19:46.583521+0000","last_peered":"2024-09-19T04:19:46.583521+0000","last_clean":"2024-09-19T04:19:46.583521+0000","last_became_active":"2024-09-19T04:19:44.227406+0000","last_became_peered":"2024-09-19T04:19:44.227406+0000","last_unstale":"2024-09-19T04:19:46.583521+0000","last_undegraded":"2024-09-19T04:19:46.583521+0000","last_fullsized":"2024-09-19T04:19:46.583521+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:42:10.435772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143832+0000","last_change":"2024-09-19T04:19:45.172103+0000","last_active":"2024-09-19T04:19:46.143832+0000","last_peered":"2024-09-19T04:19:46.143832+0000","last_clean":"2024-09-19T04:19:46.143832+0000","last_became_active":"2024-09-19T04:19:45.171125+0000","last_became_peered":"2024-09-19T04:19:45.171125+0000","last_unstale":"2024-09-19T04:19:46.143832+0000","last_undegraded":"2024-09-19T04:19:46.143832+0000","last_fullsized":"2024-09-19T04:19:46.143832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583259+0000","last_change":"2024-09-19T04:19:44.216222+0000","last_active":"2024-09-19T04:19:46.583259+0000","last_peered":"2024-09-19T04:19:46.583259+0000","last_clean":"2024-09-19T04:19:46.583259+0000","last_became_active":"2024-09-19T04:19:44.215778+0000","last_became_peered":"2024-09-19T04:19:44.215778+0000","last_unstale":"2024-09-19T04:19:46.583259+0000","last_undegraded":"2024-09-19T04:19:46.583259+0000","last_fullsized":"2024-09-19T04:19:46.583259+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:30.910768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":323,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.308651+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:19:46.308651+0000","last_peered":"2024-09-19T04:19:46.308651+0000","last_clean":"2024-09-19T04:19:46.308651+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:19:46.308651+0000","last_undegraded":"2024-09-19T04:19:46.308651+0000","last_fullsized":"2024-09-19T04:19:46.308651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542226,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1888256,"data_stored":1191356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542227,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":596352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":600828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542226,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":600920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":601410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":600862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":594170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542227,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":1192898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:19:49.215 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:19:49.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64cb1ad640 1 Processor -- start 2024-09-19T04:19:49.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64cb1ad640 1 -- start start 2024-09-19T04:19:49.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64cb1ad640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c4161a60 0x7f64c40d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64cb1ad640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f64c40d2e90 0x7f64c40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64cb1ad640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64c40d32b0 con 0x7f64c40cf450 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64cb1ad640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f64c40d0300 con 0x7f64c40d2e90 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64cb1ad640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f64c406e6c0 con 0x7f64c4161a60 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64bbfff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f64c40d2e90 0x7f64c40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64c974a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f64c40cf450 0x7f64c40d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54964/0 (socket says 172.21.15.76:54964) 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64c974a640 1 -- 172.21.15.76:0/2674914727 learned_addr learned my addr 172.21.15.76:0/2674914727 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64c8f49640 1 --2- 172.21.15.76:0/2674914727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c4161a60 0x7f64c40d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64bbfff640 1 -- 172.21.15.76:0/2674914727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c4161a60 msgr2=0x7f64c40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64bbfff640 1 --2- 172.21.15.76:0/2674914727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c4161a60 0x7f64c40d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64bbfff640 1 -- 172.21.15.76:0/2674914727 >> v1:172.21.15.76:6789/0 conn(0x7f64c40cf450 legacy=0x7f64c40d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64bbfff640 1 -- 172.21.15.76:0/2674914727 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c40b0f60 con 0x7f64c40d2e90 2024-09-19T04:19:49.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64bb7fe640 1 -- 172.21.15.76:0/2674914727 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 350439850 0 0) 0x7f64c40d32b0 con 0x7f64c40cf450 2024-09-19T04:19:49.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.340+0000 7f64c8f49640 1 --2- 172.21.15.76:0/2674914727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c4161a60 0x7f64c40d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:49.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64bbfff640 1 --2- 172.21.15.76:0/2674914727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c40d2e90 0x7f64c40cef10 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f64b40097c0 tx=0x7f64b401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324b27527ed94c3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:49.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64bb7fe640 1 -- 172.21.15.76:0/2674914727 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b40b1070 con 0x7f64c40d2e90 2024-09-19T04:19:49.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64bb7fe640 1 -- 172.21.15.76:0/2674914727 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64b40a8050 con 0x7f64c40d2e90 2024-09-19T04:19:49.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64bb7fe640 1 -- 172.21.15.76:0/2674914727 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c40d1930 con 0x7f64c40d2e90 2024-09-19T04:19:49.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- 172.21.15.76:0/2674914727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c40d2e90 msgr2=0x7f64c40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/2674914727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c40d2e90 0x7f64c40cef10 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f64b40097c0 tx=0x7f64b401c3e0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- 172.21.15.76:0/2674914727 shutdown_connections 2024-09-19T04:19:49.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/2674914727 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c40d2e90 0x7f64c40cef10 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/2674914727 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c4161a60 0x7f64c40d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- 172.21.15.76:0/2674914727 >> 172.21.15.76:0/2674914727 conn(0x7f64c408bf90 msgr2=0x7f64c409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:49.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- 172.21.15.76:0/2674914727 shutdown_connections 2024-09-19T04:19:49.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- 172.21.15.76:0/2674914727 wait complete. 2024-09-19T04:19:49.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 Processor -- start 2024-09-19T04:19:49.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- start start 2024-09-19T04:19:49.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64c40cf450 0x7f64c40b9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c40d2e90 0x7f64c41501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c4161a60 0x7f64c40b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f64c40cf8c0 con 0x7f64c4161a60 2024-09-19T04:19:49.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f64c406d190 con 0x7f64c40cf450 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f64c40cfa40 con 0x7f64c40d2e90 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c4161a60 0x7f64c40b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c4161a60 0x7f64c40b7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59088/0 (socket says 172.21.15.76:59088) 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 -- 172.21.15.76:0/3972494913 learned_addr learned my addr 172.21.15.76:0/3972494913 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64bbfff640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c40d2e90 0x7f64c41501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 -- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c40d2e90 msgr2=0x7f64c41501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c40d2e90 0x7f64c41501c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 -- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64c40cf450 msgr2=0x7f64c40b9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c8f49640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64c40cf450 0x7f64c40b9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64c40cf450 0x7f64c40b9c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 -- 172.21.15.76:0/3972494913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c41530a0 con 0x7f64c4161a60 2024-09-19T04:19:49.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64c974a640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c4161a60 0x7f64c40b7400 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00c7d0 tx=0x7f64bc00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:49.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64b97fa640 1 -- 172.21.15.76:0/3972494913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc018070 con 0x7f64c4161a60 2024-09-19T04:19:49.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64b97fa640 1 -- 172.21.15.76:0/3972494913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64bc00a640 con 0x7f64c4161a60 2024-09-19T04:19:49.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c4153380 con 0x7f64c4161a60 2024-09-19T04:19:49.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.344+0000 7f64b97fa640 1 -- 172.21.15.76:0/3972494913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00aa70 con 0x7f64c4161a60 2024-09-19T04:19:49.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.348+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c4157540 con 0x7f64c4161a60 2024-09-19T04:19:49.350 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.348+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c40cef10 con 0x7f64c4161a60 2024-09-19T04:19:49.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.348+0000 7f64b97fa640 1 -- 172.21.15.76:0/3972494913 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f64bc01b070 con 0x7f64c4161a60 2024-09-19T04:19:49.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.348+0000 7f64b97fa640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64b00678c0 0x7f64b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.348+0000 7f64c8f49640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64b00678c0 0x7f64b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.348+0000 7f64b97fa640 1 -- 172.21.15.76:0/3972494913 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f64bc085470 con 0x7f64c4161a60 2024-09-19T04:19:49.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.348+0000 7f64c8f49640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64b00678c0 0x7f64b0069d80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006fd0 tx=0x7f64c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:49.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.352+0000 7f64b97fa640 1 -- 172.21.15.76:0/3972494913 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f64bc055e20 con 0x7f64c4161a60 2024-09-19T04:19:49.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.500+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64c40b1740 con 0x7f64b00678c0 2024-09-19T04:19:49.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.508+0000 7f64b97fa640 1 -- 172.21.15.76:0/3972494913 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252486 (secure 0 0 0) 0x7f64c40b1740 con 0x7f64b00678c0 2024-09-19T04:19:49.513 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:49.514 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:19:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64b00678c0 msgr2=0x7f64b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64b00678c0 0x7f64b0069d80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f64c0006fd0 tx=0x7f64c0008040 comp rx=0 tx=0).stop 2024-09-19T04:19:49.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c4161a60 msgr2=0x7f64c40b7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c4161a60 0x7f64c40b7400 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00c7d0 tx=0x7f64bc00cc90 comp rx=0 tx=0).stop 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 shutdown_connections 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f64b00678c0 0x7f64b0069d80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f64c4161a60 0x7f64c40b7400 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f64c40d2e90 0x7f64c41501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 --2- 172.21.15.76:0/3972494913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64c40cf450 0x7f64c40b9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 >> 172.21.15.76:0/3972494913 conn(0x7f64c408bf90 msgr2=0x7f64c40b3300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 shutdown_connections 2024-09-19T04:19:49.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.512+0000 7f64cb1ad640 1 -- 172.21.15.76:0/3972494913 wait complete. 2024-09-19T04:19:49.530 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-19T04:19:47.860147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260772,"kb_used_data":11248,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713948,"statfs":{"total":773094113280,"available":772827082752,"internally_reserved":0,"allocated":11517952,"data_stored":5978796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":46,"apply_latency_ms":46,"commit_latency_ns":46000000,"apply_latency_ns":46000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.673437"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143948+0000","last_change":"2024-09-19T04:19:45.179577+0000","last_active":"2024-09-19T04:19:46.143948+0000","last_peered":"2024-09-19T04:19:46.143948+0000","last_clean":"2024-09-19T04:19:46.143948+0000","last_became_active":"2024-09-19T04:19:45.178771+0000","last_became_peered":"2024-09-19T04:19:45.178771+0000","last_unstale":"2024-09-19T04:19:46.143948+0000","last_undegraded":"2024-09-19T04:19:46.143948+0000","last_fullsized":"2024-09-19T04:19:46.143948+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:58:33.078579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148040+0000","last_change":"2024-09-19T04:19:45.175091+0000","last_active":"2024-09-19T04:19:47.148040+0000","last_peered":"2024-09-19T04:19:47.148040+0000","last_clean":"2024-09-19T04:19:47.148040+0000","last_became_active":"2024-09-19T04:19:45.169137+0000","last_became_peered":"2024-09-19T04:19:45.169137+0000","last_unstale":"2024-09-19T04:19:47.148040+0000","last_undegraded":"2024-09-19T04:19:47.148040+0000","last_fullsized":"2024-09-19T04:19:47.148040+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:00:32.290167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143032+0000","last_change":"2024-09-19T04:19:45.163308+0000","last_active":"2024-09-19T04:19:46.143032+0000","last_peered":"2024-09-19T04:19:46.143032+0000","last_clean":"2024-09-19T04:19:46.143032+0000","last_became_active":"2024-09-19T04:19:45.162577+0000","last_became_peered":"2024-09-19T04:19:45.162577+0000","last_unstale":"2024-09-19T04:19:46.143032+0000","last_undegraded":"2024-09-19T04:19:46.143032+0000","last_fullsized":"2024-09-19T04:19:46.143032+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:45:37.050316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149320+0000","last_change":"2024-09-19T04:19:45.171520+0000","last_active":"2024-09-19T04:19:47.149320+0000","last_peered":"2024-09-19T04:19:47.149320+0000","last_clean":"2024-09-19T04:19:47.149320+0000","last_became_active":"2024-09-19T04:19:45.169744+0000","last_became_peered":"2024-09-19T04:19:45.169744+0000","last_unstale":"2024-09-19T04:19:47.149320+0000","last_undegraded":"2024-09-19T04:19:47.149320+0000","last_fullsized":"2024-09-19T04:19:47.149320+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:41:45.851905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148901+0000","last_change":"2024-09-19T04:19:45.185748+0000","last_active":"2024-09-19T04:19:47.148901+0000","last_peered":"2024-09-19T04:19:47.148901+0000","last_clean":"2024-09-19T04:19:47.148901+0000","last_became_active":"2024-09-19T04:19:45.184526+0000","last_became_peered":"2024-09-19T04:19:45.184526+0000","last_unstale":"2024-09-19T04:19:47.148901+0000","last_undegraded":"2024-09-19T04:19:47.148901+0000","last_fullsized":"2024-09-19T04:19:47.148901+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:12.253975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147099+0000","last_change":"2024-09-19T04:19:45.192067+0000","last_active":"2024-09-19T04:19:47.147099+0000","last_peered":"2024-09-19T04:19:47.147099+0000","last_clean":"2024-09-19T04:19:47.147099+0000","last_became_active":"2024-09-19T04:19:45.191498+0000","last_became_peered":"2024-09-19T04:19:45.191498+0000","last_unstale":"2024-09-19T04:19:47.147099+0000","last_undegraded":"2024-09-19T04:19:47.147099+0000","last_fullsized":"2024-09-19T04:19:47.147099+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147543+0000","last_change":"2024-09-19T04:19:45.194119+0000","last_active":"2024-09-19T04:19:47.147543+0000","last_peered":"2024-09-19T04:19:47.147543+0000","last_clean":"2024-09-19T04:19:47.147543+0000","last_became_active":"2024-09-19T04:19:45.193726+0000","last_became_peered":"2024-09-19T04:19:45.193726+0000","last_unstale":"2024-09-19T04:19:47.147543+0000","last_undegraded":"2024-09-19T04:19:47.147543+0000","last_fullsized":"2024-09-19T04:19:47.147543+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:51.049472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.582830+0000","last_change":"2024-09-19T04:19:45.200906+0000","last_active":"2024-09-19T04:19:46.582830+0000","last_peered":"2024-09-19T04:19:46.582830+0000","last_clean":"2024-09-19T04:19:46.582830+0000","last_became_active":"2024-09-19T04:19:45.200591+0000","last_became_peered":"2024-09-19T04:19:45.200591+0000","last_unstale":"2024-09-19T04:19:46.582830+0000","last_undegraded":"2024-09-19T04:19:46.582830+0000","last_fullsized":"2024-09-19T04:19:46.582830+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:37:23.168352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147799+0000","last_change":"2024-09-19T04:19:45.179957+0000","last_active":"2024-09-19T04:19:47.147799+0000","last_peered":"2024-09-19T04:19:47.147799+0000","last_clean":"2024-09-19T04:19:47.147799+0000","last_became_active":"2024-09-19T04:19:45.179221+0000","last_became_peered":"2024-09-19T04:19:45.179221+0000","last_unstale":"2024-09-19T04:19:47.147799+0000","last_undegraded":"2024-09-19T04:19:47.147799+0000","last_fullsized":"2024-09-19T04:19:47.147799+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:12:47.700486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148480+0000","last_change":"2024-09-19T04:19:45.189940+0000","last_active":"2024-09-19T04:19:47.148480+0000","last_peered":"2024-09-19T04:19:47.148480+0000","last_clean":"2024-09-19T04:19:47.148480+0000","last_became_active":"2024-09-19T04:19:45.184874+0000","last_became_peered":"2024-09-19T04:19:45.184874+0000","last_unstale":"2024-09-19T04:19:47.148480+0000","last_undegraded":"2024-09-19T04:19:47.148480+0000","last_fullsized":"2024-09-19T04:19:47.148480+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:53.207406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145358+0000","last_change":"2024-09-19T04:19:45.163864+0000","last_active":"2024-09-19T04:19:46.145358+0000","last_peered":"2024-09-19T04:19:46.145358+0000","last_clean":"2024-09-19T04:19:46.145358+0000","last_became_active":"2024-09-19T04:19:45.163057+0000","last_became_peered":"2024-09-19T04:19:45.163057+0000","last_unstale":"2024-09-19T04:19:46.145358+0000","last_undegraded":"2024-09-19T04:19:46.145358+0000","last_fullsized":"2024-09-19T04:19:46.145358+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:13.454807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145680+0000","last_change":"2024-09-19T04:19:45.178304+0000","last_active":"2024-09-19T04:19:46.145680+0000","last_peered":"2024-09-19T04:19:46.145680+0000","last_clean":"2024-09-19T04:19:46.145680+0000","last_became_active":"2024-09-19T04:19:45.177598+0000","last_became_peered":"2024-09-19T04:19:45.177598+0000","last_unstale":"2024-09-19T04:19:46.145680+0000","last_undegraded":"2024-09-19T04:19:46.145680+0000","last_fullsized":"2024-09-19T04:19:46.145680+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:58.578284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148944+0000","last_change":"2024-09-19T04:19:45.169051+0000","last_active":"2024-09-19T04:19:47.148944+0000","last_peered":"2024-09-19T04:19:47.148944+0000","last_clean":"2024-09-19T04:19:47.148944+0000","last_became_active":"2024-09-19T04:19:45.168267+0000","last_became_peered":"2024-09-19T04:19:45.168267+0000","last_unstale":"2024-09-19T04:19:47.148944+0000","last_undegraded":"2024-09-19T04:19:47.148944+0000","last_fullsized":"2024-09-19T04:19:47.148944+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:29:48.296104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309011+0000","last_change":"2024-09-19T04:19:45.178221+0000","last_active":"2024-09-19T04:19:46.309011+0000","last_peered":"2024-09-19T04:19:46.309011+0000","last_clean":"2024-09-19T04:19:46.309011+0000","last_became_active":"2024-09-19T04:19:45.177453+0000","last_became_peered":"2024-09-19T04:19:45.177453+0000","last_unstale":"2024-09-19T04:19:46.309011+0000","last_undegraded":"2024-09-19T04:19:46.309011+0000","last_fullsized":"2024-09-19T04:19:46.309011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:57.866440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309007+0000","last_change":"2024-09-19T04:19:45.185362+0000","last_active":"2024-09-19T04:19:46.309007+0000","last_peered":"2024-09-19T04:19:46.309007+0000","last_clean":"2024-09-19T04:19:46.309007+0000","last_became_active":"2024-09-19T04:19:45.184564+0000","last_became_peered":"2024-09-19T04:19:45.184564+0000","last_unstale":"2024-09-19T04:19:46.309007+0000","last_undegraded":"2024-09-19T04:19:46.309007+0000","last_fullsized":"2024-09-19T04:19:46.309007+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:19:04.783732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309009+0000","last_change":"2024-09-19T04:19:45.170439+0000","last_active":"2024-09-19T04:19:46.309009+0000","last_peered":"2024-09-19T04:19:46.309009+0000","last_clean":"2024-09-19T04:19:46.309009+0000","last_became_active":"2024-09-19T04:19:45.169639+0000","last_became_peered":"2024-09-19T04:19:45.169639+0000","last_unstale":"2024-09-19T04:19:46.309009+0000","last_undegraded":"2024-09-19T04:19:46.309009+0000","last_fullsized":"2024-09-19T04:19:46.309009+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:54:12.953139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144130+0000","last_change":"2024-09-19T04:19:45.168627+0000","last_active":"2024-09-19T04:19:46.144130+0000","last_peered":"2024-09-19T04:19:46.144130+0000","last_clean":"2024-09-19T04:19:46.144130+0000","last_became_active":"2024-09-19T04:19:45.167855+0000","last_became_peered":"2024-09-19T04:19:45.167855+0000","last_unstale":"2024-09-19T04:19:46.144130+0000","last_undegraded":"2024-09-19T04:19:46.144130+0000","last_fullsized":"2024-09-19T04:19:46.144130+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:05:02.245083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148208+0000","last_change":"2024-09-19T04:19:45.192276+0000","last_active":"2024-09-19T04:19:47.148208+0000","last_peered":"2024-09-19T04:19:47.148208+0000","last_clean":"2024-09-19T04:19:47.148208+0000","last_became_active":"2024-09-19T04:19:45.191610+0000","last_became_peered":"2024-09-19T04:19:45.191610+0000","last_unstale":"2024-09-19T04:19:47.148208+0000","last_undegraded":"2024-09-19T04:19:47.148208+0000","last_fullsized":"2024-09-19T04:19:47.148208+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:08:58.727906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143180+0000","last_change":"2024-09-19T04:19:45.179524+0000","last_active":"2024-09-19T04:19:46.143180+0000","last_peered":"2024-09-19T04:19:46.143180+0000","last_clean":"2024-09-19T04:19:46.143180+0000","last_became_active":"2024-09-19T04:19:45.178987+0000","last_became_peered":"2024-09-19T04:19:45.178987+0000","last_unstale":"2024-09-19T04:19:46.143180+0000","last_undegraded":"2024-09-19T04:19:46.143180+0000","last_fullsized":"2024-09-19T04:19:46.143180+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:52:44.531383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145750+0000","last_change":"2024-09-19T04:19:45.168512+0000","last_active":"2024-09-19T04:19:46.145750+0000","last_peered":"2024-09-19T04:19:46.145750+0000","last_clean":"2024-09-19T04:19:46.145750+0000","last_became_active":"2024-09-19T04:19:45.167737+0000","last_became_peered":"2024-09-19T04:19:45.167737+0000","last_unstale":"2024-09-19T04:19:46.145750+0000","last_undegraded":"2024-09-19T04:19:46.145750+0000","last_fullsized":"2024-09-19T04:19:46.145750+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:51.049472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583701+0000","last_change":"2024-09-19T04:19:45.176746+0000","last_active":"2024-09-19T04:19:46.583701+0000","last_peered":"2024-09-19T04:19:46.583701+0000","last_clean":"2024-09-19T04:19:46.583701+0000","last_became_active":"2024-09-19T04:19:45.175869+0000","last_became_peered":"2024-09-19T04:19:45.175869+0000","last_unstale":"2024-09-19T04:19:46.583701+0000","last_undegraded":"2024-09-19T04:19:46.583701+0000","last_fullsized":"2024-09-19T04:19:46.583701+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:11.483700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148761+0000","last_change":"2024-09-19T04:19:45.190525+0000","last_active":"2024-09-19T04:19:47.148761+0000","last_peered":"2024-09-19T04:19:47.148761+0000","last_clean":"2024-09-19T04:19:47.148761+0000","last_became_active":"2024-09-19T04:19:45.184256+0000","last_became_peered":"2024-09-19T04:19:45.184256+0000","last_unstale":"2024-09-19T04:19:47.148761+0000","last_undegraded":"2024-09-19T04:19:47.148761+0000","last_fullsized":"2024-09-19T04:19:47.148761+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:18.279383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144384+0000","last_change":"2024-09-19T04:19:45.167039+0000","last_active":"2024-09-19T04:19:46.144384+0000","last_peered":"2024-09-19T04:19:46.144384+0000","last_clean":"2024-09-19T04:19:46.144384+0000","last_became_active":"2024-09-19T04:19:45.166242+0000","last_became_peered":"2024-09-19T04:19:45.166242+0000","last_unstale":"2024-09-19T04:19:46.144384+0000","last_undegraded":"2024-09-19T04:19:46.144384+0000","last_fullsized":"2024-09-19T04:19:46.144384+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:24:04.971293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147689+0000","last_change":"2024-09-19T04:19:45.184944+0000","last_active":"2024-09-19T04:19:47.147689+0000","last_peered":"2024-09-19T04:19:47.147689+0000","last_clean":"2024-09-19T04:19:47.147689+0000","last_became_active":"2024-09-19T04:19:45.183888+0000","last_became_peered":"2024-09-19T04:19:45.183888+0000","last_unstale":"2024-09-19T04:19:47.147689+0000","last_undegraded":"2024-09-19T04:19:47.147689+0000","last_fullsized":"2024-09-19T04:19:47.147689+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:33:34.494505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144747+0000","last_change":"2024-09-19T04:19:45.169001+0000","last_active":"2024-09-19T04:19:46.144747+0000","last_peered":"2024-09-19T04:19:46.144747+0000","last_clean":"2024-09-19T04:19:46.144747+0000","last_became_active":"2024-09-19T04:19:45.168286+0000","last_became_peered":"2024-09-19T04:19:45.168286+0000","last_unstale":"2024-09-19T04:19:46.144747+0000","last_undegraded":"2024-09-19T04:19:46.144747+0000","last_fullsized":"2024-09-19T04:19:46.144747+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145050+0000","last_change":"2024-09-19T04:19:45.184020+0000","last_active":"2024-09-19T04:19:46.145050+0000","last_peered":"2024-09-19T04:19:46.145050+0000","last_clean":"2024-09-19T04:19:46.145050+0000","last_became_active":"2024-09-19T04:19:45.183293+0000","last_became_peered":"2024-09-19T04:19:45.183293+0000","last_unstale":"2024-09-19T04:19:46.145050+0000","last_undegraded":"2024-09-19T04:19:46.145050+0000","last_fullsized":"2024-09-19T04:19:46.145050+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145517+0000","last_change":"2024-09-19T04:19:45.173612+0000","last_active":"2024-09-19T04:19:46.145517+0000","last_peered":"2024-09-19T04:19:46.145517+0000","last_clean":"2024-09-19T04:19:46.145517+0000","last_became_active":"2024-09-19T04:19:45.172851+0000","last_became_peered":"2024-09-19T04:19:45.172851+0000","last_unstale":"2024-09-19T04:19:46.145517+0000","last_undegraded":"2024-09-19T04:19:46.145517+0000","last_fullsized":"2024-09-19T04:19:46.145517+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:24.112252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147845+0000","last_change":"2024-09-19T04:19:45.166979+0000","last_active":"2024-09-19T04:19:47.147845+0000","last_peered":"2024-09-19T04:19:47.147845+0000","last_clean":"2024-09-19T04:19:47.147845+0000","last_became_active":"2024-09-19T04:19:45.166219+0000","last_became_peered":"2024-09-19T04:19:45.166219+0000","last_unstale":"2024-09-19T04:19:47.147845+0000","last_undegraded":"2024-09-19T04:19:47.147845+0000","last_fullsized":"2024-09-19T04:19:47.147845+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:59:13.454807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583946+0000","last_change":"2024-09-19T04:19:45.172717+0000","last_active":"2024-09-19T04:19:46.583946+0000","last_peered":"2024-09-19T04:19:46.583946+0000","last_clean":"2024-09-19T04:19:46.583946+0000","last_became_active":"2024-09-19T04:19:45.171590+0000","last_became_peered":"2024-09-19T04:19:45.171590+0000","last_unstale":"2024-09-19T04:19:46.583946+0000","last_undegraded":"2024-09-19T04:19:46.583946+0000","last_fullsized":"2024-09-19T04:19:46.583946+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:34:20.010469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143759+0000","last_change":"2024-09-19T04:19:45.175929+0000","last_active":"2024-09-19T04:19:46.143759+0000","last_peered":"2024-09-19T04:19:46.143759+0000","last_clean":"2024-09-19T04:19:46.143759+0000","last_became_active":"2024-09-19T04:19:45.175243+0000","last_became_peered":"2024-09-19T04:19:45.175243+0000","last_unstale":"2024-09-19T04:19:46.143759+0000","last_undegraded":"2024-09-19T04:19:46.143759+0000","last_fullsized":"2024-09-19T04:19:46.143759+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:35:18.279383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.146996+0000","last_change":"2024-09-19T04:19:45.169266+0000","last_active":"2024-09-19T04:19:47.146996+0000","last_peered":"2024-09-19T04:19:47.146996+0000","last_clean":"2024-09-19T04:19:47.146996+0000","last_became_active":"2024-09-19T04:19:45.168439+0000","last_became_peered":"2024-09-19T04:19:45.168439+0000","last_unstale":"2024-09-19T04:19:47.146996+0000","last_undegraded":"2024-09-19T04:19:47.146996+0000","last_fullsized":"2024-09-19T04:19:47.146996+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583072+0000","last_change":"2024-09-19T04:19:45.164808+0000","last_active":"2024-09-19T04:19:46.583072+0000","last_peered":"2024-09-19T04:19:46.583072+0000","last_clean":"2024-09-19T04:19:46.583072+0000","last_became_active":"2024-09-19T04:19:45.163828+0000","last_became_peered":"2024-09-19T04:19:45.163828+0000","last_unstale":"2024-09-19T04:19:46.583072+0000","last_undegraded":"2024-09-19T04:19:46.583072+0000","last_fullsized":"2024-09-19T04:19:46.583072+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:56.653211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143466+0000","last_change":"2024-09-19T04:19:45.180066+0000","last_active":"2024-09-19T04:19:46.143466+0000","last_peered":"2024-09-19T04:19:46.143466+0000","last_clean":"2024-09-19T04:19:46.143466+0000","last_became_active":"2024-09-19T04:19:45.179314+0000","last_became_peered":"2024-09-19T04:19:45.179314+0000","last_unstale":"2024-09-19T04:19:46.143466+0000","last_undegraded":"2024-09-19T04:19:46.143466+0000","last_fullsized":"2024-09-19T04:19:46.143466+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:56.657775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"103'4","reported_seq":18,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149783+0000","last_change":"2024-09-19T04:19:44.231697+0000","last_active":"2024-09-19T04:19:47.149783+0000","last_peered":"2024-09-19T04:19:47.149783+0000","last_clean":"2024-09-19T04:19:47.149783+0000","last_became_active":"2024-09-19T04:19:44.231076+0000","last_became_peered":"2024-09-19T04:19:44.231076+0000","last_unstale":"2024-09-19T04:19:47.149783+0000","last_undegraded":"2024-09-19T04:19:47.149783+0000","last_fullsized":"2024-09-19T04:19:47.149783+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:12:46.696922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583010+0000","last_change":"2024-09-19T04:19:45.193367+0000","last_active":"2024-09-19T04:19:46.583010+0000","last_peered":"2024-09-19T04:19:46.583010+0000","last_clean":"2024-09-19T04:19:46.583010+0000","last_became_active":"2024-09-19T04:19:45.192916+0000","last_became_peered":"2024-09-19T04:19:45.192916+0000","last_unstale":"2024-09-19T04:19:46.583010+0000","last_undegraded":"2024-09-19T04:19:46.583010+0000","last_fullsized":"2024-09-19T04:19:46.583010+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:50:57.350022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583650+0000","last_change":"2024-09-19T04:19:44.213081+0000","last_active":"2024-09-19T04:19:46.583650+0000","last_peered":"2024-09-19T04:19:46.583650+0000","last_clean":"2024-09-19T04:19:46.583650+0000","last_became_active":"2024-09-19T04:19:44.212554+0000","last_became_peered":"2024-09-19T04:19:44.212554+0000","last_unstale":"2024-09-19T04:19:46.583650+0000","last_undegraded":"2024-09-19T04:19:46.583650+0000","last_fullsized":"2024-09-19T04:19:46.583650+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:16.294562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144160+0000","last_change":"2024-09-19T04:19:45.198183+0000","last_active":"2024-09-19T04:19:46.144160+0000","last_peered":"2024-09-19T04:19:46.144160+0000","last_clean":"2024-09-19T04:19:46.144160+0000","last_became_active":"2024-09-19T04:19:45.197436+0000","last_became_peered":"2024-09-19T04:19:45.197436+0000","last_unstale":"2024-09-19T04:19:46.144160+0000","last_undegraded":"2024-09-19T04:19:46.144160+0000","last_fullsized":"2024-09-19T04:19:46.144160+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:12.253975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144840+0000","last_change":"2024-09-19T04:19:44.232018+0000","last_active":"2024-09-19T04:19:46.144840+0000","last_peered":"2024-09-19T04:19:46.144840+0000","last_clean":"2024-09-19T04:19:46.144840+0000","last_became_active":"2024-09-19T04:19:44.226498+0000","last_became_peered":"2024-09-19T04:19:44.226498+0000","last_unstale":"2024-09-19T04:19:46.144840+0000","last_undegraded":"2024-09-19T04:19:46.144840+0000","last_fullsized":"2024-09-19T04:19:46.144840+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:30.396563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145023+0000","last_change":"2024-09-19T04:19:45.163758+0000","last_active":"2024-09-19T04:19:46.145023+0000","last_peered":"2024-09-19T04:19:46.145023+0000","last_clean":"2024-09-19T04:19:46.145023+0000","last_became_active":"2024-09-19T04:19:45.162990+0000","last_became_peered":"2024-09-19T04:19:45.162990+0000","last_unstale":"2024-09-19T04:19:46.145023+0000","last_undegraded":"2024-09-19T04:19:46.145023+0000","last_fullsized":"2024-09-19T04:19:46.145023+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:15:51.049472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148427+0000","last_change":"2024-09-19T04:19:44.226570+0000","last_active":"2024-09-19T04:19:47.148427+0000","last_peered":"2024-09-19T04:19:47.148427+0000","last_clean":"2024-09-19T04:19:47.148427+0000","last_became_active":"2024-09-19T04:19:44.226215+0000","last_became_peered":"2024-09-19T04:19:44.226215+0000","last_unstale":"2024-09-19T04:19:47.148427+0000","last_undegraded":"2024-09-19T04:19:47.148427+0000","last_fullsized":"2024-09-19T04:19:47.148427+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.434191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149726+0000","last_change":"2024-09-19T04:19:45.199474+0000","last_active":"2024-09-19T04:19:47.149726+0000","last_peered":"2024-09-19T04:19:47.149726+0000","last_clean":"2024-09-19T04:19:47.149726+0000","last_became_active":"2024-09-19T04:19:45.198928+0000","last_became_peered":"2024-09-19T04:19:45.198928+0000","last_unstale":"2024-09-19T04:19:47.149726+0000","last_undegraded":"2024-09-19T04:19:47.149726+0000","last_fullsized":"2024-09-19T04:19:47.149726+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:59:55.032656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309273+0000","last_change":"2024-09-19T04:19:44.222531+0000","last_active":"2024-09-19T04:19:46.309273+0000","last_peered":"2024-09-19T04:19:46.309273+0000","last_clean":"2024-09-19T04:19:46.309273+0000","last_became_active":"2024-09-19T04:19:44.218242+0000","last_became_peered":"2024-09-19T04:19:44.218242+0000","last_unstale":"2024-09-19T04:19:46.309273+0000","last_undegraded":"2024-09-19T04:19:46.309273+0000","last_fullsized":"2024-09-19T04:19:46.309273+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:04:04.937333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143477+0000","last_change":"2024-09-19T04:19:45.180374+0000","last_active":"2024-09-19T04:19:46.143477+0000","last_peered":"2024-09-19T04:19:46.143477+0000","last_clean":"2024-09-19T04:19:46.143477+0000","last_became_active":"2024-09-19T04:19:45.179795+0000","last_became_peered":"2024-09-19T04:19:45.179795+0000","last_unstale":"2024-09-19T04:19:46.143477+0000","last_undegraded":"2024-09-19T04:19:46.143477+0000","last_fullsized":"2024-09-19T04:19:46.143477+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:13:30.929127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144496+0000","last_change":"2024-09-19T04:19:44.241686+0000","last_active":"2024-09-19T04:19:46.144496+0000","last_peered":"2024-09-19T04:19:46.144496+0000","last_clean":"2024-09-19T04:19:46.144496+0000","last_became_active":"2024-09-19T04:19:44.241051+0000","last_became_peered":"2024-09-19T04:19:44.241051+0000","last_unstale":"2024-09-19T04:19:46.144496+0000","last_undegraded":"2024-09-19T04:19:46.144496+0000","last_fullsized":"2024-09-19T04:19:46.144496+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:21:48.538899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148294+0000","last_change":"2024-09-19T04:19:45.175667+0000","last_active":"2024-09-19T04:19:47.148294+0000","last_peered":"2024-09-19T04:19:47.148294+0000","last_clean":"2024-09-19T04:19:47.148294+0000","last_became_active":"2024-09-19T04:19:45.168151+0000","last_became_peered":"2024-09-19T04:19:45.168151+0000","last_unstale":"2024-09-19T04:19:47.148294+0000","last_undegraded":"2024-09-19T04:19:47.148294+0000","last_fullsized":"2024-09-19T04:19:47.148294+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:10:56.170958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143136+0000","last_change":"2024-09-19T04:19:44.225516+0000","last_active":"2024-09-19T04:19:46.143136+0000","last_peered":"2024-09-19T04:19:46.143136+0000","last_clean":"2024-09-19T04:19:46.143136+0000","last_became_active":"2024-09-19T04:19:44.224853+0000","last_became_peered":"2024-09-19T04:19:44.224853+0000","last_unstale":"2024-09-19T04:19:46.143136+0000","last_undegraded":"2024-09-19T04:19:46.143136+0000","last_fullsized":"2024-09-19T04:19:46.143136+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:44.458855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149613+0000","last_change":"2024-09-19T04:19:45.178996+0000","last_active":"2024-09-19T04:19:47.149613+0000","last_peered":"2024-09-19T04:19:47.149613+0000","last_clean":"2024-09-19T04:19:47.149613+0000","last_became_active":"2024-09-19T04:19:45.178210+0000","last_became_peered":"2024-09-19T04:19:45.178210+0000","last_unstale":"2024-09-19T04:19:47.149613+0000","last_undegraded":"2024-09-19T04:19:47.149613+0000","last_fullsized":"2024-09-19T04:19:47.149613+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:26:52.242923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148880+0000","last_change":"2024-09-19T04:19:44.212254+0000","last_active":"2024-09-19T04:19:47.148880+0000","last_peered":"2024-09-19T04:19:47.148880+0000","last_clean":"2024-09-19T04:19:47.148880+0000","last_became_active":"2024-09-19T04:19:44.211798+0000","last_became_peered":"2024-09-19T04:19:44.211798+0000","last_unstale":"2024-09-19T04:19:47.148880+0000","last_undegraded":"2024-09-19T04:19:47.148880+0000","last_fullsized":"2024-09-19T04:19:47.148880+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:57.574720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143084+0000","last_change":"2024-09-19T04:19:45.170015+0000","last_active":"2024-09-19T04:19:46.143084+0000","last_peered":"2024-09-19T04:19:46.143084+0000","last_clean":"2024-09-19T04:19:46.143084+0000","last_became_active":"2024-09-19T04:19:45.160059+0000","last_became_peered":"2024-09-19T04:19:45.160059+0000","last_unstale":"2024-09-19T04:19:46.143084+0000","last_undegraded":"2024-09-19T04:19:46.143084+0000","last_fullsized":"2024-09-19T04:19:46.143084+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:38.829936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145125+0000","last_change":"2024-09-19T04:19:44.232439+0000","last_active":"2024-09-19T04:19:46.145125+0000","last_peered":"2024-09-19T04:19:46.145125+0000","last_clean":"2024-09-19T04:19:46.145125+0000","last_became_active":"2024-09-19T04:19:44.231570+0000","last_became_peered":"2024-09-19T04:19:44.231570+0000","last_unstale":"2024-09-19T04:19:46.145125+0000","last_undegraded":"2024-09-19T04:19:46.145125+0000","last_fullsized":"2024-09-19T04:19:46.145125+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:01:07.475209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148081+0000","last_change":"2024-09-19T04:19:45.186053+0000","last_active":"2024-09-19T04:19:47.148081+0000","last_peered":"2024-09-19T04:19:47.148081+0000","last_clean":"2024-09-19T04:19:47.148081+0000","last_became_active":"2024-09-19T04:19:45.183824+0000","last_became_peered":"2024-09-19T04:19:45.183824+0000","last_unstale":"2024-09-19T04:19:47.148081+0000","last_undegraded":"2024-09-19T04:19:47.148081+0000","last_fullsized":"2024-09-19T04:19:47.148081+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:03:01.109409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143658+0000","last_change":"2024-09-19T04:19:44.233959+0000","last_active":"2024-09-19T04:19:46.143658+0000","last_peered":"2024-09-19T04:19:46.143658+0000","last_clean":"2024-09-19T04:19:46.143658+0000","last_became_active":"2024-09-19T04:19:44.233633+0000","last_became_peered":"2024-09-19T04:19:44.233633+0000","last_unstale":"2024-09-19T04:19:46.143658+0000","last_undegraded":"2024-09-19T04:19:46.143658+0000","last_fullsized":"2024-09-19T04:19:46.143658+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:03:16.273621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583501+0000","last_change":"2024-09-19T04:19:45.161029+0000","last_active":"2024-09-19T04:19:46.583501+0000","last_peered":"2024-09-19T04:19:46.583501+0000","last_clean":"2024-09-19T04:19:46.583501+0000","last_became_active":"2024-09-19T04:19:45.160312+0000","last_became_peered":"2024-09-19T04:19:45.160312+0000","last_unstale":"2024-09-19T04:19:46.583501+0000","last_undegraded":"2024-09-19T04:19:46.583501+0000","last_fullsized":"2024-09-19T04:19:46.583501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:34:14.593374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"103'3","reported_seq":17,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309376+0000","last_change":"2024-09-19T04:19:44.217666+0000","last_active":"2024-09-19T04:19:46.309376+0000","last_peered":"2024-09-19T04:19:46.309376+0000","last_clean":"2024-09-19T04:19:46.309376+0000","last_became_active":"2024-09-19T04:19:44.216889+0000","last_became_peered":"2024-09-19T04:19:44.216889+0000","last_unstale":"2024-09-19T04:19:46.309376+0000","last_undegraded":"2024-09-19T04:19:46.309376+0000","last_fullsized":"2024-09-19T04:19:46.309376+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:24:17.543539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149633+0000","last_change":"2024-09-19T04:19:45.190070+0000","last_active":"2024-09-19T04:19:47.149633+0000","last_peered":"2024-09-19T04:19:47.149633+0000","last_clean":"2024-09-19T04:19:47.149633+0000","last_became_active":"2024-09-19T04:19:45.185072+0000","last_became_peered":"2024-09-19T04:19:45.185072+0000","last_unstale":"2024-09-19T04:19:47.149633+0000","last_undegraded":"2024-09-19T04:19:47.149633+0000","last_fullsized":"2024-09-19T04:19:47.149633+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"103'6","reported_seq":20,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147537+0000","last_change":"2024-09-19T04:19:44.230464+0000","last_active":"2024-09-19T04:19:47.147537+0000","last_peered":"2024-09-19T04:19:47.147537+0000","last_clean":"2024-09-19T04:19:47.147537+0000","last_became_active":"2024-09-19T04:19:44.229641+0000","last_became_peered":"2024-09-19T04:19:44.229641+0000","last_unstale":"2024-09-19T04:19:47.147537+0000","last_undegraded":"2024-09-19T04:19:47.147537+0000","last_fullsized":"2024-09-19T04:19:47.147537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:47.639016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144520+0000","last_change":"2024-09-19T04:19:45.171897+0000","last_active":"2024-09-19T04:19:46.144520+0000","last_peered":"2024-09-19T04:19:46.144520+0000","last_clean":"2024-09-19T04:19:46.144520+0000","last_became_active":"2024-09-19T04:19:45.171180+0000","last_became_peered":"2024-09-19T04:19:45.171180+0000","last_unstale":"2024-09-19T04:19:46.144520+0000","last_undegraded":"2024-09-19T04:19:46.144520+0000","last_fullsized":"2024-09-19T04:19:46.144520+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:58.578284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149925+0000","last_change":"2024-09-19T04:19:44.215853+0000","last_active":"2024-09-19T04:19:47.149925+0000","last_peered":"2024-09-19T04:19:47.149925+0000","last_clean":"2024-09-19T04:19:47.149925+0000","last_became_active":"2024-09-19T04:19:44.215492+0000","last_became_peered":"2024-09-19T04:19:44.215492+0000","last_unstale":"2024-09-19T04:19:47.149925+0000","last_undegraded":"2024-09-19T04:19:47.149925+0000","last_fullsized":"2024-09-19T04:19:47.149925+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:41.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147891+0000","last_change":"2024-09-19T04:19:45.168677+0000","last_active":"2024-09-19T04:19:47.147891+0000","last_peered":"2024-09-19T04:19:47.147891+0000","last_clean":"2024-09-19T04:19:47.147891+0000","last_became_active":"2024-09-19T04:19:45.167897+0000","last_became_peered":"2024-09-19T04:19:45.167897+0000","last_unstale":"2024-09-19T04:19:47.147891+0000","last_undegraded":"2024-09-19T04:19:47.147891+0000","last_fullsized":"2024-09-19T04:19:47.147891+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:01:24.112252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149527+0000","last_change":"2024-09-19T04:19:44.218497+0000","last_active":"2024-09-19T04:19:47.149527+0000","last_peered":"2024-09-19T04:19:47.149527+0000","last_clean":"2024-09-19T04:19:47.149527+0000","last_became_active":"2024-09-19T04:19:44.218114+0000","last_became_peered":"2024-09-19T04:19:44.218114+0000","last_unstale":"2024-09-19T04:19:47.149527+0000","last_undegraded":"2024-09-19T04:19:47.149527+0000","last_fullsized":"2024-09-19T04:19:47.149527+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:45.932308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.584147+0000","last_change":"2024-09-19T04:19:45.167687+0000","last_active":"2024-09-19T04:19:46.584147+0000","last_peered":"2024-09-19T04:19:46.584147+0000","last_clean":"2024-09-19T04:19:46.584147+0000","last_became_active":"2024-09-19T04:19:45.166759+0000","last_became_peered":"2024-09-19T04:19:45.166759+0000","last_unstale":"2024-09-19T04:19:46.584147+0000","last_undegraded":"2024-09-19T04:19:46.584147+0000","last_fullsized":"2024-09-19T04:19:46.584147+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:32:37.150211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583541+0000","last_change":"2024-09-19T04:19:44.227994+0000","last_active":"2024-09-19T04:19:46.583541+0000","last_peered":"2024-09-19T04:19:46.583541+0000","last_clean":"2024-09-19T04:19:46.583541+0000","last_became_active":"2024-09-19T04:19:44.227521+0000","last_became_peered":"2024-09-19T04:19:44.227521+0000","last_unstale":"2024-09-19T04:19:46.583541+0000","last_undegraded":"2024-09-19T04:19:46.583541+0000","last_fullsized":"2024-09-19T04:19:46.583541+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:18:03.300792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144312+0000","last_change":"2024-09-19T04:19:45.169434+0000","last_active":"2024-09-19T04:19:46.144312+0000","last_peered":"2024-09-19T04:19:46.144312+0000","last_clean":"2024-09-19T04:19:46.144312+0000","last_became_active":"2024-09-19T04:19:45.168222+0000","last_became_peered":"2024-09-19T04:19:45.168222+0000","last_unstale":"2024-09-19T04:19:46.144312+0000","last_undegraded":"2024-09-19T04:19:46.144312+0000","last_fullsized":"2024-09-19T04:19:46.144312+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:49:53.342842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"103'8","reported_seq":22,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.142788+0000","last_change":"2024-09-19T04:19:44.232483+0000","last_active":"2024-09-19T04:19:46.142788+0000","last_peered":"2024-09-19T04:19:46.142788+0000","last_clean":"2024-09-19T04:19:46.142788+0000","last_became_active":"2024-09-19T04:19:44.226270+0000","last_became_peered":"2024-09-19T04:19:44.226270+0000","last_unstale":"2024-09-19T04:19:46.142788+0000","last_undegraded":"2024-09-19T04:19:46.142788+0000","last_fullsized":"2024-09-19T04:19:46.142788+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:41.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583258+0000","last_change":"2024-09-19T04:19:45.172864+0000","last_active":"2024-09-19T04:19:46.583258+0000","last_peered":"2024-09-19T04:19:46.583258+0000","last_clean":"2024-09-19T04:19:46.583258+0000","last_became_active":"2024-09-19T04:19:45.172135+0000","last_became_peered":"2024-09-19T04:19:45.172135+0000","last_unstale":"2024-09-19T04:19:46.583258+0000","last_undegraded":"2024-09-19T04:19:46.583258+0000","last_fullsized":"2024-09-19T04:19:46.583258+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:46:21.270880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148091+0000","last_change":"2024-09-19T04:19:44.239785+0000","last_active":"2024-09-19T04:19:47.148091+0000","last_peered":"2024-09-19T04:19:47.148091+0000","last_clean":"2024-09-19T04:19:47.148091+0000","last_became_active":"2024-09-19T04:19:44.239052+0000","last_became_peered":"2024-09-19T04:19:44.239052+0000","last_unstale":"2024-09-19T04:19:47.148091+0000","last_undegraded":"2024-09-19T04:19:47.148091+0000","last_fullsized":"2024-09-19T04:19:47.148091+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:36:09.434191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147157+0000","last_change":"2024-09-19T04:19:45.169150+0000","last_active":"2024-09-19T04:19:47.147157+0000","last_peered":"2024-09-19T04:19:47.147157+0000","last_clean":"2024-09-19T04:19:47.147157+0000","last_became_active":"2024-09-19T04:19:45.168374+0000","last_became_peered":"2024-09-19T04:19:45.168374+0000","last_unstale":"2024-09-19T04:19:47.147157+0000","last_undegraded":"2024-09-19T04:19:47.147157+0000","last_fullsized":"2024-09-19T04:19:47.147157+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:34:58.578284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"103'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309563+0000","last_change":"2024-09-19T04:19:44.214954+0000","last_active":"2024-09-19T04:19:46.309563+0000","last_peered":"2024-09-19T04:19:46.309563+0000","last_clean":"2024-09-19T04:19:46.309563+0000","last_became_active":"2024-09-19T04:19:44.214233+0000","last_became_peered":"2024-09-19T04:19:44.214233+0000","last_unstale":"2024-09-19T04:19:46.309563+0000","last_undegraded":"2024-09-19T04:19:46.309563+0000","last_fullsized":"2024-09-19T04:19:46.309563+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:59:39.430232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149303+0000","last_change":"2024-09-19T04:19:45.190240+0000","last_active":"2024-09-19T04:19:47.149303+0000","last_peered":"2024-09-19T04:19:47.149303+0000","last_clean":"2024-09-19T04:19:47.149303+0000","last_became_active":"2024-09-19T04:19:45.185691+0000","last_became_peered":"2024-09-19T04:19:45.185691+0000","last_unstale":"2024-09-19T04:19:47.149303+0000","last_undegraded":"2024-09-19T04:19:47.149303+0000","last_fullsized":"2024-09-19T04:19:47.149303+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144077+0000","last_change":"2024-09-19T04:19:44.231274+0000","last_active":"2024-09-19T04:19:46.144077+0000","last_peered":"2024-09-19T04:19:46.144077+0000","last_clean":"2024-09-19T04:19:46.144077+0000","last_became_active":"2024-09-19T04:19:44.230495+0000","last_became_peered":"2024-09-19T04:19:44.230495+0000","last_unstale":"2024-09-19T04:19:46.144077+0000","last_undegraded":"2024-09-19T04:19:46.144077+0000","last_fullsized":"2024-09-19T04:19:46.144077+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:48:45.711711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143637+0000","last_change":"2024-09-19T04:19:45.167264+0000","last_active":"2024-09-19T04:19:46.143637+0000","last_peered":"2024-09-19T04:19:46.143637+0000","last_clean":"2024-09-19T04:19:46.143637+0000","last_became_active":"2024-09-19T04:19:45.166457+0000","last_became_peered":"2024-09-19T04:19:45.166457+0000","last_unstale":"2024-09-19T04:19:46.143637+0000","last_undegraded":"2024-09-19T04:19:46.143637+0000","last_fullsized":"2024-09-19T04:19:46.143637+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:51.079323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583959+0000","last_change":"2024-09-19T04:19:44.208558+0000","last_active":"2024-09-19T04:19:46.583959+0000","last_peered":"2024-09-19T04:19:46.583959+0000","last_clean":"2024-09-19T04:19:46.583959+0000","last_became_active":"2024-09-19T04:19:44.208143+0000","last_became_peered":"2024-09-19T04:19:44.208143+0000","last_unstale":"2024-09-19T04:19:46.583959+0000","last_undegraded":"2024-09-19T04:19:46.583959+0000","last_fullsized":"2024-09-19T04:19:46.583959+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:00:24.355737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143332+0000","last_change":"2024-09-19T04:19:45.196062+0000","last_active":"2024-09-19T04:19:46.143332+0000","last_peered":"2024-09-19T04:19:46.143332+0000","last_clean":"2024-09-19T04:19:46.143332+0000","last_became_active":"2024-09-19T04:19:45.195515+0000","last_became_peered":"2024-09-19T04:19:45.195515+0000","last_unstale":"2024-09-19T04:19:46.143332+0000","last_undegraded":"2024-09-19T04:19:46.143332+0000","last_fullsized":"2024-09-19T04:19:46.143332+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:54:51.346757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.582875+0000","last_change":"2024-09-19T04:19:44.218216+0000","last_active":"2024-09-19T04:19:46.582875+0000","last_peered":"2024-09-19T04:19:46.582875+0000","last_clean":"2024-09-19T04:19:46.582875+0000","last_became_active":"2024-09-19T04:19:44.217207+0000","last_became_peered":"2024-09-19T04:19:44.217207+0000","last_unstale":"2024-09-19T04:19:46.582875+0000","last_undegraded":"2024-09-19T04:19:46.582875+0000","last_fullsized":"2024-09-19T04:19:46.582875+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:06:29.851224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.150111+0000","last_change":"2024-09-19T04:19:45.187264+0000","last_active":"2024-09-19T04:19:47.150111+0000","last_peered":"2024-09-19T04:19:47.150111+0000","last_clean":"2024-09-19T04:19:47.150111+0000","last_became_active":"2024-09-19T04:19:45.186733+0000","last_became_peered":"2024-09-19T04:19:45.186733+0000","last_unstale":"2024-09-19T04:19:47.150111+0000","last_undegraded":"2024-09-19T04:19:47.150111+0000","last_fullsized":"2024-09-19T04:19:47.150111+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:58:12.105319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147553+0000","last_change":"2024-09-19T04:19:44.225860+0000","last_active":"2024-09-19T04:19:47.147553+0000","last_peered":"2024-09-19T04:19:47.147553+0000","last_clean":"2024-09-19T04:19:47.147553+0000","last_became_active":"2024-09-19T04:19:44.225195+0000","last_became_peered":"2024-09-19T04:19:44.225195+0000","last_unstale":"2024-09-19T04:19:47.147553+0000","last_undegraded":"2024-09-19T04:19:47.147553+0000","last_fullsized":"2024-09-19T04:19:47.147553+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:30:41.217655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583649+0000","last_change":"2024-09-19T04:19:45.195754+0000","last_active":"2024-09-19T04:19:46.583649+0000","last_peered":"2024-09-19T04:19:46.583649+0000","last_clean":"2024-09-19T04:19:46.583649+0000","last_became_active":"2024-09-19T04:19:45.195351+0000","last_became_peered":"2024-09-19T04:19:45.195351+0000","last_unstale":"2024-09-19T04:19:46.583649+0000","last_undegraded":"2024-09-19T04:19:46.583649+0000","last_fullsized":"2024-09-19T04:19:46.583649+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:02:11.906613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583353+0000","last_change":"2024-09-19T04:19:44.212879+0000","last_active":"2024-09-19T04:19:46.583353+0000","last_peered":"2024-09-19T04:19:46.583353+0000","last_clean":"2024-09-19T04:19:46.583353+0000","last_became_active":"2024-09-19T04:19:44.212515+0000","last_became_peered":"2024-09-19T04:19:44.212515+0000","last_unstale":"2024-09-19T04:19:46.583353+0000","last_undegraded":"2024-09-19T04:19:46.583353+0000","last_fullsized":"2024-09-19T04:19:46.583353+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:47:51.882953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143045+0000","last_change":"2024-09-19T04:19:45.173519+0000","last_active":"2024-09-19T04:19:46.143045+0000","last_peered":"2024-09-19T04:19:46.143045+0000","last_clean":"2024-09-19T04:19:46.143045+0000","last_became_active":"2024-09-19T04:19:45.172808+0000","last_became_peered":"2024-09-19T04:19:45.172808+0000","last_unstale":"2024-09-19T04:19:46.143045+0000","last_undegraded":"2024-09-19T04:19:46.143045+0000","last_fullsized":"2024-09-19T04:19:46.143045+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:12:33.683069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148230+0000","last_change":"2024-09-19T04:19:44.214785+0000","last_active":"2024-09-19T04:19:47.148230+0000","last_peered":"2024-09-19T04:19:47.148230+0000","last_clean":"2024-09-19T04:19:47.148230+0000","last_became_active":"2024-09-19T04:19:44.214401+0000","last_became_peered":"2024-09-19T04:19:44.214401+0000","last_unstale":"2024-09-19T04:19:47.148230+0000","last_undegraded":"2024-09-19T04:19:47.148230+0000","last_fullsized":"2024-09-19T04:19:47.148230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:44.458855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.142814+0000","last_change":"2024-09-19T04:19:45.178122+0000","last_active":"2024-09-19T04:19:46.142814+0000","last_peered":"2024-09-19T04:19:46.142814+0000","last_clean":"2024-09-19T04:19:46.142814+0000","last_became_active":"2024-09-19T04:19:45.177383+0000","last_became_peered":"2024-09-19T04:19:45.177383+0000","last_unstale":"2024-09-19T04:19:46.142814+0000","last_undegraded":"2024-09-19T04:19:46.142814+0000","last_fullsized":"2024-09-19T04:19:46.142814+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:31:53.207406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"104'1","reported_seq":15,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.207152+0000","last_change":"2024-09-19T04:19:44.223941+0000","last_active":"2024-09-19T04:19:46.207152+0000","last_peered":"2024-09-19T04:19:46.207152+0000","last_clean":"2024-09-19T04:19:46.207152+0000","last_became_active":"2024-09-19T04:19:44.223600+0000","last_became_peered":"2024-09-19T04:19:44.223600+0000","last_unstale":"2024-09-19T04:19:46.207152+0000","last_undegraded":"2024-09-19T04:19:46.207152+0000","last_fullsized":"2024-09-19T04:19:46.207152+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:21:31.282568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.149290+0000","last_change":"2024-09-19T04:19:45.185218+0000","last_active":"2024-09-19T04:19:47.149290+0000","last_peered":"2024-09-19T04:19:47.149290+0000","last_clean":"2024-09-19T04:19:47.149290+0000","last_became_active":"2024-09-19T04:19:45.184686+0000","last_became_peered":"2024-09-19T04:19:45.184686+0000","last_unstale":"2024-09-19T04:19:47.149290+0000","last_undegraded":"2024-09-19T04:19:47.149290+0000","last_fullsized":"2024-09-19T04:19:47.149290+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:13:14.992786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147530+0000","last_change":"2024-09-19T04:19:44.223965+0000","last_active":"2024-09-19T04:19:47.147530+0000","last_peered":"2024-09-19T04:19:47.147530+0000","last_clean":"2024-09-19T04:19:47.147530+0000","last_became_active":"2024-09-19T04:19:44.223266+0000","last_became_peered":"2024-09-19T04:19:44.223266+0000","last_unstale":"2024-09-19T04:19:47.147530+0000","last_undegraded":"2024-09-19T04:19:47.147530+0000","last_fullsized":"2024-09-19T04:19:47.147530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:51:44.458855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.145044+0000","last_change":"2024-09-19T04:19:45.199988+0000","last_active":"2024-09-19T04:19:46.145044+0000","last_peered":"2024-09-19T04:19:46.145044+0000","last_clean":"2024-09-19T04:19:46.145044+0000","last_became_active":"2024-09-19T04:19:45.199288+0000","last_became_peered":"2024-09-19T04:19:45.199288+0000","last_unstale":"2024-09-19T04:19:46.145044+0000","last_undegraded":"2024-09-19T04:19:46.145044+0000","last_fullsized":"2024-09-19T04:19:46.145044+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:12:47.700486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.148715+0000","last_change":"2024-09-19T04:19:44.231683+0000","last_active":"2024-09-19T04:19:47.148715+0000","last_peered":"2024-09-19T04:19:47.148715+0000","last_clean":"2024-09-19T04:19:47.148715+0000","last_became_active":"2024-09-19T04:19:44.231251+0000","last_became_peered":"2024-09-19T04:19:44.231251+0000","last_unstale":"2024-09-19T04:19:47.148715+0000","last_undegraded":"2024-09-19T04:19:47.148715+0000","last_fullsized":"2024-09-19T04:19:47.148715+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:47.639016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583263+0000","last_change":"2024-09-19T04:19:45.201980+0000","last_active":"2024-09-19T04:19:46.583263+0000","last_peered":"2024-09-19T04:19:46.583263+0000","last_clean":"2024-09-19T04:19:46.583263+0000","last_became_active":"2024-09-19T04:19:45.201738+0000","last_became_peered":"2024-09-19T04:19:45.201738+0000","last_unstale":"2024-09-19T04:19:46.583263+0000","last_undegraded":"2024-09-19T04:19:46.583263+0000","last_fullsized":"2024-09-19T04:19:46.583263+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:18:04.519511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144118+0000","last_change":"2024-09-19T04:19:44.231967+0000","last_active":"2024-09-19T04:19:46.144118+0000","last_peered":"2024-09-19T04:19:46.144118+0000","last_clean":"2024-09-19T04:19:46.144118+0000","last_became_active":"2024-09-19T04:19:44.231185+0000","last_became_peered":"2024-09-19T04:19:44.231185+0000","last_unstale":"2024-09-19T04:19:46.144118+0000","last_undegraded":"2024-09-19T04:19:46.144118+0000","last_fullsized":"2024-09-19T04:19:46.144118+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:01:47.639016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583820+0000","last_change":"2024-09-19T04:19:45.202821+0000","last_active":"2024-09-19T04:19:46.583820+0000","last_peered":"2024-09-19T04:19:46.583820+0000","last_clean":"2024-09-19T04:19:46.583820+0000","last_became_active":"2024-09-19T04:19:45.202375+0000","last_became_peered":"2024-09-19T04:19:45.202375+0000","last_unstale":"2024-09-19T04:19:46.583820+0000","last_undegraded":"2024-09-19T04:19:46.583820+0000","last_fullsized":"2024-09-19T04:19:46.583820+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:05:38.286645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583167+0000","last_change":"2024-09-19T04:19:44.227834+0000","last_active":"2024-09-19T04:19:46.583167+0000","last_peered":"2024-09-19T04:19:46.583167+0000","last_clean":"2024-09-19T04:19:46.583167+0000","last_became_active":"2024-09-19T04:19:44.227529+0000","last_became_peered":"2024-09-19T04:19:44.227529+0000","last_unstale":"2024-09-19T04:19:46.583167+0000","last_undegraded":"2024-09-19T04:19:46.583167+0000","last_fullsized":"2024-09-19T04:19:46.583167+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:23.454086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.309655+0000","last_change":"2024-09-19T04:19:45.175959+0000","last_active":"2024-09-19T04:19:46.309655+0000","last_peered":"2024-09-19T04:19:46.309655+0000","last_clean":"2024-09-19T04:19:46.309655+0000","last_became_active":"2024-09-19T04:19:45.174859+0000","last_became_peered":"2024-09-19T04:19:45.174859+0000","last_unstale":"2024-09-19T04:19:46.309655+0000","last_undegraded":"2024-09-19T04:19:46.309655+0000","last_fullsized":"2024-09-19T04:19:46.309655+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:41.220850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:47.147797+0000","last_change":"2024-09-19T04:19:44.225348+0000","last_active":"2024-09-19T04:19:47.147797+0000","last_peered":"2024-09-19T04:19:47.147797+0000","last_clean":"2024-09-19T04:19:47.147797+0000","last_became_active":"2024-09-19T04:19:44.224288+0000","last_became_peered":"2024-09-19T04:19:44.224288+0000","last_unstale":"2024-09-19T04:19:47.147797+0000","last_undegraded":"2024-09-19T04:19:47.147797+0000","last_fullsized":"2024-09-19T04:19:47.147797+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:43:30.396563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.144487+0000","last_change":"2024-09-19T04:19:45.163616+0000","last_active":"2024-09-19T04:19:46.144487+0000","last_peered":"2024-09-19T04:19:46.144487+0000","last_clean":"2024-09-19T04:19:46.144487+0000","last_became_active":"2024-09-19T04:19:45.162812+0000","last_became_peered":"2024-09-19T04:19:45.162812+0000","last_unstale":"2024-09-19T04:19:46.144487+0000","last_undegraded":"2024-09-19T04:19:46.144487+0000","last_fullsized":"2024-09-19T04:19:46.144487+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:12:46.935872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583521+0000","last_change":"2024-09-19T04:19:44.227839+0000","last_active":"2024-09-19T04:19:46.583521+0000","last_peered":"2024-09-19T04:19:46.583521+0000","last_clean":"2024-09-19T04:19:46.583521+0000","last_became_active":"2024-09-19T04:19:44.227406+0000","last_became_peered":"2024-09-19T04:19:44.227406+0000","last_unstale":"2024-09-19T04:19:46.583521+0000","last_undegraded":"2024-09-19T04:19:46.583521+0000","last_fullsized":"2024-09-19T04:19:46.583521+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T04:42:10.435772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.143832+0000","last_change":"2024-09-19T04:19:45.172103+0000","last_active":"2024-09-19T04:19:46.143832+0000","last_peered":"2024-09-19T04:19:46.143832+0000","last_clean":"2024-09-19T04:19:46.143832+0000","last_became_active":"2024-09-19T04:19:45.171125+0000","last_became_peered":"2024-09-19T04:19:45.171125+0000","last_unstale":"2024-09-19T04:19:46.143832+0000","last_undegraded":"2024-09-19T04:19:46.143832+0000","last_fullsized":"2024-09-19T04:19:46.143832+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:44.184342+0000","last_clean_scrub_stamp":"2024-09-19T04:19:44.184342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:11:03.510293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"103'2","reported_seq":16,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.583259+0000","last_change":"2024-09-19T04:19:44.216222+0000","last_active":"2024-09-19T04:19:46.583259+0000","last_peered":"2024-09-19T04:19:46.583259+0000","last_clean":"2024-09-19T04:19:46.583259+0000","last_became_active":"2024-09-19T04:19:44.215778+0000","last_became_peered":"2024-09-19T04:19:44.215778+0000","last_unstale":"2024-09-19T04:19:46.583259+0000","last_undegraded":"2024-09-19T04:19:46.583259+0000","last_fullsized":"2024-09-19T04:19:46.583259+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:19:43.180777+0000","last_clean_scrub_stamp":"2024-09-19T04:19:43.180777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:16:30.910768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":323,"reported_epoch":104,"state":"active+clean","last_fresh":"2024-09-19T04:19:46.308651+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:19:46.308651+0000","last_peered":"2024-09-19T04:19:46.308651+0000","last_clean":"2024-09-19T04:19:46.308651+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:19:46.308651+0000","last_undegraded":"2024-09-19T04:19:46.308651+0000","last_fullsized":"2024-09-19T04:19:46.308651+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542226,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33036,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338804,"statfs":{"total":96636764160,"available":96602935296,"internally_reserved":0,"allocated":1888256,"data_stored":1191356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542227,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":596352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542227,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":600828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542226,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32456,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339384,"statfs":{"total":96636764160,"available":96603529216,"internally_reserved":0,"allocated":1294336,"data_stored":600920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542226,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":601410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542227,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":600862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542227,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":594170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542227,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":1192898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:19:49.534 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:19:49.534 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-19T04:19:49.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f136e37c640 1 Processor -- start 2024-09-19T04:19:49.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f136e37c640 1 -- start start 2024-09-19T04:19:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f136e37c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f13680cc870 0x7f13680cf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f136e37c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1368073100 0x7f13680ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f136e37c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13680cd190 con 0x7f13680cdfd0 2024-09-19T04:19:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f136e37c640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f13680ce8d0 con 0x7f1368073100 2024-09-19T04:19:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f136e37c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f13680cebb0 con 0x7f13680cc870 2024-09-19T04:19:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f13677fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1368073100 0x7f13680ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f13677fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1368073100 0x7f13680ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59104/0 (socket says 172.21.15.76:59104) 2024-09-19T04:19:49.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f13677fe640 1 -- 172.21.15.76:0/2800667474 learned_addr learned my addr 172.21.15.76:0/2800667474 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:49.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f1367fff640 1 --2- 172.21.15.76:0/2800667474 >> v2:172.21.15.116:3300/0 conn(0x7f13680cc870 0x7f13680cf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f13677fe640 1 -- 172.21.15.76:0/2800667474 >> v2:172.21.15.116:3300/0 conn(0x7f13680cc870 msgr2=0x7f13680cf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:49.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f13677fe640 1 --2- 172.21.15.76:0/2800667474 >> v2:172.21.15.116:3300/0 conn(0x7f13680cc870 0x7f13680cf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f13677fe640 1 -- 172.21.15.76:0/2800667474 >> v1:172.21.15.76:6789/0 conn(0x7f13680cdfd0 legacy=0x7f136806de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f13677fe640 1 -- 172.21.15.76:0/2800667474 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13680cee90 con 0x7f1368073100 2024-09-19T04:19:49.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.696+0000 7f1366ffd640 1 -- 172.21.15.76:0/2800667474 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1825263972 0 0) 0x7f13680cd190 con 0x7f13680cdfd0 2024-09-19T04:19:49.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f13677fe640 1 --2- 172.21.15.76:0/2800667474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1368073100 0x7f13680ccc50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1368068970 tx=0x7f136806ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3e4677100f81317 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:49.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f1366ffd640 1 -- 172.21.15.76:0/2800667474 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0aa650 con 0x7f1368073100 2024-09-19T04:19:49.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f1366ffd640 1 -- 172.21.15.76:0/2800667474 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f135c0af070 con 0x7f1368073100 2024-09-19T04:19:49.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f1366ffd640 1 -- 172.21.15.76:0/2800667474 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f135c0aaa10 con 0x7f1368073100 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 -- 172.21.15.76:0/2800667474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1368073100 msgr2=0x7f13680ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 --2- 172.21.15.76:0/2800667474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1368073100 0x7f13680ccc50 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f1368068970 tx=0x7f136806ed20 comp rx=0 tx=0).stop 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 -- 172.21.15.76:0/2800667474 shutdown_connections 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 --2- 172.21.15.76:0/2800667474 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1368073100 0x7f13680ccc50 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 --2- 172.21.15.76:0/2800667474 >> v2:172.21.15.116:3300/0 conn(0x7f13680cc870 0x7f13680cf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 -- 172.21.15.76:0/2800667474 >> 172.21.15.76:0/2800667474 conn(0x7f13680b1c80 msgr2=0x7f13680b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 -- 172.21.15.76:0/2800667474 shutdown_connections 2024-09-19T04:19:49.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 -- 172.21.15.76:0/2800667474 wait complete. 2024-09-19T04:19:49.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 Processor -- start 2024-09-19T04:19:49.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.700+0000 7f136e37c640 1 -- start start 2024-09-19T04:19:49.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1368073100 0x7f136815bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f13680cc870 0x7f136809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13680cdfd0 0x7f1368166450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1368169310 con 0x7f13680cdfd0 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1368169190 con 0x7f1368073100 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1368169490 con 0x7f13680cc870 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13680cdfd0 0x7f1368166450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13680cdfd0 0x7f1368166450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59118/0 (socket says 172.21.15.76:59118) 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 -- 172.21.15.76:0/3262645955 learned_addr learned my addr 172.21.15.76:0/3262645955 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1367fff640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1368073100 0x7f136815bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f13677fe640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f13680cc870 0x7f136809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 -- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f13680cc870 msgr2=0x7f136809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f13680cc870 0x7f136809aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 -- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1368073100 msgr2=0x7f136815bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1368073100 0x7f136815bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 -- 172.21.15.76:0/3262645955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136809b4a0 con 0x7f13680cdfd0 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1367fff640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1368073100 0x7f136815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136c919640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13680cdfd0 0x7f1368166450 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f136000c7b0 tx=0x7f136000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1364ff9640 1 -- 172.21.15.76:0/3262645955 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136000ce60 con 0x7f13680cdfd0 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1364ff9640 1 -- 172.21.15.76:0/3262645955 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1360003db0 con 0x7f13680cdfd0 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136809b780 con 0x7f13680cdfd0 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1364ff9640 1 -- 172.21.15.76:0/3262645955 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13600052b0 con 0x7f13680cdfd0 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136809bd40 con 0x7f13680cdfd0 2024-09-19T04:19:49.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13680cf970 con 0x7f13680cdfd0 2024-09-19T04:19:49.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1364ff9640 1 -- 172.21.15.76:0/3262645955 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1360005450 con 0x7f13680cdfd0 2024-09-19T04:19:49.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1364ff9640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13540678c0 0x7f1354069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:49.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1364ff9640 1 -- 172.21.15.76:0/3262645955 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f1360085d00 con 0x7f13680cdfd0 2024-09-19T04:19:49.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.704+0000 7f1367fff640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13540678c0 0x7f1354069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:49.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.708+0000 7f1367fff640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13540678c0 0x7f1354069d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1358009840 tx=0x7f1358008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:49.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.708+0000 7f1364ff9640 1 -- 172.21.15.76:0/3262645955 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f13600566b0 con 0x7f13680cdfd0 2024-09-19T04:19:49.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.912+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f1368143210 con 0x7f13680cdfd0 2024-09-19T04:19:49.918 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.916+0000 7f1364ff9640 1 -- 172.21.15.76:0/3262645955 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f1360055fb0 con 0x7f13680cdfd0 2024-09-19T04:19:49.918 INFO:teuthology.orchestra.run.smithi076.stdout:[client.0] 2024-09-19T04:19:49.918 INFO:teuthology.orchestra.run.smithi076.stdout: key = AQCppOtm1+haIxAApKTenytS0Lkqqa0YEcUQGg== 2024-09-19T04:19:49.919 INFO:teuthology.orchestra.run.smithi076.stdout: caps mds = "allow" 2024-09-19T04:19:49.919 INFO:teuthology.orchestra.run.smithi076.stdout: caps mon = "allow r" 2024-09-19T04:19:49.919 INFO:teuthology.orchestra.run.smithi076.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-19T04:19:49.919 INFO:teuthology.orchestra.run.smithi076.stderr:updated caps for client.0 2024-09-19T04:19:49.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13540678c0 msgr2=0x7f1354069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13540678c0 0x7f1354069d80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f1358009840 tx=0x7f1358008040 comp rx=0 tx=0).stop 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13680cdfd0 msgr2=0x7f1368166450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13680cdfd0 0x7f1368166450 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f136000c7b0 tx=0x7f136000cc70 comp rx=0 tx=0).stop 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 shutdown_connections 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f13540678c0 0x7f1354069d80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f13680cdfd0 0x7f1368166450 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f13680cc870 0x7f136809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 --2- 172.21.15.76:0/3262645955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1368073100 0x7f136815bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 >> 172.21.15.76:0/3262645955 conn(0x7f13680b1c80 msgr2=0x7f13680d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 shutdown_connections 2024-09-19T04:19:49.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:49.920+0000 7f136e37c640 1 -- 172.21.15.76:0/3262645955 wait complete. 2024-09-19T04:19:49.936 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:50.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.052+0000 7fcaf3925640 1 Processor -- start 2024-09-19T04:19:50.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.052+0000 7fcaf3925640 1 -- start start 2024-09-19T04:19:50.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf3925640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcaec15c2a0 0x7fcaec0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf3925640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fcaec0af2a0 0x7fcaec06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf3925640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcaec0b3290 con 0x7fcaec15bec0 2024-09-19T04:19:50.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf3925640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0b3570 con 0x7fcaec0af2a0 2024-09-19T04:19:50.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf3925640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcaec0b3850 con 0x7fcaec15c2a0 2024-09-19T04:19:50.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf16c1640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcaec15bec0 0x7fcaec0ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54976/0 (socket says 172.21.15.76:54976) 2024-09-19T04:19:50.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf16c1640 1 -- 172.21.15.76:0/936077223 learned_addr learned my addr 172.21.15.76:0/936077223 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf1ec2640 1 --2- 172.21.15.76:0/936077223 >> v2:172.21.15.76:3300/0 conn(0x7fcaec0af2a0 0x7fcaec06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf0ec0640 1 --2- 172.21.15.76:0/936077223 >> v2:172.21.15.116:3300/0 conn(0x7fcaec15c2a0 0x7fcaec0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcae3fff640 1 -- 172.21.15.76:0/936077223 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2360389579 0 0) 0x7fcaec0b3290 con 0x7fcaec15bec0 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcae3fff640 1 -- 172.21.15.76:0/936077223 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcad8003530 con 0x7fcaec15bec0 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf1ec2640 1 -- 172.21.15.76:0/936077223 >> v2:172.21.15.116:3300/0 conn(0x7fcaec15c2a0 msgr2=0x7fcaec0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf1ec2640 1 --2- 172.21.15.76:0/936077223 >> v2:172.21.15.116:3300/0 conn(0x7fcaec15c2a0 0x7fcaec0aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf1ec2640 1 -- 172.21.15.76:0/936077223 >> v1:172.21.15.76:6789/0 conn(0x7fcaec15bec0 legacy=0x7fcaec0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.058 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf1ec2640 1 -- 172.21.15.76:0/936077223 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec0b3b30 con 0x7fcaec0af2a0 2024-09-19T04:19:50.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcaf1ec2640 1 --2- 172.21.15.76:0/936077223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec0af2a0 0x7fcaec06e000 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ae510 tx=0x7fcae401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6549f2731e729e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcae3fff640 1 -- 172.21.15.76:0/936077223 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40b2070 con 0x7fcaec0af2a0 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcae3fff640 1 -- 172.21.15.76:0/936077223 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcae40a59c0 con 0x7fcaec0af2a0 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.056+0000 7fcae3fff640 1 -- 172.21.15.76:0/936077223 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae40a5d70 con 0x7fcaec0af2a0 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- 172.21.15.76:0/936077223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec0af2a0 msgr2=0x7fcaec06e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 --2- 172.21.15.76:0/936077223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec0af2a0 0x7fcaec06e000 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fcaec0ae510 tx=0x7fcae401c900 comp rx=0 tx=0).stop 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- 172.21.15.76:0/936077223 shutdown_connections 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 --2- 172.21.15.76:0/936077223 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec0af2a0 0x7fcaec06e000 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 --2- 172.21.15.76:0/936077223 >> v2:172.21.15.116:3300/0 conn(0x7fcaec15c2a0 0x7fcaec0aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- 172.21.15.76:0/936077223 >> 172.21.15.76:0/936077223 conn(0x7fcaec0b1f90 msgr2=0x7fcaec0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- 172.21.15.76:0/936077223 shutdown_connections 2024-09-19T04:19:50.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- 172.21.15.76:0/936077223 wait complete. 2024-09-19T04:19:50.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 Processor -- start 2024-09-19T04:19:50.061 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- start start 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcaec0af2a0 0x7fcaec15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec15bec0 0x7fcaec156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcaec15c2a0 0x7fcaec156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec06ed50 con 0x7fcaec15bec0 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcaec0af710 con 0x7fcaec0af2a0 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcaec06eed0 con 0x7fcaec15c2a0 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec15bec0 0x7fcaec156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec15bec0 0x7fcaec156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59124/0 (socket says 172.21.15.76:59124) 2024-09-19T04:19:50.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 -- 172.21.15.76:0/2899676594 learned_addr learned my addr 172.21.15.76:0/2899676594 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf16c1640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcaec0af2a0 0x7fcaec15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 -- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcaec15c2a0 msgr2=0x7fcaec156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcaec15c2a0 0x7fcaec156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 -- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcaec0af2a0 msgr2=0x7fcaec15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcaec0af2a0 0x7fcaec15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 -- 172.21.15.76:0/2899676594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaec153280 con 0x7fcaec15bec0 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf16c1640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcaec0af2a0 0x7fcaec15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf0ec0640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec15bec0 0x7fcaec156a20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006e90 tx=0x7fcadc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcae1ffb640 1 -- 172.21.15.76:0/2899676594 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00b6c0 con 0x7fcaec15bec0 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcae1ffb640 1 -- 172.21.15.76:0/2899676594 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcadc019070 con 0x7fcaec15bec0 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaec153560 con 0x7fcaec15bec0 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcae1ffb640 1 -- 172.21.15.76:0/2899676594 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc00bac0 con 0x7fcaec15bec0 2024-09-19T04:19:50.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.060+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaec155f40 con 0x7fcaec15bec0 2024-09-19T04:19:50.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.064+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaec06f260 con 0x7fcaec15bec0 2024-09-19T04:19:50.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.064+0000 7fcae1ffb640 1 -- 172.21.15.76:0/2899676594 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcadc00bc60 con 0x7fcaec15bec0 2024-09-19T04:19:50.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.064+0000 7fcae1ffb640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcad00678c0 0x7fcad0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.064+0000 7fcae1ffb640 1 -- 172.21.15.76:0/2899676594 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fcadc0861f0 con 0x7fcaec15bec0 2024-09-19T04:19:50.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.064+0000 7fcaf16c1640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcad00678c0 0x7fcad0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.064+0000 7fcaf16c1640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcad00678c0 0x7fcad0069d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005890 tx=0x7fcae8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.068+0000 7fcae1ffb640 1 -- 172.21.15.76:0/2899676594 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcadc056c30 con 0x7fcaec15bec0 2024-09-19T04:19:50.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaec06fcd0 con 0x7fcaec15bec0 2024-09-19T04:19:50.268 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcae1ffb640 1 -- 172.21.15.76:0/2899676594 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5230 (secure 0 0 0) 0x7fcadc056530 con 0x7fcaec15bec0 2024-09-19T04:19:50.268 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:50.268 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":102,"btime":"2024-09-19T04:19:48:158361+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6127,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8115,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:19:47.518980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127]},"id":7}]} 2024-09-19T04:19:50.269 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 102 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcad00678c0 msgr2=0x7fcad0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcad00678c0 0x7fcad0069d80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005890 tx=0x7fcae8009040 comp rx=0 tx=0).stop 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec15bec0 msgr2=0x7fcaec156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec15bec0 0x7fcaec156a20 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006e90 tx=0x7fcadc00dc60 comp rx=0 tx=0).stop 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 shutdown_connections 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcad00678c0 0x7fcad0069d80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcaec15c2a0 0x7fcaec156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcaec15bec0 0x7fcaec156a20 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 --2- 172.21.15.76:0/2899676594 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcaec0af2a0 0x7fcaec15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 >> 172.21.15.76:0/2899676594 conn(0x7fcaec0b1f90 msgr2=0x7fcaec0b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 shutdown_connections 2024-09-19T04:19:50.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.268+0000 7fcaf3925640 1 -- 172.21.15.76:0/2899676594 wait complete. 2024-09-19T04:19:50.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:19:44.552866+0000', 'modified': '2024-09-19T04:19:47.518980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8127': {'gid': 8127, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3881707282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3881707282}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3881707282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8127, 'qdb_cluster': [8127]} 2024-09-19T04:19:50.691 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-19T04:19:50.692 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:19:50.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14ad043640 1 Processor -- start 2024-09-19T04:19:50.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14ad043640 1 -- start start 2024-09-19T04:19:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14ad043640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14a8154d60 0x7f14a80b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14ad043640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a806ccd0 0x7f14a806d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14ad043640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14a8155140 con 0x7f14a80cd500 2024-09-19T04:19:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14ad043640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f14a815bcc0 con 0x7f14a8154d60 2024-09-19T04:19:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14ad043640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f14a806d660 con 0x7f14a806ccd0 2024-09-19T04:19:50.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a6d9d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f14a8154d60 0x7f14a80b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a659c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f14a80cd500 0x7f14a815b5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:54992/0 (socket says 172.21.15.76:54992) 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a659c640 1 -- 172.21.15.76:0/2793344540 learned_addr learned my addr 172.21.15.76:0/2793344540 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a5d9b640 1 -- 172.21.15.76:0/2793344540 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3294815100 0 0) 0x7f14a8155140 con 0x7f14a80cd500 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a759e640 1 --2- 172.21.15.76:0/2793344540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a806ccd0 0x7f14a806d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a5d9b640 1 -- 172.21.15.76:0/2793344540 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1490003530 con 0x7f14a80cd500 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a6d9d640 1 -- 172.21.15.76:0/2793344540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a806ccd0 msgr2=0x7f14a806d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2793344540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a806ccd0 0x7f14a806d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a6d9d640 1 -- 172.21.15.76:0/2793344540 >> v1:172.21.15.76:6789/0 conn(0x7f14a80cd500 legacy=0x7f14a815b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a6d9d640 1 -- 172.21.15.76:0/2793344540 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a80b4960 con 0x7f14a8154d60 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.820+0000 7f14a759e640 1 --2- 172.21.15.76:0/2793344540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a806ccd0 0x7f14a806d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2793344540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a8154d60 0x7f14a80b3aa0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f14980097c0 tx=0x7f149801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba0bd455d5162ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14a5d9b640 1 -- 172.21.15.76:0/2793344540 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980b0070 con 0x7f14a8154d60 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14a5d9b640 1 -- 172.21.15.76:0/2793344540 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980ab400 con 0x7f14a8154d60 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14a5d9b640 1 -- 172.21.15.76:0/2793344540 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980ab7d0 con 0x7f14a8154d60 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 -- 172.21.15.76:0/2793344540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a8154d60 msgr2=0x7f14a80b3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 --2- 172.21.15.76:0/2793344540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a8154d60 0x7f14a80b3aa0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f14980097c0 tx=0x7f149801c410 comp rx=0 tx=0).stop 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 -- 172.21.15.76:0/2793344540 shutdown_connections 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 --2- 172.21.15.76:0/2793344540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a806ccd0 0x7f14a806d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 --2- 172.21.15.76:0/2793344540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a8154d60 0x7f14a80b3aa0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 -- 172.21.15.76:0/2793344540 >> 172.21.15.76:0/2793344540 conn(0x7f14a8072670 msgr2=0x7f14a8072a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 -- 172.21.15.76:0/2793344540 shutdown_connections 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 -- 172.21.15.76:0/2793344540 wait complete. 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 Processor -- start 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.824+0000 7f14ad043640 1 -- start start 2024-09-19T04:19:50.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14a806ccd0 0x7f14a813f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a80cd500 0x7f14a8167880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a8154d60 0x7f14a8167dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f14a80b4160 con 0x7f14a80cd500 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f14a80b3fe0 con 0x7f14a806ccd0 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f14a80cecd0 con 0x7f14a8154d60 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a80cd500 0x7f14a8167880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a80cd500 0x7f14a8167880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59142/0 (socket says 172.21.15.76:59142) 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 -- 172.21.15.76:0/2418100075 learned_addr learned my addr 172.21.15.76:0/2418100075 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a759e640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a8154d60 0x7f14a8167dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14a806ccd0 0x7f14a813f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 -- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a8154d60 msgr2=0x7f14a8167dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a8154d60 0x7f14a8167dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 -- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14a806ccd0 msgr2=0x7f14a813f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14a806ccd0 0x7f14a813f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 -- 172.21.15.76:0/2418100075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14a814dcf0 con 0x7f14a80cd500 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a659c640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a80cd500 0x7f14a8167880 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f149c00c450 tx=0x7f149c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f148f7fe640 1 -- 172.21.15.76:0/2418100075 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c017070 con 0x7f14a80cd500 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f148f7fe640 1 -- 172.21.15.76:0/2418100075 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c005cf0 con 0x7f14a80cd500 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14a814dfd0 con 0x7f14a80cd500 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f148f7fe640 1 -- 172.21.15.76:0/2418100075 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c007290 con 0x7f14a80cd500 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14a814e590 con 0x7f14a80cd500 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f148f7fe640 1 -- 172.21.15.76:0/2418100075 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f149c01a070 con 0x7f14a80cd500 2024-09-19T04:19:50.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f148f7fe640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14940678c0 0x7f1494069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f148f7fe640 1 -- 172.21.15.76:0/2418100075 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f149c0851e0 con 0x7f14a80cd500 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14940678c0 0x7f1494069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.828+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14940678c0 0x7f1494069d80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f14980ad000 tx=0x7f1498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 Processor -- start 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- start start 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee855ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee8514d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee8515290 0x564ee8515670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564ee83fcb60 con 0x564ee855f440 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564ee8363c70 con 0x564ee8515290 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x564ee8309ed0 con 0x564ee855eb20 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee8514d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee8514d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59158/0 (socket says 172.21.15.76:59158) 2024-09-19T04:19:50.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 -- 172.21.15.76:0/176998400 learned_addr learned my addr 172.21.15.76:0/176998400 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a759e640 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee8515290 0x564ee8515670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee855ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 -- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 msgr2=0x564ee855ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee855ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 -- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee8515290 msgr2=0x564ee8515670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee8515290 0x564ee8515670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 -- 172.21.15.76:0/176998400 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee831be50 con 0x564ee855f440 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a759e640 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee8515290 0x564ee8515670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14a659c640 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee8514d50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f149c05db30 tx=0x7f149c055540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7549363bddec21e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f148cff9640 1 -- 172.21.15.76:0/176998400 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c053780 con 0x564ee855f440 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f148cff9640 1 -- 172.21.15.76:0/176998400 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c051af0 con 0x564ee855f440 2024-09-19T04:19:50.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f148cff9640 1 -- 172.21.15.76:0/176998400 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c05de10 con 0x564ee855f440 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- 172.21.15.76:0/176998400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 msgr2=0x564ee8514d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee8514d50 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f149c05db30 tx=0x7f149c055540 comp rx=0 tx=0).stop 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- 172.21.15.76:0/176998400 shutdown_connections 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee8515290 0x564ee8515670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee8514d50 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- 172.21.15.76:0/176998400 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee855ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- 172.21.15.76:0/176998400 >> 172.21.15.76:0/176998400 conn(0x564ee851b600 msgr2=0x564ee8513630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- 172.21.15.76:0/176998400 shutdown_connections 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- 172.21.15.76:0/176998400 wait complete. 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- start start 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 Processor -- start 2024-09-19T04:19:50.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8519e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee851fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee8520230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564ee83c5b00 con 0x564ee855f440 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564ee833e360 con 0x564ee851a390 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x564ee833b2d0 con 0x564ee855eb20 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.832+0000 7f14b00ff000 10 client.0 _ll_register_callbacks cb 0x564ee8500820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee851fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee851fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59168/0 (socket says 172.21.15.76:59168) 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 -- 172.21.15.76:0/605544774 learned_addr learned my addr 172.21.15.76:0/605544774 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a759e640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee8520230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8519e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 -- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 msgr2=0x564ee8519e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8519e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 -- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 msgr2=0x564ee8520230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee8520230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 -- 172.21.15.76:0/605544774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee830a120 con 0x564ee855f440 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8519e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a759e640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee8520230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a659c640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee851fcf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f149c055d30 tx=0x7f149c000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae88e593010f55b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 1 -- 172.21.15.76:0/605544774 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f149c072060 con 0x564ee855f440 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 1 -- 172.21.15.76:0/605544774 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f149c073a40 con 0x564ee855f440 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f149c051af0 con 0x564ee855f440 2024-09-19T04:19:50.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 1 -- 172.21.15.76:0/605544774 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f149c078050 con 0x564ee855f440 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 10 client.8265 fetch_fsmap learned FSMap version 102 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x564ee8303450 con 0x564ee855f440 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 1 -- 172.21.15.76:0/605544774 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f149c051af0 con 0x564ee855f440 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 1 -- 172.21.15.76:0/605544774 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7f149c07b020 con 0x564ee855f440 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 10 client.8265 fetch_fsmap finished waiting for FSMap version 102 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 10 client.8265 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 20 client.8265 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee856d280 0x564ee856f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 4 client.8265 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:50.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ee82fdc00 con 0x564ee856d280 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee856d280 0x564ee856f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 10 client.8265 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee856d280 0x564ee856f670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f9d4fafe341e405 server_cookie=d792e73ffcd0061d in_seq=0 out_seq=0 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 1 -- 172.21.15.76:0/605544774 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564ee82fdc00 con 0x564ee856d280 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 10 client.8265 handle_command_reply: tid=0 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 client.8265 shutdown 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 msgr2=0x564ee851fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee851fcf0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f149c055d30 tx=0x7f149c000bd0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 shutdown_connections 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee856d280 0x564ee856f670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee8520230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee851fcf0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 --2- 172.21.15.76:0/605544774 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8519e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f1485ffb640 0 client.8265 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:50.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 >> 172.21.15.76:0/605544774 conn(0x564ee8515290 msgr2=0x564ee8513d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 shutdown_connections 2024-09-19T04:19:50.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 1 -- 172.21.15.76:0/605544774 wait complete. 2024-09-19T04:19:50.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.836+0000 7f14b00ff000 20 client.8265 trim_cache size 0 max 16384 2024-09-19T04:19:50.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 Processor -- start 2024-09-19T04:19:50.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- start start 2024-09-19T04:19:50.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee851a390 0x564ee855dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee855e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee855f440 0x564ee856f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f14a8202c40 con 0x564ee851a390 2024-09-19T04:19:50.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f14a81f77f0 con 0x564ee855f440 2024-09-19T04:19:50.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f14a82025f0 con 0x564ee855eb20 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee851a390 0x564ee855dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee851a390 0x564ee855dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59176/0 (socket says 172.21.15.76:59176) 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 -- 172.21.15.76:0/2808526118 learned_addr learned my addr 172.21.15.76:0/2808526118 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a759e640 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee855f440 0x564ee856f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 -- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 msgr2=0x564ee855e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee855e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 -- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee855f440 msgr2=0x564ee856f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee855f440 0x564ee856f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 -- 172.21.15.76:0/2808526118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ee855f270 con 0x564ee851a390 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a759e640 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee855f440 0x564ee856f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee851a390 0x564ee855dc90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1498009580 tx=0x7f14980a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e4835b41494a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f1487fff640 1 -- 172.21.15.76:0/2808526118 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980b0040 con 0x564ee851a390 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f1487fff640 1 -- 172.21.15.76:0/2808526118 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980b6470 con 0x564ee851a390 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f1487fff640 1 -- 172.21.15.76:0/2808526118 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14980b6820 con 0x564ee851a390 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- 172.21.15.76:0/2808526118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee851a390 msgr2=0x564ee855dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee851a390 0x564ee855dc90 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f1498009580 tx=0x7f14980a6000 comp rx=0 tx=0).stop 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- 172.21.15.76:0/2808526118 shutdown_connections 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee855f440 0x564ee856f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee855e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 --2- 172.21.15.76:0/2808526118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee851a390 0x564ee855dc90 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- 172.21.15.76:0/2808526118 >> 172.21.15.76:0/2808526118 conn(0x564ee8515290 msgr2=0x564ee8519650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- 172.21.15.76:0/2808526118 shutdown_connections 2024-09-19T04:19:50.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- 172.21.15.76:0/2808526118 wait complete. 2024-09-19T04:19:50.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.856+0000 7f14b00ff000 1 -- start start 2024-09-19T04:19:50.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 Processor -- start 2024-09-19T04:19:50.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee85728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8572e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee858d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x564ee8362f80 con 0x564ee855f440 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x564ee831bca0 con 0x564ee851a390 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x564ee83682a0 con 0x564ee855eb20 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 10 client.0 _ll_register_callbacks cb 0x7f149c06c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee858d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee858d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59190/0 (socket says 172.21.15.76:59190) 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 -- 172.21.15.76:0/1207977150 learned_addr learned my addr 172.21.15.76:0/1207977150 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee85728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a659c640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8572e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 -- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 msgr2=0x564ee8572e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8572e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 -- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 msgr2=0x564ee85728c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee85728c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 -- 172.21.15.76:0/1207977150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ee82eafd0 con 0x564ee855f440 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a659c640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8572e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a759e640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee858d960 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f14a000fb70 tx=0x7f14a000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a552f482bd1c2bfe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 1 -- 172.21.15.76:0/1207977150 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a001d060 con 0x564ee855f440 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 1 -- 172.21.15.76:0/1207977150 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f14a000d5e0 con 0x564ee855f440 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14980b6470 con 0x564ee855f440 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 1 -- 172.21.15.76:0/1207977150 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f14a0023050 con 0x564ee855f440 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 10 client.8271 fetch_fsmap learned FSMap version 102 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x564ee83e6750 con 0x564ee855f440 2024-09-19T04:19:50.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 1 -- 172.21.15.76:0/1207977150 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14980b6470 con 0x564ee855f440 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 1 -- 172.21.15.76:0/1207977150 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7f14a0026020 con 0x564ee855f440 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 10 client.8271 fetch_fsmap finished waiting for FSMap version 102 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 10 client.8271 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 20 client.8271 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee85d1e10 0x564ee85d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 4 client.8271 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "status"}) -- 0x564ee83713a0 con 0x564ee85d1e10 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee85d1e10 0x564ee85d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14a6d9d640 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee85d1e10 0x564ee85d21f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61810da7b94c552d server_cookie=6fa8a32b92412e8 in_seq=0 out_seq=0 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 10 client.8271 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 1 -- 172.21.15.76:0/1207977150 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x564ee83713a0 con 0x564ee85d1e10 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 10 client.8271 handle_command_reply: tid=0 2024-09-19T04:19:50.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 client.8271 shutdown 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 msgr2=0x564ee858d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee858d960 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f14a000fb70 tx=0x7f14a000c7f0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 shutdown_connections 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x564ee85d1e10 0x564ee85d21f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x564ee855f440 0x564ee858d960 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x564ee855eb20 0x564ee8572e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f1485ffb640 0 client.8271 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 --2- 172.21.15.76:0/1207977150 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x564ee851a390 0x564ee85728c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 >> 172.21.15.76:0/1207977150 conn(0x564ee8515290 msgr2=0x564ee855cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 shutdown_connections 2024-09-19T04:19:50.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 1 -- 172.21.15.76:0/1207977150 wait complete. 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14b00ff000 20 client.8271 trim_cache size 0 max 16384 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8127, 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:19:50.870 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 5.6657926359999999, 2024-09-19T04:19:50.871 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 102, 2024-09-19T04:19:50.871 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 104, 2024-09-19T04:19:50.871 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 104, 2024-09-19T04:19:50.871 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 9.7428709740000006 2024-09-19T04:19:50.871 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14940678c0 msgr2=0x7f1494069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14940678c0 0x7f1494069d80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f14980ad000 tx=0x7f1498008040 comp rx=0 tx=0).stop 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a80cd500 msgr2=0x7f14a8167880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a80cd500 0x7f14a8167880 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f149c00c450 tx=0x7f149c00c910 comp rx=0 tx=0).stop 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 shutdown_connections 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f14940678c0 0x7f1494069d80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f14a8154d60 0x7f14a8167dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f14a80cd500 0x7f14a8167880 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 --2- 172.21.15.76:0/2418100075 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f14a806ccd0 0x7f14a813f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 >> 172.21.15.76:0/2418100075 conn(0x7f14a8072670 msgr2=0x7f14a80cff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:50.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 shutdown_connections 2024-09-19T04:19:50.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.860+0000 7f14ad043640 1 -- 172.21.15.76:0/2418100075 wait complete. 2024-09-19T04:19:50.879 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-19T04:19:50.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452c5e5640 1 Processor -- start 2024-09-19T04:19:50.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452c5e5640 1 -- start start 2024-09-19T04:19:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452c5e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 0x7f45240cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452c5e5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45240d0300 con 0x7f4524161a60 2024-09-19T04:19:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452c5e5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45240b0f60 con 0x7f45240cf450 2024-09-19T04:19:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452c5e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f45240cfd40 con 0x7f45240d3060 2024-09-19T04:19:50.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452a381640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f4524161a60 0x7f45240d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:55000/0 (socket says 172.21.15.76:55000) 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.988+0000 7f452a381640 1 -- 172.21.15.76:0/1700837693 learned_addr learned my addr 172.21.15.76:0/1700837693 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2623749083 0 0) 0x7f4520001120 con 0x7f4524161a60 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4510003530 con 0x7f4524161a60 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 0x7f45240cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3472159677 0 0) 0x7f4510003530 con 0x7f4524161a60 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 msgr2=0x7f45240cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 0x7f45240cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 >> v1:172.21.15.116:6789/0 conn(0x7f45240cf450 legacy=0x7f452406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45240d0020 con 0x7f4524161a60 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f4520001850 con 0x7f4524161a60 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 >> v1:172.21.15.76:6789/0 conn(0x7f4524161a60 legacy=0x7f45240d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45100039f0 0x7f4510003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4510004810 0x7f4510004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 0x7f451000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f45240cfd40 con 0x7f4510004810 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4510005530 con 0x7f45100039f0 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f451000bf10 con 0x7f45240d3060 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4510004810 0x7f4510004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452a381640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45100039f0 0x7f4510003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452ab82640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 0x7f451000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 -- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 msgr2=0x7f451000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f45240d3060 0x7f451000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 -- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45100039f0 msgr2=0x7f4510003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45100039f0 0x7f4510003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:50.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 -- 172.21.15.76:0/1700837693 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4510003530 con 0x7f4510004810 2024-09-19T04:19:51.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f4529b80640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4510004810 0x7f4510004c90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f451400ba30 tx=0x7f451401c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d265c45ee4eac8c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451400b600 con 0x7f4510004810 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45140b0070 con 0x7f4510004810 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452937f640 1 -- 172.21.15.76:0/1700837693 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45140ab8a0 con 0x7f4510004810 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452c5e5640 1 -- 172.21.15.76:0/1700837693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4510004810 msgr2=0x7f4510004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452c5e5640 1 --2- 172.21.15.76:0/1700837693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4510004810 0x7f4510004c90 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f451400ba30 tx=0x7f451401c670 comp rx=0 tx=0).stop 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452ab82640 1 -- 172.21.15.76:0/1700837693 reap_dead start 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452c5e5640 1 -- 172.21.15.76:0/1700837693 shutdown_connections 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.992+0000 7f452c5e5640 1 -- 172.21.15.76:0/1700837693 >> 172.21.15.76:0/1700837693 conn(0x7f452408bf90 msgr2=0x7f452409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- 172.21.15.76:0/1700837693 shutdown_connections 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- 172.21.15.76:0/1700837693 wait complete. 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 Processor -- start 2024-09-19T04:19:51.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- start start 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4510004810 0x7f452415b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4524162fa0 0x7f4524161f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524162460 0x7f45241628e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f452416a0a0 con 0x7f4524162460 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4524169f20 con 0x7f4510004810 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f452416a220 con 0x7f4524162fa0 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524162460 0x7f45241628e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524162460 0x7f45241628e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:59210/0 (socket says 172.21.15.76:59210) 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 -- 172.21.15.76:0/2716817401 learned_addr learned my addr 172.21.15.76:0/2716817401 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f4529b80640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4524162fa0 0x7f4524161f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452a381640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4510004810 0x7f452415b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 -- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4524162fa0 msgr2=0x7f4524161f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4524162fa0 0x7f4524161f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:51.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 -- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4510004810 msgr2=0x7f452415b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4510004810 0x7f452415b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 -- 172.21.15.76:0/2716817401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45240b96a0 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f4529b80640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4524162fa0 0x7f4524161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452a381640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4510004810 0x7f452415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452ab82640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524162460 0x7f45241628e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f45240b6f60 tx=0x7f451c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f451affd640 1 -- 172.21.15.76:0/2716817401 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c003980 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f451affd640 1 -- 172.21.15.76:0/2716817401 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f451c018070 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45240b9980 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f451affd640 1 -- 172.21.15.76:0/2716817401 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f451c003d60 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45240b9f40 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:50.996+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452406d6b0 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.000+0000 7f451affd640 1 -- 172.21.15.76:0/2716817401 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f451c0050d0 con 0x7f4524162460 2024-09-19T04:19:51.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.000+0000 7f451affd640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45080678f0 0x7f4508069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.000+0000 7f452a381640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45080678f0 0x7f4508069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.000+0000 7f451affd640 1 -- 172.21.15.76:0/2716817401 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f451c085aa0 con 0x7f4524162460 2024-09-19T04:19:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.000+0000 7f452a381640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45080678f0 0x7f4508069db0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45200012c0 tx=0x7f4520009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:51.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.004+0000 7f451affd640 1 -- 172.21.15.76:0/2716817401 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f451c0564e0 con 0x7f4524162460 2024-09-19T04:19:51.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.188+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f45240d1650 con 0x7f4524162460 2024-09-19T04:19:51.191 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f451affd640 1 -- 172.21.15.76:0/2716817401 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7f451c055de0 con 0x7f4524162460 2024-09-19T04:19:51.191 INFO:teuthology.orchestra.run.smithi076.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T04:19:51.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45080678f0 msgr2=0x7f4508069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:51.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45080678f0 0x7f4508069db0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f45200012c0 tx=0x7f4520009040 comp rx=0 tx=0).stop 2024-09-19T04:19:51.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524162460 msgr2=0x7f45241628e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:51.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524162460 0x7f45241628e0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f45240b6f60 tx=0x7f451c00ec10 comp rx=0 tx=0).stop 2024-09-19T04:19:51.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 shutdown_connections 2024-09-19T04:19:51.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45080678f0 0x7f4508069db0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:51.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4524162460 0x7f45241628e0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:51.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4524162fa0 0x7f4524161f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:51.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 --2- 172.21.15.76:0/2716817401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4510004810 0x7f452415b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:51.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 >> 172.21.15.76:0/2716817401 conn(0x7f452408bf90 msgr2=0x7f45240d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:51.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 shutdown_connections 2024-09-19T04:19:51.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51.192+0000 7f452c5e5640 1 -- 172.21.15.76:0/2716817401 wait complete. 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi076.front.sepia.ceph.com 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T04:19:51.209 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:19:51.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:51.209 DEBUG:teuthology.orchestra.run.smithi076:> ip addr 2024-09-19T04:19:51.214 INFO:teuthology.orchestra.run.smithi076.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T04:19:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T04:19:51.214 INFO:teuthology.orchestra.run.smithi076.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 ::1/128 scope host 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:66 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f0 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:d8:da:67 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: altname enp7s0f1 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e0 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T04:19:51.215 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0c:c4:7a:8f:47:e1 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.21.15.76/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::ec4:7aff:fe8f:47e1/64 scope link 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 02:42:8b:17:fd:c3 brd ff:ff:ff:ff:ff:ff 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout:9: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.216 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout:10: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout:11: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run.smithi076.stdout: valid_lft forever preferred_lft forever 2024-09-19T04:19:51.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:51.218 DEBUG:teuthology.orchestra.run.smithi076:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T04:19:51.265 INFO:teuthology.orchestra.run.smithi076.stdout:1 2024-09-19T04:19:51.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:51.267 DEBUG:teuthology.orchestra.run.smithi076:> route 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:Kernel IP routing table 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-19T04:19:51.319 INFO:teuthology.orchestra.run.smithi076.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T04:19:51.320 DEBUG:teuthology.orchestra.run.smithi076:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T04:19:51.320 DEBUG:teuthology.orchestra.run.smithi076:> set -e 2024-09-19T04:19:51.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-19T04:19:51.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-19T04:19:51.320 DEBUG:teuthology.orchestra.run.smithi076:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-19T04:19:51.320 DEBUG:teuthology.orchestra.run.smithi076:> ') 2024-09-19T04:19:51.406 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T04:19:51.432 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:51.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:51.439 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:19:51.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T04:19:51.492 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:19:51.492 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:19:51.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:51.492 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link set brx.0 up 2024-09-19T04:19:51.548 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-19T04:19:51.548 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:19:51.548 DEBUG:teuthology.orchestra.run.smithi076:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:51.596 INFO:teuthology.orchestra.run.smithi076.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T04:19:51.596 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T04:19:51.596 DEBUG:teuthology.orchestra.run.smithi076:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:51.644 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:19:51.701 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:19:51.749 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:19:51.749 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:19:51.749 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:19:51.749 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:19:51.749 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:19:51.749 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:19:51.749 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:19:51.750 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:19:51.751 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:19:51.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:51.752 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:19:51.801 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-19T04:19:51.801 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-19T04:19:51.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:19:51.859 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:19:51.909 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:19:51.910 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:19:51.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:51.911 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:19:51.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:2024-09-19T04:19:51.924+0000 7f1c721c4580 -1 init, newargv = 0x5582727d1260 newargc=15 2024-09-19T04:19:51.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[49271]: starting ceph client 2024-09-19T04:19:51.932 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[49271]: starting fuse 2024-09-19T04:19:53.031 DEBUG:teuthology.orchestra.run.smithi076:> sudo modprobe fuse 2024-09-19T04:19:53.046 DEBUG:teuthology.orchestra.run.smithi076:> cat /proc/self/mounts | awk '{print $2}' 2024-09-19T04:19:53.093 INFO:teuthology.orchestra.run.smithi076.stdout:/sys 2024-09-19T04:19:53.093 INFO:teuthology.orchestra.run.smithi076.stdout:/proc 2024-09-19T04:19:53.093 INFO:teuthology.orchestra.run.smithi076.stdout:/dev 2024-09-19T04:19:53.093 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/pts 2024-09-19T04:19:53.093 INFO:teuthology.orchestra.run.smithi076.stdout:/run 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/ 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/security 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/shm 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/run/lock 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/cgroup 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/pstore 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/bpf 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/sys/fs/binfmt_misc 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/hugepages 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/dev/mqueue 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/debug 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/tracing 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/proc/fs/nfsd 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/fs/fuse/connections 2024-09-19T04:19:53.094 INFO:teuthology.orchestra.run.smithi076.stdout:/sys/kernel/config 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/credentials/systemd-sysusers.service 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/rpc_pipefs 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/user/1000 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-0 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-1 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-2 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/var/lib/ceph/osd/ceph-3 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:19:53.095 INFO:teuthology.orchestra.run.smithi076.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:19:53.096 INFO:teuthology.orchestra.run.smithi076.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:53.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:53.096 DEBUG:teuthology.orchestra.run.smithi076:> ls /sys/fs/fuse/connections 2024-09-19T04:19:53.144 INFO:teuthology.orchestra.run.smithi076.stdout:45 2024-09-19T04:19:53.144 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> sudo stdin-killer -- python3 -c ' 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> import glob 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> import re 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> import subprocess 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> def _find_admin_socket(client_name): 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> files = glob.glob(asok_path) 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> # Given a non-glob path, it better be there 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> if "*" not in asok_path: 2024-09-19T04:19:53.145 DEBUG:teuthology.orchestra.run.smithi076:> assert(len(files) == 1) 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> return files[0] 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> for f in files: 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> contents = proc_f.read() 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> if mountpoint in contents: 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> return f 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> print(_find_admin_socket("client.0")) 2024-09-19T04:19:53.146 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:19:53.266 INFO:teuthology.orchestra.run.smithi076.stdout:/var/run/ceph/ceph-client.0.49271.asok 2024-09-19T04:19:53.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:19:53.277 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.49271.asok 2024-09-19T04:19:53.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:53.278 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.49271.asok status 2024-09-19T04:19:53.432 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:53.432 INFO:teuthology.orchestra.run.smithi076.stdout: "metadata": { 2024-09-19T04:19:53.432 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_sha1": "2f599a46ad10e37c929e5c43da2f6907ddcd1961", 2024-09-19T04:19:53.432 INFO:teuthology.orchestra.run.smithi076.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T04:19:53.432 INFO:teuthology.orchestra.run.smithi076.stdout: "entity_id": "0", 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "hostname": "smithi076", 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "pid": "49271", 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "root": "/" 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_count": 0, 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "dentry_pinned_count": 0, 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8280, 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "inst": { 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "name": { 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "client", 2024-09-19T04:19:53.433 INFO:teuthology.orchestra.run.smithi076.stdout: "num": 8280 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2264047940 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": { 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "type": "v1", 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr": "192.168.0.1:0", 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "nonce": 2264047940 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: }, 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "inst_str": "client.8280 192.168.0.1:0/2264047940", 2024-09-19T04:19:53.434 INFO:teuthology.orchestra.run.smithi076.stdout: "addr_str": "192.168.0.1:0/2264047940", 2024-09-19T04:19:53.435 INFO:teuthology.orchestra.run.smithi076.stdout: "inode_count": 1, 2024-09-19T04:19:53.435 INFO:teuthology.orchestra.run.smithi076.stdout: "mds_epoch": 102, 2024-09-19T04:19:53.435 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch": 104, 2024-09-19T04:19:53.435 INFO:teuthology.orchestra.run.smithi076.stdout: "osd_epoch_barrier": 0, 2024-09-19T04:19:53.435 INFO:teuthology.orchestra.run.smithi076.stdout: "blocklisted": false, 2024-09-19T04:19:53.435 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs" 2024-09-19T04:19:53.435 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:53.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:53.446 DEBUG:teuthology.orchestra.run.smithi076:> stat --file-system '--printf=%T 2024-09-19T04:19:53.446 DEBUG:teuthology.orchestra.run.smithi076:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:53.452 INFO:teuthology.orchestra.run.smithi076.stdout:fuseblk 2024-09-19T04:19:53.452 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:53.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:53.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T04:19:53.507 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-09-19T04:19:53.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fe658640 1 Processor -- start 2024-09-19T04:19:53.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fe658640 1 -- start start 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fe658640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f83f80b4460 0x7f83f806e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fe658640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f806ee50 con 0x7f83f806fef0 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fe658640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83f806f130 con 0x7f83f8073e60 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fe658640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f83f806f410 con 0x7f83f80b4460 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83effff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f83f806fef0 0x7f83f80b3d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45302/0 (socket says 172.21.15.76:45302) 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83effff640 1 -- 172.21.15.76:0/3886805766 learned_addr learned my addr 172.21.15.76:0/3886805766 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fcbf5640 1 --2- 172.21.15.76:0/3886805766 >> v2:172.21.15.76:3300/0 conn(0x7f83f80b4460 0x7f83f806e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fcbf5640 1 -- 172.21.15.76:0/3886805766 >> v1:172.21.15.116:6789/0 conn(0x7f83f8073e60 legacy=0x7f83f815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:53.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fcbf5640 1 -- 172.21.15.76:0/3886805766 >> v1:172.21.15.76:6789/0 conn(0x7f83f806fef0 legacy=0x7f83f80b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.680+0000 7f83fcbf5640 1 -- 172.21.15.76:0/3886805766 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f806f6f0 con 0x7f83f80b4460 2024-09-19T04:19:53.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fcbf5640 1 --2- 172.21.15.76:0/3886805766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f80b4460 0x7f83f806e800 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009d90 tx=0x7f83f001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39de032755cade93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83f77fe640 1 -- 172.21.15.76:0/3886805766 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00b2070 con 0x7f83f80b4460 2024-09-19T04:19:53.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83f77fe640 1 -- 172.21.15.76:0/3886805766 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83f00a57f0 con 0x7f83f80b4460 2024-09-19T04:19:53.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83f77fe640 1 -- 172.21.15.76:0/3886805766 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f00a5bc0 con 0x7f83f80b4460 2024-09-19T04:19:53.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- 172.21.15.76:0/3886805766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f80b4460 msgr2=0x7f83f806e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 --2- 172.21.15.76:0/3886805766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f80b4460 0x7f83f806e800 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f83f0009d90 tx=0x7f83f001c8a0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- 172.21.15.76:0/3886805766 shutdown_connections 2024-09-19T04:19:53.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 --2- 172.21.15.76:0/3886805766 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f80b4460 0x7f83f806e800 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- 172.21.15.76:0/3886805766 >> 172.21.15.76:0/3886805766 conn(0x7f83f80b1f90 msgr2=0x7f83f80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- 172.21.15.76:0/3886805766 shutdown_connections 2024-09-19T04:19:53.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- 172.21.15.76:0/3886805766 wait complete. 2024-09-19T04:19:53.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 Processor -- start 2024-09-19T04:19:53.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- start start 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f83f806fef0 0x7f83f81574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f8073e60 0x7f83f8158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f80b4460 0x7f83f815e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8169460 con 0x7f83f8073e60 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81692e0 con 0x7f83f80b4460 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f83f81695e0 con 0x7f83f806fef0 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f8073e60 0x7f83f8158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f8073e60 0x7f83f8158ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33060/0 (socket says 172.21.15.76:33060) 2024-09-19T04:19:53.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 -- 172.21.15.76:0/2325940297 learned_addr learned my addr 172.21.15.76:0/2325940297 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fcbf5640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f80b4460 0x7f83f815e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 -- 172.21.15.76:0/2325940297 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f83f806fef0 msgr2=0x7f83f81574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f83f806fef0 0x7f83f81574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 -- 172.21.15.76:0/2325940297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f80b4460 msgr2=0x7f83f815e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f80b4460 0x7f83f815e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 -- 172.21.15.76:0/2325940297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f80b4c00 con 0x7f83f8073e60 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83effff640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f8073e60 0x7f83f8158ed0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f83e000c370 tx=0x7f83e000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83f57fa640 1 -- 172.21.15.76:0/2325940297 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0017070 con 0x7f83f8073e60 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83f57fa640 1 -- 172.21.15.76:0/2325940297 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83e0005cf0 con 0x7f83f8073e60 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83f57fa640 1 -- 172.21.15.76:0/2325940297 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e00072f0 con 0x7f83f8073e60 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f8156430 con 0x7f83f8073e60 2024-09-19T04:19:53.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.684+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81569f0 con 0x7f83f8073e60 2024-09-19T04:19:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.688+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83f806e800 con 0x7f83f8073e60 2024-09-19T04:19:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.688+0000 7f83f57fa640 1 -- 172.21.15.76:0/2325940297 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f83e001a070 con 0x7f83f8073e60 2024-09-19T04:19:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.688+0000 7f83f57fa640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83d80678c0 0x7f83d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:53.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.688+0000 7f83f7fff640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83d80678c0 0x7f83d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:53.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.688+0000 7f83f57fa640 1 -- 172.21.15.76:0/2325940297 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f83e00855b0 con 0x7f83f8073e60 2024-09-19T04:19:53.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.688+0000 7f83f7fff640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83d80678c0 0x7f83d8069d80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f83f815a610 tx=0x7f83e80079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:53.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.692+0000 7f83f57fa640 1 -- 172.21.15.76:0/2325940297 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f83e0021020 con 0x7f83f8073e60 2024-09-19T04:19:53.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.888+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f83f80723f0 con 0x7f83f8073e60 2024-09-19T04:19:53.943 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83f57fa640 1 -- 172.21.15.76:0/2325940297 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v103) ==== 105+0+0 (secure 0 0 0) 0x7f83e0055fd0 con 0x7f83f8073e60 2024-09-19T04:19:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83d80678c0 msgr2=0x7f83d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83d80678c0 0x7f83d8069d80 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f83f815a610 tx=0x7f83e80079c0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f8073e60 msgr2=0x7f83f8158ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f8073e60 0x7f83f8158ed0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f83e000c370 tx=0x7f83e000c830 comp rx=0 tx=0).stop 2024-09-19T04:19:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 shutdown_connections 2024-09-19T04:19:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f83d80678c0 0x7f83d8069d80 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.946 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f83f80b4460 0x7f83f815e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f83f8073e60 0x7f83f8158ed0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 --2- 172.21.15.76:0/2325940297 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f83f806fef0 0x7f83f81574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:53.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 >> 172.21.15.76:0/2325940297 conn(0x7f83f80b1f90 msgr2=0x7f83f80751a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:53.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 shutdown_connections 2024-09-19T04:19:53.947 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:53.944+0000 7f83fe658640 1 -- 172.21.15.76:0/2325940297 wait complete. 2024-09-19T04:19:53.970 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:54.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.084+0000 7fc441c58640 1 Processor -- start 2024-09-19T04:19:54.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc441c58640 1 -- start start 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc441c58640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc43c0d3060 0x7fc43c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc441c58640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc43c0cf450 0x7fc43c06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc441c58640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43c0d0300 con 0x7fc43c161a60 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc441c58640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fc43c0b0f60 con 0x7fc43c0d3060 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc441c58640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fc43c0cfd40 con 0x7fc43c0cf450 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc43c0d3060 0x7fc43c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fc43c0d3060 0x7fc43c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33070/0 (socket says 172.21.15.76:33070) 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 -- 172.21.15.76:0/1128972878 learned_addr learned my addr 172.21.15.76:0/1128972878 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43bfff640 1 --2- 172.21.15.76:0/1128972878 >> v2:172.21.15.116:3300/0 conn(0x7fc43c0cf450 0x7fc43c06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 -- 172.21.15.76:0/1128972878 >> v2:172.21.15.116:3300/0 conn(0x7fc43c0cf450 msgr2=0x7fc43c06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 --2- 172.21.15.76:0/1128972878 >> v2:172.21.15.116:3300/0 conn(0x7fc43c0cf450 0x7fc43c06d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 -- 172.21.15.76:0/1128972878 >> v1:172.21.15.116:6789/0 conn(0x7fc43c161a60 legacy=0x7fc43c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:54.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 -- 172.21.15.76:0/1128972878 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c0d0020 con 0x7fc43c0d3060 2024-09-19T04:19:54.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43affd640 1 --2- 172.21.15.76:0/1128972878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c0d3060 0x7fc43c0cef10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0ce950 tx=0x7fc42c01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da164cca73c1423a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:54.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43a7fc640 1 -- 172.21.15.76:0/1128972878 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0b0070 con 0x7fc43c0d3060 2024-09-19T04:19:54.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43a7fc640 1 -- 172.21.15.76:0/1128972878 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42c0ab600 con 0x7fc43c0d3060 2024-09-19T04:19:54.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.088+0000 7fc43a7fc640 1 -- 172.21.15.76:0/1128972878 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c0ab9d0 con 0x7fc43c0d3060 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- 172.21.15.76:0/1128972878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c0d3060 msgr2=0x7fc43c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 --2- 172.21.15.76:0/1128972878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c0d3060 0x7fc43c0cef10 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc43c0ce950 tx=0x7fc42c01c240 comp rx=0 tx=0).stop 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- 172.21.15.76:0/1128972878 shutdown_connections 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 --2- 172.21.15.76:0/1128972878 >> v2:172.21.15.116:3300/0 conn(0x7fc43c0cf450 0x7fc43c06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 --2- 172.21.15.76:0/1128972878 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c0d3060 0x7fc43c0cef10 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- 172.21.15.76:0/1128972878 >> 172.21.15.76:0/1128972878 conn(0x7fc43c08bf90 msgr2=0x7fc43c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- 172.21.15.76:0/1128972878 shutdown_connections 2024-09-19T04:19:54.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- 172.21.15.76:0/1128972878 wait complete. 2024-09-19T04:19:54.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 Processor -- start 2024-09-19T04:19:54.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- start start 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc43c0cf450 0x7fc43c0c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc43c0d3060 0x7fc43c153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c161a60 0x7fc43c0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c16af20 con 0x7fc43c161a60 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c06e650 con 0x7fc43c0cf450 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc43c16b0a0 con 0x7fc43c0d3060 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c161a60 0x7fc43c0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c161a60 0x7fc43c0ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33080/0 (socket says 172.21.15.76:33080) 2024-09-19T04:19:54.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 -- 172.21.15.76:0/716867401 learned_addr learned my addr 172.21.15.76:0/716867401 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43b7fe640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc43c0cf450 0x7fc43c0c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43affd640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc43c0d3060 0x7fc43c153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 -- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc43c0d3060 msgr2=0x7fc43c153410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc43c0d3060 0x7fc43c153410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 -- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc43c0cf450 msgr2=0x7fc43c0c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc43c0cf450 0x7fc43c0c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 -- 172.21.15.76:0/716867401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c0c8d90 con 0x7fc43c161a60 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43affd640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc43c0d3060 0x7fc43c153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc43bfff640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c161a60 0x7fc43c0ca1f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fc43400cc60 tx=0x7fc43400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc423fff640 1 -- 172.21.15.76:0/716867401 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4340039a0 con 0x7fc43c161a60 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc423fff640 1 -- 172.21.15.76:0/716867401 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc434003b40 con 0x7fc43c161a60 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc423fff640 1 -- 172.21.15.76:0/716867401 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc434004020 con 0x7fc43c161a60 2024-09-19T04:19:54.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c15f860 con 0x7fc43c161a60 2024-09-19T04:19:54.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.092+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c0b1850 con 0x7fc43c161a60 2024-09-19T04:19:54.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.096+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43c15fd80 con 0x7fc43c161a60 2024-09-19T04:19:54.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.096+0000 7fc423fff640 1 -- 172.21.15.76:0/716867401 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc4340041c0 con 0x7fc43c161a60 2024-09-19T04:19:54.097 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.096+0000 7fc423fff640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4280678f0 0x7fc428069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:54.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.096+0000 7fc423fff640 1 -- 172.21.15.76:0/716867401 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fc434086270 con 0x7fc43c161a60 2024-09-19T04:19:54.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.096+0000 7fc43b7fe640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4280678f0 0x7fc428069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:54.098 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.096+0000 7fc43b7fe640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4280678f0 0x7fc428069db0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc43c1598d0 tx=0x7fc430009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:54.101 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.100+0000 7fc423fff640 1 -- 172.21.15.76:0/716867401 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc43c15fd80 con 0x7fc43c161a60 2024-09-19T04:19:54.182 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:19:54.185 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:19:54.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.296+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc43c0d3940 con 0x7fc43c161a60 2024-09-19T04:19:54.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.296+0000 7fc423fff640 1 -- 172.21.15.76:0/716867401 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+5247 (secure 0 0 0) 0x7fc43c0c8d90 con 0x7fc43c161a60 2024-09-19T04:19:54.298 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:54.298 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":104,"btime":"2024-09-19T04:19:53:951153+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":6127,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":104,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:19:53.951110+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1],"up":{"mds_0":8127,"mds_1":8115},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:creating","state_seq":1,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127]},"id":7}]} 2024-09-19T04:19:54.299 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 104 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4280678f0 msgr2=0x7fc428069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4280678f0 0x7fc428069db0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc43c1598d0 tx=0x7fc430009040 comp rx=0 tx=0).stop 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c161a60 msgr2=0x7fc43c0ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c161a60 0x7fc43c0ca1f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fc43400cc60 tx=0x7fc43400a5c0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 shutdown_connections 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc4280678f0 0x7fc428069db0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc43c161a60 0x7fc43c0ca1f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc43c0d3060 0x7fc43c153410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 --2- 172.21.15.76:0/716867401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc43c0cf450 0x7fc43c0c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:54.301 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 >> 172.21.15.76:0/716867401 conn(0x7fc43c08bf90 msgr2=0x7fc43c0d2950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:54.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 shutdown_connections 2024-09-19T04:19:54.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:54.300+0000 7fc441c58640 1 -- 172.21.15.76:0/716867401 wait complete. 2024-09-19T04:19:54.719 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:19:44.552866+0000', 'modified': '2024-09-19T04:19:53.951110+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1], 'up': {'mds_0': 8127, 'mds_1': 8115}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8115': {'gid': 8115, 'name': 'c', 'rank': 1, 'incarnation': 104, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.76:6835/2501269734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 2501269734}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 2501269734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8127': {'gid': 8127, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3881707282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3881707282}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3881707282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8127, 'qdb_cluster': [8127]} 2024-09-19T04:19:54.719 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8115:up:creating 2024-09-19T04:19:55.721 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:55.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa630948640 1 Processor -- start 2024-09-19T04:19:55.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa630948640 1 -- start start 2024-09-19T04:19:55.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa630948640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa62c1520a0 0x7fa62c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:55.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa630948640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c15f3a0 0x7fa62c0adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:55.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa630948640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa62c070250 con 0x7fa62c06fe50 2024-09-19T04:19:55.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa630948640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa62c152480 con 0x7fa62c1520a0 2024-09-19T04:19:55.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa630948640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c0aede0 con 0x7fa62c15f3a0 2024-09-19T04:19:55.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa629d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa62c06fe50 0x7fa62c15ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45310/0 (socket says 172.21.15.76:45310) 2024-09-19T04:19:55.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa629d9b640 1 -- 172.21.15.76:0/4276608051 learned_addr learned my addr 172.21.15.76:0/4276608051 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62a59c640 1 --2- 172.21.15.76:0/4276608051 >> v2:172.21.15.116:3300/0 conn(0x7fa62c1520a0 0x7fa62c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62ad9d640 1 --2- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c15f3a0 0x7fa62c0adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62959a640 1 -- 172.21.15.76:0/4276608051 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586154325 0 0) 0x7fa62c070250 con 0x7fa62c06fe50 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62959a640 1 -- 172.21.15.76:0/4276608051 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa614003530 con 0x7fa62c06fe50 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62959a640 1 -- 172.21.15.76:0/4276608051 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3837497332 0 0) 0x7fa614003530 con 0x7fa62c06fe50 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62a59c640 1 -- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c15f3a0 msgr2=0x7fa62c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62a59c640 1 --2- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c15f3a0 0x7fa62c0adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62a59c640 1 -- 172.21.15.76:0/4276608051 >> v1:172.21.15.76:6789/0 conn(0x7fa62c06fe50 legacy=0x7fa62c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:55.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.852+0000 7fa62a59c640 1 -- 172.21.15.76:0/4276608051 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c15fa10 con 0x7fa62c1520a0 2024-09-19T04:19:55.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa62a59c640 1 --2- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c1520a0 0x7fa62c06f910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0097c0 tx=0x7fa61c01c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db08c1964b9b6a2b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:55.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa62959a640 1 -- 172.21.15.76:0/4276608051 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c0b0070 con 0x7fa62c1520a0 2024-09-19T04:19:55.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa62959a640 1 -- 172.21.15.76:0/4276608051 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa61c0ab400 con 0x7fa62c1520a0 2024-09-19T04:19:55.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa62959a640 1 -- 172.21.15.76:0/4276608051 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa620001680 con 0x7fa62c1520a0 2024-09-19T04:19:55.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 -- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c1520a0 msgr2=0x7fa62c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:55.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 --2- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c1520a0 0x7fa62c06f910 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0097c0 tx=0x7fa61c01c3c0 comp rx=0 tx=0).stop 2024-09-19T04:19:55.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 -- 172.21.15.76:0/4276608051 shutdown_connections 2024-09-19T04:19:55.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 --2- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c15f3a0 0x7fa62c0adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:55.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 --2- 172.21.15.76:0/4276608051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c1520a0 0x7fa62c06f910 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:55.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 -- 172.21.15.76:0/4276608051 >> 172.21.15.76:0/4276608051 conn(0x7fa62c0b1f90 msgr2=0x7fa62c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:55.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 -- 172.21.15.76:0/4276608051 shutdown_connections 2024-09-19T04:19:55.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 -- 172.21.15.76:0/4276608051 wait complete. 2024-09-19T04:19:55.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 Processor -- start 2024-09-19T04:19:55.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.856+0000 7fa630948640 1 -- start start 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c06fe50 0x7fa62c143210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c1520a0 0x7fa62c099060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa62c15f3a0 0x7fa62c0995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c073d30 con 0x7fa62c15f3a0 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c073a50 con 0x7fa62c06fe50 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa62c167f10 con 0x7fa62c1520a0 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa62c15f3a0 0x7fa62c0995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa62c15f3a0 0x7fa62c0995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33086/0 (socket says 172.21.15.76:33086) 2024-09-19T04:19:55.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 -- 172.21.15.76:0/2026469206 learned_addr learned my addr 172.21.15.76:0/2026469206 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62a59c640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c06fe50 0x7fa62c143210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa629d9b640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c1520a0 0x7fa62c099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 -- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c1520a0 msgr2=0x7fa62c099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c1520a0 0x7fa62c099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 -- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c06fe50 msgr2=0x7fa62c143210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c06fe50 0x7fa62c143210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 -- 172.21.15.76:0/2026469206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c164f70 con 0x7fa62c15f3a0 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62a59c640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c06fe50 0x7fa62c143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:55.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62ad9d640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa62c15f3a0 0x7fa62c0995a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa62400efc0 tx=0x7fa62400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:55.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa61affd640 1 -- 172.21.15.76:0/2026469206 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624018020 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa61affd640 1 -- 172.21.15.76:0/2026469206 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa624014480 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c165250 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa61affd640 1 -- 172.21.15.76:0/2026469206 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624014870 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c165810 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c06f910 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa61affd640 1 -- 172.21.15.76:0/2026469206 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa62401b020 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa61affd640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa60c0678c0 0x7fa60c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa61affd640 1 -- 172.21.15.76:0/2026469206 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fa624086c30 con 0x7fa62c15f3a0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62a59c640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa60c0678c0 0x7fa60c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:55.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.860+0000 7fa62a59c640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa60c0678c0 0x7fa60c069d80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0ad000 tx=0x7fa61c0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:55.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:55.864+0000 7fa61affd640 1 -- 172.21.15.76:0/2026469206 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa624022030 con 0x7fa62c15f3a0 2024-09-19T04:19:56.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.064+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa62c165a50 con 0x7fa62c15f3a0 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa61affd640 1 -- 172.21.15.76:0/2026469206 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5270 (secure 0 0 0) 0x7fa624010040 con 0x7fa62c15f3a0 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":107,"btime":"2024-09-19T04:19:55:962543+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:19:55.962531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[6127,8115,8127]},"id":7}]} 2024-09-19T04:19:56.068 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 107 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa60c0678c0 msgr2=0x7fa60c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa60c0678c0 0x7fa60c069d80 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa61c0ad000 tx=0x7fa61c0a6000 comp rx=0 tx=0).stop 2024-09-19T04:19:56.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa62c15f3a0 msgr2=0x7fa62c0995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa62c15f3a0 0x7fa62c0995a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa62400efc0 tx=0x7fa62400c490 comp rx=0 tx=0).stop 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 shutdown_connections 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa60c0678c0 0x7fa60c069d80 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa62c15f3a0 0x7fa62c0995a0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa62c1520a0 0x7fa62c099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 --2- 172.21.15.76:0/2026469206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa62c06fe50 0x7fa62c143210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 >> 172.21.15.76:0/2026469206 conn(0x7fa62c0b1f90 msgr2=0x7fa62c15b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 shutdown_connections 2024-09-19T04:19:56.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.068+0000 7fa630948640 1 -- 172.21.15.76:0/2026469206 wait complete. 2024-09-19T04:19:56.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-19T04:19:44.552866+0000', 'modified': '2024-09-19T04:19:55.962531+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 8127, 'mds_1': 8115, 'mds_2': 6127}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6127': {'gid': 6127, 'name': 'd', 'rank': 2, 'incarnation': 106, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6835/255266111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 255266111}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 255266111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8115': {'gid': 8115, 'name': 'c', 'rank': 1, 'incarnation': 104, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.76:6835/2501269734', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6834', 'nonce': 2501269734}, {'type': 'v1', 'addr': '172.21.15.76:6835', 'nonce': 2501269734}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_8127': {'gid': 8127, 'name': 'a', 'rank': 0, 'incarnation': 99, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.76:6837/3881707282', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.76:6836', 'nonce': 3881707282}, {'type': 'v1', 'addr': '172.21.15.76:6837', 'nonce': 3881707282}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8127, 'qdb_cluster': [6127, 8115, 8127]} 2024-09-19T04:19:56.490 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T04:19:56.490 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-19T04:19:56.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 Processor -- start 2024-09-19T04:19:56.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 -- start start 2024-09-19T04:19:56.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f001520a0 0x7f5f0006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f0015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5f0015f1d0 0x7f5f00071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5f00152480 con 0x7f5f001520a0 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f5f00070250 con 0x7f5f0015f1d0 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f077f6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5f000aede0 con 0x7f5f0006fe50 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.620+0000 7f5f05592640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f001520a0 0x7f5f0006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5f001520a0 0x7f5f0006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33098/0 (socket says 172.21.15.76:33098) 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 -- 172.21.15.76:0/462562693 learned_addr learned my addr 172.21.15.76:0/462562693 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f04d91640 1 --2- 172.21.15.76:0/462562693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f0015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05d93640 1 --2- 172.21.15.76:0/462562693 >> v2:172.21.15.116:3300/0 conn(0x7f5f0015f1d0 0x7f5f00071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 -- 172.21.15.76:0/462562693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 msgr2=0x7f5f0015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.624 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 --2- 172.21.15.76:0/462562693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f0015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 -- 172.21.15.76:0/462562693 >> v2:172.21.15.116:3300/0 conn(0x7f5f0015f1d0 msgr2=0x7f5f00071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 --2- 172.21.15.76:0/462562693 >> v2:172.21.15.116:3300/0 conn(0x7f5f0015f1d0 0x7f5f00071f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 -- 172.21.15.76:0/462562693 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f0015fa10 con 0x7f5f001520a0 2024-09-19T04:19:56.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f04d91640 1 --2- 172.21.15.76:0/462562693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f0015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:56.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5f05592640 1 --2- 172.21.15.76:0/462562693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f001520a0 0x7f5f0006f910 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5f0006e250 tx=0x7f5efc01c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4a46382568fc8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5ef7fff640 1 -- 172.21.15.76:0/462562693 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0b0070 con 0x7f5f001520a0 2024-09-19T04:19:56.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5ef7fff640 1 -- 172.21.15.76:0/462562693 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc0ab600 con 0x7f5f001520a0 2024-09-19T04:19:56.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.624+0000 7f5ef7fff640 1 -- 172.21.15.76:0/462562693 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0ab9d0 con 0x7f5f001520a0 2024-09-19T04:19:56.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- 172.21.15.76:0/462562693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f001520a0 msgr2=0x7f5f0006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 --2- 172.21.15.76:0/462562693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f001520a0 0x7f5f0006f910 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f5f0006e250 tx=0x7f5efc01c240 comp rx=0 tx=0).stop 2024-09-19T04:19:56.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- 172.21.15.76:0/462562693 shutdown_connections 2024-09-19T04:19:56.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 --2- 172.21.15.76:0/462562693 >> v2:172.21.15.116:3300/0 conn(0x7f5f0015f1d0 0x7f5f00071f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 --2- 172.21.15.76:0/462562693 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f0015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 --2- 172.21.15.76:0/462562693 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f001520a0 0x7f5f0006f910 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- 172.21.15.76:0/462562693 >> 172.21.15.76:0/462562693 conn(0x7f5f000b1f90 msgr2=0x7f5f000b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- 172.21.15.76:0/462562693 shutdown_connections 2024-09-19T04:19:56.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- 172.21.15.76:0/462562693 wait complete. 2024-09-19T04:19:56.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 Processor -- start 2024-09-19T04:19:56.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- start start 2024-09-19T04:19:56.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f00099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f001520a0 0x7f5f0009c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0015f1d0 0x7f5f000b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00073d30 con 0x7f5f0015f1d0 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00073a50 con 0x7f5f001520a0 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5f0015f5e0 con 0x7f5f0006fe50 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0015f1d0 0x7f5f000b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0015f1d0 0x7f5f000b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33104/0 (socket says 172.21.15.76:33104) 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 -- 172.21.15.76:0/2461092085 learned_addr learned my addr 172.21.15.76:0/2461092085 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05592640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f00099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f04d91640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f001520a0 0x7f5f0009c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 -- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 msgr2=0x7f5f00099940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f00099940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 -- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f001520a0 msgr2=0x7f5f0009c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f001520a0 0x7f5f0009c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 -- 172.21.15.76:0/2461092085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f00099e80 con 0x7f5f0015f1d0 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f05d93640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0015f1d0 0x7f5f000b3bc0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009dd0 tx=0x7f5ef800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5ef5ffb640 1 -- 172.21.15.76:0/2461092085 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8018020 con 0x7f5f0015f1d0 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5ef5ffb640 1 -- 172.21.15.76:0/2461092085 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef8014480 con 0x7f5f0015f1d0 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f0009cec0 con 0x7f5f0015f1d0 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5ef5ffb640 1 -- 172.21.15.76:0/2461092085 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8014870 con 0x7f5f0015f1d0 2024-09-19T04:19:56.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.628+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f0009d480 con 0x7f5f0015f1d0 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5ef5ffb640 1 -- 172.21.15.76:0/2461092085 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ef801b020 con 0x7f5f0015f1d0 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5ef5ffb640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ee40678c0 0x7f5ee4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f05592640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ee40678c0 0x7f5ee4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5ef5ffb640 1 -- 172.21.15.76:0/2461092085 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5ef8086960 con 0x7f5f0015f1d0 2024-09-19T04:19:56.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f05592640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ee40678c0 0x7f5ee4069d80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0ad000 tx=0x7f5efc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 Processor -- start 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 -- start start 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad615da440 0x55ad6158fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad61590290 0x55ad61590670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ad61477b60 con 0x55ad615da440 2024-09-19T04:19:56.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ad613dec70 con 0x55ad615d9b20 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ad61384ed0 con 0x55ad61590290 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad615da440 0x55ad6158fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad615da440 0x55ad6158fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33108/0 (socket says 172.21.15.76:33108) 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 -- 172.21.15.76:0/4260942219 learned_addr learned my addr 172.21.15.76:0/4260942219 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f05592640 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 -- 172.21.15.76:0/4260942219 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad61590290 msgr2=0x55ad61590670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad61590290 0x55ad61590670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 -- 172.21.15.76:0/4260942219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 msgr2=0x55ad615d9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 -- 172.21.15.76:0/4260942219 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad61396e50 con 0x55ad615da440 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5f04d91640 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad615da440 0x55ad6158fd50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000d860 tx=0x7f5ef000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6883d071a9c8160 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5eeaffd640 1 -- 172.21.15.76:0/4260942219 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0018030 con 0x55ad615da440 2024-09-19T04:19:56.636 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5eeaffd640 1 -- 172.21.15.76:0/4260942219 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0004020 con 0x55ad615da440 2024-09-19T04:19:56.637 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.632+0000 7f5eeaffd640 1 -- 172.21.15.76:0/4260942219 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00043f0 con 0x55ad615da440 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/4260942219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad615da440 msgr2=0x55ad6158fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad615da440 0x55ad6158fd50 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000d860 tx=0x7f5ef000dd20 comp rx=0 tx=0).stop 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/4260942219 shutdown_connections 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad61590290 0x55ad61590670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad615da440 0x55ad6158fd50 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/4260942219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/4260942219 >> 172.21.15.76:0/4260942219 conn(0x55ad61596600 msgr2=0x55ad6158e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/4260942219 shutdown_connections 2024-09-19T04:19:56.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/4260942219 wait complete. 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- start start 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 Processor -- start 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61594e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad6159acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad6159b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ad61440b00 con 0x55ad61595390 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ad613b9360 con 0x55ad615d9b20 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ad613b62d0 con 0x55ad615da440 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 10 client.0 _ll_register_callbacks cb 0x55ad6157b820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad6159b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad6159b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33122/0 (socket says 172.21.15.76:33122) 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 -- 172.21.15.76:0/2474394390 learned_addr learned my addr 172.21.15.76:0/2474394390 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.641 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05592640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61594e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f04d91640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad6159acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 -- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 msgr2=0x55ad6159acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad6159acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 -- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 msgr2=0x55ad61594e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61594e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 -- 172.21.15.76:0/2474394390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad61385120 con 0x55ad61595390 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05592640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61594e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05d93640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad6159b230 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f5ef805f220 tx=0x7f5ef8056cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cad98031f2228f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5edffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5edffff640 1 -- 172.21.15.76:0/2474394390 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef805f4c0 con 0x55ad61595390 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5edffff640 1 -- 172.21.15.76:0/2474394390 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5ef8055fc0 con 0x55ad61595390 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ef0004020 con 0x55ad61595390 2024-09-19T04:19:56.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5edffff640 1 -- 172.21.15.76:0/2474394390 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f5ef8075120 con 0x55ad61595390 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 10 client.8313 fetch_fsmap learned FSMap version 107 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55ad6137e450 con 0x55ad61595390 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5edffff640 1 -- 172.21.15.76:0/2474394390 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef0004020 con 0x55ad61595390 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5edffff640 1 -- 172.21.15.76:0/2474394390 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 107) ==== 3542+0+0 (secure 0 0 0) 0x7f5ef805d130 con 0x55ad61595390 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 10 client.8313 fetch_fsmap finished waiting for FSMap version 107 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 10 client.8313 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 20 client.8313 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad615e8280 0x55ad615ea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 4 client.8313 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad61378c00 con 0x55ad615e8280 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.636+0000 7f5f05592640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad615e8280 0x55ad615ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5edffff640 10 client.8313 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:19:56.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f05592640 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad615e8280 0x55ad615ea670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a7acb587d8f7b15 server_cookie=f4da29e0ea820b55 in_seq=0 out_seq=0 2024-09-19T04:19:56.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5edffff640 1 -- 172.21.15.76:0/2474394390 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ad61378c00 con 0x55ad615e8280 2024-09-19T04:19:56.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5edffff640 10 client.8313 handle_command_reply: tid=0 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 client.8313 shutdown 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 msgr2=0x55ad6159b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad6159b230 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f5ef805f220 tx=0x7f5ef8056cb0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 shutdown_connections 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad615e8280 0x55ad615ea670 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad6159b230 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad6159acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/2474394390 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61594e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5edffff640 0 client.8313 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 >> 172.21.15.76:0/2474394390 conn(0x55ad61590290 msgr2=0x55ad6158ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 shutdown_connections 2024-09-19T04:19:56.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/2474394390 wait complete. 2024-09-19T04:19:56.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.640+0000 7f5f0a8b2000 20 client.8313 trim_cache size 0 max 16384 2024-09-19T04:19:56.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 Processor -- start 2024-09-19T04:19:56.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- start start 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad615ea670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00200f60 con 0x55ad61595390 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5f00201630 con 0x55ad615d9b20 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5f000b0600 con 0x55ad615da440 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615d8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33130/0 (socket says 172.21.15.76:33130) 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 -- 172.21.15.76:0/497928902 learned_addr learned my addr 172.21.15.76:0/497928902 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 -- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 msgr2=0x55ad615ea670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad615ea670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 -- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 msgr2=0x55ad615d9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f04d91640 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 -- 172.21.15.76:0/497928902 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad61365fd0 con 0x55ad61595390 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f04d91640 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f05592640 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615d8cc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5efc01c280 tx=0x7f5efc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7165a20dc98c020 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5eea7fc640 1 -- 172.21.15.76:0/497928902 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0d1070 con 0x55ad61595390 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5eea7fc640 1 -- 172.21.15.76:0/497928902 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc0cc450 con 0x55ad61595390 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5eea7fc640 1 -- 172.21.15.76:0/497928902 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0b0040 con 0x55ad61595390 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/497928902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 msgr2=0x55ad615d8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615d8cc0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f5efc01c280 tx=0x7f5efc004740 comp rx=0 tx=0).stop 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/497928902 shutdown_connections 2024-09-19T04:19:56.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad615ea670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad615d9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/497928902 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615d8cc0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/497928902 >> 172.21.15.76:0/497928902 conn(0x55ad61590290 msgr2=0x55ad61594650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/497928902 shutdown_connections 2024-09-19T04:19:56.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.660+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/497928902 wait complete. 2024-09-19T04:19:56.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- start start 2024-09-19T04:19:56.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 Processor -- start 2024-09-19T04:19:56.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61608960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad61608ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ad613ddf80 con 0x55ad61595390 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ad61396ca0 con 0x55ad615d9b20 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ad613e32a0 con 0x55ad615da440 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 10 client.0 _ll_register_callbacks cb 0x7f5ef800cf90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615ed590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33146/0 (socket says 172.21.15.76:33146) 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 -- 172.21.15.76:0/5586127 learned_addr learned my addr 172.21.15.76:0/5586127 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 -- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 msgr2=0x55ad61608ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad61608ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 -- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 msgr2=0x55ad61608960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61608960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 -- 172.21.15.76:0/5586127 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad61461750 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f05592640 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615ed590 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009270 tx=0x7f5efc01aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3c56263f8a0c00 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 1 -- 172.21.15.76:0/5586127 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0d1030 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 1 -- 172.21.15.76:0/5586127 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f5efc0abde0 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5efc0cc450 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 1 -- 172.21.15.76:0/5586127 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f5efc0b0040 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 10 client.8319 fetch_fsmap learned FSMap version 107 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55ad6136ca30 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 1 -- 172.21.15.76:0/5586127 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5efc0cc450 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 1 -- 172.21.15.76:0/5586127 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 107) ==== 3542+0+0 (secure 0 0 0) 0x7f5efc0bf040 con 0x55ad61595390 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 10 client.8319 fetch_fsmap finished waiting for FSMap version 107 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 10 client.8319 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-19T04:19:56.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 20 client.8319 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad616093e0 0x55ad6164cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 4 client.8319 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "status"}) -- 0x55ad613ec3a0 con 0x55ad616093e0 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f04d91640 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad616093e0 0x55ad6164cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 10 client.8319 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f04d91640 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad616093e0 0x55ad6164cec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7c8d2ba03e26167 server_cookie=cc4e8d56c10b808f in_seq=0 out_seq=0 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 1 -- 172.21.15.76:0/5586127 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55ad613ec3a0 con 0x55ad616093e0 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5edffff640 10 client.8319 handle_command_reply: tid=0 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 client.8319 shutdown 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.664+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 msgr2=0x55ad615ed590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615ed590 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009270 tx=0x7f5efc01aa80 comp rx=0 tx=0).stop 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 shutdown_connections 2024-09-19T04:19:56.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55ad616093e0 0x55ad6164cec0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ad615da440 0x55ad61608ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ad615d9b20 0x55ad61608960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5edffff640 0 client.8319 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 --2- 172.21.15.76:0/5586127 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ad61595390 0x55ad615ed590 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 >> 172.21.15.76:0/5586127 conn(0x55ad61590290 msgr2=0x55ad615d7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 shutdown_connections 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 1 -- 172.21.15.76:0/5586127 wait complete. 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f0a8b2000 20 client.8319 trim_cache size 0 max 16384 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:56.672 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 2, 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 6127, 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 1.70484836, 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 107, 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 104, 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 104, 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 15.538155151 2024-09-19T04:19:56.673 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ee40678c0 msgr2=0x7f5ee4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ee40678c0 0x7f5ee4069d80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0ad000 tx=0x7f5efc008040 comp rx=0 tx=0).stop 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0015f1d0 msgr2=0x7f5f000b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0015f1d0 0x7f5f000b3bc0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009dd0 tx=0x7f5ef800c490 comp rx=0 tx=0).stop 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 shutdown_connections 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5ee40678c0 0x7f5ee4069d80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5f0015f1d0 0x7f5f000b3bc0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5f001520a0 0x7f5f0009c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 --2- 172.21.15.76:0/2461092085 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5f0006fe50 0x7f5f00099940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 >> 172.21.15.76:0/2461092085 conn(0x7f5f000b1f90 msgr2=0x7f5f00150a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 shutdown_connections 2024-09-19T04:19:56.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.668+0000 7f5f077f6640 1 -- 172.21.15.76:0/2461092085 wait complete. 2024-09-19T04:19:56.685 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-19T04:19:56.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.796+0000 7f77a50a1640 1 Processor -- start 2024-09-19T04:19:56.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.796+0000 7f77a50a1640 1 -- start start 2024-09-19T04:19:56.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.796+0000 7f77a50a1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a00d3060 0x7f77a00cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.796+0000 7f77a50a1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77a00cf450 0x7f77a006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.796+0000 7f77a50a1640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77a00d0300 con 0x7f77a0161a60 2024-09-19T04:19:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.796+0000 7f77a50a1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f77a00b0f60 con 0x7f77a00cf450 2024-09-19T04:19:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.796+0000 7f77a50a1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f77a00cfd40 con 0x7f77a00d3060 2024-09-19T04:19:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779f59e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77a00cf450 0x7f77a006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779e59c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a00d3060 0x7f77a00cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779f59e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77a00cf450 0x7f77a006d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:47978/0 (socket says 172.21.15.76:47978) 2024-09-19T04:19:56.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779f59e640 1 -- 172.21.15.76:0/2487210728 learned_addr learned my addr 172.21.15.76:0/2487210728 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779e59c640 1 -- 172.21.15.76:0/2487210728 >> v2:172.21.15.116:3300/0 conn(0x7f77a00cf450 msgr2=0x7f77a006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779e59c640 1 --2- 172.21.15.76:0/2487210728 >> v2:172.21.15.116:3300/0 conn(0x7f77a00cf450 0x7f77a006d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779e59c640 1 -- 172.21.15.76:0/2487210728 >> v1:172.21.15.116:6789/0 conn(0x7f77a0161a60 legacy=0x7f77a00d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779e59c640 1 -- 172.21.15.76:0/2487210728 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a00d0020 con 0x7f77a00d3060 2024-09-19T04:19:56.802 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779e59c640 1 --2- 172.21.15.76:0/2487210728 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a00d3060 0x7f77a00cef10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77940097c0 tx=0x7f779401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f6f69e9b98237c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779dd9b640 1 -- 172.21.15.76:0/2487210728 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940b1070 con 0x7f77a00d3060 2024-09-19T04:19:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779dd9b640 1 -- 172.21.15.76:0/2487210728 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77940a8050 con 0x7f77a00d3060 2024-09-19T04:19:56.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.800+0000 7f779dd9b640 1 -- 172.21.15.76:0/2487210728 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940ac860 con 0x7f77a00d3060 2024-09-19T04:19:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- 172.21.15.76:0/2487210728 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a00d3060 msgr2=0x7f77a00cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 --2- 172.21.15.76:0/2487210728 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a00d3060 0x7f77a00cef10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f77940097c0 tx=0x7f779401c3e0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- 172.21.15.76:0/2487210728 shutdown_connections 2024-09-19T04:19:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 --2- 172.21.15.76:0/2487210728 >> v2:172.21.15.116:3300/0 conn(0x7f77a00cf450 0x7f77a006d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 --2- 172.21.15.76:0/2487210728 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a00d3060 0x7f77a00cef10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- 172.21.15.76:0/2487210728 >> 172.21.15.76:0/2487210728 conn(0x7f77a008bf90 msgr2=0x7f77a009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- 172.21.15.76:0/2487210728 shutdown_connections 2024-09-19T04:19:56.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- 172.21.15.76:0/2487210728 wait complete. 2024-09-19T04:19:56.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 Processor -- start 2024-09-19T04:19:56.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- start start 2024-09-19T04:19:56.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a00cf450 0x7f77a0151bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77a00d3060 0x7f77a0152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a0161a60 0x7f77a00b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f77a016af20 con 0x7f77a00d3060 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77a006e610 con 0x7f77a00cf450 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f77a016b0a0 con 0x7f77a0161a60 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77a00d3060 0x7f77a0152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77a00d3060 0x7f77a0152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33160/0 (socket says 172.21.15.76:33160) 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 -- 172.21.15.76:0/1198361666 learned_addr learned my addr 172.21.15.76:0/1198361666 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779f59e640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a0161a60 0x7f77a00b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779ed9d640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a00cf450 0x7f77a0151bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.808 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 -- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a0161a60 msgr2=0x7f77a00b24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a0161a60 0x7f77a00b24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 -- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a00cf450 msgr2=0x7f77a0151bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a00cf450 0x7f77a0151bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 -- 172.21.15.76:0/1198361666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a0152130 con 0x7f77a00d3060 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779f59e640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a0161a60 0x7f77a00b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f779e59c640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77a00d3060 0x7f77a0152350 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f779401a010 tx=0x7f779401ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77877fe640 1 -- 172.21.15.76:0/1198361666 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940b1040 con 0x7f77a00d3060 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77877fe640 1 -- 172.21.15.76:0/1198361666 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77940acbd0 con 0x7f77a00d3060 2024-09-19T04:19:56.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a0152cc0 con 0x7f77a00d3060 2024-09-19T04:19:56.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77877fe640 1 -- 172.21.15.76:0/1198361666 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940b7450 con 0x7f77a00d3060 2024-09-19T04:19:56.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.804+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a0153280 con 0x7f77a00d3060 2024-09-19T04:19:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77877fe640 1 -- 172.21.15.76:0/1198361666 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f77940be030 con 0x7f77a00d3060 2024-09-19T04:19:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77877fe640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f778c0678c0 0x7f778c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77877fe640 1 -- 172.21.15.76:0/1198361666 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f779411f7f0 con 0x7f77a00d3060 2024-09-19T04:19:56.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f778c0678c0 0x7f778c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f778c0678c0 0x7f778c069d80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7790000f70 tx=0x7f7790009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 Processor -- start 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 -- start start 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b55f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b0bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b0c290 0x55db19b0c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55db199f3b60 con 0x55db19b55b20 2024-09-19T04:19:56.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55db1995ac70 con 0x55db19b56440 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55db19900ed0 con 0x55db19b0c290 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b55f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b55f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33170/0 (socket says 172.21.15.76:33170) 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 -- 172.21.15.76:0/2742922165 learned_addr learned my addr 172.21.15.76:0/2742922165 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779e59c640 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b0bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 -- 172.21.15.76:0/2742922165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b0c290 msgr2=0x55db19b0c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b0c290 0x55db19b0c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 -- 172.21.15.76:0/2742922165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 msgr2=0x55db19b0bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b0bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 -- 172.21.15.76:0/2742922165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db19912e50 con 0x55db19b55b20 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f779ed9d640 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b55f00 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f77900052a0 tx=0x7f7790011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77fa1eddac1a497c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f7784ff9640 1 -- 172.21.15.76:0/2742922165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790005450 con 0x55db19b55b20 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f7784ff9640 1 -- 172.21.15.76:0/2742922165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779000e6e0 con 0x55db19b55b20 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.808+0000 7f7784ff9640 1 -- 172.21.15.76:0/2742922165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779000ea90 con 0x55db19b55b20 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/2742922165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 msgr2=0x55db19b55f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b55f00 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f77900052a0 tx=0x7f7790011000 comp rx=0 tx=0).stop 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/2742922165 shutdown_connections 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b0c290 0x55db19b0c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b0bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- 172.21.15.76:0/2742922165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b55f00 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/2742922165 >> 172.21.15.76:0/2742922165 conn(0x55db19b12600 msgr2=0x55db19b0a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/2742922165 shutdown_connections 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/2742922165 wait complete. 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- start start 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 Processor -- start 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b10df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b16cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55db199bcb00 con 0x55db19b55b20 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55db19935360 con 0x55db19b56440 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55db199322d0 con 0x55db19b11330 2024-09-19T04:19:56.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 10 client.0 _ll_register_callbacks cb 0x55db19af7820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b10df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b10df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33176/0 (socket says 172.21.15.76:33176) 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 -- 172.21.15.76:0/4094485230 learned_addr learned my addr 172.21.15.76:0/4094485230 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 -- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 msgr2=0x55db19b17230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779f59e640 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b17230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779e59c640 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b16cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 -- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 msgr2=0x55db19b16cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b16cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 -- 172.21.15.76:0/4094485230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db19901120 con 0x55db19b55b20 2024-09-19T04:19:56.822 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779ed9d640 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b10df0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f7790003fd0 tx=0x7f7790013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3b6fef49764751 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f777dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f777dffb640 1 -- 172.21.15.76:0/4094485230 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7790021450 con 0x55db19b55b20 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f777dffb640 1 -- 172.21.15.76:0/4094485230 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7790010050 con 0x55db19b55b20 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f779000e6e0 con 0x55db19b55b20 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f777dffb640 1 -- 172.21.15.76:0/4094485230 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f7790031050 con 0x55db19b55b20 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 10 client.8328 fetch_fsmap learned FSMap version 107 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55db198fa450 con 0x55db19b55b20 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f777dffb640 1 -- 172.21.15.76:0/4094485230 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779000e6e0 con 0x55db19b55b20 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f777dffb640 1 -- 172.21.15.76:0/4094485230 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 107) ==== 3542+0+0 (secure 0 0 0) 0x7f77900215f0 con 0x55db19b55b20 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 10 client.8328 fetch_fsmap finished waiting for FSMap version 107 2024-09-19T04:19:56.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 10 client.8328 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 20 client.8328 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b64280 0x55db19b66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 4 client.8328 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db198f4c00 con 0x55db19b64280 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779e59c640 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b64280 0x55db19b66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.812+0000 7f779e59c640 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b64280 0x55db19b66670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f1bb9bf8e10f4b12 server_cookie=965b54fd32e6af8d in_seq=0 out_seq=0 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f777dffb640 10 client.8328 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f777dffb640 1 -- 172.21.15.76:0/4094485230 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55db198f4c00 con 0x55db19b64280 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f777dffb640 10 client.8328 handle_command_reply: tid=0 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 client.8328 shutdown 2024-09-19T04:19:56.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 msgr2=0x55db19b10df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b10df0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f7790003fd0 tx=0x7f7790013000 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 shutdown_connections 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b64280 0x55db19b66670 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b56440 0x55db19b16cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 --2- 172.21.15.76:0/4094485230 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b55b20 0x55db19b10df0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f777dffb640 0 client.8328 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 >> 172.21.15.76:0/4094485230 conn(0x55db19b0c290 msgr2=0x55db19b0ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 shutdown_connections 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 1 -- 172.21.15.76:0/4094485230 wait complete. 2024-09-19T04:19:56.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.816+0000 7f77a815d000 20 client.8328 trim_cache size 0 max 16384 2024-09-19T04:19:56.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.832+0000 7f77a815d000 1 Processor -- start 2024-09-19T04:19:56.834 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.832+0000 7f77a815d000 1 -- start start 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b105f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b10b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b56440 0x55db19b66670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f77a02033c0 con 0x55db19b56440 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f77a0203a90 con 0x55db19b55b20 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f77a00b8610 con 0x55db19b11330 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b56440 0x55db19b66670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b56440 0x55db19b66670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33180/0 (socket says 172.21.15.76:33180) 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 -- 172.21.15.76:0/1614028707 learned_addr learned my addr 172.21.15.76:0/1614028707 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 -- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 msgr2=0x55db19b105f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b105f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b105f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779e59c640 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b10b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 -- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 msgr2=0x55db19b10b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b10b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 -- 172.21.15.76:0/1614028707 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db19b618e0 con 0x55db19b56440 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779e59c640 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b10b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b56440 0x55db19b66670 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7798009b60 tx=0x7f7798009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e5d10de66b09a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f777ffff640 1 -- 172.21.15.76:0/1614028707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798009e60 con 0x55db19b56440 2024-09-19T04:19:56.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f777ffff640 1 -- 172.21.15.76:0/1614028707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7798015420 con 0x55db19b56440 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f777ffff640 1 -- 172.21.15.76:0/1614028707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77980157f0 con 0x55db19b56440 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- 172.21.15.76:0/1614028707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b56440 msgr2=0x55db19b66670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b56440 0x55db19b66670 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7798009b60 tx=0x7f7798009810 comp rx=0 tx=0).stop 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- 172.21.15.76:0/1614028707 shutdown_connections 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b56440 0x55db19b66670 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b10b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- 172.21.15.76:0/1614028707 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b11330 0x55db19b105f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- 172.21.15.76:0/1614028707 >> 172.21.15.76:0/1614028707 conn(0x55db19b0c290 msgr2=0x55db19b698c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- 172.21.15.76:0/1614028707 shutdown_connections 2024-09-19T04:19:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- 172.21.15.76:0/1614028707 wait complete. 2024-09-19T04:19:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- start start 2024-09-19T04:19:56.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 Processor -- start 2024-09-19T04:19:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b11330 0x55db19b68b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b84960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b56440 0x55db19b84ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55db19959f80 con 0x55db19b11330 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55db19912ca0 con 0x55db19b55b20 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55db1995f2a0 con 0x55db19b56440 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 10 client.0 _ll_register_callbacks cb 0x7f7790005170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b11330 0x55db19b68b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b11330 0x55db19b68b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33186/0 (socket says 172.21.15.76:33186) 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 -- 172.21.15.76:0/4118770385 learned_addr learned my addr 172.21.15.76:0/4118770385 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779e59c640 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b84960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779f59e640 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b56440 0x55db19b84ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 -- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b56440 msgr2=0x55db19b84ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b56440 0x55db19b84ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 -- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 msgr2=0x55db19b84960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b84960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 -- 172.21.15.76:0/4118770385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db19b56270 con 0x55db19b11330 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f779ed9d640 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b11330 0x55db19b68b90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f7790007b10 tx=0x7f7790000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9584950e109efa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f777dffb640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f777dffb640 1 -- 172.21.15.76:0/4118770385 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779003a060 con 0x55db19b11330 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f777dffb640 1 -- 172.21.15.76:0/4118770385 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f7790029040 con 0x55db19b11330 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.836+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7798015420 con 0x55db19b11330 2024-09-19T04:19:56.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f777dffb640 1 -- 172.21.15.76:0/4118770385 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f7790040050 con 0x55db19b11330 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 10 client.8334 fetch_fsmap learned FSMap version 107 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55db198e1fd0 con 0x55db19b11330 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f777dffb640 1 -- 172.21.15.76:0/4118770385 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7798015420 con 0x55db19b11330 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f777dffb640 1 -- 172.21.15.76:0/4118770385 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 107) ==== 3542+0+0 (secure 0 0 0) 0x7f7790043020 con 0x55db19b11330 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 10 client.8334 fetch_fsmap finished waiting for FSMap version 107 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 10 client.8334 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 20 client.8334 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b853e0 0x55db19bc9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 4 client.8334 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "status"}) -- 0x55db199683a0 con 0x55db19b853e0 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f779e59c640 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b853e0 0x55db19bc9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f777dffb640 10 client.8334 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f779e59c640 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b853e0 0x55db19bc9200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1e6b4e60b420abe5 server_cookie=e5dafb307fda2b6e in_seq=0 out_seq=0 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f777dffb640 1 -- 172.21.15.76:0/4118770385 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55db199683a0 con 0x55db19b853e0 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f777dffb640 10 client.8334 handle_command_reply: tid=0 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 client.8334 shutdown 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b11330 msgr2=0x55db19b68b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b11330 0x55db19b68b90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f7790007b10 tx=0x7f7790000ee0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 shutdown_connections 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55db19b853e0 0x55db19bc9200 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55db19b56440 0x55db19b84ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55db19b55b20 0x55db19b84960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.848 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f777dffb640 0 client.8334 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 --2- 172.21.15.76:0/4118770385 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55db19b11330 0x55db19b68b90 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 >> 172.21.15.76:0/4118770385 conn(0x55db19b0c290 msgr2=0x55db19b105f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 shutdown_connections 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 1 -- 172.21.15.76:0/4118770385 wait complete. 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a815d000 20 client.8334 trim_cache size 0 max 16384 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:19:56.849 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 1, 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8115, 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 2.8889847089999998, 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 107, 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 104, 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 104, 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 15.731362412999999 2024-09-19T04:19:56.850 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:56.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f778c0678c0 msgr2=0x7f778c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f778c0678c0 0x7f778c069d80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7790000f70 tx=0x7f7790009040 comp rx=0 tx=0).stop 2024-09-19T04:19:56.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77a00d3060 msgr2=0x7f77a0152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77a00d3060 0x7f77a0152350 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f779401a010 tx=0x7f779401ac20 comp rx=0 tx=0).stop 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 shutdown_connections 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f778c0678c0 0x7f778c069d80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f77a0161a60 0x7f77a00b24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f77a00d3060 0x7f77a0152350 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 --2- 172.21.15.76:0/1198361666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77a00cf450 0x7f77a0151bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 >> 172.21.15.76:0/1198361666 conn(0x7f77a008bf90 msgr2=0x7f77a00d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 shutdown_connections 2024-09-19T04:19:56.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.840+0000 7f77a50a1640 1 -- 172.21.15.76:0/1198361666 wait complete. 2024-09-19T04:19:56.857 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-19T04:19:56.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4592f3f640 1 Processor -- start 2024-09-19T04:19:56.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4592f3f640 1 -- start start 2024-09-19T04:19:56.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4592f3f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c0ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4592f3f640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0b3290 con 0x7f458c15c2a0 2024-09-19T04:19:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4592f3f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f458c0b3570 con 0x7f458c0af470 2024-09-19T04:19:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4592f3f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f458c0b3850 con 0x7f458c15bec0 2024-09-19T04:19:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f458bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f458c15c2a0 0x7f458c0aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45326/0 (socket says 172.21.15.76:45326) 2024-09-19T04:19:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f458bfff640 1 -- 172.21.15.76:0/1403932530 learned_addr learned my addr 172.21.15.76:0/1403932530 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:19:56.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4590cdb640 1 --2- 172.21.15.76:0/1403932530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c0ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4590cdb640 1 -- 172.21.15.76:0/1403932530 >> v1:172.21.15.116:6789/0 conn(0x7f458c0af470 legacy=0x7f458c06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4590cdb640 1 -- 172.21.15.76:0/1403932530 >> v1:172.21.15.76:6789/0 conn(0x7f458c15c2a0 legacy=0x7f458c0aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.968+0000 7f4590cdb640 1 -- 172.21.15.76:0/1403932530 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c0b3b30 con 0x7f458c15bec0 2024-09-19T04:19:56.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.972+0000 7f4590cdb640 1 --2- 172.21.15.76:0/1403932530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c0ad200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45800097f0 tx=0x7f458001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=545c87e3bc4a510a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.972+0000 7f458b7fe640 1 -- 172.21.15.76:0/1403932530 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45800aa610 con 0x7f458c15bec0 2024-09-19T04:19:56.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.972+0000 7f458b7fe640 1 -- 172.21.15.76:0/1403932530 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45800af070 con 0x7f458c15bec0 2024-09-19T04:19:56.975 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.972+0000 7f458b7fe640 1 -- 172.21.15.76:0/1403932530 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45800aa9d0 con 0x7f458c15bec0 2024-09-19T04:19:56.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- 172.21.15.76:0/1403932530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 msgr2=0x7f458c0ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 --2- 172.21.15.76:0/1403932530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c0ad200 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f45800097f0 tx=0x7f458001c4b0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- 172.21.15.76:0/1403932530 shutdown_connections 2024-09-19T04:19:56.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 --2- 172.21.15.76:0/1403932530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c0ad200 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- 172.21.15.76:0/1403932530 >> 172.21.15.76:0/1403932530 conn(0x7f458c0b1f90 msgr2=0x7f458c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- 172.21.15.76:0/1403932530 shutdown_connections 2024-09-19T04:19:56.976 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- 172.21.15.76:0/1403932530 wait complete. 2024-09-19T04:19:56.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 Processor -- start 2024-09-19T04:19:56.977 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- start start 2024-09-19T04:19:56.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f458c0af470 0x7f458c1f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.978 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c09dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f458c15c2a0 0x7f458c0998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f458c06ef20 con 0x7f458c15c2a0 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f458c0ad740 con 0x7f458c0af470 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f458c06f0a0 con 0x7f458c15bec0 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f458c15c2a0 0x7f458c0998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f458c15c2a0 0x7f458c0998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33202/0 (socket says 172.21.15.76:33202) 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 -- 172.21.15.76:0/415548860 learned_addr learned my addr 172.21.15.76:0/415548860 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4590cdb640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f458c0af470 0x7f458c1f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f458bfff640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c09dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 -- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 msgr2=0x7f458c09dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c09dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 -- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f458c0af470 msgr2=0x7f458c1f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.979 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f458c0af470 0x7f458c1f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 -- 172.21.15.76:0/415548860 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c09bcf0 con 0x7f458c15c2a0 2024-09-19T04:19:56.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45914dc640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f458c15c2a0 0x7f458c0998b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f458400c760 tx=0x7f458400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45897fa640 1 -- 172.21.15.76:0/415548860 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584003b80 con 0x7f458c15c2a0 2024-09-19T04:19:56.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45897fa640 1 -- 172.21.15.76:0/415548860 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4584003d20 con 0x7f458c15c2a0 2024-09-19T04:19:56.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c09bfd0 con 0x7f458c15c2a0 2024-09-19T04:19:56.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f45897fa640 1 -- 172.21.15.76:0/415548860 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458400aa30 con 0x7f458c15c2a0 2024-09-19T04:19:56.980 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.976+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c09c590 con 0x7f458c15c2a0 2024-09-19T04:19:56.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f45897fa640 1 -- 172.21.15.76:0/415548860 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f458400abd0 con 0x7f458c15c2a0 2024-09-19T04:19:56.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f45897fa640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45700678c0 0x7f4570069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f45897fa640 1 -- 172.21.15.76:0/415548860 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f4584085d80 con 0x7f458c15c2a0 2024-09-19T04:19:56.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4590cdb640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45700678c0 0x7f4570069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.981 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4590cdb640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45700678c0 0x7f4570069d80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f45800ae000 tx=0x7f45800a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 Processor -- start 2024-09-19T04:19:56.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 -- start start 2024-09-19T04:19:56.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fc1ab20 0x55798fc1af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fbd0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fbd1290 0x55798fbd1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55798fab8b60 con 0x55798fc1b440 2024-09-19T04:19:56.984 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55798fa1fc70 con 0x55798fc1ab20 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55798f9c5ed0 con 0x55798fbd1290 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fbd0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fbd0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33206/0 (socket says 172.21.15.76:33206) 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 -- 172.21.15.76:0/901125225 learned_addr learned my addr 172.21.15.76:0/901125225 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f4590cdb640 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fc1ab20 0x55798fc1af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 -- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fbd1290 msgr2=0x55798fbd1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f45914dc640 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fbd1290 0x55798fbd1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fbd1290 0x55798fbd1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 -- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fc1ab20 msgr2=0x55798fc1af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fc1ab20 0x55798fc1af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 -- 172.21.15.76:0/901125225 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798f9d7e50 con 0x55798fc1b440 2024-09-19T04:19:56.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f458bfff640 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fbd0d50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f458c0ae510 tx=0x7f4578001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe74fb136a86461 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f45767fc640 1 -- 172.21.15.76:0/901125225 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578019070 con 0x55798fc1b440 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f45767fc640 1 -- 172.21.15.76:0/901125225 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45780058c0 con 0x55798fc1b440 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.980+0000 7f45767fc640 1 -- 172.21.15.76:0/901125225 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578005c70 con 0x55798fc1b440 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/901125225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 msgr2=0x55798fbd0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fbd0d50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f458c0ae510 tx=0x7f4578001a40 comp rx=0 tx=0).stop 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/901125225 shutdown_connections 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fbd1290 0x55798fbd1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fbd0d50 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- 172.21.15.76:0/901125225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fc1ab20 0x55798fc1af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/901125225 >> 172.21.15.76:0/901125225 conn(0x55798fbd7600 msgr2=0x55798fbcf630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/901125225 shutdown_connections 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/901125225 wait complete. 2024-09-19T04:19:56.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- start start 2024-09-19T04:19:56.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 Processor -- start 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fbd5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 0x55798fbdbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fbdc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55798fa81b00 con 0x55798fc1ab20 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55798f9fa360 con 0x55798fbd6330 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55798f9f72d0 con 0x55798fc1b440 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 10 client.0 _ll_register_callbacks cb 0x55798fbbc820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fbd5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fbd5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33218/0 (socket says 172.21.15.76:33218) 2024-09-19T04:19:56.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 -- 172.21.15.76:0/3666572450 learned_addr learned my addr 172.21.15.76:0/3666572450 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f45914dc640 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fbdc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 -- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 msgr2=0x55798fbdbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 0x55798fbdbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 -- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 msgr2=0x55798fbdc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fbdc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 -- 172.21.15.76:0/3666572450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798f9c6120 con 0x55798fc1ab20 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f45914dc640 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fbdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4590cdb640 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fbd5df0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f45800ac040 tx=0x7f458001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de0c9f12a51549c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f456b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f456b7fe640 1 -- 172.21.15.76:0/3666572450 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45800ba070 con 0x55798fc1ab20 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f456b7fe640 1 -- 172.21.15.76:0/3666572450 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f45800bfce0 con 0x55798fc1ab20 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f45780058c0 con 0x55798fc1ab20 2024-09-19T04:19:56.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f456b7fe640 1 -- 172.21.15.76:0/3666572450 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f45800b4400 con 0x55798fc1ab20 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 10 client.8343 fetch_fsmap learned FSMap version 107 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55798f9bf450 con 0x55798fc1ab20 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f456b7fe640 1 -- 172.21.15.76:0/3666572450 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45780058c0 con 0x55798fc1ab20 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f456b7fe640 1 -- 172.21.15.76:0/3666572450 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 107) ==== 3542+0+0 (secure 0 0 0) 0x7f45800b45a0 con 0x55798fc1ab20 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 10 client.8343 fetch_fsmap finished waiting for FSMap version 107 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 10 client.8343 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 20 client.8343 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc29280 0x55798fc2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 4 client.8343 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798f9b9c00 con 0x55798fc29280 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.984+0000 7f458bfff640 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc29280 0x55798fc2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f456b7fe640 10 client.8343 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:56.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f458bfff640 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc29280 0x55798fc2b670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55c6c1f180edb5d server_cookie=e02493b8022d933d in_seq=0 out_seq=0 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f456b7fe640 1 -- 172.21.15.76:0/3666572450 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55798f9b9c00 con 0x55798fc29280 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f456b7fe640 10 client.8343 handle_command_reply: tid=0 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 client.8343 shutdown 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 msgr2=0x55798fbd5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fbd5df0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f45800ac040 tx=0x7f458001acf0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 shutdown_connections 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc29280 0x55798fc2b670 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fbdc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 0x55798fbdbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f456b7fe640 0 client.8343 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3666572450 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fbd5df0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 >> 172.21.15.76:0/3666572450 conn(0x55798fbd1290 msgr2=0x55798fbcfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:56.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 shutdown_connections 2024-09-19T04:19:56.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 1 -- 172.21.15.76:0/3666572450 wait complete. 2024-09-19T04:19:56.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:56.988+0000 7f4595ffb000 20 client.8343 trim_cache size 0 max 16384 2024-09-19T04:19:57.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 Processor -- start 2024-09-19T04:19:57.006 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 -- start start 2024-09-19T04:19:57.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc19c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1ab20 0x55798fc1a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fc2b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f458c203160 con 0x55798fc1b440 2024-09-19T04:19:57.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f458c203810 con 0x55798fbd6330 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f458c153a70 con 0x55798fc1ab20 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f45914dc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fc2b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f45914dc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fc2b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33230/0 (socket says 172.21.15.76:33230) 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.004+0000 7f45914dc640 1 -- 172.21.15.76:0/2982510723 learned_addr learned my addr 172.21.15.76:0/2982510723 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4590cdb640 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc19c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f45914dc640 1 -- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1ab20 msgr2=0x55798fc1a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f45914dc640 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1ab20 0x55798fc1a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f45914dc640 1 -- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 msgr2=0x55798fc19c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f45914dc640 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc19c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f45914dc640 1 -- 172.21.15.76:0/2982510723 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798f9a6fd0 con 0x55798fc1b440 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4590cdb640 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc19c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:19:57.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f45914dc640 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fc2b670 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f458405e640 tx=0x7f4584055e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a83f9ba72f15ebb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4575ffb640 1 -- 172.21.15.76:0/2982510723 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584055f90 con 0x55798fc1b440 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4575ffb640 1 -- 172.21.15.76:0/2982510723 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458405a340 con 0x55798fc1b440 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4575ffb640 1 -- 172.21.15.76:0/2982510723 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458405eb70 con 0x55798fc1b440 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/2982510723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 msgr2=0x55798fc2b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fc2b670 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f458405e640 tx=0x7f4584055e60 comp rx=0 tx=0).stop 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/2982510723 shutdown_connections 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1b440 0x55798fc2b670 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1ab20 0x55798fc1a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- 172.21.15.76:0/2982510723 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc19c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/2982510723 >> 172.21.15.76:0/2982510723 conn(0x55798fbd1290 msgr2=0x55798fbd55f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/2982510723 shutdown_connections 2024-09-19T04:19:57.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/2982510723 wait complete. 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- start start 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 Processor -- start 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fc49960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 0x55798fc49ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55798fa1ef80 con 0x55798fc1ab20 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55798f9d7ca0 con 0x55798fbd6330 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55798fa242a0 con 0x55798fc1b440 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 10 client.0 _ll_register_callbacks cb 0x7f4580009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:19:57.014 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fc49960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fc49960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33244/0 (socket says 172.21.15.76:33244) 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 -- 172.21.15.76:0/3684031425 learned_addr learned my addr 172.21.15.76:0/3684031425 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4590cdb640 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f45914dc640 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 0x55798fc49ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 -- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 msgr2=0x55798fc49ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 0x55798fc49ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 -- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 msgr2=0x55798fc2e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc2e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 -- 172.21.15.76:0/3684031425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798faa2750 con 0x55798fc1ab20 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f458bfff640 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fc49960 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4578017040 tx=0x7f4578001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5f118223744ac9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f456b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:19:57.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f456b7fe640 1 -- 172.21.15.76:0/3684031425 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4578019040 con 0x55798fc1ab20 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f456b7fe640 1 -- 172.21.15.76:0/3684031425 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7f45780070d0 con 0x55798fc1ab20 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f458405a340 con 0x55798fc1ab20 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f456b7fe640 1 -- 172.21.15.76:0/3684031425 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) ==== 24+0+0 (secure 0 0 0) 0x7f4578030050 con 0x55798fc1ab20 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 10 client.8349 fetch_fsmap learned FSMap version 107 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) -- 0x55798f9ada30 con 0x55798fc1ab20 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f456b7fe640 1 -- 172.21.15.76:0/3684031425 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f458405a340 con 0x55798fc1ab20 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f456b7fe640 1 -- 172.21.15.76:0/3684031425 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 107) ==== 3542+0+0 (secure 0 0 0) 0x7f4578033020 con 0x55798fc1ab20 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 10 client.8349 fetch_fsmap finished waiting for FSMap version 107 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 10 client.8349 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 20 client.8349 populate_metadata read hostname 'smithi076' 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc4a3e0 0x55798fc8de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 4 client.8349 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T04:19:57.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "status"}) -- 0x55798fa2d3a0 con 0x55798fc4a3e0 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.008+0000 7f4590cdb640 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc4a3e0 0x55798fc8de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f456b7fe640 10 client.8349 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4590cdb640 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc4a3e0 0x55798fc8de50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcb3d507d203a92 server_cookie=28181f67ef0bf87d in_seq=0 out_seq=0 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f456b7fe640 1 -- 172.21.15.76:0/3684031425 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55798fa2d3a0 con 0x55798fc4a3e0 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f456b7fe640 10 client.8349 handle_command_reply: tid=0 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 client.8349 shutdown 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 msgr2=0x55798fc49960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fc49960 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f4578017040 tx=0x7f4578001190 comp rx=0 tx=0).stop 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 shutdown_connections 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55798fc4a3e0 0x55798fc8de50 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55798fc1b440 0x55798fc49ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55798fc1ab20 0x55798fc49960 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 --2- 172.21.15.76:0/3684031425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55798fbd6330 0x55798fc2e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f456b7fe640 0 client.8349 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 >> 172.21.15.76:0/3684031425 conn(0x55798fbd1290 msgr2=0x55798fc18c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 shutdown_connections 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 1 -- 172.21.15.76:0/3684031425 wait complete. 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4595ffb000 20 client.8349 trim_cache size 0 max 16384 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stdout: "cluster_fsid": "2bcf595d-5a10-4e84-8b11-2533969843a0", 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stdout: "whoami": 0, 2024-09-19T04:19:57.018 INFO:teuthology.orchestra.run.smithi076.stdout: "id": 8127, 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "want_state": "up:active", 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "state": "up:active", 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "fs_name": "cephfs", 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "rank_uptime": 11.815344176, 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "mdsmap_epoch": 107, 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch": 104, 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "osdmap_epoch_barrier": 104, 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout: "uptime": 15.892422592999999 2024-09-19T04:19:57.019 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:19:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45700678c0 msgr2=0x7f4570069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45700678c0 0x7f4570069d80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f45800ae000 tx=0x7f45800a7000 comp rx=0 tx=0).stop 2024-09-19T04:19:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f458c15c2a0 msgr2=0x7f458c0998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f458c15c2a0 0x7f458c0998b0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f458400c760 tx=0x7f458400cc20 comp rx=0 tx=0).stop 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 shutdown_connections 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f45700678c0 0x7f4570069d80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f458c15c2a0 0x7f458c0998b0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f458c15bec0 0x7f458c09dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 --2- 172.21.15.76:0/415548860 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f458c0af470 0x7f458c1f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 >> 172.21.15.76:0/415548860 conn(0x7f458c0b1f90 msgr2=0x7f458c0b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 shutdown_connections 2024-09-19T04:19:57.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.012+0000 7f4592f3f640 1 -- 172.21.15.76:0/415548860 wait complete. 2024-09-19T04:19:57.028 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:19:57.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8fdf31640 1 Processor -- start 2024-09-19T04:19:57.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8fdf31640 1 -- start start 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8fdf31640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe8f80cdb00 0x7fe8f80b3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8fdf31640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f80b3ba0 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8fdf31640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8f80ce140 con 0x7fe8f80b3280 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8fdf31640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe8f80ce420 con 0x7fe8f80cdb00 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe8f80cdb00 0x7fe8f80b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe8f80cdb00 0x7fe8f80b3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33246/0 (socket says 172.21.15.76:33246) 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f6ffd640 1 -- 172.21.15.76:0/1036447717 learned_addr learned my addr 172.21.15.76:0/1036447717 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f7fff640 1 --1- 172.21.15.76:0/1036447717 >> v1:172.21.15.76:6789/0 conn(0x7fe8f80cf1e0 0x7fe8f815d810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45342/0 (socket says 172.21.15.76:45342) 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f6ffd640 1 -- 172.21.15.76:0/1036447717 >> v1:172.21.15.116:6789/0 conn(0x7fe8f80b3280 legacy=0x7fe8f815d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f67fc640 1 -- 172.21.15.76:0/1036447717 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4226091085 0 0) 0x7fe8f80b3ba0 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f6ffd640 1 -- 172.21.15.76:0/1036447717 >> v1:172.21.15.76:6789/0 conn(0x7fe8f80cf1e0 legacy=0x7fe8f815d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.140+0000 7fe8f6ffd640 1 -- 172.21.15.76:0/1036447717 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f80ce700 con 0x7fe8f80cdb00 2024-09-19T04:19:57.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8f6ffd640 1 --2- 172.21.15.76:0/1036447717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cdb00 0x7fe8f80b3660 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac168e91499e839 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:57.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8f67fc640 1 -- 172.21.15.76:0/1036447717 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00b1070 con 0x7fe8f80cdb00 2024-09-19T04:19:57.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8f67fc640 1 -- 172.21.15.76:0/1036447717 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8e00ac620 con 0x7fe8f80cdb00 2024-09-19T04:19:57.144 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8f67fc640 1 -- 172.21.15.76:0/1036447717 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00ac9f0 con 0x7fe8f80cdb00 2024-09-19T04:19:57.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 -- 172.21.15.76:0/1036447717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cdb00 msgr2=0x7fe8f80b3660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/1036447717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cdb00 0x7fe8f80b3660 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0009870 tx=0x7fe8e001c4d0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 -- 172.21.15.76:0/1036447717 shutdown_connections 2024-09-19T04:19:57.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/1036447717 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cdb00 0x7fe8f80b3660 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 -- 172.21.15.76:0/1036447717 >> 172.21.15.76:0/1036447717 conn(0x7fe8f80b6870 msgr2=0x7fe8f80b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:57.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 -- 172.21.15.76:0/1036447717 shutdown_connections 2024-09-19T04:19:57.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 -- 172.21.15.76:0/1036447717 wait complete. 2024-09-19T04:19:57.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 Processor -- start 2024-09-19T04:19:57.146 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.144+0000 7fe8fdf31640 1 -- start start 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8f80b3280 0x7fe8f814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f80cdb00 0x7fe8f81366e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cf1e0 0x7fe8f806cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f815a060 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f8159ee0 con 0x7fe8f80cdb00 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe8f816a4b0 con 0x7fe8f80b3280 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cf1e0 0x7fe8f806cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cf1e0 0x7fe8f806cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:33252/0 (socket says 172.21.15.76:33252) 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 -- 172.21.15.76:0/2672336361 learned_addr learned my addr 172.21.15.76:0/2672336361 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f77fe640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8f80b3280 0x7fe8f814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f6ffd640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f80cdb00 0x7fe8f81366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 -- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8f80b3280 msgr2=0x7fe8f814c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8f80b3280 0x7fe8f814c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 -- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f80cdb00 msgr2=0x7fe8f81366e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f80cdb00 0x7fe8f81366e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 -- 172.21.15.76:0/2672336361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f8139380 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f6ffd640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f80cdb00 0x7fe8f81366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f7fff640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cf1e0 0x7fe8f806cda0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c780 tx=0x7fe8f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8effff640 1 -- 172.21.15.76:0/2672336361 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0018070 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8effff640 1 -- 172.21.15.76:0/2672336361 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8f000a640 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f8139660 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8effff640 1 -- 172.21.15.76:0/2672336361 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f000aa30 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f8156970 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8f81562c0 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8effff640 1 -- 172.21.15.76:0/2672336361 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe8f001b070 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8effff640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8d80678c0 0x7fe8d8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:19:57.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8effff640 1 -- 172.21.15.76:0/2672336361 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 6101+0+0 (secure 0 0 0) 0x7fe8f0085450 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f77fe640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8d80678c0 0x7fe8d8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:19:57.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.148+0000 7fe8f77fe640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8d80678c0 0x7fe8d8069d80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8001650 tx=0x7fe8e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:19:57.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.152+0000 7fe8effff640 1 -- 172.21.15.76:0/2672336361 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe8f0055e00 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8f80b3660 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8effff640 1 -- 172.21.15.76:0/2672336361 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5270 (secure 0 0 0) 0x7fe8f0055700 con 0x7fe8f80cf1e0 2024-09-19T04:19:57.345 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:19:57.345 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":107,"btime":"2024-09-19T04:19:55:962543+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:19:55.962531+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[6127,8115,8127]},"id":7}]} 2024-09-19T04:19:57.345 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 107 2024-09-19T04:19:57.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8d80678c0 msgr2=0x7fe8d8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8d80678c0 0x7fe8d8069d80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8001650 tx=0x7fe8e8009040 comp rx=0 tx=0).stop 2024-09-19T04:19:57.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cf1e0 msgr2=0x7fe8f806cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:19:57.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cf1e0 0x7fe8f806cda0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c780 tx=0x7fe8f000cc40 comp rx=0 tx=0).stop 2024-09-19T04:19:57.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 shutdown_connections 2024-09-19T04:19:57.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe8d80678c0 0x7fe8d8069d80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe8f80cf1e0 0x7fe8f806cda0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe8f80cdb00 0x7fe8f81366e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 --2- 172.21.15.76:0/2672336361 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe8f80b3280 0x7fe8f814c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:19:57.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 >> 172.21.15.76:0/2672336361 conn(0x7fe8f80b6870 msgr2=0x7fe8f80b6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:19:57.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 shutdown_connections 2024-09-19T04:19:57.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:19:57.344+0000 7fe8fdf31640 1 -- 172.21.15.76:0/2672336361 wait complete. 2024-09-19T04:19:57.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:57.362 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:19:57.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:57.396 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec sync d1/d2/d3/d4/d5/d6/d7/d8) 2024-09-19T04:19:57.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:57.453 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 0 d1/d2) 2024-09-19T04:19:57.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:57.506 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1/d2/d3/d4) 2024-09-19T04:19:57.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:19:57.555 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 2 d1/d2/d3/d4/d5/d6) 2024-09-19T04:19:57.609 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:19:57.796 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:19:57.796 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:57.797 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:19:58.148 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:19:58.148+0000 7f3e47568640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:19:59.798 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:19:59.944 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"5","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":1,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":1},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:19:59.945 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 1, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 1, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "5" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:19:59.945 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:20:01.947 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:20:02.094 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:20:02.095 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:20:02.095 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:20:04.096 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:20:04.244 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"7","version":"7","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:20:04.245 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "7", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "7" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:20:04.245 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:20:06.246 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:20:06.394 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":0,"dirty":1,"authpin":0},"nref":3}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":1,"authpin":0},"nref":3}}] 2024-09-19T04:20:06.394 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 3, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 0, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 3, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:20:06.395 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:20:08.395 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json get subtrees 2024-09-19T04:20:08.541 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"13","version":"13","committing_version":"1","committed_version":"1","is_rep":true,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds0","dirfrag":"0x100","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"0","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":true,"auth_first":0,"auth_second":-2,"export_pin":0,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":0,"dir":{"path":"/d1/d2","dirfrag":"0x10000000001","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"0","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[0,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"replicated":1,"dirty":1,"authpin":0},"nref":4}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":1,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":2}}] 2024-09-19T04:20:08.542 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x1", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 4, "path": "", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "13", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "0", "dirfrag": "0x100", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds0", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x10000000001", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2", "pins": { "authpin": 0, "child": 1, "dirty": 1, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 0, "export_pin_target": 0, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "authpin": 0, "child": 1, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false } ] 2024-09-19T04:20:08.542 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] =?= [('/d1/d2', 0), ('/d1/d2/d3/d4', 1)] 2024-09-19T04:20:08.543 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:20:08.682 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:20:08.683 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:20:08.683 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:20:10.684 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:20:10.831 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:20:10.832 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:20:10.832 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:20:12.833 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json get subtrees 2024-09-19T04:20:12.975 INFO:teuthology.orchestra.run.smithi076.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"14","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1,"2":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":false,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"request":0,"child":0,"frozen":0,"subtree":1,"replicated":0,"dirty":0,"waiter":0,"authpin":0,"tempexporting":0},"nref":1}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:20:12.976 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1, "2": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "14", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 1, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "authpin": 0, "child": 0, "dirty": 0, "frozen": 0, "ptrwaiter": 0, "replicated": 0, "request": 0, "subtree": 1, "tempexporting": 0, "waiter": 0 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:20:12.976 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:20:12.976 DEBUG:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2024-09-19T04:20:13.127 INFO:teuthology.orchestra.run.smithi116.stdout:[{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"13","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":false,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1/d2/d3/d4","dirfrag":"0x10000000003","snapid_first":2,"projected_version":"0","version":"14","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"dnwaiter":0,"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":2,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":2,"dir":{"path":"/d1/d2/d3/d4/d5/d6","dirfrag":"0x10000000005","snapid_first":2,"projected_version":"6","version":"6","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"2","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"1":1}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}},{"is_auth":true,"auth_first":2,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds2","dirfrag":"0x102","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"2","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[2,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2024-09-19T04:20:13.128 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "13" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000003", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "/d1/d2/d3/d4", "pins": { "child": 1, "dnwaiter": 0, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "14" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "1": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "2", "dirfrag": "0x10000000005", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1/d2/d3/d4/d5/d6", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "6", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "6" }, "distributed_ephemeral_pin": false, "export_pin": 2, "export_pin_target": 2, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 2, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "2", "dirfrag": "0x102", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds2", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 2, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2024-09-19T04:20:13.128 INFO:tasks.cephfs.cephfs_test_case:[('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] =?= [('/d1/d2/d3/d4', 1), ('/d1/d2/d3/d4/d5/d6', 2)] 2024-09-19T04:20:13.128 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:0 flush journal 2024-09-19T04:20:13.261 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ee74b640 1 Processor -- start 2024-09-19T04:20:13.262 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ee74b640 1 -- start start 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ee74b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e8144d00 0x7fb1e8070880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ee74b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1e806c3d0 0x7fb1e80af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ee74b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1e80b5b90 con 0x7fb1e80b41c0 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ee74b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb1e8070dc0 con 0x7fb1e806c3d0 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ee74b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e80aec50 con 0x7fb1e8144d00 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ecce8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1e806c3d0 0x7fb1e80af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ecce8640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb1e806c3d0 0x7fb1e80af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51226/0 (socket says 172.21.15.76:51226) 2024-09-19T04:20:13.263 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.260+0000 7fb1ecce8640 1 -- 172.21.15.76:0/139339009 learned_addr learned my addr 172.21.15.76:0/139339009 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/139339009 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e8144d00 0x7fb1e8070880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ecce8640 1 -- 172.21.15.76:0/139339009 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e8144d00 msgr2=0x7fb1e8070880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/139339009 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e8144d00 0x7fb1e8070880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ecce8640 1 -- 172.21.15.76:0/139339009 >> v1:172.21.15.116:6789/0 conn(0x7fb1e80b41c0 legacy=0x7fb1e806bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.264 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ecce8640 1 -- 172.21.15.76:0/139339009 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e80b5e30 con 0x7fb1e806c3d0 2024-09-19T04:20:13.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/139339009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e806c3d0 0x7fb1e80af0e0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009870 tx=0x7fb1d001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9419b81ffb470a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/139339009 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00b0070 con 0x7fb1e806c3d0 2024-09-19T04:20:13.265 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/139339009 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d00ab600 con 0x7fb1e806c3d0 2024-09-19T04:20:13.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1e6ffd640 1 -- 172.21.15.76:0/139339009 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00ab9d0 con 0x7fb1e806c3d0 2024-09-19T04:20:13.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 -- 172.21.15.76:0/139339009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e806c3d0 msgr2=0x7fb1e80af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/139339009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e806c3d0 0x7fb1e80af0e0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0009870 tx=0x7fb1d001c490 comp rx=0 tx=0).stop 2024-09-19T04:20:13.266 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 -- 172.21.15.76:0/139339009 shutdown_connections 2024-09-19T04:20:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/139339009 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e806c3d0 0x7fb1e80af0e0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/139339009 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e8144d00 0x7fb1e8070880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 -- 172.21.15.76:0/139339009 >> 172.21.15.76:0/139339009 conn(0x7fb1e80b6650 msgr2=0x7fb1e80b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 -- 172.21.15.76:0/139339009 shutdown_connections 2024-09-19T04:20:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.264+0000 7fb1ee74b640 1 -- 172.21.15.76:0/139339009 wait complete. 2024-09-19T04:20:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 Processor -- start 2024-09-19T04:20:13.267 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 -- start start 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e806c3d0 0x7fb1e8168450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e80b41c0 0x7fb1e8158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e8144d00 0x7fb1e8142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80affb0 con 0x7fb1e80b41c0 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80afe30 con 0x7fb1e8144d00 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e80b0130 con 0x7fb1e806c3d0 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e80b41c0 0x7fb1e8158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.270 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e80b41c0 0x7fb1e8158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51228/0 (socket says 172.21.15.76:51228) 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 -- 172.21.15.76:0/3915173886 learned_addr learned my addr 172.21.15.76:0/3915173886 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e8144d00 0x7fb1e8142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e806c3d0 0x7fb1e8168450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 -- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e806c3d0 msgr2=0x7fb1e8168450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e806c3d0 0x7fb1e8168450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 -- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e8144d00 msgr2=0x7fb1e8142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e8144d00 0x7fb1e8142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 -- 172.21.15.76:0/3915173886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e81430e0 con 0x7fb1e80b41c0 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e8144d00 0x7fb1e8142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e806c3d0 0x7fb1e8168450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e80b41c0 0x7fb1e8158070 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c440 tx=0x7fb1e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.271 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e4ff9640 1 -- 172.21.15.76:0/3915173886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0016020 con 0x7fb1e80b41c0 2024-09-19T04:20:13.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e4ff9640 1 -- 172.21.15.76:0/3915173886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1e0012830 con 0x7fb1e80b41c0 2024-09-19T04:20:13.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e809b3c0 con 0x7fb1e80b41c0 2024-09-19T04:20:13.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1e4ff9640 1 -- 172.21.15.76:0/3915173886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0012c20 con 0x7fb1e80b41c0 2024-09-19T04:20:13.272 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.268+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e809b980 con 0x7fb1e80b41c0 2024-09-19T04:20:13.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e4ff9640 1 -- 172.21.15.76:0/3915173886 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb1e0019020 con 0x7fb1e80b41c0 2024-09-19T04:20:13.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e4ff9640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1c80678c0 0x7fb1c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e4ff9640 1 -- 172.21.15.76:0/3915173886 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fb1e0085b10 con 0x7fb1e80b41c0 2024-09-19T04:20:13.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1c80678c0 0x7fb1c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1c80678c0 0x7fb1c8069d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb1e809cec0 tx=0x7fb1d8007420 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 Processor -- start 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- start start 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d407150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d407690 0x55729d3ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d3bb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55729d2a4960 con 0x55729d406d70 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55729d1e8900 con 0x55729d3badb0 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55729d20bd40 con 0x55729d407690 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d407150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d407150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51236/0 (socket says 172.21.15.76:51236) 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 -- 172.21.15.76:0/2108071242 learned_addr learned my addr 172.21.15.76:0/2108071242 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d3bb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d407690 0x55729d3ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 -- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d407690 msgr2=0x55729d3ba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d407690 0x55729d3ba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 -- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 msgr2=0x55729d3bb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d3bb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 -- 172.21.15.76:0/2108071242 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55729d1b2580 con 0x55729d406d70 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d3bb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d407150 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8007800 tx=0x7fb1d8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55311164bc4efb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1ddffb640 1 -- 172.21.15.76:0/2108071242 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d801d440 con 0x55729d406d70 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1ddffb640 1 -- 172.21.15.76:0/2108071242 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d8022070 con 0x55729d406d70 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1ddffb640 1 -- 172.21.15.76:0/2108071242 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d801d7e0 con 0x55729d406d70 2024-09-19T04:20:13.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- 172.21.15.76:0/2108071242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 msgr2=0x55729d407150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d407150 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8007800 tx=0x7fb1d8010000 comp rx=0 tx=0).stop 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- 172.21.15.76:0/2108071242 shutdown_connections 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d3bb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d407690 0x55729d3ba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 --2- 172.21.15.76:0/2108071242 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d407150 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- 172.21.15.76:0/2108071242 >> 172.21.15.76:0/2108071242 conn(0x55729d3e8040 msgr2=0x55729d3ea430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- 172.21.15.76:0/2108071242 shutdown_connections 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.272+0000 7fb1f1807000 1 -- 172.21.15.76:0/2108071242 wait complete. 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- start start 2024-09-19T04:20:13.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 Processor -- start 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3badb0 0x55729d3c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d3c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d407690 0x55729d3e8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55729d207d00 con 0x55729d3badb0 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55729d1c6aa0 con 0x55729d407690 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55729d1a5340 con 0x55729d406d70 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 10 client.0 _ll_register_callbacks cb 0x55729d3a8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3badb0 0x55729d3c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3badb0 0x55729d3c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51252/0 (socket says 172.21.15.76:51252) 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 -- 172.21.15.76:0/3719992745 learned_addr learned my addr 172.21.15.76:0/3719992745 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 -- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 msgr2=0x55729d3c2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d407690 0x55729d3e8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d3c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d3c2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 -- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d407690 msgr2=0x55729d3e8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d407690 0x55729d3e8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 -- 172.21.15.76:0/3719992745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55729d1aedc0 con 0x55729d3badb0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d407690 0x55729d3e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3badb0 0x55729d3c25a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x55729d1cfc90 tx=0x7fb1d8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80523c12ac852404 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/3719992745 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d801dc00 con 0x55729d3badb0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/3719992745 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fb1d801dda0 con 0x55729d3badb0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d8022070 con 0x55729d3badb0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/3719992745 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fb1d8039050 con 0x55729d3badb0 2024-09-19T04:20:13.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 10 client.8370 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55729d1ab7f0 con 0x55729d3badb0 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/3719992745 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d8022070 con 0x55729d3badb0 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/3719992745 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7fb1d803c020 con 0x55729d3badb0 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 10 client.8370 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 10 client.8370 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 20 client.8370 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d3e8e30 0x55729d418020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 4 client.8370 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55729d1a5e10 con 0x55729d3e8e30 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d3e8e30 0x55729d418020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 10 client.8370 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d3e8e30 0x55729d418020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7230a08c64380b7d server_cookie=63e7cc2dfe7e3e05 in_seq=0 out_seq=0 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/3719992745 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55729d1a5e10 con 0x55729d3e8e30 2024-09-19T04:20:13.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 10 client.8370 handle_command_reply: tid=0 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 client.8370 shutdown 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3badb0 msgr2=0x55729d3c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3badb0 0x55729d3c25a0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x55729d1cfc90 tx=0x7fb1d8004ac0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 shutdown_connections 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d3e8e30 0x55729d418020 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d407690 0x55729d3e8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d3c2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1c6ffd640 0 client.8370 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 --2- 172.21.15.76:0/3719992745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3badb0 0x55729d3c25a0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.276+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 >> 172.21.15.76:0/3719992745 conn(0x55729d3e8040 msgr2=0x55729d3b9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.280+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 shutdown_connections 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.280+0000 7fb1f1807000 1 -- 172.21.15.76:0/3719992745 wait complete. 2024-09-19T04:20:13.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.280+0000 7fb1f1807000 20 client.8370 trim_cache size 0 max 16384 2024-09-19T04:20:13.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.292+0000 7fb1f1807000 1 Processor -- start 2024-09-19T04:20:13.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.292+0000 7fb1f1807000 1 -- start start 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3e8040 0x55729d3e6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d3badb0 0x55729d3e7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d3c2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8204200 con 0x55729d406d70 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e82033d0 con 0x55729d3e8040 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e8203ad0 con 0x55729d3badb0 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d3c2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d3c2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51266/0 (socket says 172.21.15.76:51266) 2024-09-19T04:20:13.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 -- 172.21.15.76:0/185805582 learned_addr learned my addr 172.21.15.76:0/185805582 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 -- 172.21.15.76:0/185805582 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d3badb0 msgr2=0x55729d3e7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/185805582 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d3badb0 0x55729d3e7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 -- 172.21.15.76:0/185805582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3e8040 msgr2=0x55729d3e6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/185805582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3e8040 0x55729d3e6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 -- 172.21.15.76:0/185805582 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55729d1c9470 con 0x55729d406d70 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/185805582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d3c2330 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb1d001cd30 tx=0x7fb1d0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebf167c0bbf4b37 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1dd7fa640 1 -- 172.21.15.76:0/185805582 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00b0040 con 0x55729d406d70 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1dd7fa640 1 -- 172.21.15.76:0/185805582 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d00ab810 con 0x55729d406d70 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1dd7fa640 1 -- 172.21.15.76:0/185805582 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d00abbe0 con 0x55729d406d70 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- 172.21.15.76:0/185805582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 msgr2=0x55729d3c2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- 172.21.15.76:0/185805582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d3c2330 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb1d001cd30 tx=0x7fb1d0004740 comp rx=0 tx=0).stop 2024-09-19T04:20:13.298 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- 172.21.15.76:0/185805582 shutdown_connections 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- 172.21.15.76:0/185805582 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d406d70 0x55729d3c2330 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- 172.21.15.76:0/185805582 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d3badb0 0x55729d3e7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- 172.21.15.76:0/185805582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3e8040 0x55729d3e6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- 172.21.15.76:0/185805582 >> 172.21.15.76:0/185805582 conn(0x55729d3bf890 msgr2=0x55729d406180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- 172.21.15.76:0/185805582 shutdown_connections 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- 172.21.15.76:0/185805582 wait complete. 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- start start 2024-09-19T04:20:13.299 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 Processor -- start 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d47b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3e8040 0x55729d47bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d4194d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55729d2c80c0 con 0x55729d3e8040 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55729d20b050 con 0x55729d3badb0 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55729d207d00 con 0x55729d406d70 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 10 client.0 _ll_register_callbacks cb 0x7fb1d8007c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3e8040 0x55729d47bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.302 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3e8040 0x55729d47bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51280/0 (socket says 172.21.15.76:51280) 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 -- 172.21.15.76:0/255794796 learned_addr learned my addr 172.21.15.76:0/255794796 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d4194d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d47b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 -- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 msgr2=0x55729d4194d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d4194d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 -- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 msgr2=0x55729d47b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d47b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 -- 172.21.15.76:0/255794796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55729d19f640 con 0x55729d3e8040 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1ecce8640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d4194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d47b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1e77fe640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3e8040 0x55729d47bc60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c410 tx=0x7fb1e005a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795454a839b476e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/255794796 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0052490 con 0x55729d3e8040 2024-09-19T04:20:13.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/255794796 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fb1e0055270 con 0x55729d3e8040 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d00ab810 con 0x55729d3e8040 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/255794796 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fb1e0052630 con 0x55729d3e8040 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 10 client.8376 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.296+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55729d1a33b0 con 0x55729d3e8040 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/255794796 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1d00ab810 con 0x55729d3e8040 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/255794796 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7fb1e005c3d0 con 0x55729d3e8040 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1f1807000 10 client.8376 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1f1807000 10 client.8376 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1f1807000 20 client.8376 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1f1807000 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d407690 0x55729d419a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1f1807000 4 client.8376 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55729d219470 con 0x55729d407690 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d407690 0x55729d419a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1c6ffd640 10 client.8376 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:13.305 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.300+0000 7fb1e7fff640 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d407690 0x55729d419a10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac3868c4a613468 server_cookie=5ed5608d1a17b947 in_seq=0 out_seq=0 2024-09-19T04:20:13.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1c6ffd640 1 -- 172.21.15.76:0/255794796 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55729d219470 con 0x55729d407690 2024-09-19T04:20:13.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1c6ffd640 10 client.8376 handle_command_reply: tid=0 2024-09-19T04:20:13.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 client.8376 shutdown 2024-09-19T04:20:13.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3e8040 msgr2=0x55729d47bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3e8040 0x55729d47bc60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c410 tx=0x7fb1e005a290 comp rx=0 tx=0).stop 2024-09-19T04:20:13.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 shutdown_connections 2024-09-19T04:20:13.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55729d407690 0x55729d419a10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55729d406d70 0x55729d4194d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1c6ffd640 0 client.8376 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55729d3e8040 0x55729d47bc60 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 --2- 172.21.15.76:0/255794796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55729d3badb0 0x55729d47b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.316+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 >> 172.21.15.76:0/255794796 conn(0x55729d3bf890 msgr2=0x55729d3e5d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 shutdown_connections 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1f1807000 1 -- 172.21.15.76:0/255794796 wait complete. 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1f1807000 20 client.8376 trim_cache size 0 max 16384 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:13.320 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:20:13.321 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:20:13.321 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:13.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1c80678c0 msgr2=0x7fb1c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1c80678c0 0x7fb1c8069d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fb1e809cec0 tx=0x7fb1d8007420 comp rx=0 tx=0).stop 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e80b41c0 msgr2=0x7fb1e8158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e80b41c0 0x7fb1e8158070 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c440 tx=0x7fb1e000c900 comp rx=0 tx=0).stop 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 shutdown_connections 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb1c80678c0 0x7fb1c8069d80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e8144d00 0x7fb1e8142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb1e80b41c0 0x7fb1e8158070 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 --2- 172.21.15.76:0/3915173886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb1e806c3d0 0x7fb1e8168450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 >> 172.21.15.76:0/3915173886 conn(0x7fb1e80b6650 msgr2=0x7fb1e8161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 shutdown_connections 2024-09-19T04:20:13.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.320+0000 7fb1ee74b640 1 -- 172.21.15.76:0/3915173886 wait complete. 2024-09-19T04:20:13.336 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:1 flush journal 2024-09-19T04:20:13.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a6ad24640 1 Processor -- start 2024-09-19T04:20:13.452 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a6ad24640 1 -- start start 2024-09-19T04:20:13.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a6ad24640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8a640d3060 0x7f8a640cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a6ad24640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a640cf450 0x7f8a6406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a6ad24640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a640d0300 con 0x7f8a64161a60 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a6ad24640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8a640b0f60 con 0x7f8a640cf450 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a6ad24640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8a640cfd40 con 0x7f8a640d3060 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a692c1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8a640cf450 0x7f8a6406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a68ac0640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8a64161a60 0x7f8a640d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51750/0 (socket says 172.21.15.76:51750) 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a68ac0640 1 -- 172.21.15.76:0/461028581 learned_addr learned my addr 172.21.15.76:0/461028581 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a63fff640 1 --2- 172.21.15.76:0/461028581 >> v2:172.21.15.116:3300/0 conn(0x7f8a640d3060 0x7f8a640cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a692c1640 1 -- 172.21.15.76:0/461028581 >> v2:172.21.15.116:3300/0 conn(0x7f8a640d3060 msgr2=0x7f8a640cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a692c1640 1 --2- 172.21.15.76:0/461028581 >> v2:172.21.15.116:3300/0 conn(0x7f8a640d3060 0x7f8a640cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a692c1640 1 -- 172.21.15.76:0/461028581 >> v1:172.21.15.76:6789/0 conn(0x7f8a64161a60 legacy=0x7f8a640d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.452+0000 7f8a692c1640 1 -- 172.21.15.76:0/461028581 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a640d0020 con 0x7f8a640cf450 2024-09-19T04:20:13.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a692c1640 1 --2- 172.21.15.76:0/461028581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a6406d6b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8a6408d850 tx=0x7f8a5401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19590614e1d8dcb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a637fe640 1 -- 172.21.15.76:0/461028581 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a540b1070 con 0x7f8a640cf450 2024-09-19T04:20:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a637fe640 1 -- 172.21.15.76:0/461028581 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a540ac620 con 0x7f8a640cf450 2024-09-19T04:20:13.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a637fe640 1 -- 172.21.15.76:0/461028581 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a540ac9f0 con 0x7f8a640cf450 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 -- 172.21.15.76:0/461028581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 msgr2=0x7f8a6406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/461028581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a6406d6b0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f8a6408d850 tx=0x7f8a5401c3c0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 -- 172.21.15.76:0/461028581 shutdown_connections 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/461028581 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a6406d6b0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/461028581 >> v2:172.21.15.116:3300/0 conn(0x7f8a640d3060 0x7f8a640cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 -- 172.21.15.76:0/461028581 >> 172.21.15.76:0/461028581 conn(0x7f8a6408bf90 msgr2=0x7f8a6409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 -- 172.21.15.76:0/461028581 shutdown_connections 2024-09-19T04:20:13.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 -- 172.21.15.76:0/461028581 wait complete. 2024-09-19T04:20:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 Processor -- start 2024-09-19T04:20:13.459 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.456+0000 7f8a6ad24640 1 -- start start 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a641590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a640d3060 0x7f8a64150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a64161a60 0x7f8a64159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6406e400 con 0x7f8a640cf450 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8a640cfad0 con 0x7f8a640d3060 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8a6406e580 con 0x7f8a64161a60 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a641590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a641590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51310/0 (socket says 172.21.15.76:51310) 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 -- 172.21.15.76:0/329733542 learned_addr learned my addr 172.21.15.76:0/329733542 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 -- 172.21.15.76:0/329733542 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a64161a60 msgr2=0x7f8a64159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a64161a60 0x7f8a64159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 -- 172.21.15.76:0/329733542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a640d3060 msgr2=0x7f8a64150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a63fff640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a640d3060 0x7f8a64150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a640d3060 0x7f8a64150b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 -- 172.21.15.76:0/329733542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a64152bd0 con 0x7f8a640cf450 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a641590e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00caa0 tx=0x7f8a5c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a617fa640 1 -- 172.21.15.76:0/329733542 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c017070 con 0x7f8a640cf450 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a617fa640 1 -- 172.21.15.76:0/329733542 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00ac00 con 0x7f8a640cf450 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a641fa990 con 0x7f8a640cf450 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a617fa640 1 -- 172.21.15.76:0/329733542 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c012400 con 0x7f8a640cf450 2024-09-19T04:20:13.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a641faf50 con 0x7f8a640cf450 2024-09-19T04:20:13.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a617fa640 1 -- 172.21.15.76:0/329733542 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8a5c01a070 con 0x7f8a640cf450 2024-09-19T04:20:13.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a617fa640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8a500678c0 0x7f8a50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a617fa640 1 -- 172.21.15.76:0/329733542 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8a5c085bf0 con 0x7f8a640cf450 2024-09-19T04:20:13.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.460+0000 7f8a63fff640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8a500678c0 0x7f8a50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a63fff640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8a500678c0 0x7f8a50069d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8a640d0fe0 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 Processor -- start 2024-09-19T04:20:13.464 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- start start 2024-09-19T04:20:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a591d70 0x56098a592150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a545870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a545db0 0x56098a546190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56098a42f960 con 0x56098a545db0 2024-09-19T04:20:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56098a373900 con 0x56098a592690 2024-09-19T04:20:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56098a396d40 con 0x56098a591d70 2024-09-19T04:20:13.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a545db0 0x56098a546190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a545db0 0x56098a546190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51324/0 (socket says 172.21.15.76:51324) 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 -- 172.21.15.76:0/2675486146 learned_addr learned my addr 172.21.15.76:0/2675486146 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a591d70 0x56098a592150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a63fff640 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a545870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 -- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a591d70 msgr2=0x56098a592150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a591d70 0x56098a592150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 -- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 msgr2=0x56098a545870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a545870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 -- 172.21.15.76:0/2675486146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098a33d580 con 0x56098a545db0 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a692c1640 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a545db0 0x56098a546190 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8a540af040 tx=0x7f8a540046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fb76268a282c87 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a4a7fc640 1 -- 172.21.15.76:0/2675486146 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a540b1040 con 0x56098a545db0 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a4a7fc640 1 -- 172.21.15.76:0/2675486146 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a540ba070 con 0x56098a545db0 2024-09-19T04:20:13.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a4a7fc640 1 -- 172.21.15.76:0/2675486146 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a540ac860 con 0x56098a545db0 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- 172.21.15.76:0/2675486146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a545db0 msgr2=0x56098a546190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a545db0 0x56098a546190 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8a540af040 tx=0x7f8a540046d0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- 172.21.15.76:0/2675486146 shutdown_connections 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a545db0 0x56098a546190 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a545870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/2675486146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a591d70 0x56098a592150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- 172.21.15.76:0/2675486146 >> 172.21.15.76:0/2675486146 conn(0x56098a573040 msgr2=0x56098a575430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- 172.21.15.76:0/2675486146 shutdown_connections 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- 172.21.15.76:0/2675486146 wait complete. 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 -- start start 2024-09-19T04:20:13.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.464+0000 7f8a6dde0000 1 Processor -- start 2024-09-19T04:20:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a598ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a599220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 0x56098a599760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56098a392d00 con 0x56098a591d70 2024-09-19T04:20:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56098a351aa0 con 0x56098a545db0 2024-09-19T04:20:13.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56098a330340 con 0x56098a592690 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 10 client.0 _ll_register_callbacks cb 0x56098a533760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a599220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a599220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51336/0 (socket says 172.21.15.76:51336) 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 -- 172.21.15.76:0/3877410999 learned_addr learned my addr 172.21.15.76:0/3877410999 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a692c1640 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 0x56098a599760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a598ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 -- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 msgr2=0x56098a599760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 0x56098a599760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 -- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 msgr2=0x56098a598ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a598ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 -- 172.21.15.76:0/3877410999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56098a339dc0 con 0x56098a591d70 2024-09-19T04:20:13.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a63fff640 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a599220 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8a58015d40 tx=0x7f8a58004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be1368f2e0646c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 1 -- 172.21.15.76:0/3877410999 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5800f040 con 0x56098a591d70 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 1 -- 172.21.15.76:0/3877410999 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8a5800ebf0 con 0x56098a591d70 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a540ba070 con 0x56098a591d70 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 1 -- 172.21.15.76:0/3877410999 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f8a5802e050 con 0x56098a591d70 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 10 client.8391 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x56098a3367f0 con 0x56098a591d70 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 1 -- 172.21.15.76:0/3877410999 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a540ba070 con 0x56098a591d70 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 1 -- 172.21.15.76:0/3877410999 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7f8a58031020 con 0x56098a591d70 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 10 client.8391 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 10 client.8391 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 20 client.8391 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a5a10b0 0x56098a5a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 4 client.8391 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56098a330e10 con 0x56098a5a10b0 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a5a10b0 0x56098a5a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 10 client.8391 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a5a10b0 0x56098a5a34a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1612f15792f9e216 server_cookie=6e3a85134f052b73 in_seq=0 out_seq=0 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 1 -- 172.21.15.76:0/3877410999 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56098a330e10 con 0x56098a5a10b0 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 10 client.8391 handle_command_reply: tid=0 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 client.8391 shutdown 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 msgr2=0x56098a599220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a599220 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f8a58015d40 tx=0x7f8a58004740 comp rx=0 tx=0).stop 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 shutdown_connections 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a5a10b0 0x56098a5a34a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 0x56098a599760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a599220 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/3877410999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a598ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a437fe640 0 client.8391 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 >> 172.21.15.76:0/3877410999 conn(0x56098a573040 msgr2=0x56098a544f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 shutdown_connections 2024-09-19T04:20:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 1 -- 172.21.15.76:0/3877410999 wait complete. 2024-09-19T04:20:13.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.468+0000 7f8a6dde0000 20 client.8391 trim_cache size 0 max 16384 2024-09-19T04:20:13.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 Processor -- start 2024-09-19T04:20:13.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 -- start start 2024-09-19T04:20:13.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a54dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a59f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 0x56098a59f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64203f20 con 0x56098a591d70 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64203110 con 0x56098a545db0 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8a642037f0 con 0x56098a592690 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a59f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a59f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51342/0 (socket says 172.21.15.76:51342) 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 -- 172.21.15.76:0/1965258262 learned_addr learned my addr 172.21.15.76:0/1965258262 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 -- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 msgr2=0x56098a59f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 0x56098a59f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 -- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 msgr2=0x56098a54dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a54dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a54dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 -- 172.21.15.76:0/1965258262 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56098a354470 con 0x56098a591d70 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a54dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a63fff640 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a59f1e0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f8a58017010 tx=0x7f8a58018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846d6f7447ace6e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a49ffb640 1 -- 172.21.15.76:0/1965258262 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a58037070 con 0x56098a591d70 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a49ffb640 1 -- 172.21.15.76:0/1965258262 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a58026070 con 0x56098a591d70 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.484+0000 7f8a49ffb640 1 -- 172.21.15.76:0/1965258262 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a580218a0 con 0x56098a591d70 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1965258262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 msgr2=0x56098a59f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a59f1e0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f8a58017010 tx=0x7f8a58018000 comp rx=0 tx=0).stop 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1965258262 shutdown_connections 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a592690 0x56098a59f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a59f1e0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1965258262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a545db0 0x56098a54dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1965258262 >> 172.21.15.76:0/1965258262 conn(0x56098a573040 msgr2=0x56098a59ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1965258262 shutdown_connections 2024-09-19T04:20:13.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1965258262 wait complete. 2024-09-19T04:20:13.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- start start 2024-09-19T04:20:13.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 Processor -- start 2024-09-19T04:20:13.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a545db0 0x56098a5a5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a5a6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a60a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8a64152170 con 0x56098a591d70 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56098a3557c0 con 0x56098a592690 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56098a3d6780 con 0x56098a545db0 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 10 client.0 _ll_register_callbacks cb 0x7f8a58015440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a5a6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a5a6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51358/0 (socket says 172.21.15.76:51358) 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 -- 172.21.15.76:0/1597191791 learned_addr learned my addr 172.21.15.76:0/1597191791 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a545db0 0x56098a5a5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a692c1640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a60a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 -- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a545db0 msgr2=0x56098a5a5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a545db0 0x56098a5a5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 -- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 msgr2=0x56098a60a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a60a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 -- 172.21.15.76:0/1597191791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56098a32a640 con 0x56098a591d70 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a692c1640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a60a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a63fff640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a5a6230 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f8a58035040 tx=0x7f8a58015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7236e0359899ed5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a437fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a437fe640 1 -- 172.21.15.76:0/1597191791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5803f060 con 0x56098a591d70 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a437fe640 1 -- 172.21.15.76:0/1597191791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8a5800ebf0 con 0x56098a591d70 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a58026070 con 0x56098a591d70 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a437fe640 1 -- 172.21.15.76:0/1597191791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f8a58037070 con 0x56098a591d70 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 10 client.8397 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x56098a32e3b0 con 0x56098a591d70 2024-09-19T04:20:13.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a437fe640 1 -- 172.21.15.76:0/1597191791 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a58026070 con 0x56098a591d70 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a437fe640 1 -- 172.21.15.76:0/1597191791 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7f8a58031040 con 0x56098a591d70 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 10 client.8397 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 10 client.8397 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 20 client.8397 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a60adf0 0x56098a60b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 4 client.8397 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56098a3a4470 con 0x56098a60adf0 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a60adf0 0x56098a60b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a68ac0640 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a60adf0 0x56098a60b200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=46cb957ba458b76a server_cookie=9aca3649b83df9b6 in_seq=0 out_seq=0 2024-09-19T04:20:13.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.488+0000 7f8a437fe640 10 client.8397 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:13.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a437fe640 1 -- 172.21.15.76:0/1597191791 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56098a3a4470 con 0x56098a60adf0 2024-09-19T04:20:13.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a437fe640 10 client.8397 handle_command_reply: tid=0 2024-09-19T04:20:13.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 client.8397 shutdown 2024-09-19T04:20:13.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 msgr2=0x56098a5a6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a5a6230 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f8a58035040 tx=0x7f8a58015000 comp rx=0 tx=0).stop 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 shutdown_connections 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56098a60adf0 0x56098a60b200 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56098a592690 0x56098a60a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56098a591d70 0x56098a5a6230 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 --2- 172.21.15.76:0/1597191791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56098a545db0 0x56098a5a5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 >> 172.21.15.76:0/1597191791 conn(0x56098a573040 msgr2=0x56098a59f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a437fe640 0 client.8397 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.504+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 shutdown_connections 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6dde0000 1 -- 172.21.15.76:0/1597191791 wait complete. 2024-09-19T04:20:13.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6dde0000 20 client.8397 trim_cache size 0 max 16384 2024-09-19T04:20:13.509 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:20:13.509 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:20:13.509 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:13.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8a500678c0 msgr2=0x7f8a50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8a500678c0 0x7f8a50069d80 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8a640d0fe0 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2024-09-19T04:20:13.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 msgr2=0x7f8a641590e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a641590e0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c00caa0 tx=0x7f8a5c00cf60 comp rx=0 tx=0).stop 2024-09-19T04:20:13.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 shutdown_connections 2024-09-19T04:20:13.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8a500678c0 0x7f8a50069d80 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8a64161a60 0x7f8a64159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a640d3060 0x7f8a64150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 --2- 172.21.15.76:0/329733542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8a640cf450 0x7f8a641590e0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 >> 172.21.15.76:0/329733542 conn(0x7f8a6408bf90 msgr2=0x7f8a6415d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 shutdown_connections 2024-09-19T04:20:13.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.508+0000 7f8a6ad24640 1 -- 172.21.15.76:0/329733542 wait complete. 2024-09-19T04:20:13.525 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.7:2 flush journal 2024-09-19T04:20:13.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900f044640 1 Processor -- start 2024-09-19T04:20:13.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900f044640 1 -- start start 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900f044640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f90080ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900f044640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f90080af470 0x7f900806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900f044640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90080b3290 con 0x7f900815c2a0 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900f044640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f90080b3570 con 0x7f90080af470 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900f044640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f90080b3850 con 0x7f900815bec0 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900d5e1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f90080af470 0x7f900806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900d5e1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f90080af470 0x7f900806e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51366/0 (socket says 172.21.15.76:51366) 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900d5e1640 1 -- 172.21.15.76:0/2364939356 learned_addr learned my addr 172.21.15.76:0/2364939356 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900cde0640 1 --2- 172.21.15.76:0/2364939356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f90080ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900d5e1640 1 -- 172.21.15.76:0/2364939356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 msgr2=0x7f90080ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900d5e1640 1 --2- 172.21.15.76:0/2364939356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f90080ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.642 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900d5e1640 1 -- 172.21.15.76:0/2364939356 >> v1:172.21.15.116:6789/0 conn(0x7f900815c2a0 legacy=0x7f90080aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900d5e1640 1 -- 172.21.15.76:0/2364939356 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90080b3b30 con 0x7f90080af470 2024-09-19T04:20:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.640+0000 7f900cde0640 1 --2- 172.21.15.76:0/2364939356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f90080ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900d5e1640 1 --2- 172.21.15.76:0/2364939356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900806e000 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9000009e50 tx=0x7f900001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a12bbda1f8b790f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f90077fe640 1 -- 172.21.15.76:0/2364939356 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5840 con 0x7f90080af470 2024-09-19T04:20:13.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f90077fe640 1 -- 172.21.15.76:0/2364939356 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90000a59e0 con 0x7f90080af470 2024-09-19T04:20:13.644 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f90077fe640 1 -- 172.21.15.76:0/2364939356 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5db0 con 0x7f90080af470 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- 172.21.15.76:0/2364939356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 msgr2=0x7f900806e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 --2- 172.21.15.76:0/2364939356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900806e000 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9000009e50 tx=0x7f900001c9c0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- 172.21.15.76:0/2364939356 shutdown_connections 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 --2- 172.21.15.76:0/2364939356 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900806e000 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 --2- 172.21.15.76:0/2364939356 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f90080ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- 172.21.15.76:0/2364939356 >> 172.21.15.76:0/2364939356 conn(0x7f90080b1f90 msgr2=0x7f90080b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- 172.21.15.76:0/2364939356 shutdown_connections 2024-09-19T04:20:13.645 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- 172.21.15.76:0/2364939356 wait complete. 2024-09-19T04:20:13.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 Processor -- start 2024-09-19T04:20:13.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- start start 2024-09-19T04:20:13.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900814dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.648 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f900814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900815c2a0 0x7f9008157a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f900806ed50 con 0x7f90080af470 2024-09-19T04:20:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f90080ad740 con 0x7f900815c2a0 2024-09-19T04:20:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900f044640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f900806eed0 con 0x7f900815bec0 2024-09-19T04:20:13.649 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900cde0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900814dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900cde0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900814dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51368/0 (socket says 172.21.15.76:51368) 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900cde0640 1 -- 172.21.15.76:0/3186049424 learned_addr learned my addr 172.21.15.76:0/3186049424 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f9007fff640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f900814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900d5e1640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900815c2a0 0x7f9008157a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900cde0640 1 -- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 msgr2=0x7f900814d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.644+0000 7f900cde0640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f900814d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f900cde0640 1 -- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900815c2a0 msgr2=0x7f9008157a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f900cde0640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900815c2a0 0x7f9008157a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f900cde0640 1 -- 172.21.15.76:0/3186049424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9008157f40 con 0x7f90080af470 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f9007fff640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f900814d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f900cde0640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900814dcf0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c7f0 tx=0x7f8ff800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f90057fa640 1 -- 172.21.15.76:0/3186049424 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8007a30 con 0x7f90080af470 2024-09-19T04:20:13.650 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f90057fa640 1 -- 172.21.15.76:0/3186049424 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff8007bd0 con 0x7f90080af470 2024-09-19T04:20:13.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90080b0c60 con 0x7f90080af470 2024-09-19T04:20:13.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f90057fa640 1 -- 172.21.15.76:0/3186049424 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff80113f0 con 0x7f90080af470 2024-09-19T04:20:13.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9008152380 con 0x7f90080af470 2024-09-19T04:20:13.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f90057fa640 1 -- 172.21.15.76:0/3186049424 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8ff8011590 con 0x7f90080af470 2024-09-19T04:20:13.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f90057fa640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ff40678c0 0x7f8ff4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f9007fff640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ff40678c0 0x7f8ff4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f90057fa640 1 -- 172.21.15.76:0/3186049424 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8ff8086060 con 0x7f90080af470 2024-09-19T04:20:13.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f9007fff640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ff40678c0 0x7f8ff4069d80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f900814af40 tx=0x7f8ffc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f9012100000 1 Processor -- start 2024-09-19T04:20:13.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.648+0000 7f9012100000 1 -- start start 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd11150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bcc4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bcc5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf8bbae960 con 0x55bf8bd10d70 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf8baf2900 con 0x55bf8bd11690 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bf8bb15d40 con 0x55bf8bcc4db0 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd11150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd11150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51374/0 (socket says 172.21.15.76:51374) 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 -- 172.21.15.76:0/1691735427 learned_addr learned my addr 172.21.15.76:0/1691735427 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 -- 172.21.15.76:0/1691735427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 msgr2=0x55bf8bcc5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bcc5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 -- 172.21.15.76:0/1691735427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 msgr2=0x55bf8bcc4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bcc4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bcc4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 -- 172.21.15.76:0/1691735427 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8babc580 con 0x55bf8bd10d70 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd11150 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8055fd0 tx=0x7f8ff8056a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf19c34e0c66c378 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f8fee7fc640 1 -- 172.21.15.76:0/1691735427 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff806e020 con 0x55bf8bd10d70 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f8fee7fc640 1 -- 172.21.15.76:0/1691735427 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff805eeb0 con 0x55bf8bd10d70 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f8fee7fc640 1 -- 172.21.15.76:0/1691735427 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ff801e070 con 0x55bf8bd10d70 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- 172.21.15.76:0/1691735427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 msgr2=0x55bf8bd11150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd11150 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f8ff8055fd0 tx=0x7f8ff8056a90 comp rx=0 tx=0).stop 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- 172.21.15.76:0/1691735427 shutdown_connections 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bcc5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bcc4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- 172.21.15.76:0/1691735427 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd11150 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.659 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- 172.21.15.76:0/1691735427 >> 172.21.15.76:0/1691735427 conn(0x55bf8bcf2040 msgr2=0x55bf8bcf4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- 172.21.15.76:0/1691735427 shutdown_connections 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- 172.21.15.76:0/1691735427 wait complete. 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- start start 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 Processor -- start 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd17ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd18220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bd18760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf8bb11d00 con 0x55bf8bd10d70 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf8bad0aa0 con 0x55bf8bd11690 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bf8baaf340 con 0x55bf8bcc4db0 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9012100000 10 client.0 _ll_register_callbacks cb 0x55bf8bcb2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd18220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.660 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd18220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51388/0 (socket says 172.21.15.76:51388) 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 -- 172.21.15.76:0/1736482352 learned_addr learned my addr 172.21.15.76:0/1736482352 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bd18760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900cde0640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd17ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 -- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 msgr2=0x55bf8bd17ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd17ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 -- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 msgr2=0x55bf8bd18760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bd18760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 -- 172.21.15.76:0/1736482352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf8bab8dc0 con 0x55bf8bd10d70 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bd18760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f9007fff640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd18220 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00dd30 tx=0x7f8ffc004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205032b26122f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.652+0000 7f8fe77fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1736482352 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc01f410 con 0x55bf8bd10d70 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1736482352 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8ffc01f5b0 con 0x55bf8bd10d70 2024-09-19T04:20:13.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ff805eeb0 con 0x55bf8bd10d70 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1736482352 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f8ffc005920 con 0x55bf8bd10d70 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 10 client.8409 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55bf8bab57f0 con 0x55bf8bd10d70 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1736482352 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff805eeb0 con 0x55bf8bd10d70 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1736482352 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7f8ffc02f080 con 0x55bf8bd10d70 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 10 client.8409 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 10 client.8409 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 20 client.8409 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd200b0 0x55bf8bd224a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 4 client.8409 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf8baafe10 con 0x55bf8bd200b0 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f900cde0640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd200b0 0x55bf8bd224a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 10 client.8409 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:13.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f900cde0640 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd200b0 0x55bf8bd224a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8e3d654088bbb1f server_cookie=a477cce7587e25aa in_seq=0 out_seq=0 2024-09-19T04:20:13.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1736482352 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bf8baafe10 con 0x55bf8bd200b0 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 10 client.8409 handle_command_reply: tid=0 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 client.8409 shutdown 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 msgr2=0x55bf8bd18220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd18220 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00dd30 tx=0x7f8ffc004690 comp rx=0 tx=0).stop 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 shutdown_connections 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd200b0 0x55bf8bd224a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bd11690 0x55bf8bd18760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd18220 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 --2- 172.21.15.76:0/1736482352 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd17ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f8fe77fe640 0 client.8409 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 >> 172.21.15.76:0/1736482352 conn(0x55bf8bcf2040 msgr2=0x55bf8bcc3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 shutdown_connections 2024-09-19T04:20:13.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 1 -- 172.21.15.76:0/1736482352 wait complete. 2024-09-19T04:20:13.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.656+0000 7f9012100000 20 client.8409 trim_cache size 0 max 16384 2024-09-19T04:20:13.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 Processor -- start 2024-09-19T04:20:13.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 -- start start 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcf2040 0x55bf8bcccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcc4db0 0x55bf8bccd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd1ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9008203eb0 con 0x55bf8bd10d70 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9008203080 con 0x55bf8bcc4db0 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9008203780 con 0x55bf8bcf2040 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd1ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd1ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51394/0 (socket says 172.21.15.76:51394) 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 -- 172.21.15.76:0/1101435528 learned_addr learned my addr 172.21.15.76:0/1101435528 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 -- 172.21.15.76:0/1101435528 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcf2040 msgr2=0x55bf8bcccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcf2040 0x55bf8bcccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 -- 172.21.15.76:0/1101435528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcc4db0 msgr2=0x55bf8bccd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f9007fff640 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcc4db0 0x55bf8bccd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcc4db0 0x55bf8bccd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 -- 172.21.15.76:0/1101435528 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf8bad3470 con 0x55bf8bd10d70 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd1ad00 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f900001d260 tx=0x7f9000004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7372bf9e173afc56 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f8fedffb640 1 -- 172.21.15.76:0/1101435528 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5840 con 0x55bf8bd10d70 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f8fedffb640 1 -- 172.21.15.76:0/1101435528 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90000a59e0 con 0x55bf8bd10d70 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.672+0000 7f8fedffb640 1 -- 172.21.15.76:0/1101435528 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5db0 con 0x55bf8bd10d70 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1101435528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 msgr2=0x55bf8bd1ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd1ad00 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f900001d260 tx=0x7f9000004a50 comp rx=0 tx=0).stop 2024-09-19T04:20:13.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1101435528 shutdown_connections 2024-09-19T04:20:13.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd1ad00 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcc4db0 0x55bf8bccd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- 172.21.15.76:0/1101435528 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcf2040 0x55bf8bcccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1101435528 >> 172.21.15.76:0/1101435528 conn(0x55bf8bcc9890 msgr2=0x55bf8bd17aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1101435528 shutdown_connections 2024-09-19T04:20:13.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1101435528 wait complete. 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- start start 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 Processor -- start 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd1da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcf2040 0x55bf8bd85140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd85680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bf8babbf20 con 0x55bf8bd10d70 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bf8bb1a370 con 0x55bf8bcf2040 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bf8bad43f0 con 0x55bf8bcc4db0 2024-09-19T04:20:13.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 10 client.0 _ll_register_callbacks cb 0x7f8ffc009160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd85680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd85680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51398/0 (socket says 172.21.15.76:51398) 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 -- 172.21.15.76:0/1454741313 learned_addr learned my addr 172.21.15.76:0/1454741313 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 -- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 msgr2=0x55bf8bd1da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900cde0640 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd1da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9007fff640 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcf2040 0x55bf8bd85140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd1da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 -- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcf2040 msgr2=0x55bf8bd85140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcf2040 0x55bf8bd85140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 -- 172.21.15.76:0/1454741313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf8baa9640 con 0x55bf8bd10d70 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900d5e1640 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd85680 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9000000f90 tx=0x7f900001b200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7888f2c9ebd55046 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f8fe77fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:13.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1454741313 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90000a5840 con 0x55bf8bd10d70 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1454741313 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f90000cbc70 con 0x55bf8bd10d70 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90000a59e0 con 0x55bf8bd10d70 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1454741313 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f90000cbe70 con 0x55bf8bd10d70 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 10 client.8415 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55bf8baad3b0 con 0x55bf8bd10d70 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1454741313 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90000a59e0 con 0x55bf8bd10d70 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1454741313 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7f9000008040 con 0x55bf8bd10d70 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 10 client.8415 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 10 client.8415 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 20 client.8415 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd11690 0x55bf8bd85bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 4 client.8415 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T04:20:13.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf8bb23470 con 0x55bf8bd11690 2024-09-19T04:20:13.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900cde0640 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd11690 0x55bf8bd85bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:13.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f8fe77fe640 10 client.8415 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:13.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.676+0000 7f900cde0640 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd11690 0x55bf8bd85bc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1c000bc61e413a1 server_cookie=3f7d72410bbb4fdc in_seq=0 out_seq=0 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f8fe77fe640 1 -- 172.21.15.76:0/1454741313 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf8bb23470 con 0x55bf8bd11690 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f8fe77fe640 10 client.8415 handle_command_reply: tid=0 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 client.8415 shutdown 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 msgr2=0x55bf8bd85680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd85680 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f9000000f90 tx=0x7f900001b200 comp rx=0 tx=0).stop 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 shutdown_connections 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55bf8bd11690 0x55bf8bd85bc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bf8bd10d70 0x55bf8bd85680 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bf8bcf2040 0x55bf8bd85140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 --2- 172.21.15.76:0/1454741313 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bf8bcc4db0 0x55bf8bd1da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f8fe77fe640 0 client.8415 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 >> 172.21.15.76:0/1454741313 conn(0x55bf8bcc9890 msgr2=0x55bf8bccbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 shutdown_connections 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 1 -- 172.21.15.76:0/1454741313 wait complete. 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f9012100000 20 client.8415 trim_cache size 0 max 16384 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:20:13.697 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:13.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ff40678c0 msgr2=0x7f8ff4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ff40678c0 0x7f8ff4069d80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f900814af40 tx=0x7f8ffc008040 comp rx=0 tx=0).stop 2024-09-19T04:20:13.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 msgr2=0x7f900814dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:13.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900814dcf0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f8ff800c7f0 tx=0x7f8ff800ccb0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 shutdown_connections 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8ff40678c0 0x7f8ff4069d80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900815c2a0 0x7f9008157a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f900815bec0 0x7f900814d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 --2- 172.21.15.76:0/3186049424 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f90080af470 0x7f900814dcf0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 >> 172.21.15.76:0/3186049424 conn(0x7f90080b1f90 msgr2=0x7f90081598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 shutdown_connections 2024-09-19T04:20:13.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13.696+0000 7f900f044640 1 -- 172.21.15.76:0/3186049424 wait complete. 2024-09-19T04:20:13.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:13.712 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest/mnt.0 && exec find d1/d2/d3/) 2024-09-19T04:20:13.741 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/ 2024-09-19T04:20:13.741 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4 2024-09-19T04:20:13.741 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5 2024-09-19T04:20:13.741 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6 2024-09-19T04:20:13.741 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7 2024-09-19T04:20:13.741 INFO:teuthology.orchestra.run.smithi076.stdout:d1/d2/d3/d4/d5/d6/d7/d8 2024-09-19T04:20:13.741 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:20:13.742 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:20:13.742 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:20:13.742 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:20:13.742 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/").st_ino) 2024-09-19T04:20:13.742 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:20:13.839 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627778 2024-09-19T04:20:13.841 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:13.848 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:20:13.848 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:20:13.848 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:20:13.848 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:20:13.848 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4").st_ino) 2024-09-19T04:20:13.848 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:20:13.949 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627779 2024-09-19T04:20:13.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:13.958 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:20:13.958 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:20:13.958 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:20:13.958 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:20:13.959 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5").st_ino) 2024-09-19T04:20:13.959 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:20:14.060 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627780 2024-09-19T04:20:14.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:14.069 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:20:14.069 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:20:14.069 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:20:14.069 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:20:14.069 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6").st_ino) 2024-09-19T04:20:14.070 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:20:14.175 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627781 2024-09-19T04:20:14.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:14.185 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:20:14.185 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:20:14.185 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:20:14.185 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:20:14.185 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7").st_ino) 2024-09-19T04:20:14.185 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:20:14.291 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627782 2024-09-19T04:20:14.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:14.301 DEBUG:teuthology.orchestra.run.smithi076:> stdin-killer -- python3 -c ' 2024-09-19T04:20:14.302 DEBUG:teuthology.orchestra.run.smithi076:> import os 2024-09-19T04:20:14.302 DEBUG:teuthology.orchestra.run.smithi076:> import stat 2024-09-19T04:20:14.302 DEBUG:teuthology.orchestra.run.smithi076:> 2024-09-19T04:20:14.302 DEBUG:teuthology.orchestra.run.smithi076:> print(os.stat("/home/ubuntu/cephtest/mnt.0/d1/d2/d3/d4/d5/d6/d7/d8").st_ino) 2024-09-19T04:20:14.302 DEBUG:teuthology.orchestra.run.smithi076:> ' 2024-09-19T04:20:14.408 INFO:teuthology.orchestra.run.smithi076.stdout:1099511627783 2024-09-19T04:20:14.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T04:20:14.419 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000002.00000000 parent 2024-09-19T04:20:14.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 Processor -- start 2024-09-19T04:20:14.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- start start 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7861d0 0x55f73a846510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a7b8dd0 0x55f73a7ac010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a843bb0 0x55f73a841160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f73a7b9550 con 0x55f73a843bb0 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f73a6d71b0 con 0x55f73a7b8dd0 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f73a7b97f0 con 0x55f73a7861d0 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a843bb0 0x55f73a841160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a843bb0 0x55f73a841160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51412/0 (socket says 172.21.15.76:51412) 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 -- 172.21.15.76:0/3501281629 learned_addr learned my addr 172.21.15.76:0/3501281629 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff516eb5640 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7861d0 0x55f73a846510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5166b4640 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a7b8dd0 0x55f73a7ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 -- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7861d0 msgr2=0x55f73a846510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7861d0 0x55f73a846510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 -- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a7b8dd0 msgr2=0x55f73a7ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a7b8dd0 0x55f73a7ac010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 -- 172.21.15.76:0/3501281629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73a846bd0 con 0x55f73a843bb0 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5176b6640 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a843bb0 0x55f73a841160 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff510001670 tx=0x7ff510001b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe821f6827da5f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5156b2640 1 -- 172.21.15.76:0/3501281629 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510013620 con 0x55f73a843bb0 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5156b2640 1 -- 172.21.15.76:0/3501281629 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5100137c0 con 0x55f73a843bb0 2024-09-19T04:20:14.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5156b2640 1 -- 172.21.15.76:0/3501281629 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510013bb0 con 0x55f73a843bb0 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- 172.21.15.76:0/3501281629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a843bb0 msgr2=0x55f73a841160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a843bb0 0x55f73a841160 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff510001670 tx=0x7ff510001b30 comp rx=0 tx=0).stop 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- 172.21.15.76:0/3501281629 shutdown_connections 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a843bb0 0x55f73a841160 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a7b8dd0 0x55f73a7ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3501281629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7861d0 0x55f73a846510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- 172.21.15.76:0/3501281629 >> 172.21.15.76:0/3501281629 conn(0x55f73a786dc0 msgr2=0x55f73a845e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- 172.21.15.76:0/3501281629 shutdown_connections 2024-09-19T04:20:14.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.500+0000 7ff5193a2080 1 -- 172.21.15.76:0/3501281629 wait complete. 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 Processor -- start 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 -- start start 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a7861d0 0x55f73a7ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7b8dd0 0x55f73a842090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a843bb0 0x55f73a8425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55f73a85c0a0 con 0x55f73a7861d0 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55f73a846e70 con 0x55f73a843bb0 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55f73a85c340 con 0x55f73a7b8dd0 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a7861d0 0x55f73a7ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a7861d0 0x55f73a7ae980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51426/0 (socket says 172.21.15.76:51426) 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 -- 172.21.15.76:0/3202118762 learned_addr learned my addr 172.21.15.76:0/3202118762 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5176b6640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a843bb0 0x55f73a8425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5166b4640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7b8dd0 0x55f73a842090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 -- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7b8dd0 msgr2=0x55f73a842090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7b8dd0 0x55f73a842090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 -- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a843bb0 msgr2=0x55f73a8425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a843bb0 0x55f73a8425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 -- 172.21.15.76:0/3202118762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f73a8f7c00 con 0x55f73a7861d0 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5176b6640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55f73a843bb0 0x55f73a8425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5166b4640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55f73a7b8dd0 0x55f73a842090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff516eb5640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a7861d0 0x55f73a7ae980 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff508096d60 tx=0x7ff5080180f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff50ffff640 1 -- 172.21.15.76:0/3202118762 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5080b9070 con 0x55f73a7861d0 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff50ffff640 1 -- 172.21.15.76:0/3202118762 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5080057d0 con 0x55f73a7861d0 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff50ffff640 1 -- 172.21.15.76:0/3202118762 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5080af040 con 0x55f73a7861d0 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f73a8f7ea0 con 0x55f73a7861d0 2024-09-19T04:20:14.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f73a8f83e0 con 0x55f73a7861d0 2024-09-19T04:20:14.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.504+0000 7ff50ffff640 1 -- 172.21.15.76:0/3202118762 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff5080bc070 con 0x55f73a7861d0 2024-09-19T04:20:14.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff50ffff640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4f80662d0 0x7ff4f8068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff50ffff640 1 -- 172.21.15.76:0/3202118762 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff50811f440 con 0x55f73a7861d0 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5166b4640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4f80662d0 0x7ff4f8068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55f73a8f6fa0 0x55f73a8f7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 --> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] -- osd_op(unknown.0.0:1 15.12 15:4bec3886:::10000000002.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55f73a8f8b20 con 0x55f73a8f6fa0 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5176b6640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55f73a8f6fa0 0x55f73a8f7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5166b4640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4f80662d0 0x7ff4f8068790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff5000098c0 tx=0x7ff50000c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5176b6640 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55f73a8f6fa0 0x55f73a8f7380 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5176b6640 1 -- 172.21.15.76:0/3202118762 <== osd.3 v2:172.21.15.76:6816/1627524440 1 ==== osd_op_reply(1 10000000002.00000000 [rmxattr] v106'8 uv8 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7ff51000f030 con 0x55f73a8f6fa0 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55f73a8f6fa0 msgr2=0x55f73a8f7380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55f73a8f6fa0 0x55f73a8f7380 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4f80662d0 msgr2=0x7ff4f8068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4f80662d0 0x7ff4f8068790 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff5000098c0 tx=0x7ff50000c040 comp rx=0 tx=0).stop 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a7861d0 msgr2=0x55f73a7ae980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 --2- 172.21.15.76:0/3202118762 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55f73a7861d0 0x55f73a7ae980 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff508096d60 tx=0x7ff5080180f0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5176b6640 1 -- 172.21.15.76:0/3202118762 reap_dead start 2024-09-19T04:20:14.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 shutdown_connections 2024-09-19T04:20:14.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 >> 172.21.15.76:0/3202118762 conn(0x55f73a786dc0 msgr2=0x55f73a85c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 shutdown_connections 2024-09-19T04:20:14.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.508+0000 7ff5193a2080 1 -- 172.21.15.76:0/3202118762 wait complete. 2024-09-19T04:20:14.513 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000003.00000000 parent 2024-09-19T04:20:14.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 Processor -- start 2024-09-19T04:20:14.589 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 -- start start 2024-09-19T04:20:14.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e35f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e35fe050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e5e3524640 con 0x55e5e35fa560 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e5e34721b0 con 0x55e5e35fdc70 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa4339080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55e5e35247c0 con 0x55e5e35f80f0 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e35f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e35f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51450/0 (socket says 172.21.15.76:51450) 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 -- 172.21.15.76:0/677319534 learned_addr learned my addr 172.21.15.76:0/677319534 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa1e4c640 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa264d640 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e35fe050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 -- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 msgr2=0x55e5e35f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 -- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 msgr2=0x55e5e35fe050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.592 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e35fe050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 -- 172.21.15.76:0/677319534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e3552840 con 0x55e5e35fa560 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa1e4c640 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa264d640 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e35fe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7aa164b640 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e35f84d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00d700 tx=0x7f7a8c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52d74052185e612 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7a9bfff640 1 -- 172.21.15.76:0/677319534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c00bcf0 con 0x55e5e35fa560 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7a9bfff640 1 -- 172.21.15.76:0/677319534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8c012030 con 0x55e5e35fa560 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.588+0000 7f7a9bfff640 1 -- 172.21.15.76:0/677319534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c017660 con 0x55e5e35fa560 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- 172.21.15.76:0/677319534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 msgr2=0x55e5e35f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e35f84d0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00d700 tx=0x7f7a8c00dbc0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- 172.21.15.76:0/677319534 shutdown_connections 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e35fe050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e35f84d0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 --2- 172.21.15.76:0/677319534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.593 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- 172.21.15.76:0/677319534 >> 172.21.15.76:0/677319534 conn(0x55e5e354def0 msgr2=0x55e5e35f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- 172.21.15.76:0/677319534 shutdown_connections 2024-09-19T04:20:14.594 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- 172.21.15.76:0/677319534 wait complete. 2024-09-19T04:20:14.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 Processor -- start 2024-09-19T04:20:14.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- start start 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e3544930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e354c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55e5e35f5b90 con 0x55e5e35fa560 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55e5e3552ae0 con 0x55e5e35fdc70 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55e5e35f5e30 con 0x55e5e35f80f0 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e3544930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e3544930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51460/0 (socket says 172.21.15.76:51460) 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 -- 172.21.15.76:0/1968408996 learned_addr learned my addr 172.21.15.76:0/1968408996 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa264d640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e354c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa1e4c640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 -- 172.21.15.76:0/1968408996 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 msgr2=0x55e5e35676b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.597 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55e5e35f80f0 0x55e5e35676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 -- 172.21.15.76:0/1968408996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 msgr2=0x55e5e354c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55e5e35fdc70 0x55e5e354c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 -- 172.21.15.76:0/1968408996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5e354c690 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa164b640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e3544930 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c001ee0 tx=0x7f7a8c001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7a9affd640 1 -- 172.21.15.76:0/1968408996 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c028070 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7a9affd640 1 -- 172.21.15.76:0/1968408996 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a8c017830 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e5e3567ce0 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7a9affd640 1 -- 172.21.15.76:0/1968408996 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c001780 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.592+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5e3568220 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7a9affd640 1 -- 172.21.15.76:0/1968408996 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a8c02b070 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7a9affd640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a880662a0 0x7f7a88068760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7a9affd640 1 -- 172.21.15.76:0/1968408996 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f7a8c086450 con 0x55e5e35fa560 2024-09-19T04:20:14.598 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7aa1e4c640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a880662a0 0x7f7a88068760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7aa4339080 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55e5e36928b0 0x55e5e3692c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 --> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] -- osd_op(unknown.0.0:1 15.12 15:4def72d4:::10000000003.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55e5e36931d0 con 0x55e5e36928b0 2024-09-19T04:20:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7aa264d640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55e5e36928b0 0x55e5e3692c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7aa1e4c640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a880662a0 0x7f7a88068760 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00a7e0 tx=0x7f7a9c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.599 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.596+0000 7f7aa264d640 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55e5e36928b0 0x55e5e3692c90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa264d640 1 -- 172.21.15.76:0/1968408996 <== osd.3 v2:172.21.15.76:6816/1627524440 1 ==== osd_op_reply(1 10000000003.00000000 [rmxattr] v106'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a940b2040 con 0x55e5e36928b0 2024-09-19T04:20:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55e5e36928b0 msgr2=0x55e5e3692c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x55e5e36928b0 0x55e5e3692c90 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a880662a0 msgr2=0x7f7a88068760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a880662a0 0x7f7a88068760 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00a7e0 tx=0x7f7a9c007040 comp rx=0 tx=0).stop 2024-09-19T04:20:14.600 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 msgr2=0x55e5e3544930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 --2- 172.21.15.76:0/1968408996 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55e5e35fa560 0x55e5e3544930 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c001ee0 tx=0x7f7a8c001a50 comp rx=0 tx=0).stop 2024-09-19T04:20:14.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa264d640 1 -- 172.21.15.76:0/1968408996 reap_dead start 2024-09-19T04:20:14.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 shutdown_connections 2024-09-19T04:20:14.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 >> 172.21.15.76:0/1968408996 conn(0x55e5e354def0 msgr2=0x55e5e35fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 shutdown_connections 2024-09-19T04:20:14.601 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.600+0000 7f7aa4339080 1 -- 172.21.15.76:0/1968408996 wait complete. 2024-09-19T04:20:14.602 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000004.00000000 parent 2024-09-19T04:20:14.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 Processor -- start 2024-09-19T04:20:14.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 -- start start 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2808ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd277d750 0x562fd280aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd2799b10 0x562fd2827bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562fd269f1b0 con 0x7f6044003be0 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562fd26f87b0 con 0x562fd2799b10 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6057fc7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562fd2780fe0 con 0x562fd277d750 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6055ada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2808ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6055ada640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2808ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51470/0 (socket says 172.21.15.76:51470) 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.680+0000 7f6055ada640 1 -- 172.21.15.76:0/3784775600 learned_addr learned my addr 172.21.15.76:0/3784775600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd2799b10 0x562fd2827bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60552d9640 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd277d750 0x562fd280aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 -- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd277d750 msgr2=0x562fd280aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd277d750 0x562fd280aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 -- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd2799b10 msgr2=0x562fd2827bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd2799b10 0x562fd2827bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 -- 172.21.15.76:0/3784775600 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd28095f0 con 0x7f6044003be0 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60552d9640 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd277d750 0x562fd280aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2808ae0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f604800c310 tx=0x7f604800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0e24f665038e61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f604ffff640 1 -- 172.21.15.76:0/3784775600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048010440 con 0x7f6044003be0 2024-09-19T04:20:14.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f604ffff640 1 -- 172.21.15.76:0/3784775600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60480105e0 con 0x7f6044003be0 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f604ffff640 1 -- 172.21.15.76:0/3784775600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60480109b0 con 0x7f6044003be0 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- 172.21.15.76:0/3784775600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 msgr2=0x562fd2808ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2808ae0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f604800c310 tx=0x7f604800c7d0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- 172.21.15.76:0/3784775600 shutdown_connections 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd2799b10 0x562fd2827bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd277d750 0x562fd280aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 --2- 172.21.15.76:0/3784775600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2808ae0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- 172.21.15.76:0/3784775600 >> 172.21.15.76:0/3784775600 conn(0x562fd274edc0 msgr2=0x562fd28097d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- 172.21.15.76:0/3784775600 shutdown_connections 2024-09-19T04:20:14.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- 172.21.15.76:0/3784775600 wait complete. 2024-09-19T04:20:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 Processor -- start 2024-09-19T04:20:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- start start 2024-09-19T04:20:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd277d750 0x562fd2825110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd2799b10 0x562fd2776ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.689 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2825b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562fd280ea20 con 0x7f6044003be0 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562fd280e780 con 0x562fd277d750 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562fd280ecc0 con 0x562fd2799b10 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2825b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2825b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51472/0 (socket says 172.21.15.76:51472) 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 -- 172.21.15.76:0/1350890962 learned_addr learned my addr 172.21.15.76:0/1350890962 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60552d9640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd2799b10 0x562fd2776ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd277d750 0x562fd2825110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 -- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd2799b10 msgr2=0x562fd2776ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562fd2799b10 0x562fd2776ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 -- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd277d750 msgr2=0x562fd2825110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd277d750 0x562fd2825110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 -- 172.21.15.76:0/1350890962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fd28bf800 con 0x7f6044003be0 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6055ada640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562fd277d750 0x562fd2825110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:14.690 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f60562db640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2825b80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f605000a790 tx=0x7f6050005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f604effd640 1 -- 172.21.15.76:0/1350890962 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050004610 con 0x7f6044003be0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f604effd640 1 -- 172.21.15.76:0/1350890962 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60500bd070 con 0x7f6044003be0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562fd27776b0 con 0x7f6044003be0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f604effd640 1 -- 172.21.15.76:0/1350890962 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500049b0 con 0x7f6044003be0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.684+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fd2777bf0 con 0x7f6044003be0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f604effd640 1 -- 172.21.15.76:0/1350890962 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f60500b5040 con 0x7f6044003be0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f604effd640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60380662d0 0x7f6038068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f604effd640 1 -- 172.21.15.76:0/1350890962 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f605011f630 con 0x7f6044003be0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f6055ada640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60380662d0 0x7f6038068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f6057fc7080 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x562fd27785d0 0x562fd27789b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- osd_op(unknown.0.0:1 15.5 15:a367a3e8:::10000000004.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x562fd2825650 con 0x562fd27785d0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f6055ada640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60380662d0 0x7f6038068790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6048001ee0 tx=0x7f6048002000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f60552d9640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x562fd27785d0 0x562fd27789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.688+0000 7f60552d9640 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x562fd27785d0 0x562fd27789b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f60552d9640 1 -- 172.21.15.76:0/1350890962 <== osd.6 v2:172.21.15.116:6816/2407229160 1 ==== osd_op_reply(1 10000000004.00000000 [rmxattr] v106'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f6040008040 con 0x562fd27785d0 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x562fd27785d0 msgr2=0x562fd27789b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x562fd27785d0 0x562fd27789b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60380662d0 msgr2=0x7f6038068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f60380662d0 0x7f6038068790 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6048001ee0 tx=0x7f6048002000 comp rx=0 tx=0).stop 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 msgr2=0x562fd2825b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 --2- 172.21.15.76:0/1350890962 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6044003be0 0x562fd2825b80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f605000a790 tx=0x7f6050005d90 comp rx=0 tx=0).stop 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f60562db640 1 -- 172.21.15.76:0/1350890962 reap_dead start 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 shutdown_connections 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 >> 172.21.15.76:0/1350890962 conn(0x562fd274edc0 msgr2=0x562fd280ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 shutdown_connections 2024-09-19T04:20:14.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.692+0000 7f6057fc7080 1 -- 172.21.15.76:0/1350890962 wait complete. 2024-09-19T04:20:14.695 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000005.00000000 parent 2024-09-19T04:20:14.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 Processor -- start 2024-09-19T04:20:14.772 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- start start 2024-09-19T04:20:14.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0a23370 0x5577d0ac0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 0x5577d09f4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0ac7150 0x5577d09f4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5577d0a3de10 con 0x5577d0a23370 2024-09-19T04:20:14.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5577d09441b0 con 0x5577d0a3a030 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5577d0ad2110 con 0x5577d0ac7150 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0a23370 0x5577d0ac0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0a23370 0x5577d0ac0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51492/0 (socket says 172.21.15.76:51492) 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 -- 172.21.15.76:0/2943558397 learned_addr learned my addr 172.21.15.76:0/2943558397 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f2798ad2640 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 0x5577d09f4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f2799ad4640 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0ac7150 0x5577d09f4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 -- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0ac7150 msgr2=0x5577d09f4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0ac7150 0x5577d09f4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 -- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 msgr2=0x5577d09f4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 0x5577d09f4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 -- 172.21.15.76:0/2943558397 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d0abffd0 con 0x5577d0a23370 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f27992d3640 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0a23370 0x5577d0ac0bd0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f27940016a0 tx=0x7f2794001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ba0fccfddfd6ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f278b7fe640 1 -- 172.21.15.76:0/2943558397 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794017070 con 0x5577d0a23370 2024-09-19T04:20:14.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f278b7fe640 1 -- 172.21.15.76:0/2943558397 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f279400f730 con 0x5577d0a23370 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f278b7fe640 1 -- 172.21.15.76:0/2943558397 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279400fb20 con 0x5577d0a23370 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- 172.21.15.76:0/2943558397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0a23370 msgr2=0x5577d0ac0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0a23370 0x5577d0ac0bd0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f27940016a0 tx=0x7f2794001b60 comp rx=0 tx=0).stop 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- 172.21.15.76:0/2943558397 shutdown_connections 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0ac7150 0x5577d09f4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 0x5577d09f4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2943558397 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0a23370 0x5577d0ac0bd0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- 172.21.15.76:0/2943558397 >> 172.21.15.76:0/2943558397 conn(0x5577d0a1e060 msgr2=0x5577d09f6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- 172.21.15.76:0/2943558397 shutdown_connections 2024-09-19T04:20:14.776 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.772+0000 7f279b7c0080 1 -- 172.21.15.76:0/2943558397 wait complete. 2024-09-19T04:20:14.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 Processor -- start 2024-09-19T04:20:14.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 -- start start 2024-09-19T04:20:14.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0a23370 0x5577d0abe0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 0x5577d0a33e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0ac7150 0x5577d09f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5577d0ac0510 con 0x5577d0ac7150 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5577d0ac0270 con 0x5577d0a3a030 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5577d0abe810 con 0x5577d0a23370 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0ac7150 0x5577d09f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0ac7150 0x5577d09f19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51496/0 (socket says 172.21.15.76:51496) 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 -- 172.21.15.76:0/2041524972 learned_addr learned my addr 172.21.15.76:0/2041524972 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f27992d3640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0a23370 0x5577d0abe0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2798ad2640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 0x5577d0a33e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 -- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0a23370 msgr2=0x5577d0abe0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5577d0a23370 0x5577d0abe0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 -- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 msgr2=0x5577d0a33e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5577d0a3a030 0x5577d0a33e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 -- 172.21.15.76:0/2041524972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577d0a348b0 con 0x5577d0ac7150 2024-09-19T04:20:14.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f2799ad4640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0ac7150 0x5577d09f19e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f278c017fd0 tx=0x7f278c018160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f278a7fc640 1 -- 172.21.15.76:0/2041524972 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0b9070 con 0x5577d0ac7150 2024-09-19T04:20:14.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f278a7fc640 1 -- 172.21.15.76:0/2041524972 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f278c0b5440 con 0x5577d0ac7150 2024-09-19T04:20:14.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5577d0a34b50 con 0x5577d0ac7150 2024-09-19T04:20:14.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f278a7fc640 1 -- 172.21.15.76:0/2041524972 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c0af040 con 0x5577d0ac7150 2024-09-19T04:20:14.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577d0a35090 con 0x5577d0ac7150 2024-09-19T04:20:14.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.776+0000 7f278a7fc640 1 -- 172.21.15.76:0/2041524972 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f278c0bc070 con 0x5577d0ac7150 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f278a7fc640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f277c0662d0 0x7f277c068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f278a7fc640 1 -- 172.21.15.76:0/2041524972 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f278c11f7f0 con 0x5577d0ac7150 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f27992d3640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f277c0662d0 0x7f277c068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5577d0a34390 0x5577d0a380a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- osd_op(unknown.0.0:1 15.9 15:9258b67d:::10000000005.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x5577d0a385e0 con 0x5577d0a34390 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f27992d3640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f277c0662d0 0x7f277c068790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2794001360 tx=0x7f2794008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f2798ad2640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5577d0a34390 0x5577d0a380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f2798ad2640 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5577d0a34390 0x5577d0a380a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f2798ad2640 1 -- 172.21.15.76:0/2041524972 <== osd.6 v2:172.21.15.116:6816/2407229160 1 ==== osd_op_reply(1 10000000005.00000000 [rmxattr] v106'6 uv6 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f27900099b0 con 0x5577d0a34390 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5577d0a34390 msgr2=0x5577d0a380a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x5577d0a34390 0x5577d0a380a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f277c0662d0 msgr2=0x7f277c068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f277c0662d0 0x7f277c068790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2794001360 tx=0x7f2794008030 comp rx=0 tx=0).stop 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0ac7150 msgr2=0x5577d09f19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 --2- 172.21.15.76:0/2041524972 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5577d0ac7150 0x5577d09f19e0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f278c017fd0 tx=0x7f278c018160 comp rx=0 tx=0).stop 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f2799ad4640 1 -- 172.21.15.76:0/2041524972 reap_dead start 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 shutdown_connections 2024-09-19T04:20:14.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 >> 172.21.15.76:0/2041524972 conn(0x5577d0a1e060 msgr2=0x5577d0a1f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 shutdown_connections 2024-09-19T04:20:14.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.780+0000 7f279b7c0080 1 -- 172.21.15.76:0/2041524972 wait complete. 2024-09-19T04:20:14.786 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000006.00000000 parent 2024-09-19T04:20:14.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 Processor -- start 2024-09-19T04:20:14.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 -- start start 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a64d390 0x561c3a72a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a67de20 0x561c3a721ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a725730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561c3a6981b0 con 0x561c3a729750 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561c3a59f1b0 con 0x561c3a64d390 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c2ba7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561c3a698330 con 0x561c3a67de20 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a725730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a725730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51520/0 (socket says 172.21.15.76:51520) 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 -- 172.21.15.76:0/3166748746 learned_addr learned my addr 172.21.15.76:0/3166748746 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 -- 172.21.15.76:0/3166748746 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a67de20 msgr2=0x561c3a721ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 --2- 172.21.15.76:0/3166748746 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a67de20 0x561c3a721ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 -- 172.21.15.76:0/3166748746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a64d390 msgr2=0x561c3a72a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 --2- 172.21.15.76:0/3166748746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a64d390 0x561c3a72a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 -- 172.21.15.76:0/3166748746 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3a651190 con 0x561c3a729750 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17c0ebb640 1 --2- 172.21.15.76:0/3166748746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a725730 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b660 tx=0x7f17b800bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe5186273b4dcef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17beeb7640 1 -- 172.21.15.76:0/3166748746 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8014030 con 0x561c3a729750 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17beeb7640 1 -- 172.21.15.76:0/3166748746 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b80096b0 con 0x561c3a729750 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.864+0000 7f17beeb7640 1 -- 172.21.15.76:0/3166748746 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8009aa0 con 0x561c3a729750 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- 172.21.15.76:0/3166748746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 msgr2=0x561c3a725730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/3166748746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a725730 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b660 tx=0x7f17b800bb20 comp rx=0 tx=0).stop 2024-09-19T04:20:14.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- 172.21.15.76:0/3166748746 shutdown_connections 2024-09-19T04:20:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/3166748746 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a725730 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/3166748746 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a67de20 0x561c3a721ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/3166748746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a64d390 0x561c3a72a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- 172.21.15.76:0/3166748746 >> 172.21.15.76:0/3166748746 conn(0x561c3a679220 msgr2=0x561c3a67e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- 172.21.15.76:0/3166748746 shutdown_connections 2024-09-19T04:20:14.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- 172.21.15.76:0/3166748746 wait complete. 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 Processor -- start 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- start start 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a64d390 0x561c3a6949f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a67de20 0x561c3a64fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a64ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561c3a6516d0 con 0x561c3a729750 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561c3a651430 con 0x561c3a67de20 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561c3a651970 con 0x561c3a64d390 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a64ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a64ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51534/0 (socket says 172.21.15.76:51534) 2024-09-19T04:20:14.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 -- 172.21.15.76:0/2620765206 learned_addr learned my addr 172.21.15.76:0/2620765206 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c06ba640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a64d390 0x561c3a6949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17bfeb9640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a67de20 0x561c3a64fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 -- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a64d390 msgr2=0x561c3a6949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561c3a64d390 0x561c3a6949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 -- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a67de20 msgr2=0x561c3a64fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a67de20 0x561c3a64fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 -- 172.21.15.76:0/2620765206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c3a690c70 con 0x561c3a729750 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17bfeb9640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561c3a67de20 0x561c3a64fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c0ebb640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a64ff70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f17b80094a0 tx=0x7f17b80033b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17bdeb5640 1 -- 172.21.15.76:0/2620765206 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8010c00 con 0x561c3a729750 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17bdeb5640 1 -- 172.21.15.76:0/2620765206 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17b8010dc0 con 0x561c3a729750 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17bdeb5640 1 -- 172.21.15.76:0/2620765206 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b8023680 con 0x561c3a729750 2024-09-19T04:20:14.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561c3a690f10 con 0x561c3a729750 2024-09-19T04:20:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.868+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c3a6929a0 con 0x561c3a729750 2024-09-19T04:20:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17bdeb5640 1 -- 172.21.15.76:0/2620765206 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f17b8017070 con 0x561c3a729750 2024-09-19T04:20:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17bdeb5640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17ac0662d0 0x7f17ac068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17bdeb5640 1 -- 172.21.15.76:0/2620765206 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f17b8086a90 con 0x561c3a729750 2024-09-19T04:20:14.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17c06ba640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17ac0662d0 0x7f17ac068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x561c3a693e50 0x561c3a68fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- osd_op(unknown.0.0:1 15.5 15:a5feb6f3:::10000000006.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x561c3a690050 con 0x561c3a693e50 2024-09-19T04:20:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17c06ba640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17ac0662d0 0x7f17ac068790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000f70 tx=0x7f17b0001480 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17bfeb9640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x561c3a693e50 0x561c3a68fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.872+0000 7f17bfeb9640 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x561c3a693e50 0x561c3a68fb10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17bfeb9640 1 -- 172.21.15.76:0/2620765206 <== osd.6 v2:172.21.15.116:6816/2407229160 1 ==== osd_op_reply(1 10000000006.00000000 [rmxattr] v106'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f17b40a8070 con 0x561c3a693e50 2024-09-19T04:20:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x561c3a693e50 msgr2=0x561c3a68fb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x561c3a693e50 0x561c3a68fb10 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17ac0662d0 msgr2=0x7f17ac068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f17ac0662d0 0x7f17ac068790 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f17b0000f70 tx=0x7f17b0001480 comp rx=0 tx=0).stop 2024-09-19T04:20:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 msgr2=0x561c3a64ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 --2- 172.21.15.76:0/2620765206 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561c3a729750 0x561c3a64ff70 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f17b80094a0 tx=0x7f17b80033b0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c0ebb640 1 -- 172.21.15.76:0/2620765206 reap_dead start 2024-09-19T04:20:14.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 shutdown_connections 2024-09-19T04:20:14.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 >> 172.21.15.76:0/2620765206 conn(0x561c3a679220 msgr2=0x561c3a67e290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 shutdown_connections 2024-09-19T04:20:14.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.876+0000 7f17c2ba7080 1 -- 172.21.15.76:0/2620765206 wait complete. 2024-09-19T04:20:14.879 DEBUG:teuthology.orchestra.run.smithi076:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000007.00000000 parent 2024-09-19T04:20:14.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 Processor -- start 2024-09-19T04:20:14.962 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 -- start start 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07e231d0 0x55bd07edfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07ee0bb0 0x55bd07ee3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bd07e56550 con 0x55bd07e55dd0 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bd07d741b0 con 0x55bd07ee0bb0 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b78db4080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bd07e567f0 con 0x55bd07e231d0 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.964 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51544/0 (socket says 172.21.15.76:51544) 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 -- 172.21.15.76:0/2504383106 learned_addr learned my addr 172.21.15.76:0/2504383106 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 -- 172.21.15.76:0/2504383106 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07e231d0 msgr2=0x55bd07edfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 --2- 172.21.15.76:0/2504383106 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07e231d0 0x55bd07edfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 -- 172.21.15.76:0/2504383106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07ee0bb0 msgr2=0x55bd07ee3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 --2- 172.21.15.76:0/2504383106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07ee0bb0 0x55bd07ee3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 -- 172.21.15.76:0/2504383106 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd07efd2c0 con 0x55bd07e55dd0 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.960+0000 7f1b6ffff640 1 --2- 172.21.15.76:0/2504383106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee37c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000a720 tx=0x7f1b6000abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4982aadd1d6cadb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b758c5640 1 -- 172.21.15.76:0/2504383106 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6001a070 con 0x55bd07e55dd0 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b758c5640 1 -- 172.21.15.76:0/2504383106 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6000ae30 con 0x55bd07e55dd0 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b758c5640 1 -- 172.21.15.76:0/2504383106 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60008b90 con 0x55bd07e55dd0 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- 172.21.15.76:0/2504383106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 msgr2=0x55bd07ee37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.965 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 --2- 172.21.15.76:0/2504383106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee37c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000a720 tx=0x7f1b6000abe0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- 172.21.15.76:0/2504383106 shutdown_connections 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 --2- 172.21.15.76:0/2504383106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07ee0bb0 0x55bd07ee3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 --2- 172.21.15.76:0/2504383106 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee37c0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 --2- 172.21.15.76:0/2504383106 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07e231d0 0x55bd07edfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- 172.21.15.76:0/2504383106 >> 172.21.15.76:0/2504383106 conn(0x55bd07e23dc0 msgr2=0x55bd07efd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- 172.21.15.76:0/2504383106 shutdown_connections 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- 172.21.15.76:0/2504383106 wait complete. 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 Processor -- start 2024-09-19T04:20:14.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- start start 2024-09-19T04:20:14.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07e231d0 0x55bd07ef5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07ee0bb0 0x55bd07ee3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55bd07edeae0 con 0x55bd07e55dd0 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55bd07eded80 con 0x55bd07e231d0 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55bd07f04880 con 0x55bd07ee0bb0 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51554/0 (socket says 172.21.15.76:51554) 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 -- 172.21.15.76:0/3048762783 learned_addr learned my addr 172.21.15.76:0/3048762783 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b768c7640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07e231d0 0x55bd07ef5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b770c8640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07ee0bb0 0x55bd07ee3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 -- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07ee0bb0 msgr2=0x55bd07ee3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55bd07ee0bb0 0x55bd07ee3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 -- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07e231d0 msgr2=0x55bd07ef5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55bd07e231d0 0x55bd07ef5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 -- 172.21.15.76:0/3048762783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd07f04b20 con 0x55bd07e55dd0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b6ffff640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee35f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1b60005aa0 tx=0x7f1b60003030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b748c3640 1 -- 172.21.15.76:0/3048762783 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60024020 con 0x55bd07e55dd0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b748c3640 1 -- 172.21.15.76:0/3048762783 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b60008e80 con 0x55bd07e55dd0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bd07f93590 con 0x55bd07e55dd0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b748c3640 1 -- 172.21.15.76:0/3048762783 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6001a040 con 0x55bd07e55dd0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.964+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd07f04dc0 con 0x55bd07e55dd0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b748c3640 1 -- 172.21.15.76:0/3048762783 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1b60027020 con 0x55bd07e55dd0 2024-09-19T04:20:14.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b748c3640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b580662d0 0x7f1b58068790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b768c7640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b580662d0 0x7f1b58068790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b748c3640 1 -- 172.21.15.76:0/3048762783 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f1b60086050 con 0x55bd07e55dd0 2024-09-19T04:20:14.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b78db4080 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55bd07f93cd0 0x55bd07f940b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:14.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 --> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] -- osd_op(unknown.0.0:1 15.1d 15:bbf5850a:::10000000007.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106) -- 0x55bd07f945f0 con 0x55bd07f93cd0 2024-09-19T04:20:14.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b768c7640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b580662d0 0x7f1b58068790 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1b680ac000 tx=0x7f1b68003030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b770c8640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55bd07f93cd0 0x55bd07f940b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:14.971 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.968+0000 7f1b770c8640 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55bd07f93cd0 0x55bd07f940b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:14.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b770c8640 1 -- 172.21.15.76:0/3048762783 <== osd.4 v2:172.21.15.116:6800/3265669393 1 ==== osd_op_reply(1 10000000007.00000000 [rmxattr] v106'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f1b70001660 con 0x55bd07f93cd0 2024-09-19T04:20:14.972 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55bd07f93cd0 msgr2=0x55bd07f940b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x55bd07f93cd0 0x55bd07f940b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b580662d0 msgr2=0x7f1b58068790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1b580662d0 0x7f1b58068790 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1b680ac000 tx=0x7f1b68003030 comp rx=0 tx=0).stop 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 msgr2=0x55bd07ee35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 --2- 172.21.15.76:0/3048762783 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55bd07e55dd0 0x55bd07ee35f0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f1b60005aa0 tx=0x7f1b60003030 comp rx=0 tx=0).stop 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b770c8640 1 -- 172.21.15.76:0/3048762783 reap_dead start 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 shutdown_connections 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 >> 172.21.15.76:0/3048762783 conn(0x55bd07e23dc0 msgr2=0x55bd07efa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 shutdown_connections 2024-09-19T04:20:14.973 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:14.972+0000 7f1b78db4080 1 -- 172.21.15.76:0/3048762783 wait complete. 2024-09-19T04:20:14.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:14.976 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /d1/d2/d3 recursive,force 2024-09-19T04:20:15.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd958528640 1 Processor -- start 2024-09-19T04:20:15.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd958528640 1 -- start start 2024-09-19T04:20:15.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd958528640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 0x7fd95006e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd958528640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95006ee50 con 0x7fd950073e60 2024-09-19T04:20:15.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd958528640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd95006f130 con 0x7fd95006fef0 2024-09-19T04:20:15.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd958528640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd95006f410 con 0x7fd9500b4460 2024-09-19T04:20:15.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd9562c4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd950073e60 0x7fd95015b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51812/0 (socket says 172.21.15.76:51812) 2024-09-19T04:20:15.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd9562c4640 1 -- 172.21.15.76:0/231191604 learned_addr learned my addr 172.21.15.76:0/231191604 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:15.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd956ac5640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 0x7fd95006e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422622185 0 0) 0x7fd95006ee50 con 0x7fd950073e60 2024-09-19T04:20:15.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.148+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd93c003530 con 0x7fd950073e60 2024-09-19T04:20:15.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1030018587 0 0) 0x7fd93c003530 con 0x7fd950073e60 2024-09-19T04:20:15.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 msgr2=0x7fd95006e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 0x7fd95006e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 >> v1:172.21.15.116:6789/0 conn(0x7fd95006fef0 legacy=0x7fd9500b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95006f6f0 con 0x7fd950073e60 2024-09-19T04:20:15.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fd94c001a00 con 0x7fd950073e60 2024-09-19T04:20:15.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 >> v1:172.21.15.76:6789/0 conn(0x7fd950073e60 legacy=0x7fd95015b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd93c0039f0 0x7fd93c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 0x7fd93c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd93c004d50 0x7fd93c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd95006f410 con 0x7fd93c0039f0 2024-09-19T04:20:15.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd93c0053e0 con 0x7fd93c004d50 2024-09-19T04:20:15.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd93c005590 con 0x7fd9500b4460 2024-09-19T04:20:15.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9562c4640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd93c0039f0 0x7fd93c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd956ac5640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd93c004d50 0x7fd93c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd955ac3640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 0x7fd93c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9562c4640 1 -- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 msgr2=0x7fd93c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9562c4640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd9500b4460 0x7fd93c004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9562c4640 1 -- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd93c004d50 msgr2=0x7fd93c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9562c4640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd93c004d50 0x7fd93c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9562c4640 1 -- 172.21.15.76:0/231191604 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93c003530 con 0x7fd93c0039f0 2024-09-19T04:20:15.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd956ac5640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd93c004d50 0x7fd93c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9562c4640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd93c0039f0 0x7fd93c003e10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00a860 tx=0x7fd94c01d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c7739ea9d5506e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00a480 con 0x7fd93c0039f0 2024-09-19T04:20:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd94c0a5e50 con 0x7fd93c0039f0 2024-09-19T04:20:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd9552c2640 1 -- 172.21.15.76:0/231191604 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c0ad8b0 con 0x7fd93c0039f0 2024-09-19T04:20:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd958528640 1 -- 172.21.15.76:0/231191604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd93c0039f0 msgr2=0x7fd93c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd958528640 1 --2- 172.21.15.76:0/231191604 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd93c0039f0 0x7fd93c003e10 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00a860 tx=0x7fd94c01d380 comp rx=0 tx=0).stop 2024-09-19T04:20:15.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd956ac5640 1 -- 172.21.15.76:0/231191604 reap_dead start 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd958528640 1 -- 172.21.15.76:0/231191604 shutdown_connections 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd958528640 1 -- 172.21.15.76:0/231191604 >> 172.21.15.76:0/231191604 conn(0x7fd9500b1f90 msgr2=0x7fd9500b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd958528640 1 -- 172.21.15.76:0/231191604 shutdown_connections 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.152+0000 7fd958528640 1 -- 172.21.15.76:0/231191604 wait complete. 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 Processor -- start 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 -- start start 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd95014b060 0x7fd95009bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9501f2a40 0x7fd95009c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9501f2ec0 0x7fd95009ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd95006eb10 con 0x7fd9501f2a40 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd95006e990 con 0x7fd9501f2ec0 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd950165790 con 0x7fd95014b060 2024-09-19T04:20:15.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9501f2a40 0x7fd95009c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9501f2a40 0x7fd95009c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51574/0 (socket says 172.21.15.76:51574) 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 -- 172.21.15.76:0/1302099534 learned_addr learned my addr 172.21.15.76:0/1302099534 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd956ac5640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9501f2ec0 0x7fd95009ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd9562c4640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd95014b060 0x7fd95009bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 -- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd95014b060 msgr2=0x7fd95009bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd95014b060 0x7fd95009bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 -- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9501f2ec0 msgr2=0x7fd95009ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9501f2ec0 0x7fd95009ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 -- 172.21.15.76:0/1302099534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd95009b150 con 0x7fd9501f2a40 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd955ac3640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9501f2a40 0x7fd95009c550 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd94000d570 tx=0x7fd94000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd946ffd640 1 -- 172.21.15.76:0/1302099534 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940005480 con 0x7fd9501f2a40 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd946ffd640 1 -- 172.21.15.76:0/1302099534 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd940017070 con 0x7fd9501f2a40 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd95009b430 con 0x7fd9501f2a40 2024-09-19T04:20:15.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd946ffd640 1 -- 172.21.15.76:0/1302099534 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940003820 con 0x7fd9501f2a40 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.156+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501523d0 con 0x7fd9501f2a40 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 Processor -- start 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd946ffd640 1 -- 172.21.15.76:0/1302099534 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd9400039c0 con 0x7fd9501f2a40 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- start start 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd946ffd640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9340678f0 0x7fd934069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd946ffd640 1 -- 172.21.15.76:0/1302099534 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fd940085ca0 con 0x7fd9501f2a40 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9340678f0 0x7fd934069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc06daf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06912f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc0691c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ccc04dfbd0 con 0x55ccc0691830 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ccc04852f0 con 0x55ccc06db460 2024-09-19T04:20:15.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ccc04dbb90 con 0x55ccc06dab40 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc0691c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc0691c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51590/0 (socket says 172.21.15.76:51590) 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 -- 172.21.15.76:0/323623657 learned_addr learned my addr 172.21.15.76:0/323623657 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9340678f0 0x7fd934069db0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd95006dd30 tx=0x7fd948008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd955ac3640 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06912f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd9562c4640 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc06daf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 -- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 msgr2=0x55ccc06daf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc06daf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 -- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 msgr2=0x55ccc06912f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06912f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 -- 172.21.15.76:0/323623657 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc04abb10 con 0x55ccc0691830 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd956ac5640 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc0691c10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd95015f4a0 tx=0x7fd948017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc17990672576c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.177 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd933fff640 1 -- 172.21.15.76:0/323623657 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948027070 con 0x55ccc0691830 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd933fff640 1 -- 172.21.15.76:0/323623657 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd948014e00 con 0x55ccc0691830 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd933fff640 1 -- 172.21.15.76:0/323623657 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948016b30 con 0x55ccc0691830 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- 172.21.15.76:0/323623657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 msgr2=0x55ccc0691c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc0691c10 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fd95015f4a0 tx=0x7fd948017000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- 172.21.15.76:0/323623657 shutdown_connections 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc0691c10 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06912f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/323623657 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc06daf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- 172.21.15.76:0/323623657 >> 172.21.15.76:0/323623657 conn(0x55ccc068d7e0 msgr2=0x55ccc068fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- 172.21.15.76:0/323623657 shutdown_connections 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.160+0000 7fd95b5e4000 1 -- 172.21.15.76:0/323623657 wait complete. 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 -- start start 2024-09-19T04:20:15.178 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 Processor -- start 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc0691830 0x55ccc069b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06dab40 0x55ccc069be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06db460 0x55ccc069c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ccc0561700 con 0x55ccc06dab40 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55ccc04795c0 con 0x55ccc0691830 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ccc0489500 con 0x55ccc06db460 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 10 client.0 _ll_register_callbacks cb 0x55ccc05935a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06dab40 0x55ccc069be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06dab40 0x55ccc069be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51592/0 (socket says 172.21.15.76:51592) 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 -- 172.21.15.76:0/83634644 learned_addr learned my addr 172.21.15.76:0/83634644 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9562c4640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc0691830 0x55ccc069b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd956ac5640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06db460 0x55ccc069c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 -- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06db460 msgr2=0x55ccc069c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06db460 0x55ccc069c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 -- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc0691830 msgr2=0x55ccc069b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc0691830 0x55ccc069b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 -- 172.21.15.76:0/83634644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc0485500 con 0x55ccc06dab40 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd956ac5640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06db460 0x55ccc069c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd955ac3640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06dab40 0x55ccc069be30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd94000df30 tx=0x7fd940056630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f405712152814f8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 1 -- 172.21.15.76:0/83634644 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940081070 con 0x55ccc06dab40 2024-09-19T04:20:15.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 1 -- 172.21.15.76:0/83634644 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fd9400554c0 con 0x55ccc06dab40 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd948014e00 con 0x55ccc06dab40 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 1 -- 172.21.15.76:0/83634644 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fd94005eae0 con 0x55ccc06dab40 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 10 client.8484 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55ccc047f550 con 0x55ccc06dab40 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 1 -- 172.21.15.76:0/83634644 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd948014e00 con 0x55ccc06dab40 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 1 -- 172.21.15.76:0/83634644 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7fd940051bc0 con 0x55ccc06dab40 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 10 client.8484 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 10 client.8484 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 20 client.8484 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06e9bd0 0x55ccc06ebfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 4 client.8484 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:15.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccc0479d00 con 0x55ccc06e9bd0 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9562c4640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06e9bd0 0x55ccc06ebfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 10 client.8484 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9562c4640 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06e9bd0 0x55ccc06ebfc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faee4f047b55f854 server_cookie=9dfbe12a0f0a4362 in_seq=0 out_seq=0 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 1 -- 172.21.15.76:0/83634644 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ccc0479d00 con 0x55ccc06e9bd0 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd9317fa640 10 client.8484 handle_command_reply: tid=0 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.164+0000 7fd95b5e4000 1 client.8484 shutdown 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06dab40 msgr2=0x55ccc069be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06dab40 0x55ccc069be30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fd94000df30 tx=0x7fd940056630 comp rx=0 tx=0).stop 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 shutdown_connections 2024-09-19T04:20:15.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06e9bd0 0x55ccc06ebfc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06db460 0x55ccc069c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06dab40 0x55ccc069be30 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/83634644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc0691830 0x55ccc069b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd9317fa640 0 client.8484 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 >> 172.21.15.76:0/83634644 conn(0x55ccc068d7e0 msgr2=0x55ccc06903f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 shutdown_connections 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 1 -- 172.21.15.76:0/83634644 wait complete. 2024-09-19T04:20:15.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.168+0000 7fd95b5e4000 20 client.8484 trim_cache size 0 max 16384 2024-09-19T04:20:15.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 Processor -- start 2024-09-19T04:20:15.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 -- start start 2024-09-19T04:20:15.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc0691830 0x55ccc07496a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06dab40 0x55ccc0749be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06db460 0x55ccc074a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.187 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd9502005f0 con 0x55ccc06db460 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501ff7c0 con 0x55ccc06dab40 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd9501ffec0 con 0x55ccc0691830 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06db460 0x55ccc074a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06db460 0x55ccc074a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51594/0 (socket says 172.21.15.76:51594) 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 -- 172.21.15.76:0/3323901811 learned_addr learned my addr 172.21.15.76:0/3323901811 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 -- 172.21.15.76:0/3323901811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc0691830 msgr2=0x55ccc07496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 --2- 172.21.15.76:0/3323901811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc0691830 0x55ccc07496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 -- 172.21.15.76:0/3323901811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06dab40 msgr2=0x55ccc0749be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 --2- 172.21.15.76:0/3323901811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06dab40 0x55ccc0749be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 -- 172.21.15.76:0/3323901811 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccc0561b90 con 0x55ccc06db460 2024-09-19T04:20:15.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd956ac5640 1 --2- 172.21.15.76:0/3323901811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06db460 0x55ccc074a120 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd948016800 tx=0x7fd94801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4af98a1561d77d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd9337fe640 1 -- 172.21.15.76:0/3323901811 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948035070 con 0x55ccc06db460 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd9337fe640 1 -- 172.21.15.76:0/3323901811 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd948016a30 con 0x55ccc06db460 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.184+0000 7fd9337fe640 1 -- 172.21.15.76:0/3323901811 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948027070 con 0x55ccc06db460 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3323901811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06db460 msgr2=0x55ccc074a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3323901811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06db460 0x55ccc074a120 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd948016800 tx=0x7fd94801a000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3323901811 shutdown_connections 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3323901811 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc06db460 0x55ccc074a120 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3323901811 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06dab40 0x55ccc0749be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3323901811 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc0691830 0x55ccc07496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3323901811 >> 172.21.15.76:0/3323901811 conn(0x55ccc068d7e0 msgr2=0x55ccc0695cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3323901811 shutdown_connections 2024-09-19T04:20:15.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3323901811 wait complete. 2024-09-19T04:20:15.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- start start 2024-09-19T04:20:15.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 Processor -- start 2024-09-19T04:20:15.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc074df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc074e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06ee300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55ccc04deee0 con 0x55ccc0691830 2024-09-19T04:20:15.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd950154c40 con 0x55ccc06db460 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55ccc0561700 con 0x55ccc06dab40 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 10 client.0 _ll_register_callbacks cb 0x7fd940001240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc074df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc074df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51598/0 (socket says 172.21.15.76:51598) 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 -- 172.21.15.76:0/3293490231 learned_addr learned my addr 172.21.15.76:0/3293490231 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd956ac5640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd955ac3640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc074e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 -- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 msgr2=0x55ccc074e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc074e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 -- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 msgr2=0x55ccc06ee300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06ee300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 -- 172.21.15.76:0/3293490231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccc0538ca0 con 0x55ccc0691830 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd956ac5640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9562c4640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc074df90 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00a860 tx=0x7fd93c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a9d5296d2d464c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9317fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9317fa640 1 -- 172.21.15.76:0/3293490231 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94c00a480 con 0x55ccc0691830 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9317fa640 1 -- 172.21.15.76:0/3293490231 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fd94c001740 con 0x55ccc0691830 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd948016a30 con 0x55ccc0691830 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9317fa640 1 -- 172.21.15.76:0/3293490231 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fd94c0cc050 con 0x55ccc0691830 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 10 client.8490 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x55ccc019a4c0 con 0x55ccc0691830 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9317fa640 1 -- 172.21.15.76:0/3293490231 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd948016a30 con 0x55ccc0691830 2024-09-19T04:20:15.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9317fa640 1 -- 172.21.15.76:0/3293490231 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7fd94c0cf020 con 0x55ccc0691830 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 10 client.8490 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 10 client.8490 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 20 client.8490 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06ee840 0x55ccc06eec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 4 client.8490 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}] 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub start", "path": "/d1/d2/d3", "scrubops": ["recursive", "force"]}) -- 0x55ccc04ed300 con 0x55ccc06ee840 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd955ac3640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06ee840 0x55ccc06eec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd9317fa640 10 client.8490 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.188+0000 7fd955ac3640 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06ee840 0x55ccc06eec50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a53a232482dcb4a server_cookie=add967e28103ae7d in_seq=0 out_seq=0 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd9317fa640 1 -- 172.21.15.76:0/3293490231 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ccc04ed300 con 0x55ccc06ee840 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd9317fa640 10 client.8490 handle_command_reply: tid=0 2024-09-19T04:20:15.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 client.8490 shutdown 2024-09-19T04:20:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 msgr2=0x55ccc074df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc074df90 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00a860 tx=0x7fd93c005840 comp rx=0 tx=0).stop 2024-09-19T04:20:15.199 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:20:15.192+0000 7f3e43560640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:19:57.385751+0000 1=0+1) n(v3 rc2024-09-19T04:19:57.608824+0000 6=0+6) (inest mix dirty) (ifile excl) caps={8280=pAsLsXsFsx/AsLsXsFsx@10},l=8280 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558bc0175b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.385751+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.608824+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.385751+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:19:57.608824+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 shutdown_connections 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55ccc06ee840 0x55ccc06eec50 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ccc06db460 0x55ccc06ee300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55ccc06dab40 0x55ccc074e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 --2- 172.21.15.76:0/3293490231 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55ccc0691830 0x55ccc074df90 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd9317fa640 0 client.8490 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 >> 172.21.15.76:0/3293490231 conn(0x55ccc068d7e0 msgr2=0x55ccc06964e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 shutdown_connections 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 1 -- 172.21.15.76:0/3293490231 wait complete. 2024-09-19T04:20:15.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd95b5e4000 20 client.8490 trim_cache size 0 max 16384 2024-09-19T04:20:15.201 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:15.201 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0, 2024-09-19T04:20:15.201 INFO:teuthology.orchestra.run.smithi076.stdout: "scrub_tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:15.201 INFO:teuthology.orchestra.run.smithi076.stdout: "mode": "asynchronous" 2024-09-19T04:20:15.201 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:15.202 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:20:15.192+0000 7faffbf0a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:19:57.390751+0000 1=0+1) n(v2 rc2024-09-19T04:19:57.608824+0000 4=0+4) (inest lock) caps={8280=pAsLsXsFs/-@7},l=8280 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x564e4d11db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.390751+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.608824+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.390751+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:19:57.608824+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9340678f0 msgr2=0x7fd934069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9340678f0 0x7fd934069db0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd95006dd30 tx=0x7fd948008040 comp rx=0 tx=0).stop 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9501f2a40 msgr2=0x7fd95009c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9501f2a40 0x7fd95009c550 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7fd94000d570 tx=0x7fd94000da30 comp rx=0 tx=0).stop 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 shutdown_connections 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd9340678f0 0x7fd934069db0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9501f2ec0 0x7fd95009ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd9501f2a40 0x7fd95009c550 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 --2- 172.21.15.76:0/1302099534 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd95014b060 0x7fd95009bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 >> 172.21.15.76:0/1302099534 conn(0x7fd9500b1f90 msgr2=0x7fd95015b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 shutdown_connections 2024-09-19T04:20:15.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fd958528640 1 -- 172.21.15.76:0/1302099534 wait complete. 2024-09-19T04:20:15.204 INFO:tasks.ceph.mds.a.smithi076.stderr:2024-09-19T04:20:15.192+0000 7f3e49d6d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:19:57.388277+0000 1=0+1) n(v4 rc2024-09-19T04:19:57.608824+0000 5=0+5) (inest sync->mix g=1,2) caps={8280=pAsLsXsFs/-@12} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x558bc0176100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.388277+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.608824+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.388277+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:19:57.608824+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:20:15.204 INFO:tasks.ceph.mds.c.smithi076.stderr:2024-09-19T04:20:15.192+0000 7fb002717640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:19:57.393459+0000 1=0+1) n(v1 rc2024-09-19T04:19:57.608824+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={8280=pAsLsXsFs/-@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x564e4d11d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.393459+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.396196+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.393459+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:19:57.608824+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:20:15.205 INFO:tasks.ceph.mds.d.smithi116.stderr:2024-09-19T04:20:15.193+0000 7f7e951de640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:19:57.396196+0000 1=0+1) n(v0 rc2024-09-19T04:19:57.396196+0000 2=0+2) (inest lock) (ifile excl) caps={8280=pAsLsXsFsx/-@6},l=8280 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x562e08673600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.396196+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.396196+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.396196+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:19:57.396196+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:20:15.205 INFO:tasks.ceph.mds.d.smithi116.stderr:2024-09-19T04:20:15.193+0000 7f7e951de640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={8280=pAsLsXsFs/-@3},l=8280 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562e08673b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:20:15.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:15.210 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-09-19T04:20:15.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff15ddca640 1 Processor -- start 2024-09-19T04:20:15.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff15ddca640 1 -- start start 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff15ddca640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff15815bec0 0x7ff1580ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff15ddca640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff15815c2a0 0x7ff1580aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff15ddca640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1580b3290 con 0x7ff1580af2a0 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff15ddca640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff1580b3570 con 0x7ff15815c2a0 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff15ddca640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff1580b3850 con 0x7ff15815bec0 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff156ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff15815c2a0 0x7ff1580aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff156ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff15815c2a0 0x7ff1580aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51614/0 (socket says 172.21.15.76:51614) 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff156ffd640 1 -- 172.21.15.76:0/1876531479 learned_addr learned my addr 172.21.15.76:0/1876531479 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff1577fe640 1 --2- 172.21.15.76:0/1876531479 >> v2:172.21.15.116:3300/0 conn(0x7ff15815bec0 0x7ff1580ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff156ffd640 1 -- 172.21.15.76:0/1876531479 >> v2:172.21.15.116:3300/0 conn(0x7ff15815bec0 msgr2=0x7ff1580ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1876531479 >> v2:172.21.15.116:3300/0 conn(0x7ff15815bec0 0x7ff1580ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff156ffd640 1 -- 172.21.15.76:0/1876531479 >> v1:172.21.15.116:6789/0 conn(0x7ff1580af2a0 legacy=0x7ff15815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.320+0000 7ff156ffd640 1 -- 172.21.15.76:0/1876531479 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1580b3b30 con 0x7ff15815c2a0 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1876531479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815c2a0 0x7ff1580aed60 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff158071710 tx=0x7ff14001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b812c1acd23041 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff1567fc640 1 -- 172.21.15.76:0/1876531479 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400aa610 con 0x7ff15815c2a0 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff1567fc640 1 -- 172.21.15.76:0/1876531479 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400af070 con 0x7ff15815c2a0 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff1567fc640 1 -- 172.21.15.76:0/1876531479 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400aa9d0 con 0x7ff15815c2a0 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 -- 172.21.15.76:0/1876531479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815c2a0 msgr2=0x7ff1580aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 --2- 172.21.15.76:0/1876531479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815c2a0 0x7ff1580aed60 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7ff158071710 tx=0x7ff14001c4b0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 -- 172.21.15.76:0/1876531479 shutdown_connections 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 --2- 172.21.15.76:0/1876531479 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815c2a0 0x7ff1580aed60 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 --2- 172.21.15.76:0/1876531479 >> v2:172.21.15.116:3300/0 conn(0x7ff15815bec0 0x7ff1580ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 -- 172.21.15.76:0/1876531479 >> 172.21.15.76:0/1876531479 conn(0x7ff1580b1f90 msgr2=0x7ff1580b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 -- 172.21.15.76:0/1876531479 shutdown_connections 2024-09-19T04:20:15.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.324+0000 7ff15ddca640 1 -- 172.21.15.76:0/1876531479 wait complete. 2024-09-19T04:20:15.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 Processor -- start 2024-09-19T04:20:15.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 -- start start 2024-09-19T04:20:15.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1580af2a0 0x7ff1580d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815bec0 0x7ff15809aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff15815c2a0 0x7ff15809c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580ad740 con 0x7ff15815bec0 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1580af710 con 0x7ff1580af2a0 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff158169000 con 0x7ff15815c2a0 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815bec0 0x7ff15809aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815bec0 0x7ff15809aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51624/0 (socket says 172.21.15.76:51624) 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 -- 172.21.15.76:0/2130934029 learned_addr learned my addr 172.21.15.76:0/2130934029 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff1577fe640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1580af2a0 0x7ff1580d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff157fff640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff15815c2a0 0x7ff15809c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 -- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff15815c2a0 msgr2=0x7ff15809c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff15815c2a0 0x7ff15809c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 -- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1580af2a0 msgr2=0x7ff1580d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1580af2a0 0x7ff1580d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 -- 172.21.15.76:0/2130934029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15809cc00 con 0x7ff15815bec0 2024-09-19T04:20:15.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff156ffd640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815bec0 0x7ff15809aa80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff14001a0e0 tx=0x7ff140019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff14ffff640 1 -- 172.21.15.76:0/2130934029 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400be610 con 0x7ff15815bec0 2024-09-19T04:20:15.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff14ffff640 1 -- 172.21.15.76:0/2130934029 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400be7b0 con 0x7ff15815bec0 2024-09-19T04:20:15.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15809cee0 con 0x7ff15815bec0 2024-09-19T04:20:15.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff14ffff640 1 -- 172.21.15.76:0/2130934029 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400beba0 con 0x7ff15815bec0 2024-09-19T04:20:15.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15809d4a0 con 0x7ff15815bec0 2024-09-19T04:20:15.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.328+0000 7ff14ffff640 1 -- 172.21.15.76:0/2130934029 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff1400bc020 con 0x7ff15815bec0 2024-09-19T04:20:15.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff14ffff640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1380678c0 0x7ff138069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff14ffff640 1 -- 172.21.15.76:0/2130934029 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff140120580 con 0x7ff15815bec0 2024-09-19T04:20:15.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1380678c0 0x7ff138069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1380678c0 0x7ff138069d80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff150003fd0 tx=0x7ff1500039a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 Processor -- start 2024-09-19T04:20:15.333 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- start start 2024-09-19T04:20:15.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a00ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649b4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.339 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x558864805cc0 con 0x558864a00c10 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5588647c0a20 con 0x558864a01530 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x558864801c80 con 0x5588649b4780 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a00ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a00ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51634/0 (socket says 172.21.15.76:51634) 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 -- 172.21.15.76:0/580975540 learned_addr learned my addr 172.21.15.76:0/580975540 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff156ffd640 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff157fff640 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 -- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 msgr2=0x5588649b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.340 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 -- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 msgr2=0x5588649b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 -- 172.21.15.76:0/580975540 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588647a8d40 con 0x558864a00c10 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff1577fe640 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a00ff0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff1500049e0 tx=0x7ff150012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed8199b3f93c3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff14d7fa640 1 -- 172.21.15.76:0/580975540 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150026070 con 0x558864a00c10 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff14d7fa640 1 -- 172.21.15.76:0/580975540 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1500147e0 con 0x558864a00c10 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff14d7fa640 1 -- 172.21.15.76:0/580975540 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff150014b90 con 0x558864a00c10 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- 172.21.15.76:0/580975540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 msgr2=0x558864a00ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a00ff0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff1500049e0 tx=0x7ff150012000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- 172.21.15.76:0/580975540 shutdown_connections 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 --2- 172.21.15.76:0/580975540 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a00ff0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- 172.21.15.76:0/580975540 >> 172.21.15.76:0/580975540 conn(0x5588649e19b0 msgr2=0x5588649e3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- 172.21.15.76:0/580975540 shutdown_connections 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.332+0000 7ff160e86000 1 -- 172.21.15.76:0/580975540 wait complete. 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 -- start start 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 Processor -- start 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55886479f2c0 con 0x558864a00c10 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5588647c4370 con 0x558864a01530 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x558864813f50 con 0x5588649b4780 2024-09-19T04:20:15.342 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 10 client.0 _ll_register_callbacks cb 0x558864866db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51646/0 (socket says 172.21.15.76:51646) 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 -- 172.21.15.76:0/1192930358 learned_addr learned my addr 172.21.15.76:0/1192930358 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff157fff640 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 -- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 msgr2=0x5588649e0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 -- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 msgr2=0x5588649ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 -- 172.21.15.76:0/1192930358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588647a5770 con 0x558864a00c10 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff157fff640 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba2b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff140009280 tx=0x7ff1400f0ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baec5cb0fbc42b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 1 -- 172.21.15.76:0/1192930358 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400f93d0 con 0x558864a00c10 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 1 -- 172.21.15.76:0/1192930358 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff140120070 con 0x558864a00c10 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1500147e0 con 0x558864a00c10 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 1 -- 172.21.15.76:0/1192930358 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7ff1400f70b0 con 0x558864a00c10 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 10 client.8502 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x5588647c33f0 con 0x558864a00c10 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 1 -- 172.21.15.76:0/1192930358 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1500147e0 con 0x558864a00c10 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 1 -- 172.21.15.76:0/1192930358 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7ff1400c6030 con 0x558864a00c10 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 10 client.8502 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 10 client.8502 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:15.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 20 client.8502 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5588649bb200 0x5588649bb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 4 client.8502 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55886479fd90 con 0x5588649bb200 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1577fe640 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5588649bb200 0x5588649bb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1577fe640 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5588649bb200 0x5588649bb610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46df06699cc1264a server_cookie=2839f5e4c995ae1c in_seq=0 out_seq=0 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 10 client.8502 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 1 -- 172.21.15.76:0/1192930358 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55886479fd90 con 0x5588649bb200 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff1367fc640 10 client.8502 handle_command_reply: tid=0 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.336+0000 7ff160e86000 1 client.8502 shutdown 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 msgr2=0x5588649ba2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba2b0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff140009280 tx=0x7ff1400f0ef0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 shutdown_connections 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5588649bb200 0x5588649bb610 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x5588649ba7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba2b0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff1367fc640 0 client.8502 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 --2- 172.21.15.76:0/1192930358 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 >> 172.21.15.76:0/1192930358 conn(0x5588649e19b0 msgr2=0x5588649b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 shutdown_connections 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 1 -- 172.21.15.76:0/1192930358 wait complete. 2024-09-19T04:20:15.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.340+0000 7ff160e86000 20 client.8502 trim_cache size 0 max 16384 2024-09-19T04:20:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 Processor -- start 2024-09-19T04:20:15.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 -- start start 2024-09-19T04:20:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588649b4780 0x5588649b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558864a01530 0x5588649df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff1582012e0 con 0x558864a00c10 2024-09-19T04:20:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff1582019e0 con 0x5588649b4780 2024-09-19T04:20:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff15815d010 con 0x558864a01530 2024-09-19T04:20:15.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51656/0 (socket says 172.21.15.76:51656) 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 -- 172.21.15.76:0/432004547 learned_addr learned my addr 172.21.15.76:0/432004547 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 -- 172.21.15.76:0/432004547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558864a01530 msgr2=0x5588649df690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 --2- 172.21.15.76:0/432004547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558864a01530 0x5588649df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 -- 172.21.15.76:0/432004547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588649b4780 msgr2=0x5588649b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 --2- 172.21.15.76:0/432004547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588649b4780 0x5588649b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 -- 172.21.15.76:0/432004547 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558864a01360 con 0x558864a00c10 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff156ffd640 1 --2- 172.21.15.76:0/432004547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba320 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff140009630 tx=0x7ff140120000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c6463f8f4488c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff14cff9640 1 -- 172.21.15.76:0/432004547 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140120070 con 0x558864a00c10 2024-09-19T04:20:15.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff14cff9640 1 -- 172.21.15.76:0/432004547 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400af040 con 0x558864a00c10 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.356+0000 7ff14cff9640 1 -- 172.21.15.76:0/432004547 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400efc40 con 0x558864a00c10 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/432004547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 msgr2=0x5588649ba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/432004547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba320 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff140009630 tx=0x7ff140120000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/432004547 shutdown_connections 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/432004547 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x558864a01530 0x5588649df690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/432004547 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x5588649ba320 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/432004547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5588649b4780 0x5588649b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/432004547 >> 172.21.15.76:0/432004547 conn(0x5588649e19b0 msgr2=0x5588649b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/432004547 shutdown_connections 2024-09-19T04:20:15.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/432004547 wait complete. 2024-09-19T04:20:15.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- start start 2024-09-19T04:20:15.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 Processor -- start 2024-09-19T04:20:15.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649e0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a14000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x558864a14540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5588647ad120 con 0x558864a00c10 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x558864886e90 con 0x558864a01530 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5588647afb90 con 0x5588649b4780 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 10 client.0 _ll_register_callbacks cb 0x7ff1400aaf30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a14000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a14000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51672/0 (socket says 172.21.15.76:51672) 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 -- 172.21.15.76:0/1488477592 learned_addr learned my addr 172.21.15.76:0/1488477592 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 -- 172.21.15.76:0/1488477592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 msgr2=0x5588649e0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649e0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 -- 172.21.15.76:0/1488477592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 msgr2=0x558864a14540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x558864a14540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 -- 172.21.15.76:0/1488477592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558864887320 con 0x558864a00c10 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff156ffd640 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a14000 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff1400be3e0 tx=0x7ff1400ac000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecb0a34ecebcc36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 1 -- 172.21.15.76:0/1488477592 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400efe50 con 0x558864a00c10 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 1 -- 172.21.15.76:0/1488477592 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff1400f1b20 con 0x558864a00c10 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1400af040 con 0x558864a00c10 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 1 -- 172.21.15.76:0/1488477592 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7ff1400f0330 con 0x558864a00c10 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 10 client.8508 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x5588647995c0 con 0x558864a00c10 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 1 -- 172.21.15.76:0/1488477592 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400af040 con 0x558864a00c10 2024-09-19T04:20:15.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 1 -- 172.21.15.76:0/1488477592 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7ff1400c6040 con 0x558864a00c10 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 10 client.8508 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 10 client.8508 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 20 client.8508 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x558864a14a80 0x558864a7d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 4 client.8508 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub pause"}) -- 0x5588648133f0 con 0x558864a14a80 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1577fe640 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x558864a14a80 0x558864a7d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1577fe640 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x558864a14a80 0x558864a7d240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87c2227b15c53f5a server_cookie=ffcb531dfc523930 in_seq=0 out_seq=0 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 10 client.8508 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 1 -- 172.21.15.76:0/1488477592 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588648133f0 con 0x558864a14a80 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 10 client.8508 handle_command_reply: tid=0 2024-09-19T04:20:15.368 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 client.8508 shutdown 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 msgr2=0x558864a14000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a14000 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff1400be3e0 tx=0x7ff1400ac000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 shutdown_connections 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x558864a14a80 0x558864a7d240 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x558864a01530 0x558864a14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff1367fc640 0 client.8508 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x558864a00c10 0x558864a14000 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 --2- 172.21.15.76:0/1488477592 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5588649b4780 0x5588649e0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 >> 172.21.15.76:0/1488477592 conn(0x5588649e19b0 msgr2=0x5588649b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 shutdown_connections 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.360+0000 7ff160e86000 1 -- 172.21.15.76:0/1488477592 wait complete. 2024-09-19T04:20:15.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff160e86000 20 client.8508 trim_cache size 0 max 16384 2024-09-19T04:20:15.370 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:15.370 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:20:15.370 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:20:15.370 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1380678c0 msgr2=0x7ff138069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1380678c0 0x7ff138069d80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff150003fd0 tx=0x7ff1500039a0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815bec0 msgr2=0x7ff15809aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815bec0 0x7ff15809aa80 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7ff14001a0e0 tx=0x7ff140019250 comp rx=0 tx=0).stop 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 shutdown_connections 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff1380678c0 0x7ff138069d80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff15815c2a0 0x7ff15809c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff15815bec0 0x7ff15809aa80 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 --2- 172.21.15.76:0/2130934029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1580af2a0 0x7ff1580d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 >> 172.21.15.76:0/2130934029 conn(0x7ff1580b1f90 msgr2=0x7ff1580cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 shutdown_connections 2024-09-19T04:20:15.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.364+0000 7ff15ddca640 1 -- 172.21.15.76:0/2130934029 wait complete. 2024-09-19T04:20:15.379 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:15.379 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-19T04:20:15.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb7685a7640 1 Processor -- start 2024-09-19T04:20:15.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb7685a7640 1 -- start start 2024-09-19T04:20:15.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb7685a7640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7601520a0 0x7fb76006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb7685a7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76006fe50 0x7fb76015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb7685a7640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb760070250 con 0x7fb76015f1d0 2024-09-19T04:20:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb7685a7640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb760152480 con 0x7fb7601520a0 2024-09-19T04:20:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb7685a7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb7600aede0 con 0x7fb76006fe50 2024-09-19T04:20:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb765b42640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76006fe50 0x7fb76015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb765b42640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76006fe50 0x7fb76015ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:47380/0 (socket says 172.21.15.76:47380) 2024-09-19T04:20:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb766343640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb7601520a0 0x7fb76006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb765b42640 1 -- 172.21.15.76:0/2653017004 learned_addr learned my addr 172.21.15.76:0/2653017004 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb766343640 1 -- 172.21.15.76:0/2653017004 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76006fe50 msgr2=0x7fb76015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb766343640 1 --2- 172.21.15.76:0/2653017004 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76006fe50 0x7fb76015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb766343640 1 -- 172.21.15.76:0/2653017004 >> v1:172.21.15.116:6789/0 conn(0x7fb76015f1d0 legacy=0x7fb760071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.495 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.492+0000 7fb766343640 1 -- 172.21.15.76:0/2653017004 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76015fa10 con 0x7fb7601520a0 2024-09-19T04:20:15.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb766343640 1 --2- 172.21.15.76:0/2653017004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7601520a0 0x7fb76006f910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0097c0 tx=0x7fb75c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f290d905bcf098af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb765341640 1 -- 172.21.15.76:0/2653017004 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0b1070 con 0x7fb7601520a0 2024-09-19T04:20:15.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb765341640 1 -- 172.21.15.76:0/2653017004 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0ac600 con 0x7fb7601520a0 2024-09-19T04:20:15.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb765341640 1 -- 172.21.15.76:0/2653017004 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0ac9d0 con 0x7fb7601520a0 2024-09-19T04:20:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 -- 172.21.15.76:0/2653017004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7601520a0 msgr2=0x7fb76006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 --2- 172.21.15.76:0/2653017004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7601520a0 0x7fb76006f910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0097c0 tx=0x7fb75c01c480 comp rx=0 tx=0).stop 2024-09-19T04:20:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 -- 172.21.15.76:0/2653017004 shutdown_connections 2024-09-19T04:20:15.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 --2- 172.21.15.76:0/2653017004 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb76006fe50 0x7fb76015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 --2- 172.21.15.76:0/2653017004 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7601520a0 0x7fb76006f910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 -- 172.21.15.76:0/2653017004 >> 172.21.15.76:0/2653017004 conn(0x7fb7600b1f90 msgr2=0x7fb7600b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 -- 172.21.15.76:0/2653017004 shutdown_connections 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 -- 172.21.15.76:0/2653017004 wait complete. 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.496+0000 7fb7685a7640 1 Processor -- start 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 -- start start 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76006fe50 0x7fb760099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7601520a0 0x7fb7601640d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76015f1d0 0x7fb7600b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb760073d30 con 0x7fb76015f1d0 2024-09-19T04:20:15.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb760073a50 con 0x7fb76006fe50 2024-09-19T04:20:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb76015f600 con 0x7fb7601520a0 2024-09-19T04:20:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76015f1d0 0x7fb7600b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76015f1d0 0x7fb7600b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51684/0 (socket says 172.21.15.76:51684) 2024-09-19T04:20:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 -- 172.21.15.76:0/1208367199 learned_addr learned my addr 172.21.15.76:0/1208367199 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766343640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76006fe50 0x7fb760099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb765b42640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7601520a0 0x7fb7601640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 -- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7601520a0 msgr2=0x7fb7601640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7601520a0 0x7fb7601640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 -- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76006fe50 msgr2=0x7fb760099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76006fe50 0x7fb760099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 -- 172.21.15.76:0/1208367199 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76009c320 con 0x7fb76015f1d0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb765b42640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7601520a0 0x7fb7601640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766b44640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76015f1d0 0x7fb7600b37e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb75800c740 tx=0x7fb75800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb752ffd640 1 -- 172.21.15.76:0/1208367199 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758016020 con 0x7fb76015f1d0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb752ffd640 1 -- 172.21.15.76:0/1208367199 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75800ce50 con 0x7fb76015f1d0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb760164a00 con 0x7fb76015f1d0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb752ffd640 1 -- 172.21.15.76:0/1208367199 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758003980 con 0x7fb76015f1d0 2024-09-19T04:20:15.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76009c590 con 0x7fb76015f1d0 2024-09-19T04:20:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb752ffd640 1 -- 172.21.15.76:0/1208367199 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb758003b20 con 0x7fb76015f1d0 2024-09-19T04:20:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb752ffd640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7440678c0 0x7fb744069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb752ffd640 1 -- 172.21.15.76:0/1208367199 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fb7580859a0 con 0x7fb76015f1d0 2024-09-19T04:20:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.500+0000 7fb766343640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7440678c0 0x7fb744069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb766343640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7440678c0 0x7fb744069d80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0ae000 tx=0x7fb75c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 Processor -- start 2024-09-19T04:20:15.511 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- start start 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31bd70 0x56165d31c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d31c690 0x56165d2cf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d2cfdb0 0x56165d2d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56165d1b9960 con 0x56165d31c690 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56165d0fd900 con 0x56165d2cfdb0 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56165d120d40 con 0x56165d31bd70 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d31c690 0x56165d2cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d31c690 0x56165d2cf870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51694/0 (socket says 172.21.15.76:51694) 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 -- 172.21.15.76:0/1987949769 learned_addr learned my addr 172.21.15.76:0/1987949769 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb766343640 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31bd70 0x56165d31c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb766b44640 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d2cfdb0 0x56165d2d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 -- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31bd70 msgr2=0x56165d31c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.512 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31bd70 0x56165d31c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 -- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d2cfdb0 msgr2=0x56165d2d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d2cfdb0 0x56165d2d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 -- 172.21.15.76:0/1987949769 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165d0c7580 con 0x56165d31c690 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb766b44640 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d2cfdb0 0x56165d2d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb765b42640 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d31c690 0x56165d2cf870 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb754009b50 tx=0x7fb754009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=939c31c3989fcff7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb743fff640 1 -- 172.21.15.76:0/1987949769 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754019070 con 0x56165d31c690 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb743fff640 1 -- 172.21.15.76:0/1987949769 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb754009c40 con 0x56165d31c690 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb743fff640 1 -- 172.21.15.76:0/1987949769 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754015460 con 0x56165d31c690 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- 172.21.15.76:0/1987949769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d31c690 msgr2=0x56165d2cf870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d31c690 0x56165d2cf870 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fb754009b50 tx=0x7fb754009240 comp rx=0 tx=0).stop 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- 172.21.15.76:0/1987949769 shutdown_connections 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d2cfdb0 0x56165d2d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d31c690 0x56165d2cf870 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 --2- 172.21.15.76:0/1987949769 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31bd70 0x56165d31c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- 172.21.15.76:0/1987949769 >> 172.21.15.76:0/1987949769 conn(0x56165d2fd040 msgr2=0x56165d2ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- 172.21.15.76:0/1987949769 shutdown_connections 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- 172.21.15.76:0/1987949769 wait complete. 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 -- start start 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.504+0000 7fb76b663000 1 Processor -- start 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d322ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d323220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31c690 0x56165d323760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56165d11cd00 con 0x56165d2cfdb0 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56165d0dbaa0 con 0x56165d31bd70 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56165d0ba340 con 0x56165d31c690 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 10 client.0 _ll_register_callbacks cb 0x56165d2bd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d322ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d322ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51696/0 (socket says 172.21.15.76:51696) 2024-09-19T04:20:15.514 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 -- 172.21.15.76:0/3008900184 learned_addr learned my addr 172.21.15.76:0/3008900184 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb765b42640 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d323220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766b44640 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31c690 0x56165d323760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 -- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31c690 msgr2=0x56165d323760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31c690 0x56165d323760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 -- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 msgr2=0x56165d323220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d323220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 -- 172.21.15.76:0/3008900184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165d0c3dc0 con 0x56165d2cfdb0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb766343640 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d322ce0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb75c01a0b0 tx=0x7fb75c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7294dd114f9078 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 1 -- 172.21.15.76:0/3008900184 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0c9060 con 0x56165d2cfdb0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 1 -- 172.21.15.76:0/3008900184 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fb75c0b95f0 con 0x56165d2cfdb0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb754009c40 con 0x56165d2cfdb0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 1 -- 172.21.15.76:0/3008900184 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fb75c0cf050 con 0x56165d2cfdb0 2024-09-19T04:20:15.515 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 10 client.8517 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x56165d0c07f0 con 0x56165d2cfdb0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 1 -- 172.21.15.76:0/3008900184 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb754009c40 con 0x56165d2cfdb0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 1 -- 172.21.15.76:0/3008900184 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7fb75c0d2020 con 0x56165d2cfdb0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 10 client.8517 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 10 client.8517 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 20 client.8517 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d32b0b0 0x56165d32d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 4 client.8517 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56165d0bae10 con 0x56165d32b0b0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb765b42640 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d32b0b0 0x56165d32d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 10 client.8517 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb765b42640 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d32b0b0 0x56165d32d4a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4ea1d7e80c457851 server_cookie=a84060871403b5cc in_seq=0 out_seq=0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 1 -- 172.21.15.76:0/3008900184 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56165d0bae10 con 0x56165d32b0b0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 10 client.8517 handle_command_reply: tid=0 2024-09-19T04:20:15.516 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 client.8517 shutdown 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 msgr2=0x56165d322ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d322ce0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb75c01a0b0 tx=0x7fb75c01acc0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 shutdown_connections 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d32b0b0 0x56165d32d4a0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d31c690 0x56165d323760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb7417fa640 0 client.8517 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d323220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 --2- 172.21.15.76:0/3008900184 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d322ce0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 >> 172.21.15.76:0/3008900184 conn(0x56165d2fd040 msgr2=0x56165d2cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 shutdown_connections 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 1 -- 172.21.15.76:0/3008900184 wait complete. 2024-09-19T04:20:15.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.508+0000 7fb76b663000 20 client.8517 trim_cache size 0 max 16384 2024-09-19T04:20:15.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 Processor -- start 2024-09-19T04:20:15.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- start start 2024-09-19T04:20:15.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2fd040 0x56165d2d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2cfdb0 0x56165d2d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d325d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb760203fb0 con 0x56165d2fd040 2024-09-19T04:20:15.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb760203180 con 0x56165d31bd70 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb760203880 con 0x56165d2cfdb0 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2fd040 0x56165d2d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2fd040 0x56165d2d7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51702/0 (socket says 172.21.15.76:51702) 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 -- 172.21.15.76:0/1015339111 learned_addr learned my addr 172.21.15.76:0/1015339111 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 -- 172.21.15.76:0/1015339111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2cfdb0 msgr2=0x56165d2d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 --2- 172.21.15.76:0/1015339111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2cfdb0 0x56165d2d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 -- 172.21.15.76:0/1015339111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 msgr2=0x56165d325d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 --2- 172.21.15.76:0/1015339111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d325d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 -- 172.21.15.76:0/1015339111 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56165d0de470 con 0x56165d2fd040 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb766343640 1 --2- 172.21.15.76:0/1015339111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2fd040 0x56165d2d7b90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000c10 tx=0x7fb75c0bb050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b00893cd463eac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb7437fe640 1 -- 172.21.15.76:0/1015339111 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0b1040 con 0x56165d2fd040 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb7437fe640 1 -- 172.21.15.76:0/1015339111 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0b95f0 con 0x56165d2fd040 2024-09-19T04:20:15.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb7437fe640 1 -- 172.21.15.76:0/1015339111 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0b6430 con 0x56165d2fd040 2024-09-19T04:20:15.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- 172.21.15.76:0/1015339111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2fd040 msgr2=0x56165d2d7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 --2- 172.21.15.76:0/1015339111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2fd040 0x56165d2d7b90 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb75c000c10 tx=0x7fb75c0bb050 comp rx=0 tx=0).stop 2024-09-19T04:20:15.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- 172.21.15.76:0/1015339111 shutdown_connections 2024-09-19T04:20:15.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 --2- 172.21.15.76:0/1015339111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d325d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 --2- 172.21.15.76:0/1015339111 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2cfdb0 0x56165d2d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 --2- 172.21.15.76:0/1015339111 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2fd040 0x56165d2d7b90 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- 172.21.15.76:0/1015339111 >> 172.21.15.76:0/1015339111 conn(0x56165d2d4890 msgr2=0x56165d322aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- 172.21.15.76:0/1015339111 shutdown_connections 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- 172.21.15.76:0/1015339111 wait complete. 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 -- start start 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.528+0000 7fb76b663000 1 Processor -- start 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d328950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2fd040 0x56165d34b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d34bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56165d0c6f20 con 0x56165d2cfdb0 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb76009d3c0 con 0x56165d31bd70 2024-09-19T04:20:15.536 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56165d125370 con 0x56165d2fd040 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 10 client.0 _ll_register_callbacks cb 0x7fb7540090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d328950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d328950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51716/0 (socket says 172.21.15.76:51716) 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 -- 172.21.15.76:0/1273807765 learned_addr learned my addr 172.21.15.76:0/1273807765 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 -- 172.21.15.76:0/1273807765 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2fd040 msgr2=0x56165d34b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2fd040 0x56165d34b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 -- 172.21.15.76:0/1273807765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 msgr2=0x56165d34bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d34bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 -- 172.21.15.76:0/1273807765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56165d0b4640 con 0x56165d2cfdb0 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb766343640 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d328950 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0096c0 tx=0x7fb75c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a91be23026d9e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 1 -- 172.21.15.76:0/1273807765 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb75c0d7070 con 0x56165d2cfdb0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 1 -- 172.21.15.76:0/1273807765 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fb75c0ac400 con 0x56165d2cfdb0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb75c0b95f0 con 0x56165d2cfdb0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 1 -- 172.21.15.76:0/1273807765 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fb75c0b1070 con 0x56165d2cfdb0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 10 client.8523 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x56165d0b83b0 con 0x56165d2cfdb0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 1 -- 172.21.15.76:0/1273807765 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb75c0b95f0 con 0x56165d2cfdb0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 1 -- 172.21.15.76:0/1273807765 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7fb75c0d2040 con 0x56165d2cfdb0 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 10 client.8523 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 10 client.8523 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 20 client.8523 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d31c690 0x56165d34c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 4 client.8523 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:15.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56165d12e470 con 0x56165d31c690 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb765b42640 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d31c690 0x56165d34c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 10 client.8523 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb765b42640 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d31c690 0x56165d34c210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77f174d7ba9811b7 server_cookie=f5f2f0b4e96a01bd in_seq=0 out_seq=0 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 1 -- 172.21.15.76:0/1273807765 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x56165d12e470 con 0x56165d31c690 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 10 client.8523 handle_command_reply: tid=0 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 client.8523 shutdown 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 msgr2=0x56165d328950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d328950 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0096c0 tx=0x7fb75c008040 comp rx=0 tx=0).stop 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 shutdown_connections 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56165d31c690 0x56165d34c210 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x56165d31bd70 0x56165d34bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56165d2fd040 0x56165d34b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 --2- 172.21.15.76:0/1273807765 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x56165d2cfdb0 0x56165d328950 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7417fa640 0 client.8523 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 >> 172.21.15.76:0/1273807765 conn(0x56165d2d4890 msgr2=0x56165d2d73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 shutdown_connections 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 1 -- 172.21.15.76:0/1273807765 wait complete. 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb76b663000 20 client.8523 trim_cache size 0 max 16384 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:15.540 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:15.541 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:15.541 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:15.541 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:15.541 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:15.541 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:15.541 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:15.541 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:15.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7440678c0 msgr2=0x7fb744069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7440678c0 0x7fb744069d80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb75c0ae000 tx=0x7fb75c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76015f1d0 msgr2=0x7fb7600b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76015f1d0 0x7fb7600b37e0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fb75800c740 tx=0x7fb75800cc00 comp rx=0 tx=0).stop 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 shutdown_connections 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb7440678c0 0x7fb744069d80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb76015f1d0 0x7fb7600b37e0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb7601520a0 0x7fb7601640d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 --2- 172.21.15.76:0/1208367199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb76006fe50 0x7fb760099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 >> 172.21.15.76:0/1208367199 conn(0x7fb7600b1f90 msgr2=0x7fb76006e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 shutdown_connections 2024-09-19T04:20:15.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.532+0000 7fb7685a7640 1 -- 172.21.15.76:0/1208367199 wait complete. 2024-09-19T04:20:15.550 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:15.551 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:15.551 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-19T04:20:15.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.668+0000 7efe3041f640 1 Processor -- start 2024-09-19T04:20:15.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.668+0000 7efe3041f640 1 -- start start 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe3041f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe280cb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe3041f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe280b3e00 0x7efe280b4290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe3041f640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe28159070 con 0x7efe28158c90 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe3041f640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7efe2806d070 con 0x7efe280b3e00 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe3041f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efe2806d2c0 con 0x7efe2815f840 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe280b3e00 0x7efe280b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7efe280b3e00 0x7efe280b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51732/0 (socket says 172.21.15.76:51732) 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 -- 172.21.15.76:0/681370663 learned_addr learned my addr 172.21.15.76:0/681370663 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/681370663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe280cb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 -- 172.21.15.76:0/681370663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 msgr2=0x7efe280cb9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 --2- 172.21.15.76:0/681370663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe280cb9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 -- 172.21.15.76:0/681370663 >> v1:172.21.15.116:6789/0 conn(0x7efe28158c90 legacy=0x7efe2806e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 -- 172.21.15.76:0/681370663 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe2806d5a0 con 0x7efe280b3e00 2024-09-19T04:20:15.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/681370663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe280cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2e9bc640 1 --2- 172.21.15.76:0/681370663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe280b4290 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7efe20009e50 tx=0x7efe2001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48f123957aeb1b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2d1b9640 1 -- 172.21.15.76:0/681370663 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe200a5670 con 0x7efe280b3e00 2024-09-19T04:20:15.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2d1b9640 1 -- 172.21.15.76:0/681370663 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe200a5810 con 0x7efe280b3e00 2024-09-19T04:20:15.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe2d1b9640 1 -- 172.21.15.76:0/681370663 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe200a5be0 con 0x7efe280b3e00 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe3041f640 1 -- 172.21.15.76:0/681370663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 msgr2=0x7efe280b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.672+0000 7efe3041f640 1 --2- 172.21.15.76:0/681370663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe280b4290 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7efe20009e50 tx=0x7efe2001c9f0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- 172.21.15.76:0/681370663 shutdown_connections 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 --2- 172.21.15.76:0/681370663 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe280b4290 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 --2- 172.21.15.76:0/681370663 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe280cb9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- 172.21.15.76:0/681370663 >> 172.21.15.76:0/681370663 conn(0x7efe2806bcc0 msgr2=0x7efe2806c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- 172.21.15.76:0/681370663 shutdown_connections 2024-09-19T04:20:15.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- 172.21.15.76:0/681370663 wait complete. 2024-09-19T04:20:15.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 Processor -- start 2024-09-19T04:20:15.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- start start 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe281568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe28158c90 0x7efe281522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe281556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efe280cd180 con 0x7efe280b3e00 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe280cd000 con 0x7efe28158c90 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efe280cd300 con 0x7efe2815f840 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe281568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.679 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe281568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51734/0 (socket says 172.21.15.76:51734) 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 -- 172.21.15.76:0/529474410 learned_addr learned my addr 172.21.15.76:0/529474410 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe28158c90 0x7efe281522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e9bc640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe281556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 -- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 msgr2=0x7efe281556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe281556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 -- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe28158c90 msgr2=0x7efe281522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe28158c90 0x7efe281522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 -- 172.21.15.76:0/529474410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe281527f0 con 0x7efe280b3e00 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe281568a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7efe2400c480 tx=0x7efe2400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe1effd640 1 -- 172.21.15.76:0/529474410 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24012830 con 0x7efe280b3e00 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe1effd640 1 -- 172.21.15.76:0/529474410 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe24017070 con 0x7efe280b3e00 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe28156ec0 con 0x7efe280b3e00 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe1effd640 1 -- 172.21.15.76:0/529474410 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24012c30 con 0x7efe280b3e00 2024-09-19T04:20:15.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.676+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe28157480 con 0x7efe280b3e00 2024-09-19T04:20:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe1effd640 1 -- 172.21.15.76:0/529474410 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efe2401c460 con 0x7efe280b3e00 2024-09-19T04:20:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe1effd640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efe0c0678c0 0x7efe0c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe1effd640 1 -- 172.21.15.76:0/529474410 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7efe24085c20 con 0x7efe280b3e00 2024-09-19T04:20:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efe0c0678c0 0x7efe0c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efe0c0678c0 0x7efe0c069d80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe28156830 tx=0x7efe18008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 Processor -- start 2024-09-19T04:20:15.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 -- start start 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 0x561958b31150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958b31690 0x561958ae4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958ae4db0 0x561958ae5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5619589ce960 con 0x561958b31690 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561958912900 con 0x561958ae4db0 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561958935d40 con 0x561958b30d70 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958b31690 0x561958ae4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958b31690 0x561958ae4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51738/0 (socket says 172.21.15.76:51738) 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 -- 172.21.15.76:0/3013979797 learned_addr learned my addr 172.21.15.76:0/3013979797 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 0x561958b31150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2e9bc640 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958ae4db0 0x561958ae5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 -- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 msgr2=0x561958b31150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 0x561958b31150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 -- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958ae4db0 msgr2=0x561958ae5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958ae4db0 0x561958ae5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2d9ba640 1 -- 172.21.15.76:0/3013979797 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619588dc580 con 0x561958b31690 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.680+0000 7efe2e9bc640 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958ae4db0 0x561958ae5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958b31690 0x561958ae4870 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7efe1800fa30 tx=0x7efe18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7691321492d7c1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe0bfff640 1 -- 172.21.15.76:0/3013979797 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe1801b050 con 0x561958b31690 2024-09-19T04:20:15.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe0bfff640 1 -- 172.21.15.76:0/3013979797 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe18025070 con 0x561958b31690 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe0bfff640 1 -- 172.21.15.76:0/3013979797 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18014e00 con 0x561958b31690 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/3013979797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958b31690 msgr2=0x561958ae4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958b31690 0x561958ae4870 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7efe1800fa30 tx=0x7efe18012000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/3013979797 shutdown_connections 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958ae4db0 0x561958ae5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958b31690 0x561958ae4870 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- 172.21.15.76:0/3013979797 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 0x561958b31150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/3013979797 >> 172.21.15.76:0/3013979797 conn(0x561958b12040 msgr2=0x561958b14430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/3013979797 shutdown_connections 2024-09-19T04:20:15.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/3013979797 wait complete. 2024-09-19T04:20:15.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- start start 2024-09-19T04:20:15.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 Processor -- start 2024-09-19T04:20:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b37ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958b38220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 0x561958b38760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561958931d00 con 0x561958ae4db0 2024-09-19T04:20:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5619588f0aa0 con 0x561958b30d70 2024-09-19T04:20:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5619588cf340 con 0x561958b31690 2024-09-19T04:20:15.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 10 client.0 _ll_register_callbacks cb 0x561958ad2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b37ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b37ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51748/0 (socket says 172.21.15.76:51748) 2024-09-19T04:20:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2866224694 learned_addr learned my addr 172.21.15.76:0/2866224694 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958b38220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 msgr2=0x561958b38760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 0x561958b38760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 msgr2=0x561958b38220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958b38220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2866224694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619588d8dc0 con 0x561958ae4db0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958b38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b37ce0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7efe2400ce40 tx=0x7efe24055ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ea7660305aa610 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe097fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe097fa640 1 -- 172.21.15.76:0/2866224694 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24081020 con 0x561958ae4db0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe097fa640 1 -- 172.21.15.76:0/2866224694 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7efe2405e680 con 0x561958ae4db0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe18025070 con 0x561958ae4db0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe097fa640 1 -- 172.21.15.76:0/2866224694 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7efe2405e880 con 0x561958ae4db0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 10 client.8535 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x5619588d57f0 con 0x561958ae4db0 2024-09-19T04:20:15.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe097fa640 1 -- 172.21.15.76:0/2866224694 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe18025070 con 0x561958ae4db0 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe097fa640 1 -- 172.21.15.76:0/2866224694 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7efe24055380 con 0x561958ae4db0 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 10 client.8535 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 10 client.8535 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 20 client.8535 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958b400b0 0x561958b424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 4 client.8535 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.684+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619588cfe10 con 0x561958b400b0 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958b400b0 0x561958b424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958b400b0 0x561958b424a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1cc244beb955fb5d server_cookie=a79d3920e7b11fa9 in_seq=0 out_seq=0 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe097fa640 10 client.8535 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe097fa640 1 -- 172.21.15.76:0/2866224694 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619588cfe10 con 0x561958b400b0 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe097fa640 10 client.8535 handle_command_reply: tid=0 2024-09-19T04:20:15.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 client.8535 shutdown 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 msgr2=0x561958b37ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b37ce0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7efe2400ce40 tx=0x7efe24055ff0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 shutdown_connections 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958b400b0 0x561958b424a0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 0x561958b38760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958b38220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe097fa640 0 client.8535 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 --2- 172.21.15.76:0/2866224694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b37ce0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 >> 172.21.15.76:0/2866224694 conn(0x561958b12040 msgr2=0x561958ae3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 shutdown_connections 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 1 -- 172.21.15.76:0/2866224694 wait complete. 2024-09-19T04:20:15.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.688+0000 7efe334db000 20 client.8535 trim_cache size 0 max 16384 2024-09-19T04:20:15.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 Processor -- start 2024-09-19T04:20:15.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- start start 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958aecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 0x561958aed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b31690 0x561958b3ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efe28203e80 con 0x561958ae4db0 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe28203050 con 0x561958b31690 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efe28203750 con 0x561958b30d70 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958aecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958aecc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51760/0 (socket says 172.21.15.76:51760) 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2799342362 learned_addr learned my addr 172.21.15.76:0/2799342362 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2799342362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 msgr2=0x561958aed140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2799342362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 0x561958aed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2799342362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b31690 msgr2=0x561958b3ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2799342362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b31690 0x561958b3ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2799342362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561958aea960 con 0x561958ae4db0 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2799342362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958aecc00 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7efe2405e5e0 tx=0x7efe24058000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d596f8ecff08a716 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe0b7fe640 1 -- 172.21.15.76:0/2799342362 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24081040 con 0x561958ae4db0 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe0b7fe640 1 -- 172.21.15.76:0/2799342362 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe240525a0 con 0x561958ae4db0 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe0b7fe640 1 -- 172.21.15.76:0/2799342362 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2407d3f0 con 0x561958ae4db0 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- 172.21.15.76:0/2799342362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 msgr2=0x561958aecc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.711 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 --2- 172.21.15.76:0/2799342362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958aecc00 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7efe2405e5e0 tx=0x7efe24058000 comp rx=0 tx=0).stop 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- 172.21.15.76:0/2799342362 shutdown_connections 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 --2- 172.21.15.76:0/2799342362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b31690 0x561958b3ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 --2- 172.21.15.76:0/2799342362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b30d70 0x561958aed140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 --2- 172.21.15.76:0/2799342362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958aecc00 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- 172.21.15.76:0/2799342362 >> 172.21.15.76:0/2799342362 conn(0x561958b12040 msgr2=0x561958b37aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- 172.21.15.76:0/2799342362 shutdown_connections 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- 172.21.15.76:0/2799342362 wait complete. 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 -- start start 2024-09-19T04:20:15.712 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.708+0000 7efe334db000 1 Processor -- start 2024-09-19T04:20:15.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b3d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958ba0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 0x561958ba1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561958931d00 con 0x561958ae4db0 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efe2814fa10 con 0x561958b30d70 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561958aeba30 con 0x561958b31690 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 10 client.0 _ll_register_callbacks cb 0x7efe24055d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b3d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b3d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51764/0 (socket says 172.21.15.76:51764) 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2852658785 learned_addr learned my addr 172.21.15.76:0/2852658785 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 msgr2=0x561958ba1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 0x561958ba1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 msgr2=0x561958ba0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958ba0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 -- 172.21.15.76:0/2852658785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619588f3470 con 0x561958ae4db0 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2e1bb640 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b3d700 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7efe24084070 tx=0x7efe24083ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6107aa66f9258e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 1 -- 172.21.15.76:0/2852658785 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe24063070 con 0x561958ae4db0 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 1 -- 172.21.15.76:0/2852658785 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7efe2407d600 con 0x561958ae4db0 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe240525a0 con 0x561958ae4db0 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 1 -- 172.21.15.76:0/2852658785 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7efe24047050 con 0x561958ae4db0 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 10 client.8541 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x5619588c9640 con 0x561958ae4db0 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 1 -- 172.21.15.76:0/2852658785 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe240525a0 con 0x561958ae4db0 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 1 -- 172.21.15.76:0/2852658785 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7efe2407c030 con 0x561958ae4db0 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 10 client.8541 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 10 client.8541 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 20 client.8541 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958ba1680 0x561958ba1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 4 client.8541 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561958943470 con 0x561958ba1680 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958ba1680 0x561958ba1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 10 client.8541 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe2d9ba640 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958ba1680 0x561958ba1a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=317c6875f76b4a2b server_cookie=6ec3bc55adf5eb13 in_seq=0 out_seq=0 2024-09-19T04:20:15.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 1 -- 172.21.15.76:0/2852658785 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x561958943470 con 0x561958ba1680 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 10 client.8541 handle_command_reply: tid=0 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 client.8541 shutdown 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 msgr2=0x561958b3d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b3d700 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7efe24084070 tx=0x7efe24083ef0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 shutdown_connections 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x561958ba1680 0x561958ba1a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561958b31690 0x561958ba1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561958b30d70 0x561958ba0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 --2- 172.21.15.76:0/2852658785 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561958ae4db0 0x561958b3d700 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe097fa640 0 client.8541 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 >> 172.21.15.76:0/2852658785 conn(0x561958b12040 msgr2=0x561958aebbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 shutdown_connections 2024-09-19T04:20:15.719 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 1 -- 172.21.15.76:0/2852658785 wait complete. 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe334db000 20 client.8541 trim_cache size 0 max 16384 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:15.720 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:15.721 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:15.721 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:15.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efe0c0678c0 msgr2=0x7efe0c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efe0c0678c0 0x7efe0c069d80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7efe28156830 tx=0x7efe18008040 comp rx=0 tx=0).stop 2024-09-19T04:20:15.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 msgr2=0x7efe281568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe281568a0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7efe2400c480 tx=0x7efe2400c940 comp rx=0 tx=0).stop 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 shutdown_connections 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efe0c0678c0 0x7efe0c069d80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efe2815f840 0x7efe281556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe28158c90 0x7efe281522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 --2- 172.21.15.76:0/529474410 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efe280b3e00 0x7efe281568a0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 >> 172.21.15.76:0/529474410 conn(0x7efe2806bcc0 msgr2=0x7efe2806c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 shutdown_connections 2024-09-19T04:20:15.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.712+0000 7efe3041f640 1 -- 172.21.15.76:0/529474410 wait complete. 2024-09-19T04:20:15.731 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:15.731 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:15.732 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-19T04:20:15.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.844+0000 7f350af58640 1 Processor -- start 2024-09-19T04:20:15.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.844+0000 7f350af58640 1 -- start start 2024-09-19T04:20:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f350af58640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3504070020 0x7f350415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f350af58640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f1d0 0x7f35040adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f350af58640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35040aede0 con 0x7f35041520a0 2024-09-19T04:20:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f350af58640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3504152480 con 0x7f3504070020 2024-09-19T04:20:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f350af58640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f350415fa10 con 0x7f350415f1d0 2024-09-19T04:20:15.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3504070020 0x7f350415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3504070020 0x7f350415ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51774/0 (socket says 172.21.15.76:51774) 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 -- 172.21.15.76:0/2623582802 learned_addr learned my addr 172.21.15.76:0/2623582802 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f35094f5640 1 --2- 172.21.15.76:0/2623582802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f1d0 0x7f35040adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 -- 172.21.15.76:0/2623582802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f1d0 msgr2=0x7f35040adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 --2- 172.21.15.76:0/2623582802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f1d0 0x7f35040adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 -- 172.21.15.76:0/2623582802 >> v1:172.21.15.116:6789/0 conn(0x7f35041520a0 legacy=0x7f350406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 -- 172.21.15.76:0/2623582802 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3504074250 con 0x7f3504070020 2024-09-19T04:20:15.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f35094f5640 1 --2- 172.21.15.76:0/2623582802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f1d0 0x7f35040adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f3503fff640 1 --2- 172.21.15.76:0/2623582802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3504070020 0x7f350415ec90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009870 tx=0x7f34f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa98527e62457b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f35037fe640 1 -- 172.21.15.76:0/2623582802 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80aa610 con 0x7f3504070020 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f35037fe640 1 -- 172.21.15.76:0/2623582802 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f80af070 con 0x7f3504070020 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.848+0000 7f35037fe640 1 -- 172.21.15.76:0/2623582802 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80aa9d0 con 0x7f3504070020 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- 172.21.15.76:0/2623582802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3504070020 msgr2=0x7f350415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 --2- 172.21.15.76:0/2623582802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3504070020 0x7f350415ec90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f34f8009870 tx=0x7f34f801c4f0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- 172.21.15.76:0/2623582802 shutdown_connections 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 --2- 172.21.15.76:0/2623582802 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f350415f1d0 0x7f35040adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 --2- 172.21.15.76:0/2623582802 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3504070020 0x7f350415ec90 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- 172.21.15.76:0/2623582802 >> 172.21.15.76:0/2623582802 conn(0x7f35040b1f90 msgr2=0x7f35040b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- 172.21.15.76:0/2623582802 shutdown_connections 2024-09-19T04:20:15.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- 172.21.15.76:0/2623582802 wait complete. 2024-09-19T04:20:15.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 Processor -- start 2024-09-19T04:20:15.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- start start 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3504070020 0x7f3504099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35041520a0 0x7f350414a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350415f1d0 0x7f35041502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f350415f5e0 con 0x7f350415f1d0 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3504073d30 con 0x7f3504070020 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f350415f760 con 0x7f35041520a0 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350415f1d0 0x7f35041502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350415f1d0 0x7f35041502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51778/0 (socket says 172.21.15.76:51778) 2024-09-19T04:20:15.855 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 -- 172.21.15.76:0/1835035186 learned_addr learned my addr 172.21.15.76:0/1835035186 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f3508cf4640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3504070020 0x7f3504099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f3503fff640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35041520a0 0x7f350414a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 -- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35041520a0 msgr2=0x7f350414a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35041520a0 0x7f350414a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 -- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3504070020 msgr2=0x7f3504099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3504070020 0x7f3504099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 -- 172.21.15.76:0/1835035186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350414a9b0 con 0x7f350415f1d0 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35094f5640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350415f1d0 0x7f35041502d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00cef0 tx=0x7f34fc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35017fa640 1 -- 172.21.15.76:0/1835035186 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc01a070 con 0x7f350415f1d0 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35017fa640 1 -- 172.21.15.76:0/1835035186 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc00d860 con 0x7f350415f1d0 2024-09-19T04:20:15.856 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350414f510 con 0x7f350415f1d0 2024-09-19T04:20:15.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f35017fa640 1 -- 172.21.15.76:0/1835035186 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc00dc50 con 0x7f350415f1d0 2024-09-19T04:20:15.857 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.852+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350414fad0 con 0x7f350415f1d0 2024-09-19T04:20:15.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35017fa640 1 -- 172.21.15.76:0/1835035186 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f34fc01d070 con 0x7f350415f1d0 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35017fa640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34f00678c0 0x7f34f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35017fa640 1 -- 172.21.15.76:0/1835035186 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f34fc086660 con 0x7f350415f1d0 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f3508cf4640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34f00678c0 0x7f34f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f3508cf4640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34f00678c0 0x7f34f0069d80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f34f40098e0 tx=0x7f34f4009060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 Processor -- start 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 -- start start 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955814b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x5595580fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595580ff190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559557fe8960 con 0x5595580fedb0 2024-09-19T04:20:15.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559557f2c900 con 0x55955814b690 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559557f4fd40 con 0x55955814ad70 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595580ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595580ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51780/0 (socket says 172.21.15.76:51780) 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 -- 172.21.15.76:0/3377696760 learned_addr learned my addr 172.21.15.76:0/3377696760 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f3503fff640 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x5595580fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f3508cf4640 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 -- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 msgr2=0x55955814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955814b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 -- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 msgr2=0x5595580fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x5595580fe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 -- 172.21.15.76:0/3377696760 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559557ef6580 con 0x5595580fedb0 2024-09-19T04:20:15.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f3503fff640 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x5595580fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.856+0000 7f35094f5640 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595580ff190 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00ae50 tx=0x7f34fc0568a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d869fa06c9bb305 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34ea7fc640 1 -- 172.21.15.76:0/3377696760 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc054af0 con 0x5595580fedb0 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34ea7fc640 1 -- 172.21.15.76:0/3377696760 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc054d50 con 0x5595580fedb0 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34ea7fc640 1 -- 172.21.15.76:0/3377696760 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc05d0f0 con 0x5595580fedb0 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/3377696760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 msgr2=0x5595580ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595580ff190 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00ae50 tx=0x7f34fc0568a0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/3377696760 shutdown_connections 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595580ff190 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x5595580fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- 172.21.15.76:0/3377696760 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/3377696760 >> 172.21.15.76:0/3377696760 conn(0x55955812c040 msgr2=0x55955812e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/3377696760 shutdown_connections 2024-09-19T04:20:15.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/3377696760 wait complete. 2024-09-19T04:20:15.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- start start 2024-09-19T04:20:15.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 Processor -- start 2024-09-19T04:20:15.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x559558106530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558106a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 0x55955812c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559557f4bd00 con 0x55955814ad70 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559557f0aaa0 con 0x5595580fedb0 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559557ee9340 con 0x55955814b690 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 10 client.0 _ll_register_callbacks cb 0x5595580ec760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558106a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558106a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51782/0 (socket says 172.21.15.76:51782) 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 -- 172.21.15.76:0/2851176605 learned_addr learned my addr 172.21.15.76:0/2851176605 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3508cf4640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x559558106530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f35094f5640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 0x55955812c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 -- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 msgr2=0x55955812c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 0x55955812c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 -- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 msgr2=0x559558106530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x559558106530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 -- 172.21.15.76:0/2851176605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559557ef2dc0 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3508cf4640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x559558106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3503fff640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558106a70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f34f80ac040 tx=0x7f34f801ad30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb64bd1eb80176f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34e37fe640 1 -- 172.21.15.76:0/2851176605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80aae30 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34fc054d50 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34e37fe640 1 -- 172.21.15.76:0/2851176605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f34f80bfcc0 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34e37fe640 1 -- 172.21.15.76:0/2851176605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f34f80b33f0 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 10 client.8553 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x559557eef7f0 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34e37fe640 1 -- 172.21.15.76:0/2851176605 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34fc054d50 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f34e37fe640 1 -- 172.21.15.76:0/2851176605 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7f34f80b3590 con 0x55955814ad70 2024-09-19T04:20:15.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 10 client.8553 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 10 client.8553 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 20 client.8553 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955812ce30 0x55955815c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 4 client.8553 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559557ee9e10 con 0x55955812ce30 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.860+0000 7f3508cf4640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955812ce30 0x55955815c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f3508cf4640 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955812ce30 0x55955815c020 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ac37692422fa824 server_cookie=1dc1774019f2da52 in_seq=0 out_seq=0 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f34e37fe640 10 client.8553 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f34e37fe640 1 -- 172.21.15.76:0/2851176605 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559557ee9e10 con 0x55955812ce30 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f34e37fe640 10 client.8553 handle_command_reply: tid=0 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 client.8553 shutdown 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 msgr2=0x559558106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558106a70 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f34f80ac040 tx=0x7f34f801ad30 comp rx=0 tx=0).stop 2024-09-19T04:20:15.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 shutdown_connections 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955812ce30 0x55955815c020 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 0x55955812c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558106a70 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 --2- 172.21.15.76:0/2851176605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x559558106530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f34e37fe640 0 client.8553 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 >> 172.21.15.76:0/2851176605 conn(0x55955812c040 msgr2=0x5595580fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 shutdown_connections 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 1 -- 172.21.15.76:0/2851176605 wait complete. 2024-09-19T04:20:15.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.864+0000 7f350e014000 20 client.8553 trim_cache size 0 max 16384 2024-09-19T04:20:15.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 Processor -- start 2024-09-19T04:20:15.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 -- start start 2024-09-19T04:20:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x5595581066e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558129d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 0x55955812a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f35041ffd90 con 0x55955814ad70 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f35041fef60 con 0x5595580fedb0 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.880+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f35041ff660 con 0x55955814b690 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558129d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558129d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51798/0 (socket says 172.21.15.76:51798) 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 -- 172.21.15.76:0/1503051214 learned_addr learned my addr 172.21.15.76:0/1503051214 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 -- 172.21.15.76:0/1503051214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 msgr2=0x55955812a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 --2- 172.21.15.76:0/1503051214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 0x55955812a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 -- 172.21.15.76:0/1503051214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 msgr2=0x5595581066e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 --2- 172.21.15.76:0/1503051214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x5595581066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 -- 172.21.15.76:0/1503051214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559557f0d470 con 0x55955814ad70 2024-09-19T04:20:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 --2- 172.21.15.76:0/1503051214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558129d20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f34f801c9d0 tx=0x7f34f80b5040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8ed83977f3c565 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e9ffb640 1 -- 172.21.15.76:0/1503051214 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f80b3590 con 0x55955814ad70 2024-09-19T04:20:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e9ffb640 1 -- 172.21.15.76:0/1503051214 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f8000e50 con 0x55955814ad70 2024-09-19T04:20:15.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e9ffb640 1 -- 172.21.15.76:0/1503051214 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f8004280 con 0x55955814ad70 2024-09-19T04:20:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- 172.21.15.76:0/1503051214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 msgr2=0x559558129d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 --2- 172.21.15.76:0/1503051214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558129d20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f34f801c9d0 tx=0x7f34f80b5040 comp rx=0 tx=0).stop 2024-09-19T04:20:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- 172.21.15.76:0/1503051214 shutdown_connections 2024-09-19T04:20:15.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 --2- 172.21.15.76:0/1503051214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814b690 0x55955812a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 --2- 172.21.15.76:0/1503051214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55955814ad70 0x559558129d20 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 --2- 172.21.15.76:0/1503051214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5595580fedb0 0x5595581066e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- 172.21.15.76:0/1503051214 >> 172.21.15.76:0/1503051214 conn(0x55955812c040 msgr2=0x559558149320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- 172.21.15.76:0/1503051214 shutdown_connections 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- 172.21.15.76:0/1503051214 wait complete. 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- start start 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 Processor -- start 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595581c3240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955815d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x55955815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x559557f4bd00 con 0x5595580fedb0 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x559557f5dfd0 con 0x55955814b690 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x559557f17400 con 0x55955814ad70 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 10 client.0 _ll_register_callbacks cb 0x7f34f80096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:15.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595581c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595581c3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51816/0 (socket says 172.21.15.76:51816) 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 -- 172.21.15.76:0/684822214 learned_addr learned my addr 172.21.15.76:0/684822214 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3503fff640 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955815d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 -- 172.21.15.76:0/684822214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 msgr2=0x55955815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955815d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 -- 172.21.15.76:0/684822214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 msgr2=0x55955815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x55955815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 -- 172.21.15.76:0/684822214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559557ee3640 con 0x5595580fedb0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f3508cf4640 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595581c3240 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f34f400ed80 tx=0x7f34f4004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed176cca057495b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e37fe640 1 -- 172.21.15.76:0/684822214 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34f4031060 con 0x5595580fedb0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e37fe640 1 -- 172.21.15.76:0/684822214 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f34f402c440 con 0x5595580fedb0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34f8000e50 con 0x5595580fedb0 2024-09-19T04:20:15.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e37fe640 1 -- 172.21.15.76:0/684822214 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f34f4037050 con 0x5595580fedb0 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 10 client.8559 fetch_fsmap learned FSMap version 109 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=109,monmap=2+}) -- 0x559557ee73b0 con 0x5595580fedb0 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e37fe640 1 -- 172.21.15.76:0/684822214 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34f8000e50 con 0x5595580fedb0 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f34e37fe640 1 -- 172.21.15.76:0/684822214 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 109) ==== 3554+0+0 (secure 0 0 0) 0x7f34f403a020 con 0x5595580fedb0 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 10 client.8559 fetch_fsmap finished waiting for FSMap version 109 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.884+0000 7f350e014000 10 client.8559 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 20 client.8559 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955815daf0 0x55955815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 4 client.8559 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559557f5d470 con 0x55955815daf0 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f3503fff640 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955815daf0 0x55955815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:15.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f34e37fe640 10 client.8559 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f3503fff640 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955815daf0 0x55955815ded0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f827ae9c09d558e server_cookie=ab108e4b6b9c80fc in_seq=0 out_seq=0 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f34e37fe640 1 -- 172.21.15.76:0/684822214 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x559557f5d470 con 0x55955815daf0 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f34e37fe640 10 client.8559 handle_command_reply: tid=0 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 client.8559 shutdown 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 msgr2=0x5595581c3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595581c3240 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f34f400ed80 tx=0x7f34f4004700 comp rx=0 tx=0).stop 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 shutdown_connections 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55955815daf0 0x55955815ded0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55955814b690 0x55955815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55955814ad70 0x55955815d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f34e37fe640 0 client.8559 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:15.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 --2- 172.21.15.76:0/684822214 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5595580fedb0 0x5595581c3240 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 >> 172.21.15.76:0/684822214 conn(0x55955812c040 msgr2=0x559558149b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 shutdown_connections 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 1 -- 172.21.15.76:0/684822214 wait complete. 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350e014000 20 client.8559 trim_cache size 0 max 16384 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:15.894 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:15.895 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:15.895 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34f00678c0 msgr2=0x7f34f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34f00678c0 0x7f34f0069d80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f34f40098e0 tx=0x7f34f4009060 comp rx=0 tx=0).stop 2024-09-19T04:20:15.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350415f1d0 msgr2=0x7f35041502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350415f1d0 0x7f35041502d0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00cef0 tx=0x7f34fc00a710 comp rx=0 tx=0).stop 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 shutdown_connections 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f34f00678c0 0x7f34f0069d80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f350415f1d0 0x7f35041502d0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f35041520a0 0x7f350414a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 --2- 172.21.15.76:0/1835035186 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3504070020 0x7f3504099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 >> 172.21.15.76:0/1835035186 conn(0x7f35040b1f90 msgr2=0x7f350406f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 shutdown_connections 2024-09-19T04:20:15.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:15.888+0000 7f350af58640 1 -- 172.21.15.76:0/1835035186 wait complete. 2024-09-19T04:20:15.905 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:15.905 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:16.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.016+0000 7faa66dee640 1 Processor -- start 2024-09-19T04:20:16.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.016+0000 7faa66dee640 1 -- start start 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa66dee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 0x7faa600cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa66dee640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa600cf450 0x7faa6006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa66dee640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa600d0300 con 0x7faa60161a60 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa66dee640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7faa600b0f60 con 0x7faa600cf450 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa66dee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7faa600cfd40 con 0x7faa600d3060 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa64b8a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7faa60161a60 0x7faa600d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51820/0 (socket says 172.21.15.76:51820) 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa64b8a640 1 -- 172.21.15.76:0/740839578 learned_addr learned my addr 172.21.15.76:0/740839578 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 0x7faa600cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa6538b640 1 --2- 172.21.15.76:0/740839578 >> v2:172.21.15.116:3300/0 conn(0x7faa600cf450 0x7faa6006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1292996300 0 0) 0x7faa540014a0 con 0x7faa60161a60 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faa48003530 con 0x7faa60161a60 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1030197818 0 0) 0x7faa48003530 con 0x7faa60161a60 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 msgr2=0x7faa600cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 0x7faa600cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 >> v2:172.21.15.116:3300/0 conn(0x7faa600cf450 msgr2=0x7faa6006d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 --2- 172.21.15.76:0/740839578 >> v2:172.21.15.116:3300/0 conn(0x7faa600cf450 0x7faa6006d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa600d0020 con 0x7faa60161a60 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7faa54001bd0 con 0x7faa60161a60 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 >> v1:172.21.15.76:6789/0 conn(0x7faa60161a60 legacy=0x7faa600d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 0x7faa480039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa48004d50 0x7faa480051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa600b0f60 con 0x7faa48003f40 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa600cfd40 con 0x7faa48004d50 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7faa4800ba20 con 0x7faa600d3060 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa64b8a640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 0x7faa480039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 -- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 msgr2=0x7faa480039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa600d3060 0x7faa480039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 -- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa48004d50 msgr2=0x7faa480051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa48004d50 0x7faa480051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 -- 172.21.15.76:0/740839578 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa48003530 con 0x7faa48003f40 2024-09-19T04:20:16.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.020+0000 7faa5ffff640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa48004810 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7faa50009b10 tx=0x7faa5001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9759367c66addc47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:16.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa500b2070 con 0x7faa48003f40 2024-09-19T04:20:16.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa500a5610 con 0x7faa48003f40 2024-09-19T04:20:16.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa5f7fe640 1 -- 172.21.15.76:0/740839578 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa500096b0 con 0x7faa48003f40 2024-09-19T04:20:16.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- 172.21.15.76:0/740839578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 msgr2=0x7faa48004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 --2- 172.21.15.76:0/740839578 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa48004810 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7faa50009b10 tx=0x7faa5001ca40 comp rx=0 tx=0).stop 2024-09-19T04:20:16.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa6538b640 1 -- 172.21.15.76:0/740839578 reap_dead start 2024-09-19T04:20:16.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- 172.21.15.76:0/740839578 shutdown_connections 2024-09-19T04:20:16.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- 172.21.15.76:0/740839578 >> 172.21.15.76:0/740839578 conn(0x7faa6008bf90 msgr2=0x7faa6009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:16.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- 172.21.15.76:0/740839578 shutdown_connections 2024-09-19T04:20:16.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- 172.21.15.76:0/740839578 wait complete. 2024-09-19T04:20:16.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 Processor -- start 2024-09-19T04:20:16.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- start start 2024-09-19T04:20:16.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa600b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60162070 0x7faa60159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa60163660 0x7faa601608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7faa60168f10 con 0x7faa48003f40 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7faa6006e610 con 0x7faa60163660 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7faa60169090 con 0x7faa60162070 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa600b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa600b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51832/0 (socket says 172.21.15.76:51832) 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 -- 172.21.15.76:0/1541601012 learned_addr learned my addr 172.21.15.76:0/1541601012 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa6538b640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa60163660 0x7faa601608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa5ffff640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60162070 0x7faa60159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 -- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60162070 msgr2=0x7faa60159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60162070 0x7faa60159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 -- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa60163660 msgr2=0x7faa601608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa60163660 0x7faa601608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 -- 172.21.15.76:0/1541601012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa601625c0 con 0x7faa48003f40 2024-09-19T04:20:16.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa64b8a640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa600b2f40 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7faa54001d10 tx=0x7faa54001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:16.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa5d7fa640 1 -- 172.21.15.76:0/1541601012 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa54019070 con 0x7faa48003f40 2024-09-19T04:20:16.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa5d7fa640 1 -- 172.21.15.76:0/1541601012 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa54004d70 con 0x7faa48003f40 2024-09-19T04:20:16.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa601628a0 con 0x7faa48003f40 2024-09-19T04:20:16.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa5d7fa640 1 -- 172.21.15.76:0/1541601012 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa540072d0 con 0x7faa48003f40 2024-09-19T04:20:16.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.024+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa60162e60 con 0x7faa48003f40 2024-09-19T04:20:16.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.028+0000 7faa5d7fa640 1 -- 172.21.15.76:0/1541601012 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7faa5401c070 con 0x7faa48003f40 2024-09-19T04:20:16.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.028+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa6006d6b0 con 0x7faa48003f40 2024-09-19T04:20:16.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.028+0000 7faa5d7fa640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa4c0678c0 0x7faa4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.028+0000 7faa5d7fa640 1 -- 172.21.15.76:0/1541601012 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7faa54086090 con 0x7faa48003f40 2024-09-19T04:20:16.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.028+0000 7faa5ffff640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa4c0678c0 0x7faa4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.028+0000 7faa5ffff640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa4c0678c0 0x7faa4c069d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faa500af000 tx=0x7faa50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:16.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.032+0000 7faa5d7fa640 1 -- 172.21.15.76:0/1541601012 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7faa600d0300 con 0x7faa48003f40 2024-09-19T04:20:16.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.220+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa6006bcc0 con 0x7faa48003f40 2024-09-19T04:20:16.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.220+0000 7faa5d7fa640 1 -- 172.21.15.76:0/1541601012 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5274 (secure 0 0 0) 0x7faa54056a10 con 0x7faa48003f40 2024-09-19T04:20:16.224 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:16.224 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":109,"btime":"2024-09-19T04:20:12:044551+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":109,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:11.115053+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[6127,8115,8127]},"id":7}]} 2024-09-19T04:20:16.224 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 109 2024-09-19T04:20:16.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa4c0678c0 msgr2=0x7faa4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa4c0678c0 0x7faa4c069d80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7faa500af000 tx=0x7faa50008040 comp rx=0 tx=0).stop 2024-09-19T04:20:16.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 msgr2=0x7faa600b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa600b2f40 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7faa54001d10 tx=0x7faa54001d40 comp rx=0 tx=0).stop 2024-09-19T04:20:16.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 shutdown_connections 2024-09-19T04:20:16.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7faa4c0678c0 0x7faa4c069d80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa60163660 0x7faa601608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7faa60162070 0x7faa60159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 --2- 172.21.15.76:0/1541601012 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7faa48003f40 0x7faa600b2f40 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 >> 172.21.15.76:0/1541601012 conn(0x7faa6008bf90 msgr2=0x7faa600b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:16.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 shutdown_connections 2024-09-19T04:20:16.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.224+0000 7faa66dee640 1 -- 172.21.15.76:0/1541601012 wait complete. 2024-09-19T04:20:16.242 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:16.242 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:16.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7fd8b0640 1 Processor -- start 2024-09-19T04:20:16.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7fd8b0640 1 -- start start 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7fd8b0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815bec0 0x7fe7f80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7fd8b0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7f815c2a0 0x7fe7f80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7fd8b0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7f80b3290 con 0x7fe7f80af2a0 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7fd8b0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe7f80b3570 con 0x7fe7f815c2a0 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7fd8b0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f80b3850 con 0x7fe7f815bec0 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7f815c2a0 0x7fe7f80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe7f80af2a0 0x7fe7f815fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51826/0 (socket says 172.21.15.76:51826) 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe7f815c2a0 0x7fe7f80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51836/0 (socket says 172.21.15.76:51836) 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f77fe640 1 -- 172.21.15.76:0/3168762398 learned_addr learned my addr 172.21.15.76:0/3168762398 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:16.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f6ffd640 1 --2- 172.21.15.76:0/3168762398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815bec0 0x7fe7f80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f67fc640 1 -- 172.21.15.76:0/3168762398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815bec0 msgr2=0x7fe7f80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:16.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f67fc640 1 --2- 172.21.15.76:0/3168762398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815bec0 0x7fe7f80ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f67fc640 1 -- 172.21.15.76:0/3168762398 >> v1:172.21.15.76:6789/0 conn(0x7fe7f80af2a0 legacy=0x7fe7f815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f5ffb640 1 -- 172.21.15.76:0/3168762398 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 530216936 0 0) 0x7fe7f80b3290 con 0x7fe7f80af2a0 2024-09-19T04:20:16.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f67fc640 1 -- 172.21.15.76:0/3168762398 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f80b3b30 con 0x7fe7f815c2a0 2024-09-19T04:20:16.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.356+0000 7fe7f6ffd640 1 --2- 172.21.15.76:0/3168762398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815bec0 0x7fe7f80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:16.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 --2- 172.21.15.76:0/3168762398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815c2a0 0x7fe7f80aed60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80097c0 tx=0x7fe7e801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2917cd670f35265 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:16.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f5ffb640 1 -- 172.21.15.76:0/3168762398 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80a7070 con 0x7fe7f815c2a0 2024-09-19T04:20:16.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f5ffb640 1 -- 172.21.15.76:0/3168762398 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e80b0070 con 0x7fe7f815c2a0 2024-09-19T04:20:16.359 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f5ffb640 1 -- 172.21.15.76:0/3168762398 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80ab890 con 0x7fe7f815c2a0 2024-09-19T04:20:16.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/3168762398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815c2a0 msgr2=0x7fe7f80aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/3168762398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815c2a0 0x7fe7f80aed60 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fe7e80097c0 tx=0x7fe7e801c470 comp rx=0 tx=0).stop 2024-09-19T04:20:16.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/3168762398 shutdown_connections 2024-09-19T04:20:16.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/3168762398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815c2a0 0x7fe7f80aed60 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/3168762398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815bec0 0x7fe7f80ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/3168762398 >> 172.21.15.76:0/3168762398 conn(0x7fe7f80b1f90 msgr2=0x7fe7f80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:16.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/3168762398 shutdown_connections 2024-09-19T04:20:16.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/3168762398 wait complete. 2024-09-19T04:20:16.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 Processor -- start 2024-09-19T04:20:16.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- start start 2024-09-19T04:20:16.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7f80af2a0 0x7fe7f814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815bec0 0x7fe7f8157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815c2a0 0x7fe7f8157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80af6f0 con 0x7fe7f815bec0 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f80ad740 con 0x7fe7f80af2a0 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7fd8b0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f8169000 con 0x7fe7f815c2a0 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815bec0 0x7fe7f8157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815bec0 0x7fe7f8157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51852/0 (socket says 172.21.15.76:51852) 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 -- 172.21.15.76:0/261609912 learned_addr learned my addr 172.21.15.76:0/261609912 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f6ffd640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7f80af2a0 0x7fe7f814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f77fe640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815c2a0 0x7fe7f8157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 -- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815c2a0 msgr2=0x7fe7f8157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815c2a0 0x7fe7f8157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 -- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7f80af2a0 msgr2=0x7fe7f814aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7f80af2a0 0x7fe7f814aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f67fc640 1 -- 172.21.15.76:0/261609912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f815c980 con 0x7fe7f815bec0 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.360+0000 7fe7f6ffd640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7f80af2a0 0x7fe7f814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7f67fc640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815bec0 0x7fe7f8157980 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe7e801ab20 tx=0x7fe7e8019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7dffff640 1 -- 172.21.15.76:0/261609912 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80bf450 con 0x7fe7f815bec0 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7dffff640 1 -- 172.21.15.76:0/261609912 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e80bf5f0 con 0x7fe7f815bec0 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f815cc60 con 0x7fe7f815bec0 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7dffff640 1 -- 172.21.15.76:0/261609912 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e80bf9e0 con 0x7fe7f815bec0 2024-09-19T04:20:16.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f815d220 con 0x7fe7f815bec0 2024-09-19T04:20:16.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7f80ad200 con 0x7fe7f815bec0 2024-09-19T04:20:16.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7dffff640 1 -- 172.21.15.76:0/261609912 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe7e80bfb80 con 0x7fe7f815bec0 2024-09-19T04:20:16.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7dffff640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7e40678c0 0x7fe7e4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:16.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7dffff640 1 -- 172.21.15.76:0/261609912 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fe7e8120e20 con 0x7fe7f815bec0 2024-09-19T04:20:16.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7f6ffd640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7e40678c0 0x7fe7e4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:16.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.364+0000 7fe7f6ffd640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7e40678c0 0x7fe7e4069d80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8159d90 tx=0x7fe7ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:16.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.368+0000 7fe7dffff640 1 -- 172.21.15.76:0/261609912 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe7e8016090 con 0x7fe7f815bec0 2024-09-19T04:20:16.587 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe7f8075740 con 0x7fe7f815bec0 2024-09-19T04:20:16.588 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7dffff640 1 -- 172.21.15.76:0/261609912 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1363 (secure 0 0 0) 0x7fe7e80b3e60 con 0x7fe7f815bec0 2024-09-19T04:20:16.588 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:16.589 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":435,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":62,"data_bytes":664919,"bytes_used":267460608,"bytes_avail":772826652672,"bytes_total":773094113280,"write_bytes_sec":2303,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":109,"btime":"2024-09-19T04:20:12:044551+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-09-19T04:19:33.856403+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7e40678c0 msgr2=0x7fe7e4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7e40678c0 0x7fe7e4069d80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8159d90 tx=0x7fe7ec008040 comp rx=0 tx=0).stop 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815bec0 msgr2=0x7fe7f8157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815bec0 0x7fe7f8157980 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fe7e801ab20 tx=0x7fe7e8019210 comp rx=0 tx=0).stop 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 shutdown_connections 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe7e40678c0 0x7fe7e4069d80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe7f815c2a0 0x7fe7f8157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe7f815bec0 0x7fe7f8157980 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 --2- 172.21.15.76:0/261609912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7f80af2a0 0x7fe7f814aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 >> 172.21.15.76:0/261609912 conn(0x7fe7f80b1f90 msgr2=0x7fe7f8074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 shutdown_connections 2024-09-19T04:20:16.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:16.588+0000 7fe7fd8b0640 1 -- 172.21.15.76:0/261609912 wait complete. 2024-09-19T04:20:16.606 INFO:tasks.ceph.ceph_manager.ceph:{} 2024-09-19T04:20:17.606 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:17.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65f966640 1 Processor -- start 2024-09-19T04:20:17.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65f966640 1 -- start start 2024-09-19T04:20:17.736 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65f966640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa65815bec0 0x7fa6580ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65f966640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa65815c2a0 0x7fa6580aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65f966640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6580b3290 con 0x7fa6580af2a0 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65f966640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fa6580b3570 con 0x7fa65815c2a0 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65f966640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa6580b3850 con 0x7fa65815bec0 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65cf01640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fa65815c2a0 0x7fa6580aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65df03640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa6580af2a0 0x7fa65815fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51828/0 (socket says 172.21.15.76:51828) 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65df03640 1 -- 172.21.15.76:0/943620876 learned_addr learned my addr 172.21.15.76:0/943620876 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65d702640 1 --2- 172.21.15.76:0/943620876 >> v2:172.21.15.116:3300/0 conn(0x7fa65815bec0 0x7fa6580ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:17.737 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65cf01640 1 -- 172.21.15.76:0/943620876 >> v2:172.21.15.116:3300/0 conn(0x7fa65815bec0 msgr2=0x7fa6580ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:17.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65cf01640 1 --2- 172.21.15.76:0/943620876 >> v2:172.21.15.116:3300/0 conn(0x7fa65815bec0 0x7fa6580ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65cf01640 1 -- 172.21.15.76:0/943620876 >> v1:172.21.15.76:6789/0 conn(0x7fa6580af2a0 legacy=0x7fa65815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:17.738 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.736+0000 7fa65cf01640 1 -- 172.21.15.76:0/943620876 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6580b3b30 con 0x7fa65815c2a0 2024-09-19T04:20:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65cf01640 1 --2- 172.21.15.76:0/943620876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa65815c2a0 0x7fa6580aed60 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa658071710 tx=0x7fa64801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22be1edc54c02687 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa64ffff640 1 -- 172.21.15.76:0/943620876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6480ab650 con 0x7fa65815c2a0 2024-09-19T04:20:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa64ffff640 1 -- 172.21.15.76:0/943620876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6480b0070 con 0x7fa65815c2a0 2024-09-19T04:20:17.739 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa64ffff640 1 -- 172.21.15.76:0/943620876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6480aba10 con 0x7fa65815c2a0 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- 172.21.15.76:0/943620876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa65815c2a0 msgr2=0x7fa6580aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 --2- 172.21.15.76:0/943620876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa65815c2a0 0x7fa6580aed60 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fa658071710 tx=0x7fa64801c450 comp rx=0 tx=0).stop 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- 172.21.15.76:0/943620876 shutdown_connections 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 --2- 172.21.15.76:0/943620876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa65815c2a0 0x7fa6580aed60 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 --2- 172.21.15.76:0/943620876 >> v2:172.21.15.116:3300/0 conn(0x7fa65815bec0 0x7fa6580ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- 172.21.15.76:0/943620876 >> 172.21.15.76:0/943620876 conn(0x7fa6580b1f90 msgr2=0x7fa6580b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- 172.21.15.76:0/943620876 shutdown_connections 2024-09-19T04:20:17.740 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- 172.21.15.76:0/943620876 wait complete. 2024-09-19T04:20:17.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 Processor -- start 2024-09-19T04:20:17.741 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- start start 2024-09-19T04:20:17.743 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6580af2a0 0x7fa6581522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa65815bec0 0x7fa65809dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa65815c2a0 0x7fa658157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6580af6b0 con 0x7fa6580af2a0 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa6580ad740 con 0x7fa65815bec0 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65f966640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa658169000 con 0x7fa65815c2a0 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65d702640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6580af2a0 0x7fa6581522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65d702640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6580af2a0 0x7fa6581522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51872/0 (socket says 172.21.15.76:51872) 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.740+0000 7fa65d702640 1 -- 172.21.15.76:0/3234770868 learned_addr learned my addr 172.21.15.76:0/3234770868 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65cf01640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa65815bec0 0x7fa65809dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65d702640 1 -- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa65815c2a0 msgr2=0x7fa658157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65df03640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa65815c2a0 0x7fa658157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:17.744 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65d702640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa65815c2a0 0x7fa658157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65d702640 1 -- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa65815bec0 msgr2=0x7fa65809dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65d702640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa65815bec0 0x7fa65809dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65d702640 1 -- 172.21.15.76:0/3234770868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa658070280 con 0x7fa6580af2a0 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65cf01640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa65815bec0 0x7fa65809dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65d702640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6580af2a0 0x7fa6581522e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fa65400d800 tx=0x7fa65400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa64dffb640 1 -- 172.21.15.76:0/3234770868 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65400b6c0 con 0x7fa6580af2a0 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa64dffb640 1 -- 172.21.15.76:0/3234770868 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa654019070 con 0x7fa6580af2a0 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa64dffb640 1 -- 172.21.15.76:0/3234770868 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65400ba60 con 0x7fa6580af2a0 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65809b1d0 con 0x7fa6580af2a0 2024-09-19T04:20:17.745 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa65809b790 con 0x7fa6580af2a0 2024-09-19T04:20:17.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6580ad200 con 0x7fa6580af2a0 2024-09-19T04:20:17.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa64dffb640 1 -- 172.21.15.76:0/3234770868 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa65400bc00 con 0x7fa6580af2a0 2024-09-19T04:20:17.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa64dffb640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa63c0678c0 0x7fa63c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:17.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa64dffb640 1 -- 172.21.15.76:0/3234770868 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fa6540862c0 con 0x7fa6580af2a0 2024-09-19T04:20:17.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65cf01640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa63c0678c0 0x7fa63c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:17.746 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.744+0000 7fa65cf01640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa63c0678c0 0x7fa63c069d80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa6480af000 tx=0x7fa6480a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:17.750 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.748+0000 7fa64dffb640 1 -- 172.21.15.76:0/3234770868 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa654056c40 con 0x7fa6580af2a0 2024-09-19T04:20:17.948 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.948+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa65806fcd0 con 0x7fa6580af2a0 2024-09-19T04:20:17.949 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.948+0000 7fa64dffb640 1 -- 172.21.15.76:0/3234770868 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) ==== 95+0+5276 (secure 0 0 0) 0x7fa6540860e0 con 0x7fa6580af2a0 2024-09-19T04:20:17.950 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:17.950 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":110,"btime":"2024-09-19T04:20:17:064361+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:16.114867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":5,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":3,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:17.950 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 110 2024-09-19T04:20:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa63c0678c0 msgr2=0x7fa63c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa63c0678c0 0x7fa63c069d80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa6480af000 tx=0x7fa6480a7000 comp rx=0 tx=0).stop 2024-09-19T04:20:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6580af2a0 msgr2=0x7fa6581522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6580af2a0 0x7fa6581522e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fa65400d800 tx=0x7fa65400dcc0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.952 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 shutdown_connections 2024-09-19T04:20:17.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa63c0678c0 0x7fa63c069d80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa65815c2a0 0x7fa658157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa65815bec0 0x7fa65809dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 --2- 172.21.15.76:0/3234770868 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6580af2a0 0x7fa6581522e0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:17.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 >> 172.21.15.76:0/3234770868 conn(0x7fa6580b1f90 msgr2=0x7fa658074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:17.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 shutdown_connections 2024-09-19T04:20:17.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:17.952+0000 7fa65f966640 1 -- 172.21.15.76:0/3234770868 wait complete. 2024-09-19T04:20:17.968 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:17.968 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:18.073 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:20:18.072+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:20:18.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f693ab64640 1 Processor -- start 2024-09-19T04:20:18.085 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f693ab64640 1 -- start start 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f693ab64640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f693415e340 0x7f693409aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f693ab64640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69341629a0 0x7f693406ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f693ab64640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69340ae000 con 0x7f6934161e50 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f693ab64640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f69340b4780 con 0x7f69341629a0 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f693ab64640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f693415e720 con 0x7f693415e340 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6939101640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69341629a0 0x7f693406ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6939101640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f69341629a0 0x7f693406ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51882/0 (socket says 172.21.15.76:51882) 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6939101640 1 -- 172.21.15.76:0/2593376707 learned_addr learned my addr 172.21.15.76:0/2593376707 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6933fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6934161e50 0x7f6934162290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51834/0 (socket says 172.21.15.76:51834) 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6938900640 1 --2- 172.21.15.76:0/2593376707 >> v2:172.21.15.116:3300/0 conn(0x7f693415e340 0x7f693409aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6939101640 1 -- 172.21.15.76:0/2593376707 >> v2:172.21.15.116:3300/0 conn(0x7f693415e340 msgr2=0x7f693409aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:18.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6939101640 1 --2- 172.21.15.76:0/2593376707 >> v2:172.21.15.116:3300/0 conn(0x7f693415e340 0x7f693409aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f69337fe640 1 -- 172.21.15.76:0/2593376707 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3690822720 0 0) 0x7f69340ae000 con 0x7f6934161e50 2024-09-19T04:20:18.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6939101640 1 -- 172.21.15.76:0/2593376707 >> v1:172.21.15.76:6789/0 conn(0x7f6934161e50 legacy=0x7f6934162290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.084+0000 7f6939101640 1 -- 172.21.15.76:0/2593376707 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69340b41c0 con 0x7f69341629a0 2024-09-19T04:20:18.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f6939101640 1 --2- 172.21.15.76:0/2593376707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f693406ef90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f69340716a0 tx=0x7f692c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a918ebb621d4ff40 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:18.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f69337fe640 1 -- 172.21.15.76:0/2593376707 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0a56c0 con 0x7f69341629a0 2024-09-19T04:20:18.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f69337fe640 1 -- 172.21.15.76:0/2593376707 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c0a5860 con 0x7f69341629a0 2024-09-19T04:20:18.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f69337fe640 1 -- 172.21.15.76:0/2593376707 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0a5c30 con 0x7f69341629a0 2024-09-19T04:20:18.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 -- 172.21.15.76:0/2593376707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 msgr2=0x7f693406ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.090 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 --2- 172.21.15.76:0/2593376707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f693406ef90 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f69340716a0 tx=0x7f692c01c9f0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 -- 172.21.15.76:0/2593376707 shutdown_connections 2024-09-19T04:20:18.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 --2- 172.21.15.76:0/2593376707 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f693406ef90 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 --2- 172.21.15.76:0/2593376707 >> v2:172.21.15.116:3300/0 conn(0x7f693415e340 0x7f693409aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 -- 172.21.15.76:0/2593376707 >> 172.21.15.76:0/2593376707 conn(0x7f69340733a0 msgr2=0x7f69340b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:18.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 -- 172.21.15.76:0/2593376707 shutdown_connections 2024-09-19T04:20:18.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.088+0000 7f693ab64640 1 -- 172.21.15.76:0/2593376707 wait complete. 2024-09-19T04:20:18.091 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 Processor -- start 2024-09-19T04:20:18.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 -- start start 2024-09-19T04:20:18.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f693415e340 0x7f69340ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.093 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6934161e50 0x7f6934156e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f6934150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f693416b1f0 con 0x7f69341629a0 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f693406fef0 con 0x7f6934161e50 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f693416b370 con 0x7f693415e340 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f6934150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f6934150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51884/0 (socket says 172.21.15.76:51884) 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 -- 172.21.15.76:0/1476083146 learned_addr learned my addr 172.21.15.76:0/1476083146 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6933fff640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6934161e50 0x7f6934156e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6938900640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f693415e340 0x7f69340ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 -- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f693415e340 msgr2=0x7f69340ae200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f693415e340 0x7f69340ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.094 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 -- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6934161e50 msgr2=0x7f6934156e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6934161e50 0x7f6934156e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 -- 172.21.15.76:0/1476083146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6934153190 con 0x7f69341629a0 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6933fff640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6934161e50 0x7f6934156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f6939101640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f6934150700 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f692c01a620 tx=0x7f692c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f69317fa640 1 -- 172.21.15.76:0/1476083146 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0b8670 con 0x7f69341629a0 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f69317fa640 1 -- 172.21.15.76:0/1476083146 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c0be070 con 0x7f69341629a0 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6934152eb0 con 0x7f69341629a0 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f69317fa640 1 -- 172.21.15.76:0/1476083146 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0b8a90 con 0x7f69341629a0 2024-09-19T04:20:18.095 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69340cf4b0 con 0x7f69341629a0 2024-09-19T04:20:18.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693409aa00 con 0x7f69341629a0 2024-09-19T04:20:18.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.092+0000 7f69317fa640 1 -- 172.21.15.76:0/1476083146 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f692c0b8c30 con 0x7f69341629a0 2024-09-19T04:20:18.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.096+0000 7f69317fa640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69180678c0 0x7f6918069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:18.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.096+0000 7f69317fa640 1 -- 172.21.15.76:0/1476083146 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f692c121070 con 0x7f69341629a0 2024-09-19T04:20:18.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.096+0000 7f6938900640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69180678c0 0x7f6918069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:18.096 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.096+0000 7f6938900640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69180678c0 0x7f6918069d80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f69280096f0 tx=0x7f6928008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:18.099 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.100+0000 7f69317fa640 1 -- 172.21.15.76:0/1476083146 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f692c0165e0 con 0x7f69341629a0 2024-09-19T04:20:18.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f69340b2a90 con 0x7f69341629a0 2024-09-19T04:20:18.304 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f69317fa640 1 -- 172.21.15.76:0/1476083146 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1928 (secure 0 0 0) 0x7f692c0f1b10 con 0x7f69341629a0 2024-09-19T04:20:18.305 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:18.305 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"2 MDSs report damaged metadata","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":437,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":694960,"bytes_used":267493376,"bytes_avail":772826619904,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":8003,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":111,"btime":"2024-09-19T04:20:18:076669+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-09-19T04:20:17.870600+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69180678c0 msgr2=0x7f6918069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69180678c0 0x7f6918069d80 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f69280096f0 tx=0x7f6928008040 comp rx=0 tx=0).stop 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 msgr2=0x7f6934150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f6934150700 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f692c01a620 tx=0x7f692c019790 comp rx=0 tx=0).stop 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 shutdown_connections 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69180678c0 0x7f6918069d80 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f69341629a0 0x7f6934150700 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6934161e50 0x7f6934156e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 --2- 172.21.15.76:0/1476083146 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f693415e340 0x7f69340ae200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 >> 172.21.15.76:0/1476083146 conn(0x7f69340733a0 msgr2=0x7f6934154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 shutdown_connections 2024-09-19T04:20:18.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:18.304+0000 7f693ab64640 1 -- 172.21.15.76:0/1476083146 wait complete. 2024-09-19T04:20:18.322 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:19.324 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:19.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e82d2640 1 Processor -- start 2024-09-19T04:20:19.451 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e82d2640 1 -- start start 2024-09-19T04:20:19.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e82d2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f58e015c2a0 0x7f58e00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e82d2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58e00af2a0 0x7f58e006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e82d2640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58e00b3290 con 0x7f58e015bec0 2024-09-19T04:20:19.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e82d2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f58e00b3570 con 0x7f58e00af2a0 2024-09-19T04:20:19.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e82d2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f58e00b3850 con 0x7f58e015c2a0 2024-09-19T04:20:19.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e686f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58e00af2a0 0x7f58e006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.453 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e686f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f58e00af2a0 0x7f58e006e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51890/0 (socket says 172.21.15.76:51890) 2024-09-19T04:20:19.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e686f640 1 -- 172.21.15.76:0/1193692137 learned_addr learned my addr 172.21.15.76:0/1193692137 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:19.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e586d640 1 --2- 172.21.15.76:0/1193692137 >> v2:172.21.15.116:3300/0 conn(0x7f58e015c2a0 0x7f58e00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e686f640 1 -- 172.21.15.76:0/1193692137 >> v2:172.21.15.116:3300/0 conn(0x7f58e015c2a0 msgr2=0x7f58e00aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:19.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e686f640 1 --2- 172.21.15.76:0/1193692137 >> v2:172.21.15.116:3300/0 conn(0x7f58e015c2a0 0x7f58e00aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e686f640 1 -- 172.21.15.76:0/1193692137 >> v1:172.21.15.116:6789/0 conn(0x7f58e015bec0 legacy=0x7f58e00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:19.454 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.452+0000 7f58e686f640 1 -- 172.21.15.76:0/1193692137 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e00b3b30 con 0x7f58e00af2a0 2024-09-19T04:20:19.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e686f640 1 --2- 172.21.15.76:0/1193692137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e006e000 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ae510 tx=0x7f58d801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682eae4c00b0f914 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:19.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e506c640 1 -- 172.21.15.76:0/1193692137 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80a5840 con 0x7f58e00af2a0 2024-09-19T04:20:19.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e506c640 1 -- 172.21.15.76:0/1193692137 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d80a59e0 con 0x7f58e00af2a0 2024-09-19T04:20:19.456 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e506c640 1 -- 172.21.15.76:0/1193692137 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d80a5db0 con 0x7f58e00af2a0 2024-09-19T04:20:19.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 -- 172.21.15.76:0/1193692137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 msgr2=0x7f58e006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 --2- 172.21.15.76:0/1193692137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e006e000 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ae510 tx=0x7f58d801c980 comp rx=0 tx=0).stop 2024-09-19T04:20:19.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 -- 172.21.15.76:0/1193692137 shutdown_connections 2024-09-19T04:20:19.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 --2- 172.21.15.76:0/1193692137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e006e000 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 --2- 172.21.15.76:0/1193692137 >> v2:172.21.15.116:3300/0 conn(0x7f58e015c2a0 0x7f58e00aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 -- 172.21.15.76:0/1193692137 >> 172.21.15.76:0/1193692137 conn(0x7f58e00b1f90 msgr2=0x7f58e00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:19.457 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 -- 172.21.15.76:0/1193692137 shutdown_connections 2024-09-19T04:20:19.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 -- 172.21.15.76:0/1193692137 wait complete. 2024-09-19T04:20:19.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 Processor -- start 2024-09-19T04:20:19.458 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.456+0000 7f58e82d2640 1 -- start start 2024-09-19T04:20:19.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58e015bec0 0x7f58e0155f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.460 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e015c2a0 0x7f58e014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f58e006ed50 con 0x7f58e00af2a0 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f58e00af710 con 0x7f58e015c2a0 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f58e006eed0 con 0x7f58e015bec0 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e015d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51894/0 (socket says 172.21.15.76:51894) 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 -- 172.21.15.76:0/431833694 learned_addr learned my addr 172.21.15.76:0/431833694 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 -- 172.21.15.76:0/431833694 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58e015bec0 msgr2=0x7f58e0155f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58e015bec0 0x7f58e0155f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 -- 172.21.15.76:0/431833694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e015c2a0 msgr2=0x7f58e014a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e015c2a0 0x7f58e014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 -- 172.21.15.76:0/431833694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e0156790 con 0x7f58e00af2a0 2024-09-19T04:20:19.461 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e606e640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e015d280 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00c450 tx=0x7f58dc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:19.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58d6ffd640 1 -- 172.21.15.76:0/431833694 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc012830 con 0x7f58e00af2a0 2024-09-19T04:20:19.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58d6ffd640 1 -- 172.21.15.76:0/431833694 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58dc017070 con 0x7f58e00af2a0 2024-09-19T04:20:19.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e0156a70 con 0x7f58e00af2a0 2024-09-19T04:20:19.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58d6ffd640 1 -- 172.21.15.76:0/431833694 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc012c30 con 0x7f58e00af2a0 2024-09-19T04:20:19.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e0157030 con 0x7f58e00af2a0 2024-09-19T04:20:19.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58d6ffd640 1 -- 172.21.15.76:0/431833694 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f58dc01c460 con 0x7f58e00af2a0 2024-09-19T04:20:19.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e006f260 con 0x7f58e00af2a0 2024-09-19T04:20:19.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58d6ffd640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58c40678c0 0x7f58c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58e586d640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58c40678c0 0x7f58c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.460+0000 7f58d6ffd640 1 -- 172.21.15.76:0/431833694 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f58dc085c30 con 0x7f58e00af2a0 2024-09-19T04:20:19.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.464+0000 7f58e586d640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58c40678c0 0x7f58c4069d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f58e006d2a0 tx=0x7f58d0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:19.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.468+0000 7f58d6ffd640 1 -- 172.21.15.76:0/431833694 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f58dc0565b0 con 0x7f58e00af2a0 2024-09-19T04:20:19.657 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.656+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58e006fcd0 con 0x7f58e00af2a0 2024-09-19T04:20:19.658 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.656+0000 7f58d6ffd640 1 -- 172.21.15.76:0/431833694 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f58dc055eb0 con 0x7f58e00af2a0 2024-09-19T04:20:19.658 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:19.659 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:19.659 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58c40678c0 msgr2=0x7f58c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58c40678c0 0x7f58c4069d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f58e006d2a0 tx=0x7f58d0008040 comp rx=0 tx=0).stop 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 msgr2=0x7f58e015d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e015d280 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00c450 tx=0x7f58dc00c910 comp rx=0 tx=0).stop 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 shutdown_connections 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f58c40678c0 0x7f58c4069d80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58e015c2a0 0x7f58e014a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f58e015bec0 0x7f58e0155f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 --2- 172.21.15.76:0/431833694 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f58e00af2a0 0x7f58e015d280 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 >> 172.21.15.76:0/431833694 conn(0x7f58e00b1f90 msgr2=0x7f58e00b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:19.661 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 shutdown_connections 2024-09-19T04:20:19.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.660+0000 7f58e82d2640 1 -- 172.21.15.76:0/431833694 wait complete. 2024-09-19T04:20:19.676 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:19.676 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:19.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bc6ac640 1 Processor -- start 2024-09-19T04:20:19.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bc6ac640 1 -- start start 2024-09-19T04:20:19.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bc6ac640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 0x7f03b41634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bc6ac640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b406e470 con 0x7f03b40b3290 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bc6ac640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b406e750 con 0x7f03b41630c0 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bc6ac640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f03b406fc20 con 0x7f03b406df70 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bac49640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f03b40b3290 0x7f03b40ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51848/0 (socket says 172.21.15.76:51848) 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bac49640 1 -- 172.21.15.76:0/1663344057 learned_addr learned my addr 172.21.15.76:0/1663344057 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9c47640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 0x7f03b41634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2186369360 0 0) 0x7f03b406e470 con 0x7f03b40b3290 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03a0003530 con 0x7f03b40b3290 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4278659913 0 0) 0x7f03a0003530 con 0x7f03b40b3290 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 msgr2=0x7f03b41634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:19.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 0x7f03b41634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 >> v1:172.21.15.116:6789/0 conn(0x7f03b41630c0 legacy=0x7f03b406f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b406ea30 con 0x7f03b40b3290 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f03ac001a00 con 0x7f03b40b3290 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 >> v1:172.21.15.76:6789/0 conn(0x7f03b40b3290 legacy=0x7f03b40ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03a0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03a0004810 0x7f03a0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 0x7f03a00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03b406fc20 con 0x7f03a00039f0 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03a000ba80 con 0x7f03a0004810 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f03a000bc80 con 0x7f03b406df70 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03ba448640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03a0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9c47640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03a0004810 0x7f03a0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03bac49640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 0x7f03a00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03ba448640 1 -- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 msgr2=0x7f03a00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:19.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03ba448640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b406df70 0x7f03a00051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03ba448640 1 -- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03a0004810 msgr2=0x7f03a0004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:19.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03ba448640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03a0004810 0x7f03a0004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03ba448640 1 -- 172.21.15.76:0/1663344057 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a0003530 con 0x7f03a00039f0 2024-09-19T04:20:19.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.780+0000 7f03b9c47640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03a0004810 0x7f03a0004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03ba448640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03a0003e10 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f03b0001750 tx=0x7f03b001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47e4b93ad320fa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b00013a0 con 0x7f03a00039f0 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b00052f0 con 0x7f03a00039f0 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03b9446640 1 -- 172.21.15.76:0/1663344057 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b00a4e60 con 0x7f03a00039f0 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1663344057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 msgr2=0x7f03a0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 --2- 172.21.15.76:0/1663344057 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03a0003e10 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f03b0001750 tx=0x7f03b001c3c0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bac49640 1 -- 172.21.15.76:0/1663344057 reap_dead start 2024-09-19T04:20:19.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1663344057 shutdown_connections 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1663344057 >> 172.21.15.76:0/1663344057 conn(0x7f03b40b1f90 msgr2=0x7f03b40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1663344057 shutdown_connections 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1663344057 wait complete. 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 Processor -- start 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 -- start start 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03b409df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b4159980 0x7f03b415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.784+0000 7f03bc6ac640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b4157b30 0x7f03b415c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03bc6ac640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03b416e770 con 0x7f03a00039f0 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03bc6ac640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03b416e5f0 con 0x7f03b4159980 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03bc6ac640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f03b416e8f0 con 0x7f03b4157b30 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03b409df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03b409df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51906/0 (socket says 172.21.15.76:51906) 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 -- 172.21.15.76:0/1764032105 learned_addr learned my addr 172.21.15.76:0/1764032105 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03bac49640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b4157b30 0x7f03b415c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03b9c47640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b4159980 0x7f03b415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 -- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b4157b30 msgr2=0x7f03b415c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b4157b30 0x7f03b415c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 -- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b4159980 msgr2=0x7f03b415bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b4159980 0x7f03b415bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 -- 172.21.15.76:0/1764032105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b41671b0 con 0x7f03a00039f0 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03b9c47640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b4159980 0x7f03b415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03ba448640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03b409df90 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f03b0001750 tx=0x7f03a0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03aaffd640 1 -- 172.21.15.76:0/1764032105 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0007a40 con 0x7f03a00039f0 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03aaffd640 1 -- 172.21.15.76:0/1764032105 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03b00bb070 con 0x7f03a00039f0 2024-09-19T04:20:19.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b4167490 con 0x7f03a00039f0 2024-09-19T04:20:19.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03aaffd640 1 -- 172.21.15.76:0/1764032105 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0000ee0 con 0x7f03a00039f0 2024-09-19T04:20:19.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b4158de0 con 0x7f03a00039f0 2024-09-19T04:20:19.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b40ad200 con 0x7f03a00039f0 2024-09-19T04:20:19.791 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03aaffd640 1 -- 172.21.15.76:0/1764032105 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f03b00b4040 con 0x7f03a00039f0 2024-09-19T04:20:19.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03aaffd640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0398067a60 0x7f0398069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:19.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03aaffd640 1 -- 172.21.15.76:0/1764032105 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f03b011fbe0 con 0x7f03a00039f0 2024-09-19T04:20:19.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03b9c47640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0398067a60 0x7f0398069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:19.792 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.788+0000 7f03b9c47640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0398067a60 0x7f0398069f20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f03b415cf60 tx=0x7f03a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:19.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.792+0000 7f03aaffd640 1 -- 172.21.15.76:0/1764032105 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f03b00160d0 con 0x7f03a00039f0 2024-09-19T04:20:19.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:19.996+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f03b4070080 con 0x7f03a00039f0 2024-09-19T04:20:19.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03aaffd640 1 -- 172.21.15.76:0/1764032105 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1928 (secure 0 0 0) 0x7f03b00f0710 con 0x7f03a00039f0 2024-09-19T04:20:19.999 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:20.000 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":438,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":694960,"bytes_used":267493376,"bytes_avail":772826619904,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":8003,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-09-19T04:20:17.870600+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:20:20.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0398067a60 msgr2=0x7f0398069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0398067a60 0x7f0398069f20 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f03b415cf60 tx=0x7f03a4008040 comp rx=0 tx=0).stop 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 msgr2=0x7f03b409df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03b409df90 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f03b0001750 tx=0x7f03a0005840 comp rx=0 tx=0).stop 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 shutdown_connections 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0398067a60 0x7f0398069f20 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03b4157b30 0x7f03b415c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03b4159980 0x7f03b415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 --2- 172.21.15.76:0/1764032105 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03a00039f0 0x7f03b409df90 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 >> 172.21.15.76:0/1764032105 conn(0x7f03b40b1f90 msgr2=0x7f03b40b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 shutdown_connections 2024-09-19T04:20:20.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:20.000+0000 7f03bc6ac640 1 -- 172.21.15.76:0/1764032105 wait complete. 2024-09-19T04:20:20.016 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:21.017 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:21.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.148+0000 7f3f5bc98640 1 Processor -- start 2024-09-19T04:20:21.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.148+0000 7f3f5bc98640 1 -- start start 2024-09-19T04:20:21.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f5bc98640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f3f54067e30 0x7f3f54161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f5bc98640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f541623e0 0x7f3f540b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f5bc98640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f540afbb0 con 0x7f3f5415e260 2024-09-19T04:20:21.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f5bc98640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f3f54162830 con 0x7f3f54067e30 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f5bc98640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3f540b0700 con 0x7f3f541623e0 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59a34640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f3f5415e260 0x7f3f540afff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51864/0 (socket says 172.21.15.76:51864) 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59a34640 1 -- 172.21.15.76:0/4137429825 learned_addr learned my addr 172.21.15.76:0/4137429825 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59233640 1 --2- 172.21.15.76:0/4137429825 >> v2:172.21.15.76:3300/0 conn(0x7f3f54067e30 0x7f3f54161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f5a235640 1 --2- 172.21.15.76:0/4137429825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f541623e0 0x7f3f540b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59233640 1 -- 172.21.15.76:0/4137429825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f541623e0 msgr2=0x7f3f540b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59233640 1 --2- 172.21.15.76:0/4137429825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f541623e0 0x7f3f540b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59233640 1 -- 172.21.15.76:0/4137429825 >> v1:172.21.15.76:6789/0 conn(0x7f3f5415e260 legacy=0x7f3f540afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59233640 1 -- 172.21.15.76:0/4137429825 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f540af310 con 0x7f3f54067e30 2024-09-19T04:20:21.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f5a235640 1 --2- 172.21.15.76:0/4137429825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f541623e0 0x7f3f540b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:21.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f59233640 1 --2- 172.21.15.76:0/4137429825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f54067e30 0x7f3f54161ea0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097c0 tx=0x7f3f4401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0ec3ea3e90c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f58a32640 1 -- 172.21.15.76:0/4137429825 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f440aa600 con 0x7f3f54067e30 2024-09-19T04:20:21.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f58a32640 1 -- 172.21.15.76:0/4137429825 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f440af070 con 0x7f3f54067e30 2024-09-19T04:20:21.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.152+0000 7f3f58a32640 1 -- 172.21.15.76:0/4137429825 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f440aa9c0 con 0x7f3f54067e30 2024-09-19T04:20:21.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- 172.21.15.76:0/4137429825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f54067e30 msgr2=0x7f3f54161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/4137429825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f54067e30 0x7f3f54161ea0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f3f440097c0 tx=0x7f3f4401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- 172.21.15.76:0/4137429825 shutdown_connections 2024-09-19T04:20:21.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/4137429825 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f541623e0 0x7f3f540b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/4137429825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f54067e30 0x7f3f54161ea0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- 172.21.15.76:0/4137429825 >> 172.21.15.76:0/4137429825 conn(0x7f3f54073580 msgr2=0x7f3f5406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- 172.21.15.76:0/4137429825 shutdown_connections 2024-09-19T04:20:21.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- 172.21.15.76:0/4137429825 wait complete. 2024-09-19T04:20:21.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 Processor -- start 2024-09-19T04:20:21.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- start start 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f5415e260 0x7f3f541596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f541623e0 0x7f3f54151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f54155a70 0x7f3f5406c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f3f540b5090 con 0x7f3f5415e260 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f3f540b4f10 con 0x7f3f541623e0 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f3f540b5210 con 0x7f3f54155a70 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f5415e260 0x7f3f541596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f5415e260 0x7f3f541596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51918/0 (socket says 172.21.15.76:51918) 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 -- 172.21.15.76:0/2129284025 learned_addr learned my addr 172.21.15.76:0/2129284025 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59233640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f541623e0 0x7f3f54151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 -- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f54155a70 msgr2=0x7f3f5406c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5a235640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f54155a70 0x7f3f5406c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f54155a70 0x7f3f5406c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 -- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f541623e0 msgr2=0x7f3f54151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f541623e0 0x7f3f54151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 -- 172.21.15.76:0/2129284025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f54155f50 con 0x7f3f5415e260 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59233640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f541623e0 0x7f3f54151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5a235640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f54155a70 0x7f3f5406c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f59a34640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f5415e260 0x7f3f541596e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000c420 tx=0x7f3f5000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f4a7fc640 1 -- 172.21.15.76:0/2129284025 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50017070 con 0x7f3f5415e260 2024-09-19T04:20:21.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f4a7fc640 1 -- 172.21.15.76:0/2129284025 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f50005cf0 con 0x7f3f5415e260 2024-09-19T04:20:21.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f4a7fc640 1 -- 172.21.15.76:0/2129284025 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f500072f0 con 0x7f3f5415e260 2024-09-19T04:20:21.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f54156230 con 0x7f3f5415e260 2024-09-19T04:20:21.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.156+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f541567f0 con 0x7f3f5415e260 2024-09-19T04:20:21.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.160+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f54161ea0 con 0x7f3f5415e260 2024-09-19T04:20:21.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.160+0000 7f3f4a7fc640 1 -- 172.21.15.76:0/2129284025 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f3f5001a070 con 0x7f3f5415e260 2024-09-19T04:20:21.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.160+0000 7f3f4a7fc640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3f380678c0 0x7f3f38069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.160+0000 7f3f4a7fc640 1 -- 172.21.15.76:0/2129284025 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f3f50085310 con 0x7f3f5415e260 2024-09-19T04:20:21.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.160+0000 7f3f59233640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3f380678c0 0x7f3f38069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.160+0000 7f3f59233640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3f380678c0 0x7f3f38069d80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f440ae000 tx=0x7f3f44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.164+0000 7f3f4a7fc640 1 -- 172.21.15.76:0/2129284025 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f3f50055bf0 con 0x7f3f5415e260 2024-09-19T04:20:21.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.360+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3f54159c20 con 0x7f3f5415e260 2024-09-19T04:20:21.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.360+0000 7f3f4a7fc640 1 -- 172.21.15.76:0/2129284025 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f3f500554f0 con 0x7f3f5415e260 2024-09-19T04:20:21.362 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:21.362 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:21.362 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3f380678c0 msgr2=0x7f3f38069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3f380678c0 0x7f3f38069d80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3f440ae000 tx=0x7f3f44008040 comp rx=0 tx=0).stop 2024-09-19T04:20:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f5415e260 msgr2=0x7f3f541596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f5415e260 0x7f3f541596e0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f3f5000c420 tx=0x7f3f5000c8e0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 shutdown_connections 2024-09-19T04:20:21.364 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f3f380678c0 0x7f3f38069d80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3f54155a70 0x7f3f5406c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f541623e0 0x7f3f54151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 --2- 172.21.15.76:0/2129284025 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3f5415e260 0x7f3f541596e0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 >> 172.21.15.76:0/2129284025 conn(0x7f3f54073580 msgr2=0x7f3f54153ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 shutdown_connections 2024-09-19T04:20:21.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.364+0000 7f3f5bc98640 1 -- 172.21.15.76:0/2129284025 wait complete. 2024-09-19T04:20:21.381 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:21.381 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:21.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.496+0000 7f9d69fb3640 1 Processor -- start 2024-09-19T04:20:21.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.496+0000 7f9d69fb3640 1 -- start start 2024-09-19T04:20:21.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.496+0000 7f9d69fb3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 0x7f9d64165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.496+0000 7f9d69fb3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d640aede0 con 0x7f9d641520a0 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.496+0000 7f9d69fb3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d6415fa10 con 0x7f9d64070020 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.496+0000 7f9d69fb3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d64152480 con 0x7f9d6415f3a0 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d637fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9d641520a0 0x7f9d6406f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51866/0 (socket says 172.21.15.76:51866) 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d637fe640 1 -- 172.21.15.76:0/3254566530 learned_addr learned my addr 172.21.15.76:0/3254566530 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 0x7f9d64165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1448199450 0 0) 0x7f9d640aede0 con 0x7f9d641520a0 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d4c003530 con 0x7f9d641520a0 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 533971741 0 0) 0x7f9d4c003530 con 0x7f9d641520a0 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 msgr2=0x7f9d64165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 0x7f9d64165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 >> v1:172.21.15.116:6789/0 conn(0x7f9d64070020 legacy=0x7f9d6415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d64074250 con 0x7f9d641520a0 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f9d54001680 con 0x7f9d641520a0 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 >> v1:172.21.15.76:6789/0 conn(0x7f9d641520a0 legacy=0x7f9d6406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c0039f0 0x7f9d4c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 0x7f9d4c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d4c004d50 0x7f9d4c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c00bab0 con 0x7f9d4c004d50 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d4c00bc60 con 0x7f9d4c0039f0 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d4c00be60 con 0x7f9d6415f3a0 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d4c004d50 0x7f9d4c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d62ffd640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 0x7f9d4c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d637fe640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c0039f0 0x7f9d4c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.502 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 -- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 msgr2=0x7f9d4c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d6415f3a0 0x7f9d4c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 -- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c0039f0 msgr2=0x7f9d4c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c0039f0 0x7f9d4c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 -- 172.21.15.76:0/3254566530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4c003530 con 0x7f9d4c004d50 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d4c004d50 0x7f9d4c0051d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009f90 tx=0x7f9d5c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5656cbff6238ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0b2070 con 0x7f9d4c004d50 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5c0a5c50 con 0x7f9d4c004d50 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.500+0000 7f9d627fc640 1 -- 172.21.15.76:0/3254566530 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c009c10 con 0x7f9d4c004d50 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- 172.21.15.76:0/3254566530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d4c004d50 msgr2=0x7f9d4c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 --2- 172.21.15.76:0/3254566530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d4c004d50 0x7f9d4c0051d0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c009f90 tx=0x7f9d5c01cc40 comp rx=0 tx=0).stop 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d63fff640 1 -- 172.21.15.76:0/3254566530 reap_dead start 2024-09-19T04:20:21.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- 172.21.15.76:0/3254566530 shutdown_connections 2024-09-19T04:20:21.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- 172.21.15.76:0/3254566530 >> 172.21.15.76:0/3254566530 conn(0x7f9d640b1f90 msgr2=0x7f9d640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- 172.21.15.76:0/3254566530 shutdown_connections 2024-09-19T04:20:21.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- 172.21.15.76:0/3254566530 wait complete. 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 Processor -- start 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- start start 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c004d50 0x7f9d64145530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6406e800 0x7f9d6406ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d641e5950 0x7f9d641e5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64073a50 con 0x7f9d6406e800 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64073770 con 0x7f9d4c004d50 2024-09-19T04:20:21.507 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d64073d30 con 0x7f9d641e5950 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6406e800 0x7f9d6406ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6406e800 0x7f9d6406ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51934/0 (socket says 172.21.15.76:51934) 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 -- 172.21.15.76:0/745703866 learned_addr learned my addr 172.21.15.76:0/745703866 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d637fe640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c004d50 0x7f9d64145530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d63fff640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d641e5950 0x7f9d641e5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 -- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d641e5950 msgr2=0x7f9d641e5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d641e5950 0x7f9d641e5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 -- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c004d50 msgr2=0x7f9d64145530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c004d50 0x7f9d64145530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 -- 172.21.15.76:0/745703866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d640ccd60 con 0x7f9d6406e800 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d62ffd640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6406e800 0x7f9d6406ec80 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800f820 tx=0x7f9d5800fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.508 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d4bfff640 1 -- 172.21.15.76:0/745703866 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580050d0 con 0x7f9d6406e800 2024-09-19T04:20:21.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d4bfff640 1 -- 172.21.15.76:0/745703866 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d58018070 con 0x7f9d6406e800 2024-09-19T04:20:21.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d640cd000 con 0x7f9d6406e800 2024-09-19T04:20:21.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d4bfff640 1 -- 172.21.15.76:0/745703866 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58005490 con 0x7f9d6406e800 2024-09-19T04:20:21.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.504+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d640cd5c0 con 0x7f9d6406e800 2024-09-19T04:20:21.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.508+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d64151250 con 0x7f9d6406e800 2024-09-19T04:20:21.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.508+0000 7f9d4bfff640 1 -- 172.21.15.76:0/745703866 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9d5800d730 con 0x7f9d6406e800 2024-09-19T04:20:21.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.508+0000 7f9d4bfff640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d50067a60 0x7f9d50069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.508+0000 7f9d4bfff640 1 -- 172.21.15.76:0/745703866 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f9d58085b90 con 0x7f9d6406e800 2024-09-19T04:20:21.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.508+0000 7f9d637fe640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d50067a60 0x7f9d50069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.510 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.508+0000 7f9d637fe640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d50067a60 0x7f9d50069f20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d54001060 tx=0x7f9d54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.513 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.512+0000 7f9d4bfff640 1 -- 172.21.15.76:0/745703866 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9d580565a0 con 0x7f9d6406e800 2024-09-19T04:20:21.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9d6415ec90 con 0x7f9d6406e800 2024-09-19T04:20:21.732 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d4bfff640 1 -- 172.21.15.76:0/745703866 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7f9d58055ea0 con 0x7f9d6406e800 2024-09-19T04:20:21.732 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:21.732 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":440,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":694960,"bytes_used":267493376,"bytes_avail":772826619904,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7266,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-19T04:20:19.871079+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-19T04:20:21.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d50067a60 msgr2=0x7f9d50069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d50067a60 0x7f9d50069f20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9d54001060 tx=0x7f9d54009040 comp rx=0 tx=0).stop 2024-09-19T04:20:21.734 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6406e800 msgr2=0x7f9d6406ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6406e800 0x7f9d6406ec80 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800f820 tx=0x7f9d5800fce0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 shutdown_connections 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d50067a60 0x7f9d50069f20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d641e5950 0x7f9d641e5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6406e800 0x7f9d6406ec80 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 --2- 172.21.15.76:0/745703866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d4c004d50 0x7f9d64145530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 >> 172.21.15.76:0/745703866 conn(0x7f9d640b1f90 msgr2=0x7f9d640afe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 shutdown_connections 2024-09-19T04:20:21.735 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.732+0000 7f9d69fb3640 1 -- 172.21.15.76:0/745703866 wait complete. 2024-09-19T04:20:21.751 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:21.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:21.751 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub abort 2024-09-19T04:20:21.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.864+0000 7f74e2b92640 1 Processor -- start 2024-09-19T04:20:21.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e2b92640 1 -- start start 2024-09-19T04:20:21.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e2b92640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f74dc073e60 0x7f74dc06e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e2b92640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74dc0b3d50 0x7f74dc0b4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e2b92640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f74dc0b4670 con 0x7f74dc06fef0 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e2b92640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f74dc06ee50 con 0x7f74dc0b3d50 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e2b92640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f74dc06f130 con 0x7f74dc073e60 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e112f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74dc0b3d50 0x7f74dc0b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e112f640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f74dc0b3d50 0x7f74dc0b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51940/0 (socket says 172.21.15.76:51940) 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e112f640 1 -- 172.21.15.76:0/230960454 learned_addr learned my addr 172.21.15.76:0/230960454 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e092e640 1 --2- 172.21.15.76:0/230960454 >> v2:172.21.15.116:3300/0 conn(0x7f74dc073e60 0x7f74dc06e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e112f640 1 -- 172.21.15.76:0/230960454 >> v2:172.21.15.116:3300/0 conn(0x7f74dc073e60 msgr2=0x7f74dc06e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e112f640 1 --2- 172.21.15.76:0/230960454 >> v2:172.21.15.116:3300/0 conn(0x7f74dc073e60 0x7f74dc06e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e112f640 1 -- 172.21.15.76:0/230960454 >> v1:172.21.15.116:6789/0 conn(0x7f74dc06fef0 legacy=0x7f74dc15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e112f640 1 -- 172.21.15.76:0/230960454 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc06f410 con 0x7f74dc0b3d50 2024-09-19T04:20:21.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.868+0000 7f74e092e640 1 --2- 172.21.15.76:0/230960454 >> v2:172.21.15.116:3300/0 conn(0x7f74dc073e60 0x7f74dc06e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:21.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e112f640 1 --2- 172.21.15.76:0/230960454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc0b3d50 0x7f74dc0b4130 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f74dc070340 tx=0x7f74d401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d07e5c74eded7c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74db7fe640 1 -- 172.21.15.76:0/230960454 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d40b0070 con 0x7f74dc0b3d50 2024-09-19T04:20:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74db7fe640 1 -- 172.21.15.76:0/230960454 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d40ab600 con 0x7f74dc0b3d50 2024-09-19T04:20:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74db7fe640 1 -- 172.21.15.76:0/230960454 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d40ab9d0 con 0x7f74dc0b3d50 2024-09-19T04:20:21.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- 172.21.15.76:0/230960454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc0b3d50 msgr2=0x7f74dc0b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 --2- 172.21.15.76:0/230960454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc0b3d50 0x7f74dc0b4130 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f74dc070340 tx=0x7f74d401c490 comp rx=0 tx=0).stop 2024-09-19T04:20:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- 172.21.15.76:0/230960454 shutdown_connections 2024-09-19T04:20:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 --2- 172.21.15.76:0/230960454 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc0b3d50 0x7f74dc0b4130 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 --2- 172.21.15.76:0/230960454 >> v2:172.21.15.116:3300/0 conn(0x7f74dc073e60 0x7f74dc06e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- 172.21.15.76:0/230960454 >> 172.21.15.76:0/230960454 conn(0x7f74dc0b1f90 msgr2=0x7f74dc0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- 172.21.15.76:0/230960454 shutdown_connections 2024-09-19T04:20:21.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- 172.21.15.76:0/230960454 wait complete. 2024-09-19T04:20:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 Processor -- start 2024-09-19T04:20:21.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- start start 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74dc06fef0 0x7f74dc151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc073e60 0x7f74dc073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74dc0b3d50 0x7f74dc099540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc0729a0 con 0x7f74dc073e60 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc072820 con 0x7f74dc06fef0 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.872+0000 7f74e2b92640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f74dc072b20 con 0x7f74dc0b3d50 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc073e60 0x7f74dc073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc073e60 0x7f74dc073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51950/0 (socket says 172.21.15.76:51950) 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 -- 172.21.15.76:0/2931753708 learned_addr learned my addr 172.21.15.76:0/2931753708 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74e092e640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74dc06fef0 0x7f74dc151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74e112f640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74dc0b3d50 0x7f74dc099540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 -- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74dc0b3d50 msgr2=0x7f74dc099540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74dc0b3d50 0x7f74dc099540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 -- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74dc06fef0 msgr2=0x7f74dc151180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74dc06fef0 0x7f74dc151180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 -- 172.21.15.76:0/2931753708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74dc09d070 con 0x7f74dc073e60 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74dbfff640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc073e60 0x7f74dc073140 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c390 tx=0x7f74d000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74d97fa640 1 -- 172.21.15.76:0/2931753708 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0017070 con 0x7f74dc073e60 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74d97fa640 1 -- 172.21.15.76:0/2931753708 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d00052d0 con 0x7f74dc073e60 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74d97fa640 1 -- 172.21.15.76:0/2931753708 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d0003820 con 0x7f74dc073e60 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74dc09d350 con 0x7f74dc073e60 2024-09-19T04:20:21.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74dc09d910 con 0x7f74dc073e60 2024-09-19T04:20:21.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74d97fa640 1 -- 172.21.15.76:0/2931753708 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f74d001a070 con 0x7f74dc073e60 2024-09-19T04:20:21.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74d97fa640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c80678c0 0x7f74c8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74d97fa640 1 -- 172.21.15.76:0/2931753708 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f74d0085170 con 0x7f74dc073e60 2024-09-19T04:20:21.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74e092e640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c80678c0 0x7f74c8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74e092e640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c80678c0 0x7f74c8069d80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009a80 tx=0x7f74cc003a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.876+0000 7f74e5c4e000 1 Processor -- start 2024-09-19T04:20:21.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- start start 2024-09-19T04:20:21.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9249ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.881 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea91fd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea91fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ea904ecc0 con 0x561ea91fd780 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ea9009a20 con 0x561ea9249c10 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561ea904ac80 con 0x561ea924a530 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea91fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea91fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51956/0 (socket says 172.21.15.76:51956) 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 -- 172.21.15.76:0/299204588 learned_addr learned my addr 172.21.15.76:0/299204588 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e092e640 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9249ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74dbfff640 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea91fd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 -- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 msgr2=0x561ea91fd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea91fd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 -- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 msgr2=0x561ea9249ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9249ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 -- 172.21.15.76:0/299204588 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea8ff1d40 con 0x561ea91fd780 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e112f640 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea91fdb60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f74d401a0c0 tx=0x7f74d40a6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56ff6b19d9de14a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74c27fc640 1 -- 172.21.15.76:0/299204588 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d40b0040 con 0x561ea91fd780 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74c27fc640 1 -- 172.21.15.76:0/299204588 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d40bb070 con 0x561ea91fd780 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74c27fc640 1 -- 172.21.15.76:0/299204588 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d40aba70 con 0x561ea91fd780 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- 172.21.15.76:0/299204588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 msgr2=0x561ea91fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea91fdb60 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f74d401a0c0 tx=0x7f74d40a6000 comp rx=0 tx=0).stop 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- 172.21.15.76:0/299204588 shutdown_connections 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea91fdb60 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea91fd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/299204588 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9249ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- 172.21.15.76:0/299204588 >> 172.21.15.76:0/299204588 conn(0x561ea922a9b0 msgr2=0x561ea922cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- 172.21.15.76:0/299204588 shutdown_connections 2024-09-19T04:20:21.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- 172.21.15.76:0/299204588 wait complete. 2024-09-19T04:20:21.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- start start 2024-09-19T04:20:21.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 Processor -- start 2024-09-19T04:20:21.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea9250d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92512d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea9251810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ea8fe82c0 con 0x561ea91fd780 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ea900d370 con 0x561ea924a530 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561ea905cf50 con 0x561ea9249c10 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e5c4e000 10 client.0 _ll_register_callbacks cb 0x561ea90afdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e092e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea9250d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e092e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea9250d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51968/0 (socket says 172.21.15.76:51968) 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.880+0000 7f74e092e640 1 -- 172.21.15.76:0/1062108191 learned_addr learned my addr 172.21.15.76:0/1062108191 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e112f640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea9251810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74dbfff640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e092e640 1 -- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 msgr2=0x561ea92512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e092e640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e092e640 1 -- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 msgr2=0x561ea9251810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e092e640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea9251810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e092e640 1 -- 172.21.15.76:0/1062108191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea8fee770 con 0x561ea91fd780 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e112f640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea9251810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e092e640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea9250d90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f74cc016a00 tx=0x7f74cc004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b921e3b1db52b28b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1062108191 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc004870 con 0x561ea91fd780 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1062108191 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f74cc014b60 con 0x561ea91fd780 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74d40bb070 con 0x561ea91fd780 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1062108191 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f74cc014d60 con 0x561ea91fd780 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 10 client.8631 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:21.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x561ea900c3f0 con 0x561ea91fd780 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1062108191 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d40bb070 con 0x561ea91fd780 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1062108191 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f74cc022420 con 0x561ea91fd780 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 10 client.8631 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 10 client.8631 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 20 client.8631 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea9259160 0x561ea925b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 4 client.8631 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ea8fe8d90 con 0x561ea9259160 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74dbfff640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea9259160 0x561ea925b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74dbfff640 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea9259160 0x561ea925b550 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91ea7d66abf61458 server_cookie=6cbc6ec85b2a573d in_seq=0 out_seq=0 2024-09-19T04:20:21.890 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 10 client.8631 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1062108191 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ea8fe8d90 con 0x561ea9259160 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 10 client.8631 handle_command_reply: tid=0 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 client.8631 shutdown 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 msgr2=0x561ea9250d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea9250d90 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f74cc016a00 tx=0x7f74cc004ac0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 shutdown_connections 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea9259160 0x561ea925b550 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea9251810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74bb7fe640 0 client.8631 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1062108191 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea9250d90 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 >> 172.21.15.76:0/1062108191 conn(0x561ea922a9b0 msgr2=0x561ea91fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.891 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 shutdown_connections 2024-09-19T04:20:21.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1062108191 wait complete. 2024-09-19T04:20:21.892 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.884+0000 7f74e5c4e000 20 client.8631 trim_cache size 0 max 16384 2024-09-19T04:20:21.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 Processor -- start 2024-09-19T04:20:21.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 -- start start 2024-09-19T04:20:21.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea925ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9256970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea9256eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc200fb0 con 0x561ea91fd780 2024-09-19T04:20:21.904 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f74dc2016e0 con 0x561ea9249c10 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.900+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f74dc152cd0 con 0x561ea924a530 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea925ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea925ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51974/0 (socket says 172.21.15.76:51974) 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/3339080130 learned_addr learned my addr 172.21.15.76:0/3339080130 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e112f640 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea9256eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 msgr2=0x561ea9256eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74dbfff640 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9256970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea9256eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 msgr2=0x561ea9256970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9256970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/3339080130 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ea9205d90 con 0x561ea91fd780 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea925ebb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f74cc014fb0 tx=0x7f74cc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653036026b949443 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74c1ffb640 1 -- 172.21.15.76:0/3339080130 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc014b60 con 0x561ea91fd780 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74c1ffb640 1 -- 172.21.15.76:0/3339080130 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74cc039070 con 0x561ea91fd780 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74c1ffb640 1 -- 172.21.15.76:0/3339080130 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc022420 con 0x561ea91fd780 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- 172.21.15.76:0/3339080130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 msgr2=0x561ea925ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea925ebb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f74cc014fb0 tx=0x7f74cc020000 comp rx=0 tx=0).stop 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- 172.21.15.76:0/3339080130 shutdown_connections 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea924a530 0x561ea9256eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea9249c10 0x561ea9256970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/3339080130 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea925ebb0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- 172.21.15.76:0/3339080130 >> 172.21.15.76:0/3339080130 conn(0x561ea922a9b0 msgr2=0x561ea91fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- 172.21.15.76:0/3339080130 shutdown_connections 2024-09-19T04:20:21.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- 172.21.15.76:0/3339080130 wait complete. 2024-09-19T04:20:21.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- start start 2024-09-19T04:20:21.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 Processor -- start 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea92bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92be060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea925d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x561ea8ff6120 con 0x561ea91fd780 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x561ea90cfe90 con 0x561ea924a530 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x561ea8ff8b90 con 0x561ea9249c10 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 10 client.0 _ll_register_callbacks cb 0x7f74cc016650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea92bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea92bdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51980/0 (socket says 172.21.15.76:51980) 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/1931072147 learned_addr learned my addr 172.21.15.76:0/1931072147 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e112f640 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea925d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74dbfff640 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92be060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 msgr2=0x561ea92be060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92be060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 msgr2=0x561ea925d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea925d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 -- 172.21.15.76:0/1931072147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ea90d0320 con 0x561ea91fd780 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e092e640 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea92bdb20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f74cc036040 tx=0x7f74cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce63a65aa707960 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1931072147 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc022b70 con 0x561ea91fd780 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1931072147 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f74cc022d10 con 0x561ea91fd780 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74cc039070 con 0x561ea91fd780 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1931072147 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f74cc034460 con 0x561ea91fd780 2024-09-19T04:20:21.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 10 client.8637 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x561ea8fe25c0 con 0x561ea91fd780 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.904+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1931072147 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74cc039070 con 0x561ea91fd780 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1931072147 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f74cc034600 con 0x561ea91fd780 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 10 client.8637 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 10 client.8637 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 20 client.8637 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea925ddd0 0x561ea925e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 4 client.8637 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub abort"}) -- 0x561ea905c3f0 con 0x561ea925ddd0 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74dbfff640 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea925ddd0 0x561ea925e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74bb7fe640 10 client.8637 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:21.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74dbfff640 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea925ddd0 0x561ea925e1e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=110ec7337e354289 server_cookie=eb70a960b5411b08 in_seq=0 out_seq=0 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74bb7fe640 1 -- 172.21.15.76:0/1931072147 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ea905c3f0 con 0x561ea925ddd0 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74bb7fe640 10 client.8637 handle_command_reply: tid=0 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 client.8637 shutdown 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 msgr2=0x561ea92bdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea92bdb20 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f74cc036040 tx=0x7f74cc014000 comp rx=0 tx=0).stop 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 shutdown_connections 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x561ea925ddd0 0x561ea925e1e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561ea924a530 0x561ea925d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x561ea9249c10 0x561ea92be060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74bb7fe640 0 client.8637 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 --2- 172.21.15.76:0/1931072147 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x561ea91fd780 0x561ea92bdb20 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 >> 172.21.15.76:0/1931072147 conn(0x561ea922a9b0 msgr2=0x561ea91fba10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 shutdown_connections 2024-09-19T04:20:21.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 1 -- 172.21.15.76:0/1931072147 wait complete. 2024-09-19T04:20:21.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e5c4e000 20 client.8637 trim_cache size 0 max 16384 2024-09-19T04:20:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "message": "", 2024-09-19T04:20:21.915 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:20:21.915 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:21.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c80678c0 msgr2=0x7f74c8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c80678c0 0x7f74c8069d80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009a80 tx=0x7f74cc003a00 comp rx=0 tx=0).stop 2024-09-19T04:20:21.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc073e60 msgr2=0x7f74dc073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:21.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc073e60 0x7f74dc073140 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f74d000c390 tx=0x7f74d000c850 comp rx=0 tx=0).stop 2024-09-19T04:20:21.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 shutdown_connections 2024-09-19T04:20:21.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f74c80678c0 0x7f74c8069d80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f74dc0b3d50 0x7f74dc099540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f74dc073e60 0x7f74dc073140 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 --2- 172.21.15.76:0/2931753708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74dc06fef0 0x7f74dc151180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:21.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 >> 172.21.15.76:0/2931753708 conn(0x7f74dc0b1f90 msgr2=0x7f74dc15d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:21.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 shutdown_connections 2024-09-19T04:20:21.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:21.908+0000 7f74e2b92640 1 -- 172.21.15.76:0/2931753708 wait complete. 2024-09-19T04:20:21.923 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:21.923 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-19T04:20:22.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f04040a9640 1 Processor -- start 2024-09-19T04:20:22.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f04040a9640 1 -- start start 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f04040a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc161a60 0x7f03fc0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f04040a9640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03fc0d2e90 0x7f03fc0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f04040a9640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03fc0d32b0 con 0x7f03fc0cf450 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f04040a9640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f03fc0d0300 con 0x7f03fc0d2e90 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f04040a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc06e6c0 con 0x7f03fc161a60 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0401e45640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc161a60 0x7f03fc0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0402646640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f03fc0cf450 0x7f03fc0d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:54868/0 (socket says 172.21.15.76:54868) 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0401e45640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc161a60 0x7f03fc0d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:47696/0 (socket says 172.21.15.76:47696) 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0401644640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f03fc0d2e90 0x7f03fc0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0402646640 1 -- 172.21.15.76:0/3798540644 learned_addr learned my addr 172.21.15.76:0/3798540644 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:22.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0401e45640 1 -- 172.21.15.76:0/3798540644 >> v2:172.21.15.116:3300/0 conn(0x7f03fc0d2e90 msgr2=0x7f03fc0cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0401e45640 1 --2- 172.21.15.76:0/3798540644 >> v2:172.21.15.116:3300/0 conn(0x7f03fc0d2e90 0x7f03fc0cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0401e45640 1 -- 172.21.15.76:0/3798540644 >> v1:172.21.15.116:6789/0 conn(0x7f03fc0cf450 legacy=0x7f03fc0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0401e45640 1 -- 172.21.15.76:0/3798540644 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc0b0f60 con 0x7f03fc161a60 2024-09-19T04:20:22.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.032+0000 7f0400e43640 1 -- 172.21.15.76:0/3798540644 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1338504481 0 0) 0x7f03fc0d32b0 con 0x7f03fc0cf450 2024-09-19T04:20:22.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f0401e45640 1 --2- 172.21.15.76:0/3798540644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc161a60 0x7f03fc0d2950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0097c0 tx=0x7f03ec01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=603747b299acbc09 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f0400e43640 1 -- 172.21.15.76:0/3798540644 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0b1070 con 0x7f03fc161a60 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f0400e43640 1 -- 172.21.15.76:0/3798540644 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec0a8050 con 0x7f03fc161a60 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f0400e43640 1 -- 172.21.15.76:0/3798540644 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0d1930 con 0x7f03fc161a60 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 -- 172.21.15.76:0/3798540644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc161a60 msgr2=0x7f03fc0d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 --2- 172.21.15.76:0/3798540644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc161a60 0x7f03fc0d2950 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0097c0 tx=0x7f03ec01c3e0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 -- 172.21.15.76:0/3798540644 shutdown_connections 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 --2- 172.21.15.76:0/3798540644 >> v2:172.21.15.116:3300/0 conn(0x7f03fc0d2e90 0x7f03fc0cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 --2- 172.21.15.76:0/3798540644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc161a60 0x7f03fc0d2950 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 -- 172.21.15.76:0/3798540644 >> 172.21.15.76:0/3798540644 conn(0x7f03fc08bf90 msgr2=0x7f03fc09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 -- 172.21.15.76:0/3798540644 shutdown_connections 2024-09-19T04:20:22.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 -- 172.21.15.76:0/3798540644 wait complete. 2024-09-19T04:20:22.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 Processor -- start 2024-09-19T04:20:22.038 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.036+0000 7f04040a9640 1 -- start start 2024-09-19T04:20:22.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc0cf450 0x7f03fc0b9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc0d2e90 0x7f03fc14f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc151c20 0x7f03fc1520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc0cf8c0 con 0x7f03fc0cf450 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc06d190 con 0x7f03fc0d2e90 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc0cfa40 con 0x7f03fc151c20 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc0cf450 0x7f03fc0b9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc0cf450 0x7f03fc0b9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51982/0 (socket says 172.21.15.76:51982) 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 -- 172.21.15.76:0/518150342 learned_addr learned my addr 172.21.15.76:0/518150342 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401644640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc0d2e90 0x7f03fc14f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0402646640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc151c20 0x7f03fc1520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 -- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc151c20 msgr2=0x7f03fc1520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc151c20 0x7f03fc1520d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 -- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc0d2e90 msgr2=0x7f03fc14f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc0d2e90 0x7f03fc14f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 -- 172.21.15.76:0/518150342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc158720 con 0x7f03fc0cf450 2024-09-19T04:20:22.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401e45640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc0cf450 0x7f03fc0b9e10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0af040 tx=0x7f03ec01ac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f03fa7fc640 1 -- 172.21.15.76:0/518150342 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0b1050 con 0x7f03fc0cf450 2024-09-19T04:20:22.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f03fa7fc640 1 -- 172.21.15.76:0/518150342 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec0bd070 con 0x7f03fc0cf450 2024-09-19T04:20:22.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03fc158a00 con 0x7f03fc0cf450 2024-09-19T04:20:22.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f03fa7fc640 1 -- 172.21.15.76:0/518150342 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec006e20 con 0x7f03fc0cf450 2024-09-19T04:20:22.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03fc158fc0 con 0x7f03fc0cf450 2024-09-19T04:20:22.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f03fa7fc640 1 -- 172.21.15.76:0/518150342 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f03ec0b8890 con 0x7f03fc0cf450 2024-09-19T04:20:22.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f03fa7fc640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03e80678f0 0x7f03e8069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401644640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03e80678f0 0x7f03e8069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f03fa7fc640 1 -- 172.21.15.76:0/518150342 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f03ec120590 con 0x7f03fc0cf450 2024-09-19T04:20:22.045 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.040+0000 7f0401644640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03e80678f0 0x7f03e8069db0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f03fc1fa580 tx=0x7f03f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 Processor -- start 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- start start 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 0x55dbb2cb8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2c6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2c6c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dbb2b55960 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dbb2a99900 con 0x55dbb2cb8690 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55dbb2abcd40 con 0x55dbb2cb7d70 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2c6c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2c6c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51996/0 (socket says 172.21.15.76:51996) 2024-09-19T04:20:22.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 -- 172.21.15.76:0/2866672095 learned_addr learned my addr 172.21.15.76:0/2866672095 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 -- 172.21.15.76:0/2866672095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 msgr2=0x55dbb2cb8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 0x55dbb2cb8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 -- 172.21.15.76:0/2866672095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 msgr2=0x55dbb2c6b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401644640 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2c6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2c6b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 -- 172.21.15.76:0/2866672095 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb2a63580 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2c6c190 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f03f40072e0 tx=0x7f03f40077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d79b655cd8e454 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f03e37fe640 1 -- 172.21.15.76:0/2866672095 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4019070 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f03e37fe640 1 -- 172.21.15.76:0/2866672095 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4004d80 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f03e37fe640 1 -- 172.21.15.76:0/2866672095 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f4001e60 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- 172.21.15.76:0/2866672095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 msgr2=0x55dbb2c6c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2c6c190 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f03f40072e0 tx=0x7f03f40077a0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.047 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- 172.21.15.76:0/2866672095 shutdown_connections 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2c6c190 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2c6b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- 172.21.15.76:0/2866672095 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 0x55dbb2cb8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- 172.21.15.76:0/2866672095 >> 172.21.15.76:0/2866672095 conn(0x55dbb2c99040 msgr2=0x55dbb2c9b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- 172.21.15.76:0/2866672095 shutdown_connections 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- 172.21.15.76:0/2866672095 wait complete. 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- start start 2024-09-19T04:20:22.048 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 Processor -- start 2024-09-19T04:20:22.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cbeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.051 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb7d70 0x55dbb2cbf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb8690 0x55dbb2cbf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dbb2ab8d00 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55dbb2a77aa0 con 0x55dbb2cb7d70 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55dbb2a56340 con 0x55dbb2cb8690 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0407165000 10 client.0 _ll_register_callbacks cb 0x55dbb2c59760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cbeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cbeef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52012/0 (socket says 172.21.15.76:52012) 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 -- 172.21.15.76:0/1938580807 learned_addr learned my addr 172.21.15.76:0/1938580807 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401644640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb7d70 0x55dbb2cbf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 -- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb8690 msgr2=0x55dbb2cbf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0402646640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb8690 0x55dbb2cbf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.052 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb8690 0x55dbb2cbf970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 -- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb7d70 msgr2=0x55dbb2cbf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb7d70 0x55dbb2cbf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401e45640 1 -- 172.21.15.76:0/1938580807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb2a5fdc0 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.044+0000 7f0401644640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb7d70 0x55dbb2cbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0401e45640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cbeef0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0f0780 tx=0x7f03ec0ace40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ccf000a210684a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 1 -- 172.21.15.76:0/1938580807 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0b5e10 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03f4004d80 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 1 -- 172.21.15.76:0/1938580807 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f03ec0c3cd0 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 1 -- 172.21.15.76:0/1938580807 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f03ec0c3ed0 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.053 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 10 client.8646 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55dbb2a5c7f0 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 1 -- 172.21.15.76:0/1938580807 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03f4004d80 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 1 -- 172.21.15.76:0/1938580807 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f03ec0b8b80 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 10 client.8646 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 10 client.8646 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 20 client.8646 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc72c0 0x55dbb2cc96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 4 client.8646 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbb2a56e10 con 0x55dbb2cc72c0 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0401644640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc72c0 0x55dbb2cc96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.054 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 10 client.8646 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0401644640 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc72c0 0x55dbb2cc96b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b91aac742844f0 server_cookie=73d3fe13f64d6cd6 in_seq=0 out_seq=0 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 1 -- 172.21.15.76:0/1938580807 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dbb2a56e10 con 0x55dbb2cc72c0 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 10 client.8646 handle_command_reply: tid=0 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 client.8646 shutdown 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 msgr2=0x55dbb2cbeef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cbeef0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0f0780 tx=0x7f03ec0ace40 comp rx=0 tx=0).stop 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 shutdown_connections 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc72c0 0x55dbb2cc96b0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb8690 0x55dbb2cbf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb7d70 0x55dbb2cbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f03e0ff9640 0 client.8646 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 --2- 172.21.15.76:0/1938580807 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cbeef0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 >> 172.21.15.76:0/1938580807 conn(0x55dbb2c99040 msgr2=0x55dbb2c6af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 shutdown_connections 2024-09-19T04:20:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 1 -- 172.21.15.76:0/1938580807 wait complete. 2024-09-19T04:20:22.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.048+0000 7f0407165000 20 client.8646 trim_cache size 0 max 16384 2024-09-19T04:20:22.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.064+0000 7f0407165000 1 Processor -- start 2024-09-19T04:20:22.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.064+0000 7f0407165000 1 -- start start 2024-09-19T04:20:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cc5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 0x55dbb2cbf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.068 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2cbf6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc204090 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc203230 con 0x55dbb2cb8690 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc203960 con 0x55dbb2cb7d70 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cc5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cc5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52020/0 (socket says 172.21.15.76:52020) 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 -- 172.21.15.76:0/1638813243 learned_addr learned my addr 172.21.15.76:0/1638813243 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 -- 172.21.15.76:0/1638813243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 msgr2=0x55dbb2cbf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 --2- 172.21.15.76:0/1638813243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 0x55dbb2cbf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 -- 172.21.15.76:0/1638813243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 msgr2=0x55dbb2cbf6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 --2- 172.21.15.76:0/1638813243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2cbf6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.069 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 -- 172.21.15.76:0/1638813243 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbb2c71960 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 --2- 172.21.15.76:0/1638813243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cc5570 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006aa0 tx=0x7f03ec0b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a30dba57286810a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e2ffd640 1 -- 172.21.15.76:0/1638813243 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0f4bf0 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e2ffd640 1 -- 172.21.15.76:0/1638813243 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec0a8060 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e2ffd640 1 -- 172.21.15.76:0/1638813243 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ec0eca90 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/1638813243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 msgr2=0x55dbb2cc5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- 172.21.15.76:0/1638813243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cc5570 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f03ec006aa0 tx=0x7f03ec0b9000 comp rx=0 tx=0).stop 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/1638813243 shutdown_connections 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- 172.21.15.76:0/1638813243 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2cbf6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- 172.21.15.76:0/1638813243 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2cb7d70 0x55dbb2cbf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- 172.21.15.76:0/1638813243 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2c6bdb0 0x55dbb2cc5570 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/1638813243 >> 172.21.15.76:0/1638813243 conn(0x55dbb2c99040 msgr2=0x55dbb2cbeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.070 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/1638813243 shutdown_connections 2024-09-19T04:20:22.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/1638813243 wait complete. 2024-09-19T04:20:22.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- start start 2024-09-19T04:20:22.071 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 Processor -- start 2024-09-19T04:20:22.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2c6bdb0 0x55dbb2d344d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.074 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2cb7d70 0x55dbb2d34a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2cc4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55dbb2a62f20 con 0x55dbb2cb7d70 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc160db0 con 0x55dbb2cb8690 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55dbb2a84400 con 0x55dbb2c6bdb0 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 10 client.0 _ll_register_callbacks cb 0x7f03ec0069d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2cb7d70 0x55dbb2d34a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2cb7d70 0x55dbb2d34a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52022/0 (socket says 172.21.15.76:52022) 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 -- 172.21.15.76:0/3637954097 learned_addr learned my addr 172.21.15.76:0/3637954097 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 -- 172.21.15.76:0/3637954097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2c6bdb0 msgr2=0x55dbb2d344d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2c6bdb0 0x55dbb2d344d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 -- 172.21.15.76:0/3637954097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 msgr2=0x55dbb2cc4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2cc4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.075 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 -- 172.21.15.76:0/3637954097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbb2a7a470 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401644640 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2cb7d70 0x55dbb2d34a10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f03f0011bb0 tx=0x7f03f00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771310daad24b2b1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e0ff9640 1 -- 172.21.15.76:0/3637954097 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0031060 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e0ff9640 1 -- 172.21.15.76:0/3637954097 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f03f002c480 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ec0a8060 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e0ff9640 1 -- 172.21.15.76:0/3637954097 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f03f0037050 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 10 client.8652 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55dbb2a50640 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e0ff9640 1 -- 172.21.15.76:0/3637954097 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03ec0a8060 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f03e0ff9640 1 -- 172.21.15.76:0/3637954097 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f03f003a020 con 0x55dbb2cb7d70 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 10 client.8652 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 10 client.8652 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:22.076 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 20 client.8652 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc47a0 0x55dbb2cc4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 4 client.8652 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dbb2aca470 con 0x55dbb2cc47a0 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.068+0000 7f0401e45640 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc47a0 0x55dbb2cc4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f03e0ff9640 10 client.8652 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0401e45640 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc47a0 0x55dbb2cc4bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3ca55c2a25f9743 server_cookie=c704331c2d6176b9 in_seq=0 out_seq=0 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f03e0ff9640 1 -- 172.21.15.76:0/3637954097 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55dbb2aca470 con 0x55dbb2cc47a0 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f03e0ff9640 10 client.8652 handle_command_reply: tid=0 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 client.8652 shutdown 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2cb7d70 msgr2=0x55dbb2d34a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2cb7d70 0x55dbb2d34a10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f03f0011bb0 tx=0x7f03f00046d0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 shutdown_connections 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55dbb2cc47a0 0x55dbb2cc4bb0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.077 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55dbb2cb8690 0x55dbb2cc4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55dbb2cb7d70 0x55dbb2d34a10 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 --2- 172.21.15.76:0/3637954097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55dbb2c6bdb0 0x55dbb2d344d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 >> 172.21.15.76:0/3637954097 conn(0x55dbb2c99040 msgr2=0x55dbb2c73470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f03e0ff9640 0 client.8652 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 shutdown_connections 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 1 -- 172.21.15.76:0/3637954097 wait complete. 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f0407165000 20 client.8652 trim_cache size 0 max 16384 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:22.078 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:22.079 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:22.079 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:22.079 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:22.079 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:22.079 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.079 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.079 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03e80678f0 msgr2=0x7f03e8069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03e80678f0 0x7f03e8069db0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f03fc1fa580 tx=0x7f03f0008040 comp rx=0 tx=0).stop 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc0cf450 msgr2=0x7f03fc0b9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc0cf450 0x7f03fc0b9e10 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f03ec0af040 tx=0x7f03ec01ac20 comp rx=0 tx=0).stop 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 shutdown_connections 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03e80678f0 0x7f03e8069db0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc151c20 0x7f03fc1520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc0d2e90 0x7f03fc14f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 --2- 172.21.15.76:0/518150342 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc0cf450 0x7f03fc0b9e10 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 >> 172.21.15.76:0/518150342 conn(0x7f03fc08bf90 msgr2=0x7f03fc0d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 shutdown_connections 2024-09-19T04:20:22.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.072+0000 7f04040a9640 1 -- 172.21.15.76:0/518150342 wait complete. 2024-09-19T04:20:22.087 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:22.087 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:22.087 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-19T04:20:22.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.192+0000 7f7a6be17640 1 Processor -- start 2024-09-19T04:20:22.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.192+0000 7f7a6be17640 1 -- start start 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a641520a0 0x7f7a6406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a6406fe50 0x7f7a6415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a64070250 con 0x7f7a6415f1d0 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7a64152480 con 0x7f7a6406fe50 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7a640aede0 con 0x7f7a641520a0 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a693b2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f7a6406fe50 0x7f7a6415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6a3b4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7a6415f1d0 0x7f7a64071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51882/0 (socket says 172.21.15.76:51882) 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6a3b4640 1 -- 172.21.15.76:0/3931478645 learned_addr learned my addr 172.21.15.76:0/3931478645 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/3931478645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a641520a0 0x7f7a6406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a693b2640 1 -- 172.21.15.76:0/3931478645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a641520a0 msgr2=0x7f7a6406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a693b2640 1 --2- 172.21.15.76:0/3931478645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a641520a0 0x7f7a6406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a693b2640 1 -- 172.21.15.76:0/3931478645 >> v1:172.21.15.76:6789/0 conn(0x7f7a6415f1d0 legacy=0x7f7a64071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a693b2640 1 -- 172.21.15.76:0/3931478645 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6415fa10 con 0x7f7a6406fe50 2024-09-19T04:20:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a68bb1640 1 -- 172.21.15.76:0/3931478645 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3402171992 0 0) 0x7f7a64070250 con 0x7f7a6415f1d0 2024-09-19T04:20:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a693b2640 1 --2- 172.21.15.76:0/3931478645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6406fe50 0x7f7a6415ec90 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009d90 tx=0x7f7a6001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3479ae396f54823f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a68bb1640 1 -- 172.21.15.76:0/3931478645 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600b2070 con 0x7f7a6406fe50 2024-09-19T04:20:22.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a68bb1640 1 -- 172.21.15.76:0/3931478645 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600a5930 con 0x7f7a6406fe50 2024-09-19T04:20:22.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a68bb1640 1 -- 172.21.15.76:0/3931478645 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600a5d00 con 0x7f7a6406fe50 2024-09-19T04:20:22.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- 172.21.15.76:0/3931478645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6406fe50 msgr2=0x7f7a6415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 --2- 172.21.15.76:0/3931478645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6406fe50 0x7f7a6415ec90 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009d90 tx=0x7f7a6001c950 comp rx=0 tx=0).stop 2024-09-19T04:20:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- 172.21.15.76:0/3931478645 shutdown_connections 2024-09-19T04:20:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 --2- 172.21.15.76:0/3931478645 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6406fe50 0x7f7a6415ec90 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 --2- 172.21.15.76:0/3931478645 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a641520a0 0x7f7a6406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- 172.21.15.76:0/3931478645 >> 172.21.15.76:0/3931478645 conn(0x7f7a640b1f90 msgr2=0x7f7a640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- 172.21.15.76:0/3931478645 shutdown_connections 2024-09-19T04:20:22.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.196+0000 7f7a6be17640 1 -- 172.21.15.76:0/3931478645 wait complete. 2024-09-19T04:20:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 Processor -- start 2024-09-19T04:20:22.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 -- start start 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6406fe50 0x7f7a6409df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a641520a0 0x7f7a6414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6415f1d0 0x7f7a6414a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64073d30 con 0x7f7a6415f1d0 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64073a50 con 0x7f7a641520a0 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7a6415f640 con 0x7f7a6406fe50 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6415f1d0 0x7f7a6414a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6415f1d0 0x7f7a6414a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52042/0 (socket says 172.21.15.76:52042) 2024-09-19T04:20:22.202 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 -- 172.21.15.76:0/361706058 learned_addr learned my addr 172.21.15.76:0/361706058 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a693b2640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a641520a0 0x7f7a6414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 -- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6406fe50 msgr2=0x7f7a6409df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6406fe50 0x7f7a6409df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 -- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a641520a0 msgr2=0x7f7a6414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a641520a0 0x7f7a6414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 -- 172.21.15.76:0/361706058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a64151d20 con 0x7f7a6415f1d0 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a693b2640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a641520a0 0x7f7a6414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6a3b4640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6415f1d0 0x7f7a6414a340 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00c4b0 tx=0x7f7a5c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a567fc640 1 -- 172.21.15.76:0/361706058 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c018070 con 0x7f7a6415f1d0 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a567fc640 1 -- 172.21.15.76:0/361706058 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a5c0074e0 con 0x7f7a6415f1d0 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a567fc640 1 -- 172.21.15.76:0/361706058 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c004a30 con 0x7f7a6415f1d0 2024-09-19T04:20:22.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a640b0a60 con 0x7f7a6415f1d0 2024-09-19T04:20:22.204 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.200+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6414c0d0 con 0x7f7a6415f1d0 2024-09-19T04:20:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a567fc640 1 -- 172.21.15.76:0/361706058 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a5c01b070 con 0x7f7a6415f1d0 2024-09-19T04:20:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a567fc640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a480678c0 0x7f7a48069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a480678c0 0x7f7a48069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a567fc640 1 -- 172.21.15.76:0/361706058 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f7a5c0850e0 con 0x7f7a6415f1d0 2024-09-19T04:20:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a480678c0 0x7f7a48069d80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098f0 tx=0x7f7a58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 Processor -- start 2024-09-19T04:20:22.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 -- start start 2024-09-19T04:20:22.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 0x5627d6f96150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f96690 0x5627d6f49870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 0x5627d6f4a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5627d6e33960 con 0x5627d6f96690 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5627d6d77900 con 0x5627d6f95d70 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5627d6d9ad40 con 0x5627d6f49db0 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f96690 0x5627d6f49870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f96690 0x5627d6f49870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52052/0 (socket says 172.21.15.76:52052) 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 -- 172.21.15.76:0/3176105304 learned_addr learned my addr 172.21.15.76:0/3176105304 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 -- 172.21.15.76:0/3176105304 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 msgr2=0x5627d6f4a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 --2- 172.21.15.76:0/3176105304 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 0x5627d6f4a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 -- 172.21.15.76:0/3176105304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 msgr2=0x5627d6f96150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 --2- 172.21.15.76:0/3176105304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 0x5627d6f96150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 -- 172.21.15.76:0/3176105304 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d6d41580 con 0x5627d6f96690 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a693b2640 1 --2- 172.21.15.76:0/3176105304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f96690 0x5627d6f49870 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7a6001c990 tx=0x7f7a60004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44a5c310aae6ee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a477fe640 1 -- 172.21.15.76:0/3176105304 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600b2040 con 0x5627d6f96690 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a477fe640 1 -- 172.21.15.76:0/3176105304 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600a58b0 con 0x5627d6f96690 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a477fe640 1 -- 172.21.15.76:0/3176105304 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600a5ca0 con 0x5627d6f96690 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 -- 172.21.15.76:0/3176105304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f96690 msgr2=0x5627d6f49870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/3176105304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f96690 0x5627d6f49870 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f7a6001c990 tx=0x7f7a60004ac0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 -- 172.21.15.76:0/3176105304 shutdown_connections 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/3176105304 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 0x5627d6f4a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.204+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/3176105304 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f96690 0x5627d6f49870 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/3176105304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 0x5627d6f96150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- 172.21.15.76:0/3176105304 >> 172.21.15.76:0/3176105304 conn(0x5627d6f77040 msgr2=0x5627d6f79430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- 172.21.15.76:0/3176105304 shutdown_connections 2024-09-19T04:20:22.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- 172.21.15.76:0/3176105304 wait complete. 2024-09-19T04:20:22.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- start start 2024-09-19T04:20:22.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 Processor -- start 2024-09-19T04:20:22.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f49db0 0x5627d6f515a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 0x5627d6f51ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 0x5627d6f77420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5627d6d96d00 con 0x5627d6f49db0 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5627d6d55aa0 con 0x5627d6f95d70 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5627d6d34340 con 0x5627d6f96690 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 10 client.0 _ll_register_callbacks cb 0x5627d6f37760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f49db0 0x5627d6f515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f49db0 0x5627d6f515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52062/0 (socket says 172.21.15.76:52062) 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 -- 172.21.15.76:0/2326748235 learned_addr learned my addr 172.21.15.76:0/2326748235 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 -- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 msgr2=0x5627d6f77420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 0x5627d6f77420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 -- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 msgr2=0x5627d6f51ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 0x5627d6f51ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 -- 172.21.15.76:0/2326748235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d6d3ddc0 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f49db0 0x5627d6f515a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800de10 tx=0x7f7a58014800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43a237ef8fb997b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a44ff9640 1 -- 172.21.15.76:0/2326748235 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58024060 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a44ff9640 1 -- 172.21.15.76:0/2326748235 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f7a5801f400 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a600a58b0 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a44ff9640 1 -- 172.21.15.76:0/2326748235 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f7a580059a0 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 10 client.8667 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x5627d6d3a7f0 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a44ff9640 1 -- 172.21.15.76:0/2326748235 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600a58b0 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a44ff9640 1 -- 172.21.15.76:0/2326748235 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f7a5802d080 con 0x5627d6f49db0 2024-09-19T04:20:22.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 10 client.8667 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 10 client.8667 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 20 client.8667 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d6fa5010 0x5627d6fa7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 4 client.8667 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5627d6d34e10 con 0x5627d6fa5010 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a693b2640 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d6fa5010 0x5627d6fa7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a44ff9640 10 client.8667 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.208+0000 7f7a693b2640 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d6fa5010 0x5627d6fa7400 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e75906dfee445c31 server_cookie=9d0f853fd7aeb126 in_seq=0 out_seq=0 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a44ff9640 1 -- 172.21.15.76:0/2326748235 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5627d6d34e10 con 0x5627d6fa5010 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a44ff9640 10 client.8667 handle_command_reply: tid=0 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 client.8667 shutdown 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f49db0 msgr2=0x5627d6f515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f49db0 0x5627d6f515a0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f7a5800de10 tx=0x7f7a58014800 comp rx=0 tx=0).stop 2024-09-19T04:20:22.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 shutdown_connections 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d6fa5010 0x5627d6fa7400 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 0x5627d6f77420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a44ff9640 0 client.8667 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f95d70 0x5627d6f51ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/2326748235 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f49db0 0x5627d6f515a0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 >> 172.21.15.76:0/2326748235 conn(0x5627d6f77040 msgr2=0x5627d6f48f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 shutdown_connections 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 1 -- 172.21.15.76:0/2326748235 wait complete. 2024-09-19T04:20:22.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.212+0000 7f7a6eed3000 20 client.8667 trim_cache size 0 max 16384 2024-09-19T04:20:22.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 Processor -- start 2024-09-19T04:20:22.229 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 -- start start 2024-09-19T04:20:22.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 0x5627d6f951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d6f95710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.231 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f96690 0x5627d6f50110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a64200b30 con 0x5627d6f95d70 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a641ffcd0 con 0x5627d6f96690 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7a64200400 con 0x5627d6f49db0 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d6f95710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d6f95710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52068/0 (socket says 172.21.15.76:52068) 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 -- 172.21.15.76:0/944506376 learned_addr learned my addr 172.21.15.76:0/944506376 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 0x5627d6f951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 -- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 msgr2=0x5627d6f951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a6a3b4640 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f96690 0x5627d6f50110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 0x5627d6f951d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 -- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f96690 msgr2=0x5627d6f50110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.232 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f96690 0x5627d6f50110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 -- 172.21.15.76:0/944506376 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5627d6d58470 con 0x5627d6f95d70 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a693b2640 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d6f95710 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f7a6001d1f0 tx=0x7f7a60004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8375af63b8c6efa6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a46ffd640 1 -- 172.21.15.76:0/944506376 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600b2040 con 0x5627d6f95d70 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a46ffd640 1 -- 172.21.15.76:0/944506376 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600a6c50 con 0x5627d6f95d70 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.228+0000 7f7a46ffd640 1 -- 172.21.15.76:0/944506376 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600039a0 con 0x5627d6f95d70 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/944506376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 msgr2=0x5627d6f95710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d6f95710 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f7a6001d1f0 tx=0x7f7a60004a50 comp rx=0 tx=0).stop 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/944506376 shutdown_connections 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f96690 0x5627d6f50110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d6f95710 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/944506376 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f49db0 0x5627d6f951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/944506376 >> 172.21.15.76:0/944506376 conn(0x5627d6f77040 msgr2=0x5627d6f94ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.233 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/944506376 shutdown_connections 2024-09-19T04:20:22.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/944506376 wait complete. 2024-09-19T04:20:22.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- start start 2024-09-19T04:20:22.234 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 Processor -- start 2024-09-19T04:20:22.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f49db0 0x5627d700c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d700c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 0x5627d7009d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5627d6d96d00 con 0x5627d6f95d70 2024-09-19T04:20:22.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5627d6d9f370 con 0x5627d6f49db0 2024-09-19T04:20:22.237 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5627d6d9a050 con 0x5627d6f96690 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 10 client.0 _ll_register_callbacks cb 0x7f7a58009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d700c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d700c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52074/0 (socket says 172.21.15.76:52074) 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 -- 172.21.15.76:0/89190495 learned_addr learned my addr 172.21.15.76:0/89190495 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f49db0 0x5627d700c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6a3b4640 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 0x5627d7009d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 -- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 msgr2=0x5627d7009d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 0x5627d7009d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 -- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f49db0 msgr2=0x5627d700c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f49db0 0x5627d700c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 -- 172.21.15.76:0/89190495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5627d6d2e640 con 0x5627d6f95d70 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a693b2640 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d700c600 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009bd0 tx=0x7f7a6001b190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688187097d8c98b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.238 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a44ff9640 1 -- 172.21.15.76:0/89190495 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a600b2070 con 0x5627d6f95d70 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a44ff9640 1 -- 172.21.15.76:0/89190495 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f7a600cec60 con 0x5627d6f95d70 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a600a6c50 con 0x5627d6f95d70 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a44ff9640 1 -- 172.21.15.76:0/89190495 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f7a600cee80 con 0x5627d6f95d70 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 10 client.8673 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x5627d6d323b0 con 0x5627d6f95d70 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a44ff9640 1 -- 172.21.15.76:0/89190495 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a600a6c50 con 0x5627d6f95d70 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a44ff9640 1 -- 172.21.15.76:0/89190495 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f7a600cc030 con 0x5627d6f95d70 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 10 client.8673 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 10 client.8673 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 20 client.8673 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d700a2b0 0x5627d700a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 4 client.8673 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5627d6da8470 con 0x5627d700a2b0 2024-09-19T04:20:22.239 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d700a2b0 0x5627d700a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a69bb3640 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d700a2b0 0x5627d700a690 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dcc5f99a9ff7eac1 server_cookie=599ed46faea01ae7 in_seq=0 out_seq=0 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.232+0000 7f7a44ff9640 10 client.8673 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a44ff9640 1 -- 172.21.15.76:0/89190495 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x5627d6da8470 con 0x5627d700a2b0 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a44ff9640 10 client.8673 handle_command_reply: tid=0 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 client.8673 shutdown 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 msgr2=0x5627d700c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d700c600 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009bd0 tx=0x7f7a6001b190 comp rx=0 tx=0).stop 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 shutdown_connections 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x5627d700a2b0 0x5627d700a690 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5627d6f96690 0x5627d7009d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5627d6f95d70 0x5627d700c600 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.240 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 --2- 172.21.15.76:0/89190495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5627d6f49db0 0x5627d700c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a44ff9640 0 client.8673 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 >> 172.21.15.76:0/89190495 conn(0x5627d6f77040 msgr2=0x5627d6f951d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 shutdown_connections 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 1 -- 172.21.15.76:0/89190495 wait complete. 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6eed3000 20 client.8673 trim_cache size 0 max 16384 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:22.241 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:22.242 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:22.242 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:22.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.242 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.242 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:22.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a480678c0 msgr2=0x7f7a48069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.242 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a480678c0 0x7f7a48069d80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098f0 tx=0x7f7a58008040 comp rx=0 tx=0).stop 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6415f1d0 msgr2=0x7f7a6414a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6415f1d0 0x7f7a6414a340 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c00c4b0 tx=0x7f7a5c00c970 comp rx=0 tx=0).stop 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 shutdown_connections 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a480678c0 0x7f7a48069d80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6415f1d0 0x7f7a6414a340 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a641520a0 0x7f7a6414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 --2- 172.21.15.76:0/361706058 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6406fe50 0x7f7a6409df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 >> 172.21.15.76:0/361706058 conn(0x7f7a640b1f90 msgr2=0x7f7a6415d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 shutdown_connections 2024-09-19T04:20:22.243 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.236+0000 7f7a6be17640 1 -- 172.21.15.76:0/361706058 wait complete. 2024-09-19T04:20:22.251 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:22.251 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:22.251 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-19T04:20:22.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.356+0000 7fce5d9b9640 1 Processor -- start 2024-09-19T04:20:22.358 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.356+0000 7fce5d9b9640 1 -- start start 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce58161a60 0x7fce580d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fce580d2e90 0x7fce580cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce580d32b0 con 0x7fce580cf450 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fce580d0300 con 0x7fce580d2e90 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fce5806e6c0 con 0x7fce58161a60 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce567fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fce580d2e90 0x7fce580cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce577fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fce580cf450 0x7fce580d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51886/0 (socket says 172.21.15.76:51886) 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce577fe640 1 -- 172.21.15.76:0/2862066349 learned_addr learned my addr 172.21.15.76:0/2862066349 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:22.360 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2862066349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce58161a60 0x7fce580d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce567fc640 1 -- 172.21.15.76:0/2862066349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce58161a60 msgr2=0x7fce580d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce567fc640 1 --2- 172.21.15.76:0/2862066349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce58161a60 0x7fce580d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce567fc640 1 -- 172.21.15.76:0/2862066349 >> v1:172.21.15.76:6789/0 conn(0x7fce580cf450 legacy=0x7fce580d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce567fc640 1 -- 172.21.15.76:0/2862066349 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580b0f60 con 0x7fce580d2e90 2024-09-19T04:20:22.361 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce55ffb640 1 -- 172.21.15.76:0/2862066349 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3465941384 0 0) 0x7fce580d32b0 con 0x7fce580cf450 2024-09-19T04:20:22.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce567fc640 1 --2- 172.21.15.76:0/2862066349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580d2e90 0x7fce580cef10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0097c0 tx=0x7fce4c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d23c791bb77c5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce55ffb640 1 -- 172.21.15.76:0/2862066349 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0b1070 con 0x7fce580d2e90 2024-09-19T04:20:22.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce55ffb640 1 -- 172.21.15.76:0/2862066349 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4c0a8050 con 0x7fce580d2e90 2024-09-19T04:20:22.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce55ffb640 1 -- 172.21.15.76:0/2862066349 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce580d1930 con 0x7fce580d2e90 2024-09-19T04:20:22.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2862066349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580d2e90 msgr2=0x7fce580cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.362 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2862066349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580d2e90 0x7fce580cef10 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0097c0 tx=0x7fce4c01c3e0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2862066349 shutdown_connections 2024-09-19T04:20:22.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2862066349 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580d2e90 0x7fce580cef10 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2862066349 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce58161a60 0x7fce580d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2862066349 >> 172.21.15.76:0/2862066349 conn(0x7fce5808bf90 msgr2=0x7fce5809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2862066349 shutdown_connections 2024-09-19T04:20:22.363 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.360+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2862066349 wait complete. 2024-09-19T04:20:22.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 Processor -- start 2024-09-19T04:20:22.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 -- start start 2024-09-19T04:20:22.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580cf450 0x7fce580cade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce580d2e90 0x7fce58158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58161a60 0x7fce58158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.365 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fce580cf8c0 con 0x7fce580cf450 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce5806d190 con 0x7fce58161a60 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fce580cfa40 con 0x7fce580d2e90 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580cf450 0x7fce580cade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580cf450 0x7fce580cade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52094/0 (socket says 172.21.15.76:52094) 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 -- 172.21.15.76:0/2814225535 learned_addr learned my addr 172.21.15.76:0/2814225535 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce577fe640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58161a60 0x7fce58158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce567fc640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce580d2e90 0x7fce58158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 -- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce580d2e90 msgr2=0x7fce58158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce580d2e90 0x7fce58158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 -- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58161a60 msgr2=0x7fce58158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.366 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58161a60 0x7fce58158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 -- 172.21.15.76:0/2814225535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5815cd00 con 0x7fce580cf450 2024-09-19T04:20:22.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580cf450 0x7fce580cade0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c7f0 tx=0x7fce4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce3ffff640 1 -- 172.21.15.76:0/2814225535 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48015070 con 0x7fce580cf450 2024-09-19T04:20:22.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce3ffff640 1 -- 172.21.15.76:0/2814225535 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce48004240 con 0x7fce580cf450 2024-09-19T04:20:22.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5815cfe0 con 0x7fce580cf450 2024-09-19T04:20:22.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce3ffff640 1 -- 172.21.15.76:0/2814225535 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48003620 con 0x7fce580cf450 2024-09-19T04:20:22.367 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5815d5a0 con 0x7fce580cf450 2024-09-19T04:20:22.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce3ffff640 1 -- 172.21.15.76:0/2814225535 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fce480039d0 con 0x7fce580cf450 2024-09-19T04:20:22.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.364+0000 7fce3ffff640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fce440678c0 0x7fce44069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce3ffff640 1 -- 172.21.15.76:0/2814225535 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fce48085ee0 con 0x7fce580cf450 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce567fc640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fce440678c0 0x7fce44069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce567fc640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fce440678c0 0x7fce44069d80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0ae000 tx=0x7fce4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 Processor -- start 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- start start 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x562347983150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x562347936870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x562347937190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562347820960 con 0x562347936db0 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562347764900 con 0x562347983690 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562347787d40 con 0x562347982d70 2024-09-19T04:20:22.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x562347937190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x562347937190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52102/0 (socket says 172.21.15.76:52102) 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 -- 172.21.15.76:0/3167765937 learned_addr learned my addr 172.21.15.76:0/3167765937 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce56ffd640 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x562347983150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce567fc640 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x562347936870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 -- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 msgr2=0x562347983150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x562347983150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 -- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 msgr2=0x562347936870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x562347936870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 -- 172.21.15.76:0/3167765937 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234772e580 con 0x562347936db0 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce567fc640 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x562347936870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce577fe640 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x562347937190 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fce500072e0 tx=0x7fce500077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858af0ac3353ea3e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce3d7fa640 1 -- 172.21.15.76:0/3167765937 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50019070 con 0x562347936db0 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce3d7fa640 1 -- 172.21.15.76:0/3167765937 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce50004d80 con 0x562347936db0 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce3d7fa640 1 -- 172.21.15.76:0/3167765937 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50001e60 con 0x562347936db0 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- 172.21.15.76:0/3167765937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 msgr2=0x562347937190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x562347937190 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fce500072e0 tx=0x7fce500077a0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- 172.21.15.76:0/3167765937 shutdown_connections 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x562347937190 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x562347936870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 --2- 172.21.15.76:0/3167765937 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x562347983150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- 172.21.15.76:0/3167765937 >> 172.21.15.76:0/3167765937 conn(0x562347964040 msgr2=0x562347966430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.373 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- 172.21.15.76:0/3167765937 shutdown_connections 2024-09-19T04:20:22.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- 172.21.15.76:0/3167765937 wait complete. 2024-09-19T04:20:22.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 -- start start 2024-09-19T04:20:22.374 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.368+0000 7fce60a75000 1 Processor -- start 2024-09-19T04:20:22.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347936db0 0x562347989ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x56234798a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347983690 0x56234798a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.377 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562347783d00 con 0x562347983690 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x562347742aa0 con 0x562347936db0 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x562347721340 con 0x562347982d70 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 10 client.0 _ll_register_callbacks cb 0x562347924760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347983690 0x56234798a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347983690 0x56234798a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52106/0 (socket says 172.21.15.76:52106) 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 -- 172.21.15.76:0/2057992017 learned_addr learned my addr 172.21.15.76:0/2057992017 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 -- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 msgr2=0x56234798a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347936db0 0x562347989ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce567fc640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x56234798a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x56234798a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 -- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347936db0 msgr2=0x562347989ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.378 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347936db0 0x562347989ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 -- 172.21.15.76:0/2057992017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234772adc0 con 0x562347983690 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347936db0 0x562347989ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce577fe640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347983690 0x56234798a760 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fce50000f90 tx=0x7fce50007da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58dc078447745b8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 1 -- 172.21.15.76:0/2057992017 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50026060 con 0x562347983690 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 1 -- 172.21.15.76:0/2057992017 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fce50021c60 con 0x562347983690 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce50004d80 con 0x562347983690 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 1 -- 172.21.15.76:0/2057992017 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fce50019070 con 0x562347983690 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 10 client.8688 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x5623477277f0 con 0x562347983690 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 1 -- 172.21.15.76:0/2057992017 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce50004d80 con 0x562347983690 2024-09-19T04:20:22.379 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 1 -- 172.21.15.76:0/2057992017 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7fce5001f070 con 0x562347983690 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 10 client.8688 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 10 client.8688 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 20 client.8688 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479920b0 0x5623479944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 4 client.8688 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562347721e10 con 0x5623479920b0 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479920b0 0x5623479944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 10 client.8688 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479920b0 0x5623479944a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc679eb97c30a329 server_cookie=e207784328b684c in_seq=0 out_seq=0 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 1 -- 172.21.15.76:0/2057992017 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562347721e10 con 0x5623479920b0 2024-09-19T04:20:22.380 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 10 client.8688 handle_command_reply: tid=0 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 client.8688 shutdown 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347983690 msgr2=0x56234798a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347983690 0x56234798a760 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fce50000f90 tx=0x7fce50007da0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 shutdown_connections 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479920b0 0x5623479944a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347983690 0x56234798a760 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x56234798a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce367fc640 0 client.8688 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 --2- 172.21.15.76:0/2057992017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347936db0 0x562347989ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 >> 172.21.15.76:0/2057992017 conn(0x562347964040 msgr2=0x562347935f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 shutdown_connections 2024-09-19T04:20:22.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 1 -- 172.21.15.76:0/2057992017 wait complete. 2024-09-19T04:20:22.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.372+0000 7fce60a75000 20 client.8688 trim_cache size 0 max 16384 2024-09-19T04:20:22.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 Processor -- start 2024-09-19T04:20:22.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- start start 2024-09-19T04:20:22.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x5623479901e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x562347990720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x5623479976e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fce58203df0 con 0x562347936db0 2024-09-19T04:20:22.394 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fce58202f90 con 0x562347983690 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fce582036c0 con 0x562347982d70 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x5623479901e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x5623479901e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52112/0 (socket says 172.21.15.76:52112) 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 -- 172.21.15.76:0/1221661913 learned_addr learned my addr 172.21.15.76:0/1221661913 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 -- 172.21.15.76:0/1221661913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 msgr2=0x562347990720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 --2- 172.21.15.76:0/1221661913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x562347990720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 -- 172.21.15.76:0/1221661913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 msgr2=0x5623479976e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 --2- 172.21.15.76:0/1221661913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x5623479976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 -- 172.21.15.76:0/1221661913 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562347745470 con 0x562347936db0 2024-09-19T04:20:22.395 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce56ffd640 1 --2- 172.21.15.76:0/1221661913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x5623479901e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fce48007410 tx=0x7fce480524a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e0da341aa9ffcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce3cff9640 1 -- 172.21.15.76:0/1221661913 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4801e040 con 0x562347936db0 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce3cff9640 1 -- 172.21.15.76:0/1221661913 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce48084360 con 0x562347936db0 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce3cff9640 1 -- 172.21.15.76:0/1221661913 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48054460 con 0x562347936db0 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- 172.21.15.76:0/1221661913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 msgr2=0x5623479901e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 --2- 172.21.15.76:0/1221661913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x5623479901e0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fce48007410 tx=0x7fce480524a0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- 172.21.15.76:0/1221661913 shutdown_connections 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 --2- 172.21.15.76:0/1221661913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x5623479976e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 --2- 172.21.15.76:0/1221661913 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347982d70 0x562347990720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 --2- 172.21.15.76:0/1221661913 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347936db0 0x5623479901e0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.396 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- 172.21.15.76:0/1221661913 >> 172.21.15.76:0/1221661913 conn(0x562347964040 msgr2=0x56234798fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- 172.21.15.76:0/1221661913 shutdown_connections 2024-09-19T04:20:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- 172.21.15.76:0/1221661913 wait complete. 2024-09-19T04:20:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 -- start start 2024-09-19T04:20:22.397 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.392+0000 7fce60a75000 1 Processor -- start 2024-09-19T04:20:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347936db0 0x562347996c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347982d70 0x562347997180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x5623479f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x562347787050 con 0x562347982d70 2024-09-19T04:20:22.400 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x56234793da00 con 0x562347983690 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x56234774f400 con 0x562347936db0 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 10 client.0 _ll_register_callbacks cb 0x7fce50017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347982d70 0x562347997180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347982d70 0x562347997180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52116/0 (socket says 172.21.15.76:52116) 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 -- 172.21.15.76:0/2306627456 learned_addr learned my addr 172.21.15.76:0/2306627456 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 -- 172.21.15.76:0/2306627456 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347936db0 msgr2=0x562347996c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347936db0 0x562347996c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce577fe640 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x5623479f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 -- 172.21.15.76:0/2306627456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 msgr2=0x5623479f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x5623479f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 -- 172.21.15.76:0/2306627456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234771b640 con 0x562347982d70 2024-09-19T04:20:22.401 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce567fc640 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347982d70 0x562347997180 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000c10 tx=0x7fce4c019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b70e963361f567 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 1 -- 172.21.15.76:0/2306627456 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0b8a80 con 0x562347982d70 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 1 -- 172.21.15.76:0/2306627456 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fce4c0b8c20 con 0x562347982d70 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce48084360 con 0x562347982d70 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 1 -- 172.21.15.76:0/2306627456 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fce4c0d0050 con 0x562347982d70 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 10 client.8694 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x56234771f3b0 con 0x562347982d70 2024-09-19T04:20:22.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 1 -- 172.21.15.76:0/2306627456 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce48084360 con 0x562347982d70 2024-09-19T04:20:22.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 1 -- 172.21.15.76:0/2306627456 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7fce4c0d3020 con 0x562347982d70 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 10 client.8694 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 10 client.8694 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 20 client.8694 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479f3b00 0x5623479ff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 4 client.8694 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562347795470 con 0x5623479f3b00 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479f3b00 0x5623479ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 10 client.8694 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce56ffd640 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479f3b00 0x5623479ff460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da05c58979c3e65f server_cookie=b436746cf554348d in_seq=0 out_seq=0 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 1 -- 172.21.15.76:0/2306627456 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x562347795470 con 0x5623479f3b00 2024-09-19T04:20:22.404 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 10 client.8694 handle_command_reply: tid=0 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 client.8694 shutdown 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347982d70 msgr2=0x562347997180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347982d70 0x562347997180 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000c10 tx=0x7fce4c019180 comp rx=0 tx=0).stop 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 shutdown_connections 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5623479f3b00 0x5623479ff460 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x562347983690 0x5623479f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x562347982d70 0x562347997180 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce367fc640 0 client.8694 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 --2- 172.21.15.76:0/2306627456 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x562347936db0 0x562347996c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 >> 172.21.15.76:0/2306627456 conn(0x562347964040 msgr2=0x5623479902f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 shutdown_connections 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 1 -- 172.21.15.76:0/2306627456 wait complete. 2024-09-19T04:20:22.405 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce60a75000 20 client.8694 trim_cache size 0 max 16384 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.406 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:22.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fce440678c0 msgr2=0x7fce44069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.407 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fce440678c0 0x7fce44069d80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0ae000 tx=0x7fce4c008040 comp rx=0 tx=0).stop 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580cf450 msgr2=0x7fce580cade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580cf450 0x7fce580cade0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fce4800c7f0 tx=0x7fce4800ccb0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 shutdown_connections 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fce440678c0 0x7fce44069d80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fce58161a60 0x7fce58158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fce580d2e90 0x7fce58158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 --2- 172.21.15.76:0/2814225535 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fce580cf450 0x7fce580cade0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 >> 172.21.15.76:0/2814225535 conn(0x7fce5808bf90 msgr2=0x7fce580d04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 shutdown_connections 2024-09-19T04:20:22.408 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.396+0000 7fce5d9b9640 1 -- 172.21.15.76:0/2814225535 wait complete. 2024-09-19T04:20:22.412 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:22.412 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:22.413 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-19T04:20:22.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.516+0000 7f04185c6640 1 Processor -- start 2024-09-19T04:20:22.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f04185c6640 1 -- start start 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f04185c6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 0x7f0410071730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f04185c6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04100720b0 con 0x7f0410141a60 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f04185c6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0410072380 con 0x7f0410071c70 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f04185c6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0410072660 con 0x7f0410140f70 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0410141a60 0x7f04100d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:51894/0 (socket says 172.21.15.76:51894) 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 -- 172.21.15.76:0/1515301763 learned_addr learned my addr 172.21.15.76:0/1515301763 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416362640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 0x7f0410071730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1398862513 0 0) 0x7f04100720b0 con 0x7f0410141a60 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f03fc003530 con 0x7f0410141a60 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3276655758 0 0) 0x7f03fc003530 con 0x7f0410141a60 2024-09-19T04:20:22.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 msgr2=0x7f0410071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 0x7f0410071730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 >> v1:172.21.15.116:6789/0 conn(0x7f0410071c70 legacy=0x7f0410141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04100d0f90 con 0x7f0410141a60 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f040c001a00 con 0x7f0410141a60 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 >> v1:172.21.15.76:6789/0 conn(0x7f0410141a60 legacy=0x7f04100d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 0x7f03fc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc003fc0 0x7f03fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc004d50 0x7f03fc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0410072660 con 0x7f03fc004d50 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f03fc00bb10 con 0x7f03fc003fc0 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f03fc00bd10 con 0x7f0410140f70 2024-09-19T04:20:22.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc004d50 0x7f03fc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416362640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 0x7f03fc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0415b61640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc003fc0 0x7f03fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 -- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 msgr2=0x7f03fc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0410140f70 0x7f03fc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 -- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc003fc0 msgr2=0x7f03fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03fc003fc0 0x7f03fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.520+0000 7f0416b63640 1 -- 172.21.15.76:0/1515301763 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03fc003530 con 0x7f03fc004d50 2024-09-19T04:20:22.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0416b63640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc004d50 0x7f03fc0051d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f040c00a770 tx=0x7f040c01d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326cd63c45179a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00a390 con 0x7f03fc004d50 2024-09-19T04:20:22.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c0ad630 con 0x7f03fc004d50 2024-09-19T04:20:22.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415360640 1 -- 172.21.15.76:0/1515301763 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0ada60 con 0x7f03fc004d50 2024-09-19T04:20:22.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- 172.21.15.76:0/1515301763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc004d50 msgr2=0x7f03fc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 --2- 172.21.15.76:0/1515301763 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f03fc004d50 0x7f03fc0051d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f040c00a770 tx=0x7f040c01d2a0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0416b63640 1 -- 172.21.15.76:0/1515301763 reap_dead start 2024-09-19T04:20:22.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- 172.21.15.76:0/1515301763 shutdown_connections 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- 172.21.15.76:0/1515301763 >> 172.21.15.76:0/1515301763 conn(0x7f04100b3e70 msgr2=0x7f04100b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- 172.21.15.76:0/1515301763 shutdown_connections 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- 172.21.15.76:0/1515301763 wait complete. 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 Processor -- start 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- start start 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc004d50 0x7f041013e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04101e5a30 0x7f04101e5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04101e63f0 0x7f041015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0410141ed0 con 0x7f04101e5a30 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04100d15e0 con 0x7f04101e63f0 2024-09-19T04:20:22.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f04185c6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0410142050 con 0x7f03fc004d50 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04101e5a30 0x7f04101e5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04101e5a30 0x7f04101e5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52122/0 (socket says 172.21.15.76:52122) 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 -- 172.21.15.76:0/3264684207 learned_addr learned my addr 172.21.15.76:0/3264684207 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0416b63640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04101e63f0 0x7f041015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0416362640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc004d50 0x7f041013e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 -- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc004d50 msgr2=0x7f041013e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc004d50 0x7f041013e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 -- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04101e63f0 msgr2=0x7f041015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04101e63f0 0x7f041015d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 -- 172.21.15.76:0/3264684207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04101e6870 con 0x7f04101e5a30 2024-09-19T04:20:22.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0416362640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc004d50 0x7f041013e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.524+0000 7f0415b61640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04101e5a30 0x7f04101e5eb0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f040000e440 tx=0x7f040000e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0406ffd640 1 -- 172.21.15.76:0/3264684207 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000052e0 con 0x7f04101e5a30 2024-09-19T04:20:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0406ffd640 1 -- 172.21.15.76:0/3264684207 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0400018070 con 0x7f04101e5a30 2024-09-19T04:20:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0410159380 con 0x7f04101e5a30 2024-09-19T04:20:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0406ffd640 1 -- 172.21.15.76:0/3264684207 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400005480 con 0x7f04101e5a30 2024-09-19T04:20:22.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041015caf0 con 0x7f04101e5a30 2024-09-19T04:20:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0406ffd640 1 -- 172.21.15.76:0/3264684207 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0400003820 con 0x7f04101e5a30 2024-09-19T04:20:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0406ffd640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03f40678f0 0x7f03f4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0416362640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03f40678f0 0x7f03f4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0406ffd640 1 -- 172.21.15.76:0/3264684207 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f0400085b40 con 0x7f04101e5a30 2024-09-19T04:20:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f0416362640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03f40678f0 0x7f03f4069db0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0408009a20 tx=0x7f0408008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 Processor -- start 2024-09-19T04:20:22.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 -- start start 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655041bd150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bd690 0x565504170870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 0x565504171190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56550405a960 con 0x5655041bcd70 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565503f9e900 con 0x5655041bd690 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.528+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x565503fc1d40 con 0x565504170db0 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655041bd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655041bd150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52132/0 (socket says 172.21.15.76:52132) 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 -- 172.21.15.76:0/3650362572 learned_addr learned my addr 172.21.15.76:0/3650362572 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0415b61640 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bd690 0x565504170870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 0x565504171190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 -- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 msgr2=0x565504171190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.539 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 0x565504171190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 -- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bd690 msgr2=0x565504170870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bd690 0x565504170870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 -- 172.21.15.76:0/3650362572 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565503f68580 con 0x5655041bcd70 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416362640 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655041bd150 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0408012d50 tx=0x7f0408015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b44dfb26061c131 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f3fff640 1 -- 172.21.15.76:0/3650362572 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408022440 con 0x5655041bcd70 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f3fff640 1 -- 172.21.15.76:0/3650362572 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0408027070 con 0x5655041bcd70 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f3fff640 1 -- 172.21.15.76:0/3650362572 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04080227e0 con 0x5655041bcd70 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/3650362572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 msgr2=0x5655041bd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655041bd150 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0408012d50 tx=0x7f0408015000 comp rx=0 tx=0).stop 2024-09-19T04:20:22.540 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/3650362572 shutdown_connections 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 0x565504171190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bd690 0x565504170870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- 172.21.15.76:0/3650362572 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655041bd150 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/3650362572 >> 172.21.15.76:0/3650362572 conn(0x56550419e040 msgr2=0x5655041a0430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/3650362572 shutdown_connections 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/3650362572 wait complete. 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- start start 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 Processor -- start 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 0x565504178560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.541 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bcd70 0x565504178aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bd690 0x56550419e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x565503fbdd00 con 0x5655041bd690 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565503f7caa0 con 0x5655041bcd70 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x565503f5b340 con 0x565504170db0 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 10 client.0 _ll_register_callbacks cb 0x56550415e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bd690 0x56550419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bd690 0x56550419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52142/0 (socket says 172.21.15.76:52142) 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 -- 172.21.15.76:0/2743380971 learned_addr learned my addr 172.21.15.76:0/2743380971 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0415b61640 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bcd70 0x565504178aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 -- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 msgr2=0x565504178560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 0x565504178560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 -- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bcd70 msgr2=0x565504178aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.542 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bcd70 0x565504178aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 -- 172.21.15.76:0/2743380971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565503f64dc0 con 0x5655041bd690 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0415b61640 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bcd70 0x565504178aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f0416b63640 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bd690 0x56550419e420 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f040c00a770 tx=0x7f040c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df3412ea2ae5226 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f17fa640 1 -- 172.21.15.76:0/2743380971 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00a390 con 0x5655041bd690 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f17fa640 1 -- 172.21.15.76:0/2743380971 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f040c0b9cb0 con 0x5655041bd690 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0408027070 con 0x5655041bd690 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f17fa640 1 -- 172.21.15.76:0/2743380971 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f040c001800 con 0x5655041bd690 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 10 client.8706 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x565503f617f0 con 0x5655041bd690 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f17fa640 1 -- 172.21.15.76:0/2743380971 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0408027070 con 0x5655041bd690 2024-09-19T04:20:22.543 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f03f17fa640 1 -- 172.21.15.76:0/2743380971 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f040c004b70 con 0x5655041bd690 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 10 client.8706 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 10 client.8706 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 20 client.8706 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56550419ee30 0x5655041ce020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 4 client.8706 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.532+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565503f5be10 con 0x56550419ee30 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f0416362640 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56550419ee30 0x5655041ce020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f0416362640 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56550419ee30 0x5655041ce020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9c76d6a20a599fb server_cookie=480715d7723ae746 in_seq=0 out_seq=0 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f03f17fa640 10 client.8706 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f03f17fa640 1 -- 172.21.15.76:0/2743380971 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565503f5be10 con 0x56550419ee30 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f03f17fa640 10 client.8706 handle_command_reply: tid=0 2024-09-19T04:20:22.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 client.8706 shutdown 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bd690 msgr2=0x56550419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bd690 0x56550419e420 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f040c00a770 tx=0x7f040c01a040 comp rx=0 tx=0).stop 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 shutdown_connections 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x56550419ee30 0x5655041ce020 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bd690 0x56550419e420 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5655041bcd70 0x565504178aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f03f17fa640 0 client.8706 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 --2- 172.21.15.76:0/2743380971 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x565504170db0 0x565504178560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 >> 172.21.15.76:0/2743380971 conn(0x56550419e040 msgr2=0x56550416ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 shutdown_connections 2024-09-19T04:20:22.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 1 -- 172.21.15.76:0/2743380971 wait complete. 2024-09-19T04:20:22.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.536+0000 7f041b682000 20 client.8706 trim_cache size 0 max 16384 2024-09-19T04:20:22.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 Processor -- start 2024-09-19T04:20:22.553 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 -- start start 2024-09-19T04:20:22.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 0x5655041bc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 0x5655041bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655042157b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f04101ff670 con 0x5655041bcd70 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f04101fe810 con 0x565504170db0 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f04101fef40 con 0x56550419e040 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655042157b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655042157b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52148/0 (socket says 172.21.15.76:52148) 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 -- 172.21.15.76:0/3781959124 learned_addr learned my addr 172.21.15.76:0/3781959124 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 -- 172.21.15.76:0/3781959124 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 msgr2=0x5655041bc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 --2- 172.21.15.76:0/3781959124 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 0x5655041bc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 -- 172.21.15.76:0/3781959124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 msgr2=0x5655041bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 --2- 172.21.15.76:0/3781959124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 0x5655041bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 -- 172.21.15.76:0/3781959124 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565503f7f470 con 0x5655041bcd70 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f0416b63640 1 --2- 172.21.15.76:0/3781959124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655042157b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f040c01cd10 tx=0x7f040c0b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b549c13f2552088 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f03f37fe640 1 -- 172.21.15.76:0/3781959124 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0c7070 con 0x5655041bcd70 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f03f37fe640 1 -- 172.21.15.76:0/3781959124 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c0019a0 con 0x5655041bcd70 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.552+0000 7f03f37fe640 1 -- 172.21.15.76:0/3781959124 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00a390 con 0x5655041bcd70 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/3781959124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 msgr2=0x5655042157b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/3781959124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655042157b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f040c01cd10 tx=0x7f040c0b7000 comp rx=0 tx=0).stop 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/3781959124 shutdown_connections 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/3781959124 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x5655042157b0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/3781959124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 0x5655041bc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/3781959124 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 0x5655041bc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/3781959124 >> 172.21.15.76:0/3781959124 conn(0x565504175890 msgr2=0x5655041d1640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/3781959124 shutdown_connections 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/3781959124 wait complete. 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- start start 2024-09-19T04:20:22.558 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 Processor -- start 2024-09-19T04:20:22.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 0x565504235e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 0x565504236350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x565504219cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x56550417c3f0 con 0x5655041bcd70 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x565503fbdd00 con 0x565504170db0 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x565503fcffd0 con 0x56550419e040 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 10 client.0 _ll_register_callbacks cb 0x7f03fc003700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x565504219cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x565504219cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52158/0 (socket says 172.21.15.76:52158) 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 -- 172.21.15.76:0/754200040 learned_addr learned my addr 172.21.15.76:0/754200040 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 -- 172.21.15.76:0/754200040 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 msgr2=0x565504236350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 0x565504236350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416362640 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 0x565504235e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 -- 172.21.15.76:0/754200040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 msgr2=0x565504235e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 0x565504235e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 -- 172.21.15.76:0/754200040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565503f55640 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416b63640 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x565504219cf0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f040c01d780 tx=0x7f040c002140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a49b0afceb88616 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 1 -- 172.21.15.76:0/754200040 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0b8650 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 1 -- 172.21.15.76:0/754200040 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f040c0b87f0 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f040c0019a0 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 1 -- 172.21.15.76:0/754200040 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f040c0d9050 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 10 client.8712 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x565503f593b0 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 1 -- 172.21.15.76:0/754200040 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f040c0019a0 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 1 -- 172.21.15.76:0/754200040 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f040c0dc020 con 0x5655041bcd70 2024-09-19T04:20:22.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 10 client.8712 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 10 client.8712 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 20 client.8712 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x5655041bd690 0x56550421a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 4 client.8712 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565503fcf470 con 0x5655041bd690 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416362640 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x5655041bd690 0x56550421a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 10 client.8712 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f0416362640 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x5655041bd690 0x56550421a230 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0117be3d7763dc server_cookie=7596d3222ba57056 in_seq=0 out_seq=0 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 1 -- 172.21.15.76:0/754200040 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x565503fcf470 con 0x5655041bd690 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 10 client.8712 handle_command_reply: tid=0 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 client.8712 shutdown 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 msgr2=0x565504219cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x565504219cf0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f040c01d780 tx=0x7f040c002140 comp rx=0 tx=0).stop 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 shutdown_connections 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x5655041bd690 0x56550421a230 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5655041bcd70 0x565504219cf0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x56550419e040 0x565504236350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f03f17fa640 0 client.8712 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 --2- 172.21.15.76:0/754200040 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x565504170db0 0x565504235e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 >> 172.21.15.76:0/754200040 conn(0x565504175890 msgr2=0x5655041d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 shutdown_connections 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 1 -- 172.21.15.76:0/754200040 wait complete. 2024-09-19T04:20:22.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.556+0000 7f041b682000 20 client.8712 trim_cache size 0 max 16384 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.566 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:22.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03f40678f0 msgr2=0x7f03f4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03f40678f0 0x7f03f4069db0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0408009a20 tx=0x7f0408008040 comp rx=0 tx=0).stop 2024-09-19T04:20:22.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04101e5a30 msgr2=0x7f04101e5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04101e5a30 0x7f04101e5eb0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f040000e440 tx=0x7f040000e900 comp rx=0 tx=0).stop 2024-09-19T04:20:22.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 shutdown_connections 2024-09-19T04:20:22.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f03f40678f0 0x7f03f4069db0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f04101e63f0 0x7f041015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f04101e5a30 0x7f04101e5eb0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 --2- 172.21.15.76:0/3264684207 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f03fc004d50 0x7f041013e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 >> 172.21.15.76:0/3264684207 conn(0x7f04100b3e70 msgr2=0x7f04100b3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 shutdown_connections 2024-09-19T04:20:22.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.560+0000 7f04185c6640 1 -- 172.21.15.76:0/3264684207 wait complete. 2024-09-19T04:20:22.573 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:22.574 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:22.574 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-19T04:20:22.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.676+0000 7f8c5342b640 1 Processor -- start 2024-09-19T04:20:22.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.676+0000 7f8c5342b640 1 -- start start 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c5342b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c161a60 0x7f8c4c0d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c5342b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8c4c0d2e90 0x7f8c4c0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c5342b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8c4c0cf450 0x7f8c4c0d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c5342b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f8c4c0d32b0 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c5342b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8c4c0d0300 con 0x7f8c4c0cf450 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c5342b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4c0b0f60 con 0x7f8c4c161a60 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8c4c0d2e90 0x7f8c4c0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f8c4c0d2e90 0x7f8c4c0cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57286/0 (socket says 172.21.15.76:57286) 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 -- 172.21.15.76:0/2043136875 learned_addr learned my addr 172.21.15.76:0/2043136875 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c519c8640 1 --2- 172.21.15.76:0/2043136875 >> v2:172.21.15.116:3300/0 conn(0x7f8c4c0cf450 0x7f8c4c0d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c511c7640 1 --2- 172.21.15.76:0/2043136875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c161a60 0x7f8c4c0d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 -- 172.21.15.76:0/2043136875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c161a60 msgr2=0x7f8c4c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 --2- 172.21.15.76:0/2043136875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c161a60 0x7f8c4c0d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 -- 172.21.15.76:0/2043136875 >> v2:172.21.15.116:3300/0 conn(0x7f8c4c0cf450 msgr2=0x7f8c4c0d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 --2- 172.21.15.76:0/2043136875 >> v2:172.21.15.116:3300/0 conn(0x7f8c4c0cf450 0x7f8c4c0d0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 -- 172.21.15.76:0/2043136875 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c0cfd40 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c519c8640 1 --2- 172.21.15.76:0/2043136875 >> v2:172.21.15.116:3300/0 conn(0x7f8c4c0cf450 0x7f8c4c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c509c6640 1 --2- 172.21.15.76:0/2043136875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0cef10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0d23a0 tx=0x7f8c3c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e789cde6723637c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c43fff640 1 -- 172.21.15.76:0/2043136875 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0ab620 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c43fff640 1 -- 172.21.15.76:0/2043136875 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0b0070 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.680+0000 7f8c43fff640 1 -- 172.21.15.76:0/2043136875 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0ab9e0 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- 172.21.15.76:0/2043136875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 msgr2=0x7f8c4c0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 --2- 172.21.15.76:0/2043136875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0cef10 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c0d23a0 tx=0x7f8c3c01c460 comp rx=0 tx=0).stop 2024-09-19T04:20:22.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- 172.21.15.76:0/2043136875 shutdown_connections 2024-09-19T04:20:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 --2- 172.21.15.76:0/2043136875 >> v2:172.21.15.116:3300/0 conn(0x7f8c4c0cf450 0x7f8c4c0d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 --2- 172.21.15.76:0/2043136875 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0cef10 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 --2- 172.21.15.76:0/2043136875 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c161a60 0x7f8c4c0d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- 172.21.15.76:0/2043136875 >> 172.21.15.76:0/2043136875 conn(0x7f8c4c08bf90 msgr2=0x7f8c4c09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- 172.21.15.76:0/2043136875 shutdown_connections 2024-09-19T04:20:22.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- 172.21.15.76:0/2043136875 wait complete. 2024-09-19T04:20:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 Processor -- start 2024-09-19T04:20:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- start start 2024-09-19T04:20:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c0cf450 0x7f8c4c1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c4c161a60 0x7f8c4c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.686 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c06d830 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c06d6b0 con 0x7f8c4c161a60 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4c06d9b0 con 0x7f8c4c0cf450 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0b71a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57294/0 (socket says 172.21.15.76:57294) 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 -- 172.21.15.76:0/1880705668 learned_addr learned my addr 172.21.15.76:0/1880705668 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c519c8640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c4c161a60 0x7f8c4c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c511c7640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c0cf450 0x7f8c4c1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 -- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c0cf450 msgr2=0x7f8c4c1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c0cf450 0x7f8c4c1546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 -- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c4c161a60 msgr2=0x7f8c4c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c4c161a60 0x7f8c4c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.687 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 -- 172.21.15.76:0/1880705668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4c0b79c0 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c519c8640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c4c161a60 0x7f8c4c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c509c6640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0b71a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0ad040 tx=0x7f8c3c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c41ffb640 1 -- 172.21.15.76:0/1880705668 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0abdb0 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c41ffb640 1 -- 172.21.15.76:0/1880705668 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0be3f0 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c41ffb640 1 -- 172.21.15.76:0/1880705668 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0be7a0 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c4c0b7ca0 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.688 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.684+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c4c0b8260 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c41ffb640 1 -- 172.21.15.76:0/1880705668 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8c3c0c7440 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c41ffb640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c300678f0 0x7f8c30069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.691 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c41ffb640 1 -- 172.21.15.76:0/1880705668 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8c3c120340 con 0x7f8c4c0d2e90 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c511c7640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c300678f0 0x7f8c30069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c511c7640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c300678f0 0x7f8c30069db0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8c480096f0 tx=0x7f8c48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 Processor -- start 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- start start 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5a1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a554870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a555190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.692 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55791a43e960 con 0x55791a554db0 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55791a382900 con 0x55791a5a1690 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55791a3a5d40 con 0x55791a5a0d70 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a555190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a555190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57310/0 (socket says 172.21.15.76:57310) 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 -- 172.21.15.76:0/271380257 learned_addr learned my addr 172.21.15.76:0/271380257 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c509c6640 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a554870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c511c7640 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 -- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 msgr2=0x55791a5a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 -- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 msgr2=0x55791a554870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a554870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 -- 172.21.15.76:0/271380257 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55791a34c580 con 0x55791a554db0 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c519c8640 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a555190 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8c440073b0 tx=0x7f8c44007870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d91f7083da5e73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c2effd640 1 -- 172.21.15.76:0/271380257 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c440137d0 con 0x55791a554db0 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c2effd640 1 -- 172.21.15.76:0/271380257 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c44019070 con 0x55791a554db0 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c2effd640 1 -- 172.21.15.76:0/271380257 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c44013b70 con 0x55791a554db0 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- 172.21.15.76:0/271380257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 msgr2=0x55791a555190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a555190 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8c440073b0 tx=0x7f8c44007870 comp rx=0 tx=0).stop 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- 172.21.15.76:0/271380257 shutdown_connections 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a555190 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a554870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 --2- 172.21.15.76:0/271380257 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- 172.21.15.76:0/271380257 >> 172.21.15.76:0/271380257 conn(0x55791a582040 msgr2=0x55791a584430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- 172.21.15.76:0/271380257 shutdown_connections 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.688+0000 7f8c564e7000 1 -- 172.21.15.76:0/271380257 wait complete. 2024-09-19T04:20:22.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 -- start start 2024-09-19T04:20:22.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 Processor -- start 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a554db0 0x55791a5a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 0x55791a5a8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a5a1690 0x55791a5a8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55791a3a1d00 con 0x55791a5a1690 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55791a360aa0 con 0x55791a5a0d70 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55791a33f340 con 0x55791a554db0 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 10 client.0 _ll_register_callbacks cb 0x55791a542760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a5a1690 0x55791a5a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a5a1690 0x55791a5a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57322/0 (socket says 172.21.15.76:57322) 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 -- 172.21.15.76:0/4234414059 learned_addr learned my addr 172.21.15.76:0/4234414059 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 -- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a554db0 msgr2=0x55791a5a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a554db0 0x55791a5a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 -- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 msgr2=0x55791a5a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 0x55791a5a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 -- 172.21.15.76:0/4234414059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791a348dc0 con 0x55791a5a1690 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c519c8640 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a5a1690 0x55791a5a8760 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8c44016040 tx=0x7f8c44004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa61ed6e7d20fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c27fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c27fff640 1 -- 172.21.15.76:0/4234414059 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4402b060 con 0x55791a5a1690 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c44019070 con 0x55791a5a1690 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c27fff640 1 -- 172.21.15.76:0/4234414059 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8c44014670 con 0x55791a5a1690 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c27fff640 1 -- 172.21.15.76:0/4234414059 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f8c44031050 con 0x55791a5a1690 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 10 client.8724 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55791a3457f0 con 0x55791a5a1690 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c27fff640 1 -- 172.21.15.76:0/4234414059 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c44019070 con 0x55791a5a1690 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c27fff640 1 -- 172.21.15.76:0/4234414059 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f8c44034020 con 0x55791a5a1690 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 10 client.8724 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 10 client.8724 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 20 client.8724 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5b00b0 0x55791a5b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 4 client.8724 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55791a33fe10 con 0x55791a5b00b0 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c511c7640 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5b00b0 0x55791a5b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c27fff640 10 client.8724 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.692+0000 7f8c511c7640 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5b00b0 0x55791a5b24a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8ad7831a6a91f14 server_cookie=8f94660c712043ef in_seq=0 out_seq=0 2024-09-19T04:20:22.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c27fff640 1 -- 172.21.15.76:0/4234414059 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55791a33fe10 con 0x55791a5b00b0 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c27fff640 10 client.8724 handle_command_reply: tid=0 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 client.8724 shutdown 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a5a1690 msgr2=0x55791a5a8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a5a1690 0x55791a5a8760 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8c44016040 tx=0x7f8c44004ac0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 shutdown_connections 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5b00b0 0x55791a5b24a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a5a1690 0x55791a5a8760 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 0x55791a5a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c27fff640 0 client.8724 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 --2- 172.21.15.76:0/4234414059 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a554db0 0x55791a5a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 >> 172.21.15.76:0/4234414059 conn(0x55791a582040 msgr2=0x55791a553f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 shutdown_connections 2024-09-19T04:20:22.701 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 1 -- 172.21.15.76:0/4234414059 wait complete. 2024-09-19T04:20:22.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.696+0000 7f8c564e7000 20 client.8724 trim_cache size 0 max 16384 2024-09-19T04:20:22.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 Processor -- start 2024-09-19T04:20:22.713 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 -- start start 2024-09-19T04:20:22.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a6103c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 0x55791a610900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a1690 0x55791a610e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c204550 con 0x55791a554db0 2024-09-19T04:20:22.715 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c2036f0 con 0x55791a5a0d70 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4c203e20 con 0x55791a5a1690 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a6103c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a6103c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57334/0 (socket says 172.21.15.76:57334) 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 -- 172.21.15.76:0/1166748201 learned_addr learned my addr 172.21.15.76:0/1166748201 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 -- 172.21.15.76:0/1166748201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a1690 msgr2=0x55791a610e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 --2- 172.21.15.76:0/1166748201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a1690 0x55791a610e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 -- 172.21.15.76:0/1166748201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 msgr2=0x55791a610900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 --2- 172.21.15.76:0/1166748201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 0x55791a610900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 -- 172.21.15.76:0/1166748201 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55791a363470 con 0x55791a554db0 2024-09-19T04:20:22.716 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c511c7640 1 --2- 172.21.15.76:0/1166748201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a6103c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800ea30 tx=0x7f8c48011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2197bc44db341f0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c2e7fc640 1 -- 172.21.15.76:0/1166748201 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4800f040 con 0x55791a554db0 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c2e7fc640 1 -- 172.21.15.76:0/1166748201 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c48015700 con 0x55791a554db0 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.712+0000 7f8c2e7fc640 1 -- 172.21.15.76:0/1166748201 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c48015ab0 con 0x55791a554db0 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/1166748201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 msgr2=0x55791a6103c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- 172.21.15.76:0/1166748201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a6103c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800ea30 tx=0x7f8c48011000 comp rx=0 tx=0).stop 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/1166748201 shutdown_connections 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- 172.21.15.76:0/1166748201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a1690 0x55791a610e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- 172.21.15.76:0/1166748201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a0d70 0x55791a610900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.717 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- 172.21.15.76:0/1166748201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a6103c0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/1166748201 >> 172.21.15.76:0/1166748201 conn(0x55791a582040 msgr2=0x55791a55bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/1166748201 shutdown_connections 2024-09-19T04:20:22.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/1166748201 wait complete. 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- start start 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 Processor -- start 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a5ad950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a5d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55791a34bf20 con 0x55791a554db0 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4c1fb450 con 0x55791a5a1690 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55791a5603f0 con 0x55791a5a0d70 2024-09-19T04:20:22.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 10 client.0 _ll_register_callbacks cb 0x7f8c44009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a5ad950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a5ad950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57336/0 (socket says 172.21.15.76:57336) 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 -- 172.21.15.76:0/3251743793 learned_addr learned my addr 172.21.15.76:0/3251743793 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c509c6640 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 -- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 msgr2=0x55791a5d0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c519c8640 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a5d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 -- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 msgr2=0x55791a5d0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a5d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 -- 172.21.15.76:0/3251743793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791a339640 con 0x55791a554db0 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c511c7640 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a5ad950 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802d040 tx=0x7f8c48004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4424cd07878f672a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c27fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c27fff640 1 -- 172.21.15.76:0/3251743793 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4803b020 con 0x55791a554db0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c27fff640 1 -- 172.21.15.76:0/3251743793 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8c48015cc0 con 0x55791a554db0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c48015700 con 0x55791a554db0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c27fff640 1 -- 172.21.15.76:0/3251743793 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f8c48045050 con 0x55791a554db0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 10 client.8730 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55791a33d3b0 con 0x55791a554db0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c27fff640 1 -- 172.21.15.76:0/3251743793 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c48015700 con 0x55791a554db0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c27fff640 1 -- 172.21.15.76:0/3251743793 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f8c48048020 con 0x55791a554db0 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 10 client.8730 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 10 client.8730 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 20 client.8730 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5d1210 0x55791a614a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 4 client.8730 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:22.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55791a3b3470 con 0x55791a5d1210 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.716+0000 7f8c509c6640 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5d1210 0x55791a614a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c27fff640 10 client.8730 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c509c6640 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5d1210 0x55791a614a30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d33e639b7d04c69f server_cookie=cb4a9822ef6d4422 in_seq=0 out_seq=0 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c27fff640 1 -- 172.21.15.76:0/3251743793 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55791a3b3470 con 0x55791a5d1210 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c27fff640 10 client.8730 handle_command_reply: tid=0 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 client.8730 shutdown 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 msgr2=0x55791a5ad950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a5ad950 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8c4802d040 tx=0x7f8c48004740 comp rx=0 tx=0).stop 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 shutdown_connections 2024-09-19T04:20:22.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55791a5d1210 0x55791a614a30 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55791a5a1690 0x55791a5d0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55791a5a0d70 0x55791a5d0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c27fff640 0 client.8730 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 --2- 172.21.15.76:0/3251743793 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55791a554db0 0x55791a5ad950 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 >> 172.21.15.76:0/3251743793 conn(0x55791a582040 msgr2=0x55791a55c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 shutdown_connections 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 1 -- 172.21.15.76:0/3251743793 wait complete. 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c564e7000 20 client.8730 trim_cache size 0 max 16384 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:22.726 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.727 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:22.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c300678f0 msgr2=0x7f8c30069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c300678f0 0x7f8c30069db0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8c480096f0 tx=0x7f8c48008040 comp rx=0 tx=0).stop 2024-09-19T04:20:22.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 msgr2=0x7f8c4c0b71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0b71a0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c0ad040 tx=0x7f8c3c01aca0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 shutdown_connections 2024-09-19T04:20:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8c300678f0 0x7f8c30069db0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c4c161a60 0x7f8c4c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8c4c0d2e90 0x7f8c4c0b71a0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 --2- 172.21.15.76:0/1880705668 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8c4c0cf450 0x7f8c4c1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 >> 172.21.15.76:0/1880705668 conn(0x7f8c4c08bf90 msgr2=0x7f8c4c15a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 shutdown_connections 2024-09-19T04:20:22.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.720+0000 7f8c5342b640 1 -- 172.21.15.76:0/1880705668 wait complete. 2024-09-19T04:20:22.734 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:22.734 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:22.735 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-19T04:20:22.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.840+0000 7fa6c7dcf640 1 Processor -- start 2024-09-19T04:20:22.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.840+0000 7fa6c7dcf640 1 -- start start 2024-09-19T04:20:22.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c7dcf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 0x7fa6c006ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c7dcf640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa6c006d210 0x7fa6c015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c7dcf640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6c015c4a0 con 0x7fa6c015c0c0 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c7dcf640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa6c015d340 con 0x7fa6c006d210 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c7dcf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c015c730 con 0x7fa6c00b5910 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c5b6b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fa6c015c0c0 0x7fa6c00b4540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52766/0 (socket says 172.21.15.76:52766) 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/3824605386 learned_addr learned my addr 172.21.15.76:0/3824605386 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c636c640 1 --2- 172.21.15.76:0/3824605386 >> v2:172.21.15.116:3300/0 conn(0x7fa6c006d210 0x7fa6c015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 0x7fa6c006ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3104537635 0 0) 0x7fa6c015c4a0 con 0x7fa6c015c0c0 2024-09-19T04:20:22.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa6ac003530 con 0x7fa6c015c0c0 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 462212954 0 0) 0x7fa6ac003530 con 0x7fa6c015c0c0 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 msgr2=0x7fa6c006ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 0x7fa6c006ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 >> v2:172.21.15.116:3300/0 conn(0x7fa6c006d210 msgr2=0x7fa6c015d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 --2- 172.21.15.76:0/3824605386 >> v2:172.21.15.116:3300/0 conn(0x7fa6c006d210 0x7fa6c015d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c015ca10 con 0x7fa6c015c0c0 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fa6bc001680 con 0x7fa6c015c0c0 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 >> v1:172.21.15.76:6789/0 conn(0x7fa6c015c0c0 legacy=0x7fa6c00b4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 0x7fa6ac0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6ac003f40 0x7fa6ac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6ac004d50 0x7fa6ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c015c730 con 0x7fa6ac003f40 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa6ac00b9d0 con 0x7fa6ac004d50 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa6ac00bb50 con 0x7fa6c00b5910 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6ac003f40 0x7fa6ac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 0x7fa6ac0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c636c640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6ac004d50 0x7fa6ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 -- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 msgr2=0x7fa6ac0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00b5910 0x7fa6ac0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 -- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6ac004d50 msgr2=0x7fa6ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6ac004d50 0x7fa6ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 -- 172.21.15.76:0/3824605386 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6ac003530 con 0x7fa6ac003f40 2024-09-19T04:20:22.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c636c640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6ac004d50 0x7fa6ac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c536a640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6ac003f40 0x7fa6ac004810 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009ab0 tx=0x7fa6b001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a439ecf720d18f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00b2070 con 0x7fa6ac003f40 2024-09-19T04:20:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b00a5530 con 0x7fa6ac003f40 2024-09-19T04:20:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c4b69640 1 -- 172.21.15.76:0/3824605386 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00096b0 con 0x7fa6ac003f40 2024-09-19T04:20:22.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/3824605386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6ac003f40 msgr2=0x7fa6ac004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.844+0000 7fa6c7dcf640 1 --2- 172.21.15.76:0/3824605386 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6ac003f40 0x7fa6ac004810 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009ab0 tx=0x7fa6b001c9f0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 -- 172.21.15.76:0/3824605386 reap_dead start 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/3824605386 shutdown_connections 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/3824605386 >> 172.21.15.76:0/3824605386 conn(0x7fa6c0072690 msgr2=0x7fa6c0072aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/3824605386 shutdown_connections 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/3824605386 wait complete. 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 Processor -- start 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- start start 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6c00afee0 0x7fa6c00a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00a9f00 0x7fa6c0157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6c01f4940 0x7fa6c0157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c006ccd0 con 0x7fa6c01f4940 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c015cfd0 con 0x7fa6c00afee0 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c006ce50 con 0x7fa6c00a9f00 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6c01f4940 0x7fa6c0157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6c01f4940 0x7fa6c0157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57348/0 (socket says 172.21.15.76:57348) 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 -- 172.21.15.76:0/1482995215 learned_addr learned my addr 172.21.15.76:0/1482995215 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6c00afee0 0x7fa6c00a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c536a640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00a9f00 0x7fa6c0157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 -- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00a9f00 msgr2=0x7fa6c0157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00a9f00 0x7fa6c0157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 -- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6c00afee0 msgr2=0x7fa6c00a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6c00afee0 0x7fa6c00a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 -- 172.21.15.76:0/1482995215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01f38a0 con 0x7fa6c01f4940 2024-09-19T04:20:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c636c640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6c01f4940 0x7fa6c0157970 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0073b30 tx=0x7fa6b800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6b67fc640 1 -- 172.21.15.76:0/1482995215 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b801c070 con 0x7fa6c01f4940 2024-09-19T04:20:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6b67fc640 1 -- 172.21.15.76:0/1482995215 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b8017400 con 0x7fa6c01f4940 2024-09-19T04:20:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c01f3b80 con 0x7fa6c01f4940 2024-09-19T04:20:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6b67fc640 1 -- 172.21.15.76:0/1482995215 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80177d0 con 0x7fa6c01f4940 2024-09-19T04:20:22.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.848+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c01f4140 con 0x7fa6c01f4940 2024-09-19T04:20:22.858 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6b67fc640 1 -- 172.21.15.76:0/1482995215 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa6b801f070 con 0x7fa6c01f4940 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6b67fc640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6a40678c0 0x7fa6a4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6b67fc640 1 -- 172.21.15.76:0/1482995215 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fa6b8085fd0 con 0x7fa6c01f4940 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6a40678c0 0x7fa6a4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6a40678c0 0x7fa6a4069d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc004850 tx=0x7fa6bc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 Processor -- start 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- start start 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a50a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 0x55d46a4bd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4bddb0 0x55d46a4be190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d46a3a7960 con 0x55d46a509d70 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d46a2eb900 con 0x55d46a50a690 2024-09-19T04:20:22.859 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d46a30ed40 con 0x55d46a4bddb0 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a50a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a50a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57360/0 (socket says 172.21.15.76:57360) 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/1331925542 learned_addr learned my addr 172.21.15.76:0/1331925542 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/1331925542 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4bddb0 msgr2=0x55d46a4be190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/1331925542 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4bddb0 0x55d46a4be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/1331925542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 msgr2=0x55d46a4bd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/1331925542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 0x55d46a4bd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/1331925542 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d46a2b5580 con 0x55d46a509d70 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/1331925542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a50a150 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc013b40 tx=0x7fa6bc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118a7cc6efe27be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6a37fe640 1 -- 172.21.15.76:0/1331925542 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc029070 con 0x55d46a509d70 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6a37fe640 1 -- 172.21.15.76:0/1331925542 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6bc016c30 con 0x55d46a509d70 2024-09-19T04:20:22.860 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6a37fe640 1 -- 172.21.15.76:0/1331925542 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc010040 con 0x55d46a509d70 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- 172.21.15.76:0/1331925542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 msgr2=0x55d46a50a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/1331925542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a50a150 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc013b40 tx=0x7fa6bc016000 comp rx=0 tx=0).stop 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- 172.21.15.76:0/1331925542 shutdown_connections 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/1331925542 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4bddb0 0x55d46a4be190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/1331925542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 0x55d46a4bd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/1331925542 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a50a150 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- 172.21.15.76:0/1331925542 >> 172.21.15.76:0/1331925542 conn(0x55d46a4eb040 msgr2=0x55d46a4ed430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- 172.21.15.76:0/1331925542 shutdown_connections 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.852+0000 7fa6cae8b000 1 -- 172.21.15.76:0/1331925542 wait complete. 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- start start 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 Processor -- start 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.861 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a509d70 0x55d46a4c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 0x55d46a4eb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d46a30ad00 con 0x55d46a4bddb0 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d46a2c9aa0 con 0x55d46a50a690 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d46a2a8340 con 0x55d46a509d70 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 10 client.0 _ll_register_callbacks cb 0x55d46a4ab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4c55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57374/0 (socket says 172.21.15.76:57374) 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/3750904554 learned_addr learned my addr 172.21.15.76:0/3750904554 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/3750904554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a509d70 msgr2=0x55d46a4c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a509d70 0x55d46a4c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/3750904554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 msgr2=0x55d46a4eb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 0x55d46a4eb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 -- 172.21.15.76:0/3750904554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46a2b1dc0 con 0x55d46a4bddb0 2024-09-19T04:20:22.862 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4c55a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc027040 tx=0x7fa6bc000ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72fb07b3b1aad3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3750904554 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc029070 con 0x55d46a4bddb0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3750904554 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fa6bc023970 con 0x55d46a4bddb0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6bc016c30 con 0x55d46a4bddb0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3750904554 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fa6bc03c050 con 0x55d46a4bddb0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 10 client.8742 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55d46a2ae7f0 con 0x55d46a4bddb0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3750904554 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6bc016c30 con 0x55d46a4bddb0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3750904554 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7fa6bc03f020 con 0x55d46a4bddb0 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 10 client.8742 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 10 client.8742 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 20 client.8742 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a4ebe30 0x55d46a51b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 4 client.8742 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d46a2a8e10 con 0x55d46a4ebe30 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c536a640 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a4ebe30 0x55d46a51b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 10 client.8742 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6c536a640 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a4ebe30 0x55d46a51b020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=319655d0c35f7794 server_cookie=9e87d56f557e2ee3 in_seq=0 out_seq=0 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3750904554 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d46a2a8e10 con 0x55d46a4ebe30 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 10 client.8742 handle_command_reply: tid=0 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 client.8742 shutdown 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 msgr2=0x55d46a4c55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4c55a0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc027040 tx=0x7fa6bc000ff0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.864 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 shutdown_connections 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a4ebe30 0x55d46a51b020 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a50a690 0x55d46a4eb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a509d70 0x55d46a4c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3750904554 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4c55a0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6a0ff9640 0 client.8742 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 >> 172.21.15.76:0/3750904554 conn(0x55d46a4eb040 msgr2=0x55d46a4bcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 shutdown_connections 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3750904554 wait complete. 2024-09-19T04:20:22.865 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.856+0000 7fa6cae8b000 20 client.8742 trim_cache size 0 max 16384 2024-09-19T04:20:22.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 Processor -- start 2024-09-19T04:20:22.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- start start 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 0x55d46a4e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4ea270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a509d70 0x55d46a51b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01ffab0 con 0x55d46a4bddb0 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01fec50 con 0x55d46a509d70 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa6c01ff380 con 0x55d46a4eb040 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4ea270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4ea270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57380/0 (socket says 172.21.15.76:57380) 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 -- 172.21.15.76:0/2514413646 learned_addr learned my addr 172.21.15.76:0/2514413646 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 -- 172.21.15.76:0/2514413646 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 msgr2=0x55d46a4e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 0x55d46a4e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a509d70 0x55d46a51b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 -- 172.21.15.76:0/2514413646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a509d70 msgr2=0x55d46a51b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a509d70 0x55d46a51b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 -- 172.21.15.76:0/2514413646 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d46a2cc470 con 0x55d46a4bddb0 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c536a640 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4ea270 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa6b001c820 tx=0x7fa6b00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb144d6b579634a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6a2ffd640 1 -- 172.21.15.76:0/2514413646 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00b2040 con 0x55d46a4bddb0 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6a2ffd640 1 -- 172.21.15.76:0/2514413646 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b00a5530 con 0x55d46a4bddb0 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6a2ffd640 1 -- 172.21.15.76:0/2514413646 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b00a58e0 con 0x55d46a4bddb0 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- 172.21.15.76:0/2514413646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 msgr2=0x55d46a4ea270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4ea270 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fa6b001c820 tx=0x7fa6b00046d0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- 172.21.15.76:0/2514413646 shutdown_connections 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a509d70 0x55d46a51b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a4bddb0 0x55d46a4ea270 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/2514413646 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 0x55d46a4e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- 172.21.15.76:0/2514413646 >> 172.21.15.76:0/2514413646 conn(0x55d46a4c2890 msgr2=0x55d46a509180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- 172.21.15.76:0/2514413646 shutdown_connections 2024-09-19T04:20:22.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- 172.21.15.76:0/2514413646 wait complete. 2024-09-19T04:20:22.882 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- start start 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 Processor -- start 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a4bddb0 0x55d46a51e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 0x55d46a539310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a539850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55d46a4c67b0 con 0x55d46a509d70 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55d46a30ad00 con 0x55d46a4bddb0 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55d46a31cfd0 con 0x55d46a4eb040 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6cae8b000 10 client.0 _ll_register_callbacks cb 0x7fa6bc027070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a539850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a539850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57386/0 (socket says 172.21.15.76:57386) 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 -- 172.21.15.76:0/3401155165 learned_addr learned my addr 172.21.15.76:0/3401155165 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:22.883 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 -- 172.21.15.76:0/3401155165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 msgr2=0x55d46a539310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 0x55d46a539310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 -- 172.21.15.76:0/3401155165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a4bddb0 msgr2=0x55d46a51e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a4bddb0 0x55d46a51e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.876+0000 7fa6c636c640 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a4bddb0 0x55d46a51e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c636c640 1 -- 172.21.15.76:0/3401155165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46a2a2640 con 0x55d46a509d70 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c636c640 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a539850 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80561e0 tx=0x7fa6b8056a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd6f4bcf3603a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3401155165 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b80566b0 con 0x55d46a509d70 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3401155165 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fa6b805e9a0 con 0x55d46a509d70 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b00a5530 con 0x55d46a509d70 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3401155165 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7fa6b805ee30 con 0x55d46a509d70 2024-09-19T04:20:22.884 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 10 client.8748 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55d46a2a63b0 con 0x55d46a509d70 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3401155165 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6b00a5530 con 0x55d46a509d70 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3401155165 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7fa6b805a0d0 con 0x55d46a509d70 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 10 client.8748 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 10 client.8748 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 20 client.8748 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a50a690 0x55d46a539d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 4 client.8748 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d46a31c470 con 0x55d46a50a690 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a50a690 0x55d46a539d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 10 client.8748 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.885 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c5b6b640 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a50a690 0x55d46a539d90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e9bf860a98c64 server_cookie=9608560e22746f7c in_seq=0 out_seq=0 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 1 -- 172.21.15.76:0/3401155165 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55d46a31c470 con 0x55d46a50a690 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 10 client.8748 handle_command_reply: tid=0 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 client.8748 shutdown 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 msgr2=0x55d46a539850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a539850 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80561e0 tx=0x7fa6b8056a90 comp rx=0 tx=0).stop 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 shutdown_connections 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x55d46a50a690 0x55d46a539d90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55d46a509d70 0x55d46a539850 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55d46a4eb040 0x55d46a539310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6a0ff9640 0 client.8748 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 --2- 172.21.15.76:0/3401155165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55d46a4bddb0 0x55d46a51e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 >> 172.21.15.76:0/3401155165 conn(0x55d46a4c2890 msgr2=0x55d46a4e8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 shutdown_connections 2024-09-19T04:20:22.886 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 1 -- 172.21.15.76:0/3401155165 wait complete. 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6cae8b000 20 client.8748 trim_cache size 0 max 16384 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.887 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:22.888 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:22.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6a40678c0 msgr2=0x7fa6a4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.888 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6a40678c0 0x7fa6a4069d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc004850 tx=0x7fa6bc009040 comp rx=0 tx=0).stop 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6c01f4940 msgr2=0x7fa6c0157970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6c01f4940 0x7fa6c0157970 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fa6c0073b30 tx=0x7fa6b800f4a0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 shutdown_connections 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa6a40678c0 0x7fa6a4069d80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa6c01f4940 0x7fa6c0157970 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa6c00a9f00 0x7fa6c0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 --2- 172.21.15.76:0/1482995215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6c00afee0 0x7fa6c00a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 >> 172.21.15.76:0/1482995215 conn(0x7fa6c0072690 msgr2=0x7fa6c015a690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 shutdown_connections 2024-09-19T04:20:22.889 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:22.880+0000 7fa6c7dcf640 1 -- 172.21.15.76:0/1482995215 wait complete. 2024-09-19T04:20:22.895 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:22.896 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:22.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc290fbe640 1 Processor -- start 2024-09-19T04:20:22.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc290fbe640 1 -- start start 2024-09-19T04:20:23.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc290fbe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 0x7fc28c141350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc290fbe640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc28c072380 con 0x7fc28c141890 2024-09-19T04:20:23.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc290fbe640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc28c0d0f90 con 0x7fc28c140f70 2024-09-19T04:20:23.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc290fbe640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c072660 con 0x7fc28c071e40 2024-09-19T04:20:23.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc28ad9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fc28c141890 0x7fc28c0d1a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52782/0 (socket says 172.21.15.76:52782) 2024-09-19T04:20:23.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc28ad9d640 1 -- 172.21.15.76:0/2291203365 learned_addr learned my addr 172.21.15.76:0/2291203365 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc281d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 0x7fc28c141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2357623558 0 0) 0x7fc28c072380 con 0x7fc28c141890 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc278003530 con 0x7fc28c141890 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2396777910 0 0) 0x7fc278003530 con 0x7fc28c141890 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 msgr2=0x7fc28c141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 0x7fc28c141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 >> v1:172.21.15.116:6789/0 conn(0x7fc28c140f70 legacy=0x7fc28c071730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c0d1270 con 0x7fc28c141890 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fc284001a00 con 0x7fc28c141890 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 >> v1:172.21.15.76:6789/0 conn(0x7fc28c141890 legacy=0x7fc28c0d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 0x7fc2780039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc278004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc278004d50 0x7fc2780051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c072660 con 0x7fc278003f70 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc27800bab0 con 0x7fc278004d50 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc27800bcb0 con 0x7fc28c071e40 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc281d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc278004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc28a59c640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 0x7fc2780039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc28ad9d640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc278004d50 0x7fc2780051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc281d9b640 1 -- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 msgr2=0x7fc2780039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc281d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c071e40 0x7fc2780039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc281d9b640 1 -- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc278004d50 msgr2=0x7fc2780051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc281d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc278004d50 0x7fc2780051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc281d9b640 1 -- 172.21.15.76:0/2291203365 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc278003530 con 0x7fc278003f70 2024-09-19T04:20:23.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.000+0000 7fc28ad9d640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc278004d50 0x7fc2780051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:23.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc281d9b640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc278004810 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc274009ab0 tx=0x7fc27401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8d22ac4d8c156b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740b2070 con 0x7fc278003f70 2024-09-19T04:20:23.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2740a5640 con 0x7fc278003f70 2024-09-19T04:20:23.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc289d9b640 1 -- 172.21.15.76:0/2291203365 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2740096b0 con 0x7fc278003f70 2024-09-19T04:20:23.007 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- 172.21.15.76:0/2291203365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 msgr2=0x7fc278004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 --2- 172.21.15.76:0/2291203365 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc278004810 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fc274009ab0 tx=0x7fc27401ca50 comp rx=0 tx=0).stop 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28ad9d640 1 -- 172.21.15.76:0/2291203365 reap_dead start 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- 172.21.15.76:0/2291203365 shutdown_connections 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- 172.21.15.76:0/2291203365 >> 172.21.15.76:0/2291203365 conn(0x7fc28c0b3e70 msgr2=0x7fc28c0b4280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- 172.21.15.76:0/2291203365 shutdown_connections 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- 172.21.15.76:0/2291203365 wait complete. 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 Processor -- start 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- start start 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc28c157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c06cd30 0x7fc28c13ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc28c158410 0x7fc28c158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c1416e0 con 0x7fc278003f70 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c141560 con 0x7fc28c158410 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc290fbe640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fc28c0d15e0 con 0x7fc28c06cd30 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc28c157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc28c157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57410/0 (socket says 172.21.15.76:57410) 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 -- 172.21.15.76:0/577922417 learned_addr learned my addr 172.21.15.76:0/577922417 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28ad9d640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc28c158410 0x7fc28c158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 -- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c06cd30 msgr2=0x7fc28c13ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc281d9b640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c06cd30 0x7fc28c13ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c06cd30 0x7fc28c13ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 -- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc28c158410 msgr2=0x7fc28c158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc28c158410 0x7fc28c158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 -- 172.21.15.76:0/577922417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c06d4a0 con 0x7fc278003f70 2024-09-19T04:20:23.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc28a59c640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc28c157bb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00e580 tx=0x7fc27c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.004+0000 7fc2837fe640 1 -- 172.21.15.76:0/577922417 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0052e0 con 0x7fc278003f70 2024-09-19T04:20:23.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc2837fe640 1 -- 172.21.15.76:0/577922417 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc27c018070 con 0x7fc278003f70 2024-09-19T04:20:23.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c06d780 con 0x7fc278003f70 2024-09-19T04:20:23.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc2837fe640 1 -- 172.21.15.76:0/577922417 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c003820 con 0x7fc278003f70 2024-09-19T04:20:23.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c06dd40 con 0x7fc278003f70 2024-09-19T04:20:23.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c068930 con 0x7fc278003f70 2024-09-19T04:20:23.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc2837fe640 1 -- 172.21.15.76:0/577922417 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fc27c0039c0 con 0x7fc278003f70 2024-09-19T04:20:23.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc2837fe640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc26c0678f0 0x7fc26c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc2837fe640 1 -- 172.21.15.76:0/577922417 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fc27c085710 con 0x7fc278003f70 2024-09-19T04:20:23.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc281d9b640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc26c0678f0 0x7fc26c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.008+0000 7fc281d9b640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc26c0678f0 0x7fc26c069db0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc2740af000 tx=0x7fc2740046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.012+0000 7fc2837fe640 1 -- 172.21.15.76:0/577922417 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fc27c0560a0 con 0x7fc278003f70 2024-09-19T04:20:23.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc28c0b5840 con 0x7fc278003f70 2024-09-19T04:20:23.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc2837fe640 1 -- 172.21.15.76:0/577922417 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7fc27c005480 con 0x7fc278003f70 2024-09-19T04:20:23.196 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:23.196 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:23.197 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc26c0678f0 msgr2=0x7fc26c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc26c0678f0 0x7fc26c069db0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc2740af000 tx=0x7fc2740046d0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 msgr2=0x7fc28c157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc28c157bb0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00e580 tx=0x7fc27c00ea40 comp rx=0 tx=0).stop 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 shutdown_connections 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fc26c0678f0 0x7fc26c069db0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc28c158410 0x7fc28c158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fc28c06cd30 0x7fc28c13ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 --2- 172.21.15.76:0/577922417 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fc278003f70 0x7fc28c157bb0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.196+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 >> 172.21.15.76:0/577922417 conn(0x7fc28c0b3e70 msgr2=0x7fc28c141f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.200+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 shutdown_connections 2024-09-19T04:20:23.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.200+0000 7fc290fbe640 1 -- 172.21.15.76:0/577922417 wait complete. 2024-09-19T04:20:23.214 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:23.214 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:23.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.316+0000 7f987d464640 1 Processor -- start 2024-09-19T04:20:23.319 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.316+0000 7f987d464640 1 -- start start 2024-09-19T04:20:23.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f987d464640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98780b0f00 0x7f9878166150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f987d464640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f987809ae60 con 0x7f987809aa00 2024-09-19T04:20:23.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f987d464640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98780ae8b0 con 0x7f987815b170 2024-09-19T04:20:23.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f987d464640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f98780aeb90 con 0x7f98780b0f00 2024-09-19T04:20:23.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98777fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f98780b0f00 0x7f9878166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f986e7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f987809aa00 0x7f98780b07f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52796/0 (socket says 172.21.15.76:52796) 2024-09-19T04:20:23.320 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f986e7fc640 1 -- 172.21.15.76:0/1459059380 learned_addr learned my addr 172.21.15.76:0/1459059380 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:23.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98777fe640 1 -- 172.21.15.76:0/1459059380 >> v1:172.21.15.116:6789/0 conn(0x7f987815b170 legacy=0x7f98780af950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98777fe640 1 -- 172.21.15.76:0/1459059380 >> v1:172.21.15.76:6789/0 conn(0x7f987809aa00 legacy=0x7f98780b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98777fe640 1 -- 172.21.15.76:0/1459059380 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98780aee70 con 0x7f98780b0f00 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98777fe640 1 --2- 172.21.15.76:0/1459059380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98780b0f00 0x7f9878166150 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9870009d90 tx=0x7f987001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0042bad83ac592 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98767fc640 1 -- 172.21.15.76:0/1459059380 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98700b2070 con 0x7f98780b0f00 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98767fc640 1 -- 172.21.15.76:0/1459059380 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98700a5860 con 0x7f98780b0f00 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.320+0000 7f98767fc640 1 -- 172.21.15.76:0/1459059380 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98700a5c30 con 0x7f98780b0f00 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- 172.21.15.76:0/1459059380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98780b0f00 msgr2=0x7f9878166150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 --2- 172.21.15.76:0/1459059380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98780b0f00 0x7f9878166150 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f9870009d90 tx=0x7f987001c800 comp rx=0 tx=0).stop 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- 172.21.15.76:0/1459059380 shutdown_connections 2024-09-19T04:20:23.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 --2- 172.21.15.76:0/1459059380 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f98780b0f00 0x7f9878166150 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- 172.21.15.76:0/1459059380 >> 172.21.15.76:0/1459059380 conn(0x7f98780b6800 msgr2=0x7f98780b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- 172.21.15.76:0/1459059380 shutdown_connections 2024-09-19T04:20:23.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- 172.21.15.76:0/1459059380 wait complete. 2024-09-19T04:20:23.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 Processor -- start 2024-09-19T04:20:23.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- start start 2024-09-19T04:20:23.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f987809aa00 0x7f987813f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f98780b0f00 0x7f98780b4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f987815b170 0x7f98780b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f98780b0060 con 0x7f987815b170 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f98780af430 con 0x7f987809aa00 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f987815b550 con 0x7f98780b0f00 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f987815b170 0x7f98780b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f987815b170 0x7f98780b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57428/0 (socket says 172.21.15.76:57428) 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 -- 172.21.15.76:0/2370212567 learned_addr learned my addr 172.21.15.76:0/2370212567 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f9876ffd640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f987809aa00 0x7f987813f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 -- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f98780b0f00 msgr2=0x7f98780b4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f986e7fc640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f98780b0f00 0x7f98780b4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f98780b0f00 0x7f98780b4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 -- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f987809aa00 msgr2=0x7f987813f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f987809aa00 0x7f987813f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 -- 172.21.15.76:0/2370212567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98781556f0 con 0x7f987815b170 2024-09-19T04:20:23.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f98777fe640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f987815b170 0x7f98780b50a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f98700b0040 tx=0x7f98700195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f986ffff640 1 -- 172.21.15.76:0/2370212567 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98700b2050 con 0x7f987815b170 2024-09-19T04:20:23.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f986ffff640 1 -- 172.21.15.76:0/2370212567 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98700be070 con 0x7f987815b170 2024-09-19T04:20:23.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98781559d0 con 0x7f987815b170 2024-09-19T04:20:23.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f986ffff640 1 -- 172.21.15.76:0/2370212567 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98700b9450 con 0x7f987815b170 2024-09-19T04:20:23.328 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.324+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9878154950 con 0x7f987815b170 2024-09-19T04:20:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.328+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98780b07f0 con 0x7f987815b170 2024-09-19T04:20:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.328+0000 7f986ffff640 1 -- 172.21.15.76:0/2370212567 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f98700b95f0 con 0x7f987815b170 2024-09-19T04:20:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.328+0000 7f986ffff640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98580678c0 0x7f9858069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.328+0000 7f986ffff640 1 -- 172.21.15.76:0/2370212567 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f9870121580 con 0x7f987815b170 2024-09-19T04:20:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.328+0000 7f9876ffd640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98580678c0 0x7f9858069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.328+0000 7f9876ffd640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98580678c0 0x7f9858069d80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9868000f70 tx=0x7f9868009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.332+0000 7f986ffff640 1 -- 172.21.15.76:0/2370212567 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9870016480 con 0x7f987815b170 2024-09-19T04:20:23.534 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.532+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f987806e7c0 con 0x7f987815b170 2024-09-19T04:20:23.535 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f986ffff640 1 -- 172.21.15.76:0/2370212567 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7f98700f1fc0 con 0x7f987815b170 2024-09-19T04:20:23.535 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:23.536 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":442,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":694960,"bytes_used":267493376,"bytes_avail":772826619904,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":6568,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-19T04:20:19.871079+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-19T04:20:23.537 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98580678c0 msgr2=0x7f9858069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98580678c0 0x7f9858069d80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9868000f70 tx=0x7f9868009040 comp rx=0 tx=0).stop 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f987815b170 msgr2=0x7f98780b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f987815b170 0x7f98780b50a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f98700b0040 tx=0x7f98700195a0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 shutdown_connections 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f98580678c0 0x7f9858069d80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f987815b170 0x7f98780b50a0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f98780b0f00 0x7f98780b4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 --2- 172.21.15.76:0/2370212567 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f987809aa00 0x7f987813f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 >> 172.21.15.76:0/2370212567 conn(0x7f98780b6800 msgr2=0x7f987806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 shutdown_connections 2024-09-19T04:20:23.538 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.536+0000 7f987d464640 1 -- 172.21.15.76:0/2370212567 wait complete. 2024-09-19T04:20:23.553 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:23.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:23.553 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-09-19T04:20:23.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.660+0000 7f8dfcdfc640 1 Processor -- start 2024-09-19T04:20:23.662 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.660+0000 7f8dfcdfc640 1 -- start start 2024-09-19T04:20:23.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8dfcdfc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 0x7f8df80d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8dfcdfc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8df80d2e90 0x7f8df80cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8dfcdfc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8df80d32b0 con 0x7f8df80cf450 2024-09-19T04:20:23.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8dfcdfc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f8df80d0300 con 0x7f8df80d2e90 2024-09-19T04:20:23.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8dfcdfc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8df806e6c0 con 0x7f8df8161a60 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df6d9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8df80cf450 0x7f8df80d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52808/0 (socket says 172.21.15.76:52808) 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df6d9d640 1 -- 172.21.15.76:0/2742441549 learned_addr learned my addr 172.21.15.76:0/2742441549 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/2742441549 >> v2:172.21.15.116:3300/0 conn(0x7f8df80d2e90 0x7f8df80cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2196590318 0 0) 0x7f8df80d32b0 con 0x7f8df80cf450 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 0x7f8df80d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8de0003530 con 0x7f8df80cf450 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2621975848 0 0) 0x7f8de0003530 con 0x7f8df80cf450 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 msgr2=0x7f8df80d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 0x7f8df80d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 >> v2:172.21.15.116:3300/0 conn(0x7f8df80d2e90 msgr2=0x7f8df80cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 --2- 172.21.15.76:0/2742441549 >> v2:172.21.15.116:3300/0 conn(0x7f8df80d2e90 0x7f8df80cef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df80b0f60 con 0x7f8df80cf450 2024-09-19T04:20:23.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f8df0001a00 con 0x7f8df80cf450 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 >> v1:172.21.15.76:6789/0 conn(0x7f8df80cf450 legacy=0x7f8df80d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8de0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 0x7f8de0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de0004d50 0x7f8de00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8df80d0300 con 0x7f8de00039c0 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8df806e6c0 con 0x7f8de0004d50 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8de000b9d0 con 0x7f8df8161a60 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8de0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 0x7f8de0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df6d9d640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de0004d50 0x7f8de00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 -- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 msgr2=0x7f8de0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df8161a60 0x7f8de0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 -- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de0004d50 msgr2=0x7f8de00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8de0004d50 0x7f8de00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.667 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 -- 172.21.15.76:0/2742441549 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de0003530 con 0x7f8de00039c0 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df659c640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8de0003de0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009ae0 tx=0x7f8de801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1383ff98302f430 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80b2070 con 0x7f8de00039c0 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de80a5610 con 0x7f8de00039c0 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.664+0000 7f8df559a640 1 -- 172.21.15.76:0/2742441549 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80096b0 con 0x7f8de00039c0 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/2742441549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 msgr2=0x7f8de0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 --2- 172.21.15.76:0/2742441549 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8de0003de0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009ae0 tx=0x7f8de801ca40 comp rx=0 tx=0).stop 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df6d9d640 1 -- 172.21.15.76:0/2742441549 reap_dead start 2024-09-19T04:20:23.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/2742441549 shutdown_connections 2024-09-19T04:20:23.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/2742441549 >> 172.21.15.76:0/2742441549 conn(0x7f8df808bf90 msgr2=0x7f8df809d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/2742441549 shutdown_connections 2024-09-19T04:20:23.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/2742441549 wait complete. 2024-09-19T04:20:23.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 Processor -- start 2024-09-19T04:20:23.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- start start 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8df81f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8df81f4350 0x7f8df80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df80d04e0 0x7f8df80d0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8df806d190 con 0x7f8de00039c0 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8df80d1930 con 0x7f8df81f4350 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8df80cf880 con 0x7f8df80d04e0 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8df81f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8df81f79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57438/0 (socket says 172.21.15.76:57438) 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 -- 172.21.15.76:0/4078310195 learned_addr learned my addr 172.21.15.76:0/4078310195 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.671 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8df81f4350 0x7f8df80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df6d9d640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df80d04e0 0x7f8df80d0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 -- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df80d04e0 msgr2=0x7f8df80d0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df80d04e0 0x7f8df80d0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 -- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8df81f4350 msgr2=0x7f8df80cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8df81f4350 0x7f8df80cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 -- 172.21.15.76:0/4078310195 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8df80ce1e0 con 0x7f8de00039c0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8df81f4350 0x7f8df80cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8df659c640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8df81f79d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f8de80b0040 tx=0x7f8de0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8de6ffd640 1 -- 172.21.15.76:0/4078310195 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80bc030 con 0x7f8de00039c0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8de6ffd640 1 -- 172.21.15.76:0/4078310195 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de800f000 con 0x7f8de00039c0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8de6ffd640 1 -- 172.21.15.76:0/4078310195 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80b2040 con 0x7f8de00039c0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8df80ce460 con 0x7f8de00039c0 2024-09-19T04:20:23.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.668+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8df80b84d0 con 0x7f8de00039c0 2024-09-19T04:20:23.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8de6ffd640 1 -- 172.21.15.76:0/4078310195 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8de80bf070 con 0x7f8de00039c0 2024-09-19T04:20:23.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8de6ffd640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8dd80678c0 0x7f8dd8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8dd80678c0 0x7f8dd8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8de6ffd640 1 -- 172.21.15.76:0/4078310195 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8de811fdb0 con 0x7f8de00039c0 2024-09-19T04:20:23.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8dd80678c0 0x7f8dd8069d80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8df80d1030 tx=0x7f8dec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 Processor -- start 2024-09-19T04:20:23.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 -- start start 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 0x5639cf32b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32b690 0x5639cf2de870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf2df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5639cf1c8960 con 0x5639cf32b690 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5639cf10c900 con 0x5639cf32ad70 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5639cf12fd40 con 0x5639cf2dedb0 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32b690 0x5639cf2de870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32b690 0x5639cf2de870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57448/0 (socket says 172.21.15.76:57448) 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 -- 172.21.15.76:0/1071214201 learned_addr learned my addr 172.21.15.76:0/1071214201 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 -- 172.21.15.76:0/1071214201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 msgr2=0x5639cf2df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/1071214201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf2df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.676 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 -- 172.21.15.76:0/1071214201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 msgr2=0x5639cf32b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/1071214201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 0x5639cf32b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 -- 172.21.15.76:0/1071214201 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cf0d6580 con 0x5639cf32b690 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/1071214201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32b690 0x5639cf2de870 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f8dec012ce0 tx=0x7f8dec015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f470c1f85da057 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dd7fff640 1 -- 172.21.15.76:0/1071214201 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec022440 con 0x5639cf32b690 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dd7fff640 1 -- 172.21.15.76:0/1071214201 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dec027070 con 0x5639cf32b690 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dd7fff640 1 -- 172.21.15.76:0/1071214201 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dec0227e0 con 0x5639cf32b690 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1071214201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32b690 msgr2=0x5639cf2de870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.672+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1071214201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32b690 0x5639cf2de870 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f8dec012ce0 tx=0x7f8dec015000 comp rx=0 tx=0).stop 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1071214201 shutdown_connections 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1071214201 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf2df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1071214201 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32b690 0x5639cf2de870 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1071214201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 0x5639cf32b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1071214201 >> 172.21.15.76:0/1071214201 conn(0x5639cf30c040 msgr2=0x5639cf30e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1071214201 shutdown_connections 2024-09-19T04:20:23.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1071214201 wait complete. 2024-09-19T04:20:23.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- start start 2024-09-19T04:20:23.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 Processor -- start 2024-09-19T04:20:23.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf2dedb0 0x5639cf331ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf32ad70 0x5639cf332220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32b690 0x5639cf332760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5639cf12bd00 con 0x5639cf2dedb0 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x5639cf0eaaa0 con 0x5639cf32b690 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5639cf0c9340 con 0x5639cf32ad70 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 10 client.0 _ll_register_callbacks cb 0x5639cf2cc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf2dedb0 0x5639cf331ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf2dedb0 0x5639cf331ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57450/0 (socket says 172.21.15.76:57450) 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 -- 172.21.15.76:0/1569584561 learned_addr learned my addr 172.21.15.76:0/1569584561 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 -- 172.21.15.76:0/1569584561 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf32ad70 msgr2=0x5639cf332220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf32ad70 0x5639cf332220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.682 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 -- 172.21.15.76:0/1569584561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32b690 msgr2=0x5639cf332760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32b690 0x5639cf332760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 -- 172.21.15.76:0/1569584561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cf0d2dc0 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df659c640 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf2dedb0 0x5639cf331ce0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f8de80eff20 tx=0x7f8de80f0750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8261a593a7f6a86b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 1 -- 172.21.15.76:0/1569584561 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80f8b70 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 1 -- 172.21.15.76:0/1569584561 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8de8105c90 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8dec027070 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 1 -- 172.21.15.76:0/1569584561 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f8de8105eb0 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 10 client.8775 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x5639cf0cf7f0 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 1 -- 172.21.15.76:0/1569584561 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8dec027070 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 1 -- 172.21.15.76:0/1569584561 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f8de80b5030 con 0x5639cf2dedb0 2024-09-19T04:20:23.683 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 10 client.8775 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 10 client.8775 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 20 client.8775 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf33a0b0 0x5639cf33c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 4 client.8775 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639cf0c9e10 con 0x5639cf33a0b0 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf33a0b0 0x5639cf33c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf33a0b0 0x5639cf33c4a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebba4b47f8635ebf server_cookie=550a6b6369f85317 in_seq=0 out_seq=0 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 10 client.8775 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 1 -- 172.21.15.76:0/1569584561 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5639cf0c9e10 con 0x5639cf33a0b0 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dd57fa640 10 client.8775 handle_command_reply: tid=0 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.676+0000 7f8dffeb8000 1 client.8775 shutdown 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf2dedb0 msgr2=0x5639cf331ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.684 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf2dedb0 0x5639cf331ce0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f8de80eff20 tx=0x7f8de80f0750 comp rx=0 tx=0).stop 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 shutdown_connections 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf33a0b0 0x5639cf33c4a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32b690 0x5639cf332760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf32ad70 0x5639cf332220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1569584561 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf2dedb0 0x5639cf331ce0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dd57fa640 0 client.8775 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 >> 172.21.15.76:0/1569584561 conn(0x5639cf30c040 msgr2=0x5639cf2ddf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 shutdown_connections 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1569584561 wait complete. 2024-09-19T04:20:23.685 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.680+0000 7f8dffeb8000 20 client.8775 trim_cache size 0 max 16384 2024-09-19T04:20:23.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 Processor -- start 2024-09-19T04:20:23.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- start start 2024-09-19T04:20:23.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf30c040 0x5639cf2e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf2e7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 0x5639cf334d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8df82045b0 con 0x5639cf30c040 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8df8203750 con 0x5639cf32ad70 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8df8203e80 con 0x5639cf2dedb0 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf30c040 0x5639cf2e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf30c040 0x5639cf2e6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57460/0 (socket says 172.21.15.76:57460) 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 -- 172.21.15.76:0/1320909119 learned_addr learned my addr 172.21.15.76:0/1320909119 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 -- 172.21.15.76:0/1320909119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 msgr2=0x5639cf2e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 --2- 172.21.15.76:0/1320909119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf2e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 -- 172.21.15.76:0/1320909119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 msgr2=0x5639cf334d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 --2- 172.21.15.76:0/1320909119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 0x5639cf334d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 -- 172.21.15.76:0/1320909119 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639cf0ed470 con 0x5639cf30c040 2024-09-19T04:20:23.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8df659c640 1 --2- 172.21.15.76:0/1320909119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf30c040 0x5639cf2e6c00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8de801c870 tx=0x7f8de80f2000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45624b9cdb1c3e81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dd77fe640 1 -- 172.21.15.76:0/1320909119 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80ec230 con 0x5639cf30c040 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dd77fe640 1 -- 172.21.15.76:0/1320909119 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de80f8b70 con 0x5639cf30c040 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dd77fe640 1 -- 172.21.15.76:0/1320909119 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8de80ef3e0 con 0x5639cf30c040 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1320909119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf30c040 msgr2=0x5639cf2e6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1320909119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf30c040 0x5639cf2e6c00 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f8de801c870 tx=0x7f8de80f2000 comp rx=0 tx=0).stop 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1320909119 shutdown_connections 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1320909119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf32ad70 0x5639cf334d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1320909119 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf2e7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/1320909119 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf30c040 0x5639cf2e6c00 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1320909119 >> 172.21.15.76:0/1320909119 conn(0x5639cf2e3770 msgr2=0x5639cf3322c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1320909119 shutdown_connections 2024-09-19T04:20:23.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- 172.21.15.76:0/1320909119 wait complete. 2024-09-19T04:20:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 -- start start 2024-09-19T04:20:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.696+0000 7f8dffeb8000 1 Processor -- start 2024-09-19T04:20:23.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf3380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf30c040 0x5639cf338620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32ad70 0x5639cf39af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x5639cf13dfd0 con 0x5639cf32ad70 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8df8202800 con 0x5639cf30c040 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x5639cf2ea3f0 con 0x5639cf2dedb0 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 10 client.0 _ll_register_callbacks cb 0x7f8de80092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32ad70 0x5639cf39af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32ad70 0x5639cf39af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57476/0 (socket says 172.21.15.76:57476) 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 -- 172.21.15.76:0/2021086726 learned_addr learned my addr 172.21.15.76:0/2021086726 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df659c640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf3380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf30c040 0x5639cf338620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 -- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 msgr2=0x5639cf3380e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf3380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 -- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf30c040 msgr2=0x5639cf338620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf30c040 0x5639cf338620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 -- 172.21.15.76:0/2021086726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639cf0c3640 con 0x5639cf32ad70 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df5d9b640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf30c040 0x5639cf338620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df6d9d640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32ad70 0x5639cf39af60 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8df808e010 tx=0x7f8df0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580a9bc060c5e80d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 1 -- 172.21.15.76:0/2021086726 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df0010040 con 0x5639cf32ad70 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 1 -- 172.21.15.76:0/2021086726 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8df0014630 con 0x5639cf32ad70 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8de80f8b70 con 0x5639cf32ad70 2024-09-19T04:20:23.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 1 -- 172.21.15.76:0/2021086726 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f8df0004b70 con 0x5639cf32ad70 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 10 client.8781 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x5639cf0c73b0 con 0x5639cf32ad70 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 1 -- 172.21.15.76:0/2021086726 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8de80f8b70 con 0x5639cf32ad70 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 1 -- 172.21.15.76:0/2021086726 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f8df0013070 con 0x5639cf32ad70 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 10 client.8781 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 10 client.8781 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 20 client.8781 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf32b690 0x5639cf39be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 4 client.8781 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub resume"}) -- 0x5639cf13d470 con 0x5639cf32b690 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df659c640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf32b690 0x5639cf39be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8df659c640 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf32b690 0x5639cf39be60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=453dd78a292d3983 server_cookie=6bcd934e5bcf7704 in_seq=0 out_seq=0 2024-09-19T04:20:23.706 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 10 client.8781 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 1 -- 172.21.15.76:0/2021086726 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5639cf13d470 con 0x5639cf32b690 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 10 client.8781 handle_command_reply: tid=0 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 client.8781 shutdown 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32ad70 msgr2=0x5639cf39af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32ad70 0x5639cf39af60 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f8df808e010 tx=0x7f8df0007780 comp rx=0 tx=0).stop 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 shutdown_connections 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x5639cf32b690 0x5639cf39be60 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x5639cf32ad70 0x5639cf39af60 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x5639cf30c040 0x5639cf338620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dd57fa640 0 client.8781 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 --2- 172.21.15.76:0/2021086726 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x5639cf2dedb0 0x5639cf3380e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 >> 172.21.15.76:0/2021086726 conn(0x5639cf2e3770 msgr2=0x5639cf2e5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.707 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 shutdown_connections 2024-09-19T04:20:23.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 1 -- 172.21.15.76:0/2021086726 wait complete. 2024-09-19T04:20:23.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dffeb8000 20 client.8781 trim_cache size 0 max 16384 2024-09-19T04:20:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:23.708 INFO:teuthology.orchestra.run.smithi076.stdout: "return_code": 0 2024-09-19T04:20:23.708 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:23.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8dd80678c0 msgr2=0x7f8dd8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8dd80678c0 0x7f8dd8069d80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8df80d1030 tx=0x7f8dec008040 comp rx=0 tx=0).stop 2024-09-19T04:20:23.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 msgr2=0x7f8df81f79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8df81f79d0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f8de80b0040 tx=0x7f8de0005840 comp rx=0 tx=0).stop 2024-09-19T04:20:23.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 shutdown_connections 2024-09-19T04:20:23.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8dd80678c0 0x7f8dd8069d80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.709 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8df80d04e0 0x7f8df80d0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8df81f4350 0x7f8df80cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 --2- 172.21.15.76:0/4078310195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8de00039c0 0x7f8df81f79d0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 >> 172.21.15.76:0/4078310195 conn(0x7f8df808bf90 msgr2=0x7f8df80b46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 shutdown_connections 2024-09-19T04:20:23.710 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.700+0000 7f8dfcdfc640 1 -- 172.21.15.76:0/4078310195 wait complete. 2024-09-19T04:20:23.717 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:23.717 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:1 scrub status 2024-09-19T04:20:23.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d6a68a640 1 Processor -- start 2024-09-19T04:20:23.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d6a68a640 1 -- start start 2024-09-19T04:20:23.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d6a68a640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d6415c2a0 0x7f9d640aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d6a68a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d6406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d6a68a640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d640b3290 con 0x7f9d6415bec0 2024-09-19T04:20:23.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d6a68a640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9d640b3570 con 0x7f9d6415c2a0 2024-09-19T04:20:23.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d6a68a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d640b3850 con 0x7f9d640af2a0 2024-09-19T04:20:23.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d68c27640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d6406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d68c27640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d6406e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:35724/0 (socket says 172.21.15.76:35724) 2024-09-19T04:20:23.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d68c27640 1 -- 172.21.15.76:0/2603353490 learned_addr learned my addr 172.21.15.76:0/2603353490 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d637fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d6415c2a0 0x7f9d640aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d637fe640 1 -- 172.21.15.76:0/2603353490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 msgr2=0x7f9d6406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d637fe640 1 --2- 172.21.15.76:0/2603353490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d6406e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d637fe640 1 -- 172.21.15.76:0/2603353490 >> v1:172.21.15.116:6789/0 conn(0x7f9d6415bec0 legacy=0x7f9d640ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.828+0000 7f9d637fe640 1 -- 172.21.15.76:0/2603353490 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d640b3b30 con 0x7f9d6415c2a0 2024-09-19T04:20:23.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d637fe640 1 --2- 172.21.15.76:0/2603353490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415c2a0 0x7f9d640aed60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9d580097c0 tx=0x7f9d5801c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2c4da2a488d424a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d62ffd640 1 -- 172.21.15.76:0/2603353490 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580a7070 con 0x7f9d6415c2a0 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d62ffd640 1 -- 172.21.15.76:0/2603353490 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d580b0070 con 0x7f9d6415c2a0 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d62ffd640 1 -- 172.21.15.76:0/2603353490 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580ab890 con 0x7f9d6415c2a0 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2603353490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415c2a0 msgr2=0x7f9d640aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2603353490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415c2a0 0x7f9d640aed60 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9d580097c0 tx=0x7f9d5801c470 comp rx=0 tx=0).stop 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2603353490 shutdown_connections 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2603353490 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d6406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2603353490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415c2a0 0x7f9d640aed60 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2603353490 >> 172.21.15.76:0/2603353490 conn(0x7f9d640b1f90 msgr2=0x7f9d640b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2603353490 shutdown_connections 2024-09-19T04:20:23.835 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.832+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2603353490 wait complete. 2024-09-19T04:20:23.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 Processor -- start 2024-09-19T04:20:23.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 -- start start 2024-09-19T04:20:23.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d640991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415bec0 0x7f9d64150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6415c2a0 0x7f9d6414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d6406ed50 con 0x7f9d6415c2a0 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d640af6b0 con 0x7f9d6415bec0 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d6406eed0 con 0x7f9d640af2a0 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6415c2a0 0x7f9d6414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6415c2a0 0x7f9d6414b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57484/0 (socket says 172.21.15.76:57484) 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 -- 172.21.15.76:0/2191592323 learned_addr learned my addr 172.21.15.76:0/2191592323 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d637fe640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415bec0 0x7f9d64150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d63fff640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d640991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 -- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 msgr2=0x7f9d640991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d640991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 -- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415bec0 msgr2=0x7f9d64150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415bec0 0x7f9d64150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 -- 172.21.15.76:0/2191592323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d64152400 con 0x7f9d6415c2a0 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d68c27640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6415c2a0 0x7f9d6414b510 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e880 tx=0x7f9d5400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d60ff9640 1 -- 172.21.15.76:0/2191592323 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54017070 con 0x7f9d6415c2a0 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d60ff9640 1 -- 172.21.15.76:0/2191592323 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5400c940 con 0x7f9d6415c2a0 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d64149e30 con 0x7f9d6415c2a0 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d60ff9640 1 -- 172.21.15.76:0/2191592323 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400cd30 con 0x7f9d6415c2a0 2024-09-19T04:20:23.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6414a3f0 con 0x7f9d6415c2a0 2024-09-19T04:20:23.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d60ff9640 1 -- 172.21.15.76:0/2191592323 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9d5401a070 con 0x7f9d6415c2a0 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.836+0000 7f9d60ff9640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d500678c0 0x7f9d50069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d60ff9640 1 -- 172.21.15.76:0/2191592323 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f9d54085e10 con 0x7f9d6415c2a0 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d63fff640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d500678c0 0x7f9d50069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d63fff640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d500678c0 0x7f9d50069d80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d6415b8b0 tx=0x7f9d5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 Processor -- start 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- start start 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c933d70 0x55870c934150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c8e7870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c8e8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55870c7d1960 con 0x55870c8e7db0 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55870c715900 con 0x55870c934690 2024-09-19T04:20:23.843 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55870c738d40 con 0x55870c933d70 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c8e8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c8e8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57494/0 (socket says 172.21.15.76:57494) 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 -- 172.21.15.76:0/2683145944 learned_addr learned my addr 172.21.15.76:0/2683145944 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 -- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c933d70 msgr2=0x55870c934150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d637fe640 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c8e7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d63fff640 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c933d70 0x55870c934150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c933d70 0x55870c934150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 -- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 msgr2=0x55870c8e7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c8e7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 -- 172.21.15.76:0/2683145944 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55870c6df580 con 0x55870c8e7db0 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d637fe640 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c8e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:23.844 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d68c27640 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c8e8190 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9d540841b0 tx=0x7f9d54056960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c05548a13cf16ff server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d49ffb640 1 -- 172.21.15.76:0/2683145944 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5406e020 con 0x55870c8e7db0 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d49ffb640 1 -- 172.21.15.76:0/2683145944 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5405ed00 con 0x55870c8e7db0 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d49ffb640 1 -- 172.21.15.76:0/2683145944 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54021040 con 0x55870c8e7db0 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- 172.21.15.76:0/2683145944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 msgr2=0x55870c8e8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c8e8190 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f9d540841b0 tx=0x7f9d54056960 comp rx=0 tx=0).stop 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- 172.21.15.76:0/2683145944 shutdown_connections 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c8e8190 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c8e7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 --2- 172.21.15.76:0/2683145944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c933d70 0x55870c934150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- 172.21.15.76:0/2683145944 >> 172.21.15.76:0/2683145944 conn(0x55870c915040 msgr2=0x55870c917430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- 172.21.15.76:0/2683145944 shutdown_connections 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- 172.21.15.76:0/2683145944 wait complete. 2024-09-19T04:20:23.845 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 -- start start 2024-09-19T04:20:23.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.840+0000 7f9d6d746000 1 Processor -- start 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 0x55870c93ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c93b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c93b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55870c734d00 con 0x55870c933d70 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55870c6f3aa0 con 0x55870c934690 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55870c6d2340 con 0x55870c8e7db0 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 10 client.0 _ll_register_callbacks cb 0x55870c8d5760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:23.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c93b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c93b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57498/0 (socket says 172.21.15.76:57498) 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 -- 172.21.15.76:0/3650304690 learned_addr learned my addr 172.21.15.76:0/3650304690 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d68c27640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c93b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 0x55870c93ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 -- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 msgr2=0x55870c93ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 0x55870c93ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 -- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 msgr2=0x55870c93b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c93b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 -- 172.21.15.76:0/3650304690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870c6dbdc0 con 0x55870c933d70 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d68c27640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c93b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d637fe640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c93b220 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9d5801c950 tx=0x7f9d580046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d67137a223065 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 1 -- 172.21.15.76:0/3650304690 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58006e20 con 0x55870c933d70 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 1 -- 172.21.15.76:0/3650304690 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f9d580abcc0 con 0x55870c933d70 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5405ed00 con 0x55870c933d70 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 1 -- 172.21.15.76:0/3650304690 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f9d580b43f0 con 0x55870c933d70 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 10 client.8790 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55870c6d87f0 con 0x55870c933d70 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 1 -- 172.21.15.76:0/3650304690 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d5405ed00 con 0x55870c933d70 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 1 -- 172.21.15.76:0/3650304690 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f9d580b4590 con 0x55870c933d70 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 10 client.8790 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 10 client.8790 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 20 client.8790 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:23.851 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c9430b0 0x55870c9454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 4 client.8790 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:23.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55870c6d2e10 con 0x55870c9430b0 2024-09-19T04:20:23.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c9430b0 0x55870c9454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d63fff640 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c9430b0 0x55870c9454a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52b263ceb0a4e09d server_cookie=a3a1da369f75ac4a in_seq=0 out_seq=0 2024-09-19T04:20:23.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 10 client.8790 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:23.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 1 -- 172.21.15.76:0/3650304690 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55870c6d2e10 con 0x55870c9430b0 2024-09-19T04:20:23.852 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 10 client.8790 handle_command_reply: tid=0 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 client.8790 shutdown 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 msgr2=0x55870c93b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c93b220 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f9d5801c950 tx=0x7f9d580046d0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 shutdown_connections 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c9430b0 0x55870c9454a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c93b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c93b220 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d42ffd640 0 client.8790 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 --2- 172.21.15.76:0/3650304690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 0x55870c93ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 >> 172.21.15.76:0/3650304690 conn(0x55870c915040 msgr2=0x55870c8e6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.853 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 shutdown_connections 2024-09-19T04:20:23.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 1 -- 172.21.15.76:0/3650304690 wait complete. 2024-09-19T04:20:23.854 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.844+0000 7f9d6d746000 20 client.8790 trim_cache size 0 max 16384 2024-09-19T04:20:23.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 Processor -- start 2024-09-19T04:20:23.863 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- start start 2024-09-19T04:20:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 0x55870c948d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c9413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c941930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64201d10 con 0x55870c933d70 2024-09-19T04:20:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d64200eb0 con 0x55870c934690 2024-09-19T04:20:23.866 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9d642015e0 con 0x55870c8e7db0 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c9413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c9413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57514/0 (socket says 172.21.15.76:57514) 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 -- 172.21.15.76:0/184323555 learned_addr learned my addr 172.21.15.76:0/184323555 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 -- 172.21.15.76:0/184323555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 msgr2=0x55870c948d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 --2- 172.21.15.76:0/184323555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 0x55870c948d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 -- 172.21.15.76:0/184323555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 msgr2=0x55870c941930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 --2- 172.21.15.76:0/184323555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c941930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 -- 172.21.15.76:0/184323555 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55870c6f6470 con 0x55870c933d70 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d637fe640 1 --2- 172.21.15.76:0/184323555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c9413f0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f9d5801c4b0 tx=0x7f9d580a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1768b1ad5767e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d497fa640 1 -- 172.21.15.76:0/184323555 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580b4590 con 0x55870c933d70 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d497fa640 1 -- 172.21.15.76:0/184323555 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d58000e50 con 0x55870c933d70 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d497fa640 1 -- 172.21.15.76:0/184323555 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58004260 con 0x55870c933d70 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- 172.21.15.76:0/184323555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 msgr2=0x55870c9413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 --2- 172.21.15.76:0/184323555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c9413f0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f9d5801c4b0 tx=0x7f9d580a9000 comp rx=0 tx=0).stop 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- 172.21.15.76:0/184323555 shutdown_connections 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 --2- 172.21.15.76:0/184323555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c934690 0x55870c941930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 --2- 172.21.15.76:0/184323555 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c933d70 0x55870c9413f0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 --2- 172.21.15.76:0/184323555 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c8e7db0 0x55870c948d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- 172.21.15.76:0/184323555 >> 172.21.15.76:0/184323555 conn(0x55870c915040 msgr2=0x55870c940ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- 172.21.15.76:0/184323555 shutdown_connections 2024-09-19T04:20:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- 172.21.15.76:0/184323555 wait complete. 2024-09-19T04:20:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 -- start start 2024-09-19T04:20:23.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.864+0000 7f9d6d746000 1 Processor -- start 2024-09-19T04:20:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c947e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c933d70 0x55870c963790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c934690 0x55870c963cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55870c746fd0 con 0x55870c8e7db0 2024-09-19T04:20:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9d641578f0 con 0x55870c933d70 2024-09-19T04:20:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55870c778780 con 0x55870c934690 2024-09-19T04:20:23.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 10 client.0 _ll_register_callbacks cb 0x7f9d540090e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c947e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c947e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57520/0 (socket says 172.21.15.76:57520) 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 -- 172.21.15.76:0/780526624 learned_addr learned my addr 172.21.15.76:0/780526624 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 -- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c934690 msgr2=0x55870c963cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d68c27640 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c934690 0x55870c963cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d637fe640 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c933d70 0x55870c963790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c934690 0x55870c963cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 -- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c933d70 msgr2=0x55870c963790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c933d70 0x55870c963790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 -- 172.21.15.76:0/780526624 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870c6cc640 con 0x55870c8e7db0 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d63fff640 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c947e20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c003f20 tx=0x7f9d5c001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5bfb67ba40b808 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:23.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 1 -- 172.21.15.76:0/780526624 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5c0057e0 con 0x55870c8e7db0 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 1 -- 172.21.15.76:0/780526624 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f9d5c02dcc0 con 0x55870c8e7db0 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d58000e50 con 0x55870c8e7db0 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 1 -- 172.21.15.76:0/780526624 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f9d5c024420 con 0x55870c8e7db0 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 10 client.8796 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55870c6d03b0 con 0x55870c8e7db0 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 1 -- 172.21.15.76:0/780526624 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d58000e50 con 0x55870c8e7db0 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 1 -- 172.21.15.76:0/780526624 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f9d5c029020 con 0x55870c8e7db0 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 10 client.8796 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 10 client.8796 resolve_mds: resolved 7:1 to role '7:1' aka daemon mds.c 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 20 client.8796 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c964210 0x55870c9a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 4 client.8796 mds_command: new command op to 8115 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:23.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 --> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55870c746470 con 0x55870c964210 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d637fe640 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c964210 0x55870c9a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 10 client.8796 ms_handle_connect on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d637fe640 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c964210 0x55870c9a7d20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2131de689d3a690e server_cookie=9e46fb9b1347bb59 in_seq=0 out_seq=0 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 1 -- 172.21.15.76:0/780526624 <== mds.1 v2:172.21.15.76:6834/2501269734 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55870c746470 con 0x55870c964210 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 10 client.8796 handle_command_reply: tid=0 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 client.8796 shutdown 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 msgr2=0x55870c947e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c947e20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c003f20 tx=0x7f9d5c001260 comp rx=0 tx=0).stop 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 shutdown_connections 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:6834/2501269734,v1:172.21.15.76:6835/2501269734] conn(0x55870c964210 0x55870c9a7d20 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.875 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55870c934690 0x55870c963cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d42ffd640 0 client.8796 ms_handle_reset on v2:172.21.15.76:6834/2501269734 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55870c933d70 0x55870c963790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 --2- 172.21.15.76:0/780526624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55870c8e7db0 0x55870c947e20 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 >> 172.21.15.76:0/780526624 conn(0x55870c915040 msgr2=0x55870c9412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 shutdown_connections 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 1 -- 172.21.15.76:0/780526624 wait complete. 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6d746000 20 client.8796 trim_cache size 0 max 16384 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:23.876 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:23.877 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d500678c0 msgr2=0x7f9d50069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d500678c0 0x7f9d50069d80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f9d6415b8b0 tx=0x7f9d5c009040 comp rx=0 tx=0).stop 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6415c2a0 msgr2=0x7f9d6414b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6415c2a0 0x7f9d6414b510 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f9d5400e880 tx=0x7f9d5400ed40 comp rx=0 tx=0).stop 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 shutdown_connections 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9d500678c0 0x7f9d50069d80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9d6415c2a0 0x7f9d6414b510 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d6415bec0 0x7f9d64150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.878 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 --2- 172.21.15.76:0/2191592323 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9d640af2a0 0x7f9d640991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 >> 172.21.15.76:0/2191592323 conn(0x7f9d640b1f90 msgr2=0x7f9d640b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:23.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 shutdown_connections 2024-09-19T04:20:23.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.868+0000 7f9d6a68a640 1 -- 172.21.15.76:0/2191592323 wait complete. 2024-09-19T04:20:23.885 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:23.885 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:23.885 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:2 scrub status 2024-09-19T04:20:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab372b640 1 Processor -- start 2024-09-19T04:20:23.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab372b640 1 -- start start 2024-09-19T04:20:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab372b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5aac0b39a0 0x7f5aac06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab372b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac06e4b0 0x7f5aac072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab372b640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5aac06ec60 con 0x7f5aac1587a0 2024-09-19T04:20:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab372b640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5aac06f870 con 0x7f5aac0b39a0 2024-09-19T04:20:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab372b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac06ef40 con 0x7f5aac06e4b0 2024-09-19T04:20:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab0cc6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5aac0b39a0 0x7f5aac06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab0cc6640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5aac0b39a0 0x7f5aac06df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57528/0 (socket says 172.21.15.76:57528) 2024-09-19T04:20:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab0cc6640 1 -- 172.21.15.76:0/1291394524 learned_addr learned my addr 172.21.15.76:0/1291394524 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab1cc8640 1 --2- 172.21.15.76:0/1291394524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac06e4b0 0x7f5aac072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab0cc6640 1 -- 172.21.15.76:0/1291394524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac06e4b0 msgr2=0x7f5aac072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/1291394524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac06e4b0 0x7f5aac072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab0cc6640 1 -- 172.21.15.76:0/1291394524 >> v1:172.21.15.116:6789/0 conn(0x7f5aac1587a0 legacy=0x7f5aac0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:23.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:23.996+0000 7f5ab0cc6640 1 -- 172.21.15.76:0/1291394524 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac06f220 con 0x7f5aac0b39a0 2024-09-19T04:20:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/1291394524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0b39a0 0x7f5aac06df70 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009870 tx=0x7f5aa801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f417aef4ad6dbea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5aa3fff640 1 -- 172.21.15.76:0/1291394524 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80b1070 con 0x7f5aac0b39a0 2024-09-19T04:20:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5aa3fff640 1 -- 172.21.15.76:0/1291394524 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa80ac630 con 0x7f5aac0b39a0 2024-09-19T04:20:23.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5aa3fff640 1 -- 172.21.15.76:0/1291394524 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80a8050 con 0x7f5aac0b39a0 2024-09-19T04:20:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- 172.21.15.76:0/1291394524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0b39a0 msgr2=0x7f5aac06df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 --2- 172.21.15.76:0/1291394524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0b39a0 0x7f5aac06df70 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8009870 tx=0x7f5aa801c4f0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- 172.21.15.76:0/1291394524 shutdown_connections 2024-09-19T04:20:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 --2- 172.21.15.76:0/1291394524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac06e4b0 0x7f5aac072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 --2- 172.21.15.76:0/1291394524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac0b39a0 0x7f5aac06df70 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- 172.21.15.76:0/1291394524 >> 172.21.15.76:0/1291394524 conn(0x7f5aac0b1f90 msgr2=0x7f5aac0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- 172.21.15.76:0/1291394524 shutdown_connections 2024-09-19T04:20:24.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- 172.21.15.76:0/1291394524 wait complete. 2024-09-19T04:20:24.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 Processor -- start 2024-09-19T04:20:24.001 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- start start 2024-09-19T04:20:24.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac06e4b0 0x7f5aac155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac0b39a0 0x7f5aac1530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aac1587a0 0x7f5aac150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac073030 con 0x7f5aac06e4b0 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac072eb0 con 0x7f5aac1587a0 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab372b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac0731b0 con 0x7f5aac0b39a0 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac06e4b0 0x7f5aac155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac06e4b0 0x7f5aac155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57544/0 (socket says 172.21.15.76:57544) 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 -- 172.21.15.76:0/3933907331 learned_addr learned my addr 172.21.15.76:0/3933907331 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac0b39a0 0x7f5aac1530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab1cc8640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aac1587a0 0x7f5aac150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 -- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac0b39a0 msgr2=0x7f5aac1530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.004 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac0b39a0 0x7f5aac1530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 -- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aac1587a0 msgr2=0x7f5aac150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aac1587a0 0x7f5aac150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 -- 172.21.15.76:0/3933907331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aac154e30 con 0x7f5aac06e4b0 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.000+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac06e4b0 0x7f5aac155bb0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b440 tx=0x7f5a9c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5aa1ffb640 1 -- 172.21.15.76:0/3933907331 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c015020 con 0x7f5aac06e4b0 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5aa1ffb640 1 -- 172.21.15.76:0/3933907331 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c011890 con 0x7f5aac06e4b0 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5aa1ffb640 1 -- 172.21.15.76:0/3933907331 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c011c80 con 0x7f5aac06e4b0 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aac1fb730 con 0x7f5aac06e4b0 2024-09-19T04:20:24.005 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aac1553b0 con 0x7f5aac06e4b0 2024-09-19T04:20:24.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5aa1ffb640 1 -- 172.21.15.76:0/3933907331 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5a9c018020 con 0x7f5aac06e4b0 2024-09-19T04:20:24.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5aa1ffb640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a900678c0 0x7f5a90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5aa1ffb640 1 -- 172.21.15.76:0/3933907331 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f5a9c085c90 con 0x7f5aac06e4b0 2024-09-19T04:20:24.008 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a900678c0 0x7f5a90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a900678c0 0x7f5a90069d80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80ae000 tx=0x7f5aa8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 Processor -- start 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 -- start start 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db19d70 0x55897db1a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897dacd870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 0x55897dace190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55897d9b7960 con 0x55897db19d70 2024-09-19T04:20:24.009 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55897d8fb900 con 0x55897dacddb0 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.004+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55897d91ed40 con 0x55897db1a690 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db19d70 0x55897db1a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db19d70 0x55897db1a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57556/0 (socket says 172.21.15.76:57556) 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/860817912 learned_addr learned my addr 172.21.15.76:0/860817912 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/860817912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 msgr2=0x55897dacd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/860817912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897dacd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/860817912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 msgr2=0x55897dace190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/860817912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 0x55897dace190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/860817912 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897d8c5580 con 0x55897db19d70 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/860817912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db19d70 0x55897db1a150 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c083fa0 tx=0x7f5a9c011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997cecb1cb3dea70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.010 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a8effd640 1 -- 172.21.15.76:0/860817912 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c05eaf0 con 0x55897db19d70 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a8effd640 1 -- 172.21.15.76:0/860817912 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c053fb0 con 0x55897db19d70 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a8effd640 1 -- 172.21.15.76:0/860817912 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c052580 con 0x55897db19d70 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/860817912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db19d70 msgr2=0x55897db1a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/860817912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db19d70 0x55897db1a150 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c083fa0 tx=0x7f5a9c011f50 comp rx=0 tx=0).stop 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/860817912 shutdown_connections 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/860817912 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 0x55897dace190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/860817912 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897dacd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/860817912 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db19d70 0x55897db1a150 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/860817912 >> 172.21.15.76:0/860817912 conn(0x55897dafb040 msgr2=0x55897dafd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/860817912 shutdown_connections 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/860817912 wait complete. 2024-09-19T04:20:24.011 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- start start 2024-09-19T04:20:24.012 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 Processor -- start 2024-09-19T04:20:24.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db20ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 0x55897db21220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897db21760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55897d91ad00 con 0x55897dacddb0 2024-09-19T04:20:24.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55897d8d9aa0 con 0x55897db19d70 2024-09-19T04:20:24.015 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55897d8b8340 con 0x55897db1a690 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 10 client.0 _ll_register_callbacks cb 0x55897dabb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db20ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db20ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57568/0 (socket says 172.21.15.76:57568) 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/2213437558 learned_addr learned my addr 172.21.15.76:0/2213437558 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 msgr2=0x55897db21760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897db21760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 msgr2=0x55897db21220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 0x55897db21220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.016 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 -- 172.21.15.76:0/2213437558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897d8c1dc0 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db20ce0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c05e5c0 tx=0x7f5a9c01c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530808c51cd0b3b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a87fff640 1 -- 172.21.15.76:0/2213437558 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c06f070 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a87fff640 1 -- 172.21.15.76:0/2213437558 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f5a9c0743f0 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a9c053fb0 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a87fff640 1 -- 172.21.15.76:0/2213437558 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c05eaf0 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 10 client.8808 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55897d8be7f0 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a87fff640 1 -- 172.21.15.76:0/2213437558 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5a9c053fb0 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5a87fff640 1 -- 172.21.15.76:0/2213437558 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f5a9c055360 con 0x55897dacddb0 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 10 client.8808 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:24.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 10 client.8808 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 20 client.8808 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db290b0 0x55897db2b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 4 client.8808 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.008+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897d8b8e10 con 0x55897db290b0 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db290b0 0x55897db2b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5a87fff640 10 client.8808 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db290b0 0x55897db2b4a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fe1d72ce06467057 server_cookie=4ea9832efb224fdf in_seq=0 out_seq=0 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5a87fff640 1 -- 172.21.15.76:0/2213437558 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55897d8b8e10 con 0x55897db290b0 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5a87fff640 10 client.8808 handle_command_reply: tid=0 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 client.8808 shutdown 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 msgr2=0x55897db20ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db20ce0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c05e5c0 tx=0x7f5a9c01c040 comp rx=0 tx=0).stop 2024-09-19T04:20:24.018 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 shutdown_connections 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db290b0 0x55897db2b4a0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897db21760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5a87fff640 0 client.8808 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 0x55897db21220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/2213437558 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db20ce0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 >> 172.21.15.76:0/2213437558 conn(0x55897dafb040 msgr2=0x55897daccf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 shutdown_connections 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 1 -- 172.21.15.76:0/2213437558 wait complete. 2024-09-19T04:20:24.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.012+0000 7f5ab67e7000 20 client.8808 trim_cache size 0 max 16384 2024-09-19T04:20:24.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 Processor -- start 2024-09-19T04:20:24.031 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- start start 2024-09-19T04:20:24.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 0x55897db2e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db19d70 0x55897db271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db1a690 0x55897db27720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac203f30 con 0x55897db1a690 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5aac2030d0 con 0x55897dacddb0 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5aac203800 con 0x55897db19d70 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db1a690 0x55897db27720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db1a690 0x55897db27720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57576/0 (socket says 172.21.15.76:57576) 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 -- 172.21.15.76:0/1369118784 learned_addr learned my addr 172.21.15.76:0/1369118784 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 -- 172.21.15.76:0/1369118784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db19d70 msgr2=0x55897db271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 --2- 172.21.15.76:0/1369118784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db19d70 0x55897db271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 -- 172.21.15.76:0/1369118784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 msgr2=0x55897db2e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 --2- 172.21.15.76:0/1369118784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 0x55897db2e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 -- 172.21.15.76:0/1369118784 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897d8dc470 con 0x55897db1a690 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab1cc8640 1 --2- 172.21.15.76:0/1369118784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db1a690 0x55897db27720 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4007db0 tx=0x7f5aa400e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7925882339bcad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5a8e7fc640 1 -- 172.21.15.76:0/1369118784 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4018630 con 0x55897db1a690 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5a8e7fc640 1 -- 172.21.15.76:0/1369118784 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa40187d0 con 0x55897db1a690 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5a8e7fc640 1 -- 172.21.15.76:0/1369118784 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa4018bc0 con 0x55897db1a690 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- 172.21.15.76:0/1369118784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db1a690 msgr2=0x55897db27720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/1369118784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db1a690 0x55897db27720 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4007db0 tx=0x7f5aa400e760 comp rx=0 tx=0).stop 2024-09-19T04:20:24.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- 172.21.15.76:0/1369118784 shutdown_connections 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/1369118784 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897db1a690 0x55897db27720 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/1369118784 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db19d70 0x55897db271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/1369118784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897dacddb0 0x55897db2e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- 172.21.15.76:0/1369118784 >> 172.21.15.76:0/1369118784 conn(0x55897dafb040 msgr2=0x55897db26ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- 172.21.15.76:0/1369118784 shutdown_connections 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- 172.21.15.76:0/1369118784 wait complete. 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- start start 2024-09-19T04:20:24.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 Processor -- start 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db2de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 0x55897db8cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897db8d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x55897d8c4f20 con 0x55897dacddb0 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x55897d8d4ac0 con 0x55897db19d70 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x55897d9db0c0 con 0x55897db1a690 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab67e7000 10 client.0 _ll_register_callbacks cb 0x7f5a9c011e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db2de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.040 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab14c7640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db2de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57578/0 (socket says 172.21.15.76:57578) 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab14c7640 1 -- 172.21.15.76:0/803073362 learned_addr learned my addr 172.21.15.76:0/803073362 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab14c7640 1 -- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 msgr2=0x55897db8d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897db8d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab14c7640 1 -- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 msgr2=0x55897db8cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.032+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 0x55897db8cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab14c7640 1 -- 172.21.15.76:0/803073362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897d8b2640 con 0x55897dacddb0 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab14c7640 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db2de20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c054460 tx=0x7f5a9c083ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809885ab387f73e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 1 -- 172.21.15.76:0/803073362 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c06f070 con 0x55897dacddb0 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 1 -- 172.21.15.76:0/803073362 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f5a9c051e70 con 0x55897dacddb0 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa40187d0 con 0x55897dacddb0 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 1 -- 172.21.15.76:0/803073362 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f5a9c055f10 con 0x55897dacddb0 2024-09-19T04:20:24.041 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 10 client.8814 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x55897d8b63b0 con 0x55897dacddb0 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 1 -- 172.21.15.76:0/803073362 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5aa40187d0 con 0x55897dacddb0 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 1 -- 172.21.15.76:0/803073362 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f5a9c07b020 con 0x55897dacddb0 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 10 client.8814 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 10 client.8814 resolve_mds: resolved 7:2 to role '7:2' aka daemon mds.d 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 20 client.8814 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db2e360 0x55897db49790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 4 client.8814 mds_command: new command op to 6127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 --> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55897d92c470 con 0x55897db2e360 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db2e360 0x55897db49790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.042 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab0cc6640 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db2e360 0x55897db49790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87e7587a2b285836 server_cookie=a9264139787201d1 in_seq=0 out_seq=0 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 10 client.8814 ms_handle_connect on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 1 -- 172.21.15.76:0/803073362 <== mds.2 v2:172.21.15.116:6834/255266111 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x55897d92c470 con 0x55897db2e360 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 10 client.8814 handle_command_reply: tid=0 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 client.8814 shutdown 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 msgr2=0x55897db2de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db2de20 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c054460 tx=0x7f5a9c083ed0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 shutdown_connections 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:6834/255266111,v1:172.21.15.116:6835/255266111] conn(0x55897db2e360 0x55897db49790 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x55897db1a690 0x55897db8d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55897db19d70 0x55897db8cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5a87fff640 0 client.8814 ms_handle_reset on v2:172.21.15.116:6834/255266111 2024-09-19T04:20:24.043 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 --2- 172.21.15.76:0/803073362 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x55897dacddb0 0x55897db2de20 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 >> 172.21.15.76:0/803073362 conn(0x55897dafb040 msgr2=0x55897db272f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 shutdown_connections 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 1 -- 172.21.15.76:0/803073362 wait complete. 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab67e7000 20 client.8814 trim_cache size 0 max 16384 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:24.044 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:24.045 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:24.045 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:24.045 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:24.045 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:24.045 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a900678c0 msgr2=0x7f5a90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a900678c0 0x7f5a90069d80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f5aa80ae000 tx=0x7f5aa8008040 comp rx=0 tx=0).stop 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac06e4b0 msgr2=0x7f5aac155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac06e4b0 0x7f5aac155bb0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00b440 tx=0x7f5a9c00b900 comp rx=0 tx=0).stop 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 shutdown_connections 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5a900678c0 0x7f5a90069d80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5aac1587a0 0x7f5aac150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5aac0b39a0 0x7f5aac1530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 --2- 172.21.15.76:0/3933907331 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5aac06e4b0 0x7f5aac155bb0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 >> 172.21.15.76:0/3933907331 conn(0x7f5aac0b1f90 msgr2=0x7f5aac0ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 shutdown_connections 2024-09-19T04:20:24.046 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.036+0000 7f5ab372b640 1 -- 172.21.15.76:0/3933907331 wait complete. 2024-09-19T04:20:24.053 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:24.053 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-09-19T04:20:24.053 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-19T04:20:24.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.160+0000 7f59ab424640 1 Processor -- start 2024-09-19T04:20:24.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.160+0000 7f59ab424640 1 -- start start 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59a415bec0 0x7f59a40ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a40af470 0x7f59a406e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a40b3290 con 0x7f59a415c2a0 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f59a40b3570 con 0x7f59a415bec0 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f59a40b3850 con 0x7f59a40af470 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59a415bec0 0x7f59a40ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f59a415bec0 0x7f59a40ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57592/0 (socket says 172.21.15.76:57592) 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 -- 172.21.15.76:0/2894746192 learned_addr learned my addr 172.21.15.76:0/2894746192 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 -- 172.21.15.76:0/2894746192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a40af470 msgr2=0x7f59a406e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 --2- 172.21.15.76:0/2894746192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a40af470 0x7f59a406e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 -- 172.21.15.76:0/2894746192 >> v1:172.21.15.116:6789/0 conn(0x7f59a415c2a0 legacy=0x7f59a40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 -- 172.21.15.76:0/2894746192 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a40b3b30 con 0x7f59a415bec0 2024-09-19T04:20:24.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a91c0640 1 --2- 172.21.15.76:0/2894746192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415bec0 0x7f59a40ad200 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f599c009e50 tx=0x7f599c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5411f3c57cf0262 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a3fff640 1 -- 172.21.15.76:0/2894746192 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0a57b0 con 0x7f59a415bec0 2024-09-19T04:20:24.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a3fff640 1 -- 172.21.15.76:0/2894746192 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599c0a5950 con 0x7f59a415bec0 2024-09-19T04:20:24.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59a3fff640 1 -- 172.21.15.76:0/2894746192 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0a5d20 con 0x7f59a415bec0 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- 172.21.15.76:0/2894746192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415bec0 msgr2=0x7f59a40ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 --2- 172.21.15.76:0/2894746192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415bec0 0x7f59a40ad200 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f599c009e50 tx=0x7f599c01c930 comp rx=0 tx=0).stop 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- 172.21.15.76:0/2894746192 shutdown_connections 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 --2- 172.21.15.76:0/2894746192 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a40af470 0x7f59a406e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 --2- 172.21.15.76:0/2894746192 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415bec0 0x7f59a40ad200 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- 172.21.15.76:0/2894746192 >> 172.21.15.76:0/2894746192 conn(0x7f59a40b1f90 msgr2=0x7f59a40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- 172.21.15.76:0/2894746192 shutdown_connections 2024-09-19T04:20:24.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.164+0000 7f59ab424640 1 -- 172.21.15.76:0/2894746192 wait complete. 2024-09-19T04:20:24.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 Processor -- start 2024-09-19T04:20:24.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 -- start start 2024-09-19T04:20:24.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a40af470 0x7f59a40998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a415bec0 0x7f59a4073530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415c2a0 0x7f59a414b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59a406ed50 con 0x7f59a415c2a0 2024-09-19T04:20:24.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59a40ad740 con 0x7f59a40af470 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f59a406eed0 con 0x7f59a415bec0 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415c2a0 0x7f59a414b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415c2a0 0x7f59a414b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57598/0 (socket says 172.21.15.76:57598) 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 -- 172.21.15.76:0/3366813851 learned_addr learned my addr 172.21.15.76:0/3366813851 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a89bf640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a415bec0 0x7f59a4073530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a91c0640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a40af470 0x7f59a40998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 -- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a415bec0 msgr2=0x7f59a4073530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a415bec0 0x7f59a4073530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 -- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a40af470 msgr2=0x7f59a40998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a40af470 0x7f59a40998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 -- 172.21.15.76:0/3366813851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a4153de0 con 0x7f59a415c2a0 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a91c0640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a40af470 0x7f59a40998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a99c1640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415c2a0 0x7f59a414b120 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f599400b430 tx=0x7f599400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a1ffb640 1 -- 172.21.15.76:0/3366813851 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994015020 con 0x7f59a415c2a0 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a1ffb640 1 -- 172.21.15.76:0/3366813851 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5994011830 con 0x7f59a415c2a0 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59a1ffb640 1 -- 172.21.15.76:0/3366813851 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5994011c20 con 0x7f59a415c2a0 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a41540c0 con 0x7f59a415c2a0 2024-09-19T04:20:24.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.168+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a4154960 con 0x7f59a415c2a0 2024-09-19T04:20:24.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a1ffb640 1 -- 172.21.15.76:0/3366813851 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5994018020 con 0x7f59a415c2a0 2024-09-19T04:20:24.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a1ffb640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59900678f0 0x7f5990069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a1ffb640 1 -- 172.21.15.76:0/3366813851 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f5994085ed0 con 0x7f59a415c2a0 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a91c0640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59900678f0 0x7f5990069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a91c0640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59900678f0 0x7f5990069db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f599c0ae000 tx=0x7f599c0a7070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 Processor -- start 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- start start 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1ba9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1b5c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b5d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563cb1a46960 con 0x563cb1ba9690 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563cb198a900 con 0x563cb1b5cdb0 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x563cb19add40 con 0x563cb1ba8d70 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1b5c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.180 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1b5c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57602/0 (socket says 172.21.15.76:57602) 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 -- 172.21.15.76:0/890771317 learned_addr learned my addr 172.21.15.76:0/890771317 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a91c0640 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1ba9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a99c1640 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b5d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 -- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 msgr2=0x563cb1ba9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1ba9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 -- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 msgr2=0x563cb1b5d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b5d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 -- 172.21.15.76:0/890771317 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb1954580 con 0x563cb1ba9690 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59a89bf640 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1b5c870 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f59a415c6b0 tx=0x7f599800ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f059da337837ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f598affd640 1 -- 172.21.15.76:0/890771317 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998015070 con 0x563cb1ba9690 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f598affd640 1 -- 172.21.15.76:0/890771317 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5998004220 con 0x563cb1ba9690 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f598affd640 1 -- 172.21.15.76:0/890771317 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599800ae60 con 0x563cb1ba9690 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- 172.21.15.76:0/890771317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 msgr2=0x563cb1b5c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.181 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1b5c870 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f59a415c6b0 tx=0x7f599800ac10 comp rx=0 tx=0).stop 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- 172.21.15.76:0/890771317 shutdown_connections 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b5d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1b5c870 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/890771317 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1ba9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- 172.21.15.76:0/890771317 >> 172.21.15.76:0/890771317 conn(0x563cb1b8a040 msgr2=0x563cb1b8c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- 172.21.15.76:0/890771317 shutdown_connections 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.172+0000 7f59ae4e0000 1 -- 172.21.15.76:0/890771317 wait complete. 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- start start 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 Processor -- start 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1b5cdb0 0x563cb1bafce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1ba8d70 0x563cb1bb0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.182 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x563cb19a9d00 con 0x563cb1b5cdb0 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563cb1968aa0 con 0x563cb1ba8d70 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x563cb1947340 con 0x563cb1ba9690 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 10 client.0 _ll_register_callbacks cb 0x563cb1b4a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1b5cdb0 0x563cb1bafce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1b5cdb0 0x563cb1bafce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57612/0 (socket says 172.21.15.76:57612) 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 -- 172.21.15.76:0/442064810 learned_addr learned my addr 172.21.15.76:0/442064810 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a89bf640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1ba8d70 0x563cb1bb0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a99c1640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 -- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 msgr2=0x563cb1bb0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 -- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1ba8d70 msgr2=0x563cb1bb0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.183 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1ba8d70 0x563cb1bb0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 -- 172.21.15.76:0/442064810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb1950dc0 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a89bf640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1ba8d70 0x563cb1bb0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a91c0640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1b5cdb0 0x563cb1bafce0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f599c01afe0 tx=0x7f599c01b170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0b732083399752 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 1 -- 172.21.15.76:0/442064810 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0b9920 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 1 -- 172.21.15.76:0/442064810 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f599c0b9ac0 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5998004220 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 1 -- 172.21.15.76:0/442064810 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f599c0cf050 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 10 client.8826 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x563cb194d7f0 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 1 -- 172.21.15.76:0/442064810 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5998004220 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 1 -- 172.21.15.76:0/442064810 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f599c0d2020 con 0x563cb1b5cdb0 2024-09-19T04:20:24.184 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 10 client.8826 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 10 client.8826 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 20 client.8826 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1bb80b0 0x563cb1bba4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 4 client.8826 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cb1947e10 con 0x563cb1bb80b0 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a89bf640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1bb80b0 0x563cb1bba4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59a89bf640 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1bb80b0 0x563cb1bba4a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92ce686b0f1b520f server_cookie=40982b9f8a508376 in_seq=0 out_seq=0 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 10 client.8826 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 1 -- 172.21.15.76:0/442064810 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563cb1947e10 con 0x563cb1bb80b0 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 10 client.8826 handle_command_reply: tid=0 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 client.8826 shutdown 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1b5cdb0 msgr2=0x563cb1bafce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.185 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1b5cdb0 0x563cb1bafce0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f599c01afe0 tx=0x7f599c01b170 comp rx=0 tx=0).stop 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 shutdown_connections 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1bb80b0 0x563cb1bba4a0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f597ffff640 0 client.8826 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1ba8d70 0x563cb1bb0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/442064810 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1b5cdb0 0x563cb1bafce0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 >> 172.21.15.76:0/442064810 conn(0x563cb1b8a040 msgr2=0x563cb1b5bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 shutdown_connections 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 1 -- 172.21.15.76:0/442064810 wait complete. 2024-09-19T04:20:24.186 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.176+0000 7f59ae4e0000 20 client.8826 trim_cache size 0 max 16384 2024-09-19T04:20:24.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 Processor -- start 2024-09-19T04:20:24.196 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- start start 2024-09-19T04:20:24.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b64c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba8d70 0x563cb1b65170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4203ff0 con 0x563cb1ba8d70 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4203190 con 0x563cb1b5cdb0 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f59a42038c0 con 0x563cb1ba9690 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba8d70 0x563cb1b65170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba8d70 0x563cb1b65170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57614/0 (socket says 172.21.15.76:57614) 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 -- 172.21.15.76:0/1106600483 learned_addr learned my addr 172.21.15.76:0/1106600483 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a91c0640 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b64c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a99c1640 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 -- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 msgr2=0x563cb1bb2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 -- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 msgr2=0x563cb1b64c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b64c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 -- 172.21.15.76:0/1106600483 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cb196b470 con 0x563cb1ba8d70 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a91c0640 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b64c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59a89bf640 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba8d70 0x563cb1b65170 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f599800a6d0 tx=0x7f5998011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f95f0a09b7f122 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f598a7fc640 1 -- 172.21.15.76:0/1106600483 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998015040 con 0x563cb1ba8d70 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f598a7fc640 1 -- 172.21.15.76:0/1106600483 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599800cc20 con 0x563cb1ba8d70 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f598a7fc640 1 -- 172.21.15.76:0/1106600483 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980253b0 con 0x563cb1ba8d70 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1106600483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba8d70 msgr2=0x563cb1b65170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba8d70 0x563cb1b65170 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f599800a6d0 tx=0x7f5998011000 comp rx=0 tx=0).stop 2024-09-19T04:20:24.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1106600483 shutdown_connections 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba9690 0x563cb1bb2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba8d70 0x563cb1b65170 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1106600483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1b64c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1106600483 >> 172.21.15.76:0/1106600483 conn(0x563cb1b8a040 msgr2=0x563cb1bafaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1106600483 shutdown_connections 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.196+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1106600483 wait complete. 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- start start 2024-09-19T04:20:24.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 Processor -- start 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1bb5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1c18810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1c18d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f59a41574c0 con 0x563cb1ba9690 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x563cb1947480 con 0x563cb1b5cdb0 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x563cb1b5cbc0 con 0x563cb1ba8d70 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 10 client.0 _ll_register_callbacks cb 0x7f599c009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1c18d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1c18d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57616/0 (socket says 172.21.15.76:57616) 2024-09-19T04:20:24.205 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 -- 172.21.15.76:0/1546978034 learned_addr learned my addr 172.21.15.76:0/1546978034 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a89bf640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1c18810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a91c0640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1bb5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 -- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 msgr2=0x563cb1c18810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1c18810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 -- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 msgr2=0x563cb1bb5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1bb5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 -- 172.21.15.76:0/1546978034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cb1941640 con 0x563cb1ba9690 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a89bf640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1c18810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a91c0640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1bb5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a99c1640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1c18d50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f599405e760 tx=0x7f5994056020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d935a3573c1a6d16 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.206 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 10 client.0 ms_handle_connect on v2:172.21.15.76:3300/0 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 1 -- 172.21.15.76:0/1546978034 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599405ec90 con 0x563cb1ba9690 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 1 -- 172.21.15.76:0/1546978034 <== mon.0 v2:172.21.15.76:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f5994052690 con 0x563cb1ba9690 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f599800cc20 con 0x563cb1ba9690 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 1 -- 172.21.15.76:0/1546978034 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) ==== 24+0+0 (secure 0 0 0) 0x7f5994055470 con 0x563cb1ba9690 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 10 client.8832 fetch_fsmap learned FSMap version 112 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) -- 0x563cb19453b0 con 0x563cb1ba9690 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 1 -- 172.21.15.76:0/1546978034 <== mon.0 v2:172.21.15.76:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f599800cc20 con 0x563cb1ba9690 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 1 -- 172.21.15.76:0/1546978034 <== mon.0 v2:172.21.15.76:3300/0 5 ==== fsmap(e 112) ==== 3558+0+0 (secure 0 0 0) 0x7f5994055610 con 0x563cb1ba9690 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 10 client.8832 fetch_fsmap finished waiting for FSMap version 112 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 10 client.8832 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.a 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 20 client.8832 populate_metadata read hostname 'smithi076' 2024-09-19T04:20:24.207 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1c19290 0x563cb1c196a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 4 client.8832 mds_command: new command op to 8127 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 --> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563cb19bb470 con 0x563cb1c19290 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a91c0640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1c19290 0x563cb1c196a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 10 client.8832 ms_handle_connect on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59a91c0640 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1c19290 0x563cb1c196a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11c684e339632826 server_cookie=af8ecf61d369d999 in_seq=0 out_seq=0 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 1 -- 172.21.15.76:0/1546978034 <== mds.0 v2:172.21.15.76:6836/3881707282 1 ==== command_reply(tid 0: 0 ) ==== 8+0+267 (crc 0 0 0) 0x563cb19bb470 con 0x563cb1c19290 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 10 client.8832 handle_command_reply: tid=0 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 client.8832 shutdown 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 msgr2=0x563cb1c18d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1c18d50 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f599405e760 tx=0x7f5994056020 comp rx=0 tx=0).stop 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 shutdown_connections 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:6836/3881707282,v1:172.21.15.76:6837/3881707282] conn(0x563cb1c19290 0x563cb1c196a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x563cb1ba9690 0x563cb1c18d50 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x563cb1ba8d70 0x563cb1c18810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 --2- 172.21.15.76:0/1546978034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x563cb1b5cdb0 0x563cb1bb5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 >> 172.21.15.76:0/1546978034 conn(0x563cb1b8a040 msgr2=0x563cb1bb02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f597ffff640 0 client.8832 ms_handle_reset on v2:172.21.15.76:6836/3881707282 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 shutdown_connections 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 1 -- 172.21.15.76:0/1546978034 wait complete. 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ae4e0000 20 client.8832 trim_cache size 0 max 16384 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:24.209 INFO:teuthology.orchestra.run.smithi076.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: "scrubs": { 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1": { 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: "path": "/d1/d2/d3", 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: "tag": "2c8b4d1d-d0c0-4337-b857-f24bd14da4b1", 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: "options": "recursive,force" 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout: } 2024-09-19T04:20:24.210 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:24.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59900678f0 msgr2=0x7f5990069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59900678f0 0x7f5990069db0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f599c0ae000 tx=0x7f599c0a7070 comp rx=0 tx=0).stop 2024-09-19T04:20:24.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415c2a0 msgr2=0x7f59a414b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415c2a0 0x7f59a414b120 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f599400b430 tx=0x7f599400b8f0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 shutdown_connections 2024-09-19T04:20:24.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f59900678f0 0x7f5990069db0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f59a415c2a0 0x7f59a414b120 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f59a415bec0 0x7f59a4073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 --2- 172.21.15.76:0/3366813851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a40af470 0x7f59a40998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 >> 172.21.15.76:0/3366813851 conn(0x7f59a40b1f90 msgr2=0x7f59a4075230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 shutdown_connections 2024-09-19T04:20:24.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.200+0000 7f59ab424640 1 -- 172.21.15.76:0/3366813851 wait complete. 2024-09-19T04:20:24.217 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T04:20:24.218 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:24.228 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:20:24.268 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:20:24.321 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.320+0000 7fd816f36640 1 Processor -- start 2024-09-19T04:20:24.322 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.320+0000 7fd816f36640 1 -- start start 2024-09-19T04:20:24.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.320+0000 7fd816f36640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd8100cebb0 0x7fd8100cef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.323 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.320+0000 7fd816f36640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100ce6f0 0x7fd8100d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.320+0000 7fd816f36640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8100d1c70 0x7fd8100d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fd8100cf4d0 con 0x7fd8100d1c70 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd8100cc890 con 0x7fd8100cebb0 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd8100ccb70 con 0x7fd8100ce6f0 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8100d1c70 0x7fd8100d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fd8100d1c70 0x7fd8100d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57628/0 (socket says 172.21.15.76:57628) 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 -- 172.21.15.76:0/2931212414 learned_addr learned my addr 172.21.15.76:0/2931212414 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd814cd2640 1 --2- 172.21.15.76:0/2931212414 >> v2:172.21.15.116:3300/0 conn(0x7fd8100cebb0 0x7fd8100cef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 -- 172.21.15.76:0/2931212414 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100ce6f0 msgr2=0x7fd8100d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd80ffff640 1 --2- 172.21.15.76:0/2931212414 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100ce6f0 0x7fd8100d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 --2- 172.21.15.76:0/2931212414 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100ce6f0 0x7fd8100d0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.324 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 -- 172.21.15.76:0/2931212414 >> v2:172.21.15.116:3300/0 conn(0x7fd8100cebb0 msgr2=0x7fd8100cef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 --2- 172.21.15.76:0/2931212414 >> v2:172.21.15.116:3300/0 conn(0x7fd8100cebb0 0x7fd8100cef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.325 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 -- 172.21.15.76:0/2931212414 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8100cce50 con 0x7fd8100d1c70 2024-09-19T04:20:24.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd8154d3640 1 --2- 172.21.15.76:0/2931212414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100d1c70 0x7fd8100d2100 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd8100cfc30 tx=0x7fd80001c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34879db594e83025 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd80f7fe640 1 -- 172.21.15.76:0/2931212414 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000a5840 con 0x7fd8100d1c70 2024-09-19T04:20:24.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd80f7fe640 1 -- 172.21.15.76:0/2931212414 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8000a59e0 con 0x7fd8100d1c70 2024-09-19T04:20:24.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd80f7fe640 1 -- 172.21.15.76:0/2931212414 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8000a5db0 con 0x7fd8100d1c70 2024-09-19T04:20:24.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- 172.21.15.76:0/2931212414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100d1c70 msgr2=0x7fd8100d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 --2- 172.21.15.76:0/2931212414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100d1c70 0x7fd8100d2100 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fd8100cfc30 tx=0x7fd80001c9c0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- 172.21.15.76:0/2931212414 shutdown_connections 2024-09-19T04:20:24.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 --2- 172.21.15.76:0/2931212414 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100d1c70 0x7fd8100d2100 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 --2- 172.21.15.76:0/2931212414 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100ce6f0 0x7fd8100d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 --2- 172.21.15.76:0/2931212414 >> v2:172.21.15.116:3300/0 conn(0x7fd8100cebb0 0x7fd8100cef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- 172.21.15.76:0/2931212414 >> 172.21.15.76:0/2931212414 conn(0x7fd8100700a0 msgr2=0x7fd8100704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- 172.21.15.76:0/2931212414 shutdown_connections 2024-09-19T04:20:24.327 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.324+0000 7fd816f36640 1 -- 172.21.15.76:0/2931212414 wait complete. 2024-09-19T04:20:24.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 Processor -- start 2024-09-19T04:20:24.329 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 -- start start 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8100ce6f0 0x7fd81014bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100cebb0 0x7fd81014c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100d1c70 0x7fd810151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8100d0cc0 con 0x7fd8100cebb0 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd81015c3a0 con 0x7fd8100ce6f0 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd8100d0e40 con 0x7fd8100d1c70 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100cebb0 0x7fd81014c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100cebb0 0x7fd81014c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57634/0 (socket says 172.21.15.76:57634) 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 -- 172.21.15.76:0/506237403 learned_addr learned my addr 172.21.15.76:0/506237403 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd814cd2640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8100ce6f0 0x7fd81014bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd8154d3640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100d1c70 0x7fd810151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.330 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 -- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100d1c70 msgr2=0x7fd810151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100d1c70 0x7fd810151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 -- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8100ce6f0 msgr2=0x7fd81014bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8100ce6f0 0x7fd81014bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 -- 172.21.15.76:0/506237403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81006ed90 con 0x7fd8100cebb0 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80ffff640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100cebb0 0x7fd81014c410 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c550 tx=0x7fd80400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80d7fa640 1 -- 172.21.15.76:0/506237403 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804015070 con 0x7fd8100cebb0 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80d7fa640 1 -- 172.21.15.76:0/506237403 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd804004220 con 0x7fd8100cebb0 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81006e4f0 con 0x7fd8100cebb0 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80d7fa640 1 -- 172.21.15.76:0/506237403 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd804003620 con 0x7fd8100cebb0 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81006eab0 con 0x7fd8100cebb0 2024-09-19T04:20:24.331 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8100cef90 con 0x7fd8100cebb0 2024-09-19T04:20:24.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80d7fa640 1 -- 172.21.15.76:0/506237403 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8040037c0 con 0x7fd8100cebb0 2024-09-19T04:20:24.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80d7fa640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7fc0678c0 0x7fd7fc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd80d7fa640 1 -- 172.21.15.76:0/506237403 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fd804085b20 con 0x7fd8100cebb0 2024-09-19T04:20:24.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.328+0000 7fd814cd2640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7fc0678c0 0x7fd7fc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.332 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.332+0000 7fd814cd2640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7fc0678c0 0x7fd7fc069d80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd8101574f0 tx=0x7fd808008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.332+0000 7fd80d7fa640 1 -- 172.21.15.76:0/506237403 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fd804055e50 con 0x7fd8100cebb0 2024-09-19T04:20:24.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.516+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8100d1a50 con 0x7fd8100cebb0 2024-09-19T04:20:24.517 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.516+0000 7fd80d7fa640 1 -- 172.21.15.76:0/506237403 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7fd80405a150 con 0x7fd8100cebb0 2024-09-19T04:20:24.517 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:24.518 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:24.518 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7fc0678c0 msgr2=0x7fd7fc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7fc0678c0 0x7fd7fc069d80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fd8101574f0 tx=0x7fd808008040 comp rx=0 tx=0).stop 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100cebb0 msgr2=0x7fd81014c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100cebb0 0x7fd81014c410 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fd80400c550 tx=0x7fd80400ca10 comp rx=0 tx=0).stop 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 shutdown_connections 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd7fc0678c0 0x7fd7fc069d80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8100d1c70 0x7fd810151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8100cebb0 0x7fd81014c410 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 --2- 172.21.15.76:0/506237403 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8100ce6f0 0x7fd81014bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 >> 172.21.15.76:0/506237403 conn(0x7fd8100700a0 msgr2=0x7fd810152f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 shutdown_connections 2024-09-19T04:20:24.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.520+0000 7fd816f36640 1 -- 172.21.15.76:0/506237403 wait complete. 2024-09-19T04:20:24.535 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:24.536 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:24.693 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 Processor -- start 2024-09-19T04:20:24.694 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 -- start start 2024-09-19T04:20:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815bec0 0x7fae680ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae6815c2a0 0x7fae680aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fae680af2a0 0x7fae6815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fae680b3290 con 0x7fae6815c2a0 2024-09-19T04:20:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fae680b3570 con 0x7fae680af2a0 2024-09-19T04:20:24.695 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.692+0000 7fae6feb2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fae680b3850 con 0x7fae6815bec0 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae6815c2a0 0x7fae680aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fae6815c2a0 0x7fae680aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57650/0 (socket says 172.21.15.76:57650) 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 -- 172.21.15.76:0/2633089882 learned_addr learned my addr 172.21.15.76:0/2633089882 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6dc4e640 1 --2- 172.21.15.76:0/2633089882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815bec0 0x7fae680ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6e44f640 1 --2- 172.21.15.76:0/2633089882 >> v2:172.21.15.116:3300/0 conn(0x7fae680af2a0 0x7fae6815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 -- 172.21.15.76:0/2633089882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815bec0 msgr2=0x7fae680ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 --2- 172.21.15.76:0/2633089882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815bec0 0x7fae680ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 -- 172.21.15.76:0/2633089882 >> v2:172.21.15.116:3300/0 conn(0x7fae680af2a0 msgr2=0x7fae6815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 --2- 172.21.15.76:0/2633089882 >> v2:172.21.15.116:3300/0 conn(0x7fae680af2a0 0x7fae6815fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.696 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 -- 172.21.15.76:0/2633089882 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae680b3b30 con 0x7fae6815c2a0 2024-09-19T04:20:24.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6d44d640 1 --2- 172.21.15.76:0/2633089882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6815c2a0 0x7fae680aed60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fae680ada40 tx=0x7fae5801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d636b8f9ec889e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6cc4c640 1 -- 172.21.15.76:0/2633089882 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580b0070 con 0x7fae6815c2a0 2024-09-19T04:20:24.697 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6cc4c640 1 -- 172.21.15.76:0/2633089882 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae580ab600 con 0x7fae6815c2a0 2024-09-19T04:20:24.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6cc4c640 1 -- 172.21.15.76:0/2633089882 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae580ab9d0 con 0x7fae6815c2a0 2024-09-19T04:20:24.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 -- 172.21.15.76:0/2633089882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6815c2a0 msgr2=0x7fae680aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.698 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 --2- 172.21.15.76:0/2633089882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6815c2a0 0x7fae680aed60 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fae680ada40 tx=0x7fae5801c230 comp rx=0 tx=0).stop 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 -- 172.21.15.76:0/2633089882 shutdown_connections 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 --2- 172.21.15.76:0/2633089882 >> v2:172.21.15.116:3300/0 conn(0x7fae680af2a0 0x7fae6815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 --2- 172.21.15.76:0/2633089882 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae6815c2a0 0x7fae680aed60 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 --2- 172.21.15.76:0/2633089882 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815bec0 0x7fae680ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 -- 172.21.15.76:0/2633089882 >> 172.21.15.76:0/2633089882 conn(0x7fae680b1f90 msgr2=0x7fae680b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 -- 172.21.15.76:0/2633089882 shutdown_connections 2024-09-19T04:20:24.699 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.696+0000 7fae6feb2640 1 -- 172.21.15.76:0/2633089882 wait complete. 2024-09-19T04:20:24.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 Processor -- start 2024-09-19T04:20:24.700 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 -- start start 2024-09-19T04:20:24.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae680af2a0 0x7fae68152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6815bec0 0x7fae6809d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815c2a0 0x7fae68159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fae680af6b0 con 0x7fae680af2a0 2024-09-19T04:20:24.702 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fae680ad740 con 0x7fae6815bec0 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fae6806e000 con 0x7fae6815c2a0 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae680af2a0 0x7fae68152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae680af2a0 0x7fae68152630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57660/0 (socket says 172.21.15.76:57660) 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 -- 172.21.15.76:0/313277876 learned_addr learned my addr 172.21.15.76:0/313277876 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6d44d640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6815bec0 0x7fae6809d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6e44f640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815c2a0 0x7fae68159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 -- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815c2a0 msgr2=0x7fae68159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815c2a0 0x7fae68159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 -- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6815bec0 msgr2=0x7fae6809d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.703 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6815bec0 0x7fae6809d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 -- 172.21.15.76:0/313277876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae6809d900 con 0x7fae680af2a0 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6dc4e640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae680af2a0 0x7fae68152630 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fae6400d800 tx=0x7fae6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae5e7fc640 1 -- 172.21.15.76:0/313277876 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6400b6c0 con 0x7fae680af2a0 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae5e7fc640 1 -- 172.21.15.76:0/313277876 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae64019070 con 0x7fae680af2a0 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae6809dbe0 con 0x7fae680af2a0 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae5e7fc640 1 -- 172.21.15.76:0/313277876 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae6400bae0 con 0x7fae680af2a0 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae6809e1a0 con 0x7fae680af2a0 2024-09-19T04:20:24.704 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.700+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae680ad200 con 0x7fae680af2a0 2024-09-19T04:20:24.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.704+0000 7fae5e7fc640 1 -- 172.21.15.76:0/313277876 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fae6400bc80 con 0x7fae680af2a0 2024-09-19T04:20:24.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.704+0000 7fae5e7fc640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae4c0678f0 0x7fae4c069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:24.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.704+0000 7fae6d44d640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae4c0678f0 0x7fae4c069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:24.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.704+0000 7fae5e7fc640 1 -- 172.21.15.76:0/313277876 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fae640862b0 con 0x7fae680af2a0 2024-09-19T04:20:24.705 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.704+0000 7fae6d44d640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae4c0678f0 0x7fae4c069db0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae580ad000 tx=0x7fae580a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:24.708 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.708+0000 7fae5e7fc640 1 -- 172.21.15.76:0/313277876 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fae64056cc0 con 0x7fae680af2a0 2024-09-19T04:20:24.925 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.924+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fae6806fcd0 con 0x7fae680af2a0 2024-09-19T04:20:24.926 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.924+0000 7fae5e7fc640 1 -- 172.21.15.76:0/313277876 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7fae640565c0 con 0x7fae680af2a0 2024-09-19T04:20:24.926 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:24.926 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700292,"bytes_used":267501568,"bytes_avail":772826611712,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":7251,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-19T04:20:19.871079+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-19T04:20:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae4c0678f0 msgr2=0x7fae4c069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.928 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae4c0678f0 0x7fae4c069db0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fae580ad000 tx=0x7fae580a6000 comp rx=0 tx=0).stop 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae680af2a0 msgr2=0x7fae68152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae680af2a0 0x7fae68152630 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fae6400d800 tx=0x7fae6400dcc0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 shutdown_connections 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fae4c0678f0 0x7fae4c069db0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fae6815c2a0 0x7fae68159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae6815bec0 0x7fae6809d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 --2- 172.21.15.76:0/313277876 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fae680af2a0 0x7fae68152630 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 >> 172.21.15.76:0/313277876 conn(0x7fae680b1f90 msgr2=0x7fae680aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 shutdown_connections 2024-09-19T04:20:24.929 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:24.928+0000 7fae6feb2640 1 -- 172.21.15.76:0/313277876 wait complete. 2024-09-19T04:20:24.944 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:25.799 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:20:25.796+0000 7fa676b89640 -1 log_channel(cluster) log [ERR] : Health check update: 3 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-19T04:20:25.945 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:26.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.076+0000 7f1db46da640 1 Processor -- start 2024-09-19T04:20:26.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.076+0000 7f1db46da640 1 -- start start 2024-09-19T04:20:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dac15c2a0 0x7f1dac0aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1dac0af2a0 0x7f1dac06e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1dac0b3290 con 0x7f1dac15bec0 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1dac0b3570 con 0x7f1dac15c2a0 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f1dac0b3850 con 0x7f1dac0af2a0 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1c75640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1dac15c2a0 0x7f1dac0aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db2476640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1dac15bec0 0x7f1dac0ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52816/0 (socket says 172.21.15.76:52816) 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db2476640 1 -- 172.21.15.76:0/1430078889 learned_addr learned my addr 172.21.15.76:0/1430078889 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1c75640 1 -- 172.21.15.76:0/1430078889 >> v2:172.21.15.116:3300/0 conn(0x7f1dac0af2a0 msgr2=0x7f1dac06e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1c75640 1 --2- 172.21.15.76:0/1430078889 >> v2:172.21.15.116:3300/0 conn(0x7f1dac0af2a0 0x7f1dac06e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1c75640 1 -- 172.21.15.76:0/1430078889 >> v1:172.21.15.76:6789/0 conn(0x7f1dac15bec0 legacy=0x7f1dac0ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1c75640 1 -- 172.21.15.76:0/1430078889 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac0b3b30 con 0x7f1dac15c2a0 2024-09-19T04:20:26.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1474640 1 -- 172.21.15.76:0/1430078889 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2457820463 0 0) 0x7f1dac0b3290 con 0x7f1dac15bec0 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1c75640 1 --2- 172.21.15.76:0/1430078889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac15c2a0 0x7f1dac0aed60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0b2fb0 tx=0x7f1da801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49659ff7b83d35ed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1474640 1 -- 172.21.15.76:0/1430078889 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da80ab650 con 0x7f1dac15c2a0 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1474640 1 -- 172.21.15.76:0/1430078889 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da80b0070 con 0x7f1dac15c2a0 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db1474640 1 -- 172.21.15.76:0/1430078889 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da80aba10 con 0x7f1dac15c2a0 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 -- 172.21.15.76:0/1430078889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac15c2a0 msgr2=0x7f1dac0aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 --2- 172.21.15.76:0/1430078889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac15c2a0 0x7f1dac0aed60 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0b2fb0 tx=0x7f1da801c450 comp rx=0 tx=0).stop 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 -- 172.21.15.76:0/1430078889 shutdown_connections 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 --2- 172.21.15.76:0/1430078889 >> v2:172.21.15.116:3300/0 conn(0x7f1dac0af2a0 0x7f1dac06e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 --2- 172.21.15.76:0/1430078889 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac15c2a0 0x7f1dac0aed60 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.083 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 -- 172.21.15.76:0/1430078889 >> 172.21.15.76:0/1430078889 conn(0x7f1dac0b1f90 msgr2=0x7f1dac0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:26.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.080+0000 7f1db46da640 1 -- 172.21.15.76:0/1430078889 shutdown_connections 2024-09-19T04:20:26.084 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- 172.21.15.76:0/1430078889 wait complete. 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 Processor -- start 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- start start 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dac0af2a0 0x7f1dac150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dac15bec0 0x7f1dac14a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac14d0b0 0x7f1dac15f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac06ed50 con 0x7f1dac14d0b0 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1dac0af6f0 con 0x7f1dac15bec0 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1dac06eed0 con 0x7f1dac0af2a0 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac14d0b0 0x7f1dac15f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac14d0b0 0x7f1dac15f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57678/0 (socket says 172.21.15.76:57678) 2024-09-19T04:20:26.086 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 -- 172.21.15.76:0/1106497118 learned_addr learned my addr 172.21.15.76:0/1106497118 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db1c75640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dac15bec0 0x7f1dac14a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2476640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dac0af2a0 0x7f1dac150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 -- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dac0af2a0 msgr2=0x7f1dac150630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dac0af2a0 0x7f1dac150630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 -- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dac15bec0 msgr2=0x7f1dac14a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dac15bec0 0x7f1dac14a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 -- 172.21.15.76:0/1106497118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dac150b70 con 0x7f1dac14d0b0 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db2c77640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac14d0b0 0x7f1dac15f010 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b970 tx=0x7f1da400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1da2ffd640 1 -- 172.21.15.76:0/1106497118 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4019070 con 0x7f1dac14d0b0 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1da2ffd640 1 -- 172.21.15.76:0/1106497118 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1da400cb60 con 0x7f1dac14d0b0 2024-09-19T04:20:26.087 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dac14d620 con 0x7f1dac14d0b0 2024-09-19T04:20:26.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1da2ffd640 1 -- 172.21.15.76:0/1106497118 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1da4014440 con 0x7f1dac14d0b0 2024-09-19T04:20:26.088 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dac14dbe0 con 0x7f1dac14d0b0 2024-09-19T04:20:26.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.084+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1dac06f1d0 con 0x7f1dac14d0b0 2024-09-19T04:20:26.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.088+0000 7f1da2ffd640 1 -- 172.21.15.76:0/1106497118 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1da401c070 con 0x7f1dac14d0b0 2024-09-19T04:20:26.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.088+0000 7f1da2ffd640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d900678c0 0x7f1d90069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.088+0000 7f1da2ffd640 1 -- 172.21.15.76:0/1106497118 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f1da4085bd0 con 0x7f1dac14d0b0 2024-09-19T04:20:26.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.088+0000 7f1db2476640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d900678c0 0x7f1d90069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.089 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.088+0000 7f1db2476640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d900678c0 0x7f1d90069d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f70 tx=0x7f1d9c007c10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:26.092 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.092+0000 7f1da2ffd640 1 -- 172.21.15.76:0/1106497118 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1da4056550 con 0x7f1dac14d0b0 2024-09-19T04:20:26.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.276+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1dac06fcd0 con 0x7f1dac14d0b0 2024-09-19T04:20:26.279 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1da2ffd640 1 -- 172.21.15.76:0/1106497118 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f1da4055e50 con 0x7f1dac14d0b0 2024-09-19T04:20:26.279 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:26.279 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:26.280 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:26.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d900678c0 msgr2=0x7f1d90069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d900678c0 0x7f1d90069d80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f70 tx=0x7f1d9c007c10 comp rx=0 tx=0).stop 2024-09-19T04:20:26.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac14d0b0 msgr2=0x7f1dac15f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac14d0b0 0x7f1dac15f010 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f1da400b970 tx=0x7f1da400be30 comp rx=0 tx=0).stop 2024-09-19T04:20:26.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 shutdown_connections 2024-09-19T04:20:26.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d900678c0 0x7f1d90069d80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1dac14d0b0 0x7f1dac15f010 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dac15bec0 0x7f1dac14a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 --2- 172.21.15.76:0/1106497118 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1dac0af2a0 0x7f1dac150630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 >> 172.21.15.76:0/1106497118 conn(0x7f1dac0b1f90 msgr2=0x7f1dac1598f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:26.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 shutdown_connections 2024-09-19T04:20:26.284 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.280+0000 7f1db46da640 1 -- 172.21.15.76:0/1106497118 wait complete. 2024-09-19T04:20:26.297 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:26.297 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:26.411 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff4746c0640 1 Processor -- start 2024-09-19T04:20:26.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff4746c0640 1 -- start start 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff4746c0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145ae0 0x7ff46c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff4746c0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff46c0b2570 0x7ff46c15c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff4746c0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff46c0b0510 con 0x7ff46c145600 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff4746c0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff46c0b29a0 con 0x7ff46c0b2570 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff4746c0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff46c0b07f0 con 0x7ff46c145ae0 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff472c5d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff46c0b2570 0x7ff46c15c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff47245c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff46c145600 0x7ff46c0b15b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52822/0 (socket says 172.21.15.76:52822) 2024-09-19T04:20:26.413 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff47245c640 1 -- 172.21.15.76:0/3005830790 learned_addr learned my addr 172.21.15.76:0/3005830790 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:26.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff472c5d640 1 -- 172.21.15.76:0/3005830790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145ae0 msgr2=0x7ff46c0b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:26.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff472c5d640 1 --2- 172.21.15.76:0/3005830790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145ae0 0x7ff46c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff472c5d640 1 -- 172.21.15.76:0/3005830790 >> v1:172.21.15.76:6789/0 conn(0x7ff46c145600 legacy=0x7ff46c0b15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.412+0000 7ff472c5d640 1 -- 172.21.15.76:0/3005830790 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c0b0ad0 con 0x7ff46c0b2570 2024-09-19T04:20:26.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 --2- 172.21.15.76:0/3005830790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c0b2570 0x7ff46c15c830 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff4640097c0 tx=0x7ff46401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe95757c127027a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:26.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff47145a640 1 -- 172.21.15.76:0/3005830790 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640b0070 con 0x7ff46c0b2570 2024-09-19T04:20:26.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff47145a640 1 -- 172.21.15.76:0/3005830790 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4640ab600 con 0x7ff46c0b2570 2024-09-19T04:20:26.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff47145a640 1 -- 172.21.15.76:0/3005830790 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640ab9d0 con 0x7ff46c0b2570 2024-09-19T04:20:26.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- 172.21.15.76:0/3005830790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c0b2570 msgr2=0x7ff46c15c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 --2- 172.21.15.76:0/3005830790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c0b2570 0x7ff46c15c830 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7ff4640097c0 tx=0x7ff46401c330 comp rx=0 tx=0).stop 2024-09-19T04:20:26.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- 172.21.15.76:0/3005830790 shutdown_connections 2024-09-19T04:20:26.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 --2- 172.21.15.76:0/3005830790 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c0b2570 0x7ff46c15c830 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 --2- 172.21.15.76:0/3005830790 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145ae0 0x7ff46c0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- 172.21.15.76:0/3005830790 >> 172.21.15.76:0/3005830790 conn(0x7ff46c09aa00 msgr2=0x7ff46c0b66c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:26.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- 172.21.15.76:0/3005830790 shutdown_connections 2024-09-19T04:20:26.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- 172.21.15.76:0/3005830790 wait complete. 2024-09-19T04:20:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 Processor -- start 2024-09-19T04:20:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- start start 2024-09-19T04:20:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff46c0b2570 0x7ff46c159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145600 0x7ff46c09bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c145ae0 0x7ff46c142c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c0b6f20 con 0x7ff46c145ae0 2024-09-19T04:20:26.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c145f00 con 0x7ff46c0b2570 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff4746c0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff46c0b70a0 con 0x7ff46c145600 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c145ae0 0x7ff46c142c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c145ae0 0x7ff46c142c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57704/0 (socket says 172.21.15.76:57704) 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 -- 172.21.15.76:0/1694505886 learned_addr learned my addr 172.21.15.76:0/1694505886 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff47245c640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff46c0b2570 0x7ff46c159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff471c5b640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145600 0x7ff46c09bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 -- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145600 msgr2=0x7ff46c09bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145600 0x7ff46c09bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 -- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff46c0b2570 msgr2=0x7ff46c159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff46c0b2570 0x7ff46c159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff472c5d640 1 -- 172.21.15.76:0/1694505886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c15a040 con 0x7ff46c145ae0 2024-09-19T04:20:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.416+0000 7ff47245c640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff46c0b2570 0x7ff46c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff472c5d640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c145ae0 0x7ff46c142c60 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff4640097c0 tx=0x7ff46401ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff462ffd640 1 -- 172.21.15.76:0/1694505886 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640b0050 con 0x7ff46c145ae0 2024-09-19T04:20:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff462ffd640 1 -- 172.21.15.76:0/1694505886 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4640bc070 con 0x7ff46c145ae0 2024-09-19T04:20:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c09e880 con 0x7ff46c145ae0 2024-09-19T04:20:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff462ffd640 1 -- 172.21.15.76:0/1694505886 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4640b7670 con 0x7ff46c145ae0 2024-09-19T04:20:26.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c09ee40 con 0x7ff46c145ae0 2024-09-19T04:20:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c15d180 con 0x7ff46c145ae0 2024-09-19T04:20:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff462ffd640 1 -- 172.21.15.76:0/1694505886 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff4640b7810 con 0x7ff46c145ae0 2024-09-19T04:20:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff462ffd640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4500678c0 0x7ff450069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff462ffd640 1 -- 172.21.15.76:0/1694505886 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff464120660 con 0x7ff46c145ae0 2024-09-19T04:20:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff47245c640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4500678c0 0x7ff450069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:26.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.420+0000 7ff47245c640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4500678c0 0x7ff450069d80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff4680012f0 tx=0x7ff468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:26.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.424+0000 7ff462ffd640 1 -- 172.21.15.76:0/1694505886 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff464015fb0 con 0x7ff46c145ae0 2024-09-19T04:20:26.651 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff46c067ed0 con 0x7ff46c145ae0 2024-09-19T04:20:26.652 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff462ffd640 1 -- 172.21.15.76:0/1694505886 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2010 (secure 0 0 0) 0x7ff4640f5e20 con 0x7ff46c145ae0 2024-09-19T04:20:26.652 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:26.652 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":445,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700292,"bytes_used":267501568,"bytes_avail":772826611712,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5715,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-19T04:20:19.871079+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-19T04:20:26.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4500678c0 msgr2=0x7ff450069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4500678c0 0x7ff450069d80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7ff4680012f0 tx=0x7ff468009040 comp rx=0 tx=0).stop 2024-09-19T04:20:26.654 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c145ae0 msgr2=0x7ff46c142c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c145ae0 0x7ff46c142c60 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7ff4640097c0 tx=0x7ff46401ab70 comp rx=0 tx=0).stop 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 shutdown_connections 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff4500678c0 0x7ff450069d80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff46c145ae0 0x7ff46c142c60 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff46c145600 0x7ff46c09bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 --2- 172.21.15.76:0/1694505886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff46c0b2570 0x7ff46c159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 >> 172.21.15.76:0/1694505886 conn(0x7ff46c09aa00 msgr2=0x7ff46c06efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 shutdown_connections 2024-09-19T04:20:26.655 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:26.652+0000 7ff4746c0640 1 -- 172.21.15.76:0/1694505886 wait complete. 2024-09-19T04:20:26.671 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'paused paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:27.672 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1da5ded640 1 Processor -- start 2024-09-19T04:20:27.807 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1da5ded640 1 -- start start 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1da5ded640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1da006df70 0x7f1da01634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1da5ded640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da00b3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1da5ded640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1da006e470 con 0x7f1da01630c0 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1da5ded640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f1da006fc20 con 0x7f1da006df70 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1da5ded640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1da006e750 con 0x7f1da00b3290 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d97fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1da006df70 0x7f1da01634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d97fff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f1da006df70 0x7f1da01634a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57710/0 (socket says 172.21.15.76:57710) 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d9f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f1da01630c0 0x7f1da006f510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52826/0 (socket says 172.21.15.76:52826) 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d97fff640 1 -- 172.21.15.76:0/2554524870 learned_addr learned my addr 172.21.15.76:0/2554524870 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d9ffff640 1 --2- 172.21.15.76:0/2554524870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da00b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d97fff640 1 -- 172.21.15.76:0/2554524870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 msgr2=0x7f1da00b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:27.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d97fff640 1 --2- 172.21.15.76:0/2554524870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da00b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d97fff640 1 -- 172.21.15.76:0/2554524870 >> v1:172.21.15.76:6789/0 conn(0x7f1da01630c0 legacy=0x7f1da006f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d97fff640 1 -- 172.21.15.76:0/2554524870 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da006ea30 con 0x7f1da006df70 2024-09-19T04:20:27.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d9effd640 1 -- 172.21.15.76:0/2554524870 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 365789093 0 0) 0x7f1da006e470 con 0x7f1da01630c0 2024-09-19T04:20:27.810 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.808+0000 7f1d9ffff640 1 --2- 172.21.15.76:0/2554524870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da00b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:27.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d97fff640 1 --2- 172.21.15.76:0/2554524870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da006df70 0x7f1da01634a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009870 tx=0x7f1d8801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d7212a842c272b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9effd640 1 -- 172.21.15.76:0/2554524870 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880b2070 con 0x7f1da006df70 2024-09-19T04:20:27.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9effd640 1 -- 172.21.15.76:0/2554524870 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d880a9070 con 0x7f1da006df70 2024-09-19T04:20:27.811 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9effd640 1 -- 172.21.15.76:0/2554524870 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d880ad880 con 0x7f1da006df70 2024-09-19T04:20:27.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- 172.21.15.76:0/2554524870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da006df70 msgr2=0x7f1da01634a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 --2- 172.21.15.76:0/2554524870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da006df70 0x7f1da01634a0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f1d88009870 tx=0x7f1d8801c560 comp rx=0 tx=0).stop 2024-09-19T04:20:27.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- 172.21.15.76:0/2554524870 shutdown_connections 2024-09-19T04:20:27.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 --2- 172.21.15.76:0/2554524870 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da00b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 --2- 172.21.15.76:0/2554524870 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da006df70 0x7f1da01634a0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- 172.21.15.76:0/2554524870 >> 172.21.15.76:0/2554524870 conn(0x7f1da00b1f90 msgr2=0x7f1da00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:27.812 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- 172.21.15.76:0/2554524870 shutdown_connections 2024-09-19T04:20:27.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- 172.21.15.76:0/2554524870 wait complete. 2024-09-19T04:20:27.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 Processor -- start 2024-09-19T04:20:27.813 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- start start 2024-09-19T04:20:27.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1da006df70 0x7f1da0153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da0156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da01630c0 0x7f1da0156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f1da016f5f0 con 0x7f1da01630c0 2024-09-19T04:20:27.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f1da016f770 con 0x7f1da006df70 2024-09-19T04:20:27.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1da5ded640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1da016f8f0 con 0x7f1da00b3290 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da01630c0 0x7f1da0156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da01630c0 0x7f1da0156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57722/0 (socket says 172.21.15.76:57722) 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 -- 172.21.15.76:0/1874969819 learned_addr learned my addr 172.21.15.76:0/1874969819 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 -- 172.21.15.76:0/1874969819 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 msgr2=0x7f1da0156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da0156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9f7fe640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1da006df70 0x7f1da0153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 -- 172.21.15.76:0/1874969819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1da006df70 msgr2=0x7f1da0153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1da006df70 0x7f1da0153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 -- 172.21.15.76:0/1874969819 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da0159dc0 con 0x7f1da01630c0 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.812+0000 7f1d9ffff640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da01630c0 0x7f1da0156f50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800cef0 tx=0x7f1d9800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9cff9640 1 -- 172.21.15.76:0/1874969819 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9801a070 con 0x7f1da01630c0 2024-09-19T04:20:27.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9cff9640 1 -- 172.21.15.76:0/1874969819 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d9800d860 con 0x7f1da01630c0 2024-09-19T04:20:27.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da0155490 con 0x7f1da01630c0 2024-09-19T04:20:27.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9cff9640 1 -- 172.21.15.76:0/1874969819 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800dc50 con 0x7f1da01630c0 2024-09-19T04:20:27.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da0155730 con 0x7f1da01630c0 2024-09-19T04:20:27.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da01634a0 con 0x7f1da01630c0 2024-09-19T04:20:27.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9cff9640 1 -- 172.21.15.76:0/1874969819 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f1d9801d070 con 0x7f1da01630c0 2024-09-19T04:20:27.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9cff9640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d800678c0 0x7f1d80069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:27.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9f7fe640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d800678c0 0x7f1d80069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:27.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9cff9640 1 -- 172.21.15.76:0/1874969819 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f1d98086620 con 0x7f1da01630c0 2024-09-19T04:20:27.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.816+0000 7f1d9f7fe640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d800678c0 0x7f1d80069d80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f70 tx=0x7f1d90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:27.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:27.820+0000 7f1d9cff9640 1 -- 172.21.15.76:0/1874969819 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f1d980570f0 con 0x7f1da01630c0 2024-09-19T04:20:28.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.016+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1da0070030 con 0x7f1da01630c0 2024-09-19T04:20:28.019 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1d9cff9640 1 -- 172.21.15.76:0/1874969819 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f1d980569f0 con 0x7f1da01630c0 2024-09-19T04:20:28.019 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:28.019 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:28.019 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:28.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d800678c0 msgr2=0x7f1d80069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d800678c0 0x7f1d80069d80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1d90000f70 tx=0x7f1d90009040 comp rx=0 tx=0).stop 2024-09-19T04:20:28.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da01630c0 msgr2=0x7f1da0156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da01630c0 0x7f1da0156f50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800cef0 tx=0x7f1d9800a710 comp rx=0 tx=0).stop 2024-09-19T04:20:28.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 shutdown_connections 2024-09-19T04:20:28.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1d800678c0 0x7f1d80069d80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1da01630c0 0x7f1da0156f50 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f1da00b3290 0x7f1da0156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 --2- 172.21.15.76:0/1874969819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1da006df70 0x7f1da0153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 >> 172.21.15.76:0/1874969819 conn(0x7f1da00b1f90 msgr2=0x7f1da00af080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:28.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 shutdown_connections 2024-09-19T04:20:28.022 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.020+0000 7f1da5ded640 1 -- 172.21.15.76:0/1874969819 wait complete. 2024-09-19T04:20:28.038 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:28.038 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:28.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72cade1640 1 Processor -- start 2024-09-19T04:20:28.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72cade1640 1 -- start start 2024-09-19T04:20:28.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72cade1640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f72c4156720 0x7f72c40b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72cade1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 0x7f72c40b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72cade1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72c413fb30 con 0x7f72c40b5b50 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72cade1640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f72c4140740 con 0x7f72c4156720 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72cade1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f72c413fe10 con 0x7f72c41412e0 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72c3fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f72c40b5b50 0x7f72c4140bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52832/0 (socket says 172.21.15.76:52832) 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72c3fff640 1 -- 172.21.15.76:0/4192157725 learned_addr learned my addr 172.21.15.76:0/4192157725 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72c8b7d640 1 --2- 172.21.15.76:0/4192157725 >> v2:172.21.15.116:3300/0 conn(0x7f72c4156720 0x7f72c40b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.160+0000 7f72c937e640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 0x7f72c40b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3756777154 0 0) 0x7f72c413fb30 con 0x7f72c40b5b50 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72ac003530 con 0x7f72c40b5b50 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4173252835 0 0) 0x7f72ac003530 con 0x7f72c40b5b50 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 msgr2=0x7f72c40b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 0x7f72c40b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 >> v2:172.21.15.116:3300/0 conn(0x7f72c4156720 msgr2=0x7f72c40b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 --2- 172.21.15.76:0/4192157725 >> v2:172.21.15.116:3300/0 conn(0x7f72c4156720 0x7f72c40b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c41400f0 con 0x7f72c40b5b50 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f72b8001680 con 0x7f72c40b5b50 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 >> v1:172.21.15.76:6789/0 conn(0x7f72c40b5b50 legacy=0x7f72c4140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac0039c0 0x7f72ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72ac004810 0x7f72ac004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 0x7f72ac0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72c4140740 con 0x7f72ac004810 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f72c413fe10 con 0x7f72ac0039c0 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f72ac00b9d0 con 0x7f72c41412e0 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c3fff640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72ac004810 0x7f72ac004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c8b7d640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac0039c0 0x7f72ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c937e640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 0x7f72ac0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c3fff640 1 -- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 msgr2=0x7f72ac0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:28.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c3fff640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 0x7f72ac0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c3fff640 1 -- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac0039c0 msgr2=0x7f72ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:28.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c3fff640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac0039c0 0x7f72ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c3fff640 1 -- 172.21.15.76:0/4192157725 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ac003530 con 0x7f72ac004810 2024-09-19T04:20:28.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c937e640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c41412e0 0x7f72ac0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:28.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c3fff640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72ac004810 0x7f72ac004c90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f72c40b5f50 tx=0x7f72b801f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0f94f981bf2631 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:28.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b8007d60 con 0x7f72ac004810 2024-09-19T04:20:28.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72b80ac020 con 0x7f72ac004810 2024-09-19T04:20:28.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.164+0000 7f72c37fe640 1 -- 172.21.15.76:0/4192157725 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72b80058e0 con 0x7f72ac004810 2024-09-19T04:20:28.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- 172.21.15.76:0/4192157725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72ac004810 msgr2=0x7f72ac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 --2- 172.21.15.76:0/4192157725 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72ac004810 0x7f72ac004c90 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f72c40b5f50 tx=0x7f72b801f3d0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 -- 172.21.15.76:0/4192157725 reap_dead start 2024-09-19T04:20:28.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- 172.21.15.76:0/4192157725 shutdown_connections 2024-09-19T04:20:28.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- 172.21.15.76:0/4192157725 >> 172.21.15.76:0/4192157725 conn(0x7f72c406c0c0 msgr2=0x7f72c406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:28.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- 172.21.15.76:0/4192157725 shutdown_connections 2024-09-19T04:20:28.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- 172.21.15.76:0/4192157725 wait complete. 2024-09-19T04:20:28.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 Processor -- start 2024-09-19T04:20:28.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- start start 2024-09-19T04:20:28.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac004810 0x7f72c4155290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c40c7150 0x7f72c4154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72c415e5e0 0x7f72c415ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f72c413f6e0 con 0x7f72c415e5e0 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f72c40728d0 con 0x7f72ac004810 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f72c413f860 con 0x7f72c40c7150 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72c415e5e0 0x7f72c415ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72c415e5e0 0x7f72c415ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57730/0 (socket says 172.21.15.76:57730) 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 -- 172.21.15.76:0/2628963526 learned_addr learned my addr 172.21.15.76:0/2628963526 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 -- 172.21.15.76:0/2628963526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c40c7150 msgr2=0x7f72c4154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c40c7150 0x7f72c4154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 -- 172.21.15.76:0/2628963526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac004810 msgr2=0x7f72c4155290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c8b7d640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac004810 0x7f72c4155290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac004810 0x7f72c4155290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 -- 172.21.15.76:0/2628963526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72c415f8f0 con 0x7f72c415e5e0 2024-09-19T04:20:28.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c937e640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72c415e5e0 0x7f72c415ea60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f72c40b5270 tx=0x7f72bc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:28.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c17fa640 1 -- 172.21.15.76:0/2628963526 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc01c070 con 0x7f72c415e5e0 2024-09-19T04:20:28.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c17fa640 1 -- 172.21.15.76:0/2628963526 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72bc017400 con 0x7f72c415e5e0 2024-09-19T04:20:28.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72c40c7680 con 0x7f72c415e5e0 2024-09-19T04:20:28.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72c17fa640 1 -- 172.21.15.76:0/2628963526 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0177d0 con 0x7f72c415e5e0 2024-09-19T04:20:28.172 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.168+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72c40c7c40 con 0x7f72c415e5e0 2024-09-19T04:20:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.172+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72c415efa0 con 0x7f72c415e5e0 2024-09-19T04:20:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.172+0000 7f72c17fa640 1 -- 172.21.15.76:0/2628963526 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f72bc01f070 con 0x7f72c415e5e0 2024-09-19T04:20:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.172+0000 7f72c17fa640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72b00678f0 0x7f72b0069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.172+0000 7f72c17fa640 1 -- 172.21.15.76:0/2628963526 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f72bc085e80 con 0x7f72c415e5e0 2024-09-19T04:20:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.172+0000 7f72c8b7d640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72b00678f0 0x7f72b0069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:28.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.172+0000 7f72c8b7d640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72b00678f0 0x7f72b0069db0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009970 tx=0x7f72b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:28.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.176+0000 7f72c17fa640 1 -- 172.21.15.76:0/2628963526 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f72bc012070 con 0x7f72c415e5e0 2024-09-19T04:20:28.398 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.396+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f72c40b5610 con 0x7f72c415e5e0 2024-09-19T04:20:28.399 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72c17fa640 1 -- 172.21.15.76:0/2628963526 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2016 (secure 0 0 0) 0x7f72bc056890 con 0x7f72c415e5e0 2024-09-19T04:20:28.399 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:28.399 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":447,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700292,"bytes_used":267501568,"bytes_avail":772826611712,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5715,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-09-19T04:20:27.873487+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72b00678f0 msgr2=0x7f72b0069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72b00678f0 0x7f72b0069db0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009970 tx=0x7f72b4008040 comp rx=0 tx=0).stop 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72c415e5e0 msgr2=0x7f72c415ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72c415e5e0 0x7f72c415ea60 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f72c40b5270 tx=0x7f72bc00f4a0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 shutdown_connections 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72b00678f0 0x7f72b0069db0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f72c415e5e0 0x7f72c415ea60 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72c40c7150 0x7f72c4154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 --2- 172.21.15.76:0/2628963526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72ac004810 0x7f72c4155290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 >> 172.21.15.76:0/2628963526 conn(0x7f72c406c0c0 msgr2=0x7f72c4156b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:28.402 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 shutdown_connections 2024-09-19T04:20:28.403 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:28.400+0000 7f72cade1640 1 -- 172.21.15.76:0/2628963526 wait complete. 2024-09-19T04:20:28.418 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:29.420 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0ce5d52640 1 Processor -- start 2024-09-19T04:20:29.555 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0ce5d52640 1 -- start start 2024-09-19T04:20:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0ce5d52640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 0x7f0ce015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0ce5d52640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0ce015f1d0 0x7f0ce00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0ce5d52640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ce00aede0 con 0x7f0ce01520a0 2024-09-19T04:20:29.556 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0ce5d52640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0ce0152480 con 0x7f0ce015f1d0 2024-09-19T04:20:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0ce5d52640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce015fa10 con 0x7f0ce0070020 2024-09-19T04:20:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdf7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f0ce01520a0 0x7f0ce006f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52846/0 (socket says 172.21.15.76:52846) 2024-09-19T04:20:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdf7fe640 1 -- 172.21.15.76:0/3121460485 learned_addr learned my addr 172.21.15.76:0/3121460485 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 910500853 0 0) 0x7f0ce00aede0 con 0x7f0ce01520a0 2024-09-19T04:20:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 --2- 172.21.15.76:0/3121460485 >> v2:172.21.15.116:3300/0 conn(0x7f0ce015f1d0 0x7f0ce00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdeffd640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 0x7f0ce015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.557 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cc8003530 con 0x7f0ce01520a0 2024-09-19T04:20:29.559 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 215762695 0 0) 0x7f0cc8003530 con 0x7f0ce01520a0 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 msgr2=0x7f0ce015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 0x7f0ce015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 >> v2:172.21.15.116:3300/0 conn(0x7f0ce015f1d0 msgr2=0x7f0ce00adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 --2- 172.21.15.76:0/3121460485 >> v2:172.21.15.116:3300/0 conn(0x7f0ce015f1d0 0x7f0ce00adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce0074250 con 0x7f0ce01520a0 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 --2- 172.21.15.76:0/3121460485 >> v2:172.21.15.116:3300/0 conn(0x7f0ce015f1d0 0x7f0ce00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f0cd0001a00 con 0x7f0ce01520a0 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 >> v1:172.21.15.76:6789/0 conn(0x7f0ce01520a0 legacy=0x7f0ce006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cc80039c0 0x7f0cc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 0x7f0cc8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0cc80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce015fa10 con 0x7f0cc8004d50 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc800ba80 con 0x7f0cc80039c0 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc800de90 con 0x7f0ce0070020 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0cc80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdf7fe640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cc80039c0 0x7f0cc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdeffd640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 0x7f0cc8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 -- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 msgr2=0x7f0cc8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0070020 0x7f0cc8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 -- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cc80039c0 msgr2=0x7f0cc8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cc80039c0 0x7f0cc8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdffff640 1 -- 172.21.15.76:0/3121460485 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc8003530 con 0x7f0cc8004d50 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.556+0000 7f0cdf7fe640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0cc80039c0 0x7f0cc8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:29.561 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0cdffff640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0cc80051d0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80098d0 tx=0x7f0cd801c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7d053c61d54c10 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd80ac070 con 0x7f0cc8004d50 2024-09-19T04:20:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd80a6660 con 0x7f0cc8004d50 2024-09-19T04:20:29.562 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0cde7fc640 1 -- 172.21.15.76:0/3121460485 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd800f040 con 0x7f0cc8004d50 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- 172.21.15.76:0/3121460485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 msgr2=0x7f0cc80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 --2- 172.21.15.76:0/3121460485 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0cc80051d0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f0cd80098d0 tx=0x7f0cd801c650 comp rx=0 tx=0).stop 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0cdffff640 1 -- 172.21.15.76:0/3121460485 reap_dead start 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- 172.21.15.76:0/3121460485 shutdown_connections 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- 172.21.15.76:0/3121460485 >> 172.21.15.76:0/3121460485 conn(0x7f0ce00b1f90 msgr2=0x7f0ce00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- 172.21.15.76:0/3121460485 shutdown_connections 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- 172.21.15.76:0/3121460485 wait complete. 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 Processor -- start 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- start start 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0ce014b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ce014d480 0x7f0ce00b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0159fc0 0x7f0ce015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0073a50 con 0x7f0cc8004d50 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0ce0073770 con 0x7f0ce014d480 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.560+0000 7f0ce5d52640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0ce0073d30 con 0x7f0ce0159fc0 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0ce014b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0ce014b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57744/0 (socket says 172.21.15.76:57744) 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 -- 172.21.15.76:0/2476928791 learned_addr learned my addr 172.21.15.76:0/2476928791 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdeffd640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ce014d480 0x7f0ce00b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdffff640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0159fc0 0x7f0ce015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 -- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0159fc0 msgr2=0x7f0ce015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0159fc0 0x7f0ce015a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 -- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ce014d480 msgr2=0x7f0ce00b0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.566 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ce014d480 0x7f0ce00b0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 -- 172.21.15.76:0/2476928791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ce014e210 con 0x7f0cc8004d50 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdeffd640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ce014d480 0x7f0ce00b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdf7fe640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0ce014b6e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00079e0 tx=0x7f0cd000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cc7fff640 1 -- 172.21.15.76:0/2476928791 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0007ad0 con 0x7f0cc8004d50 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cc7fff640 1 -- 172.21.15.76:0/2476928791 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cd0018070 con 0x7f0cc8004d50 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cc7fff640 1 -- 172.21.15.76:0/2476928791 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0013420 con 0x7f0cc8004d50 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ce015bf90 con 0x7f0cc8004d50 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ce015c550 con 0x7f0cc8004d50 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ce00b0ba0 con 0x7f0cc8004d50 2024-09-19T04:20:29.567 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cc7fff640 1 -- 172.21.15.76:0/2476928791 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f0cd00135c0 con 0x7f0cc8004d50 2024-09-19T04:20:29.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cc7fff640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0ccc067a60 0x7f0ccc069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cc7fff640 1 -- 172.21.15.76:0/2476928791 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f0cd0086400 con 0x7f0cc8004d50 2024-09-19T04:20:29.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdeffd640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0ccc067a60 0x7f0ccc069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.564+0000 7f0cdeffd640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0ccc067a60 0x7f0ccc069f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00af6c0 tx=0x7f0cd4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.568+0000 7f0cc7fff640 1 -- 172.21.15.76:0/2476928791 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f0ce014e210 con 0x7f0cc8004d50 2024-09-19T04:20:29.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.760+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0ce0068e50 con 0x7f0cc8004d50 2024-09-19T04:20:29.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.760+0000 7f0cc7fff640 1 -- 172.21.15.76:0/2476928791 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f0cd0056d80 con 0x7f0cc8004d50 2024-09-19T04:20:29.762 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:29.762 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:29.762 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0ccc067a60 msgr2=0x7f0ccc069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0ccc067a60 0x7f0ccc069f20 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f0ce00af6c0 tx=0x7f0cd4008040 comp rx=0 tx=0).stop 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 msgr2=0x7f0ce014b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0ce014b6e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00079e0 tx=0x7f0cd000e530 comp rx=0 tx=0).stop 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 shutdown_connections 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f0ccc067a60 0x7f0ccc069f20 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f0ce0159fc0 0x7f0ce015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ce014d480 0x7f0ce00b0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 --2- 172.21.15.76:0/2476928791 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f0cc8004d50 0x7f0ce014b6e0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 >> 172.21.15.76:0/2476928791 conn(0x7f0ce00b1f90 msgr2=0x7f0ce006e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.765 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 shutdown_connections 2024-09-19T04:20:29.766 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.764+0000 7f0ce5d52640 1 -- 172.21.15.76:0/2476928791 wait complete. 2024-09-19T04:20:29.780 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:29.780 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:29.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3d4fdc640 1 Processor -- start 2024-09-19T04:20:29.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3d4fdc640 1 -- start start 2024-09-19T04:20:29.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3d4fdc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3d01520a0 0x7ff3d006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3d4fdc640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff3d015f3a0 0x7ff3d00adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3d4fdc640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff3d0070250 con 0x7ff3d006fe50 2024-09-19T04:20:29.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3d4fdc640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d0152480 con 0x7ff3d01520a0 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3d4fdc640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff3d00aede0 con 0x7ff3d015f3a0 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ce59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3d01520a0 0x7ff3d006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ce59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff3d01520a0 0x7ff3d006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57750/0 (socket says 172.21.15.76:57750) 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ce59c640 1 -- 172.21.15.76:0/1464675657 learned_addr learned my addr 172.21.15.76:0/1464675657 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ced9d640 1 --2- 172.21.15.76:0/1464675657 >> v2:172.21.15.116:3300/0 conn(0x7ff3d015f3a0 0x7ff3d00adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ce59c640 1 -- 172.21.15.76:0/1464675657 >> v2:172.21.15.116:3300/0 conn(0x7ff3d015f3a0 msgr2=0x7ff3d00adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ce59c640 1 --2- 172.21.15.76:0/1464675657 >> v2:172.21.15.116:3300/0 conn(0x7ff3d015f3a0 0x7ff3d00adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ce59c640 1 -- 172.21.15.76:0/1464675657 >> v1:172.21.15.76:6789/0 conn(0x7ff3d006fe50 legacy=0x7ff3d015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.900+0000 7ff3ce59c640 1 -- 172.21.15.76:0/1464675657 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d015fa10 con 0x7ff3d01520a0 2024-09-19T04:20:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3ce59c640 1 --2- 172.21.15.76:0/1464675657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d01520a0 0x7ff3d006f910 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0155b40 tx=0x7ff3c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120bc8423ee23ee3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3cd59a640 1 -- 172.21.15.76:0/1464675657 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00ab610 con 0x7ff3d01520a0 2024-09-19T04:20:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3cd59a640 1 -- 172.21.15.76:0/1464675657 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c00b0070 con 0x7ff3d01520a0 2024-09-19T04:20:29.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3cd59a640 1 -- 172.21.15.76:0/1464675657 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00ab9d0 con 0x7ff3d01520a0 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/1464675657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d01520a0 msgr2=0x7ff3d006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/1464675657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d01520a0 0x7ff3d006f910 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0155b40 tx=0x7ff3c001c520 comp rx=0 tx=0).stop 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/1464675657 shutdown_connections 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/1464675657 >> v2:172.21.15.116:3300/0 conn(0x7ff3d015f3a0 0x7ff3d00adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/1464675657 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d01520a0 0x7ff3d006f910 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/1464675657 >> 172.21.15.76:0/1464675657 conn(0x7ff3d00b1f90 msgr2=0x7ff3d00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/1464675657 shutdown_connections 2024-09-19T04:20:29.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.904+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/1464675657 wait complete. 2024-09-19T04:20:29.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 Processor -- start 2024-09-19T04:20:29.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 -- start start 2024-09-19T04:20:29.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d006fe50 0x7ff3d00b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff3d01520a0 0x7ff3d0150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3d015f3a0 0x7ff3d014fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0073d30 con 0x7ff3d006fe50 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0073a50 con 0x7ff3d015f3a0 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d0167f10 con 0x7ff3d01520a0 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d006fe50 0x7ff3d00b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d006fe50 0x7ff3d00b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57764/0 (socket says 172.21.15.76:57764) 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 -- 172.21.15.76:0/445594976 learned_addr learned my addr 172.21.15.76:0/445594976 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 -- 172.21.15.76:0/445594976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff3d01520a0 msgr2=0x7ff3d0150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff3d01520a0 0x7ff3d0150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 -- 172.21.15.76:0/445594976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3d015f3a0 msgr2=0x7ff3d014fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:29.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ced9d640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3d015f3a0 0x7ff3d014fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3d015f3a0 0x7ff3d014fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 -- 172.21.15.76:0/445594976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d014cd80 con 0x7ff3d006fe50 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3ce59c640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d006fe50 0x7ff3d00b1970 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00ad040 tx=0x7ff3c001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3c6ffd640 1 -- 172.21.15.76:0/445594976 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00be5f0 con 0x7ff3d006fe50 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3c6ffd640 1 -- 172.21.15.76:0/445594976 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3c00be790 con 0x7ff3d006fe50 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d014caa0 con 0x7ff3d006fe50 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3c6ffd640 1 -- 172.21.15.76:0/445594976 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c00beb40 con 0x7ff3d006fe50 2024-09-19T04:20:29.910 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d00b1190 con 0x7ff3d006fe50 2024-09-19T04:20:29.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d006f910 con 0x7ff3d006fe50 2024-09-19T04:20:29.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3c6ffd640 1 -- 172.21.15.76:0/445594976 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff3c00b43f0 con 0x7ff3d006fe50 2024-09-19T04:20:29.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3c6ffd640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff3b00678c0 0x7ff3b0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:29.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3c6ffd640 1 -- 172.21.15.76:0/445594976 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff3c0120440 con 0x7ff3d006fe50 2024-09-19T04:20:29.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.908+0000 7ff3cdd9b640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff3b00678c0 0x7ff3b0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:29.911 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.912+0000 7ff3cdd9b640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff3b00678c0 0x7ff3b0069d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8000f70 tx=0x7ff3b80079c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:29.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:29.916+0000 7ff3c6ffd640 1 -- 172.21.15.76:0/445594976 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff3c00c4060 con 0x7ff3d006fe50 2024-09-19T04:20:30.132 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.132+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7ff3d015c1a0 con 0x7ff3d006fe50 2024-09-19T04:20:30.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.132+0000 7ff3c6ffd640 1 -- 172.21.15.76:0/445594976 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2016 (secure 0 0 0) 0x7ff3c00a8050 con 0x7ff3d006fe50 2024-09-19T04:20:30.133 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:30.134 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700292,"bytes_used":267501568,"bytes_avail":772826611712,"bytes_total":773094113280,"read_bytes_sec":0,"write_bytes_sec":5715,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-09-19T04:20:27.873487+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/d1/d2/d3]"}}}}}},"progress_events":{}} 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff3b00678c0 msgr2=0x7ff3b0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff3b00678c0 0x7ff3b0069d80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8000f70 tx=0x7ff3b80079c0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d006fe50 msgr2=0x7ff3d00b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d006fe50 0x7ff3d00b1970 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff3c00ad040 tx=0x7ff3c001ad60 comp rx=0 tx=0).stop 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 shutdown_connections 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff3b00678c0 0x7ff3b0069d80 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3d015f3a0 0x7ff3d014fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff3d01520a0 0x7ff3d0150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 --2- 172.21.15.76:0/445594976 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff3d006fe50 0x7ff3d00b1970 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:30.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 >> 172.21.15.76:0/445594976 conn(0x7ff3d00b1f90 msgr2=0x7ff3d01655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:30.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 shutdown_connections 2024-09-19T04:20:30.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:30.136+0000 7ff3d4fdc640 1 -- 172.21.15.76:0/445594976 wait complete. 2024-09-19T04:20:30.151 INFO:tasks.ceph.ceph_manager.ceph:{'a': 'paused paths [/d1/d2/d3]', 'c': 'idle+waiting paths [/d1/d2/d3]', 'd': 'paused paths [/d1/d2/d3]'} 2024-09-19T04:20:31.152 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:31.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.280+0000 7f308ff7b640 1 Processor -- start 2024-09-19T04:20:31.282 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.280+0000 7f308ff7b640 1 -- start start 2024-09-19T04:20:31.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308ff7b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 0x7f30880ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308ff7b640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f308815c2a0 0x7f30880aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308ff7b640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30880b3290 con 0x7f30880af2a0 2024-09-19T04:20:31.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308ff7b640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f30880b3570 con 0x7f308815c2a0 2024-09-19T04:20:31.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308ff7b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f30880b3850 con 0x7f308815bec0 2024-09-19T04:20:31.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308e518640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f30880af2a0 0x7f308815fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52876/0 (socket says 172.21.15.76:52876) 2024-09-19T04:20:31.283 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308e518640 1 -- 172.21.15.76:0/3385732605 learned_addr learned my addr 172.21.15.76:0/3385732605 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:31.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 --2- 172.21.15.76:0/3385732605 >> v2:172.21.15.116:3300/0 conn(0x7f308815c2a0 0x7f30880aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308dd17640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 0x7f30880ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.285 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2814451103 0 0) 0x7f30880b3290 con 0x7f30880af2a0 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3074003530 con 0x7f30880af2a0 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1936503564 0 0) 0x7f3074003530 con 0x7f30880af2a0 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 msgr2=0x7f30880ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 0x7f30880ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 >> v2:172.21.15.116:3300/0 conn(0x7f308815c2a0 msgr2=0x7f30880aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 --2- 172.21.15.76:0/3385732605 >> v2:172.21.15.116:3300/0 conn(0x7f308815c2a0 0x7f30880aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30880b3b30 con 0x7f30880af2a0 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f3080001680 con 0x7f30880af2a0 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 >> v1:172.21.15.76:6789/0 conn(0x7f30880af2a0 legacy=0x7f308815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 0x7f30740039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3074004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.286 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3074004d50 0x7f30740051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30880b3570 con 0x7f3074003f90 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f30880b3850 con 0x7f3074004d50 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f307400b9d0 con 0x7f308815bec0 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3074004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308dd17640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 0x7f30740039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 -- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 msgr2=0x7f30740039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 0x7f30740039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 -- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3074004d50 msgr2=0x7f30740051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308e518640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3074004d50 0x7f30740051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3074004d50 0x7f30740051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 -- 172.21.15.76:0/3385732605 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3074003530 con 0x7f3074003f90 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308dd17640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f308815bec0 0x7f30740039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:31.287 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308e518640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3074004d50 0x7f30740051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:31.290 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.284+0000 7f308d516640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3074004810 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f3084009f90 tx=0x7f308401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8819a12074de8c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30840b2070 con 0x7f3074003f90 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30840a5c50 con 0x7f3074003f90 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308cd15640 1 -- 172.21.15.76:0/3385732605 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3084009c10 con 0x7f3074003f90 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- 172.21.15.76:0/3385732605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 msgr2=0x7f3074004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 --2- 172.21.15.76:0/3385732605 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3074004810 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f3084009f90 tx=0x7f308401cc40 comp rx=0 tx=0).stop 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308e518640 1 -- 172.21.15.76:0/3385732605 reap_dead start 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- 172.21.15.76:0/3385732605 shutdown_connections 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- 172.21.15.76:0/3385732605 >> 172.21.15.76:0/3385732605 conn(0x7f30880b1f90 msgr2=0x7f30880b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- 172.21.15.76:0/3385732605 shutdown_connections 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- 172.21.15.76:0/3385732605 wait complete. 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 Processor -- start 2024-09-19T04:20:31.291 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- start start 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3088141190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3088144d30 0x7f308813f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f308813f640 0x7f3088152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f30880af730 con 0x7f3074003f90 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f30880b40f0 con 0x7f308813f640 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f30880afc90 con 0x7f3088144d30 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3088141190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3088141190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57784/0 (socket says 172.21.15.76:57784) 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 -- 172.21.15.76:0/3961748440 learned_addr learned my addr 172.21.15.76:0/3961748440 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308e518640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f308813f640 0x7f3088152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308d516640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3088144d30 0x7f308813f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 -- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3088144d30 msgr2=0x7f308813f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.292 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3088144d30 0x7f308813f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 -- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f308813f640 msgr2=0x7f3088152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f308813f640 0x7f3088152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 -- 172.21.15.76:0/3961748440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30881420c0 con 0x7f3074003f90 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308e518640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f308813f640 0x7f3088152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308dd17640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3088141190 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f307c0108d0 tx=0x7f307c010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f307a7fc640 1 -- 172.21.15.76:0/3961748440 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00f040 con 0x7f3074003f90 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f307a7fc640 1 -- 172.21.15.76:0/3961748440 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f307c004220 con 0x7f3074003f90 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3088141de0 con 0x7f3074003f90 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f307a7fc640 1 -- 172.21.15.76:0/3961748440 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00e960 con 0x7f3074003f90 2024-09-19T04:20:31.293 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.288+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3088154ae0 con 0x7f3074003f90 2024-09-19T04:20:31.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.292+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30880aed60 con 0x7f3074003f90 2024-09-19T04:20:31.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.292+0000 7f307a7fc640 1 -- 172.21.15.76:0/3961748440 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f307c00eb00 con 0x7f3074003f90 2024-09-19T04:20:31.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.292+0000 7f307a7fc640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678c0 0x7f306c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.292+0000 7f307a7fc640 1 -- 172.21.15.76:0/3961748440 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f307c085e60 con 0x7f3074003f90 2024-09-19T04:20:31.294 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.292+0000 7f308d516640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678c0 0x7f306c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.295 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.292+0000 7f308d516640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678c0 0x7f306c069d80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f30840af000 tx=0x7f30840a7040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.297 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.296+0000 7f307a7fc640 1 -- 172.21.15.76:0/3961748440 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f307c018030 con 0x7f3074003f90 2024-09-19T04:20:31.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.484+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30881580d0 con 0x7f3074003f90 2024-09-19T04:20:31.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.484+0000 7f307a7fc640 1 -- 172.21.15.76:0/3961748440 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f307c0567e0 con 0x7f3074003f90 2024-09-19T04:20:31.486 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:31.487 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:31.487 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:31.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678c0 msgr2=0x7f306c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678c0 0x7f306c069d80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f30840af000 tx=0x7f30840a7040 comp rx=0 tx=0).stop 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 msgr2=0x7f3088141190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3088141190 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f307c0108d0 tx=0x7f307c010d90 comp rx=0 tx=0).stop 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 shutdown_connections 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f306c0678c0 0x7f306c069d80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f308813f640 0x7f3088152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f3088144d30 0x7f308813f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 --2- 172.21.15.76:0/3961748440 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f3074003f90 0x7f3088141190 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 >> 172.21.15.76:0/3961748440 conn(0x7f30880b1f90 msgr2=0x7f3088159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 shutdown_connections 2024-09-19T04:20:31.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.488+0000 7f308ff7b640 1 -- 172.21.15.76:0/3961748440 wait complete. 2024-09-19T04:20:31.504 DEBUG:tasks.cephfs.test_multimds_misc:current active=['a', 'c', 'd'] 2024-09-19T04:20:31.504 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-19T04:20:31.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f956557d640 1 Processor -- start 2024-09-19T04:20:31.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f956557d640 1 -- start start 2024-09-19T04:20:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f956557d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f95600ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f956557d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f956015c2a0 0x7f95600aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f956557d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95600b3290 con 0x7f95600af2a0 2024-09-19T04:20:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f956557d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f95600b3570 con 0x7f956015c2a0 2024-09-19T04:20:31.622 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f956557d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f95600b3850 con 0x7f956015bec0 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955e7fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f956015c2a0 0x7f95600aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f95600af2a0 0x7f956015fc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52880/0 (socket says 172.21.15.76:52880) 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955f7fe640 1 -- 172.21.15.76:0/3068498398 learned_addr learned my addr 172.21.15.76:0/3068498398 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955effd640 1 --2- 172.21.15.76:0/3068498398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f95600ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955e7fc640 1 -- 172.21.15.76:0/3068498398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 msgr2=0x7f95600ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955e7fc640 1 --2- 172.21.15.76:0/3068498398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f95600ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955e7fc640 1 -- 172.21.15.76:0/3068498398 >> v1:172.21.15.76:6789/0 conn(0x7f95600af2a0 legacy=0x7f956015fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955e7fc640 1 -- 172.21.15.76:0/3068498398 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95600b3b30 con 0x7f956015c2a0 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955dffb640 1 -- 172.21.15.76:0/3068498398 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2497348351 0 0) 0x7f95600b3290 con 0x7f95600af2a0 2024-09-19T04:20:31.623 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.620+0000 7f955effd640 1 --2- 172.21.15.76:0/3068498398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f95600ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:31.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f955e7fc640 1 --2- 172.21.15.76:0/3068498398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f95600aed60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9550009d90 tx=0x7f955001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=310bc27a31eb5adf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f955dffb640 1 -- 172.21.15.76:0/3068498398 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500b2070 con 0x7f956015c2a0 2024-09-19T04:20:31.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f955dffb640 1 -- 172.21.15.76:0/3068498398 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95500a5a00 con 0x7f956015c2a0 2024-09-19T04:20:31.625 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f955dffb640 1 -- 172.21.15.76:0/3068498398 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95500a5dd0 con 0x7f956015c2a0 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 -- 172.21.15.76:0/3068498398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 msgr2=0x7f95600aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 --2- 172.21.15.76:0/3068498398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f95600aed60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f9550009d90 tx=0x7f955001c940 comp rx=0 tx=0).stop 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 -- 172.21.15.76:0/3068498398 shutdown_connections 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 --2- 172.21.15.76:0/3068498398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f95600aed60 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 --2- 172.21.15.76:0/3068498398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f95600ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 -- 172.21.15.76:0/3068498398 >> 172.21.15.76:0/3068498398 conn(0x7f95600b1f90 msgr2=0x7f95600b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 -- 172.21.15.76:0/3068498398 shutdown_connections 2024-09-19T04:20:31.626 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 -- 172.21.15.76:0/3068498398 wait complete. 2024-09-19T04:20:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 Processor -- start 2024-09-19T04:20:31.628 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.624+0000 7f956557d640 1 -- start start 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600af2a0 0x7f956015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f956014f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f9560152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f95600af6f0 con 0x7f956015c2a0 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f95600ad740 con 0x7f95600af2a0 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9560169000 con 0x7f956015bec0 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f9560152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f9560152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57796/0 (socket says 172.21.15.76:57796) 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 -- 172.21.15.76:0/2404494667 learned_addr learned my addr 172.21.15.76:0/2404494667 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955effd640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600af2a0 0x7f956015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955e7fc640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f956014f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 -- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 msgr2=0x7f956014f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f956014f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 -- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600af2a0 msgr2=0x7f956015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600af2a0 0x7f956015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 -- 172.21.15.76:0/2404494667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956015ce50 con 0x7f956015c2a0 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955effd640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600af2a0 0x7f956015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955f7fe640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f9560152620 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f955800c4a0 tx=0x7f955800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f95477fe640 1 -- 172.21.15.76:0/2404494667 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558018070 con 0x7f956015c2a0 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f95477fe640 1 -- 172.21.15.76:0/2404494667 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95580074e0 con 0x7f956015c2a0 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f95477fe640 1 -- 172.21.15.76:0/2404494667 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9558004a30 con 0x7f956015c2a0 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f956015cb70 con 0x7f956015c2a0 2024-09-19T04:20:31.630 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f956014ec60 con 0x7f956015c2a0 2024-09-19T04:20:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95600ad200 con 0x7f956015c2a0 2024-09-19T04:20:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f95477fe640 1 -- 172.21.15.76:0/2404494667 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9558004bd0 con 0x7f956015c2a0 2024-09-19T04:20:31.631 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f95477fe640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f95477fe640 1 -- 172.21.15.76:0/2404494667 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f9558085000 con 0x7f956015c2a0 2024-09-19T04:20:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955effd640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:31.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.628+0000 7f955effd640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9554008930 tx=0x7f9554008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:31.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.632+0000 7f95477fe640 1 -- 172.21.15.76:0/2404494667 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9558055a10 con 0x7f956015c2a0 2024-09-19T04:20:31.846 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.844+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f956006fcd0 con 0x7f956015c2a0 2024-09-19T04:20:31.847 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f95477fe640 1 -- 172.21.15.76:0/2404494667 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1914 (secure 0 0 0) 0x7f9558055310 con 0x7f956015c2a0 2024-09-19T04:20:31.847 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:31.847 INFO:teuthology.orchestra.run.smithi076.stdout:{"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","health":{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"3 MDSs report damaged metadata","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":450,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":106,"num_osds":8,"num_up_osds":8,"osd_up_since":1726719192,"num_in_osds":8,"osd_in_since":1726719183,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":70,"data_bytes":700292,"bytes_used":267501568,"bytes_avail":772826611712,"bytes_total":773094113280,"write_bytes_sec":682,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","id":7,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":7,"rank":2,"name":"d","status":"up:active","gid":6127},{"filesystem_id":7,"rank":1,"name":"c","status":"up:active","gid":8115},{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":8127}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-09-19T04:20:29.873992+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/d1/d2/d3]"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-19T04:20:31.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 msgr2=0x7f954c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9554008930 tx=0x7f9554008040 comp rx=0 tx=0).stop 2024-09-19T04:20:31.849 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 msgr2=0x7f9560152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f9560152620 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f955800c4a0 tx=0x7f955800c960 comp rx=0 tx=0).stop 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 shutdown_connections 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f954c0678c0 0x7f954c069d80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f956015c2a0 0x7f9560152620 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f956015bec0 0x7f956014f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 --2- 172.21.15.76:0/2404494667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95600af2a0 0x7f956015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 >> 172.21.15.76:0/2404494667 conn(0x7f95600b1f90 msgr2=0x7f95600cfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 shutdown_connections 2024-09-19T04:20:31.850 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:31.848+0000 7f956557d640 1 -- 172.21.15.76:0/2404494667 wait complete. 2024-09-19T04:20:31.865 INFO:tasks.ceph.ceph_manager.ceph:{'c': 'idle+waiting paths [/d1/d2/d3]'} 2024-09-19T04:20:31.865 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-19T04:20:31.865 DEBUG:teuthology.orchestra.run.smithi076:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:20:31.928 DEBUG:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-19T04:20:31.990 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-19T04:20:31.990 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:20:31.990 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:20:31.990 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:31.996 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-19T04:20:32.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724f17d640 1 Processor -- start 2024-09-19T04:20:32.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724f17d640 1 -- start start 2024-09-19T04:20:32.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724f17d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f724806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724f17d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f724806fe50 0x7f724815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724f17d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7248070250 con 0x7f724815f1d0 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724f17d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f7248152480 con 0x7f724806fe50 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724f17d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f72480aede0 con 0x7f72481520a0 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f723ffff640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f724806fe50 0x7f724815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724d71a640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f724815f1d0 0x7f7248071f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:52886/0 (socket says 172.21.15.76:52886) 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724d71a640 1 -- 172.21.15.76:0/750262097 learned_addr learned my addr 172.21.15.76:0/750262097 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724cf19640 1 --2- 172.21.15.76:0/750262097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f724806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f723ffff640 1 -- 172.21.15.76:0/750262097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 msgr2=0x7f724806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f723ffff640 1 --2- 172.21.15.76:0/750262097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f724806f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f723ffff640 1 -- 172.21.15.76:0/750262097 >> v1:172.21.15.76:6789/0 conn(0x7f724815f1d0 legacy=0x7f7248071f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f723f7fe640 1 -- 172.21.15.76:0/750262097 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3472991510 0 0) 0x7f7248070250 con 0x7f724815f1d0 2024-09-19T04:20:32.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f723ffff640 1 -- 172.21.15.76:0/750262097 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f724815fa10 con 0x7f724806fe50 2024-09-19T04:20:32.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.164+0000 7f724cf19640 1 --2- 172.21.15.76:0/750262097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f724806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:32.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f723ffff640 1 --2- 172.21.15.76:0/750262097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f724815ec90 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f72440097c0 tx=0x7f724401e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9267f9ee86f5ac29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f723f7fe640 1 -- 172.21.15.76:0/750262097 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440ab680 con 0x7f724806fe50 2024-09-19T04:20:32.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f723f7fe640 1 -- 172.21.15.76:0/750262097 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72440ab820 con 0x7f724806fe50 2024-09-19T04:20:32.169 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f723f7fe640 1 -- 172.21.15.76:0/750262097 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72440abbf0 con 0x7f724806fe50 2024-09-19T04:20:32.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 -- 172.21.15.76:0/750262097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 msgr2=0x7f724815ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 --2- 172.21.15.76:0/750262097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f724815ec90 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f72440097c0 tx=0x7f724401e690 comp rx=0 tx=0).stop 2024-09-19T04:20:32.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 -- 172.21.15.76:0/750262097 shutdown_connections 2024-09-19T04:20:32.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 --2- 172.21.15.76:0/750262097 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f724815ec90 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.170 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 --2- 172.21.15.76:0/750262097 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f724806f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 -- 172.21.15.76:0/750262097 >> 172.21.15.76:0/750262097 conn(0x7f72480b1f90 msgr2=0x7f72480b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:32.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 -- 172.21.15.76:0/750262097 shutdown_connections 2024-09-19T04:20:32.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.168+0000 7f724f17d640 1 -- 172.21.15.76:0/750262097 wait complete. 2024-09-19T04:20:32.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 Processor -- start 2024-09-19T04:20:32.171 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 -- start start 2024-09-19T04:20:32.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f72481f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f72481491a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f724815f1d0 0x7f72481496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7248073d30 con 0x7f724806fe50 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7248073a50 con 0x7f724815f1d0 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f724815f640 con 0x7f72481520a0 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f72481f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f72481f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:57818/0 (socket says 172.21.15.76:57818) 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 -- 172.21.15.76:0/2149146538 learned_addr learned my addr 172.21.15.76:0/2149146538 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724d71a640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f724815f1d0 0x7f72481496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f723ffff640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f72481491a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 -- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 msgr2=0x7f72481491a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f72481491a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 -- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f724815f1d0 msgr2=0x7f72481496e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f724815f1d0 0x7f72481496e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 -- 172.21.15.76:0/2149146538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7248164320 con 0x7f724806fe50 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724d71a640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f724815f1d0 0x7f72481496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724cf19640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f72481f6a10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f723800c7c0 tx=0x7f723800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f723d7fa640 1 -- 172.21.15.76:0/2149146538 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238007a30 con 0x7f724806fe50 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f723d7fa640 1 -- 172.21.15.76:0/2149146538 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7238007bd0 con 0x7f724806fe50 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f723d7fa640 1 -- 172.21.15.76:0/2149146538 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72380113f0 con 0x7f724806fe50 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7248164600 con 0x7f724806fe50 2024-09-19T04:20:32.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7248164bc0 con 0x7f724806fe50 2024-09-19T04:20:32.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.172+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724815ec90 con 0x7f724806fe50 2024-09-19T04:20:32.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.176+0000 7f723d7fa640 1 -- 172.21.15.76:0/2149146538 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7238011590 con 0x7f724806fe50 2024-09-19T04:20:32.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.176+0000 7f723d7fa640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72340678f0 0x7f7234069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.176+0000 7f723d7fa640 1 -- 172.21.15.76:0/2149146538 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f7238086050 con 0x7f724806fe50 2024-09-19T04:20:32.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.176+0000 7f723ffff640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72340678f0 0x7f7234069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.176+0000 7f723ffff640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72340678f0 0x7f7234069db0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f72440b0000 tx=0x7f72440a9000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.179 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.180+0000 7f723d7fa640 1 -- 172.21.15.76:0/2149146538 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7238015020 con 0x7f724806fe50 2024-09-19T04:20:32.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.344+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f724806e540 con 0x7f724806fe50 2024-09-19T04:20:32.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.344+0000 7f723d7fa640 1 -- 172.21.15.76:0/2149146538 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) ==== 80+0+0 (secure 0 0 0) 0x7f72380569d0 con 0x7f724806fe50 2024-09-19T04:20:32.345 INFO:teuthology.orchestra.run.smithi076.stderr:listed 0 entries 2024-09-19T04:20:32.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.344+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72340678f0 msgr2=0x7f7234069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.344+0000 7f724f17d640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72340678f0 0x7f7234069db0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f72440b0000 tx=0x7f72440a9000 comp rx=0 tx=0).stop 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 msgr2=0x7f72481f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f72481f6a10 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f723800c7c0 tx=0x7f723800cc80 comp rx=0 tx=0).stop 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 shutdown_connections 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f72340678f0 0x7f7234069db0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f724815f1d0 0x7f72481496e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f72481520a0 0x7f72481491a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 --2- 172.21.15.76:0/2149146538 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f724806fe50 0x7f72481f6a10 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 >> 172.21.15.76:0/2149146538 conn(0x7f72480b1f90 msgr2=0x7f724815c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 shutdown_connections 2024-09-19T04:20:32.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.348+0000 7f724f17d640 1 -- 172.21.15.76:0/2149146538 wait complete. 2024-09-19T04:20:32.362 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi076.front.sepia.ceph.com... 2024-09-19T04:20:32.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:32.363 DEBUG:teuthology.orchestra.run.smithi076:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-19T04:20:32.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi076.stderr:ceph-fuse[49271]: fuse finished with error 0 and tester_r 0 2024-09-19T04:20:32.373 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:20:32.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:32.373 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T04:20:32.536 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:20:32.537 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:20:32.537 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:32.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:20:32.544 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:20:32.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:32.544 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-19T04:20:32.589 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-09-19T04:20:32.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:20:32.591 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:20:32.591 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:20:32.591 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:32.597 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:20:32.597 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:20:32.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:32.597 DEBUG:teuthology.orchestra.run.smithi076:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-19T04:20:32.643 INFO:teuthology.orchestra.run.smithi076.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-09-19T04:20:32.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:20:32.644 INFO:tasks.cephfs.mount:Terminating background process 2024-09-19T04:20:32.644 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:20:32.644 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:32.650 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:20:32.650 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:20:32.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:32.651 DEBUG:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-19T04:20:32.697 INFO:teuthology.orchestra.run.smithi116.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-09-19T04:20:32.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:20:32.699 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bef1af640 1 Processor -- start 2024-09-19T04:20:32.828 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bef1af640 1 -- start start 2024-09-19T04:20:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bef1af640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 0x7f8be815d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bef1af640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be81533f0 con 0x7f8be80b3990 2024-09-19T04:20:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bef1af640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8be8073850 con 0x7f8be8072dd0 2024-09-19T04:20:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bef1af640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8be8073b30 con 0x7f8be8153010 2024-09-19T04:20:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bed74c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f8be80b3990 0x7f8be815fb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47888/0 (socket says 172.21.15.76:47888) 2024-09-19T04:20:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bed74c640 1 -- 172.21.15.76:0/1010583348 learned_addr learned my addr 172.21.15.76:0/1010583348 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:32.831 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8becf4b640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 0x7f8be815d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635990450 0 0) 0x7f8be00014a0 con 0x7f8be80b3990 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8bd0003530 con 0x7f8be80b3990 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 860654871 0 0) 0x7f8bd0003530 con 0x7f8be80b3990 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 msgr2=0x7f8be815d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 0x7f8be815d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 >> v1:172.21.15.116:6789/0 conn(0x7f8be8072dd0 legacy=0x7f8be80b3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be815c890 con 0x7f8be80b3990 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f8be0001bd0 con 0x7f8be80b3990 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 >> v1:172.21.15.76:6789/0 conn(0x7f8be80b3990 legacy=0x7f8be815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bd00039f0 0x7f8bd0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bd0004810 0x7f8bd0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 0x7f8bd00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8be8073b30 con 0x7f8bd0004810 2024-09-19T04:20:32.832 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd000ba80 con 0x7f8bd00039f0 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd000bc80 con 0x7f8be8153010 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be7fff640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bd0004810 0x7f8bd0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8bed74c640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 0x7f8bd00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8becf4b640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bd00039f0 0x7f8bd0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be7fff640 1 -- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 msgr2=0x7f8bd00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be7fff640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be8153010 0x7f8bd00051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be7fff640 1 -- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bd00039f0 msgr2=0x7f8bd0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be7fff640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bd00039f0 0x7f8bd0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8be7fff640 1 -- 172.21.15.76:0/1010583348 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd0003530 con 0x7f8bd0004810 2024-09-19T04:20:32.833 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.828+0000 7f8becf4b640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8bd00039f0 0x7f8bd0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:32.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8be7fff640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bd0004810 0x7f8bd0004c90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc001750 tx=0x7f8bdc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a899fe2f89bba44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.836 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0013a0 con 0x7f8bd0004810 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bdc0052f0 con 0x7f8bd0004810 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8be77fe640 1 -- 172.21.15.76:0/1010583348 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc003840 con 0x7f8bd0004810 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 -- 172.21.15.76:0/1010583348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bd0004810 msgr2=0x7f8bd0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 --2- 172.21.15.76:0/1010583348 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8bd0004810 0x7f8bd0004c90 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc001750 tx=0x7f8bdc01c3c0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bed74c640 1 -- 172.21.15.76:0/1010583348 reap_dead start 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 -- 172.21.15.76:0/1010583348 shutdown_connections 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 -- 172.21.15.76:0/1010583348 >> 172.21.15.76:0/1010583348 conn(0x7f8be80b6870 msgr2=0x7f8be80b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 -- 172.21.15.76:0/1010583348 shutdown_connections 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 -- 172.21.15.76:0/1010583348 wait complete. 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 Processor -- start 2024-09-19T04:20:32.837 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.832+0000 7f8bef1af640 1 -- start start 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8be814f3d0 0x7f8be814ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be814f7b0 0x7f8be81511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8be8147720 0x7f8be8147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f8be806fe00 con 0x7f8be814f3d0 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f8be815c540 con 0x7f8be8147720 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f8be80731f0 con 0x7f8be814f7b0 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8be814f3d0 0x7f8be814ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8be814f3d0 0x7f8be814ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52230/0 (socket says 172.21.15.76:52230) 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 -- 172.21.15.76:0/4206527151 learned_addr learned my addr 172.21.15.76:0/4206527151 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be7fff640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be814f7b0 0x7f8be81511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.838 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bed74c640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8be8147720 0x7f8be8147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 -- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be814f7b0 msgr2=0x7f8be81511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be814f7b0 0x7f8be81511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 -- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8be8147720 msgr2=0x7f8be8147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8be8147720 0x7f8be8147ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 -- 172.21.15.76:0/4206527151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be8148710 con 0x7f8be814f3d0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8becf4b640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8be814f3d0 0x7f8be814ad60 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8010890 tx=0x7f8bd8010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be57fa640 1 -- 172.21.15.76:0/4206527151 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800f040 con 0x7f8be814f3d0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be57fa640 1 -- 172.21.15.76:0/4206527151 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd8004220 con 0x7f8be814f3d0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be814b360 con 0x7f8be814f3d0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be57fa640 1 -- 172.21.15.76:0/4206527151 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd800e960 con 0x7f8be814f3d0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be81480e0 con 0x7f8be814f3d0 2024-09-19T04:20:32.839 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be8148ae0 con 0x7f8be814f3d0 2024-09-19T04:20:32.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be57fa640 1 -- 172.21.15.76:0/4206527151 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f8bd800eb00 con 0x7f8be814f3d0 2024-09-19T04:20:32.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be57fa640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bd40678f0 0x7f8bd4069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:32.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be57fa640 1 -- 172.21.15.76:0/4206527151 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f8bd8086130 con 0x7f8be814f3d0 2024-09-19T04:20:32.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be7fff640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bd40678f0 0x7f8bd4069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:32.840 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.836+0000 7f8be7fff640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bd40678f0 0x7f8bd4069db0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00057f0 tx=0x7f8bdc0011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:32.842 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:32.840+0000 7f8be57fa640 1 -- 172.21.15.76:0/4206527151 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8bd8018030 con 0x7f8be814f3d0 2024-09-19T04:20:33.024 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.024+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8be81e7850 con 0x7f8be814f3d0 2024-09-19T04:20:33.025 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.024+0000 7f8be57fa640 1 -- 172.21.15.76:0/4206527151 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f8bd8056b40 con 0x7f8be814f3d0 2024-09-19T04:20:33.025 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:33.026 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:33.026 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:33.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bd40678f0 msgr2=0x7f8bd4069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bd40678f0 0x7f8bd4069db0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00057f0 tx=0x7f8bdc0011a0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8be814f3d0 msgr2=0x7f8be814ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8be814f3d0 0x7f8be814ad60 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8010890 tx=0x7f8bd8010d50 comp rx=0 tx=0).stop 2024-09-19T04:20:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 shutdown_connections 2024-09-19T04:20:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f8bd40678f0 0x7f8bd4069db0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8be8147720 0x7f8be8147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f8be814f7b0 0x7f8be81511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 --2- 172.21.15.76:0/4206527151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f8be814f3d0 0x7f8be814ad60 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 >> 172.21.15.76:0/4206527151 conn(0x7f8be80b6870 msgr2=0x7f8be80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 shutdown_connections 2024-09-19T04:20:33.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.028+0000 7f8bef1af640 1 -- 172.21.15.76:0/4206527151 wait complete. 2024-09-19T04:20:33.043 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:33.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e77133640 1 Processor -- start 2024-09-19T04:20:33.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e77133640 1 -- start start 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e77133640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6e7006f010 0x7f6e70166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e77133640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e700b3290 con 0x7f6e7015bec0 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e77133640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e700b3570 con 0x7f6e7015c2a0 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e77133640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6e700b3850 con 0x7f6e7006f010 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e74ecf640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6e7015bec0 0x7f6e700ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47894/0 (socket says 172.21.15.76:47894) 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e74ecf640 1 -- 172.21.15.76:0/1070499115 learned_addr learned my addr 172.21.15.76:0/1070499115 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e756d0640 1 --2- 172.21.15.76:0/1070499115 >> v2:172.21.15.116:3300/0 conn(0x7f6e7006f010 0x7f6e70166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1860316583 0 0) 0x7f6e700b3290 con 0x7f6e7015bec0 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e5c003530 con 0x7f6e7015bec0 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2663500507 0 0) 0x7f6e5c003530 con 0x7f6e7015bec0 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 >> v2:172.21.15.116:3300/0 conn(0x7f6e7006f010 msgr2=0x7f6e70166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.151+0000 7f6e6f7fe640 1 --2- 172.21.15.76:0/1070499115 >> v2:172.21.15.116:3300/0 conn(0x7f6e7006f010 0x7f6e70166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 >> v1:172.21.15.116:6789/0 conn(0x7f6e7015c2a0 legacy=0x7f6e700aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e700b3b30 con 0x7f6e7015bec0 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6e68001a00 con 0x7f6e7015bec0 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 >> v1:172.21.15.76:6789/0 conn(0x7f6e7015bec0 legacy=0x7f6e700ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c0039c0 0x7f6e5c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e5c004810 0x7f6e5c004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e5c0051d0 0x7f6e5c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700b3850 con 0x7f6e5c004810 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5c00c070 con 0x7f6e5c0051d0 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5c00c270 con 0x7f6e5c0039c0 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6ffff640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e5c004810 0x7f6e5c004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e756d0640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e5c0051d0 0x7f6e5c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6ffff640 1 -- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c0039c0 msgr2=0x7f6e5c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e74ecf640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c0039c0 0x7f6e5c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6ffff640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c0039c0 0x7f6e5c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6ffff640 1 -- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e5c0051d0 msgr2=0x7f6e5c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6ffff640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e5c0051d0 0x7f6e5c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6ffff640 1 -- 172.21.15.76:0/1070499115 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5c003530 con 0x7f6e5c004810 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e756d0640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e5c0051d0 0x7f6e5c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e74ecf640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c0039c0 0x7f6e5c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:33.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e756d0640 1 -- 172.21.15.76:0/1070499115 reap_dead start 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6ffff640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e5c004810 0x7f6e5c004c90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f6e58001750 tx=0x7f6e5801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14e193a677c338b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e580013a0 con 0x7f6e5c004810 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e580052f0 con 0x7f6e5c004810 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e6f7fe640 1 -- 172.21.15.76:0/1070499115 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58003840 con 0x7f6e5c004810 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e77133640 1 -- 172.21.15.76:0/1070499115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e5c004810 msgr2=0x7f6e5c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e77133640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e5c004810 0x7f6e5c004c90 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f6e58001750 tx=0x7f6e5801c3c0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e77133640 1 -- 172.21.15.76:0/1070499115 shutdown_connections 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e77133640 1 --2- 172.21.15.76:0/1070499115 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e5c004810 0x7f6e5c004c90 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e77133640 1 -- 172.21.15.76:0/1070499115 >> 172.21.15.76:0/1070499115 conn(0x7f6e700b1f90 msgr2=0x7f6e700b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e77133640 1 -- 172.21.15.76:0/1070499115 shutdown_connections 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.155+0000 7f6e77133640 1 -- 172.21.15.76:0/1070499115 wait complete. 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 Processor -- start 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 -- start start 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c004810 0x7f6e701516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e70153930 0x7f6e7015dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e70153db0 0x7f6e7015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700afc90 con 0x7f6e70153930 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700b40f0 con 0x7f6e70153db0 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6e700ad200 con 0x7f6e5c004810 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e70153930 0x7f6e7015dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e70153930 0x7f6e7015dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52256/0 (socket says 172.21.15.76:52256) 2024-09-19T04:20:33.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 -- 172.21.15.76:0/307132491 learned_addr learned my addr 172.21.15.76:0/307132491 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e756d0640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e70153db0 0x7f6e7015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 -- 172.21.15.76:0/307132491 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c004810 msgr2=0x7f6e701516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c004810 0x7f6e701516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 -- 172.21.15.76:0/307132491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e70153db0 msgr2=0x7f6e7015e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e70153db0 0x7f6e7015e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 -- 172.21.15.76:0/307132491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7015cc20 con 0x7f6e70153930 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6ffff640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e70153930 0x7f6e7015dc80 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6e5801c1f0 tx=0x7f6e5c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6d7fa640 1 -- 172.21.15.76:0/307132491 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58007a40 con 0x7f6e70153930 2024-09-19T04:20:33.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6d7fa640 1 -- 172.21.15.76:0/307132491 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e580ba070 con 0x7f6e70153930 2024-09-19T04:20:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7015cec0 con 0x7f6e70153930 2024-09-19T04:20:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6d7fa640 1 -- 172.21.15.76:0/307132491 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58007270 con 0x7f6e70153930 2024-09-19T04:20:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7015d480 con 0x7f6e70153930 2024-09-19T04:20:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e700ad590 con 0x7f6e70153930 2024-09-19T04:20:33.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6d7fa640 1 -- 172.21.15.76:0/307132491 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6e580052f0 con 0x7f6e70153930 2024-09-19T04:20:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.159+0000 7f6e6d7fa640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e500678f0 0x7f6e50069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.163+0000 7f6e6d7fa640 1 -- 172.21.15.76:0/307132491 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f6e5811ff30 con 0x7f6e70153930 2024-09-19T04:20:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.163+0000 7f6e74ecf640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e500678f0 0x7f6e50069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.163+0000 7f6e74ecf640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e500678f0 0x7f6e50069db0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000f90 tx=0x7f6e68001260 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.167 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.167+0000 7f6e6d7fa640 1 -- 172.21.15.76:0/307132491 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6e580160d0 con 0x7f6e70153930 2024-09-19T04:20:33.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.351+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e7006fcd0 con 0x7f6e70153930 2024-09-19T04:20:33.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.351+0000 7f6e6d7fa640 1 -- 172.21.15.76:0/307132491 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7f6e580b33b0 con 0x7f6e70153930 2024-09-19T04:20:33.354 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:33.355 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:33.355 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:33.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e500678f0 msgr2=0x7f6e50069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e500678f0 0x7f6e50069db0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000f90 tx=0x7f6e68001260 comp rx=0 tx=0).stop 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e70153930 msgr2=0x7f6e7015dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e70153930 0x7f6e7015dc80 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6e5801c1f0 tx=0x7f6e5c005840 comp rx=0 tx=0).stop 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 shutdown_connections 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6e500678f0 0x7f6e50069db0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6e70153db0 0x7f6e7015e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6e70153930 0x7f6e7015dc80 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 --2- 172.21.15.76:0/307132491 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6e5c004810 0x7f6e701516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 >> 172.21.15.76:0/307132491 conn(0x7f6e700b1f90 msgr2=0x7f6e7014e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 shutdown_connections 2024-09-19T04:20:33.357 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.355+0000 7f6e77133640 1 -- 172.21.15.76:0/307132491 wait complete. 2024-09-19T04:20:33.371 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:20:33.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cfbc1640 1 Processor -- start 2024-09-19T04:20:33.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cfbc1640 1 -- start start 2024-09-19T04:20:33.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cfbc1640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5c806c3d0 0x7ff5c815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cfbc1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 0x7ff5c815e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cfbc1640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5c815fc10 con 0x7ff5c815f830 2024-09-19T04:20:33.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cfbc1640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff5c815da40 con 0x7ff5c806c3d0 2024-09-19T04:20:33.478 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cfbc1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c815d800 con 0x7ff5c815d290 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd15c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff5c806c3d0 0x7ff5c815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd95d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff5c815f830 0x7ff5c806bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47900/0 (socket says 172.21.15.76:47900) 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd95d640 1 -- 172.21.15.76:0/3884009944 learned_addr learned my addr 172.21.15.76:0/3884009944 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd15c640 1 --2- 172.21.15.76:0/3884009944 >> v2:172.21.15.76:3300/0 conn(0x7ff5c806c3d0 0x7ff5c815ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52270/0 (socket says 172.21.15.76:52270) 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd15c640 1 -- 172.21.15.76:0/3884009944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 msgr2=0x7ff5c815e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd15c640 1 --2- 172.21.15.76:0/3884009944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 0x7ff5c815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd15c640 1 -- 172.21.15.76:0/3884009944 >> v1:172.21.15.76:6789/0 conn(0x7ff5c815f830 legacy=0x7ff5c806bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cd15c640 1 -- 172.21.15.76:0/3884009944 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80b3570 con 0x7ff5c806c3d0 2024-09-19T04:20:33.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.475+0000 7ff5cc95b640 1 -- 172.21.15.76:0/3884009944 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3221802632 0 0) 0x7ff5c815fc10 con 0x7ff5c815f830 2024-09-19T04:20:33.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cd15c640 1 --2- 172.21.15.76:0/3884009944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c806c3d0 0x7ff5c815ded0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80097c0 tx=0x7ff5b801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb82493adf26302d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cc95b640 1 -- 172.21.15.76:0/3884009944 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80a7070 con 0x7ff5c806c3d0 2024-09-19T04:20:33.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cc95b640 1 -- 172.21.15.76:0/3884009944 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5b80b0070 con 0x7ff5c806c3d0 2024-09-19T04:20:33.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cc95b640 1 -- 172.21.15.76:0/3884009944 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5b80ab890 con 0x7ff5c806c3d0 2024-09-19T04:20:33.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/3884009944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c806c3d0 msgr2=0x7ff5c815ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/3884009944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c806c3d0 0x7ff5c815ded0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80097c0 tx=0x7ff5b801c470 comp rx=0 tx=0).stop 2024-09-19T04:20:33.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/3884009944 shutdown_connections 2024-09-19T04:20:33.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/3884009944 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 0x7ff5c815e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/3884009944 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c806c3d0 0x7ff5c815ded0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.481 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/3884009944 >> 172.21.15.76:0/3884009944 conn(0x7ff5c80b6800 msgr2=0x7ff5c80b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/3884009944 shutdown_connections 2024-09-19T04:20:33.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/3884009944 wait complete. 2024-09-19T04:20:33.483 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 Processor -- start 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.479+0000 7ff5cfbc1640 1 -- start start 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806c3d0 0x7ff5c8159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 0x7ff5c809d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c815f830 0x7ff5c809de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8169390 con 0x7ff5c815f830 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff5c8169210 con 0x7ff5c806c3d0 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff5c8169510 con 0x7ff5c815d290 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c815f830 0x7ff5c809de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c815f830 0x7ff5c809de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52276/0 (socket says 172.21.15.76:52276) 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 -- 172.21.15.76:0/1187702843 learned_addr learned my addr 172.21.15.76:0/1187702843 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cd95d640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806c3d0 0x7ff5c8159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cd15c640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 0x7ff5c809d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 -- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 msgr2=0x7ff5c809d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 0x7ff5c809d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 -- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806c3d0 msgr2=0x7ff5c8159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806c3d0 0x7ff5c8159920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 -- 172.21.15.76:0/1187702843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5c80b2cd0 con 0x7ff5c815f830 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5ce15e640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c815f830 0x7ff5c809de10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b970 tx=0x7ff5c000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5be7fc640 1 -- 172.21.15.76:0/1187702843 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0019070 con 0x7ff5c815f830 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5be7fc640 1 -- 172.21.15.76:0/1187702843 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5c000cb60 con 0x7ff5c815f830 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5c80b1f10 con 0x7ff5c815f830 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5be7fc640 1 -- 172.21.15.76:0/1187702843 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0014440 con 0x7ff5c815f830 2024-09-19T04:20:33.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5c80b24d0 con 0x7ff5c815f830 2024-09-19T04:20:33.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5c806bcc0 con 0x7ff5c815f830 2024-09-19T04:20:33.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5be7fc640 1 -- 172.21.15.76:0/1187702843 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff5c001c070 con 0x7ff5c815f830 2024-09-19T04:20:33.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5be7fc640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5ac0678c0 0x7ff5ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5be7fc640 1 -- 172.21.15.76:0/1187702843 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff5c0086020 con 0x7ff5c815f830 2024-09-19T04:20:33.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cd95d640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5ac0678c0 0x7ff5ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.483+0000 7ff5cd95d640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5ac0678c0 0x7ff5ac069d80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4000f70 tx=0x7ff5c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.489 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.487+0000 7ff5be7fc640 1 -- 172.21.15.76:0/1187702843 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff5c0023020 con 0x7ff5c815f830 2024-09-19T04:20:33.640 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.639+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5c806e7c0 con 0x7ff5c815f830 2024-09-19T04:20:33.643 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5be7fc640 1 -- 172.21.15.76:0/1187702843 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17066 (secure 0 0 0) 0x7ff5c00569f0 con 0x7ff5c815f830 2024-09-19T04:20:33.643 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:33.644 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":106,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:20:09.017016+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:19:43.130613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T04:19:43.548160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.24","mappings":[{"from":5,"to":4}]},{"pgid":"16.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5ac0678c0 msgr2=0x7ff5ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5ac0678c0 0x7ff5ac069d80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4000f70 tx=0x7ff5c4009040 comp rx=0 tx=0).stop 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c815f830 msgr2=0x7ff5c809de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c815f830 0x7ff5c809de10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000b970 tx=0x7ff5c000be30 comp rx=0 tx=0).stop 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 shutdown_connections 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff5ac0678c0 0x7ff5ac069d80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff5c815f830 0x7ff5c809de10 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff5c815d290 0x7ff5c809d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 --2- 172.21.15.76:0/1187702843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5c806c3d0 0x7ff5c8159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 >> 172.21.15.76:0/1187702843 conn(0x7ff5c80b6800 msgr2=0x7ff5c806ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 shutdown_connections 2024-09-19T04:20:33.646 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.643+0000 7ff5cfbc1640 1 -- 172.21.15.76:0/1187702843 wait complete. 2024-09-19T04:20:33.661 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:20:33.661 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 Processor -- start 2024-09-19T04:20:33.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 -- start start 2024-09-19T04:20:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb4541520a0 0x7fb45406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb45415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb45415f1d0 0x7fb454071f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fb454152480 con 0x7fb45415f1d0 2024-09-19T04:20:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fb454070250 con 0x7fb4541520a0 2024-09-19T04:20:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb45b717640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb4540aede0 con 0x7fb45406fe50 2024-09-19T04:20:33.781 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb45415f1d0 0x7fb454071f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fb45415f1d0 0x7fb454071f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52284/0 (socket says 172.21.15.76:52284) 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 -- 172.21.15.76:0/1228140624 learned_addr learned my addr 172.21.15.76:0/1228140624 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb4594b3640 1 --2- 172.21.15.76:0/1228140624 >> v2:172.21.15.116:3300/0 conn(0x7fb4541520a0 0x7fb45406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb458cb2640 1 --2- 172.21.15.76:0/1228140624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb45415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 -- 172.21.15.76:0/1228140624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 msgr2=0x7fb45415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 --2- 172.21.15.76:0/1228140624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb45415ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 -- 172.21.15.76:0/1228140624 >> v2:172.21.15.116:3300/0 conn(0x7fb4541520a0 msgr2=0x7fb45406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 --2- 172.21.15.76:0/1228140624 >> v2:172.21.15.116:3300/0 conn(0x7fb4541520a0 0x7fb45406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb459cb4640 1 -- 172.21.15.76:0/1228140624 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45415fa10 con 0x7fb45415f1d0 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb4594b3640 1 --2- 172.21.15.76:0/1228140624 >> v2:172.21.15.116:3300/0 conn(0x7fb4541520a0 0x7fb45406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:33.782 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.779+0000 7fb458cb2640 1 --2- 172.21.15.76:0/1228140624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb45415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:33.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 --2- 172.21.15.76:0/1228140624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb454071f10 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb454071710 tx=0x7fb44c01c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e987b233f17d855 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb44bfff640 1 -- 172.21.15.76:0/1228140624 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0a5590 con 0x7fb45415f1d0 2024-09-19T04:20:33.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb44bfff640 1 -- 172.21.15.76:0/1228140624 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44c0a5730 con 0x7fb45415f1d0 2024-09-19T04:20:33.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb44bfff640 1 -- 172.21.15.76:0/1228140624 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0a5b00 con 0x7fb45415f1d0 2024-09-19T04:20:33.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- 172.21.15.76:0/1228140624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 msgr2=0x7fb454071f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 --2- 172.21.15.76:0/1228140624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb454071f10 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb454071710 tx=0x7fb44c01c710 comp rx=0 tx=0).stop 2024-09-19T04:20:33.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- 172.21.15.76:0/1228140624 shutdown_connections 2024-09-19T04:20:33.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 --2- 172.21.15.76:0/1228140624 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb454071f10 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.784 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 --2- 172.21.15.76:0/1228140624 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb45415ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 --2- 172.21.15.76:0/1228140624 >> v2:172.21.15.116:3300/0 conn(0x7fb4541520a0 0x7fb45406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- 172.21.15.76:0/1228140624 >> 172.21.15.76:0/1228140624 conn(0x7fb4540b1f90 msgr2=0x7fb4540b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- 172.21.15.76:0/1228140624 shutdown_connections 2024-09-19T04:20:33.785 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- 172.21.15.76:0/1228140624 wait complete. 2024-09-19T04:20:33.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 Processor -- start 2024-09-19T04:20:33.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- start start 2024-09-19T04:20:33.787 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb4540b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4541520a0 0x7fb45409b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb45414ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fb454073d30 con 0x7fb45415f1d0 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fb454073a50 con 0x7fb4541520a0 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb45b717640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fb45415f680 con 0x7fb45406fe50 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb45414ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb45414ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52296/0 (socket says 172.21.15.76:52296) 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 -- 172.21.15.76:0/831330690 learned_addr learned my addr 172.21.15.76:0/831330690 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb458cb2640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4541520a0 0x7fb45409b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb4594b3640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb4540b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 -- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 msgr2=0x7fb4540b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.788 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb4540b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 -- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4541520a0 msgr2=0x7fb45409b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4541520a0 0x7fb45409b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 -- 172.21.15.76:0/831330690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45414d730 con 0x7fb45415f1d0 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb4594b3640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb4540b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.783+0000 7fb459cb4640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb45414ccb0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0af040 tx=0x7fb44c0194b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb449ffb640 1 -- 172.21.15.76:0/831330690 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0b4440 con 0x7fb45415f1d0 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb449ffb640 1 -- 172.21.15.76:0/831330690 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44c0b45e0 con 0x7fb45415f1d0 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb449ffb640 1 -- 172.21.15.76:0/831330690 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0b49d0 con 0x7fb45415f1d0 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45414d1f0 con 0x7fb45415f1d0 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45414f3e0 con 0x7fb45415f1d0 2024-09-19T04:20:33.789 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45414f620 con 0x7fb45415f1d0 2024-09-19T04:20:33.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb449ffb640 1 -- 172.21.15.76:0/831330690 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fb44c0c5060 con 0x7fb45415f1d0 2024-09-19T04:20:33.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb449ffb640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb4380678c0 0x7fb438069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:33.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb449ffb640 1 -- 172.21.15.76:0/831330690 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7fb44c122310 con 0x7fb45415f1d0 2024-09-19T04:20:33.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb4594b3640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb4380678c0 0x7fb438069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:33.790 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.787+0000 7fb4594b3640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb4380678c0 0x7fb438069d80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b4900 tx=0x7fb450008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:33.793 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.791+0000 7fb449ffb640 1 -- 172.21.15.76:0/831330690 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fb45414d730 con 0x7fb45415f1d0 2024-09-19T04:20:33.990 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb45406e540 con 0x7fb45415f1d0 2024-09-19T04:20:33.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb449ffb640 1 -- 172.21.15.76:0/831330690 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7fb44c0b9070 con 0x7fb45415f1d0 2024-09-19T04:20:33.992 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:33.992 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:33.992 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb4380678c0 msgr2=0x7fb438069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb4380678c0 0x7fb438069d80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b4900 tx=0x7fb450008040 comp rx=0 tx=0).stop 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 msgr2=0x7fb45414ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb45414ccb0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0af040 tx=0x7fb44c0194b0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 shutdown_connections 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fb4380678c0 0x7fb438069d80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fb45415f1d0 0x7fb45414ccb0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4541520a0 0x7fb45409b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 --2- 172.21.15.76:0/831330690 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fb45406fe50 0x7fb4540b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:33.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 >> 172.21.15.76:0/831330690 conn(0x7fb4540b1f90 msgr2=0x7fb4540b3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:33.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 shutdown_connections 2024-09-19T04:20:33.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:33.991+0000 7fb45b717640 1 -- 172.21.15.76:0/831330690 wait complete. 2024-09-19T04:20:34.011 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:20:34.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d9512640 1 Processor -- start 2024-09-19T04:20:34.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d9512640 1 -- start start 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d9512640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d40d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d9512640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff9d40cf620 0x7ff9d406d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d9512640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9d40d32b0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d9512640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7ff9d40d0300 con 0x7ff9d40cf620 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d9512640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d40b0f60 con 0x7ff9d4161a60 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d37fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7ff9d40cf620 0x7ff9d406d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9ca7fc640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ff9d40d2e90 0x7ff9d40cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47914/0 (socket says 172.21.15.76:47914) 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9ca7fc640 1 -- 172.21.15.76:0/2627044992 learned_addr learned my addr 172.21.15.76:0/2627044992 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d2ffd640 1 --2- 172.21.15.76:0/2627044992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d40d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d37fe640 1 -- 172.21.15.76:0/2627044992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 msgr2=0x7ff9d40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d37fe640 1 --2- 172.21.15.76:0/2627044992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d40d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d37fe640 1 -- 172.21.15.76:0/2627044992 >> v1:172.21.15.76:6789/0 conn(0x7ff9d40d2e90 legacy=0x7ff9d40cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.131+0000 7ff9d37fe640 1 -- 172.21.15.76:0/2627044992 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d40cfad0 con 0x7ff9d40cf620 2024-09-19T04:20:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d37fe640 1 --2- 172.21.15.76:0/2627044992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40cf620 0x7ff9d406d6b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009d90 tx=0x7ff9cc01c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc7b72f3a79a6f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d27fc640 1 -- 172.21.15.76:0/2627044992 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0b2070 con 0x7ff9d40cf620 2024-09-19T04:20:34.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d27fc640 1 -- 172.21.15.76:0/2627044992 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0a5740 con 0x7ff9d40cf620 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d27fc640 1 -- 172.21.15.76:0/2627044992 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0a5b10 con 0x7ff9d40cf620 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- 172.21.15.76:0/2627044992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40cf620 msgr2=0x7ff9d406d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 --2- 172.21.15.76:0/2627044992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40cf620 0x7ff9d406d6b0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009d90 tx=0x7ff9cc01c740 comp rx=0 tx=0).stop 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- 172.21.15.76:0/2627044992 shutdown_connections 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 --2- 172.21.15.76:0/2627044992 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40cf620 0x7ff9d406d6b0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 --2- 172.21.15.76:0/2627044992 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d40d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- 172.21.15.76:0/2627044992 >> 172.21.15.76:0/2627044992 conn(0x7ff9d408bf90 msgr2=0x7ff9d409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- 172.21.15.76:0/2627044992 shutdown_connections 2024-09-19T04:20:34.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- 172.21.15.76:0/2627044992 wait complete. 2024-09-19T04:20:34.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 Processor -- start 2024-09-19T04:20:34.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- start start 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d40cf620 0x7ff9d4158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40d2e90 0x7ff9d40b1550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d4158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d406e400 con 0x7ff9d40d2e90 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d406d190 con 0x7ff9d40cf620 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d9512640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff9d406e580 con 0x7ff9d4161a60 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40d2e90 0x7ff9d40b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40d2e90 0x7ff9d40b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52314/0 (socket says 172.21.15.76:52314) 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 -- 172.21.15.76:0/96540702 learned_addr learned my addr 172.21.15.76:0/96540702 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d37fe640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d4158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9d2ffd640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d40cf620 0x7ff9d4158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 -- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 msgr2=0x7ff9d4158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d4158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 -- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d40cf620 msgr2=0x7ff9d4158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d40cf620 0x7ff9d4158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 -- 172.21.15.76:0/96540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d40b1fe0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9ca7fc640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40d2e90 0x7ff9d40b1550 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c800 tx=0x7ff9bc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.135+0000 7ff9cbfff640 1 -- 172.21.15.76:0/96540702 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc018070 con 0x7ff9d40d2e90 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9cbfff640 1 -- 172.21.15.76:0/96540702 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9bc0072d0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d40b22c0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9cbfff640 1 -- 172.21.15.76:0/96540702 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9bc0047f0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d40b2880 con 0x7ff9d40d2e90 2024-09-19T04:20:34.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9d406d6b0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9cbfff640 1 -- 172.21.15.76:0/96540702 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff9bc004990 con 0x7ff9d40d2e90 2024-09-19T04:20:34.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9cbfff640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9b40678c0 0x7ff9b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9d2ffd640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9b40678c0 0x7ff9b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9cbfff640 1 -- 172.21.15.76:0/96540702 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff9bc0853c0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.139+0000 7ff9d2ffd640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9b40678c0 0x7ff9b4069d80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400b9d0 tx=0x7ff9c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.143+0000 7ff9cbfff640 1 -- 172.21.15.76:0/96540702 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff9bc055cc0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.331+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9d40d1000 con 0x7ff9d40d2e90 2024-09-19T04:20:34.334 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9cbfff640 1 -- 172.21.15.76:0/96540702 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5279 (secure 0 0 0) 0x7ff9bc0555c0 con 0x7ff9d40d2e90 2024-09-19T04:20:34.335 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:34.335 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":112,"btime":"2024-09-19T04:20:19:087434+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-19T04:19:44.552866+0000","modified":"2024-09-19T04:20:18.988431+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":8127,"mds_1":8115,"mds_2":6127},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6127":{"gid":6127,"name":"d","rank":2,"incarnation":106,"state":"up:active","state_seq":11,"addr":"172.21.15.116:6835/255266111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":255266111},{"type":"v1","addr":"172.21.15.116:6835","nonce":255266111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8115":{"gid":8115,"name":"c","rank":1,"incarnation":104,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6835/2501269734","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":2501269734},{"type":"v1","addr":"172.21.15.76:6835","nonce":2501269734}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_8127":{"gid":8127,"name":"a","rank":0,"incarnation":99,"state":"up:active","state_seq":11,"addr":"172.21.15.76:6837/3881707282","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":3881707282},{"type":"v1","addr":"172.21.15.76:6837","nonce":3881707282}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8127,"qdb_cluster":[8127,8115,6127]},"id":7}]} 2024-09-19T04:20:34.335 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 112 2024-09-19T04:20:34.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9b40678c0 msgr2=0x7ff9b4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9b40678c0 0x7ff9b4069d80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff9c400b9d0 tx=0x7ff9c4008040 comp rx=0 tx=0).stop 2024-09-19T04:20:34.337 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40d2e90 msgr2=0x7ff9d40b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40d2e90 0x7ff9d40b1550 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc00c800 tx=0x7ff9bc00ccc0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 shutdown_connections 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff9b40678c0 0x7ff9b4069d80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff9d4161a60 0x7ff9d4158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff9d40d2e90 0x7ff9d40b1550 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 --2- 172.21.15.76:0/96540702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d40cf620 0x7ff9d4158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 >> 172.21.15.76:0/96540702 conn(0x7ff9d408bf90 msgr2=0x7ff9d40b5160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 shutdown_connections 2024-09-19T04:20:34.338 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.335+0000 7ff9d9512640 1 -- 172.21.15.76:0/96540702 wait complete. 2024-09-19T04:20:34.353 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:20:34.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.459+0000 7f43dd3cf640 1 Processor -- start 2024-09-19T04:20:34.462 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.459+0000 7f43dd3cf640 1 -- start start 2024-09-19T04:20:34.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43dd3cf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d80b39a0 0x7f43d806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43dd3cf640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d806ec60 con 0x7f43d806e4b0 2024-09-19T04:20:34.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43dd3cf640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d806ef40 con 0x7f43d81587a0 2024-09-19T04:20:34.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43dd3cf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f43d806f870 con 0x7f43d80b39a0 2024-09-19T04:20:34.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f43d806e4b0 0x7f43d8072160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47916/0 (socket says 172.21.15.76:47916) 2024-09-19T04:20:34.463 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d77fe640 1 -- 172.21.15.76:0/367214524 learned_addr learned my addr 172.21.15.76:0/367214524 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:34.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d80b39a0 0x7f43d806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 348183659 0 0) 0x7f43d806ec60 con 0x7f43d806e4b0 2024-09-19T04:20:34.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43c0003530 con 0x7f43d806e4b0 2024-09-19T04:20:34.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 301290057 0 0) 0x7f43c0003530 con 0x7f43d806e4b0 2024-09-19T04:20:34.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d80b39a0 msgr2=0x7f43d806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:34.466 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d80b39a0 0x7f43d806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 >> v1:172.21.15.116:6789/0 conn(0x7f43d81587a0 legacy=0x7f43d80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d806f220 con 0x7f43d806e4b0 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f43d0001a00 con 0x7f43d806e4b0 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 >> v1:172.21.15.76:6789/0 conn(0x7f43d806e4b0 legacy=0x7f43d8072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43c00039f0 0x7f43c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43c0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d80b39a0 0x7f43c00051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f43d806f870 con 0x7f43c0004810 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f43c000bab0 con 0x7f43c00039f0 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f43c000bcb0 con 0x7f43d80b39a0 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43c0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 -- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d80b39a0 msgr2=0x7f43c00051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d80b39a0 0x7f43c00051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.467 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 -- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43c00039f0 msgr2=0x7f43c0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d6ffd640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43c00039f0 0x7f43c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43c00039f0 0x7f43c0003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.468 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 -- 172.21.15.76:0/367214524 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c0003530 con 0x7f43c0004810 2024-09-19T04:20:34.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d67fc640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43c0004c90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009ab0 tx=0x7f43cc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3304a9da22782bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.471 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0b2070 con 0x7f43c0004810 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43cc0a5660 con 0x7f43c0004810 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.463+0000 7f43d5ffb640 1 -- 172.21.15.76:0/367214524 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0096b0 con 0x7f43c0004810 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- 172.21.15.76:0/367214524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 msgr2=0x7f43c0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 --2- 172.21.15.76:0/367214524 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43c0004c90 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009ab0 tx=0x7f43cc01ca70 comp rx=0 tx=0).stop 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d77fe640 1 -- 172.21.15.76:0/367214524 reap_dead start 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- 172.21.15.76:0/367214524 shutdown_connections 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- 172.21.15.76:0/367214524 >> 172.21.15.76:0/367214524 conn(0x7f43d80b1f90 msgr2=0x7f43d80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:34.472 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- 172.21.15.76:0/367214524 shutdown_connections 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- 172.21.15.76:0/367214524 wait complete. 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 Processor -- start 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- start start 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43d8153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d81f7370 0x7f43d8151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d809b570 0x7f43d809ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f43d8158b80 con 0x7f43c0004810 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f43d806e940 con 0x7f43d809b570 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f43d8153a30 con 0x7f43d81f7370 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43d8153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43d8153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52322/0 (socket says 172.21.15.76:52322) 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 -- 172.21.15.76:0/2074706377 learned_addr learned my addr 172.21.15.76:0/2074706377 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:34.473 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d77fe640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d809b570 0x7f43d809ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d67fc640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d81f7370 0x7f43d8151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 -- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d81f7370 msgr2=0x7f43d8151940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d81f7370 0x7f43d8151940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 -- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d809b570 msgr2=0x7f43d809ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d809b570 0x7f43d809ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 -- 172.21.15.76:0/2074706377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d8153d10 con 0x7f43c0004810 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d77fe640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d809b570 0x7f43d809ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43d6ffd640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43d8153450 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f43c800d540 tx=0x7f43c800da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43bf7fe640 1 -- 172.21.15.76:0/2074706377 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8005480 con 0x7f43c0004810 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43bf7fe640 1 -- 172.21.15.76:0/2074706377 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c8017070 con 0x7f43c0004810 2024-09-19T04:20:34.474 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d8153ff0 con 0x7f43c0004810 2024-09-19T04:20:34.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43bf7fe640 1 -- 172.21.15.76:0/2074706377 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c8003820 con 0x7f43c0004810 2024-09-19T04:20:34.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.467+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d81545b0 con 0x7f43c0004810 2024-09-19T04:20:34.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.471+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d8070080 con 0x7f43c0004810 2024-09-19T04:20:34.475 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.471+0000 7f43bf7fe640 1 -- 172.21.15.76:0/2074706377 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f43c80039c0 con 0x7f43c0004810 2024-09-19T04:20:34.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.471+0000 7f43bf7fe640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43c40678c0 0x7f43c4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.471+0000 7f43d67fc640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43c40678c0 0x7f43c4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.471+0000 7f43bf7fe640 1 -- 172.21.15.76:0/2074706377 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7f43c8085c80 con 0x7f43c0004810 2024-09-19T04:20:34.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.471+0000 7f43d67fc640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43c40678c0 0x7f43c4069d80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0af000 tx=0x7f43cc0046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.475+0000 7f43bf7fe640 1 -- 172.21.15.76:0/2074706377 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f43c8019310 con 0x7f43c0004810 2024-09-19T04:20:34.627 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.627+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f43d80b1800 con 0x7f43c0004810 2024-09-19T04:20:34.629 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.627+0000 7f43bf7fe640 1 -- 172.21.15.76:0/2074706377 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) ==== 74+0+17066 (secure 0 0 0) 0x7f43c8056560 con 0x7f43c0004810 2024-09-19T04:20:34.630 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:34.630 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":106,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:20:09.017016+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-09-19T04:19:43.130613+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-09-19T04:19:43.548160+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.24","mappings":[{"from":5,"to":4}]},{"pgid":"16.36","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43c40678c0 msgr2=0x7f43c4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43c40678c0 0x7f43c4069d80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0af000 tx=0x7f43cc0046d0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 msgr2=0x7f43d8153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43d8153450 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f43c800d540 tx=0x7f43c800da00 comp rx=0 tx=0).stop 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 shutdown_connections 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f43c40678c0 0x7f43c4069d80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f43d809b570 0x7f43d809ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.632 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f43d81f7370 0x7f43d8151940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 --2- 172.21.15.76:0/2074706377 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f43c0004810 0x7f43d8153450 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 >> 172.21.15.76:0/2074706377 conn(0x7f43d80b1f90 msgr2=0x7f43d80b0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:34.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 shutdown_connections 2024-09-19T04:20:34.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.631+0000 7f43dd3cf640 1 -- 172.21.15.76:0/2074706377 wait complete. 2024-09-19T04:20:34.648 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-19T04:20:34.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff571351640 1 Processor -- start 2024-09-19T04:20:34.768 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff571351640 1 -- start start 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff571351640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c0b3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff571351640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff56c06e680 0x7ff56c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff571351640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff56c06ec60 con 0x7ff56c0b37d0 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff571351640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ff56c06f870 con 0x7ff56c06e680 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff571351640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff56c0b3c10 con 0x7ff56c1587a0 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56affd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c0b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56b7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff56c06e680 0x7ff56c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56b7fe640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff56c06e680 0x7ff56c072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.76:44616/0 (socket says 172.21.15.76:44616) 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56a7fc640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff56c0b37d0 0x7ff56c06df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:41970/0 (socket says 172.21.15.76:41970) 2024-09-19T04:20:34.770 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56b7fe640 1 -- 172.21.15.76:0/4107304252 learned_addr learned my addr 172.21.15.76:0/4107304252 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56affd640 1 -- 172.21.15.76:0/4107304252 >> v2:172.21.15.116:3300/0 conn(0x7ff56c06e680 msgr2=0x7ff56c072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56affd640 1 --2- 172.21.15.76:0/4107304252 >> v2:172.21.15.116:3300/0 conn(0x7ff56c06e680 0x7ff56c072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56affd640 1 -- 172.21.15.76:0/4107304252 >> v1:172.21.15.116:6789/0 conn(0x7ff56c0b37d0 legacy=0x7ff56c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56affd640 1 -- 172.21.15.76:0/4107304252 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c06ef40 con 0x7ff56c1587a0 2024-09-19T04:20:34.771 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.767+0000 7ff56b7fe640 1 --2- 172.21.15.76:0/4107304252 >> v2:172.21.15.116:3300/0 conn(0x7ff56c06e680 0x7ff56c072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff56affd640 1 --2- 172.21.15.76:0/4107304252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c0b3290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009d90 tx=0x7ff55c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=700c9109d1dbc612 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff569ffb640 1 -- 172.21.15.76:0/4107304252 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0b2070 con 0x7ff56c1587a0 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff569ffb640 1 -- 172.21.15.76:0/4107304252 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c0a5910 con 0x7ff56c1587a0 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff569ffb640 1 -- 172.21.15.76:0/4107304252 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0a5ce0 con 0x7ff56c1587a0 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 -- 172.21.15.76:0/4107304252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 msgr2=0x7ff56c0b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 --2- 172.21.15.76:0/4107304252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c0b3290 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009d90 tx=0x7ff55c01c940 comp rx=0 tx=0).stop 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 -- 172.21.15.76:0/4107304252 shutdown_connections 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 --2- 172.21.15.76:0/4107304252 >> v2:172.21.15.116:3300/0 conn(0x7ff56c06e680 0x7ff56c072160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.774 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 --2- 172.21.15.76:0/4107304252 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c0b3290 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 -- 172.21.15.76:0/4107304252 >> 172.21.15.76:0/4107304252 conn(0x7ff56c0b1f90 msgr2=0x7ff56c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:34.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 -- 172.21.15.76:0/4107304252 shutdown_connections 2024-09-19T04:20:34.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.771+0000 7ff571351640 1 -- 172.21.15.76:0/4107304252 wait complete. 2024-09-19T04:20:34.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 Processor -- start 2024-09-19T04:20:34.775 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 -- start start 2024-09-19T04:20:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c06e680 0x7ff56c099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.777 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff56c0b37d0 0x7ff56c150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c073030 con 0x7ff56c06e680 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ff56c072eb0 con 0x7ff56c0b37d0 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ff56c0731b0 con 0x7ff56c1587a0 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c06e680 0x7ff56c099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c06e680 0x7ff56c099810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52332/0 (socket says 172.21.15.76:52332) 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 -- 172.21.15.76:0/802796834 learned_addr learned my addr 172.21.15.76:0/802796834 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56a7fc640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff56c0b37d0 0x7ff56c150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56b7fe640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 -- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 msgr2=0x7ff56c150e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.778 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c150e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 -- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff56c0b37d0 msgr2=0x7ff56c150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff56c0b37d0 0x7ff56c150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 -- 172.21.15.76:0/802796834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56c14dd00 con 0x7ff56c06e680 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff56affd640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c06e680 0x7ff56c099810 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0b0040 tx=0x7ff55c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff5637fe640 1 -- 172.21.15.76:0/802796834 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0b2050 con 0x7ff56c06e680 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff5637fe640 1 -- 172.21.15.76:0/802796834 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c0be070 con 0x7ff56c06e680 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff56c14dfe0 con 0x7ff56c06e680 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff5637fe640 1 -- 172.21.15.76:0/802796834 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0ada70 con 0x7ff56c06e680 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff56c14e5a0 con 0x7ff56c06e680 2024-09-19T04:20:34.779 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56c072160 con 0x7ff56c06e680 2024-09-19T04:20:34.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.775+0000 7ff5637fe640 1 -- 172.21.15.76:0/802796834 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ff55c0adc10 con 0x7ff56c06e680 2024-09-19T04:20:34.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.779+0000 7ff5637fe640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff54c0678c0 0x7ff54c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:34.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.779+0000 7ff5637fe640 1 -- 172.21.15.76:0/802796834 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 6159+0+0 (secure 0 0 0) 0x7ff55c1210a0 con 0x7ff56c06e680 2024-09-19T04:20:34.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.779+0000 7ff56a7fc640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff54c0678c0 0x7ff54c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:34.780 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.779+0000 7ff56a7fc640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff54c0678c0 0x7ff54c069d80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0b16a0 tx=0x7ff554007cd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:34.783 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.783+0000 7ff5637fe640 1 -- 172.21.15.76:0/802796834 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ff55c016560 con 0x7ff56c06e680 2024-09-19T04:20:34.982 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.979+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff56c070080 con 0x7ff56c06e680 2024-09-19T04:20:34.983 INFO:tasks.ceph.mon.a.smithi076.stderr:2024-09-19T04:20:34.983+0000 7fa674384640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T04:20:34.987 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.987+0000 7ff5637fe640 1 -- 172.21.15.76:0/802796834 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7ff55c0ab340 con 0x7ff56c06e680 2024-09-19T04:20:34.988 INFO:teuthology.orchestra.run.smithi076.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T04:20:34.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff54c0678c0 msgr2=0x7ff54c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff54c0678c0 0x7ff54c069d80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0b16a0 tx=0x7ff554007cd0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c06e680 msgr2=0x7ff56c099810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c06e680 0x7ff56c099810 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0b0040 tx=0x7ff55c0196e0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 shutdown_connections 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ff54c0678c0 0x7ff54c069d80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ff56c1587a0 0x7ff56c150e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff56c0b37d0 0x7ff56c150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 --2- 172.21.15.76:0/802796834 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ff56c06e680 0x7ff56c099810 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:34.992 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 >> 172.21.15.76:0/802796834 conn(0x7ff56c0b1f90 msgr2=0x7ff56c151d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 shutdown_connections 2024-09-19T04:20:34.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:34.991+0000 7ff571351640 1 -- 172.21.15.76:0/802796834 wait complete. 2024-09-19T04:20:34.998 INFO:tasks.ceph.mds.a.smithi076.stderr: -7971> 2024-09-19T04:19:58.148+0000 7f3e47568640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-19T04:20:35.007 INFO:tasks.ceph.mds.a.smithi076.stderr: -2594> 2024-09-19T04:20:15.192+0000 7f3e43560640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000002 [...2,head] /d1/d2/d3/ auth{1=1,2=1} v19 ap=1 f(v0 m2024-09-19T04:19:57.385751+0000 1=0+1) n(v3 rc2024-09-19T04:19:57.608824+0000 6=0+6) (inest mix dirty) (ifile excl) caps={8280=pAsLsXsFsx/AsLsXsFsx@10},l=8280 | dirtyscattered=1 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 0x558bc0175b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000002:[<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.385751+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.608824+0000 6=0+6)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.385751+0000 1=0+1)","memory_value.rstat":"n(v3 rc2024-09-19T04:19:57.608824+0000 6=0+6)","error_str":""},"return_code":-61} 2024-09-19T04:20:35.008 INFO:tasks.ceph.mds.a.smithi076.stderr: -2472> 2024-09-19T04:20:15.192+0000 7f3e49d6d640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000003 [...2,head] /d1/d2/d3/d4/ auth{1=1,2=1} v19 ap=2 f(v0 m2024-09-19T04:19:57.388277+0000 1=0+1) n(v4 rc2024-09-19T04:19:57.608824+0000 5=0+5) (inest sync->mix g=1,2) caps={8280=pAsLsXsFs/-@12} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=1 waiter=0 authpin=1 scrubqueue=0 export_pin=1 0x558bc0176100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000003:[<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v19>,<0x1/d1 v17>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.388277+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.608824+0000 5=0+5)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.388277+0000 1=0+1)","memory_value.rstat":"n(v4 rc2024-09-19T04:19:57.608824+0000 5=0+5)","error_str":""},"return_code":-61} 2024-09-19T04:20:35.009 INFO:tasks.ceph.mds.d.smithi116.stderr: -1105> 2024-09-19T04:20:15.193+0000 7f7e951de640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000006 [...2,head] /d1/d2/d3/d4/d5/d6/d7/ auth v4 ap=1 f(v0 m2024-09-19T04:19:57.396196+0000 1=0+1) n(v0 rc2024-09-19T04:19:57.396196+0000 2=0+2) (inest lock) (ifile excl) caps={8280=pAsLsXsFsx/-@6},l=8280 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=0 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x562e08673600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000006:[<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.396196+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.396196+0000 2=0+2)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.396196+0000 1=0+1)","memory_value.rstat":"n(v0 rc2024-09-19T04:19:57.396196+0000 2=0+2)","error_str":""},"return_code":-61} 2024-09-19T04:20:35.009 INFO:tasks.ceph.mds.d.smithi116.stderr: -1086> 2024-09-19T04:20:15.193+0000 7f7e951de640 -1 mds.2.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000007 [...2,head] /d1/d2/d3/d4/d5/d6/d7/d8/ auth v2 ap=1 f() n(v0 1=0+1) (inest lock) caps={8280=pAsLsXsFs/-@3},l=8280 | request=0 lock=0 importingcaps=0 dirfrag=0 caps=1 dirtyparent=0 replicated=0 dirty=0 authpin=1 scrubqueue=0 0x562e08673b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000007:[<0x10000000006/d8 v2>,<0x10000000005/d7 v4>,<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f()","memory_value.rstat":"n(v0 1=0+1)","error_str":""},"return_code":-61} 2024-09-19T04:20:35.013 INFO:tasks.ceph.mds.c.smithi076.stderr: -1220> 2024-09-19T04:20:15.192+0000 7faffbf0a640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000004 [...2,head] /d1/d2/d3/d4/d5/ auth{0=1,2=1} v15 ap=1 f(v0 m2024-09-19T04:19:57.390751+0000 1=0+1) n(v2 rc2024-09-19T04:19:57.608824+0000 4=0+4) (inest lock) caps={8280=pAsLsXsFs/-@7},l=8280 | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x564e4d11db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000004:[<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.390751+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.608824+0000 4=0+4)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.390751+0000 1=0+1)","memory_value.rstat":"n(v2 rc2024-09-19T04:19:57.608824+0000 4=0+4)","error_str":""},"return_code":-61} 2024-09-19T04:20:35.013 INFO:tasks.ceph.mds.c.smithi076.stderr: -1084> 2024-09-19T04:20:15.192+0000 7fb002717640 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000005 [...2,head] /d1/d2/d3/d4/d5/d6/ auth{0=1,2=1} v11 ap=2 f(v0 m2024-09-19T04:19:57.393459+0000 1=0+1) n(v1 rc2024-09-19T04:19:57.608824+0000 3=0+3) (inest sync->mix g=0,2 scatter_wanted) caps={8280=pAsLsXsFs/-@7} | request=0 lock=0 importingcaps=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 export_pin=2 0x564e4d11d080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x10000000005:[<0x10000000004/d6 v11>,<0x10000000003/d5 v15>,<0x10000000002/d4 v19>,<0x10000000001/d3 v19>,<0x10000000000/d2 v17>,<0x1/d1 v15>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T04:19:57.393459+0000 1=0+1)","ondisk_value.rstat":"n(v0 rc2024-09-19T04:19:57.396196+0000 3=0+3)","memory_value.dirstat":"f(v0 m2024-09-19T04:19:57.393459+0000 1=0+1)","memory_value.rstat":"n(v1 rc2024-09-19T04:19:57.608824+0000 3=0+3)","error_str":""},"return_code":-61} 2024-09-19T04:20:35.017 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T04:20:35.040 INFO:tasks.ceph.mds.a.smithi076.stdout:starting mds.a at 2024-09-19T04:20:35.041 INFO:tasks.ceph.mds.c.smithi076.stdout:starting mds.c at 2024-09-19T04:20:35.063 INFO:tasks.ceph.mds.d.smithi116.stdout:starting mds.d at 2024-09-19T04:20:35.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f60050d0640 1 Processor -- start 2024-09-19T04:20:35.131 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f60050d0640 1 -- start start 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f60050d0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6000161a60 0x7f60000d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f60050d0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60000cf620 0x7f600006d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f60050d0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60000d32b0 con 0x7f60000d2e90 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f60050d0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f60000d0300 con 0x7f60000cf620 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f60050d0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f60000b0f60 con 0x7f6000161a60 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5ffed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6000161a60 0x7f60000d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5fff59e640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60000cf620 0x7f600006d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5ffed9d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6000161a60 0x7f60000d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:39284/0 (socket says 172.21.15.76:39284) 2024-09-19T04:20:35.133 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5ffed9d640 1 -- 172.21.15.76:0/721683899 learned_addr learned my addr 172.21.15.76:0/721683899 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:35.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5fff59e640 1 -- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6000161a60 msgr2=0x7f60000d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:35.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5fff59e640 1 --2- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6000161a60 0x7f60000d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5fff59e640 1 -- 172.21.15.76:0/721683899 >> v1:172.21.15.116:6789/0 conn(0x7f60000d2e90 legacy=0x7f60000cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5fff59e640 1 -- 172.21.15.76:0/721683899 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60000cfad0 con 0x7f60000cf620 2024-09-19T04:20:35.134 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.131+0000 7f5ffed9d640 1 --2- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6000161a60 0x7f60000d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T04:20:35.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f5fff59e640 1 --2- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60000cf620 0x7f600006d6b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009d90 tx=0x7f5ff801c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73a199c49e67bf27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.135 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f5ffdd9b640 1 -- 172.21.15.76:0/721683899 <== mon.1 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80b2070 con 0x7f60000cf620 2024-09-19T04:20:35.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f5ffdd9b640 1 -- 172.21.15.76:0/721683899 <== mon.1 v2:172.21.15.116:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff80a5970 con 0x7f60000cf620 2024-09-19T04:20:35.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f5ffdd9b640 1 -- 172.21.15.76:0/721683899 <== mon.1 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff80a5d40 con 0x7f60000cf620 2024-09-19T04:20:35.136 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60000cf620 msgr2=0x7f600006d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 --2- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60000cf620 0x7f600006d6b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009d90 tx=0x7f5ff801c8b0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- 172.21.15.76:0/721683899 shutdown_connections 2024-09-19T04:20:35.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 --2- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60000cf620 0x7f600006d6b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 --2- 172.21.15.76:0/721683899 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6000161a60 0x7f60000d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- 172.21.15.76:0/721683899 >> 172.21.15.76:0/721683899 conn(0x7f600008bf90 msgr2=0x7f600009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- 172.21.15.76:0/721683899 shutdown_connections 2024-09-19T04:20:35.137 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- 172.21.15.76:0/721683899 wait complete. 2024-09-19T04:20:35.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 Processor -- start 2024-09-19T04:20:35.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- start start 2024-09-19T04:20:35.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60000cf620 0x7f60000b71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60000d2e90 0x7f60000b76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.139 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000161a60 0x7f6000140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f600006e400 con 0x7f60000d2e90 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f600006d190 con 0x7f6000161a60 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f60050d0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f600006e580 con 0x7f60000cf620 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f5ffe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60000d2e90 0x7f60000b76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f5ffe59c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60000d2e90 0x7f60000b76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52356/0 (socket says 172.21.15.76:52356) 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.135+0000 7f5ffe59c640 1 -- 172.21.15.76:0/534371686 learned_addr learned my addr 172.21.15.76:0/534371686 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fff59e640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000161a60 0x7f6000140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffed9d640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60000cf620 0x7f60000b71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffe59c640 1 -- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60000cf620 msgr2=0x7f60000b71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffe59c640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60000cf620 0x7f60000b71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.140 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffe59c640 1 -- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000161a60 msgr2=0x7f6000140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffe59c640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000161a60 0x7f6000140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffe59c640 1 -- 172.21.15.76:0/534371686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60000b9350 con 0x7f60000d2e90 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fff59e640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000161a60 0x7f6000140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffed9d640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60000cf620 0x7f60000b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffe59c640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60000d2e90 0x7f60000b76e0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c340 tx=0x7f5ff400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fe77fe640 1 -- 172.21.15.76:0/534371686 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40052d0 con 0x7f60000d2e90 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fe77fe640 1 -- 172.21.15.76:0/534371686 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff4005470 con 0x7f60000d2e90 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fe77fe640 1 -- 172.21.15.76:0/534371686 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4007260 con 0x7f60000d2e90 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60000b9630 con 0x7f60000d2e90 2024-09-19T04:20:35.141 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60000ba110 con 0x7f60000d2e90 2024-09-19T04:20:35.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f600006d6b0 con 0x7f60000d2e90 2024-09-19T04:20:35.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fe77fe640 1 -- 172.21.15.76:0/534371686 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ff4007400 con 0x7f60000d2e90 2024-09-19T04:20:35.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fe77fe640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5fec0678c0 0x7f5fec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:35.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffed9d640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5fec0678c0 0x7f5fec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:35.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5fe77fe640 1 -- 172.21.15.76:0/534371686 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6417+0+0 (secure 0 0 0) 0x7f5ff40853e0 con 0x7f60000d2e90 2024-09-19T04:20:35.142 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.139+0000 7f5ffed9d640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5fec0678c0 0x7f5fec069d80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009630 tx=0x7f5ff0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:35.145 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.143+0000 7f5fe77fe640 1 -- 172.21.15.76:0/534371686 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5ff4055c60 con 0x7f60000d2e90 2024-09-19T04:20:35.326 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.327+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f60000d1070 con 0x7f60000d2e90 2024-09-19T04:20:35.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f5fe77fe640 1 -- 172.21.15.76:0/534371686 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) ==== 104+0+0 (secure 0 0 0) 0x7f5ff4055560 con 0x7f60000d2e90 2024-09-19T04:20:35.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5fec0678c0 msgr2=0x7f5fec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5fec0678c0 0x7f5fec069d80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009630 tx=0x7f5ff0008040 comp rx=0 tx=0).stop 2024-09-19T04:20:35.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60000d2e90 msgr2=0x7f60000b76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:35.994 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60000d2e90 0x7f60000b76e0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400c340 tx=0x7f5ff400c800 comp rx=0 tx=0).stop 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 shutdown_connections 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5fec0678c0 0x7f5fec069d80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000161a60 0x7f6000140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f60000d2e90 0x7f60000b76e0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 --2- 172.21.15.76:0/534371686 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f60000cf620 0x7f60000b71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 >> 172.21.15.76:0/534371686 conn(0x7f600008bf90 msgr2=0x7f60000b53d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 shutdown_connections 2024-09-19T04:20:35.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:35.991+0000 7f60050d0640 1 -- 172.21.15.76:0/534371686 wait complete. 2024-09-19T04:20:36.016 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T04:20:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0fc49640 1 Processor -- start 2024-09-19T04:20:36.188 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0fc49640 1 -- start start 2024-09-19T04:20:36.189 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0fc49640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbb0806f010 0x7fbb08166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0fc49640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb080b3290 con 0x7fbb0815c2a0 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0fc49640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb080b3570 con 0x7fbb0815bec0 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0fc49640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fbb080b3850 con 0x7fbb0806f010 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0d1e4640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fbb0815c2a0 0x7fbb080aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47930/0 (socket says 172.21.15.76:47930) 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0d1e4640 1 -- 172.21.15.76:0/2564806378 learned_addr learned my addr 172.21.15.76:0/2564806378 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0e1e6640 1 --2- 172.21.15.76:0/2564806378 >> v2:172.21.15.116:3300/0 conn(0x7fbb0806f010 0x7fbb08166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3635894869 0 0) 0x7fbb080b3290 con 0x7fbb0815c2a0 2024-09-19T04:20:36.190 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbaf4003530 con 0x7fbb0815c2a0 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 632816615 0 0) 0x7fbaf4003530 con 0x7fbb0815c2a0 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 >> v2:172.21.15.116:3300/0 conn(0x7fbb0806f010 msgr2=0x7fbb08166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 --2- 172.21.15.76:0/2564806378 >> v2:172.21.15.116:3300/0 conn(0x7fbb0806f010 0x7fbb08166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 >> v1:172.21.15.116:6789/0 conn(0x7fbb0815bec0 legacy=0x7fbb080ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb080b3b30 con 0x7fbb0815c2a0 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fbaf8001680 con 0x7fbb0815c2a0 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 >> v1:172.21.15.76:6789/0 conn(0x7fbb0815c2a0 legacy=0x7fbb080aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaf40039c0 0x7fbaf4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaf4004810 0x7fbaf4004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40051d0 0x7fbaf400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.193 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb080b3850 con 0x7fbaf40039c0 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf400c070 con 0x7fbaf4004810 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf400c270 con 0x7fbaf40051d0 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.187+0000 7fbb0d9e5640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaf40039c0 0x7fbaf4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0e1e6640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40051d0 0x7fbaf400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0d1e4640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaf4004810 0x7fbaf4004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0d9e5640 1 -- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40051d0 msgr2=0x7fbaf400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0d9e5640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40051d0 0x7fbaf400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0d9e5640 1 -- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaf4004810 msgr2=0x7fbaf4004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0d9e5640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbaf4004810 0x7fbaf4004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0d9e5640 1 -- 172.21.15.76:0/2564806378 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf4003530 con 0x7fbaf40039c0 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0e1e6640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40051d0 0x7fbaf400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0e1e6640 1 -- 172.21.15.76:0/2564806378 reap_dead start 2024-09-19T04:20:36.194 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0d9e5640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaf40039c0 0x7fbaf4003de0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fbb04001780 tx=0x7fbb0401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14556ef51879c854 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:36.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040013a0 con 0x7fbaf40039c0 2024-09-19T04:20:36.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb040052f0 con 0x7fbaf40039c0 2024-09-19T04:20:36.195 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0c9e3640 1 -- 172.21.15.76:0/2564806378 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04003840 con 0x7fbaf40039c0 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2564806378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaf40039c0 msgr2=0x7fbaf4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaf40039c0 0x7fbaf4003de0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fbb04001780 tx=0x7fbb0401c3d0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2564806378 shutdown_connections 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2564806378 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbaf40039c0 0x7fbaf4003de0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2564806378 >> 172.21.15.76:0/2564806378 conn(0x7fbb080b1f90 msgr2=0x7fbb080b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2564806378 shutdown_connections 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.191+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2564806378 wait complete. 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 Processor -- start 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 -- start start 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40039c0 0x7fbb081511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb08150910 0x7fbb080b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb08071f10 0x7fbb080723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fbb080ad610 con 0x7fbb08071f10 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fbb080afc90 con 0x7fbb08150910 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fbb080ad790 con 0x7fbaf40039c0 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb08071f10 0x7fbb080723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb08071f10 0x7fbb080723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52364/0 (socket says 172.21.15.76:52364) 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 -- 172.21.15.76:0/2229284888 learned_addr learned my addr 172.21.15.76:0/2229284888 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0d9e5640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40039c0 0x7fbb081511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0d1e4640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb08150910 0x7fbb080b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 -- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40039c0 msgr2=0x7fbb081511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40039c0 0x7fbb081511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 -- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb08150910 msgr2=0x7fbb080b1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb08150910 0x7fbb080b1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.199 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 -- 172.21.15.76:0/2229284888 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb08151970 con 0x7fbb08071f10 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0e1e6640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb08071f10 0x7fbb080723a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fbb0806f460 tx=0x7fbb0000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbafe7fc640 1 -- 172.21.15.76:0/2229284888 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00016620 con 0x7fbb08071f10 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbafe7fc640 1 -- 172.21.15.76:0/2229284888 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb0001b070 con 0x7fbb08071f10 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0814fb70 con 0x7fbb08071f10 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbafe7fc640 1 -- 172.21.15.76:0/2229284888 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb000169c0 con 0x7fbb08071f10 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb08150130 con 0x7fbb08071f10 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb080af4a0 con 0x7fbb08071f10 2024-09-19T04:20:36.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbafe7fc640 1 -- 172.21.15.76:0/2229284888 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fbb00016b60 con 0x7fbb08071f10 2024-09-19T04:20:36.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbafe7fc640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbaec0678c0 0x7fbaec069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:36.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbafe7fc640 1 -- 172.21.15.76:0/2229284888 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 6417+0+0 (secure 0 0 0) 0x7fbb00086c80 con 0x7fbb08071f10 2024-09-19T04:20:36.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.195+0000 7fbb0d9e5640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbaec0678c0 0x7fbaec069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:36.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.199+0000 7fbb0d9e5640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbaec0678c0 0x7fbaec069d80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbb04001750 tx=0x7fbb040011a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:36.203 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.203+0000 7fbafe7fc640 1 -- 172.21.15.76:0/2229284888 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fbb00057580 con 0x7fbb08071f10 2024-09-19T04:20:36.356 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.355+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbb08075350 con 0x7fbb08071f10 2024-09-19T04:20:36.995 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbafe7fc640 1 -- 172.21.15.76:0/2229284888 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) ==== 181+0+0 (secure 0 0 0) 0x7fbb00009b10 con 0x7fbb08071f10 2024-09-19T04:20:36.995 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_metadata' removed 2024-09-19T04:20:36.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbaec0678c0 msgr2=0x7fbaec069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:36.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbaec0678c0 0x7fbaec069d80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fbb04001750 tx=0x7fbb040011a0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.998 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb08071f10 msgr2=0x7fbb080723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb08071f10 0x7fbb080723a0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7fbb0806f460 tx=0x7fbb0000f520 comp rx=0 tx=0).stop 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 shutdown_connections 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fbaec0678c0 0x7fbaec069d80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fbb08071f10 0x7fbb080723a0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb08150910 0x7fbb080b1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 --2- 172.21.15.76:0/2229284888 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fbaf40039c0 0x7fbb081511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.995+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 >> 172.21.15.76:0/2229284888 conn(0x7fbb080b1f90 msgr2=0x7fbb0815a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.999+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 shutdown_connections 2024-09-19T04:20:36.999 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:36.999+0000 7fbb0fc49640 1 -- 172.21.15.76:0/2229284888 wait complete. 2024-09-19T04:20:37.024 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T04:20:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feed5e77640 1 Processor -- start 2024-09-19T04:20:37.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feed5e77640 1 -- start start 2024-09-19T04:20:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feed5e77640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed0161a60 0x7feed00d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feed5e77640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feed00d2e90 0x7feed00cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feed5e77640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed00d32b0 con 0x7feed00cf450 2024-09-19T04:20:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feed5e77640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7feed00d0300 con 0x7feed00d2e90 2024-09-19T04:20:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feed5e77640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7feed006e6c0 con 0x7feed0161a60 2024-09-19T04:20:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feeceffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feed00d2e90 0x7feed00cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:37.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feeceffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7feed00d2e90 0x7feed00cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52372/0 (socket says 172.21.15.76:52372) 2024-09-19T04:20:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feeceffd640 1 -- 172.21.15.76:0/1793446928 learned_addr learned my addr 172.21.15.76:0/1793446928 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feecf7fe640 1 --2- 172.21.15.76:0/1793446928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed0161a60 0x7feed00d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feeceffd640 1 -- 172.21.15.76:0/1793446928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed0161a60 msgr2=0x7feed00d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feeceffd640 1 --2- 172.21.15.76:0/1793446928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed0161a60 0x7feed00d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feeceffd640 1 -- 172.21.15.76:0/1793446928 >> v1:172.21.15.76:6789/0 conn(0x7feed00cf450 legacy=0x7feed00d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feece7fc640 1 -- 172.21.15.76:0/1793446928 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3259321612 0 0) 0x7feed00d32b0 con 0x7feed00cf450 2024-09-19T04:20:37.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.151+0000 7feeceffd640 1 -- 172.21.15.76:0/1793446928 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed00b0f60 con 0x7feed00d2e90 2024-09-19T04:20:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feeceffd640 1 --2- 172.21.15.76:0/1793446928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed00d2e90 0x7feed00cef10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e4c99f2550fcec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feece7fc640 1 -- 172.21.15.76:0/1793446928 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed00d1930 con 0x7feed00d2e90 2024-09-19T04:20:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feece7fc640 1 -- 172.21.15.76:0/1793446928 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec40af070 con 0x7feed00d2e90 2024-09-19T04:20:37.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feece7fc640 1 -- 172.21.15.76:0/1793446928 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec40aa820 con 0x7feed00d2e90 2024-09-19T04:20:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 -- 172.21.15.76:0/1793446928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed00d2e90 msgr2=0x7feed00cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 --2- 172.21.15.76:0/1793446928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed00d2e90 0x7feed00cef10 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec401c4a0 comp rx=0 tx=0).stop 2024-09-19T04:20:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 -- 172.21.15.76:0/1793446928 shutdown_connections 2024-09-19T04:20:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 --2- 172.21.15.76:0/1793446928 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed00d2e90 0x7feed00cef10 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 --2- 172.21.15.76:0/1793446928 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed0161a60 0x7feed00d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 -- 172.21.15.76:0/1793446928 >> 172.21.15.76:0/1793446928 conn(0x7feed008bf90 msgr2=0x7feed009d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:37.157 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 -- 172.21.15.76:0/1793446928 shutdown_connections 2024-09-19T04:20:37.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 -- 172.21.15.76:0/1793446928 wait complete. 2024-09-19T04:20:37.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 Processor -- start 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.155+0000 7feed5e77640 1 -- start start 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed00cf450 0x7feed015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed00d2e90 0x7feed015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed0161a60 0x7feed015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7feed00cf8c0 con 0x7feed0161a60 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7feed006d190 con 0x7feed00cf450 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7feed00cfa40 con 0x7feed00d2e90 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed0161a60 0x7feed015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:37.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed0161a60 0x7feed015b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52374/0 (socket says 172.21.15.76:52374) 2024-09-19T04:20:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 -- 172.21.15.76:0/1153311959 learned_addr learned my addr 172.21.15.76:0/1153311959 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feeceffd640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed00d2e90 0x7feed015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 -- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed00d2e90 msgr2=0x7feed015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed00d2e90 0x7feed015ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecf7fe640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed00cf450 0x7feed015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 -- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed00cf450 msgr2=0x7feed015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:37.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed00cf450 0x7feed015f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 -- 172.21.15.76:0/1153311959 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed015de30 con 0x7feed0161a60 2024-09-19T04:20:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecffff640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed0161a60 0x7feed015b0a0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7feec800c780 tx=0x7feec800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feeb7fff640 1 -- 172.21.15.76:0/1153311959 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8003db0 con 0x7feed0161a60 2024-09-19T04:20:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feeb7fff640 1 -- 172.21.15.76:0/1153311959 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec8005070 con 0x7feed0161a60 2024-09-19T04:20:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed015e110 con 0x7feed0161a60 2024-09-19T04:20:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feeb7fff640 1 -- 172.21.15.76:0/1153311959 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8005460 con 0x7feed0161a60 2024-09-19T04:20:37.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed015e6d0 con 0x7feed0161a60 2024-09-19T04:20:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed015eab0 con 0x7feed0161a60 2024-09-19T04:20:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feeb7fff640 1 -- 172.21.15.76:0/1153311959 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7feec8005600 con 0x7feed0161a60 2024-09-19T04:20:37.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feeb7fff640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feebc0678c0 0x7feebc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:37.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feeb7fff640 1 -- 172.21.15.76:0/1153311959 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5979+0+0 (secure 0 0 0) 0x7feec8085d90 con 0x7feed0161a60 2024-09-19T04:20:37.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecf7fe640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feebc0678c0 0x7feebc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:37.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.159+0000 7feecf7fe640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feebc0678c0 0x7feebc069d80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feec00096f0 tx=0x7feec0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:37.166 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.163+0000 7feeb7fff640 1 -- 172.21.15.76:0/1153311959 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7feed015eab0 con 0x7feed0161a60 2024-09-19T04:20:37.335 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.335+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7feed00cc2a0 con 0x7feed0161a60 2024-09-19T04:20:37.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.995+0000 7feeb7fff640 1 -- 172.21.15.76:0/1153311959 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) ==== 169+0+0 (secure 0 0 0) 0x7feec80567c0 con 0x7feed0161a60 2024-09-19T04:20:37.998 INFO:teuthology.orchestra.run.smithi076.stderr:pool 'cephfs_data' removed 2024-09-19T04:20:38.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feebc0678c0 msgr2=0x7feebc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feebc0678c0 0x7feebc069d80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7feec00096f0 tx=0x7feec0008040 comp rx=0 tx=0).stop 2024-09-19T04:20:38.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed0161a60 msgr2=0x7feed015b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.000 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed0161a60 0x7feed015b0a0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7feec800c780 tx=0x7feec800cc40 comp rx=0 tx=0).stop 2024-09-19T04:20:38.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 shutdown_connections 2024-09-19T04:20:38.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7feebc0678c0 0x7feebc069d80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7feed0161a60 0x7feed015b0a0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.002 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7feed00d2e90 0x7feed015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 --2- 172.21.15.76:0/1153311959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed00cf450 0x7feed015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:37.999+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 >> 172.21.15.76:0/1153311959 conn(0x7feed008bf90 msgr2=0x7feed00b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.003+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 shutdown_connections 2024-09-19T04:20:38.003 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.003+0000 7feed5e77640 1 -- 172.21.15.76:0/1153311959 wait complete. 2024-09-19T04:20:38.028 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-09-19T04:20:38.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.151+0000 7f5baea69640 1 Processor -- start 2024-09-19T04:20:38.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.151+0000 7f5baea69640 1 -- start start 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.151+0000 7f5baea69640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba80b39a0 0x7f5ba806df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.151+0000 7f5baea69640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ba806e4b0 0x7f5ba8072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.151+0000 7f5baea69640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ba806ec60 con 0x7f5ba81587a0 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.151+0000 7f5baea69640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f5ba806f870 con 0x7f5ba806e4b0 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.151+0000 7f5baea69640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba806ef40 con 0x7f5ba80b39a0 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ba806e4b0 0x7f5ba8072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f5ba806e4b0 0x7f5ba8072160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52388/0 (socket says 172.21.15.76:52388) 2024-09-19T04:20:38.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 -- 172.21.15.76:0/1279748529 learned_addr learned my addr 172.21.15.76:0/1279748529 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5ba7fff640 1 --2- 172.21.15.76:0/1279748529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba80b39a0 0x7f5ba806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 -- 172.21.15.76:0/1279748529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba80b39a0 msgr2=0x7f5ba806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 --2- 172.21.15.76:0/1279748529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba80b39a0 0x7f5ba806df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 -- 172.21.15.76:0/1279748529 >> v1:172.21.15.76:6789/0 conn(0x7f5ba81587a0 legacy=0x7f5ba80b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5ba77fe640 1 -- 172.21.15.76:0/1279748529 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4240443750 0 0) 0x7f5ba806ec60 con 0x7f5ba81587a0 2024-09-19T04:20:38.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 -- 172.21.15.76:0/1279748529 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba806f220 con 0x7f5ba806e4b0 2024-09-19T04:20:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5bad006640 1 --2- 172.21.15.76:0/1279748529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba806e4b0 0x7f5ba8072160 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba001cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c58aa4230fba27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5ba77fe640 1 -- 172.21.15.76:0/1279748529 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00a9610 con 0x7f5ba806e4b0 2024-09-19T04:20:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5ba77fe640 1 -- 172.21.15.76:0/1279748529 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba00a97b0 con 0x7f5ba806e4b0 2024-09-19T04:20:38.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.155+0000 7f5ba77fe640 1 -- 172.21.15.76:0/1279748529 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00a9b80 con 0x7f5ba806e4b0 2024-09-19T04:20:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- 172.21.15.76:0/1279748529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba806e4b0 msgr2=0x7f5ba8072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 --2- 172.21.15.76:0/1279748529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba806e4b0 0x7f5ba8072160 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba001cf90 comp rx=0 tx=0).stop 2024-09-19T04:20:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- 172.21.15.76:0/1279748529 shutdown_connections 2024-09-19T04:20:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 --2- 172.21.15.76:0/1279748529 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba806e4b0 0x7f5ba8072160 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 --2- 172.21.15.76:0/1279748529 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba80b39a0 0x7f5ba806df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- 172.21.15.76:0/1279748529 >> 172.21.15.76:0/1279748529 conn(0x7f5ba80b1f90 msgr2=0x7f5ba80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- 172.21.15.76:0/1279748529 shutdown_connections 2024-09-19T04:20:38.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- 172.21.15.76:0/1279748529 wait complete. 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 Processor -- start 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- start start 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba806e4b0 0x7f5ba8146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba80b39a0 0x7f5ba814d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81587a0 0x7f5ba814fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8073030 con 0x7f5ba81587a0 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba8072eb0 con 0x7f5ba80b39a0 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba80731b0 con 0x7f5ba806e4b0 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81587a0 0x7f5ba814fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81587a0 0x7f5ba814fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52402/0 (socket says 172.21.15.76:52402) 2024-09-19T04:20:38.162 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 -- 172.21.15.76:0/535979697 learned_addr learned my addr 172.21.15.76:0/535979697 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5ba7fff640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba80b39a0 0x7f5ba814d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bac805640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba806e4b0 0x7f5ba8146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 -- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba806e4b0 msgr2=0x7f5ba8146920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba806e4b0 0x7f5ba8146920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 -- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba80b39a0 msgr2=0x7f5ba814d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba80b39a0 0x7f5ba814d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 -- 172.21.15.76:0/535979697 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba814e930 con 0x7f5ba81587a0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5bad006640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81587a0 0x7f5ba814fda0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5ba001a160 tx=0x7f5ba001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5ba57fa640 1 -- 172.21.15.76:0/535979697 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba00c0070 con 0x7f5ba81587a0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5ba57fa640 1 -- 172.21.15.76:0/535979697 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ba00a9490 con 0x7f5ba81587a0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ba814e090 con 0x7f5ba81587a0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5ba57fa640 1 -- 172.21.15.76:0/535979697 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0004230 con 0x7f5ba81587a0 2024-09-19T04:20:38.163 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.159+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ba8150e80 con 0x7f5ba81587a0 2024-09-19T04:20:38.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.163+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ba806e000 con 0x7f5ba81587a0 2024-09-19T04:20:38.164 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.163+0000 7f5ba57fa640 1 -- 172.21.15.76:0/535979697 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f5ba00c3070 con 0x7f5ba81587a0 2024-09-19T04:20:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.163+0000 7f5ba57fa640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5b880678c0 0x7f5b88069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.163+0000 7f5bac805640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5b880678c0 0x7f5b88069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.163+0000 7f5ba57fa640 1 -- 172.21.15.76:0/535979697 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f5ba011ff20 con 0x7f5ba81587a0 2024-09-19T04:20:38.165 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.163+0000 7f5bac805640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5b880678c0 0x7f5b88069d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000f70 tx=0x7f5b98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:38.168 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.167+0000 7f5ba57fa640 1 -- 172.21.15.76:0/535979697 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f5ba00ad020 con 0x7f5ba81587a0 2024-09-19T04:20:38.336 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.335+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5ba8070080 con 0x7f5ba81587a0 2024-09-19T04:20:38.341 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.339+0000 7f5ba57fa640 1 -- 172.21.15.76:0/535979697 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7f5ba0016150 con 0x7f5ba81587a0 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5b880678c0 msgr2=0x7f5b88069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5b880678c0 0x7f5b88069d80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000f70 tx=0x7f5b98009040 comp rx=0 tx=0).stop 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81587a0 msgr2=0x7f5ba814fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81587a0 0x7f5ba814fda0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f5ba001a160 tx=0x7f5ba001c530 comp rx=0 tx=0).stop 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 shutdown_connections 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f5b880678c0 0x7f5b88069d80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f5ba81587a0 0x7f5ba814fda0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.344 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ba80b39a0 0x7f5ba814d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 --2- 172.21.15.76:0/535979697 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f5ba806e4b0 0x7f5ba8146920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 >> 172.21.15.76:0/535979697 conn(0x7f5ba80b1f90 msgr2=0x7f5ba8157750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 shutdown_connections 2024-09-19T04:20:38.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.343+0000 7f5baea69640 1 -- 172.21.15.76:0/535979697 wait complete. 2024-09-19T04:20:38.361 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0' 2024-09-19T04:20:38.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.479+0000 7f94ad76e640 1 Processor -- start 2024-09-19T04:20:38.482 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94ad76e640 1 -- start start 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94ad76e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94a815c2a0 0x7f94a80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94ad76e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a80af2a0 0x7f94a806e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94ad76e640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a80b3290 con 0x7f94a815bec0 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94ad76e640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f94a80b3570 con 0x7f94a815c2a0 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94ad76e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f94a80b3850 con 0x7f94a80af2a0 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94a815c2a0 0x7f94a80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f94a815c2a0 0x7f94a80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52406/0 (socket says 172.21.15.76:52406) 2024-09-19T04:20:38.484 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 -- 172.21.15.76:0/2871588600 learned_addr learned my addr 172.21.15.76:0/2871588600 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a6ffd640 1 --1- 172.21.15.76:0/2871588600 >> v1:172.21.15.76:6789/0 conn(0x7f94a815bec0 0x7f94a80ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47944/0 (socket says 172.21.15.76:47944) 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a77fe640 1 --2- 172.21.15.76:0/2871588600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a80af2a0 0x7f94a806e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 -- 172.21.15.76:0/2871588600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a80af2a0 msgr2=0x7f94a806e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 --2- 172.21.15.76:0/2871588600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a80af2a0 0x7f94a806e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 -- 172.21.15.76:0/2871588600 >> v1:172.21.15.76:6789/0 conn(0x7f94a815bec0 legacy=0x7f94a80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a5ffb640 1 -- 172.21.15.76:0/2871588600 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 874933436 0 0) 0x7f94a80b3290 con 0x7f94a815bec0 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 -- 172.21.15.76:0/2871588600 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a80b3b30 con 0x7f94a815c2a0 2024-09-19T04:20:38.485 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a77fe640 1 --2- 172.21.15.76:0/2871588600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a80af2a0 0x7f94a806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:38.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a67fc640 1 --2- 172.21.15.76:0/2871588600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a80aed60 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9498009870 tx=0x7f949801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58c23fcd0eff79c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:38.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a5ffb640 1 -- 172.21.15.76:0/2871588600 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94980a95f0 con 0x7f94a815c2a0 2024-09-19T04:20:38.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a5ffb640 1 -- 172.21.15.76:0/2871588600 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94980a9790 con 0x7f94a815c2a0 2024-09-19T04:20:38.486 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.483+0000 7f94a5ffb640 1 -- 172.21.15.76:0/2871588600 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94980a9b60 con 0x7f94a815c2a0 2024-09-19T04:20:38.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- 172.21.15.76:0/2871588600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 msgr2=0x7f94a80aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2871588600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a80aed60 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f9498009870 tx=0x7f949801c530 comp rx=0 tx=0).stop 2024-09-19T04:20:38.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- 172.21.15.76:0/2871588600 shutdown_connections 2024-09-19T04:20:38.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2871588600 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a80af2a0 0x7f94a806e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2871588600 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a80aed60 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.487 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- 172.21.15.76:0/2871588600 >> 172.21.15.76:0/2871588600 conn(0x7f94a80b1f90 msgr2=0x7f94a80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:38.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- 172.21.15.76:0/2871588600 shutdown_connections 2024-09-19T04:20:38.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- 172.21.15.76:0/2871588600 wait complete. 2024-09-19T04:20:38.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 Processor -- start 2024-09-19T04:20:38.488 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- start start 2024-09-19T04:20:38.490 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a80af2a0 0x7f94a8149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a815bec0 0x7f94a814a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a814a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f94a806ed50 con 0x7f94a815c2a0 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f94a80af6b0 con 0x7f94a80af2a0 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f94a806eed0 con 0x7f94a815bec0 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a814a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a814a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52412/0 (socket says 172.21.15.76:52412) 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 -- 172.21.15.76:0/2105617508 learned_addr learned my addr 172.21.15.76:0/2105617508 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a67fc640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a815bec0 0x7f94a814a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a6ffd640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a80af2a0 0x7f94a8149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 -- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a815bec0 msgr2=0x7f94a814a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.491 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a815bec0 0x7f94a814a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 -- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a80af2a0 msgr2=0x7f94a8149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a80af2a0 0x7f94a8149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 -- 172.21.15.76:0/2105617508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a815c6b0 con 0x7f94a815c2a0 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a6ffd640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a80af2a0 0x7f94a8149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94a77fe640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a814a860 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f94a000cef0 tx=0x7f94a000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f949f7fe640 1 -- 172.21.15.76:0/2105617508 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a001a070 con 0x7f94a815c2a0 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f949f7fe640 1 -- 172.21.15.76:0/2105617508 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94a000d860 con 0x7f94a815c2a0 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a81523c0 con 0x7f94a815c2a0 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f949f7fe640 1 -- 172.21.15.76:0/2105617508 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a000dc50 con 0x7f94a815c2a0 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.487+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a815cc30 con 0x7f94a815c2a0 2024-09-19T04:20:38.492 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.491+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a806e000 con 0x7f94a815c2a0 2024-09-19T04:20:38.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.491+0000 7f949f7fe640 1 -- 172.21.15.76:0/2105617508 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f94a001d070 con 0x7f94a815c2a0 2024-09-19T04:20:38.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.491+0000 7f949f7fe640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f94880678c0 0x7f9488069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:38.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.491+0000 7f949f7fe640 1 -- 172.21.15.76:0/2105617508 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f94a0086490 con 0x7f94a815c2a0 2024-09-19T04:20:38.493 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.491+0000 7f94a6ffd640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f94880678c0 0x7f9488069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:38.494 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.491+0000 7f94a6ffd640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f94880678c0 0x7f9488069d80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9490000f70 tx=0x7f9490009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:38.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.495+0000 7f949f7fe640 1 -- 172.21.15.76:0/2105617508 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f94a0057150 con 0x7f94a815c2a0 2024-09-19T04:20:38.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:38.715+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]} v 0) -- 0x7f94a80cf760 con 0x7f94a815c2a0 2024-09-19T04:20:39.017 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.015+0000 7f949f7fe640 1 -- 172.21.15.76:0/2105617508 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_multimds_misc.TestScrub2.test_scrub_pause_and_resume_with_abort_mds0"]}]=0 v371) ==== 165+0+0 (secure 0 0 0) 0x7f94a0056a50 con 0x7f94a815c2a0 2024-09-19T04:20:39.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f94880678c0 msgr2=0x7f9488069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f94880678c0 0x7f9488069d80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9490000f70 tx=0x7f9490009040 comp rx=0 tx=0).stop 2024-09-19T04:20:39.020 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 msgr2=0x7f94a814a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a814a860 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f94a000cef0 tx=0x7f94a000a710 comp rx=0 tx=0).stop 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 shutdown_connections 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f94880678c0 0x7f9488069d80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f94a815c2a0 0x7f94a814a860 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f94a815bec0 0x7f94a814a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 --2- 172.21.15.76:0/2105617508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a80af2a0 0x7f94a8149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 >> 172.21.15.76:0/2105617508 conn(0x7f94a80b1f90 msgr2=0x7f94a80b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 shutdown_connections 2024-09-19T04:20:39.021 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.019+0000 7f94ad76e640 1 -- 172.21.15.76:0/2105617508 wait complete. 2024-09-19T04:20:39.040 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort_mds0 (tasks.cephfs.test_multimds_misc.TestScrub2) ... ok 2024-09-19T04:20:39.041 INFO:tasks.cephfs_test_runner: 2024-09-19T04:20:39.041 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-19T04:20:39.041 INFO:tasks.cephfs_test_runner:Ran 6 tests in 401.031s 2024-09-19T04:20:39.041 INFO:tasks.cephfs_test_runner: 2024-09-19T04:20:39.041 INFO:tasks.cephfs_test_runner:OK 2024-09-19T04:20:39.041 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-19T04:20:39.051 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-19T04:20:39.069 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-19T04:20:39.069 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:20:39.069 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:39.076 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:20:39.076 DEBUG:teuthology.orchestra.run.smithi076:> set -ex 2024-09-19T04:20:39.076 DEBUG:teuthology.orchestra.run.smithi076:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:39.123 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336480k,nr_inodes=4084120,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=23106 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-19T04:20:39.123 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:20:39.123 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:39.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:20:39.130 DEBUG:teuthology.orchestra.run.smithi116:> set -ex 2024-09-19T04:20:39.130 DEBUG:teuthology.orchestra.run.smithi116:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T04:20:39.178 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336472k,nr_inodes=4084118,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16168 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274680k,nr_inodes=818670,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274684k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-19T04:20:39.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.179 DEBUG:teuthology.orchestra.run.smithi116:> ip netns list 2024-09-19T04:20:39.226 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-19T04:20:39.226 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-19T04:20:39.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.226 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-19T04:20:39.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.283 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-19T04:20:39.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.338 DEBUG:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2024-09-19T04:20:39.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.491 DEBUG:teuthology.orchestra.run.smithi076:> ip netns list 2024-09-19T04:20:39.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-19T04:20:39.498 INFO:teuthology.orchestra.run.smithi076.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T04:20:39.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.498 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-19T04:20:39.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.556 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T04:20:39.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T04:20:39.623 DEBUG:teuthology.orchestra.run.smithi076:> sudo ip link delete ceph-brx 2024-09-19T04:20:39.764 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-19T04:20:39.775 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-19T04:20:39.775 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:20:39.901 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.899+0000 7f2ace4b3640 1 Processor -- start 2024-09-19T04:20:39.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.899+0000 7f2ace4b3640 1 -- start start 2024-09-19T04:20:39.902 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ace4b3640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2ac806f010 0x7f2ac8166ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ace4b3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac80b3290 con 0x7f2ac815c2a0 2024-09-19T04:20:39.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ace4b3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ac80b3570 con 0x7f2ac815bec0 2024-09-19T04:20:39.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ace4b3640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f2ac80b3850 con 0x7f2ac806f010 2024-09-19T04:20:39.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f2ac815c2a0 0x7f2ac80aed60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47954/0 (socket says 172.21.15.76:47954) 2024-09-19T04:20:39.903 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 -- 172.21.15.76:0/2384992357 learned_addr learned my addr 172.21.15.76:0/2384992357 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1883059064 0 0) 0x7f2ac80b3290 con 0x7f2ac815c2a0 2024-09-19T04:20:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2acca50640 1 --2- 172.21.15.76:0/2384992357 >> v2:172.21.15.116:3300/0 conn(0x7f2ac806f010 0x7f2ac8166ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ab0003530 con 0x7f2ac815c2a0 2024-09-19T04:20:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2253609788 0 0) 0x7f2ab0003530 con 0x7f2ac815c2a0 2024-09-19T04:20:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 >> v2:172.21.15.116:3300/0 conn(0x7f2ac806f010 msgr2=0x7f2ac8166ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 --2- 172.21.15.76:0/2384992357 >> v2:172.21.15.116:3300/0 conn(0x7f2ac806f010 0x7f2ac8166ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.905 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 >> v1:172.21.15.116:6789/0 conn(0x7f2ac815bec0 legacy=0x7f2ac80ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac80b3b30 con 0x7f2ac815c2a0 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2abc001680 con 0x7f2ac815c2a0 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 >> v1:172.21.15.76:6789/0 conn(0x7f2ac815c2a0 legacy=0x7f2ac80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab00039c0 0x7f2ab0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ab0004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab00051d0 0x7f2ab000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac80b3850 con 0x7f2ab0004810 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab000c070 con 0x7f2ab00051d0 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab000c270 con 0x7f2ab00039c0 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ab0004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2acca50640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab00051d0 0x7f2ab000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.906 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac7fff640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab00039c0 0x7f2ab0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 -- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab00039c0 msgr2=0x7f2ab0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ab00039c0 0x7f2ab0003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 -- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab00051d0 msgr2=0x7f2ab000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab00051d0 0x7f2ab000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 -- 172.21.15.76:0/2384992357 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab0003530 con 0x7f2ab0004810 2024-09-19T04:20:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2acca50640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab00051d0 0x7f2ab000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:39.907 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2acca50640 1 -- 172.21.15.76:0/2384992357 reap_dead start 2024-09-19T04:20:39.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac77fe640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ab0004c90 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00c190 tx=0x7f2abc01f310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc66164532eacf08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.908 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc007d00 con 0x7f2ab0004810 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2abc005b00 con 0x7f2ab0004810 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.903+0000 7f2ac6ffd640 1 -- 172.21.15.76:0/2384992357 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2abc007070 con 0x7f2ab0004810 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2384992357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 msgr2=0x7f2ab0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ab0004c90 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f2abc00c190 tx=0x7f2abc01f310 comp rx=0 tx=0).stop 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2384992357 shutdown_connections 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2384992357 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ab0004c90 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2384992357 >> 172.21.15.76:0/2384992357 conn(0x7f2ac80b1f90 msgr2=0x7f2ac80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2384992357 shutdown_connections 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2384992357 wait complete. 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 Processor -- start 2024-09-19T04:20:39.909 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- start start 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ac8152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ac8150360 0x7f2ac814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac8152d80 0x7f2ac8153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac80afc90 con 0x7f2ab0004810 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac80b40f0 con 0x7f2ac8152d80 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac80ad200 con 0x7f2ac8150360 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ac8152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ac8152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52434/0 (socket says 172.21.15.76:52434) 2024-09-19T04:20:39.912 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 -- 172.21.15.76:0/2216242141 learned_addr learned my addr 172.21.15.76:0/2216242141 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 -- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ac8150360 msgr2=0x7f2ac814d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac77fe640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ac8150360 0x7f2ac814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2acca50640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac8152d80 0x7f2ac8153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ac8150360 0x7f2ac814d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 -- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac8152d80 msgr2=0x7f2ac8153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac8152d80 0x7f2ac8153230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 -- 172.21.15.76:0/2216242141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac8155560 con 0x7f2ab0004810 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac7fff640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ac8152550 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800d4f0 tx=0x7f2ab800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac4ff9640 1 -- 172.21.15.76:0/2216242141 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8005480 con 0x7f2ab0004810 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac4ff9640 1 -- 172.21.15.76:0/2216242141 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ab8017070 con 0x7f2ab0004810 2024-09-19T04:20:39.913 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac815c830 con 0x7f2ab0004810 2024-09-19T04:20:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ac4ff9640 1 -- 172.21.15.76:0/2216242141 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ab8003820 con 0x7f2ab0004810 2024-09-19T04:20:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.907+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac8155840 con 0x7f2ab0004810 2024-09-19T04:20:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.911+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac80ad530 con 0x7f2ab0004810 2024-09-19T04:20:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.911+0000 7f2ac4ff9640 1 -- 172.21.15.76:0/2216242141 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f2ab80039c0 con 0x7f2ab0004810 2024-09-19T04:20:39.914 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.911+0000 7f2ac4ff9640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ab40678c0 0x7f2ab4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.911+0000 7f2ac77fe640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ab40678c0 0x7f2ab4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.911+0000 7f2ac4ff9640 1 -- 172.21.15.76:0/2216242141 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f2ab8085990 con 0x7f2ab0004810 2024-09-19T04:20:39.915 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.911+0000 7f2ac77fe640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ab40678c0 0x7f2ab4069d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ae000 tx=0x7f2abc007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:39.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:39.915+0000 7f2ac4ff9640 1 -- 172.21.15.76:0/2216242141 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2ab8056590 con 0x7f2ab0004810 2024-09-19T04:20:40.078 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.075+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2ac8153c60 con 0x7f2ab40678c0 2024-09-19T04:20:40.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ac4ff9640 1 -- 172.21.15.76:0/2216242141 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7f2ac8153c60 con 0x7f2ab40678c0 2024-09-19T04:20:40.079 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:40.079 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:20:40.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ab40678c0 msgr2=0x7f2ab4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ab40678c0 0x7f2ab4069d80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2abc0ae000 tx=0x7f2abc007040 comp rx=0 tx=0).stop 2024-09-19T04:20:40.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 msgr2=0x7f2ac8152550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ac8152550 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f2ab800d4f0 tx=0x7f2ab800d9b0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 shutdown_connections 2024-09-19T04:20:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f2ab40678c0 0x7f2ab4069d80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac8152d80 0x7f2ac8153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f2ac8150360 0x7f2ac814d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 --2- 172.21.15.76:0/2216242141 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2ab0004810 0x7f2ac8152550 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 >> 172.21.15.76:0/2216242141 conn(0x7f2ac80b1f90 msgr2=0x7f2ac80b2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 shutdown_connections 2024-09-19T04:20:40.082 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.079+0000 7f2ace4b3640 1 -- 172.21.15.76:0/2216242141 wait complete. 2024-09-19T04:20:40.098 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-19T04:20:39.876912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261236,"kb_used_data":11724,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713484,"statfs":{"total":773094113280,"available":772826607616,"internally_reserved":0,"allocated":12005376,"data_stored":6284846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-115824,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-248,"num_write_kb":-199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002197"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":331,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-19T04:20:37.254539+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:20:37.254539+0000","last_peered":"2024-09-19T04:20:37.254539+0000","last_clean":"2024-09-19T04:20:37.254539+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:20:37.254539+0000","last_undegraded":"2024-09-19T04:20:37.254539+0000","last_fullsized":"2024-09-19T04:20:37.254539+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542236,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":1233336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542236,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1288790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:20:40.098 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:20:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a665bd640 1 Processor -- start 2024-09-19T04:20:40.215 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a665bd640 1 -- start start 2024-09-19T04:20:40.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a665bd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 0x7f7a600d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a665bd640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a600d27a0 con 0x7f7a600cf800 2024-09-19T04:20:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a665bd640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a600d2a80 con 0x7f7a600b4510 2024-09-19T04:20:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a665bd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7a600d2d60 con 0x7f7a601457a0 2024-09-19T04:20:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5f7fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f7a600cf800 0x7f7a600b3e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47958/0 (socket says 172.21.15.76:47958) 2024-09-19T04:20:40.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5f7fe640 1 -- 172.21.15.76:0/3703763471 learned_addr learned my addr 172.21.15.76:0/3703763471 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 0x7f7a600d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2723616729 0 0) 0x7f7a600d27a0 con 0x7f7a600cf800 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a48003530 con 0x7f7a600cf800 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3021068761 0 0) 0x7f7a48003530 con 0x7f7a600cf800 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 msgr2=0x7f7a600d1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 0x7f7a600d1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 >> v1:172.21.15.116:6789/0 conn(0x7f7a600b4510 legacy=0x7f7a600d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a600d01f0 con 0x7f7a600cf800 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f7a54001680 con 0x7f7a600cf800 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 >> v1:172.21.15.76:6789/0 conn(0x7f7a600cf800 legacy=0x7f7a600b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a480039f0 0x7f7a48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a48004810 0x7f7a48004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 0x7f7a48005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a600d2d60 con 0x7f7a480039f0 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a480039f0 0x7f7a48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4800bb00 con 0x7f7a48004810 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7a4800bd00 con 0x7f7a601457a0 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5f7fe640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a48004810 0x7f7a48004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a64b5a640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 0x7f7a48005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5ffff640 1 -- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 msgr2=0x7f7a48005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 0x7f7a48005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5ffff640 1 -- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a48004810 msgr2=0x7f7a48004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a48004810 0x7f7a48004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a5ffff640 1 -- 172.21.15.76:0/3703763471 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a48003530 con 0x7f7a480039f0 2024-09-19T04:20:40.220 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.215+0000 7f7a64b5a640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a601457a0 0x7f7a48005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a480039f0 0x7f7a48003e10 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009ae0 tx=0x7f7a5001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa54f659d6a358a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500b2070 con 0x7f7a480039f0 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a500a5560 con 0x7f7a480039f0 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a5effd640 1 -- 172.21.15.76:0/3703763471 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a500096b0 con 0x7f7a480039f0 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 -- 172.21.15.76:0/3703763471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a480039f0 msgr2=0x7f7a48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 --2- 172.21.15.76:0/3703763471 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a480039f0 0x7f7a48003e10 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009ae0 tx=0x7f7a5001ca20 comp rx=0 tx=0).stop 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a64b5a640 1 -- 172.21.15.76:0/3703763471 reap_dead start 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 -- 172.21.15.76:0/3703763471 shutdown_connections 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 -- 172.21.15.76:0/3703763471 >> 172.21.15.76:0/3703763471 conn(0x7f7a6006bcc0 msgr2=0x7f7a6006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:40.222 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 -- 172.21.15.76:0/3703763471 shutdown_connections 2024-09-19T04:20:40.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 -- 172.21.15.76:0/3703763471 wait complete. 2024-09-19T04:20:40.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 Processor -- start 2024-09-19T04:20:40.223 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.219+0000 7f7a665bd640 1 -- start start 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a480039f0 0x7f7a601358b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6006d7e0 0x7f7a60166930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6006dc60 0x7f7a60166e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f7a601680b0 con 0x7f7a6006dc60 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f7a600b8260 con 0x7f7a480039f0 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f7a60168230 con 0x7f7a6006d7e0 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6006dc60 0x7f7a60166e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6006dc60 0x7f7a60166e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52454/0 (socket says 172.21.15.76:52454) 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 -- 172.21.15.76:0/3280262076 learned_addr learned my addr 172.21.15.76:0/3280262076 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a480039f0 0x7f7a601358b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.225 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5f7fe640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6006d7e0 0x7f7a60166930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 -- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6006d7e0 msgr2=0x7f7a60166930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6006d7e0 0x7f7a60166930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 -- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a480039f0 msgr2=0x7f7a601358b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a480039f0 0x7f7a601358b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 -- 172.21.15.76:0/3280262076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6006e230 con 0x7f7a6006dc60 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a480039f0 0x7f7a601358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a64b5a640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6006dc60 0x7f7a60166e70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7a600cfca0 tx=0x7f7a5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5cff9640 1 -- 172.21.15.76:0/3280262076 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58006d60 con 0x7f7a6006dc60 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5cff9640 1 -- 172.21.15.76:0/3280262076 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a58003980 con 0x7f7a6006dc60 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6006e510 con 0x7f7a6006dc60 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5cff9640 1 -- 172.21.15.76:0/3280262076 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a58003d70 con 0x7f7a6006dc60 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a60164430 con 0x7f7a6006dc60 2024-09-19T04:20:40.226 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a60164630 con 0x7f7a6006dc60 2024-09-19T04:20:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5cff9640 1 -- 172.21.15.76:0/3280262076 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f7a5801b020 con 0x7f7a6006dc60 2024-09-19T04:20:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5cff9640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a4c0678c0 0x7f7a4c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5cff9640 1 -- 172.21.15.76:0/3280262076 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f7a58085900 con 0x7f7a6006dc60 2024-09-19T04:20:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a4c0678c0 0x7f7a4c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.227 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.223+0000 7f7a5ffff640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a4c0678c0 0x7f7a4c069d80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7a500af000 tx=0x7f7a500046d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.230 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.227+0000 7f7a5cff9640 1 -- 172.21.15.76:0/3280262076 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f7a60164630 con 0x7f7a6006dc60 2024-09-19T04:20:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7a6015c4d0 con 0x7f7a4c0678c0 2024-09-19T04:20:40.388 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a5cff9640 1 -- 172.21.15.76:0/3280262076 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7f7a500bf4e0 con 0x7f7a4c0678c0 2024-09-19T04:20:40.389 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:40.389 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:20:40.390 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a4c0678c0 msgr2=0x7f7a4c069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a4c0678c0 0x7f7a4c069d80 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7a500af000 tx=0x7f7a500046d0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6006dc60 msgr2=0x7f7a60166e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6006dc60 0x7f7a60166e70 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f7a600cfca0 tx=0x7f7a5800dc00 comp rx=0 tx=0).stop 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 shutdown_connections 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f7a4c0678c0 0x7f7a4c069d80 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f7a6006dc60 0x7f7a60166e70 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f7a6006d7e0 0x7f7a60166930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 --2- 172.21.15.76:0/3280262076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a480039f0 0x7f7a601358b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.387+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 >> 172.21.15.76:0/3280262076 conn(0x7f7a6006bcc0 msgr2=0x7f7a60144dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.391+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 shutdown_connections 2024-09-19T04:20:40.391 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.391+0000 7f7a665bd640 1 -- 172.21.15.76:0/3280262076 wait complete. 2024-09-19T04:20:40.407 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-19T04:20:39.876912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261236,"kb_used_data":11724,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713484,"statfs":{"total":773094113280,"available":772826607616,"internally_reserved":0,"allocated":12005376,"data_stored":6284846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-115824,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-248,"num_write_kb":-199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002197"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":331,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-19T04:20:37.254539+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:20:37.254539+0000","last_peered":"2024-09-19T04:20:37.254539+0000","last_clean":"2024-09-19T04:20:37.254539+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:20:37.254539+0000","last_undegraded":"2024-09-19T04:20:37.254539+0000","last_fullsized":"2024-09-19T04:20:37.254539+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542236,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":1233336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542236,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1288790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:20:40.408 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-19T04:20:40.408 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:20:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef7584640 1 Processor -- start 2024-09-19T04:20:40.518 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef7584640 1 -- start start 2024-09-19T04:20:40.519 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef7584640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015c2a0 0x7f9ef00aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef7584640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ef00af2a0 0x7f9ef006e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef7584640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ef00b3290 con 0x7f9ef015bec0 2024-09-19T04:20:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef7584640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f9ef00b3570 con 0x7f9ef00af2a0 2024-09-19T04:20:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef7584640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef00b3850 con 0x7f9ef015c2a0 2024-09-19T04:20:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef5320640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f9ef015bec0 0x7f9ef00ad200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47960/0 (socket says 172.21.15.76:47960) 2024-09-19T04:20:40.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef5320640 1 -- 172.21.15.76:0/2683952340 learned_addr learned my addr 172.21.15.76:0/2683952340 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef4b1f640 1 --2- 172.21.15.76:0/2683952340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015c2a0 0x7f9ef00aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef5b21640 1 --2- 172.21.15.76:0/2683952340 >> v2:172.21.15.116:3300/0 conn(0x7f9ef00af2a0 0x7f9ef006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ee7fff640 1 -- 172.21.15.76:0/2683952340 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2242156006 0 0) 0x7f9ef00b3290 con 0x7f9ef015bec0 2024-09-19T04:20:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ee7fff640 1 -- 172.21.15.76:0/2683952340 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9edc003530 con 0x7f9ef015bec0 2024-09-19T04:20:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef4b1f640 1 -- 172.21.15.76:0/2683952340 >> v2:172.21.15.116:3300/0 conn(0x7f9ef00af2a0 msgr2=0x7f9ef006e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:40.521 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef4b1f640 1 --2- 172.21.15.76:0/2683952340 >> v2:172.21.15.116:3300/0 conn(0x7f9ef00af2a0 0x7f9ef006e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef4b1f640 1 -- 172.21.15.76:0/2683952340 >> v1:172.21.15.76:6789/0 conn(0x7f9ef015bec0 legacy=0x7f9ef00ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ee7fff640 1 -- 172.21.15.76:0/2683952340 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1729413265 0 0) 0x7f9edc003530 con 0x7f9ef015bec0 2024-09-19T04:20:40.522 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.519+0000 7f9ef4b1f640 1 -- 172.21.15.76:0/2683952340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef00b3b30 con 0x7f9ef015c2a0 2024-09-19T04:20:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef4b1f640 1 --2- 172.21.15.76:0/2683952340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015c2a0 0x7f9ef00aed60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0097c0 tx=0x7f9eec01c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d8ac6280d84612a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ee7fff640 1 -- 172.21.15.76:0/2683952340 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec0b0070 con 0x7f9ef015c2a0 2024-09-19T04:20:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ee7fff640 1 -- 172.21.15.76:0/2683952340 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9eec0ab400 con 0x7f9ef015c2a0 2024-09-19T04:20:40.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ee7fff640 1 -- 172.21.15.76:0/2683952340 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0001680 con 0x7f9ef015c2a0 2024-09-19T04:20:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- 172.21.15.76:0/2683952340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015c2a0 msgr2=0x7f9ef00aed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 --2- 172.21.15.76:0/2683952340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015c2a0 0x7f9ef00aed60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0097c0 tx=0x7f9eec01c390 comp rx=0 tx=0).stop 2024-09-19T04:20:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- 172.21.15.76:0/2683952340 shutdown_connections 2024-09-19T04:20:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 --2- 172.21.15.76:0/2683952340 >> v2:172.21.15.116:3300/0 conn(0x7f9ef00af2a0 0x7f9ef006e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 --2- 172.21.15.76:0/2683952340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015c2a0 0x7f9ef00aed60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- 172.21.15.76:0/2683952340 >> 172.21.15.76:0/2683952340 conn(0x7f9ef00b1f90 msgr2=0x7f9ef00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:40.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- 172.21.15.76:0/2683952340 shutdown_connections 2024-09-19T04:20:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- 172.21.15.76:0/2683952340 wait complete. 2024-09-19T04:20:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 Processor -- start 2024-09-19T04:20:40.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- start start 2024-09-19T04:20:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ef00af2a0 0x7f9ef014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015bec0 0x7f9ef014c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ef015c2a0 0x7f9ef014c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef006ed50 con 0x7f9ef00af2a0 2024-09-19T04:20:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef00af6b0 con 0x7f9ef015c2a0 2024-09-19T04:20:40.527 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef7584640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f9ef006eed0 con 0x7f9ef015bec0 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ef00af2a0 0x7f9ef014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ef00af2a0 0x7f9ef014a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52466/0 (socket says 172.21.15.76:52466) 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 -- 172.21.15.76:0/237517550 learned_addr learned my addr 172.21.15.76:0/237517550 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef4b1f640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015bec0 0x7f9ef014c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5b21640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ef015c2a0 0x7f9ef014c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 -- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015bec0 msgr2=0x7f9ef014c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015bec0 0x7f9ef014c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 -- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ef015c2a0 msgr2=0x7f9ef014c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ef015c2a0 0x7f9ef014c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 -- 172.21.15.76:0/237517550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef01549c0 con 0x7f9ef00af2a0 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5b21640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ef015c2a0 0x7f9ef014c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.523+0000 7f9ef5320640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ef00af2a0 0x7f9ef014a740 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c750 tx=0x7f9ee000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ee5ffb640 1 -- 172.21.15.76:0/237517550 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee0005ae0 con 0x7f9ef00af2a0 2024-09-19T04:20:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ee5ffb640 1 -- 172.21.15.76:0/237517550 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ee0005ca0 con 0x7f9ef00af2a0 2024-09-19T04:20:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef0154ca0 con 0x7f9ef00af2a0 2024-09-19T04:20:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ee5ffb640 1 -- 172.21.15.76:0/237517550 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee00072b0 con 0x7f9ef00af2a0 2024-09-19T04:20:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef0155260 con 0x7f9ef00af2a0 2024-09-19T04:20:40.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ef006e000 con 0x7f9ef00af2a0 2024-09-19T04:20:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ee5ffb640 1 -- 172.21.15.76:0/237517550 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f9ee0007450 con 0x7f9ef00af2a0 2024-09-19T04:20:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ee5ffb640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ed40678c0 0x7f9ed4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ee5ffb640 1 -- 172.21.15.76:0/237517550 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f9ee0086570 con 0x7f9ef00af2a0 2024-09-19T04:20:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ef4b1f640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ed40678c0 0x7f9ed4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.527+0000 7f9ef4b1f640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ed40678c0 0x7f9ed4069d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0ad000 tx=0x7f9eec0a6000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.531+0000 7f9ee5ffb640 1 -- 172.21.15.76:0/237517550 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f9ee0057250 con 0x7f9ef00af2a0 2024-09-19T04:20:40.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.675+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ef0075780 con 0x7f9ed40678c0 2024-09-19T04:20:40.678 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.675+0000 7f9ee5ffb640 1 -- 172.21.15.76:0/237517550 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13513 (secure 0 0 0) 0x7f9ef0075780 con 0x7f9ed40678c0 2024-09-19T04:20:40.678 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:40.678 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ed40678c0 msgr2=0x7f9ed4069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ed40678c0 0x7f9ed4069d80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9eec0ad000 tx=0x7f9eec0a6000 comp rx=0 tx=0).stop 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ef00af2a0 msgr2=0x7f9ef014a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ef00af2a0 0x7f9ef014a740 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f9ee000c750 tx=0x7f9ee000cc10 comp rx=0 tx=0).stop 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 shutdown_connections 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f9ed40678c0 0x7f9ed4069d80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ef015c2a0 0x7f9ef014c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f9ef015bec0 0x7f9ef014c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 --2- 172.21.15.76:0/237517550 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f9ef00af2a0 0x7f9ef014a740 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.680 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 >> 172.21.15.76:0/237517550 conn(0x7f9ef00b1f90 msgr2=0x7f9ef00b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 shutdown_connections 2024-09-19T04:20:40.681 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.679+0000 7f9ef7584640 1 -- 172.21.15.76:0/237517550 wait complete. 2024-09-19T04:20:40.695 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-19T04:20:39.876912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261236,"kb_used_data":11724,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713484,"statfs":{"total":773094113280,"available":772826607616,"internally_reserved":0,"allocated":12005376,"data_stored":6284846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-115824,"num_objects":-68,"num_object_clones":0,"num_object_copies":-136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-68,"num_whiteouts":0,"num_read":-1,"num_read_kb":0,"num_write":-248,"num_write_kb":-199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002197"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":331,"reported_epoch":108,"state":"active+clean","last_fresh":"2024-09-19T04:20:37.254539+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:20:37.254539+0000","last_peered":"2024-09-19T04:20:37.254539+0000","last_clean":"2024-09-19T04:20:37.254539+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:20:37.254539+0000","last_undegraded":"2024-09-19T04:20:37.254539+0000","last_fullsized":"2024-09-19T04:20:37.254539+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542236,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":1233336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542237,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542237,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542236,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542236,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542237,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542237,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1339392,"data_stored":627120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33152,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338688,"statfs":{"total":96636764160,"available":96602816512,"internally_reserved":0,"allocated":2011136,"data_stored":1288790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:20:40.696 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T04:20:40.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.811+0000 7fa5aa785640 1 Processor -- start 2024-09-19T04:20:40.814 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.811+0000 7fa5aa785640 1 -- start start 2024-09-19T04:20:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5aa785640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5a4161a60 0x7fa5a40d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5aa785640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a40d2e90 0x7fa5a40cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5aa785640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5a40d32b0 con 0x7fa5a40cf450 2024-09-19T04:20:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5aa785640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fa5a40d0300 con 0x7fa5a4161a60 2024-09-19T04:20:40.815 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5aa785640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a406e6c0 con 0x7fa5a40d2e90 2024-09-19T04:20:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa59b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a40d2e90 0x7fa5a40cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.816 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5a3fff640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5a4161a60 0x7fa5a40d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa59b7fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a40d2e90 0x7fa5a40cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:39456/0 (socket says 172.21.15.76:39456) 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5a8d22640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa5a40cf450 0x7fa5a40d0a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:42032/0 (socket says 172.21.15.76:42032) 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa59b7fe640 1 -- 172.21.15.76:0/3218907672 learned_addr learned my addr 172.21.15.76:0/3218907672 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa59b7fe640 1 -- 172.21.15.76:0/3218907672 >> v2:172.21.15.116:3300/0 conn(0x7fa5a4161a60 msgr2=0x7fa5a40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa59b7fe640 1 --2- 172.21.15.76:0/3218907672 >> v2:172.21.15.116:3300/0 conn(0x7fa5a4161a60 0x7fa5a40d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa59b7fe640 1 -- 172.21.15.76:0/3218907672 >> v1:172.21.15.116:6789/0 conn(0x7fa5a40cf450 legacy=0x7fa5a40d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa59b7fe640 1 -- 172.21.15.76:0/3218907672 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40b0f60 con 0x7fa5a40d2e90 2024-09-19T04:20:40.817 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.815+0000 7fa5a37fe640 1 -- 172.21.15.76:0/3218907672 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3746309493 0 0) 0x7fa5a40d32b0 con 0x7fa5a40cf450 2024-09-19T04:20:40.818 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa59b7fe640 1 --2- 172.21.15.76:0/3218907672 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a40d2e90 0x7fa5a40cef10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009870 tx=0x7fa58c01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e439629156167aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.819 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a37fe640 1 -- 172.21.15.76:0/3218907672 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40d1930 con 0x7fa5a40d2e90 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a37fe640 1 -- 172.21.15.76:0/3218907672 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa58c0af070 con 0x7fa5a40d2e90 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a37fe640 1 -- 172.21.15.76:0/3218907672 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0aa820 con 0x7fa5a40d2e90 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- 172.21.15.76:0/3218907672 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a40d2e90 msgr2=0x7fa5a40cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 --2- 172.21.15.76:0/3218907672 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a40d2e90 0x7fa5a40cef10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009870 tx=0x7fa58c01c4a0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- 172.21.15.76:0/3218907672 shutdown_connections 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 --2- 172.21.15.76:0/3218907672 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a40d2e90 0x7fa5a40cef10 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 --2- 172.21.15.76:0/3218907672 >> v2:172.21.15.116:3300/0 conn(0x7fa5a4161a60 0x7fa5a40d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- 172.21.15.76:0/3218907672 >> 172.21.15.76:0/3218907672 conn(0x7fa5a408bf90 msgr2=0x7fa5a409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- 172.21.15.76:0/3218907672 shutdown_connections 2024-09-19T04:20:40.820 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- 172.21.15.76:0/3218907672 wait complete. 2024-09-19T04:20:40.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 Processor -- start 2024-09-19T04:20:40.821 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- start start 2024-09-19T04:20:40.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40cf450 0x7fa5a40b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5a40d2e90 0x7fa5a40b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.823 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a4161a60 0x7fa5a414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a40cf8a0 con 0x7fa5a40cf450 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fa5a406d190 con 0x7fa5a40d2e90 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5aa785640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fa5a40cfa20 con 0x7fa5a4161a60 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40cf450 0x7fa5a40b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40cf450 0x7fa5a40b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52476/0 (socket says 172.21.15.76:52476) 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 -- 172.21.15.76:0/2804540194 learned_addr learned my addr 172.21.15.76:0/2804540194 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa59b7fe640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5a40d2e90 0x7fa5a40b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a8d22640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a4161a60 0x7fa5a414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 -- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a4161a60 msgr2=0x7fa5a414f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a4161a60 0x7fa5a414f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.824 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 -- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5a40d2e90 msgr2=0x7fa5a40b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5a40d2e90 0x7fa5a40b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 -- 172.21.15.76:0/2804540194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5a40b8dc0 con 0x7fa5a40cf450 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a8d22640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a4161a60 0x7fa5a414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa59b7fe640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5a40d2e90 0x7fa5a40b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a3fff640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40cf450 0x7fa5a40b4c00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9b0 tx=0x7fa59400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.819+0000 7fa5a17fa640 1 -- 172.21.15.76:0/2804540194 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594017070 con 0x7fa5a40cf450 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5a17fa640 1 -- 172.21.15.76:0/2804540194 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa59400c9a0 con 0x7fa5a40cf450 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5a40b9100 con 0x7fa5a40cf450 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5a17fa640 1 -- 172.21.15.76:0/2804540194 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59400cd70 con 0x7fa5a40cf450 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5a40b9940 con 0x7fa5a40cf450 2024-09-19T04:20:40.825 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5a40cef10 con 0x7fa5a40cf450 2024-09-19T04:20:40.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5a17fa640 1 -- 172.21.15.76:0/2804540194 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fa59401a070 con 0x7fa5a40cf450 2024-09-19T04:20:40.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5a17fa640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5840678f0 0x7fa584069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:40.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa59b7fe640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5840678f0 0x7fa584069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:40.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa5a17fa640 1 -- 172.21.15.76:0/2804540194 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fa594085c70 con 0x7fa5a40cf450 2024-09-19T04:20:40.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.823+0000 7fa59b7fe640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5840678f0 0x7fa584069db0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ae000 tx=0x7fa58c0a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:40.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.827+0000 7fa5a17fa640 1 -- 172.21.15.76:0/2804540194 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fa5a40b9100 con 0x7fa5a40cf450 2024-09-19T04:20:40.991 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.991+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5a40b18c0 con 0x7fa5a40cf450 2024-09-19T04:20:40.993 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.991+0000 7fa5a17fa640 1 -- 172.21.15.76:0/2804540194 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) ==== 74+0+13599 (secure 0 0 0) 0x7fa594056870 con 0x7fa5a40cf450 2024-09-19T04:20:40.994 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:40.994 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":109,"fsid":"2bcf595d-5a10-4e84-8b11-2533969843a0","created":"2024-09-19T04:13:01.019487+0000","modified":"2024-09-19T04:20:37.997997+0000","last_up_change":"2024-09-19T04:13:12.699761+0000","last_in_change":"2024-09-19T04:13:03.248507+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T04:13:13.812655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"04ec1863-8fa9-4395-beba-35ccf9684a78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6800","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6801","nonce":2890813412}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6802","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6803","nonce":2890813412}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6806","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6807","nonce":2890813412}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6804","nonce":2890813412},{"type":"v1","addr":"172.21.15.76:6805","nonce":2890813412}]},"public_addr":"172.21.15.76:6801/2890813412","cluster_addr":"172.21.15.76:6803/2890813412","heartbeat_back_addr":"172.21.15.76:6807/2890813412","heartbeat_front_addr":"172.21.15.76:6805/2890813412","state":["exists","up"]},{"osd":1,"uuid":"da4116da-20bf-495a-a087-724ecd3b59dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6808","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6809","nonce":1010487110}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6810","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6811","nonce":1010487110}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6814","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6815","nonce":1010487110}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6812","nonce":1010487110},{"type":"v1","addr":"172.21.15.76:6813","nonce":1010487110}]},"public_addr":"172.21.15.76:6809/1010487110","cluster_addr":"172.21.15.76:6811/1010487110","heartbeat_back_addr":"172.21.15.76:6815/1010487110","heartbeat_front_addr":"172.21.15.76:6813/1010487110","state":["exists","up"]},{"osd":2,"uuid":"047d1ca8-76db-48fc-9895-0abdd6738312","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6824","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6825","nonce":1496429724}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6826","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6827","nonce":1496429724}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6830","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6831","nonce":1496429724}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6828","nonce":1496429724},{"type":"v1","addr":"172.21.15.76:6829","nonce":1496429724}]},"public_addr":"172.21.15.76:6825/1496429724","cluster_addr":"172.21.15.76:6827/1496429724","heartbeat_back_addr":"172.21.15.76:6831/1496429724","heartbeat_front_addr":"172.21.15.76:6829/1496429724","state":["exists","up"]},{"osd":3,"uuid":"83621a44-76e6-4acd-8e4c-fcba39569ac2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6816","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6817","nonce":1627524440}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6818","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6819","nonce":1627524440}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6822","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6823","nonce":1627524440}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6820","nonce":1627524440},{"type":"v1","addr":"172.21.15.76:6821","nonce":1627524440}]},"public_addr":"172.21.15.76:6817/1627524440","cluster_addr":"172.21.15.76:6819/1627524440","heartbeat_back_addr":"172.21.15.76:6823/1627524440","heartbeat_front_addr":"172.21.15.76:6821/1627524440","state":["exists","up"]},{"osd":4,"uuid":"388909a1-37ed-4039-85ef-fe9f10af682c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6801","nonce":3265669393}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6803","nonce":3265669393}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6807","nonce":3265669393}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6804","nonce":3265669393},{"type":"v1","addr":"172.21.15.116:6805","nonce":3265669393}]},"public_addr":"172.21.15.116:6801/3265669393","cluster_addr":"172.21.15.116:6803/3265669393","heartbeat_back_addr":"172.21.15.116:6807/3265669393","heartbeat_front_addr":"172.21.15.116:6805/3265669393","state":["exists","up"]},{"osd":5,"uuid":"2f3b2716-0ce3-487d-add8-be076a5138c4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6808","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6809","nonce":3134608166}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6811","nonce":3134608166}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6815","nonce":3134608166}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6812","nonce":3134608166},{"type":"v1","addr":"172.21.15.116:6813","nonce":3134608166}]},"public_addr":"172.21.15.116:6809/3134608166","cluster_addr":"172.21.15.116:6811/3134608166","heartbeat_back_addr":"172.21.15.116:6815/3134608166","heartbeat_front_addr":"172.21.15.116:6813/3134608166","state":["exists","up"]},{"osd":6,"uuid":"c4170a51-44cc-44f1-b384-ae0df43b54dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6816","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6817","nonce":2407229160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6819","nonce":2407229160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6823","nonce":2407229160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":2407229160},{"type":"v1","addr":"172.21.15.116:6821","nonce":2407229160}]},"public_addr":"172.21.15.116:6817/2407229160","cluster_addr":"172.21.15.116:6819/2407229160","heartbeat_back_addr":"172.21.15.116:6823/2407229160","heartbeat_front_addr":"172.21.15.116:6821/2407229160","state":["exists","up"]},{"osd":7,"uuid":"ec38642c-af4a-4d58-9af5-9200d729196d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6825","nonce":2361616500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6827","nonce":2361616500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6831","nonce":2361616500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":2361616500},{"type":"v1","addr":"172.21.15.116:6829","nonce":2361616500}]},"public_addr":"172.21.15.116:6825/2361616500","cluster_addr":"172.21.15.116:6827/2361616500","heartbeat_back_addr":"172.21.15.116:6831/2361616500","heartbeat_front_addr":"172.21.15.116:6829/2361616500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.520554+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.499083+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:10.170913+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.665602+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.286236+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.222662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.572404+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T04:13:09.960803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.76:6835/2501269734":"2024-09-20T04:20:34.986813+0000","172.21.15.76:6834/2501269734":"2024-09-20T04:20:34.986813+0000","172.21.15.116:6835/255266111":"2024-09-20T04:20:34.986784+0000","172.21.15.116:6834/255266111":"2024-09-20T04:20:34.986784+0000","172.21.15.76:6837/3881707282":"2024-09-20T04:20:34.986831+0000","172.21.15.76:6836/3881707282":"2024-09-20T04:20:34.986831+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5840678f0 msgr2=0x7fa584069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5840678f0 0x7fa584069db0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0ae000 tx=0x7fa58c0a7000 comp rx=0 tx=0).stop 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40cf450 msgr2=0x7fa5a40b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40cf450 0x7fa5a40b4c00 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e9b0 tx=0x7fa59400ee70 comp rx=0 tx=0).stop 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 shutdown_connections 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fa5840678f0 0x7fa584069db0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fa5a4161a60 0x7fa5a414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa5a40d2e90 0x7fa5a40b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.996 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 --2- 172.21.15.76:0/2804540194 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fa5a40cf450 0x7fa5a40b4c00 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:40.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 >> 172.21.15.76:0/2804540194 conn(0x7fa5a408bf90 msgr2=0x7fa5a414ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:40.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 shutdown_connections 2024-09-19T04:20:40.997 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:40.995+0000 7fa5aa785640 1 -- 172.21.15.76:0/2804540194 wait complete. 2024-09-19T04:20:42.014 INFO:tasks.ceph:Scrubbing osd.0 2024-09-19T04:20:42.014 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:42.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.147+0000 7f0979388640 1 Processor -- start 2024-09-19T04:20:42.149 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.147+0000 7f0979388640 1 -- start start 2024-09-19T04:20:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.147+0000 7f0979388640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09741520a0 0x7f097406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.147+0000 7f0979388640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f097415f3a0 0x7f09740adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.147+0000 7f0979388640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0974070250 con 0x7f097406fe50 2024-09-19T04:20:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.147+0000 7f0979388640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f0974152480 con 0x7f09741520a0 2024-09-19T04:20:42.150 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.147+0000 7f0979388640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f09740aede0 con 0x7f097415f3a0 2024-09-19T04:20:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09737fe640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f097415f3a0 0x7f09740adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f0972ffd640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09741520a0 0x7f097406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f096a7fc640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f097406fe50 0x7f097415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:42042/0 (socket says 172.21.15.76:42042) 2024-09-19T04:20:42.151 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f096a7fc640 1 -- 172.21.15.76:0/3586729644 learned_addr learned my addr 172.21.15.76:0/3586729644 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09727fc640 1 -- 172.21.15.76:0/3586729644 <== mon.1 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 617754646 0 0) 0x7f0974070250 con 0x7f097406fe50 2024-09-19T04:20:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09727fc640 1 -- 172.21.15.76:0/3586729644 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0960003530 con 0x7f097406fe50 2024-09-19T04:20:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09737fe640 1 -- 172.21.15.76:0/3586729644 >> v2:172.21.15.116:3300/0 conn(0x7f09741520a0 msgr2=0x7f097406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09737fe640 1 --2- 172.21.15.76:0/3586729644 >> v2:172.21.15.116:3300/0 conn(0x7f09741520a0 0x7f097406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09737fe640 1 -- 172.21.15.76:0/3586729644 >> v1:172.21.15.116:6789/0 conn(0x7f097406fe50 legacy=0x7f097415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.152 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09737fe640 1 -- 172.21.15.76:0/3586729644 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f097415fa10 con 0x7f097415f3a0 2024-09-19T04:20:42.153 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09737fe640 1 --2- 172.21.15.76:0/3586729644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f097415f3a0 0x7f09740adc80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f096c009d90 tx=0x7f096c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b223aeebc0d61823 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09727fc640 1 -- 172.21.15.76:0/3586729644 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c0b2070 con 0x7f097415f3a0 2024-09-19T04:20:42.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09727fc640 1 -- 172.21.15.76:0/3586729644 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c0a5920 con 0x7f097415f3a0 2024-09-19T04:20:42.154 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f09727fc640 1 -- 172.21.15.76:0/3586729644 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c0a5cf0 con 0x7f097415f3a0 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f0979388640 1 -- 172.21.15.76:0/3586729644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f097415f3a0 msgr2=0x7f09740adc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f0979388640 1 --2- 172.21.15.76:0/3586729644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f097415f3a0 0x7f09740adc80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f096c009d90 tx=0x7f096c01c970 comp rx=0 tx=0).stop 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f0979388640 1 -- 172.21.15.76:0/3586729644 shutdown_connections 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f0979388640 1 --2- 172.21.15.76:0/3586729644 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f097415f3a0 0x7f09740adc80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f0979388640 1 --2- 172.21.15.76:0/3586729644 >> v2:172.21.15.116:3300/0 conn(0x7f09741520a0 0x7f097406f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.151+0000 7f0979388640 1 -- 172.21.15.76:0/3586729644 >> 172.21.15.76:0/3586729644 conn(0x7f09740b1f90 msgr2=0x7f09740b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- 172.21.15.76:0/3586729644 shutdown_connections 2024-09-19T04:20:42.155 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- 172.21.15.76:0/3586729644 wait complete. 2024-09-19T04:20:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 Processor -- start 2024-09-19T04:20:42.156 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- start start 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f097406fe50 0x7f0974142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09741520a0 0x7f097409cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f097415f3a0 0x7f097409d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f0974073d30 con 0x7f097415f3a0 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f0974073a50 con 0x7f097406fe50 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f0974167f10 con 0x7f09741520a0 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f097415f3a0 0x7f097409d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f097415f3a0 0x7f097409d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52480/0 (socket says 172.21.15.76:52480) 2024-09-19T04:20:42.158 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 -- 172.21.15.76:0/4229336518 learned_addr learned my addr 172.21.15.76:0/4229336518 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0972ffd640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f097406fe50 0x7f0974142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f096a7fc640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09741520a0 0x7f097409cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 -- 172.21.15.76:0/4229336518 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09741520a0 msgr2=0x7f097409cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f09741520a0 0x7f097409cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 -- 172.21.15.76:0/4229336518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f097406fe50 msgr2=0x7f0974142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f097406fe50 0x7f0974142300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 -- 172.21.15.76:0/4229336518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0974160970 con 0x7f097415f3a0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f09737fe640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f097415f3a0 0x7f097409d060 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f096c009d90 tx=0x7f096c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c0b2050 con 0x7f097415f3a0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096c0be070 con 0x7f097415f3a0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f097409d730 con 0x7f097415f3a0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096c0ada90 con 0x7f097415f3a0 2024-09-19T04:20:42.159 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f097409dcf0 con 0x7f097415f3a0 2024-09-19T04:20:42.160 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.155+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f097406f910 con 0x7f097415f3a0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f096c0a57a0 con 0x7f097415f3a0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096bfff640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f09540678c0 0x7f0954069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f096c121800 con 0x7f097415f3a0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f0972ffd640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f09540678c0 0x7f0954069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096bfff640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7f095406dea0 0x7f09540702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f096c0c5e70 con 0x7f095406dea0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f096c121c90 con 0x7f097415f3a0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096a7fc640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7f095406dea0 0x7f09540702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f0972ffd640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f09540678c0 0x7f0954069d80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0964009af0 tx=0x7f0964008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096a7fc640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7f095406dea0 0x7f09540702e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.161 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.159+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== osd.0 v2:172.21.15.76:6800/2890813412 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f096c0c5e70 con 0x7f095406dea0 2024-09-19T04:20:42.173 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 --> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0974142a90 con 0x7f095406dea0 2024-09-19T04:20:42.174 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f096bfff640 1 -- 172.21.15.76:0/4229336518 <== osd.0 v2:172.21.15.76:6800/2890813412 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0974142a90 con 0x7f095406dea0 2024-09-19T04:20:42.174 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:42.174 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:42.174 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7f095406dea0 msgr2=0x7f09540702e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6800/2890813412,v1:172.21.15.76:6801/2890813412] conn(0x7f095406dea0 0x7f09540702e0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f09540678c0 msgr2=0x7f0954069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f09540678c0 0x7f0954069d80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0964009af0 tx=0x7f0964008040 comp rx=0 tx=0).stop 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f097415f3a0 msgr2=0x7f097409d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 --2- 172.21.15.76:0/4229336518 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f097415f3a0 0x7f097409d060 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f096c009d90 tx=0x7f096c019710 comp rx=0 tx=0).stop 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f09737fe640 1 -- 172.21.15.76:0/4229336518 reap_dead start 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 shutdown_connections 2024-09-19T04:20:42.175 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 >> 172.21.15.76:0/4229336518 conn(0x7f09740b1f90 msgr2=0x7f09740b3530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 shutdown_connections 2024-09-19T04:20:42.176 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.171+0000 7f0979388640 1 -- 172.21.15.76:0/4229336518 wait complete. 2024-09-19T04:20:42.190 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-19T04:20:42.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf926a5640 1 Processor -- start 2024-09-19T04:20:42.303 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf926a5640 1 -- start start 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf926a5640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcf8c073e60 0x7fcf8c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf926a5640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8c0742f0 con 0x7fcf8c06fef0 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf926a5640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf8c0b46a0 con 0x7fcf8c0b4290 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf926a5640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fcf8c06ee50 con 0x7fcf8c073e60 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fcf8c06fef0 0x7fcf8c15b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:47974/0 (socket says 172.21.15.76:47974) 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8bfff640 1 -- 172.21.15.76:0/727531195 learned_addr learned my addr 172.21.15.76:0/727531195 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8b7fe640 1 --2- 172.21.15.76:0/727531195 >> v2:172.21.15.116:3300/0 conn(0x7fcf8c073e60 0x7fcf8c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1596056671 0 0) 0x7fcf7c001120 con 0x7fcf8c06fef0 2024-09-19T04:20:42.306 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcf74003530 con 0x7fcf8c06fef0 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2177343697 0 0) 0x7fcf74003530 con 0x7fcf8c06fef0 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 >> v2:172.21.15.116:3300/0 conn(0x7fcf8c073e60 msgr2=0x7fcf8c0b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 --2- 172.21.15.76:0/727531195 >> v2:172.21.15.116:3300/0 conn(0x7fcf8c073e60 0x7fcf8c0b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 >> v1:172.21.15.116:6789/0 conn(0x7fcf8c0b4290 legacy=0x7fcf8c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c06f130 con 0x7fcf8c06fef0 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fcf7c001870 con 0x7fcf8c06fef0 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 >> v1:172.21.15.76:6789/0 conn(0x7fcf8c06fef0 legacy=0x7fcf8c15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740039c0 0x7fcf74003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf74004810 0x7fcf74004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf740051d0 0x7fcf7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.307 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c06ee50 con 0x7fcf740051d0 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7400c170 con 0x7fcf74004810 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7400c370 con 0x7fcf740039c0 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf90c42640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf740051d0 0x7fcf7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8b7fe640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf74004810 0x7fcf74004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf8bfff640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740039c0 0x7fcf74003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf90c42640 1 -- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740039c0 msgr2=0x7fcf74003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf90c42640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740039c0 0x7fcf74003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf90c42640 1 -- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf74004810 msgr2=0x7fcf74004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf90c42640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf74004810 0x7fcf74004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.308 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.303+0000 7fcf90c42640 1 -- 172.21.15.76:0/727531195 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf74003530 con 0x7fcf740051d0 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf90c42640 1 -- 172.21.15.76:0/727531195 reap_dead start 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf90c42640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf740051d0 0x7fcf7400b9d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001e70 tx=0x7fcf8401c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10afb84ebe1c485 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84001a90 con 0x7fcf740051d0 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf84003ba0 con 0x7fcf740051d0 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf8affd640 1 -- 172.21.15.76:0/727531195 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf840a6040 con 0x7fcf740051d0 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 -- 172.21.15.76:0/727531195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf740051d0 msgr2=0x7fcf7400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf740051d0 0x7fcf7400b9d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fcf84001e70 tx=0x7fcf8401c5a0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 -- 172.21.15.76:0/727531195 shutdown_connections 2024-09-19T04:20:42.310 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 --2- 172.21.15.76:0/727531195 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf740051d0 0x7fcf7400b9d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 -- 172.21.15.76:0/727531195 >> 172.21.15.76:0/727531195 conn(0x7fcf8c0b1f90 msgr2=0x7fcf8c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 -- 172.21.15.76:0/727531195 shutdown_connections 2024-09-19T04:20:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 -- 172.21.15.76:0/727531195 wait complete. 2024-09-19T04:20:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 Processor -- start 2024-09-19T04:20:42.311 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 -- start start 2024-09-19T04:20:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.307+0000 7fcf926a5640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740051d0 0x7fcf8c1405f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf8c152630 0x7fcf8c09ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.313 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf8c141c70 0x7fcf8c09b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c0b3d50 con 0x7fcf8c141c70 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8c06f6f0 con 0x7fcf8c152630 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8c0b3ed0 con 0x7fcf740051d0 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf8c141c70 0x7fcf8c09b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf8c141c70 0x7fcf8c09b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52496/0 (socket says 172.21.15.76:52496) 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 -- 172.21.15.76:0/390935137 learned_addr learned my addr 172.21.15.76:0/390935137 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf8bfff640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740051d0 0x7fcf8c1405f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf8b7fe640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf8c152630 0x7fcf8c09ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 -- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740051d0 msgr2=0x7fcf8c1405f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740051d0 0x7fcf8c1405f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.314 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 -- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf8c152630 msgr2=0x7fcf8c09ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf8c152630 0x7fcf8c09ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 -- 172.21.15.76:0/390935137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8c140bd0 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf90c42640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf8c141c70 0x7fcf8c09b6d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000f90 tx=0x7fcf74005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf88ff9640 1 -- 172.21.15.76:0/390935137 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf840af070 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf88ff9640 1 -- 172.21.15.76:0/390935137 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf84006b60 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8c140eb0 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf88ff9640 1 -- 172.21.15.76:0/390935137 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf84001a90 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8c141470 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8c15b860 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf88ff9640 1 -- 172.21.15.76:0/390935137 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fcf84007240 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf88ff9640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf780678c0 0x7fcf78069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf8bfff640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf780678c0 0x7fcf78069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf88ff9640 1 -- 172.21.15.76:0/390935137 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fcf8411f390 con 0x7fcf8c141c70 2024-09-19T04:20:42.315 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.311+0000 7fcf8bfff640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf780678c0 0x7fcf78069d80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007970 tx=0x7fcf7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.317 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.315+0000 7fcf88ff9640 1 -- 172.21.15.76:0/390935137 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fcf8c0b46a0 con 0x7fcf8c141c70 2024-09-19T04:20:42.476 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.475+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fcf8c0723f0 con 0x7fcf780678c0 2024-09-19T04:20:42.477 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.475+0000 7fcf88ff9640 1 -- 172.21.15.76:0/390935137 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcf8c0723f0 con 0x7fcf780678c0 2024-09-19T04:20:42.477 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 0 to deep-scrub 2024-09-19T04:20:42.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf780678c0 msgr2=0x7fcf78069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf780678c0 0x7fcf78069d80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c007970 tx=0x7fcf7c009040 comp rx=0 tx=0).stop 2024-09-19T04:20:42.479 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf8c141c70 msgr2=0x7fcf8c09b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf8c141c70 0x7fcf8c09b6d0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fcf84000f90 tx=0x7fcf74005840 comp rx=0 tx=0).stop 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 shutdown_connections 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fcf780678c0 0x7fcf78069d80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fcf8c141c70 0x7fcf8c09b6d0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf8c152630 0x7fcf8c09ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 --2- 172.21.15.76:0/390935137 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fcf740051d0 0x7fcf8c1405f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 >> 172.21.15.76:0/390935137 conn(0x7fcf8c0b1f90 msgr2=0x7fcf8c0b15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 shutdown_connections 2024-09-19T04:20:42.480 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.479+0000 7fcf926a5640 1 -- 172.21.15.76:0/390935137 wait complete. 2024-09-19T04:20:42.495 INFO:tasks.ceph:Scrubbing osd.1 2024-09-19T04:20:42.495 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:42.609 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.607+0000 7f6a0a9f0640 1 Processor -- start 2024-09-19T04:20:42.610 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.607+0000 7f6a0a9f0640 1 -- start start 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a04070020 0x7f6a0415ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a040aede0 con 0x7f6a041520a0 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a0415fa10 con 0x7f6a0415f1d0 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f6a04152480 con 0x7f6a04070020 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a037fe640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6a04070020 0x7f6a0415ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a03fff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6a041520a0 0x7f6a0406f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43162/0 (socket says 172.21.15.76:43162) 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a03fff640 1 -- 172.21.15.76:0/1664187391 learned_addr learned my addr 172.21.15.76:0/1664187391 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a037fe640 1 -- 172.21.15.76:0/1664187391 >> v1:172.21.15.116:6789/0 conn(0x7f6a0415f1d0 legacy=0x7f6a040adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:42.611 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a037fe640 1 -- 172.21.15.76:0/1664187391 >> v1:172.21.15.76:6789/0 conn(0x7f6a041520a0 legacy=0x7f6a0406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a037fe640 1 -- 172.21.15.76:0/1664187391 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a04074250 con 0x7f6a04070020 2024-09-19T04:20:42.612 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a02ffd640 1 -- 172.21.15.76:0/1664187391 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 444560532 0 0) 0x7f6a040aede0 con 0x7f6a041520a0 2024-09-19T04:20:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a037fe640 1 --2- 172.21.15.76:0/1664187391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a04070020 0x7f6a0415ec90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f69f800ba60 tx=0x7f69f801eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad47b188c694e9e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a02ffd640 1 -- 172.21.15.76:0/1664187391 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f80b4070 con 0x7f6a04070020 2024-09-19T04:20:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a02ffd640 1 -- 172.21.15.76:0/1664187391 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f80a7bd0 con 0x7f6a04070020 2024-09-19T04:20:42.613 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a02ffd640 1 -- 172.21.15.76:0/1664187391 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a040aeba0 con 0x7f6a04070020 2024-09-19T04:20:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/1664187391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a04070020 msgr2=0x7f6a0415ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 --2- 172.21.15.76:0/1664187391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a04070020 0x7f6a0415ec90 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f69f800ba60 tx=0x7f69f801eb00 comp rx=0 tx=0).stop 2024-09-19T04:20:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/1664187391 shutdown_connections 2024-09-19T04:20:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 --2- 172.21.15.76:0/1664187391 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a04070020 0x7f6a0415ec90 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/1664187391 >> 172.21.15.76:0/1664187391 conn(0x7f6a040b1f90 msgr2=0x7f6a040b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/1664187391 shutdown_connections 2024-09-19T04:20:42.614 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.611+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/1664187391 wait complete. 2024-09-19T04:20:42.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 Processor -- start 2024-09-19T04:20:42.615 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 -- start start 2024-09-19T04:20:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a04070020 0x7f6a040b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a041520a0 0x7f6a04098e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a0415f1d0 0x7f6a04148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0415f680 con 0x7f6a0415f1d0 2024-09-19T04:20:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04073d30 con 0x7f6a041520a0 2024-09-19T04:20:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6a0406e800 con 0x7f6a04070020 2024-09-19T04:20:42.617 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a0415f1d0 0x7f6a04148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a0415f1d0 0x7f6a04148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51896/0 (socket says 172.21.15.76:51896) 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 -- 172.21.15.76:0/2383416565 learned_addr learned my addr 172.21.15.76:0/2383416565 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a03fff640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a04070020 0x7f6a040b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 -- 172.21.15.76:0/2383416565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a04070020 msgr2=0x7f6a040b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a04070020 0x7f6a040b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 -- 172.21.15.76:0/2383416565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a041520a0 msgr2=0x7f6a04098e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a037fe640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a041520a0 0x7f6a04098e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a041520a0 0x7f6a04098e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 -- 172.21.15.76:0/2383416565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0414b960 con 0x7f6a0415f1d0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a08f8d640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a0415f1d0 0x7f6a04148220 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c780 tx=0x7f69fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc017070 con 0x7f6a0415f1d0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69fc003db0 con 0x7f6a0415f1d0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc0052b0 con 0x7f6a0415f1d0 2024-09-19T04:20:42.618 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a0414c870 con 0x7f6a0415f1d0 2024-09-19T04:20:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a0414b680 con 0x7f6a0415f1d0 2024-09-19T04:20:42.619 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6a0415ec90 con 0x7f6a0415f1d0 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.615+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f69fc01a070 con 0x7f6a0415f1d0 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a00ff9640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69f00678c0 0x7f69f0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a03fff640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69f00678c0 0x7f69f0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f69fc0859c0 con 0x7f6a0415f1d0 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a00ff9640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7f69f006dea0 0x7f69f00702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69f00709b0 con 0x7f69f006dea0 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f69fc085e50 con 0x7f6a0415f1d0 2024-09-19T04:20:42.620 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a037fe640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7f69f006dea0 0x7f69f00702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a03fff640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69f00678c0 0x7f69f0069d80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f69f4000f70 tx=0x7f69f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a037fe640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7f69f006dea0 0x7f69f00702e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.621 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.619+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== osd.1 v2:172.21.15.76:6808/1010487110 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f69f00709b0 con 0x7f69f006dea0 2024-09-19T04:20:42.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 --> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6a0414bbb0 con 0x7f69f006dea0 2024-09-19T04:20:42.633 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a00ff9640 1 -- 172.21.15.76:0/2383416565 <== osd.1 v2:172.21.15.76:6808/1010487110 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6a0414bbb0 con 0x7f69f006dea0 2024-09-19T04:20:42.633 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:42.633 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:42.633 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:42.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7f69f006dea0 msgr2=0x7f69f00702e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6808/1010487110,v1:172.21.15.76:6809/1010487110] conn(0x7f69f006dea0 0x7f69f00702e0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69f00678c0 msgr2=0x7f69f0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.634 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f69f00678c0 0x7f69f0069d80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f69f4000f70 tx=0x7f69f4009040 comp rx=0 tx=0).stop 2024-09-19T04:20:42.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a0415f1d0 msgr2=0x7f6a04148220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 --2- 172.21.15.76:0/2383416565 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a0415f1d0 0x7f6a04148220 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00c780 tx=0x7f69fc00cc40 comp rx=0 tx=0).stop 2024-09-19T04:20:42.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a08f8d640 1 -- 172.21.15.76:0/2383416565 reap_dead start 2024-09-19T04:20:42.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 shutdown_connections 2024-09-19T04:20:42.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 >> 172.21.15.76:0/2383416565 conn(0x7f6a040b1f90 msgr2=0x7f6a0414d550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 shutdown_connections 2024-09-19T04:20:42.635 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.631+0000 7f6a0a9f0640 1 -- 172.21.15.76:0/2383416565 wait complete. 2024-09-19T04:20:42.647 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-19T04:20:42.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d37a0640 1 Processor -- start 2024-09-19T04:20:42.751 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d37a0640 1 -- start start 2024-09-19T04:20:42.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d37a0640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5cc0d3060 0x7fe5cc0cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d37a0640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe5cc0cf450 0x7fe5cc06d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d37a0640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5cc0d0300 con 0x7fe5cc161a60 2024-09-19T04:20:42.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d37a0640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe5cc0b0f60 con 0x7fe5cc0d3060 2024-09-19T04:20:42.752 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d37a0640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe5cc0cfd40 con 0x7fe5cc0cf450 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d0d3b640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe5cc0d3060 0x7fe5cc0cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d153c640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fe5cc161a60 0x7fe5cc0d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43170/0 (socket says 172.21.15.76:43170) 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d153c640 1 -- 172.21.15.76:0/736166088 learned_addr learned my addr 172.21.15.76:0/736166088 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d0d3b640 1 -- 172.21.15.76:0/736166088 >> v2:172.21.15.116:3300/0 conn(0x7fe5cc0cf450 msgr2=0x7fe5cc06d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d0d3b640 1 --2- 172.21.15.76:0/736166088 >> v2:172.21.15.116:3300/0 conn(0x7fe5cc0cf450 0x7fe5cc06d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d0d3b640 1 -- 172.21.15.76:0/736166088 >> v1:172.21.15.76:6789/0 conn(0x7fe5cc161a60 legacy=0x7fe5cc0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5d0d3b640 1 -- 172.21.15.76:0/736166088 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc0d0020 con 0x7fe5cc0d3060 2024-09-19T04:20:42.753 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.751+0000 7fe5cbfff640 1 -- 172.21.15.76:0/736166088 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1677980007 0 0) 0x7fe5c4001120 con 0x7fe5cc161a60 2024-09-19T04:20:42.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 --2- 172.21.15.76:0/736166088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc0cef10 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0ce950 tx=0x7fe5c001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86657171c26234b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.754 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5cbfff640 1 -- 172.21.15.76:0/736166088 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00b1070 con 0x7fe5cc0d3060 2024-09-19T04:20:42.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5cbfff640 1 -- 172.21.15.76:0/736166088 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c00ac620 con 0x7fe5cc0d3060 2024-09-19T04:20:42.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5cbfff640 1 -- 172.21.15.76:0/736166088 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00ac9f0 con 0x7fe5cc0d3060 2024-09-19T04:20:42.755 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- 172.21.15.76:0/736166088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 msgr2=0x7fe5cc0cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/736166088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc0cef10 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0ce950 tx=0x7fe5c001c3c0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- 172.21.15.76:0/736166088 shutdown_connections 2024-09-19T04:20:42.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/736166088 >> v2:172.21.15.116:3300/0 conn(0x7fe5cc0cf450 0x7fe5cc06d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/736166088 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc0cef10 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- 172.21.15.76:0/736166088 >> 172.21.15.76:0/736166088 conn(0x7fe5cc08bf90 msgr2=0x7fe5cc09d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- 172.21.15.76:0/736166088 shutdown_connections 2024-09-19T04:20:42.756 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- 172.21.15.76:0/736166088 wait complete. 2024-09-19T04:20:42.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 Processor -- start 2024-09-19T04:20:42.758 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- start start 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5cc0cf450 0x7fe5cc156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc1541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5cc161a60 0x7fe5cc154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc06e400 con 0x7fe5cc0d3060 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe5cc0cfab0 con 0x7fe5cc0cf450 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe5cc06e580 con 0x7fe5cc161a60 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc1541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc1541d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51912/0 (socket says 172.21.15.76:51912) 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 -- 172.21.15.76:0/3267487716 learned_addr learned my addr 172.21.15.76:0/3267487716 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d153c640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5cc0cf450 0x7fe5cc156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.759 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d1d3d640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5cc161a60 0x7fe5cc154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 -- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5cc161a60 msgr2=0x7fe5cc154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5cc161a60 0x7fe5cc154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 -- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5cc0cf450 msgr2=0x7fe5cc156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5cc0cf450 0x7fe5cc156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 -- 172.21.15.76:0/3267487716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5cc0b13d0 con 0x7fe5cc0d3060 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d1d3d640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5cc161a60 0x7fe5cc154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d0d3b640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc1541d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00af040 tx=0x7fe5c001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5c9ffb640 1 -- 172.21.15.76:0/3267487716 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00b1050 con 0x7fe5cc0d3060 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5c9ffb640 1 -- 172.21.15.76:0/3267487716 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5c00bd070 con 0x7fe5cc0d3060 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5cc0b1640 con 0x7fe5cc0d3060 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5c9ffb640 1 -- 172.21.15.76:0/3267487716 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00b88b0 con 0x7fe5cc0d3060 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.755+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5cc0b1c00 con 0x7fe5cc0d3060 2024-09-19T04:20:42.760 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.759+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5cc06d6b0 con 0x7fe5cc0d3060 2024-09-19T04:20:42.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.759+0000 7fe5c9ffb640 1 -- 172.21.15.76:0/3267487716 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe5c00ac4e0 con 0x7fe5cc0d3060 2024-09-19T04:20:42.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.759+0000 7fe5c9ffb640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5b80678c0 0x7fe5b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:42.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.759+0000 7fe5c9ffb640 1 -- 172.21.15.76:0/3267487716 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fe5c0120500 con 0x7fe5cc0d3060 2024-09-19T04:20:42.761 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.759+0000 7fe5d153c640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5b80678c0 0x7fe5b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:42.762 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.759+0000 7fe5d153c640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5b80678c0 0x7fe5b8069d80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000f70 tx=0x7fe5c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:42.764 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.763+0000 7fe5c9ffb640 1 -- 172.21.15.76:0/3267487716 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe5c0016070 con 0x7fe5cc0d3060 2024-09-19T04:20:42.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.915+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fe5cc0d2cd0 con 0x7fe5b80678c0 2024-09-19T04:20:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.915+0000 7fe5c9ffb640 1 -- 172.21.15.76:0/3267487716 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe5cc0d2cd0 con 0x7fe5b80678c0 2024-09-19T04:20:42.917 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 1 to deep-scrub 2024-09-19T04:20:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5b80678c0 msgr2=0x7fe5b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5b80678c0 0x7fe5b8069d80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe5c4000f70 tx=0x7fe5c4009040 comp rx=0 tx=0).stop 2024-09-19T04:20:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 msgr2=0x7fe5cc1541d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc1541d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00af040 tx=0x7fe5c001ac00 comp rx=0 tx=0).stop 2024-09-19T04:20:42.919 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 shutdown_connections 2024-09-19T04:20:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe5b80678c0 0x7fe5b8069d80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe5cc161a60 0x7fe5cc154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe5cc0d3060 0x7fe5cc1541d0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 --2- 172.21.15.76:0/3267487716 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5cc0cf450 0x7fe5cc156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 >> 172.21.15.76:0/3267487716 conn(0x7fe5cc08bf90 msgr2=0x7fe5cc0d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 shutdown_connections 2024-09-19T04:20:42.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:42.919+0000 7fe5d37a0640 1 -- 172.21.15.76:0/3267487716 wait complete. 2024-09-19T04:20:42.934 INFO:tasks.ceph:Scrubbing osd.2 2024-09-19T04:20:42.935 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:43.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcf0c17640 1 Processor -- start 2024-09-19T04:20:43.055 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcf0c17640 1 -- start start 2024-09-19T04:20:43.056 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcf0c17640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcec1457a0 0x7fdcec0d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcf0c17640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcec0d27a0 con 0x7fdcec0b4510 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcf0c17640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcec0d2a80 con 0x7fdcec0cf800 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcf0c17640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fdcec0d2d60 con 0x7fdcec1457a0 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcea59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcec1457a0 0x7fdcec0d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcea59c640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fdcec1457a0 0x7fdcec0d1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51916/0 (socket says 172.21.15.76:51916) 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcead9d640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fdcec0b4510 0x7fdcec0d0710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43182/0 (socket says 172.21.15.76:43182) 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcea59c640 1 -- 172.21.15.76:0/3648755660 learned_addr learned my addr 172.21.15.76:0/3648755660 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcea59c640 1 -- 172.21.15.76:0/3648755660 >> v1:172.21.15.116:6789/0 conn(0x7fdcec0cf800 legacy=0x7fdcec0b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcea59c640 1 -- 172.21.15.76:0/3648755660 >> v1:172.21.15.76:6789/0 conn(0x7fdcec0b4510 legacy=0x7fdcec0d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdcea59c640 1 -- 172.21.15.76:0/3648755660 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec0d01f0 con 0x7fdcec1457a0 2024-09-19T04:20:43.057 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.055+0000 7fdce959a640 1 -- 172.21.15.76:0/3648755660 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077042278 0 0) 0x7fdcec0d27a0 con 0x7fdcec0b4510 2024-09-19T04:20:43.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcea59c640 1 --2- 172.21.15.76:0/3648755660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec0d1ea0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0098a0 tx=0x7fdcdc01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a090884c5f88fbdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdce959a640 1 -- 172.21.15.76:0/3648755660 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0b0070 con 0x7fdcec1457a0 2024-09-19T04:20:43.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdce959a640 1 -- 172.21.15.76:0/3648755660 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0ab600 con 0x7fdcec1457a0 2024-09-19T04:20:43.059 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdce959a640 1 -- 172.21.15.76:0/3648755660 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc0ab9d0 con 0x7fdcec1457a0 2024-09-19T04:20:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- 172.21.15.76:0/3648755660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 msgr2=0x7fdcec0d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 --2- 172.21.15.76:0/3648755660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec0d1ea0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0098a0 tx=0x7fdcdc01c470 comp rx=0 tx=0).stop 2024-09-19T04:20:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- 172.21.15.76:0/3648755660 shutdown_connections 2024-09-19T04:20:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 --2- 172.21.15.76:0/3648755660 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec0d1ea0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- 172.21.15.76:0/3648755660 >> 172.21.15.76:0/3648755660 conn(0x7fdcec06bcc0 msgr2=0x7fdcec06c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- 172.21.15.76:0/3648755660 shutdown_connections 2024-09-19T04:20:43.060 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- 172.21.15.76:0/3648755660 wait complete. 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 Processor -- start 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- start start 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcec0b4510 0x7fdcec15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdcec0cf800 0x7fdcec143d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec1442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec16a450 con 0x7fdcec1457a0 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fdcec16a2d0 con 0x7fdcec0b4510 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcf0c17640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fdcec16a5d0 con 0x7fdcec0cf800 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec1442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.063 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec1442b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51920/0 (socket says 172.21.15.76:51920) 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 -- 172.21.15.76:0/1613128039 learned_addr learned my addr 172.21.15.76:0/1613128039 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 -- 172.21.15.76:0/1613128039 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdcec0cf800 msgr2=0x7fdcec143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fdcec0cf800 0x7fdcec143d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 -- 172.21.15.76:0/1613128039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcec0b4510 msgr2=0x7fdcec15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdcec0b4510 0x7fdcec15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 -- 172.21.15.76:0/1613128039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcec15f3d0 con 0x7fdcec1457a0 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcead9d640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec1442b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c7d0 tx=0x7fdce400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.059+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce4018070 con 0x7fdcec1457a0 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdce400a640 con 0x7fdcec1457a0 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcec15e330 con 0x7fdcec1457a0 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce400aa70 con 0x7fdcec1457a0 2024-09-19T04:20:43.064 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcec15e8f0 con 0x7fdcec1457a0 2024-09-19T04:20:43.065 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcec0d1ea0 con 0x7fdcec1457a0 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fdce401b070 con 0x7fdcec1457a0 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdccc0678c0 0x7fdccc069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcea59c640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdccc0678c0 0x7fdccc069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fdce40857e0 con 0x7fdcec1457a0 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fdccc06dea0 0x7fdccc0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 --> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdccc0709b0 con 0x7fdccc06dea0 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fdce4085c70 con 0x7fdcec1457a0 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdce9d9b640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fdccc06dea0 0x7fdccc0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.066 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcea59c640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdccc0678c0 0x7fdccc069d80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ad000 tx=0x7fdcdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdce9d9b640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fdccc06dea0 0x7fdccc0702e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.067 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.063+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== osd.2 v2:172.21.15.76:6824/1496429724 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdccc0709b0 con 0x7fdccc06dea0 2024-09-19T04:20:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 --> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdcec15f5b0 con 0x7fdccc06dea0 2024-09-19T04:20:43.079 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcdaffd640 1 -- 172.21.15.76:0/1613128039 <== osd.2 v2:172.21.15.76:6824/1496429724 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdcec15f5b0 con 0x7fdccc06dea0 2024-09-19T04:20:43.079 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:43.079 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:43.079 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fdccc06dea0 msgr2=0x7fdccc0702e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6824/1496429724,v1:172.21.15.76:6825/1496429724] conn(0x7fdccc06dea0 0x7fdccc0702e0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdccc0678c0 msgr2=0x7fdccc069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fdccc0678c0 0x7fdccc069d80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc0ad000 tx=0x7fdcdc008040 comp rx=0 tx=0).stop 2024-09-19T04:20:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 msgr2=0x7fdcec1442b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.080 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 --2- 172.21.15.76:0/1613128039 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fdcec1457a0 0x7fdcec1442b0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fdce400c7d0 tx=0x7fdce400cc90 comp rx=0 tx=0).stop 2024-09-19T04:20:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcead9d640 1 -- 172.21.15.76:0/1613128039 reap_dead start 2024-09-19T04:20:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 shutdown_connections 2024-09-19T04:20:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 >> 172.21.15.76:0/1613128039 conn(0x7fdcec06bcc0 msgr2=0x7fdcec0b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 shutdown_connections 2024-09-19T04:20:43.081 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.079+0000 7fdcf0c17640 1 -- 172.21.15.76:0/1613128039 wait complete. 2024-09-19T04:20:43.095 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-19T04:20:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6299a46640 1 Processor -- start 2024-09-19T04:20:43.208 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6299a46640 1 -- start start 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6299a46640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6294156850 0x7f629415c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6299a46640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f62940cdc30 0x7f62940b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6299a46640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62940ce3a0 con 0x7f62940b6500 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6299a46640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f62940ce680 con 0x7f6294156850 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6299a46640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f62940ce960 con 0x7f62940cdc30 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f62937fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f62940b6500 0x7f629415fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43188/0 (socket says 172.21.15.76:43188) 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6292ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f6294156850 0x7f629415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f62937fe640 1 -- 172.21.15.76:0/883847151 learned_addr learned my addr 172.21.15.76:0/883847151 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f62927fc640 1 --2- 172.21.15.76:0/883847151 >> v2:172.21.15.116:3300/0 conn(0x7f62940cdc30 0x7f62940b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.210 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6291ffb640 1 -- 172.21.15.76:0/883847151 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099896200 0 0) 0x7f62940ce3a0 con 0x7f62940b6500 2024-09-19T04:20:43.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6291ffb640 1 -- 172.21.15.76:0/883847151 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6280003530 con 0x7f62940b6500 2024-09-19T04:20:43.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6292ffd640 1 -- 172.21.15.76:0/883847151 >> v2:172.21.15.116:3300/0 conn(0x7f62940cdc30 msgr2=0x7f62940b5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6292ffd640 1 --2- 172.21.15.76:0/883847151 >> v2:172.21.15.116:3300/0 conn(0x7f62940cdc30 0x7f62940b5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6292ffd640 1 -- 172.21.15.76:0/883847151 >> v1:172.21.15.76:6789/0 conn(0x7f62940b6500 legacy=0x7f629415fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.211 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.207+0000 7f6292ffd640 1 -- 172.21.15.76:0/883847151 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62940cec40 con 0x7f6294156850 2024-09-19T04:20:43.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6292ffd640 1 --2- 172.21.15.76:0/883847151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6294156850 0x7f629415c670 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f62940cd9f0 tx=0x7f628c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a5da77acbdfb80 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6291ffb640 1 -- 172.21.15.76:0/883847151 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0b0070 con 0x7f6294156850 2024-09-19T04:20:43.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6291ffb640 1 -- 172.21.15.76:0/883847151 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f628c0ab600 con 0x7f6294156850 2024-09-19T04:20:43.212 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6291ffb640 1 -- 172.21.15.76:0/883847151 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628c0ab9d0 con 0x7f6294156850 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 -- 172.21.15.76:0/883847151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6294156850 msgr2=0x7f629415c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 --2- 172.21.15.76:0/883847151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6294156850 0x7f629415c670 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f62940cd9f0 tx=0x7f628c01c480 comp rx=0 tx=0).stop 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 -- 172.21.15.76:0/883847151 shutdown_connections 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 --2- 172.21.15.76:0/883847151 >> v2:172.21.15.116:3300/0 conn(0x7f62940cdc30 0x7f62940b5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 --2- 172.21.15.76:0/883847151 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6294156850 0x7f629415c670 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 -- 172.21.15.76:0/883847151 >> 172.21.15.76:0/883847151 conn(0x7f62940736d0 msgr2=0x7f629406bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 -- 172.21.15.76:0/883847151 shutdown_connections 2024-09-19T04:20:43.213 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 -- 172.21.15.76:0/883847151 wait complete. 2024-09-19T04:20:43.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 Processor -- start 2024-09-19T04:20:43.214 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.211+0000 7f6299a46640 1 -- start start 2024-09-19T04:20:43.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62940b6500 0x7f62940cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62940cdc30 0x7f62941591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6294156850 0x7f6294157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6294169060 con 0x7f62940cdc30 2024-09-19T04:20:43.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f629406fa80 con 0x7f62940b6500 2024-09-19T04:20:43.216 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f62941691e0 con 0x7f6294156850 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62940cdc30 0x7f62941591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62940cdc30 0x7f62941591d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51932/0 (socket says 172.21.15.76:51932) 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 -- 172.21.15.76:0/1603019159 learned_addr learned my addr 172.21.15.76:0/1603019159 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6292ffd640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62940b6500 0x7f62940cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62937fe640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6294156850 0x7f6294157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 -- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6294156850 msgr2=0x7f6294157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6294156850 0x7f6294157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 -- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62940b6500 msgr2=0x7f62940cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62940b6500 0x7f62940cd670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 -- 172.21.15.76:0/1603019159 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6294156e30 con 0x7f62940cdc30 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f62927fc640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62940cdc30 0x7f62941591d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f627c00d5f0 tx=0x7f627c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.217 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f628bfff640 1 -- 172.21.15.76:0/1603019159 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c0125f0 con 0x7f62940cdc30 2024-09-19T04:20:43.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f628bfff640 1 -- 172.21.15.76:0/1603019159 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f627c019070 con 0x7f62940cdc30 2024-09-19T04:20:43.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62941570d0 con 0x7f62940cdc30 2024-09-19T04:20:43.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f628bfff640 1 -- 172.21.15.76:0/1603019159 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627c0129d0 con 0x7f62940cdc30 2024-09-19T04:20:43.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6294157690 con 0x7f62940cdc30 2024-09-19T04:20:43.218 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62940b5fc0 con 0x7f62940cdc30 2024-09-19T04:20:43.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f628bfff640 1 -- 172.21.15.76:0/1603019159 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f627c012b70 con 0x7f62940cdc30 2024-09-19T04:20:43.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f628bfff640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62740678c0 0x7f6274069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f628bfff640 1 -- 172.21.15.76:0/1603019159 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f627c0860d0 con 0x7f62940cdc30 2024-09-19T04:20:43.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6292ffd640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62740678c0 0x7f6274069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.219 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.215+0000 7f6292ffd640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62740678c0 0x7f6274069d80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f628c0ad000 tx=0x7f628c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.221 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.219+0000 7f628bfff640 1 -- 172.21.15.76:0/1603019159 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f627c056cd0 con 0x7f62940cdc30 2024-09-19T04:20:43.381 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.379+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f6294166e30 con 0x7f62740678c0 2024-09-19T04:20:43.382 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.379+0000 7f628bfff640 1 -- 172.21.15.76:0/1603019159 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6294166e30 con 0x7f62740678c0 2024-09-19T04:20:43.382 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 2 to deep-scrub 2024-09-19T04:20:43.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62740678c0 msgr2=0x7f6274069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62740678c0 0x7f6274069d80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f628c0ad000 tx=0x7f628c008040 comp rx=0 tx=0).stop 2024-09-19T04:20:43.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62940cdc30 msgr2=0x7f62941591d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62940cdc30 0x7f62941591d0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f627c00d5f0 tx=0x7f627c00dab0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 shutdown_connections 2024-09-19T04:20:43.384 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f62740678c0 0x7f6274069d80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6294156850 0x7f6294157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f62940cdc30 0x7f62941591d0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 --2- 172.21.15.76:0/1603019159 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f62940b6500 0x7f62940cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 >> 172.21.15.76:0/1603019159 conn(0x7f62940736d0 msgr2=0x7f62940b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 shutdown_connections 2024-09-19T04:20:43.385 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.383+0000 7f6299a46640 1 -- 172.21.15.76:0/1603019159 wait complete. 2024-09-19T04:20:43.400 INFO:tasks.ceph:Scrubbing osd.3 2024-09-19T04:20:43.400 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:43.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c6f56640 1 Processor -- start 2024-09-19T04:20:43.520 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c6f56640 1 -- start start 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c6f56640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd8c015f3a0 0x7fd8c0165f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c6f56640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c00aede0 con 0x7fd8c01520a0 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c6f56640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c015fa10 con 0x7fd8c0070020 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c6f56640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c0152480 con 0x7fd8c015f3a0 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c4cf2640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fd8c01520a0 0x7fd8c006f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43202/0 (socket says 172.21.15.76:43202) 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c4cf2640 1 -- 172.21.15.76:0/657681742 learned_addr learned my addr 172.21.15.76:0/657681742 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413450154 0 0) 0x7fd8c00aede0 con 0x7fd8c01520a0 2024-09-19T04:20:43.523 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c54f3640 1 --2- 172.21.15.76:0/657681742 >> v2:172.21.15.116:3300/0 conn(0x7fd8c015f3a0 0x7fd8c0165f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd8a8003530 con 0x7fd8c01520a0 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 447230238 0 0) 0x7fd8a8003530 con 0x7fd8c01520a0 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 >> v2:172.21.15.116:3300/0 conn(0x7fd8c015f3a0 msgr2=0x7fd8c0165f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 --2- 172.21.15.76:0/657681742 >> v2:172.21.15.116:3300/0 conn(0x7fd8c015f3a0 0x7fd8c0165f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 >> v1:172.21.15.116:6789/0 conn(0x7fd8c0070020 legacy=0x7fd8c015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c0074250 con 0x7fd8c01520a0 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fd8b0001a00 con 0x7fd8c01520a0 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 >> v1:172.21.15.76:6789/0 conn(0x7fd8c01520a0 legacy=0x7fd8c006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a80039c0 0x7fd8a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a8004810 0x7fd8a8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8a80051d0 0x7fd8a800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.524 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800e300 con 0x7fd8a80039c0 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd8a800e4b0 con 0x7fd8a8004810 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd8a800e6b0 con 0x7fd8a80051d0 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.519+0000 7fd8c4cf2640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a80039c0 0x7fd8a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8bffff640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a8004810 0x7fd8a8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c54f3640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8a80051d0 0x7fd8a800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c4cf2640 1 -- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8a80051d0 msgr2=0x7fd8a800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c4cf2640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8a80051d0 0x7fd8a800dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c4cf2640 1 -- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a8004810 msgr2=0x7fd8a8004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c4cf2640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a8004810 0x7fd8a8004c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c4cf2640 1 -- 172.21.15.76:0/657681742 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8a8003530 con 0x7fd8a80039c0 2024-09-19T04:20:43.525 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c54f3640 1 -- 172.21.15.76:0/657681742 reap_dead start 2024-09-19T04:20:43.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c4cf2640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a80039c0 0x7fd8a8003de0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000a860 tx=0x7fd8b001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d737e923d27a3da8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000a480 con 0x7fd8a80039c0 2024-09-19T04:20:43.528 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b00ad630 con 0x7fd8a80039c0 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8bf7fe640 1 -- 172.21.15.76:0/657681742 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b00adaa0 con 0x7fd8a80039c0 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c6f56640 1 -- 172.21.15.76:0/657681742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a80039c0 msgr2=0x7fd8a8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c6f56640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a80039c0 0x7fd8a8003de0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fd8b000a860 tx=0x7fd8b001d340 comp rx=0 tx=0).stop 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c6f56640 1 -- 172.21.15.76:0/657681742 shutdown_connections 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c6f56640 1 --2- 172.21.15.76:0/657681742 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8a80039c0 0x7fd8a8003de0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c6f56640 1 -- 172.21.15.76:0/657681742 >> 172.21.15.76:0/657681742 conn(0x7fd8c00b1f90 msgr2=0x7fd8c00b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c6f56640 1 -- 172.21.15.76:0/657681742 shutdown_connections 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.523+0000 7fd8c6f56640 1 -- 172.21.15.76:0/657681742 wait complete. 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 Processor -- start 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 -- start start 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a80039c0 0x7fd8c01f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.529 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8c01f3dd0 0x7fd8c014d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c014a2b0 0x7fd8c014a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0073d30 con 0x7fd8c014a2b0 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c0073a50 con 0x7fd8a80039c0 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c015f0a0 con 0x7fd8c01f3dd0 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c014a2b0 0x7fd8c014a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c014a2b0 0x7fd8c014a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51954/0 (socket says 172.21.15.76:51954) 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 -- 172.21.15.76:0/68867189 learned_addr learned my addr 172.21.15.76:0/68867189 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c4cf2640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a80039c0 0x7fd8c01f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8bffff640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8c01f3dd0 0x7fd8c014d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 -- 172.21.15.76:0/68867189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8c01f3dd0 msgr2=0x7fd8c014d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8c01f3dd0 0x7fd8c014d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.530 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 -- 172.21.15.76:0/68867189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a80039c0 msgr2=0x7fd8c01f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8a80039c0 0x7fd8c01f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 -- 172.21.15.76:0/68867189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c014ba50 con 0x7fd8c014a2b0 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8bffff640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fd8c01f3dd0 0x7fd8c014d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c54f3640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c014a2b0 0x7fd8c014a730 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00b5960 tx=0x7fd8b800e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b800f040 con 0x7fd8c014a2b0 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b8019070 con 0x7fd8c014a2b0 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8013850 con 0x7fd8c014a2b0 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c015c770 con 0x7fd8c014a2b0 2024-09-19T04:20:43.531 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c015ca10 con 0x7fd8c014a2b0 2024-09-19T04:20:43.532 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd8c0068e50 con 0x7fd8c014a2b0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fd8b80139f0 con 0x7fd8c014a2b0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.527+0000 7fd8bd7fa640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8ac0678c0 0x7fd8ac069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fd8b8085f10 con 0x7fd8c014a2b0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8c4cf2640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8ac0678c0 0x7fd8ac069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8bd7fa640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7fd8ac06dea0 0x7fd8ac0702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 --> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd8b800be50 con 0x7fd8ac06dea0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fd8b8086380 con 0x7fd8c014a2b0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8bffff640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7fd8ac06dea0 0x7fd8ac0702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8c4cf2640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8ac0678c0 0x7fd8ac069d80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00ad000 tx=0x7fd8b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8bffff640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7fd8ac06dea0 0x7fd8ac0702e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.533 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.531+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== osd.3 v2:172.21.15.76:6816/1627524440 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd8b800be50 con 0x7fd8ac06dea0 2024-09-19T04:20:43.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 --> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd8c00aede0 con 0x7fd8ac06dea0 2024-09-19T04:20:43.544 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8bd7fa640 1 -- 172.21.15.76:0/68867189 <== osd.3 v2:172.21.15.76:6816/1627524440 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd8c00aede0 con 0x7fd8ac06dea0 2024-09-19T04:20:43.544 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:43.544 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:43.544 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:43.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7fd8ac06dea0 msgr2=0x7fd8ac0702e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6816/1627524440,v1:172.21.15.76:6817/1627524440] conn(0x7fd8ac06dea0 0x7fd8ac0702e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8ac0678c0 msgr2=0x7fd8ac069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.545 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fd8ac0678c0 0x7fd8ac069d80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00ad000 tx=0x7fd8b0009040 comp rx=0 tx=0).stop 2024-09-19T04:20:43.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c014a2b0 msgr2=0x7fd8c014a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 --2- 172.21.15.76:0/68867189 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fd8c014a2b0 0x7fd8c014a730 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7fd8c00b5960 tx=0x7fd8b800e830 comp rx=0 tx=0).stop 2024-09-19T04:20:43.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c54f3640 1 -- 172.21.15.76:0/68867189 reap_dead start 2024-09-19T04:20:43.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 shutdown_connections 2024-09-19T04:20:43.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 >> 172.21.15.76:0/68867189 conn(0x7fd8c00b1f90 msgr2=0x7fd8c00b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 shutdown_connections 2024-09-19T04:20:43.546 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.543+0000 7fd8c6f56640 1 -- 172.21.15.76:0/68867189 wait complete. 2024-09-19T04:20:43.559 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-19T04:20:43.663 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 Processor -- start 2024-09-19T04:20:43.664 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 -- start start 2024-09-19T04:20:43.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f84a815bec0 0x7f84a80ad200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a815c2a0 0x7f84a80aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f84a80af2a0 0x7f84a815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.665 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f84a80b3290 con 0x7f84a815bec0 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f84a80b3570 con 0x7f84a80af2a0 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84ad734640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f84a80b3850 con 0x7f84a815c2a0 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f84a815bec0 0x7f84a80ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f84a815bec0 0x7f84a80ad200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51958/0 (socket says 172.21.15.76:51958) 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 -- 172.21.15.76:0/1483020051 learned_addr learned my addr 172.21.15.76:0/1483020051 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f849e7fc640 1 --2- 172.21.15.76:0/1483020051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a815c2a0 0x7f84a80aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 -- 172.21.15.76:0/1483020051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a815c2a0 msgr2=0x7f84a80aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 --2- 172.21.15.76:0/1483020051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a815c2a0 0x7f84a80aed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 -- 172.21.15.76:0/1483020051 >> v2:172.21.15.116:3300/0 conn(0x7f84a80af2a0 msgr2=0x7f84a815fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a77fe640 1 --2- 172.21.15.76:0/1483020051 >> v2:172.21.15.116:3300/0 conn(0x7f84a80af2a0 0x7f84a815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 --2- 172.21.15.76:0/1483020051 >> v2:172.21.15.116:3300/0 conn(0x7f84a80af2a0 0x7f84a815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.666 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.663+0000 7f84a6ffd640 1 -- 172.21.15.76:0/1483020051 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a80b3b30 con 0x7f84a815bec0 2024-09-19T04:20:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84a6ffd640 1 --2- 172.21.15.76:0/1483020051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a80ad200 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f84a806dc90 tx=0x7f849801c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c8e1c7eb9e6c1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84a67fc640 1 -- 172.21.15.76:0/1483020051 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980b0070 con 0x7f84a815bec0 2024-09-19T04:20:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84a67fc640 1 -- 172.21.15.76:0/1483020051 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84980ab600 con 0x7f84a815bec0 2024-09-19T04:20:43.668 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84a67fc640 1 -- 172.21.15.76:0/1483020051 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84980ab9d0 con 0x7f84a815bec0 2024-09-19T04:20:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 -- 172.21.15.76:0/1483020051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 msgr2=0x7f84a80ad200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 --2- 172.21.15.76:0/1483020051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a80ad200 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f84a806dc90 tx=0x7f849801c230 comp rx=0 tx=0).stop 2024-09-19T04:20:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 -- 172.21.15.76:0/1483020051 shutdown_connections 2024-09-19T04:20:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 --2- 172.21.15.76:0/1483020051 >> v2:172.21.15.116:3300/0 conn(0x7f84a80af2a0 0x7f84a815fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 --2- 172.21.15.76:0/1483020051 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a815c2a0 0x7f84a80aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 --2- 172.21.15.76:0/1483020051 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a80ad200 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 -- 172.21.15.76:0/1483020051 >> 172.21.15.76:0/1483020051 conn(0x7f84a80b1f90 msgr2=0x7f84a80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 -- 172.21.15.76:0/1483020051 shutdown_connections 2024-09-19T04:20:43.670 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 -- 172.21.15.76:0/1483020051 wait complete. 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 Processor -- start 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.667+0000 7f84ad734640 1 -- start start 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a80af2a0 0x7f84a815cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a815a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84a815c2a0 0x7f84a81f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f84a80af6b0 con 0x7f84a815bec0 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f84a80ad740 con 0x7f84a815c2a0 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f84a806e000 con 0x7f84a80af2a0 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a815a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a815a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51970/0 (socket says 172.21.15.76:51970) 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 -- 172.21.15.76:0/4083432398 learned_addr learned my addr 172.21.15.76:0/4083432398 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:43.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84a6ffd640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a80af2a0 0x7f84a815cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84a77fe640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84a815c2a0 0x7f84a81f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 -- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a80af2a0 msgr2=0x7f84a815cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a80af2a0 0x7f84a815cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 -- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84a815c2a0 msgr2=0x7f84a81f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84a815c2a0 0x7f84a81f7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 -- 172.21.15.76:0/4083432398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84a8159920 con 0x7f84a815bec0 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84a6ffd640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a80af2a0 0x7f84a815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849e7fc640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a815a9c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f849000c7f0 tx=0x7f849000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849ffff640 1 -- 172.21.15.76:0/4083432398 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490015070 con 0x7f84a815bec0 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849ffff640 1 -- 172.21.15.76:0/4083432398 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8490004240 con 0x7f84a815bec0 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84a8159c00 con 0x7f84a815bec0 2024-09-19T04:20:43.673 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849ffff640 1 -- 172.21.15.76:0/4083432398 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8490003620 con 0x7f84a815bec0 2024-09-19T04:20:43.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84a815a1c0 con 0x7f84a815bec0 2024-09-19T04:20:43.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84a815a6b0 con 0x7f84a815bec0 2024-09-19T04:20:43.674 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849ffff640 1 -- 172.21.15.76:0/4083432398 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f84900039d0 con 0x7f84a815bec0 2024-09-19T04:20:43.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849ffff640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84880678c0 0x7f8488069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f849ffff640 1 -- 172.21.15.76:0/4083432398 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f8490085b70 con 0x7f84a815bec0 2024-09-19T04:20:43.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84a6ffd640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84880678c0 0x7f8488069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.675 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.671+0000 7f84a6ffd640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84880678c0 0x7f8488069d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f84980ad000 tx=0x7f8498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.677 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.675+0000 7f849ffff640 1 -- 172.21.15.76:0/4083432398 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f8490056800 con 0x7f84a815bec0 2024-09-19T04:20:43.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.823+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f84a80ad200 con 0x7f84880678c0 2024-09-19T04:20:43.826 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f849ffff640 1 -- 172.21.15.76:0/4083432398 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f84a80ad200 con 0x7f84880678c0 2024-09-19T04:20:43.827 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 3 to deep-scrub 2024-09-19T04:20:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84880678c0 msgr2=0x7f8488069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84880678c0 0x7f8488069d80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f84980ad000 tx=0x7f8498008040 comp rx=0 tx=0).stop 2024-09-19T04:20:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 msgr2=0x7f84a815a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a815a9c0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f849000c7f0 tx=0x7f849000ccb0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 shutdown_connections 2024-09-19T04:20:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f84880678c0 0x7f8488069d80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.829 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84a815c2a0 0x7f84a81f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f84a815bec0 0x7f84a815a9c0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 --2- 172.21.15.76:0/4083432398 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f84a80af2a0 0x7f84a815cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 >> 172.21.15.76:0/4083432398 conn(0x7f84a80b1f90 msgr2=0x7f84a8166030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 shutdown_connections 2024-09-19T04:20:43.830 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.827+0000 7f84ad734640 1 -- 172.21.15.76:0/4083432398 wait complete. 2024-09-19T04:20:43.844 INFO:tasks.ceph:Scrubbing osd.4 2024-09-19T04:20:43.845 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:43.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd94d35640 1 Processor -- start 2024-09-19T04:20:43.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd94d35640 1 -- start start 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd94d35640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efd901520a0 0x7efd9006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd94d35640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd9015f3a0 0x7efd900adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd94d35640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd90070250 con 0x7efd9006fe50 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd94d35640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7efd90152480 con 0x7efd901520a0 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd94d35640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efd900aede0 con 0x7efd9015f3a0 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd85d9b640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7efd9006fe50 0x7efd9015ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43218/0 (socket says 172.21.15.76:43218) 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd85d9b640 1 -- 172.21.15.76:0/2563722315 learned_addr learned my addr 172.21.15.76:0/2563722315 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1236698994 0 0) 0x7efd90070250 con 0x7efd9006fe50 2024-09-19T04:20:43.959 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd9015f3a0 0x7efd900adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd7c003530 con 0x7efd9006fe50 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.955+0000 7efd8e59c640 1 --2- 172.21.15.76:0/2563722315 >> v2:172.21.15.116:3300/0 conn(0x7efd901520a0 0x7efd9006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2977208416 0 0) 0x7efd7c003530 con 0x7efd9006fe50 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd9015f3a0 msgr2=0x7efd900adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd9015f3a0 0x7efd900adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 >> v2:172.21.15.116:3300/0 conn(0x7efd901520a0 msgr2=0x7efd9006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 --2- 172.21.15.76:0/2563722315 >> v2:172.21.15.116:3300/0 conn(0x7efd901520a0 0x7efd9006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9015fa10 con 0x7efd9006fe50 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7efd78001680 con 0x7efd9006fe50 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 >> v1:172.21.15.76:6789/0 conn(0x7efd9006fe50 legacy=0x7efd9015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c0039c0 0x7efd7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.960 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd9015f3a0 0x7efd7c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c004d50 0x7efd7c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd900aede0 con 0x7efd7c004d50 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c00dbe0 con 0x7efd7c0039c0 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efd7c00dd90 con 0x7efd9015f3a0 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8e59c640 1 --2- 172.21.15.76:0/2563722315 >> v2:172.21.15.116:3300/0 conn(0x7efd901520a0 0x7efd9006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c004d50 0x7efd7c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8e59c640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c0039c0 0x7efd7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 -- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd9015f3a0 msgr2=0x7efd7c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd9015f3a0 0x7efd7c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 -- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c0039c0 msgr2=0x7efd7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c0039c0 0x7efd7c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.961 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 -- 172.21.15.76:0/2563722315 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c003530 con 0x7efd7c004d50 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c004d50 0x7efd7c0051d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7efd88009fc0 tx=0x7efd8801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f10563bb6dbbdfd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd880b2070 con 0x7efd7c004d50 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd880a5cd0 con 0x7efd7c004d50 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8dd9b640 1 -- 172.21.15.76:0/2563722315 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88009c10 con 0x7efd7c004d50 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd94d35640 1 -- 172.21.15.76:0/2563722315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c004d50 msgr2=0x7efd7c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd94d35640 1 --2- 172.21.15.76:0/2563722315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd7c004d50 0x7efd7c0051d0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7efd88009fc0 tx=0x7efd8801ccc0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd8ed9d640 1 -- 172.21.15.76:0/2563722315 reap_dead start 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd94d35640 1 -- 172.21.15.76:0/2563722315 shutdown_connections 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.959+0000 7efd94d35640 1 -- 172.21.15.76:0/2563722315 >> 172.21.15.76:0/2563722315 conn(0x7efd900b1f90 msgr2=0x7efd900b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- 172.21.15.76:0/2563722315 shutdown_connections 2024-09-19T04:20:43.963 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- 172.21.15.76:0/2563722315 wait complete. 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 Processor -- start 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- start start 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c004d50 0x7efd900afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd9014c7f0 0x7efd9015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd900ccc30 0x7efd900cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7efd90073770 con 0x7efd9014c7f0 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7efd90073490 con 0x7efd7c004d50 2024-09-19T04:20:43.966 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7efd90073a50 con 0x7efd900ccc30 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd9014c7f0 0x7efd9015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd9014c7f0 0x7efd9015df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51984/0 (socket says 172.21.15.76:51984) 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 -- 172.21.15.76:0/1668396382 learned_addr learned my addr 172.21.15.76:0/1668396382 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd900ccc30 0x7efd900cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd8e59c640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c004d50 0x7efd900afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 -- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd900ccc30 msgr2=0x7efd900cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd900ccc30 0x7efd900cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 -- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c004d50 msgr2=0x7efd900afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c004d50 0x7efd900afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 -- 172.21.15.76:0/1668396382 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd900b03e0 con 0x7efd9014c7f0 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7efd900ccc30 0x7efd900cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:43.967 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd85d9b640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd9014c7f0 0x7efd9015df40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c700 tx=0x7efd7800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd78018070 con 0x7efd9014c7f0 2024-09-19T04:20:43.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd780072f0 con 0x7efd9014c7f0 2024-09-19T04:20:43.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd900b06c0 con 0x7efd9014c7f0 2024-09-19T04:20:43.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd780047f0 con 0x7efd9014c7f0 2024-09-19T04:20:43.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd900b0c80 con 0x7efd9014c7f0 2024-09-19T04:20:43.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.963+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efd90073d30 con 0x7efd9014c7f0 2024-09-19T04:20:43.968 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7efd7801b070 con 0x7efd9014c7f0 2024-09-19T04:20:43.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd877fe640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd700678f0 0x7efd70069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7efd78084ff0 con 0x7efd9014c7f0 2024-09-19T04:20:43.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd8e59c640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd700678f0 0x7efd70069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd877fe640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7efd7006ded0 0x7efd70070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:43.969 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 --> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd7801ee50 con 0x7efd7006ded0 2024-09-19T04:20:43.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7efd7808c050 con 0x7efd9014c7f0 2024-09-19T04:20:43.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd8e59c640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd700678f0 0x7efd70069db0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efd8000b970 tx=0x7efd80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7efd7006ded0 0x7efd70070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:43.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd8ed9d640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7efd7006ded0 0x7efd70070310 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:43.970 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.967+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== osd.4 v2:172.21.15.116:6800/3265669393 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efd7801ee50 con 0x7efd7006ded0 2024-09-19T04:20:43.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 --> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7efd9006e540 con 0x7efd7006ded0 2024-09-19T04:20:43.983 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd877fe640 1 -- 172.21.15.76:0/1668396382 <== osd.4 v2:172.21.15.116:6800/3265669393 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7efd9006e540 con 0x7efd7006ded0 2024-09-19T04:20:43.984 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:43.984 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:43.984 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7efd7006ded0 msgr2=0x7efd70070310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.116:6800/3265669393,v1:172.21.15.116:6801/3265669393] conn(0x7efd7006ded0 0x7efd70070310 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd700678f0 msgr2=0x7efd70069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7efd700678f0 0x7efd70069db0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efd8000b970 tx=0x7efd80008040 comp rx=0 tx=0).stop 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd9014c7f0 msgr2=0x7efd9015df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 --2- 172.21.15.76:0/1668396382 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7efd9014c7f0 0x7efd9015df40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c700 tx=0x7efd7800cbc0 comp rx=0 tx=0).stop 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd8ed9d640 1 -- 172.21.15.76:0/1668396382 reap_dead start 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 shutdown_connections 2024-09-19T04:20:43.985 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 >> 172.21.15.76:0/1668396382 conn(0x7efd900b1f90 msgr2=0x7efd901500d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:43.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 shutdown_connections 2024-09-19T04:20:43.986 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:43.983+0000 7efd94d35640 1 -- 172.21.15.76:0/1668396382 wait complete. 2024-09-19T04:20:43.998 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-19T04:20:44.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481f097640 1 Processor -- start 2024-09-19T04:20:44.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481f097640 1 -- start start 2024-09-19T04:20:44.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481f097640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4818073170 0x7f481806e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481f097640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481806ec60 con 0x7f48181587a0 2024-09-19T04:20:44.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481f097640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481806ef40 con 0x7f48180b39a0 2024-09-19T04:20:44.108 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481f097640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f481806f870 con 0x7f4818073170 2024-09-19T04:20:44.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f48181587a0 0x7f48180b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43226/0 (socket says 172.21.15.76:43226) 2024-09-19T04:20:44.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 -- 172.21.15.76:0/1946988463 learned_addr learned my addr 172.21.15.76:0/1946988463 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:44.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481d634640 1 --2- 172.21.15.76:0/1946988463 >> v2:172.21.15.116:3300/0 conn(0x7f4818073170 0x7f481806e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2674853650 0 0) 0x7f481806ec60 con 0x7f48181587a0 2024-09-19T04:20:44.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4800003530 con 0x7f48181587a0 2024-09-19T04:20:44.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2753936258 0 0) 0x7f4800003530 con 0x7f48181587a0 2024-09-19T04:20:44.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 >> v2:172.21.15.116:3300/0 conn(0x7f4818073170 msgr2=0x7f481806e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 --2- 172.21.15.76:0/1946988463 >> v2:172.21.15.116:3300/0 conn(0x7f4818073170 0x7f481806e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 >> v1:172.21.15.116:6789/0 conn(0x7f48180b39a0 legacy=0x7f481806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481806f220 con 0x7f48181587a0 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f4810001680 con 0x7f48181587a0 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 >> v1:172.21.15.76:6789/0 conn(0x7f48181587a0 legacy=0x7f48180b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f4800003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4800004810 0x7f4800004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000051d0 0x7f480000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f481806f870 con 0x7f48000039c0 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f480000c0a0 con 0x7f48000051d0 2024-09-19T04:20:44.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f480000c2a0 con 0x7f4800004810 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f4800003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f4817fff640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4800004810 0x7f4800004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481d634640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000051d0 0x7f480000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 -- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4800004810 msgr2=0x7f4800004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4800004810 0x7f4800004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 -- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000051d0 msgr2=0x7f480000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48000051d0 0x7f480000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481ce33640 1 -- 172.21.15.76:0/1946988463 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4800003530 con 0x7f48000039c0 2024-09-19T04:20:44.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.107+0000 7f481d634640 1 -- 172.21.15.76:0/1946988463 reap_dead start 2024-09-19T04:20:44.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481ce33640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f4800003e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f481000c1c0 tx=0x7f481001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5665675f5d371f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810007d00 con 0x7f48000039c0 2024-09-19T04:20:44.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100a9020 con 0x7f48000039c0 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f48177fe640 1 -- 172.21.15.76:0/1946988463 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810005b00 con 0x7f48000039c0 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 -- 172.21.15.76:0/1946988463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 msgr2=0x7f4800003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f4800003e00 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f481000c1c0 tx=0x7f481001cbf0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 -- 172.21.15.76:0/1946988463 shutdown_connections 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 --2- 172.21.15.76:0/1946988463 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f4800003e00 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 -- 172.21.15.76:0/1946988463 >> 172.21.15.76:0/1946988463 conn(0x7f48180b1f90 msgr2=0x7f48180b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 -- 172.21.15.76:0/1946988463 shutdown_connections 2024-09-19T04:20:44.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 -- 172.21.15.76:0/1946988463 wait complete. 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 Processor -- start 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.111+0000 7f481f097640 1 -- start start 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f48181f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48181fa8c0 0x7f48181fad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4818153eb0 0x7f4818154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f481806e900 con 0x7f48000039c0 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f481806e780 con 0x7f4818153eb0 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4818072160 con 0x7f48181fa8c0 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f48181f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f48181f45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51992/0 (socket says 172.21.15.76:51992) 2024-09-19T04:20:44.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 -- 172.21.15.76:0/3823904175 learned_addr learned my addr 172.21.15.76:0/3823904175 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 -- 172.21.15.76:0/3823904175 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48181fa8c0 msgr2=0x7f48181fad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48181fa8c0 0x7f48181fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 -- 172.21.15.76:0/3823904175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4818153eb0 msgr2=0x7f4818154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4818153eb0 0x7f4818154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 -- 172.21.15.76:0/3823904175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4818154b90 con 0x7f48000039c0 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481ce33640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f48181f45c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4810000c10 tx=0x7f4800005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f48157fa640 1 -- 172.21.15.76:0/3823904175 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48100bb070 con 0x7f48000039c0 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f48157fa640 1 -- 172.21.15.76:0/3823904175 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48100a5830 con 0x7f48000039c0 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4818154e70 con 0x7f48000039c0 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f48157fa640 1 -- 172.21.15.76:0/3823904175 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810001c40 con 0x7f48000039c0 2024-09-19T04:20:44.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48181f89c0 con 0x7f48000039c0 2024-09-19T04:20:44.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48180724f0 con 0x7f48000039c0 2024-09-19T04:20:44.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f48157fa640 1 -- 172.21.15.76:0/3823904175 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f48100be070 con 0x7f48000039c0 2024-09-19T04:20:44.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f48157fa640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48040678f0 0x7f4804069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f4817fff640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48040678f0 0x7f4804069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f48157fa640 1 -- 172.21.15.76:0/3823904175 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f481011ec70 con 0x7f48000039c0 2024-09-19T04:20:44.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.115+0000 7f4817fff640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48040678f0 0x7f4804069db0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f48180d1140 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.122 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.119+0000 7f48157fa640 1 -- 172.21.15.76:0/3823904175 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f4810016970 con 0x7f48000039c0 2024-09-19T04:20:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.271+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f4818070080 con 0x7f48040678f0 2024-09-19T04:20:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.271+0000 7f48157fa640 1 -- 172.21.15.76:0/3823904175 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4818070080 con 0x7f48040678f0 2024-09-19T04:20:44.273 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 4 to deep-scrub 2024-09-19T04:20:44.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48040678f0 msgr2=0x7f4804069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48040678f0 0x7f4804069db0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f48180d1140 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-09-19T04:20:44.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 msgr2=0x7f48181f45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f48181f45c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4810000c10 tx=0x7f4800005840 comp rx=0 tx=0).stop 2024-09-19T04:20:44.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 shutdown_connections 2024-09-19T04:20:44.275 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f48040678f0 0x7f4804069db0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4818153eb0 0x7f4818154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f48181fa8c0 0x7f48181fad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 --2- 172.21.15.76:0/3823904175 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f48000039c0 0x7f48181f45c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 >> 172.21.15.76:0/3823904175 conn(0x7f48180b1f90 msgr2=0x7f48180b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 shutdown_connections 2024-09-19T04:20:44.276 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.275+0000 7f481f097640 1 -- 172.21.15.76:0/3823904175 wait complete. 2024-09-19T04:20:44.290 INFO:tasks.ceph:Scrubbing osd.5 2024-09-19T04:20:44.290 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42797c3640 1 Processor -- start 2024-09-19T04:20:44.412 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42797c3640 1 -- start start 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42797c3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 0x7f427406e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42797c3640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f427406ee50 con 0x7f42740b3d50 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42797c3640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f427406f130 con 0x7f4274073e60 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42797c3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f427406f410 con 0x7f427406fef0 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42737fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f42740b3d50 0x7f4274072610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43232/0 (socket says 172.21.15.76:43232) 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42737fe640 1 -- 172.21.15.76:0/2668290843 learned_addr learned my addr 172.21.15.76:0/2668290843 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f426a7fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 0x7f427406e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 904041222 0 0) 0x7f427406ee50 con 0x7f42740b3d50 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4260003530 con 0x7f42740b3d50 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1926208633 0 0) 0x7f4260003530 con 0x7f42740b3d50 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 msgr2=0x7f427406e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 0x7f427406e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 >> v1:172.21.15.116:6789/0 conn(0x7f4274073e60 legacy=0x7f427415b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f427406f6f0 con 0x7f42740b3d50 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f426c001a00 con 0x7f42740b3d50 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 >> v1:172.21.15.76:6789/0 conn(0x7f42740b3d50 legacy=0x7f4274072610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42600039f0 0x7f4260003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4260004810 0x7f4260004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 0x7f4260005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.416 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f427406f410 con 0x7f4260004810 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f426000bb60 con 0x7f42600039f0 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f426000bd60 con 0x7f427406fef0 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f426a7fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4260004810 0x7f4260004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f42737fe640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 0x7f4260005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f4272ffd640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42600039f0 0x7f4260003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f426a7fc640 1 -- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 msgr2=0x7f4260005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f426a7fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f427406fef0 0x7f4260005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f426a7fc640 1 -- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42600039f0 msgr2=0x7f4260003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f426a7fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42600039f0 0x7f4260003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.411+0000 7f426a7fc640 1 -- 172.21.15.76:0/2668290843 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4260003530 con 0x7f4260004810 2024-09-19T04:20:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f426a7fc640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4260004810 0x7f4260004c90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f425c00aae0 tx=0x7f425c01dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9930dbd848d5088 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0b3070 con 0x7f4260004810 2024-09-19T04:20:44.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c0a6da0 con 0x7f4260004810 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42727fc640 1 -- 172.21.15.76:0/2668290843 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c00b040 con 0x7f4260004810 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42797c3640 1 -- 172.21.15.76:0/2668290843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4260004810 msgr2=0x7f4260004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42797c3640 1 --2- 172.21.15.76:0/2668290843 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4260004810 0x7f4260004c90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f425c00aae0 tx=0x7f425c01dfc0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42737fe640 1 -- 172.21.15.76:0/2668290843 reap_dead start 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42797c3640 1 -- 172.21.15.76:0/2668290843 shutdown_connections 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42797c3640 1 -- 172.21.15.76:0/2668290843 >> 172.21.15.76:0/2668290843 conn(0x7f42740b1f90 msgr2=0x7f42740b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42797c3640 1 -- 172.21.15.76:0/2668290843 shutdown_connections 2024-09-19T04:20:44.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.415+0000 7f42797c3640 1 -- 172.21.15.76:0/2668290843 wait complete. 2024-09-19T04:20:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 Processor -- start 2024-09-19T04:20:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 -- start start 2024-09-19T04:20:44.421 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4260004810 0x7f4274148a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f427415d480 0x7f427415d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42741f5290 0x7f42741f5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f427406e380 con 0x7f42741f5290 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f427406e200 con 0x7f427415d480 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f427406e500 con 0x7f4260004810 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42741f5290 0x7f42741f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42741f5290 0x7f42741f5710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52014/0 (socket says 172.21.15.76:52014) 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 -- 172.21.15.76:0/1339731675 learned_addr learned my addr 172.21.15.76:0/1339731675 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f4272ffd640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4260004810 0x7f4274148a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f426a7fc640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f427415d480 0x7f427415d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 -- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4260004810 msgr2=0x7f4274148a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.422 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4260004810 0x7f4274148a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 -- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f427415d480 msgr2=0x7f427415d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f427415d480 0x7f427415d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 -- 172.21.15.76:0/1339731675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42741f5fc0 con 0x7f42741f5290 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42737fe640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42741f5290 0x7f42741f5710 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f42740afd80 tx=0x7f426c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c007ae0 con 0x7f42741f5290 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f426c00ee20 con 0x7f42741f5290 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4274156a10 con 0x7f42741f5290 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426c014670 con 0x7f42741f5290 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4274156fd0 con 0x7f42741f5290 2024-09-19T04:20:44.423 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.419+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f427406e8e0 con 0x7f42741f5290 2024-09-19T04:20:44.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f426c014810 con 0x7f42741f5290 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426bfff640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42540678f0 0x7f4254069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f426c086730 con 0x7f42741f5290 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f4272ffd640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42540678f0 0x7f4254069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426bfff640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7f425406ded0 0x7f4254070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 --> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f426c01fec0 con 0x7f425406ded0 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f426c086bc0 con 0x7f42741f5290 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f4272ffd640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42540678f0 0x7f4254069db0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4264005490 tx=0x7f4264009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426a7fc640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7f425406ded0 0x7f4254070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426a7fc640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7f425406ded0 0x7f4254070310 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.423+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== osd.5 v2:172.21.15.116:6808/3134608166 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f426c01fec0 con 0x7f425406ded0 2024-09-19T04:20:44.437 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.435+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 --> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f42740723f0 con 0x7f425406ded0 2024-09-19T04:20:44.438 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f426bfff640 1 -- 172.21.15.76:0/1339731675 <== osd.5 v2:172.21.15.116:6808/3134608166 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f42740723f0 con 0x7f425406ded0 2024-09-19T04:20:44.439 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:44.439 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:44.439 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7f425406ded0 msgr2=0x7f4254070310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.116:6808/3134608166,v1:172.21.15.116:6809/3134608166] conn(0x7f425406ded0 0x7f4254070310 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42540678f0 msgr2=0x7f4254069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f42540678f0 0x7f4254069db0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4264005490 tx=0x7f4264009040 comp rx=0 tx=0).stop 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42741f5290 msgr2=0x7f42741f5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 --2- 172.21.15.76:0/1339731675 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f42741f5290 0x7f42741f5710 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f42740afd80 tx=0x7f426c00e530 comp rx=0 tx=0).stop 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42737fe640 1 -- 172.21.15.76:0/1339731675 reap_dead start 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 shutdown_connections 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 >> 172.21.15.76:0/1339731675 conn(0x7f42740b1f90 msgr2=0x7f42740b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.440 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 shutdown_connections 2024-09-19T04:20:44.441 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.439+0000 7f42797c3640 1 -- 172.21.15.76:0/1339731675 wait complete. 2024-09-19T04:20:44.453 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-19T04:20:44.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac221b6640 1 Processor -- start 2024-09-19T04:20:44.560 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac221b6640 1 -- start start 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac221b6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 0x7fac1c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac221b6640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1c0aede0 con 0x7fac1c15f1d0 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac221b6640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac1c15fa10 con 0x7fac1c1520a0 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac221b6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c152480 con 0x7fac1c070020 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fac1c15f1d0 0x7fac1c0adc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43242/0 (socket says 172.21.15.76:43242) 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 -- 172.21.15.76:0/458169628 learned_addr learned my addr 172.21.15.76:0/458169628 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1affd640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 0x7fac1c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235376664 0 0) 0x7fac1c0aede0 con 0x7fac1c15f1d0 2024-09-19T04:20:44.563 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac04005740 con 0x7fac1c15f1d0 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4124717462 0 0) 0x7fac04005740 con 0x7fac1c15f1d0 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 msgr2=0x7fac1c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 0x7fac1c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 >> v1:172.21.15.116:6789/0 conn(0x7fac1c1520a0 legacy=0x7fac1c06f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c074250 con 0x7fac1c15f1d0 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fac14001a00 con 0x7fac1c15f1d0 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 >> v1:172.21.15.76:6789/0 conn(0x7fac1c15f1d0 legacy=0x7fac1c0adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac04005c00 0x7fac04006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 0x7fac04006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac04006f60 0x7fac040073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac0400dc90 con 0x7fac04006f60 2024-09-19T04:20:44.564 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fac0400de10 con 0x7fac04005c00 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fac0400e010 con 0x7fac1c070020 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac04006f60 0x7fac040073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1b7fe640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac04005c00 0x7fac04006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1affd640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 0x7fac04006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 -- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 msgr2=0x7fac04006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c070020 0x7fac04006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 -- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac04005c00 msgr2=0x7fac04006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac04005c00 0x7fac04006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.565 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.559+0000 7fac1bfff640 1 -- 172.21.15.76:0/458169628 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac04005740 con 0x7fac04006f60 2024-09-19T04:20:44.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac1bfff640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac04006f60 0x7fac040073e0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fac1400a830 tx=0x7fac1401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976c700ec2b27834 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1400a480 con 0x7fac04006f60 2024-09-19T04:20:44.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac140ad630 con 0x7fac04006f60 2024-09-19T04:20:44.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac1a7fc640 1 -- 172.21.15.76:0/458169628 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140ada60 con 0x7fac04006f60 2024-09-19T04:20:44.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 -- 172.21.15.76:0/458169628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac04006f60 msgr2=0x7fac040073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 --2- 172.21.15.76:0/458169628 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac04006f60 0x7fac040073e0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fac1400a830 tx=0x7fac1401d360 comp rx=0 tx=0).stop 2024-09-19T04:20:44.568 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac1bfff640 1 -- 172.21.15.76:0/458169628 reap_dead start 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 -- 172.21.15.76:0/458169628 shutdown_connections 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 -- 172.21.15.76:0/458169628 >> 172.21.15.76:0/458169628 conn(0x7fac1c0b1f90 msgr2=0x7fac1c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 -- 172.21.15.76:0/458169628 shutdown_connections 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 -- 172.21.15.76:0/458169628 wait complete. 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 Processor -- start 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.563+0000 7fac221b6640 1 -- start start 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c14c5c0 0x7fac1c150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c1f4270 0x7fac1c06f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac1c153320 0x7fac1c06f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c073a50 con 0x7fac1c14c5c0 2024-09-19T04:20:44.569 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c073770 con 0x7fac1c153320 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c073d30 con 0x7fac1c1f4270 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c14c5c0 0x7fac1c150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c14c5c0 0x7fac1c150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52030/0 (socket says 172.21.15.76:52030) 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 -- 172.21.15.76:0/1731825326 learned_addr learned my addr 172.21.15.76:0/1731825326 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1affd640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c1f4270 0x7fac1c06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 -- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c1f4270 msgr2=0x7fac1c06f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c1f4270 0x7fac1c06f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 -- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac1c153320 msgr2=0x7fac1c06f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1bfff640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac1c153320 0x7fac1c06f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac1c153320 0x7fac1c06f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 -- 172.21.15.76:0/1731825326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c1f4780 con 0x7fac1c14c5c0 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1affd640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c1f4270 0x7fac1c06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:20:44.570 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1b7fe640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c14c5c0 0x7fac1c150590 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e550 tx=0x7fac0c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac03fff640 1 -- 172.21.15.76:0/1731825326 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c0052e0 con 0x7fac1c14c5c0 2024-09-19T04:20:44.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac03fff640 1 -- 172.21.15.76:0/1731825326 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac0c018070 con 0x7fac1c14c5c0 2024-09-19T04:20:44.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c1f4a60 con 0x7fac1c14c5c0 2024-09-19T04:20:44.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac03fff640 1 -- 172.21.15.76:0/1731825326 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0c003820 con 0x7fac1c14c5c0 2024-09-19T04:20:44.571 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1f5020 con 0x7fac1c14c5c0 2024-09-19T04:20:44.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c15f680 con 0x7fac1c14c5c0 2024-09-19T04:20:44.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac03fff640 1 -- 172.21.15.76:0/1731825326 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fac0c0039c0 con 0x7fac1c14c5c0 2024-09-19T04:20:44.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac03fff640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fac080678c0 0x7fac08069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac03fff640 1 -- 172.21.15.76:0/1731825326 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fac0c085950 con 0x7fac1c14c5c0 2024-09-19T04:20:44.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1affd640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fac080678c0 0x7fac08069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.572 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.567+0000 7fac1affd640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fac080678c0 0x7fac08069d80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fac1c1f46b0 tx=0x7fac10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.574 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.571+0000 7fac03fff640 1 -- 172.21.15.76:0/1731825326 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fac0c0564d0 con 0x7fac1c14c5c0 2024-09-19T04:20:44.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.719+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fac1c0ce310 con 0x7fac080678c0 2024-09-19T04:20:44.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac03fff640 1 -- 172.21.15.76:0/1731825326 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fac1c0ce310 con 0x7fac080678c0 2024-09-19T04:20:44.723 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 5 to deep-scrub 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fac080678c0 msgr2=0x7fac08069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fac080678c0 0x7fac08069d80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fac1c1f46b0 tx=0x7fac10008040 comp rx=0 tx=0).stop 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c14c5c0 msgr2=0x7fac1c150590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c14c5c0 0x7fac1c150590 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fac0c00e550 tx=0x7fac0c00ea10 comp rx=0 tx=0).stop 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 shutdown_connections 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fac080678c0 0x7fac08069d80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac1c153320 0x7fac1c06f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fac1c1f4270 0x7fac1c06f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 --2- 172.21.15.76:0/1731825326 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fac1c14c5c0 0x7fac1c150590 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 >> 172.21.15.76:0/1731825326 conn(0x7fac1c0b1f90 msgr2=0x7fac1c0b3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.725 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 shutdown_connections 2024-09-19T04:20:44.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.723+0000 7fac221b6640 1 -- 172.21.15.76:0/1731825326 wait complete. 2024-09-19T04:20:44.741 INFO:tasks.ceph:Scrubbing osd.6 2024-09-19T04:20:44.741 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:44.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9db732640 1 Processor -- start 2024-09-19T04:20:44.867 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9db732640 1 -- start start 2024-09-19T04:20:44.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9db732640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d41520a0 0x7fe9d406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9db732640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9d415f3a0 0x7fe9d40adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9db732640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d4070250 con 0x7fe9d406fe50 2024-09-19T04:20:44.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9db732640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d4152480 con 0x7fe9d415f3a0 2024-09-19T04:20:44.868 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9db732640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d40aede0 con 0x7fe9d41520a0 2024-09-19T04:20:44.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d94ce640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d41520a0 0x7fe9d406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.869 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d9ccf640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9d415f3a0 0x7fe9d40adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d8ccd640 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe9d406fe50 0x7fe9d415ec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.76:47854/0 (socket says 172.21.15.76:47854) 2024-09-19T04:20:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d94ce640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d41520a0 0x7fe9d406f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.116:3301/0 says I am v2:172.21.15.76:34102/0 (socket says 172.21.15.76:34102) 2024-09-19T04:20:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d8ccd640 1 -- 172.21.15.76:0/3546636340 learned_addr learned my addr 172.21.15.76:0/3546636340 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d94ce640 1 -- 172.21.15.76:0/3546636340 >> v2:172.21.15.116:3300/0 conn(0x7fe9d415f3a0 msgr2=0x7fe9d40adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d94ce640 1 --2- 172.21.15.76:0/3546636340 >> v2:172.21.15.116:3300/0 conn(0x7fe9d415f3a0 0x7fe9d40adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d94ce640 1 -- 172.21.15.76:0/3546636340 >> v1:172.21.15.116:6789/0 conn(0x7fe9d406fe50 legacy=0x7fe9d415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.870 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.867+0000 7fe9d94ce640 1 -- 172.21.15.76:0/3546636340 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d415fa10 con 0x7fe9d41520a0 2024-09-19T04:20:44.871 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9d94ce640 1 --2- 172.21.15.76:0/3546636340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d41520a0 0x7fe9d406f910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000ba60 tx=0x7fe9d001eb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12f868ac995edc79 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9cbfff640 1 -- 172.21.15.76:0/3546636340 <== mon.2 v2:172.21.15.116:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00b4070 con 0x7fe9d41520a0 2024-09-19T04:20:44.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9cbfff640 1 -- 172.21.15.76:0/3546636340 <== mon.2 v2:172.21.15.116:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d00a7c40 con 0x7fe9d41520a0 2024-09-19T04:20:44.872 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9cbfff640 1 -- 172.21.15.76:0/3546636340 <== mon.2 v2:172.21.15.116:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d00af440 con 0x7fe9d41520a0 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 -- 172.21.15.76:0/3546636340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d41520a0 msgr2=0x7fe9d406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 --2- 172.21.15.76:0/3546636340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d41520a0 0x7fe9d406f910 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9d000ba60 tx=0x7fe9d001eb70 comp rx=0 tx=0).stop 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 -- 172.21.15.76:0/3546636340 shutdown_connections 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 --2- 172.21.15.76:0/3546636340 >> v2:172.21.15.116:3300/0 conn(0x7fe9d415f3a0 0x7fe9d40adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 --2- 172.21.15.76:0/3546636340 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d41520a0 0x7fe9d406f910 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 -- 172.21.15.76:0/3546636340 >> 172.21.15.76:0/3546636340 conn(0x7fe9d40b1f90 msgr2=0x7fe9d40b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 -- 172.21.15.76:0/3546636340 shutdown_connections 2024-09-19T04:20:44.873 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 -- 172.21.15.76:0/3546636340 wait complete. 2024-09-19T04:20:44.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 Processor -- start 2024-09-19T04:20:44.874 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.871+0000 7fe9db732640 1 -- start start 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d406fe50 0x7fe9d414b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d41520a0 0x7fe9d41f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d415f3a0 0x7fe9d414c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4073d30 con 0x7fe9d41520a0 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4073a50 con 0x7fe9d406fe50 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d406e800 con 0x7fe9d415f3a0 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d41520a0 0x7fe9d41f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d41520a0 0x7fe9d41f7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52044/0 (socket says 172.21.15.76:52044) 2024-09-19T04:20:44.876 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 -- 172.21.15.76:0/129350629 learned_addr learned my addr 172.21.15.76:0/129350629 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d94ce640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d406fe50 0x7fe9d414b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d9ccf640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d415f3a0 0x7fe9d414c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 -- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d415f3a0 msgr2=0x7fe9d414c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe9d415f3a0 0x7fe9d414c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 -- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d406fe50 msgr2=0x7fe9d414b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9d406fe50 0x7fe9d414b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 -- 172.21.15.76:0/129350629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d40b1cb0 con 0x7fe9d41520a0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d8ccd640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d41520a0 0x7fe9d41f7150 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4001ec0 tx=0x7fe9c400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4017070 con 0x7fe9d41520a0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c4005cf0 con 0x7fe9d41520a0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d40b0ef0 con 0x7fe9d41520a0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c40072d0 con 0x7fe9d41520a0 2024-09-19T04:20:44.877 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d40b14b0 con 0x7fe9d41520a0 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9d40adc80 con 0x7fe9d41520a0 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe9c401a070 con 0x7fe9d41520a0 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe9b80678c0 0x7fe9b8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fe9c4084e00 con 0x7fe9d41520a0 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d94ce640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe9b80678c0 0x7fe9b8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7fe9b806dea0 0x7fe9b80702e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9b80709b0 con 0x7fe9b806dea0 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7fe9c4087500 con 0x7fe9d41520a0 2024-09-19T04:20:44.879 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d94ce640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe9b80678c0 0x7fe9b8069d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00b1000 tx=0x7fe9d00a9070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.875+0000 7fe9d9ccf640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7fe9b806dea0 0x7fe9b80702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:44.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.879+0000 7fe9d9ccf640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7fe9b806dea0 0x7fe9b80702e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:44.880 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.879+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== osd.6 v2:172.21.15.116:6816/2407229160 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9b80709b0 con 0x7fe9b806dea0 2024-09-19T04:20:44.893 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.891+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 --> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe9d406e1c0 con 0x7fe9b806dea0 2024-09-19T04:20:44.894 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.891+0000 7fe9c9ffb640 1 -- 172.21.15.76:0/129350629 <== osd.6 v2:172.21.15.116:6816/2407229160 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe9d406e1c0 con 0x7fe9b806dea0 2024-09-19T04:20:44.894 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:44.894 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:44.894 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7fe9b806dea0 msgr2=0x7fe9b80702e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.116:6816/2407229160,v1:172.21.15.116:6817/2407229160] conn(0x7fe9b806dea0 0x7fe9b80702e0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe9b80678c0 msgr2=0x7fe9b8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe9b80678c0 0x7fe9b8069d80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe9d00b1000 tx=0x7fe9d00a9070 comp rx=0 tx=0).stop 2024-09-19T04:20:44.895 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d41520a0 msgr2=0x7fe9d41f7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:44.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 --2- 172.21.15.76:0/129350629 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe9d41520a0 0x7fe9d41f7150 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4001ec0 tx=0x7fe9c400c890 comp rx=0 tx=0).stop 2024-09-19T04:20:44.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9d9ccf640 1 -- 172.21.15.76:0/129350629 reap_dead start 2024-09-19T04:20:44.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 shutdown_connections 2024-09-19T04:20:44.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 >> 172.21.15.76:0/129350629 conn(0x7fe9d40b1f90 msgr2=0x7fe9d406ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:44.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 shutdown_connections 2024-09-19T04:20:44.896 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:44.895+0000 7fe9db732640 1 -- 172.21.15.76:0/129350629 wait complete. 2024-09-19T04:20:44.910 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-19T04:20:45.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6ab052d640 1 Processor -- start 2024-09-19T04:20:45.023 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6ab052d640 1 -- start start 2024-09-19T04:20:45.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6ab052d640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6aa815d290 0x7f6aa815e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6ab052d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa815fc10 con 0x7f6aa815f830 2024-09-19T04:20:45.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6ab052d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6aa815d800 con 0x7f6aa806c3d0 2024-09-19T04:20:45.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6ab052d640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f6aa815da40 con 0x7f6aa815d290 2024-09-19T04:20:45.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f6aa815f830 0x7f6aa806bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43248/0 (socket says 172.21.15.76:43248) 2024-09-19T04:20:45.026 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 -- 172.21.15.76:0/1367474704 learned_addr learned my addr 172.21.15.76:0/1367474704 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aaeaca640 1 --2- 172.21.15.76:0/1367474704 >> v2:172.21.15.116:3300/0 conn(0x7f6aa815d290 0x7f6aa815e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1537819736 0 0) 0x7f6aa815fc10 con 0x7f6aa815f830 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6a90003530 con 0x7f6aa815f830 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3534046585 0 0) 0x7f6a90003530 con 0x7f6aa815f830 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 >> v2:172.21.15.116:3300/0 conn(0x7f6aa815d290 msgr2=0x7f6aa815e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 --2- 172.21.15.76:0/1367474704 >> v2:172.21.15.116:3300/0 conn(0x7f6aa815d290 0x7f6aa815e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 >> v1:172.21.15.116:6789/0 conn(0x7f6aa806c3d0 legacy=0x7f6aa815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa80b3570 con 0x7f6aa815f830 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f6aa0001a00 con 0x7f6aa815f830 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 >> v1:172.21.15.76:6789/0 conn(0x7f6aa815f830 legacy=0x7f6aa806bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6a90003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a90004810 0x7f6a90004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a900051d0 0x7f6a9000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.027 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa815da40 con 0x7f6a900039c0 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6a9000c070 con 0x7f6a900051d0 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6a9000c270 con 0x7f6a90004810 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6a90003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aaeaca640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a900051d0 0x7f6a9000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aadac8640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a90004810 0x7f6a90004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 -- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a90004810 msgr2=0x7f6a90004c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6a90004810 0x7f6a90004c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 -- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a900051d0 msgr2=0x7f6a9000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a900051d0 0x7f6a9000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aaeaca640 1 -- 172.21.15.76:0/1367474704 reap_dead start 2024-09-19T04:20:45.028 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.023+0000 7f6aae2c9640 1 -- 172.21.15.76:0/1367474704 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a90003530 con 0x7f6a900039c0 2024-09-19T04:20:45.029 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6aae2c9640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6a90003de0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000a840 tx=0x7f6aa001d2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a3472c58a2277d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa000a460 con 0x7f6a900039c0 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa00ad650 con 0x7f6a900039c0 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6aad2c7640 1 -- 172.21.15.76:0/1367474704 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa00ada80 con 0x7f6a900039c0 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 -- 172.21.15.76:0/1367474704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 msgr2=0x7f6a90003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6a90003de0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000a840 tx=0x7f6aa001d2f0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 -- 172.21.15.76:0/1367474704 shutdown_connections 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 --2- 172.21.15.76:0/1367474704 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6a90003de0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 -- 172.21.15.76:0/1367474704 >> 172.21.15.76:0/1367474704 conn(0x7f6aa80b6800 msgr2=0x7f6aa80b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 -- 172.21.15.76:0/1367474704 shutdown_connections 2024-09-19T04:20:45.030 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 -- 172.21.15.76:0/1367474704 wait complete. 2024-09-19T04:20:45.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 Processor -- start 2024-09-19T04:20:45.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.027+0000 7f6ab052d640 1 -- start start 2024-09-19T04:20:45.032 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6aa8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6aa80b0290 0x7f6aa81f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6aa8156550 0x7f6aa81f9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa80735e0 con 0x7f6a900039c0 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f6aa8073460 con 0x7f6aa80b0290 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f6aa8160680 con 0x7f6aa8156550 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6aa8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6aa8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52070/0 (socket says 172.21.15.76:52070) 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 -- 172.21.15.76:0/3765450381 learned_addr learned my addr 172.21.15.76:0/3765450381 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aaeaca640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6aa8156550 0x7f6aa81f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aadac8640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6aa80b0290 0x7f6aa81f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 -- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6aa8156550 msgr2=0x7f6aa81f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.033 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6aa8156550 0x7f6aa81f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 -- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6aa80b0290 msgr2=0x7f6aa81f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6aa80b0290 0x7f6aa81f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 -- 172.21.15.76:0/3765450381 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6aa81f6910 con 0x7f6a900039c0 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aae2c9640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6aa8156010 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000a840 tx=0x7f6a90005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6a9effd640 1 -- 172.21.15.76:0/3765450381 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0001260 con 0x7f6a900039c0 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6a9effd640 1 -- 172.21.15.76:0/3765450381 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6aa00ad430 con 0x7f6a900039c0 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6aa8156a00 con 0x7f6a900039c0 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6a9effd640 1 -- 172.21.15.76:0/3765450381 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6aa0001800 con 0x7f6a900039c0 2024-09-19T04:20:45.034 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6aa8156fc0 con 0x7f6a900039c0 2024-09-19T04:20:45.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6aa8160980 con 0x7f6a900039c0 2024-09-19T04:20:45.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6a9effd640 1 -- 172.21.15.76:0/3765450381 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f6aa00bc020 con 0x7f6a900039c0 2024-09-19T04:20:45.035 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6a9effd640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a94067a60 0x7f6a94069f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6a9effd640 1 -- 172.21.15.76:0/3765450381 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f6aa0120790 con 0x7f6a900039c0 2024-09-19T04:20:45.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.031+0000 7f6aadac8640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a94067a60 0x7f6a94069f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.036 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.035+0000 7f6aadac8640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a94067a60 0x7f6a94069f20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80d0560 tx=0x7f6aa4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.037 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.035+0000 7f6a9effd640 1 -- 172.21.15.76:0/3765450381 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f6aa0017030 con 0x7f6a900039c0 2024-09-19T04:20:45.197 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.195+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f6aa806e7c0 con 0x7f6a94067a60 2024-09-19T04:20:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.195+0000 7f6a9effd640 1 -- 172.21.15.76:0/3765450381 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6aa806e7c0 con 0x7f6a94067a60 2024-09-19T04:20:45.198 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 6 to deep-scrub 2024-09-19T04:20:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a94067a60 msgr2=0x7f6a94069f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a94067a60 0x7f6a94069f20 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6aa80d0560 tx=0x7f6aa4009040 comp rx=0 tx=0).stop 2024-09-19T04:20:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 msgr2=0x7f6aa8156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6aa8156010 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f6aa000a840 tx=0x7f6a90005840 comp rx=0 tx=0).stop 2024-09-19T04:20:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 shutdown_connections 2024-09-19T04:20:45.200 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f6a94067a60 0x7f6a94069f20 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f6aa8156550 0x7f6aa81f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6aa80b0290 0x7f6aa81f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 --2- 172.21.15.76:0/3765450381 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f6a900039c0 0x7f6aa8156010 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 >> 172.21.15.76:0/3765450381 conn(0x7f6aa80b6800 msgr2=0x7f6aa815bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 shutdown_connections 2024-09-19T04:20:45.201 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.199+0000 7f6ab052d640 1 -- 172.21.15.76:0/3765450381 wait complete. 2024-09-19T04:20:45.216 INFO:tasks.ceph:Scrubbing osd.7 2024-09-19T04:20:45.216 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-19T04:20:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bbd187640 1 Processor -- start 2024-09-19T04:20:45.343 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bbd187640 1 -- start start 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bbd187640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4bb815e450 0x7f4bb809aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bbd187640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4bb806c3d0 0x7f4bb815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bbd187640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bb80b5c90 con 0x7f4bb80b40c0 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bbd187640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb80b4860 con 0x7f4bb806c3d0 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bbd187640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f4bb80b4580 con 0x7f4bb815e450 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb759e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4bb806c3d0 0x7f4bb815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb759e640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f4bb806c3d0 0x7f4bb815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52072/0 (socket says 172.21.15.76:52072) 2024-09-19T04:20:45.345 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb759e640 1 -- 172.21.15.76:0/1098095745 learned_addr learned my addr 172.21.15.76:0/1098095745 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb6d9d640 1 --2- 172.21.15.76:0/1098095745 >> v2:172.21.15.116:3300/0 conn(0x7f4bb815e450 0x7f4bb809aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb759e640 1 -- 172.21.15.76:0/1098095745 >> v2:172.21.15.116:3300/0 conn(0x7f4bb815e450 msgr2=0x7f4bb809aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb759e640 1 --2- 172.21.15.76:0/1098095745 >> v2:172.21.15.116:3300/0 conn(0x7f4bb815e450 0x7f4bb809aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb759e640 1 -- 172.21.15.76:0/1098095745 >> v1:172.21.15.116:6789/0 conn(0x7f4bb80b40c0 legacy=0x7f4bb806bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.346 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.343+0000 7f4bb759e640 1 -- 172.21.15.76:0/1098095745 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb80b5f10 con 0x7f4bb806c3d0 2024-09-19T04:20:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bb759e640 1 --2- 172.21.15.76:0/1098095745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb806c3d0 0x7f4bb815fc60 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8071880 tx=0x7f4ba801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca400a193be3595 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bb5d9b640 1 -- 172.21.15.76:0/1098095745 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80b1070 con 0x7f4bb806c3d0 2024-09-19T04:20:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bb5d9b640 1 -- 172.21.15.76:0/1098095745 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba80ac600 con 0x7f4bb806c3d0 2024-09-19T04:20:45.347 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bb5d9b640 1 -- 172.21.15.76:0/1098095745 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80ac9d0 con 0x7f4bb806c3d0 2024-09-19T04:20:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- 172.21.15.76:0/1098095745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb806c3d0 msgr2=0x7f4bb815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 --2- 172.21.15.76:0/1098095745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb806c3d0 0x7f4bb815fc60 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8071880 tx=0x7f4ba801c490 comp rx=0 tx=0).stop 2024-09-19T04:20:45.348 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- 172.21.15.76:0/1098095745 shutdown_connections 2024-09-19T04:20:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 --2- 172.21.15.76:0/1098095745 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb806c3d0 0x7f4bb815fc60 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 --2- 172.21.15.76:0/1098095745 >> v2:172.21.15.116:3300/0 conn(0x7f4bb815e450 0x7f4bb809aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- 172.21.15.76:0/1098095745 >> 172.21.15.76:0/1098095745 conn(0x7f4bb80b6800 msgr2=0x7f4bb80b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- 172.21.15.76:0/1098095745 shutdown_connections 2024-09-19T04:20:45.349 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- 172.21.15.76:0/1098095745 wait complete. 2024-09-19T04:20:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 Processor -- start 2024-09-19T04:20:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- start start 2024-09-19T04:20:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bb806c3d0 0x7f4bb815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bb80b40c0 0x7f4bb8156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb815e450 0x7f4bb81575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.351 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8162d00 con 0x7f4bb815e450 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f4bb8162b80 con 0x7f4bb80b40c0 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bbd187640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f4bb8162e80 con 0x7f4bb806c3d0 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bb759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb815e450 0x7f4bb81575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bb759e640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb815e450 0x7f4bb81575b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52082/0 (socket says 172.21.15.76:52082) 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.347+0000 7f4bb759e640 1 -- 172.21.15.76:0/3009209036 learned_addr learned my addr 172.21.15.76:0/3009209036 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb659c640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bb80b40c0 0x7f4bb8156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb6d9d640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bb806c3d0 0x7f4bb815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb759e640 1 -- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bb806c3d0 msgr2=0x7f4bb815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb759e640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f4bb806c3d0 0x7f4bb815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb759e640 1 -- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bb80b40c0 msgr2=0x7f4bb8156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.352 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb759e640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4bb80b40c0 0x7f4bb8156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb759e640 1 -- 172.21.15.76:0/3009209036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bb815b0e0 con 0x7f4bb815e450 2024-09-19T04:20:45.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb759e640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb815e450 0x7f4bb81575b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f4ba801a0c0 tx=0x7f4ba801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80b1040 con 0x7f4bb815e450 2024-09-19T04:20:45.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ba80bc070 con 0x7f4bb815e450 2024-09-19T04:20:45.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80b7690 con 0x7f4bb815e450 2024-09-19T04:20:45.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bb8156ad0 con 0x7f4bb815e450 2024-09-19T04:20:45.353 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bb81566a0 con 0x7f4bb815e450 2024-09-19T04:20:45.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4bb815fc60 con 0x7f4bb815e450 2024-09-19T04:20:45.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f4ba80b7830 con 0x7f4bb815e450 2024-09-19T04:20:45.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b980678f0 0x7f4b98069db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.354 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f4ba81204b0 con 0x7f4bb815e450 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb6d9d640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b980678f0 0x7f4b98069db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f4b9806ded0 0x7f4b98070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4ba80ac600 con 0x7f4b9806ded0 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) ==== 24+0+0 (secure 0 0 0) 0x7f4ba8120940 con 0x7f4bb815e450 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb6d9d640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b980678f0 0x7f4b98069db0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009f60 tx=0x7f4bb000b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb659c640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f4b9806ded0 0x7f4b98070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.351+0000 7f4bb659c640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f4b9806ded0 0x7f4b98070310 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.355 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.355+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== osd.7 v2:172.21.15.116:6824/2361616500 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4ba80ac600 con 0x7f4b9806ded0 2024-09-19T04:20:45.369 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 --> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4bb806e7c0 con 0x7f4b9806ded0 2024-09-19T04:20:45.370 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4baf7fe640 1 -- 172.21.15.76:0/3009209036 <== osd.7 v2:172.21.15.116:6824/2361616500 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4bb806e7c0 con 0x7f4b9806ded0 2024-09-19T04:20:45.370 INFO:teuthology.orchestra.run.smithi076.stdout:{ 2024-09-19T04:20:45.370 INFO:teuthology.orchestra.run.smithi076.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-19T04:20:45.370 INFO:teuthology.orchestra.run.smithi076.stdout:} 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f4b9806ded0 msgr2=0x7f4b98070310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4bbd187640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.116:6824/2361616500,v1:172.21.15.116:6825/2361616500] conn(0x7f4b9806ded0 0x7f4b98070310 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b980678f0 msgr2=0x7f4b98069db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4bbd187640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f4b980678f0 0x7f4b98069db0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0009f60 tx=0x7f4bb000b040 comp rx=0 tx=0).stop 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb815e450 msgr2=0x7f4bb81575b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.367+0000 7f4bbd187640 1 --2- 172.21.15.76:0/3009209036 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f4bb815e450 0x7f4bb81575b0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f4ba801a0c0 tx=0x7f4ba801acd0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.371+0000 7f4bb759e640 1 -- 172.21.15.76:0/3009209036 reap_dead start 2024-09-19T04:20:45.371 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.371+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 shutdown_connections 2024-09-19T04:20:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.371+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 >> 172.21.15.76:0/3009209036 conn(0x7f4bb80b6800 msgr2=0x7f4bb806bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.371+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 shutdown_connections 2024-09-19T04:20:45.372 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.371+0000 7f4bbd187640 1 -- 172.21.15.76:0/3009209036 wait complete. 2024-09-19T04:20:45.386 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-19T04:20:45.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f265c01d640 1 Processor -- start 2024-09-19T04:20:45.496 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f265c01d640 1 -- start start 2024-09-19T04:20:45.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f265c01d640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2654161a60 0x7f26540d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f265c01d640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26540d32b0 con 0x7f26540d2e90 2024-09-19T04:20:45.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f265c01d640 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26540d0300 con 0x7f26540cf620 2024-09-19T04:20:45.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f265c01d640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7f26540cfa90 con 0x7f2654161a60 2024-09-19T04:20:45.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f2659db9640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7f2654161a60 0x7f26540d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.497 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f26595b8640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f26540d2e90 0x7f26540cef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:43258/0 (socket says 172.21.15.76:43258) 2024-09-19T04:20:45.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f26595b8640 1 -- 172.21.15.76:0/555521825 learned_addr learned my addr 172.21.15.76:0/555521825 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:20:45.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f2659db9640 1 -- 172.21.15.76:0/555521825 >> v1:172.21.15.116:6789/0 conn(0x7f26540cf620 legacy=0x7f265406d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f2659db9640 1 -- 172.21.15.76:0/555521825 >> v1:172.21.15.76:6789/0 conn(0x7f26540d2e90 legacy=0x7f26540cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f2659db9640 1 -- 172.21.15.76:0/555521825 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26540b0f60 con 0x7f2654161a60 2024-09-19T04:20:45.498 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.495+0000 7f2658db7640 1 -- 172.21.15.76:0/555521825 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4159380629 0 0) 0x7f26540d32b0 con 0x7f26540d2e90 2024-09-19T04:20:45.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f2659db9640 1 --2- 172.21.15.76:0/555521825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540d2950 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f26480097c0 tx=0x7f264801c1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5efd95bb3a72a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f2658db7640 1 -- 172.21.15.76:0/555521825 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480b0070 con 0x7f2654161a60 2024-09-19T04:20:45.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f2658db7640 1 -- 172.21.15.76:0/555521825 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26480a4e50 con 0x7f2654161a60 2024-09-19T04:20:45.499 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f2658db7640 1 -- 172.21.15.76:0/555521825 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26480ab820 con 0x7f2654161a60 2024-09-19T04:20:45.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- 172.21.15.76:0/555521825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 msgr2=0x7f26540d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 --2- 172.21.15.76:0/555521825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540d2950 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f26480097c0 tx=0x7f264801c1c0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.500 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- 172.21.15.76:0/555521825 shutdown_connections 2024-09-19T04:20:45.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 --2- 172.21.15.76:0/555521825 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540d2950 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- 172.21.15.76:0/555521825 >> 172.21.15.76:0/555521825 conn(0x7f265408bf90 msgr2=0x7f265409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- 172.21.15.76:0/555521825 shutdown_connections 2024-09-19T04:20:45.501 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- 172.21.15.76:0/555521825 wait complete. 2024-09-19T04:20:45.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 Processor -- start 2024-09-19T04:20:45.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- start start 2024-09-19T04:20:45.503 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26540cf620 0x7f26540cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26540d2e90 0x7f265414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f265416bf30 con 0x7f2654161a60 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f265406d190 con 0x7f26540cf620 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265c01d640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f265416c0b0 con 0x7f26540d2e90 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265a5ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265a5ba640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52108/0 (socket says 172.21.15.76:52108) 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.499+0000 7f265a5ba640 1 -- 172.21.15.76:0/2644450087 learned_addr learned my addr 172.21.15.76:0/2644450087 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f26595b8640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26540d2e90 0x7f265414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265a5ba640 1 -- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26540d2e90 msgr2=0x7f265414dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.504 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f2659db9640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26540cf620 0x7f26540cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265a5ba640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26540d2e90 0x7f265414dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265a5ba640 1 -- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26540cf620 msgr2=0x7f26540cb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265a5ba640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26540cf620 0x7f26540cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265a5ba640 1 -- 172.21.15.76:0/2644450087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265415af10 con 0x7f2654161a60 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265a5ba640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540b2cd0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f264c00c780 tx=0x7f264c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f26527fc640 1 -- 172.21.15.76:0/2644450087 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c017070 con 0x7f2654161a60 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f26527fc640 1 -- 172.21.15.76:0/2644450087 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f264c003db0 con 0x7f2654161a60 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265415b1b0 con 0x7f2654161a60 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f26527fc640 1 -- 172.21.15.76:0/2644450087 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0052b0 con 0x7f2654161a60 2024-09-19T04:20:45.505 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f265414dff0 con 0x7f2654161a60 2024-09-19T04:20:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f265415b380 con 0x7f2654161a60 2024-09-19T04:20:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f26527fc640 1 -- 172.21.15.76:0/2644450087 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f264c01a070 con 0x7f2654161a60 2024-09-19T04:20:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f26527fc640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26380678c0 0x7f2638069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f2659db9640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26380678c0 0x7f2638069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f26527fc640 1 -- 172.21.15.76:0/2644450087 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f264c085260 con 0x7f2654161a60 2024-09-19T04:20:45.506 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.503+0000 7f2659db9640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26380678c0 0x7f2638069d80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26480ad000 tx=0x7f2648008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.509 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.507+0000 7f26527fc640 1 -- 172.21.15.76:0/2644450087 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f264c055e60 con 0x7f2654161a60 2024-09-19T04:20:45.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.667+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f265406dac0 con 0x7f26380678c0 2024-09-19T04:20:45.669 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.667+0000 7f26527fc640 1 -- 172.21.15.76:0/2644450087 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f265406dac0 con 0x7f26380678c0 2024-09-19T04:20:45.669 INFO:teuthology.orchestra.run.smithi076.stderr:instructed osd(s) 7 to deep-scrub 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26380678c0 msgr2=0x7f2638069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26380678c0 0x7f2638069d80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26480ad000 tx=0x7f2648008040 comp rx=0 tx=0).stop 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 msgr2=0x7f26540b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540b2cd0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f264c00c780 tx=0x7f264c00cc40 comp rx=0 tx=0).stop 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 shutdown_connections 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f26380678c0 0x7f2638069d80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f2654161a60 0x7f26540b2cd0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f26540d2e90 0x7f265414dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 --2- 172.21.15.76:0/2644450087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26540cf620 0x7f26540cb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 >> 172.21.15.76:0/2644450087 conn(0x7f265408bf90 msgr2=0x7f26540b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 shutdown_connections 2024-09-19T04:20:45.672 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.671+0000 7f265c01d640 1 -- 172.21.15.76:0/2644450087 wait complete. 2024-09-19T04:20:45.687 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:20:45.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 Processor -- start 2024-09-19T04:20:45.795 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 -- start start 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe094161a60 0x7fe0940d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0940d2e90 0x7fe0940cef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940cf450 0x7fe0940d0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 -- --> v2:172.21.15.76:3300/0 -- mon_getmap magic: 0 -- 0x7fe0940d32b0 con 0x7fe0940d2e90 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fe0940d0300 con 0x7fe094161a60 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe099d48640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe0940b0f60 con 0x7fe0940cf450 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0940d2e90 0x7fe0940cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 --2- >> v2:172.21.15.76:3300/0 conn(0x7fe0940d2e90 0x7fe0940cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52118/0 (socket says 172.21.15.76:52118) 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 -- 172.21.15.76:0/2310102315 learned_addr learned my addr 172.21.15.76:0/2310102315 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 -- 172.21.15.76:0/2310102315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940cf450 msgr2=0x7fe0940d0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.797 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 --2- 172.21.15.76:0/2310102315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940cf450 0x7fe0940d0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 -- 172.21.15.76:0/2310102315 >> v2:172.21.15.116:3300/0 conn(0x7fe094161a60 msgr2=0x7fe0940d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:20:45.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe0937fe640 1 --2- 172.21.15.76:0/2310102315 >> v2:172.21.15.116:3300/0 conn(0x7fe094161a60 0x7fe0940d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 --2- 172.21.15.76:0/2310102315 >> v2:172.21.15.116:3300/0 conn(0x7fe094161a60 0x7fe0940d2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.798 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.795+0000 7fe092ffd640 1 -- 172.21.15.76:0/2310102315 --> v2:172.21.15.76:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0940cfd40 con 0x7fe0940d2e90 2024-09-19T04:20:45.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe092ffd640 1 --2- 172.21.15.76:0/2310102315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940d2e90 0x7fe0940cef10 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe09408e010 tx=0x7fe08401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=343432b24d08b7f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0927fc640 1 -- 172.21.15.76:0/2310102315 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840ab620 con 0x7fe0940d2e90 2024-09-19T04:20:45.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0927fc640 1 -- 172.21.15.76:0/2310102315 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0840b0070 con 0x7fe0940d2e90 2024-09-19T04:20:45.799 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0927fc640 1 -- 172.21.15.76:0/2310102315 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0840ab9e0 con 0x7fe0940d2e90 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- 172.21.15.76:0/2310102315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940d2e90 msgr2=0x7fe0940cef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 --2- 172.21.15.76:0/2310102315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940d2e90 0x7fe0940cef10 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fe09408e010 tx=0x7fe08401c460 comp rx=0 tx=0).stop 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- 172.21.15.76:0/2310102315 shutdown_connections 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 --2- 172.21.15.76:0/2310102315 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940cf450 0x7fe0940d0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 --2- 172.21.15.76:0/2310102315 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940d2e90 0x7fe0940cef10 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 --2- 172.21.15.76:0/2310102315 >> v2:172.21.15.116:3300/0 conn(0x7fe094161a60 0x7fe0940d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- 172.21.15.76:0/2310102315 >> 172.21.15.76:0/2310102315 conn(0x7fe09408bf90 msgr2=0x7fe09409d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- 172.21.15.76:0/2310102315 shutdown_connections 2024-09-19T04:20:45.800 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- 172.21.15.76:0/2310102315 wait complete. 2024-09-19T04:20:45.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 Processor -- start 2024-09-19T04:20:45.801 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- start start 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940cf450 0x7fe094151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940d2e90 0x7fe0941576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe094161a60 0x7fe094157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940cfa40 con 0x7fe0940cf450 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fe0940cf8c0 con 0x7fe094161a60 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe099d48640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fe09406d6b0 con 0x7fe0940d2e90 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940cf450 0x7fe094151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.803 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940cf450 0x7fe094151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:52134/0 (socket says 172.21.15.76:52134) 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 -- 172.21.15.76:0/3639957530 learned_addr learned my addr 172.21.15.76:0/3639957530 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe092ffd640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940d2e90 0x7fe0941576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe093fff640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe094161a60 0x7fe094157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 -- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940d2e90 msgr2=0x7fe0941576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940d2e90 0x7fe0941576c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 -- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe094161a60 msgr2=0x7fe094157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe094161a60 0x7fe094157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 -- 172.21.15.76:0/3639957530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe094152780 con 0x7fe0940cf450 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe0937fe640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940cf450 0x7fe094151f00 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe08800c7f0 tx=0x7fe08800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.799+0000 7fe07bfff640 1 -- 172.21.15.76:0/3639957530 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088015070 con 0x7fe0940cf450 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe07bfff640 1 -- 172.21.15.76:0/3639957530 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe088007820 con 0x7fe0940cf450 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe094152a60 con 0x7fe0940cf450 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe07bfff640 1 -- 172.21.15.76:0/3639957530 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe088007c10 con 0x7fe0940cf450 2024-09-19T04:20:45.804 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe094153020 con 0x7fe0940cf450 2024-09-19T04:20:45.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0940cef10 con 0x7fe0940cf450 2024-09-19T04:20:45.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe07bfff640 1 -- 172.21.15.76:0/3639957530 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fe088018070 con 0x7fe0940cf450 2024-09-19T04:20:45.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe07bfff640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe0800678c0 0x7fe080069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:20:45.805 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe07bfff640 1 -- 172.21.15.76:0/3639957530 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fe088085cb0 con 0x7fe0940cf450 2024-09-19T04:20:45.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe092ffd640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe0800678c0 0x7fe080069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:20:45.806 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.803+0000 7fe092ffd640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe0800678c0 0x7fe080069d80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe0840af000 tx=0x7fe0840a7000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:20:45.809 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.807+0000 7fe07bfff640 1 -- 172.21.15.76:0/3639957530 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fe0880568b0 con 0x7fe0940cf450 2024-09-19T04:20:45.953 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.951+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe0940b47c0 con 0x7fe0800678c0 2024-09-19T04:20:45.954 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.951+0000 7fe07bfff640 1 -- 172.21.15.76:0/3639957530 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13483 (secure 0 0 0) 0x7fe0940b47c0 con 0x7fe0800678c0 2024-09-19T04:20:45.954 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:20:45.954 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:20:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe0800678c0 msgr2=0x7fe080069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe0800678c0 0x7fe080069d80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe0840af000 tx=0x7fe0840a7000 comp rx=0 tx=0).stop 2024-09-19T04:20:45.956 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940cf450 msgr2=0x7fe094151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940cf450 0x7fe094151f00 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fe08800c7f0 tx=0x7fe08800ccb0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 shutdown_connections 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fe0800678c0 0x7fe080069d80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe094161a60 0x7fe094157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fe0940d2e90 0x7fe0941576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 --2- 172.21.15.76:0/3639957530 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fe0940cf450 0x7fe094151f00 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 >> 172.21.15.76:0/3639957530 conn(0x7fe09408bf90 msgr2=0x7fe0940b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 shutdown_connections 2024-09-19T04:20:45.957 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:20:45.955+0000 7fe099d48640 1 -- 172.21.15.76:0/3639957530 wait complete. 2024-09-19T04:20:45.971 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-19T04:20:45.878093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261276,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713444,"statfs":{"total":773094113280,"available":772826566656,"internally_reserved":0,"allocated":12034048,"data_stored":6247000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.877450"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":333,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-19T04:20:38.891688+0000","last_change":"2024-09-19T04:13:14.844758+0000","last_active":"2024-09-19T04:20:38.891688+0000","last_peered":"2024-09-19T04:20:38.891688+0000","last_clean":"2024-09-19T04:20:38.891688+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:20:38.891688+0000","last_undegraded":"2024-09-19T04:20:38.891688+0000","last_fullsized":"2024-09-19T04:20:38.891688+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T04:13:13.818788+0000","last_clean_scrub_stamp":"2024-09-19T04:13:13.818788+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:55:24.464218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542237,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33096,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338744,"statfs":{"total":96636764160,"available":96602873856,"internally_reserved":0,"allocated":1949696,"data_stored":1223651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542238,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542238,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542237,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542237,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542238,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542238,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542238,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33092,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338748,"statfs":{"total":96636764160,"available":96602877952,"internally_reserved":0,"allocated":1949696,"data_stored":1223651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:20:45.975 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-19T04:13:13.818788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=4, tm_min=13, tm_sec=13, tm_wday=3, tm_yday=263, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=19, tm_hour=4, tm_min=20, tm_sec=41, tm_wday=3, tm_yday=263, tm_isdst=0) 2024-09-19T04:20:45.975 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-19T04:20:54.310 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:20:54.313 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:21:05.976 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T04:21:06.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f201360c640 1 Processor -- start 2024-09-19T04:21:06.107 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f201360c640 1 -- start start 2024-09-19T04:21:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f201360c640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f200c0adad0 0x7f200c15f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.109 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f201360c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 0x7f200c0cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f201360c640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f200c0af300 con 0x7f200c0aee70 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f201360c640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7f200c0af5e0 con 0x7f200c0adad0 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f201360c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f200c0af8c0 con 0x7f200c155450 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7f200c0aee70 0x7f200c154d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45494/0 (socket says 172.21.15.76:45494) 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 -- 172.21.15.76:0/850246224 learned_addr learned my addr 172.21.15.76:0/850246224 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2011ba9640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 0x7f200c0cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f20113a8640 1 --2- 172.21.15.76:0/850246224 >> v2:172.21.15.116:3300/0 conn(0x7f200c0adad0 0x7f200c15f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3985244160 0 0) 0x7f200c0af300 con 0x7f200c0aee70 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1ff8003530 con 0x7f200c0aee70 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3859573510 0 0) 0x7f1ff8003530 con 0x7f200c0aee70 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 msgr2=0x7f200c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 0x7f200c0cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.110 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 >> v2:172.21.15.116:3300/0 conn(0x7f200c0adad0 msgr2=0x7f200c15f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 --2- 172.21.15.76:0/850246224 >> v2:172.21.15.116:3300/0 conn(0x7f200c0adad0 0x7f200c15f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c0afba0 con 0x7f200c0aee70 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7f2000001680 con 0x7f200c0aee70 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 >> v1:172.21.15.76:6789/0 conn(0x7f200c0aee70 legacy=0x7f200c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ff80039c0 0x7f1ff8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f1ff8004c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 0x7f1ff80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f200c0af5e0 con 0x7f1ff8004810 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f200c0af8c0 con 0x7f1ff80039c0 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f1ff800b9d0 con 0x7f200c155450 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f1ff8004c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.111 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f20113a8640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ff80039c0 0x7f1ff8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2011ba9640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 0x7f1ff80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 -- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 msgr2=0x7f1ff80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c155450 0x7f1ff80051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 -- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ff80039c0 msgr2=0x7f1ff8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1ff80039c0 0x7f1ff8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.112 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.107+0000 7f2010ba7640 1 -- 172.21.15.76:0/850246224 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff8003530 con 0x7f1ff8004810 2024-09-19T04:21:06.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f2010ba7640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f1ff8004c90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f200c15c400 tx=0x7f200001f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75526042111359aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000007d60 con 0x7f1ff8004810 2024-09-19T04:21:06.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20000ac020 con 0x7f1ff8004810 2024-09-19T04:21:06.113 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f1ffffff640 1 -- 172.21.15.76:0/850246224 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20000058e0 con 0x7f1ff8004810 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 -- 172.21.15.76:0/850246224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 msgr2=0x7f1ff8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 --2- 172.21.15.76:0/850246224 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f1ff8004c90 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f200c15c400 tx=0x7f200001f3d0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f2011ba9640 1 -- 172.21.15.76:0/850246224 reap_dead start 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 -- 172.21.15.76:0/850246224 shutdown_connections 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 -- 172.21.15.76:0/850246224 >> 172.21.15.76:0/850246224 conn(0x7f200c0736d0 msgr2=0x7f200c06bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 -- 172.21.15.76:0/850246224 shutdown_connections 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 -- 172.21.15.76:0/850246224 wait complete. 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 Processor -- start 2024-09-19T04:21:06.114 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.111+0000 7f201360c640 1 -- start start 2024-09-19T04:21:06.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f200c158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f200c15d450 0x7f200c14eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c06c4c0 0x7f200c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7f200c0b03a0 con 0x7f1ff8004810 2024-09-19T04:21:06.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7f200c067c00 con 0x7f200c15d450 2024-09-19T04:21:06.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7f200c0b0520 con 0x7f200c06c4c0 2024-09-19T04:21:06.116 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f200c158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f200c158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:50978/0 (socket says 172.21.15.76:50978) 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 -- 172.21.15.76:0/590611844 learned_addr learned my addr 172.21.15.76:0/590611844 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 -- 172.21.15.76:0/590611844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c06c4c0 msgr2=0x7f200c06c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c06c4c0 0x7f200c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 -- 172.21.15.76:0/590611844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f200c15d450 msgr2=0x7f200c14eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f200c15d450 0x7f200c14eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 -- 172.21.15.76:0/590611844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200c15dc10 con 0x7f1ff8004810 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f20113a8640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f200c158730 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f2008011540 tx=0x7f2008011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f1ffdffb640 1 -- 172.21.15.76:0/590611844 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008015930 con 0x7f1ff8004810 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f1ffdffb640 1 -- 172.21.15.76:0/590611844 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2008015ad0 con 0x7f1ff8004810 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200c15def0 con 0x7f1ff8004810 2024-09-19T04:21:06.117 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f1ffdffb640 1 -- 172.21.15.76:0/590611844 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200800f460 con 0x7f1ff8004810 2024-09-19T04:21:06.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200c155930 con 0x7f1ff8004810 2024-09-19T04:21:06.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200c0adeb0 con 0x7f1ff8004810 2024-09-19T04:21:06.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f1ffdffb640 1 -- 172.21.15.76:0/590611844 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7f200800f600 con 0x7f1ff8004810 2024-09-19T04:21:06.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f1ffdffb640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ff00678c0 0x7f1ff0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.118 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f2010ba7640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ff00678c0 0x7f1ff0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f1ffdffb640 1 -- 172.21.15.76:0/590611844 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7f2008086450 con 0x7f1ff8004810 2024-09-19T04:21:06.119 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.115+0000 7f2010ba7640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ff00678c0 0x7f1ff0069d80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20000ad000 tx=0x7f200c0b41c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.121 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.119+0000 7f1ffdffb640 1 -- 172.21.15.76:0/590611844 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7f2008019030 con 0x7f1ff8004810 2024-09-19T04:21:06.277 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.275+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 --> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f200c0d0970 con 0x7f1ff00678c0 2024-09-19T04:21:06.278 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.275+0000 7f1ffdffb640 1 -- 172.21.15.76:0/590611844 <== mgr.4101 v2:172.21.15.76:6832/3111825053 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13379 (secure 0 0 0) 0x7f200c0d0970 con 0x7f1ff00678c0 2024-09-19T04:21:06.278 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:21:06.278 INFO:teuthology.orchestra.run.smithi076.stderr:dumped all 2024-09-19T04:21:06.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ff00678c0 msgr2=0x7f1ff0069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ff00678c0 0x7f1ff0069d80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20000ad000 tx=0x7f200c0b41c0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 msgr2=0x7f200c158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f200c158730 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f2008011540 tx=0x7f2008011a00 comp rx=0 tx=0).stop 2024-09-19T04:21:06.280 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 shutdown_connections 2024-09-19T04:21:06.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7f1ff00678c0 0x7f1ff0069d80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7f200c06c4c0 0x7f200c06c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f200c15d450 0x7f200c14eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 --2- 172.21.15.76:0/590611844 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7f1ff8004810 0x7f200c158730 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 >> 172.21.15.76:0/590611844 conn(0x7f200c0736d0 msgr2=0x7f200c0b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:06.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 shutdown_connections 2024-09-19T04:21:06.281 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.279+0000 7f201360c640 1 -- 172.21.15.76:0/590611844 wait complete. 2024-09-19T04:21:06.295 INFO:teuthology.orchestra.run.smithi076.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-19T04:21:05.883161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":261264,"kb_used_data":11752,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713456,"statfs":{"total":773094113280,"available":772826578944,"internally_reserved":0,"allocated":12034048,"data_stored":6247000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003169"},"pg_stats":[{"pgid":"1.0","version":"109'119","reported_seq":344,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-09-19T04:20:46.217296+0000","last_change":"2024-09-19T04:20:46.217451+0000","last_active":"2024-09-19T04:20:46.217296+0000","last_peered":"2024-09-19T04:20:46.217296+0000","last_clean":"2024-09-19T04:20:46.217296+0000","last_became_active":"2024-09-19T04:13:14.844133+0000","last_became_peered":"2024-09-19T04:13:14.844133+0000","last_unstale":"2024-09-19T04:20:46.217296+0000","last_undegraded":"2024-09-19T04:20:46.217296+0000","last_fullsized":"2024-09-19T04:20:46.217296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'119","last_scrub_stamp":"2024-09-19T04:20:46.217076+0000","last_deep_scrub":"109'119","last_deep_scrub_stamp":"2024-09-19T04:20:46.217076+0000","last_clean_scrub_stamp":"2024-09-19T04:20:46.217076+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T05:49:13.571514+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542241,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1949696,"data_stored":1223651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542241,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339328,"statfs":{"total":96636764160,"available":96603471872,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542242,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32516,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339324,"statfs":{"total":96636764160,"available":96603467776,"internally_reserved":0,"allocated":1355776,"data_stored":633283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542242,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33088,"kb_used_data":1904,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338752,"statfs":{"total":96636764160,"available":96602882048,"internally_reserved":0,"allocated":1949696,"data_stored":1223651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T04:21:06.296 DEBUG:teuthology.orchestra.run.smithi076:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-19T04:21:06.414 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fdddd878640 1 Processor -- start 2024-09-19T04:21:06.415 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fdddd878640 1 -- start start 2024-09-19T04:21:06.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fdddd878640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 0x7fddd815ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.417 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fdddd878640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fddd815f1d0 0x7fddd80adc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fdddd878640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddd80aede0 con 0x7fddd81520a0 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fdddd878640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7fddd8152480 con 0x7fddd815f1d0 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fdddd878640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fddd815fa10 con 0x7fddd8070020 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd6ffd640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7fddd81520a0 0x7fddd806f910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45500/0 (socket says 172.21.15.76:45500) 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd6ffd640 1 -- 172.21.15.76:0/3287065044 learned_addr learned my addr 172.21.15.76:0/3287065044 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddcffff640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 0x7fddd815ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1639908599 0 0) 0x7fddd80aede0 con 0x7fddd81520a0 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fddc4005740 con 0x7fddd81520a0 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd77fe640 1 --2- 172.21.15.76:0/3287065044 >> v2:172.21.15.116:3300/0 conn(0x7fddd815f1d0 0x7fddd80adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2477438360 0 0) 0x7fddc4005740 con 0x7fddd81520a0 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 msgr2=0x7fddd815ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 0x7fddd815ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.418 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 >> v2:172.21.15.116:3300/0 conn(0x7fddd815f1d0 msgr2=0x7fddd80adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 --2- 172.21.15.76:0/3287065044 >> v2:172.21.15.116:3300/0 conn(0x7fddd815f1d0 0x7fddd80adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd8074250 con 0x7fddd81520a0 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7fddc8001680 con 0x7fddd81520a0 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 >> v1:172.21.15.76:6789/0 conn(0x7fddd81520a0 legacy=0x7fddd806f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 0x7fddc4005bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc4006150 0x7fddc4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc4006f60 0x7fddc40073e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fddd815fa10 con 0x7fddc4006150 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fddc400dbe0 con 0x7fddc4006f60 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fddc400dd60 con 0x7fddd8070020 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddcffff640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc4006150 0x7fddc4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd6ffd640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 0x7fddc4005bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.419 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddcffff640 1 -- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 msgr2=0x7fddc4005bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddd77fe640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc4006f60 0x7fddc40073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddcffff640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddd8070020 0x7fddc4005bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddcffff640 1 -- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc4006f60 msgr2=0x7fddc40073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddcffff640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddc4006f60 0x7fddc40073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.420 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.415+0000 7fddcffff640 1 -- 172.21.15.76:0/3287065044 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc4005740 con 0x7fddc4006150 2024-09-19T04:21:06.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddcffff640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc4006150 0x7fddc4006a20 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009a90 tx=0x7fddc001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79c1f87df43c138f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00b2070 con 0x7fddc4006150 2024-09-19T04:21:06.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc00a5620 con 0x7fddc4006150 2024-09-19T04:21:06.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddd67fc640 1 -- 172.21.15.76:0/3287065044 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0009690 con 0x7fddc4006150 2024-09-19T04:21:06.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- 172.21.15.76:0/3287065044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc4006150 msgr2=0x7fddc4006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.424 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 --2- 172.21.15.76:0/3287065044 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddc4006150 0x7fddc4006a20 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009a90 tx=0x7fddc001c9d0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddd77fe640 1 -- 172.21.15.76:0/3287065044 reap_dead start 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- 172.21.15.76:0/3287065044 shutdown_connections 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- 172.21.15.76:0/3287065044 >> 172.21.15.76:0/3287065044 conn(0x7fddd80b1f90 msgr2=0x7fddd80b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- 172.21.15.76:0/3287065044 shutdown_connections 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- 172.21.15.76:0/3287065044 wait complete. 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 Processor -- start 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- start start 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddc4006150 0x7fddd814df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddd815e050 0x7fddd814a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd80b1180 0x7fddd80b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.425 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8073a50 con 0x7fddd815e050 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7fddd8073770 con 0x7fddd80b1180 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fdddd878640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7fddd8073d30 con 0x7fddc4006150 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddcffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddd815e050 0x7fddd814a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddcffff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddd815e050 0x7fddd814a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51000/0 (socket says 172.21.15.76:51000) 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.419+0000 7fddcffff640 1 -- 172.21.15.76:0/3442005045 learned_addr learned my addr 172.21.15.76:0/3442005045 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddcffff640 1 -- 172.21.15.76:0/3442005045 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddc4006150 msgr2=0x7fddd814df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddcffff640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddc4006150 0x7fddd814df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddcffff640 1 -- 172.21.15.76:0/3442005045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd80b1180 msgr2=0x7fddd80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.426 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddcffff640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd80b1180 0x7fddd80b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddcffff640 1 -- 172.21.15.76:0/3442005045 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd814e5c0 con 0x7fddd815e050 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddcffff640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddd815e050 0x7fddd814a8f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fddc001c800 tx=0x7fddc4007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd4ff9640 1 -- 172.21.15.76:0/3442005045 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00bc030 con 0x7fddd815e050 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd4ff9640 1 -- 172.21.15.76:0/3442005045 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddc000f000 con 0x7fddd815e050 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd814e8a0 con 0x7fddd815e050 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd4ff9640 1 -- 172.21.15.76:0/3442005045 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc00b2040 con 0x7fddd815e050 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd814ee60 con 0x7fddd815e050 2024-09-19T04:21:06.427 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd8068e50 con 0x7fddd815e050 2024-09-19T04:21:06.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd4ff9640 1 -- 172.21.15.76:0/3442005045 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7fddc00bf070 con 0x7fddd815e050 2024-09-19T04:21:06.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd4ff9640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddb80678c0 0x7fddb8069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.429 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd4ff9640 1 -- 172.21.15.76:0/3442005045 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7fddc011fc80 con 0x7fddd815e050 2024-09-19T04:21:06.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd6ffd640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddb80678c0 0x7fddb8069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.423+0000 7fddd6ffd640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddb80678c0 0x7fddb8069d80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddc8004880 tx=0x7fddc8001120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.430 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.427+0000 7fddd4ff9640 1 -- 172.21.15.76:0/3442005045 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7fddc00c9070 con 0x7fddd815e050 2024-09-19T04:21:06.586 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.583+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fddd80b4690 con 0x7fddd815e050 2024-09-19T04:21:06.591 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.591+0000 7fddd4ff9640 1 -- 172.21.15.76:0/3442005045 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7fddc0016740 con 0x7fddd815e050 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddb80678c0 msgr2=0x7fddb8069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddb80678c0 0x7fddb8069d80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fddc8004880 tx=0x7fddc8001120 comp rx=0 tx=0).stop 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddd815e050 msgr2=0x7fddd814a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddd815e050 0x7fddd814a8f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fddc001c800 tx=0x7fddc4007a50 comp rx=0 tx=0).stop 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 shutdown_connections 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7fddb80678c0 0x7fddb8069d80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fddd80b1180 0x7fddd80b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7fddd815e050 0x7fddd814a8f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.595 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 --2- 172.21.15.76:0/3442005045 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7fddc4006150 0x7fddd814df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 >> 172.21.15.76:0/3442005045 conn(0x7fddd80b1f90 msgr2=0x7fddd806f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:06.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 shutdown_connections 2024-09-19T04:21:06.596 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.595+0000 7fdddd878640 1 -- 172.21.15.76:0/3442005045 wait complete. 2024-09-19T04:21:06.611 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T04:21:06.611 DEBUG:teuthology.orchestra.run.smithi076:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-19T04:21:06.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.715+0000 7ffb83462640 1 Processor -- start 2024-09-19T04:21:06.718 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.715+0000 7ffb83462640 1 -- start start 2024-09-19T04:21:06.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb83462640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 0x7ffb7c06df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb83462640 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffb7c06e4b0 0x7ffb7c072160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb83462640 1 -- --> v1:172.21.15.76:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb7c06ec60 con 0x7ffb7c1587a0 2024-09-19T04:21:06.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb83462640 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 -- 0x7ffb7c06f870 con 0x7ffb7c06e4b0 2024-09-19T04:21:06.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb83462640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7c06ef40 con 0x7ffb7c0b39a0 2024-09-19T04:21:06.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb811fe640 1 --1- >> v1:172.21.15.76:6789/0 conn(0x7ffb7c1587a0 0x7ffb7c0b3290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.76:6789/0 says I am v1:172.21.15.76:45516/0 (socket says 172.21.15.76:45516) 2024-09-19T04:21:06.721 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb811fe640 1 -- 172.21.15.76:0/1140410093 learned_addr learned my addr 172.21.15.76:0/1140410093 (peer_addr_for_me v1:172.21.15.76:0/0) 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 <== mon.0 v1:172.21.15.76:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1992496837 0 0) 0x7ffb7c06ec60 con 0x7ffb7c1587a0 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 --> v1:172.21.15.76:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb64003530 con 0x7ffb7c1587a0 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb819ff640 1 --2- 172.21.15.76:0/1140410093 >> v2:172.21.15.116:3300/0 conn(0x7ffb7c06e4b0 0x7ffb7c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb809fd640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 0x7ffb7c06df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 <== mon.0 v1:172.21.15.76:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2530891276 0 0) 0x7ffb64003530 con 0x7ffb7c1587a0 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 msgr2=0x7ffb7c06df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 0x7ffb7c06df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 >> v2:172.21.15.116:3300/0 conn(0x7ffb7c06e4b0 msgr2=0x7ffb7c072160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 --2- 172.21.15.76:0/1140410093 >> v2:172.21.15.116:3300/0 conn(0x7ffb7c06e4b0 0x7ffb7c072160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 --> v1:172.21.15.76:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c06f220 con 0x7ffb7c1587a0 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 <== mon.0 v1:172.21.15.76:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2921992612 0 0) 0x7ffb6c001680 con 0x7ffb7c1587a0 2024-09-19T04:21:06.722 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 >> v1:172.21.15.76:6789/0 conn(0x7ffb7c1587a0 legacy=0x7ffb7c0b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 0x7ffb640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb64003ff0 0x7ffb64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb64004d50 0x7ffb640051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c06f870 con 0x7ffb64004d50 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c06ef40 con 0x7ffb64003ff0 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb6400bad0 con 0x7ffb7c0b39a0 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb819ff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb64004d50 0x7ffb640051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb809fd640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb64003ff0 0x7ffb64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb811fe640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 0x7ffb640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb819ff640 1 -- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 msgr2=0x7ffb640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb819ff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb7c0b39a0 0x7ffb640039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.723 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb819ff640 1 -- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb64003ff0 msgr2=0x7ffb64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T04:21:06.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb819ff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb64003ff0 0x7ffb64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb819ff640 1 -- 172.21.15.76:0/1140410093 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb64003530 con 0x7ffb64004d50 2024-09-19T04:21:06.724 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.719+0000 7ffb809fd640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb64003ff0 0x7ffb64004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T04:21:06.726 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb64004d50 0x7ffb640051d0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400ba30 tx=0x7ffb7401c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e289e7280d3cbc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400b600 con 0x7ffb64004d50 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ffb740aa650 con 0x7ffb64004d50 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb73fff640 1 -- 172.21.15.76:0/1140410093 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb740aaa60 con 0x7ffb64004d50 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- 172.21.15.76:0/1140410093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb64004d50 msgr2=0x7ffb640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 --2- 172.21.15.76:0/1140410093 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb64004d50 0x7ffb640051d0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ffb7400ba30 tx=0x7ffb7401c6e0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 -- 172.21.15.76:0/1140410093 reap_dead start 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- 172.21.15.76:0/1140410093 shutdown_connections 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- 172.21.15.76:0/1140410093 >> 172.21.15.76:0/1140410093 conn(0x7ffb7c0b1f90 msgr2=0x7ffb7c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- 172.21.15.76:0/1140410093 shutdown_connections 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- 172.21.15.76:0/1140410093 wait complete. 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 Processor -- start 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- start start 2024-09-19T04:21:06.727 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 --2- >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb64004d50 0x7ffb7c14c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb7c09db40 0x7ffb7c09dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb7c160310 0x7ffb7c160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c158b80 con 0x7ffb7c160310 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7c06e940 con 0x7ffb7c09db40 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- --> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7c14cd30 con 0x7ffb64004d50 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb7c160310 0x7ffb7c160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 --2- >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb7c160310 0x7ffb7c160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.76:3300/0 says I am v2:172.21.15.76:51030/0 (socket says 172.21.15.76:51030) 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 -- 172.21.15.76:0/4089078102 learned_addr learned my addr 172.21.15.76:0/4089078102 (peer_addr_for_me v2:172.21.15.76:0/0) 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 -- 172.21.15.76:0/4089078102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb64004d50 msgr2=0x7ffb7c14c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb64004d50 0x7ffb7c14c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 -- 172.21.15.76:0/4089078102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb7c09db40 msgr2=0x7ffb7c09dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb7c09db40 0x7ffb7c09dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 -- 172.21.15.76:0/4089078102 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7c14fbc0 con 0x7ffb7c160310 2024-09-19T04:21:06.728 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb819ff640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb7c160310 0x7ffb7c160790 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffb7401cbc0 tx=0x7ffb74019480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb71ffb640 1 -- 172.21.15.76:0/4089078102 <== mon.0 v2:172.21.15.76:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7400b600 con 0x7ffb7c160310 2024-09-19T04:21:06.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb71ffb640 1 -- 172.21.15.76:0/4089078102 <== mon.0 v2:172.21.15.76:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7ffb74004020 con 0x7ffb7c160310 2024-09-19T04:21:06.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7c14fea0 con 0x7ffb7c160310 2024-09-19T04:21:06.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb71ffb640 1 -- 172.21.15.76:0/4089078102 <== mon.0 v2:172.21.15.76:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb740043d0 con 0x7ffb7c160310 2024-09-19T04:21:06.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.723+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb7c150460 con 0x7ffb7c160310 2024-09-19T04:21:06.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.727+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7c06eac0 con 0x7ffb7c160310 2024-09-19T04:21:06.729 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.727+0000 7ffb71ffb640 1 -- 172.21.15.76:0/4089078102 <== mon.0 v2:172.21.15.76:3300/0 4 ==== mgrmap(e 5) ==== 87445+0+0 (secure 0 0 0) 0x7ffb740a8630 con 0x7ffb7c160310 2024-09-19T04:21:06.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.727+0000 7ffb71ffb640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb680678c0 0x7ffb68069d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T04:21:06.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.727+0000 7ffb71ffb640 1 -- 172.21.15.76:0/4089078102 <== mon.0 v2:172.21.15.76:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 5507+0+0 (secure 0 0 0) 0x7ffb74120500 con 0x7ffb7c160310 2024-09-19T04:21:06.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.727+0000 7ffb811fe640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb680678c0 0x7ffb68069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T04:21:06.730 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.727+0000 7ffb811fe640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb680678c0 0x7ffb68069d80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001ec0 tx=0x7ffb6c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T04:21:06.731 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.731+0000 7ffb71ffb640 1 -- 172.21.15.76:0/4089078102 <== mon.0 v2:172.21.15.76:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+180342 (secure 0 0 0) 0x7ffb740ae020 con 0x7ffb7c160310 2024-09-19T04:21:06.916 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.915+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 --> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb7c070080 con 0x7ffb7c160310 2024-09-19T04:21:06.917 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.915+0000 7ffb71ffb640 1 -- 172.21.15.76:0/4089078102 <== mon.0 v2:172.21.15.76:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+3918 (secure 0 0 0) 0x7ffb740163f0 con 0x7ffb7c160310 2024-09-19T04:21:06.918 INFO:teuthology.orchestra.run.smithi076.stdout: 2024-09-19T04:21:06.918 INFO:teuthology.orchestra.run.smithi076.stdout:{"epoch":114,"btime":"2024-09-19T04:20:35:990928+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6041,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6835/2612434451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2612434451},{"type":"v1","addr":"172.21.15.116:6835","nonce":2612434451}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114},{"gid":6124,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/2453624779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2453624779},{"type":"v1","addr":"172.21.15.116:6833","nonce":2453624779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":98},{"gid":8970,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6835/116354661","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6834","nonce":116354661},{"type":"v1","addr":"172.21.15.76:6835","nonce":116354661}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114},{"gid":8973,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.76:6837/1710984462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.76:6836","nonce":1710984462},{"type":"v1","addr":"172.21.15.76:6837","nonce":1710984462}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[]} 2024-09-19T04:21:06.918 INFO:teuthology.orchestra.run.smithi076.stderr:dumped fsmap epoch 114 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb680678c0 msgr2=0x7ffb68069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb680678c0 0x7ffb68069d80 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c001ec0 tx=0x7ffb6c009040 comp rx=0 tx=0).stop 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb7c160310 msgr2=0x7ffb7c160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb7c160310 0x7ffb7c160790 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7ffb7401cbc0 tx=0x7ffb74019480 comp rx=0 tx=0).stop 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 shutdown_connections 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:6832/3111825053,v1:172.21.15.76:6833/3111825053] conn(0x7ffb680678c0 0x7ffb68069d80 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.76:3300/0,v1:172.21.15.76:6789/0] conn(0x7ffb7c160310 0x7ffb7c160790 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffb7c09db40 0x7ffb7c09dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 --2- 172.21.15.76:0/4089078102 >> [v2:172.21.15.116:3301/0,v1:172.21.15.116:6790/0] conn(0x7ffb64004d50 0x7ffb7c14c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 >> 172.21.15.76:0/4089078102 conn(0x7ffb7c0b1f90 msgr2=0x7ffb7c0d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T04:21:06.920 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 shutdown_connections 2024-09-19T04:21:06.921 INFO:teuthology.orchestra.run.smithi076.stderr:2024-09-19T04:21:06.919+0000 7ffb83462640 1 -- 172.21.15.76:0/4089078102 wait complete. 2024-09-19T04:21:06.935 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-19T04:21:06.936 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-19T04:21:06.936 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-19T04:21:06.936 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:06.945 INFO:tasks.ceph.mds.a:Stopped 2024-09-19T04:21:06.945 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-19T04:21:06.945 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.002 INFO:tasks.ceph.mds.c:Stopped 2024-09-19T04:21:07.002 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-19T04:21:07.002 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.015 INFO:tasks.ceph.mds.b:Stopped 2024-09-19T04:21:07.015 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-19T04:21:07.016 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.070 INFO:tasks.ceph.mds.d:Stopped 2024-09-19T04:21:07.070 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-19T04:21:07.070 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-19T04:21:07.070 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.147 INFO:tasks.ceph.osd.0:Stopped 2024-09-19T04:21:07.147 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-19T04:21:07.148 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.234 INFO:tasks.ceph.osd.1:Stopped 2024-09-19T04:21:07.234 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-19T04:21:07.234 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.321 INFO:tasks.ceph.osd.2:Stopped 2024-09-19T04:21:07.321 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-19T04:21:07.321 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.403 INFO:tasks.ceph.osd.3:Stopped 2024-09-19T04:21:07.404 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-19T04:21:07.404 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:07.496 INFO:tasks.ceph.osd.4:Stopped 2024-09-19T04:21:07.496 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-19T04:21:07.496 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:10.828 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-19T04:21:13.598 INFO:tasks.ceph.osd.5:Stopped 2024-09-19T04:21:13.599 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-19T04:21:13.599 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:13.672 INFO:tasks.ceph.osd.6:Stopped 2024-09-19T04:21:13.672 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-19T04:21:13.672 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:13.747 INFO:tasks.ceph.osd.7:Stopped 2024-09-19T04:21:13.748 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-19T04:21:13.748 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-19T04:21:13.748 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:13.778 INFO:tasks.ceph.mgr.y:Stopped 2024-09-19T04:21:13.778 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-19T04:21:13.778 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:13.806 INFO:tasks.ceph.mgr.x:Stopped 2024-09-19T04:21:13.806 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-19T04:21:13.806 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-19T04:21:13.806 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:13.839 INFO:tasks.ceph.mon.a:Stopped 2024-09-19T04:21:13.839 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-19T04:21:13.839 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:13.860 INFO:tasks.ceph.mon.b:Stopped 2024-09-19T04:21:13.861 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-19T04:21:13.861 INFO:teuthology.orchestra.run:waiting for 300 2024-09-19T04:21:13.927 INFO:tasks.ceph.mon.c:Stopped 2024-09-19T04:21:13.927 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-19T04:21:13.927 DEBUG:teuthology.orchestra.run.smithi076:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'Scrub error on inode' | head -n 1 2024-09-19T04:21:13.946 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:21:13.947 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-19T04:21:14.305 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:21:14.305 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-19T04:21:14.533 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:21:14.533 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-19T04:21:14.706 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:21:14.706 DEBUG:teuthology.orchestra.run.smithi076:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-19T04:21:14.881 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:21:14.882 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-19T04:21:15.117 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:21:15.117 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-19T04:21:15.301 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:21:15.302 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-19T04:21:15.483 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:21:15.484 DEBUG:teuthology.orchestra.run.smithi116:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-19T04:21:15.642 INFO:tasks.ceph:Archiving mon data... 2024-09-19T04:21:15.644 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/data/mon.a.tgz 2024-09-19T04:21:15.644 DEBUG:teuthology.orchestra.run.smithi076:> mktemp 2024-09-19T04:21:15.650 INFO:teuthology.orchestra.run.smithi076.stdout:/tmp/tmp.dX6TZOVVh7 2024-09-19T04:21:15.650 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.dX6TZOVVh7 2024-09-19T04:21:15.803 DEBUG:teuthology.orchestra.run.smithi076:> sudo chmod 0666 /tmp/tmp.dX6TZOVVh7 2024-09-19T04:21:15.829 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-19T04:21:15.866 DEBUG:teuthology.orchestra.remote:smithi076:/tmp/tmp.dX6TZOVVh7 is 450KB 2024-09-19T04:21:15.928 DEBUG:teuthology.orchestra.run.smithi076:> rm -fr /tmp/tmp.dX6TZOVVh7 2024-09-19T04:21:15.934 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/data/mon.b.tgz 2024-09-19T04:21:15.934 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-09-19T04:21:15.939 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.AQOX4BBrCB 2024-09-19T04:21:15.940 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.AQOX4BBrCB 2024-09-19T04:21:16.088 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.AQOX4BBrCB 2024-09-19T04:21:16.153 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.AQOX4BBrCB is 456KB 2024-09-19T04:21:16.215 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.AQOX4BBrCB 2024-09-19T04:21:16.220 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/data/mon.c.tgz 2024-09-19T04:21:16.220 DEBUG:teuthology.orchestra.run.smithi116:> mktemp 2024-09-19T04:21:16.269 INFO:teuthology.orchestra.run.smithi116.stdout:/tmp/tmp.5buzzPtI38 2024-09-19T04:21:16.269 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.5buzzPtI38 2024-09-19T04:21:16.421 DEBUG:teuthology.orchestra.run.smithi116:> sudo chmod 0666 /tmp/tmp.5buzzPtI38 2024-09-19T04:21:16.480 DEBUG:teuthology.orchestra.remote:smithi116:/tmp/tmp.5buzzPtI38 is 456KB 2024-09-19T04:21:16.543 DEBUG:teuthology.orchestra.run.smithi116:> rm -fr /tmp/tmp.5buzzPtI38 2024-09-19T04:21:16.548 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-19T04:21:16.548 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T04:21:16.550 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-19T04:21:16.610 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T04:21:16.648 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-19T04:21:16.649 INFO:tasks.ceph:Archiving crash dumps... 2024-09-19T04:21:16.650 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/remote/smithi076/crash 2024-09-19T04:21:16.651 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T04:21:16.667 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/remote/smithi116/crash 2024-09-19T04:21:16.667 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-19T04:21:16.705 INFO:tasks.ceph:Shutting down logrotate 2024-09-19T04:21:16.705 DEBUG:teuthology.orchestra.run.smithi076:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:21:16.711 DEBUG:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-19T04:21:16.801 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T04:21:16.815 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-19T04:21:16.827 INFO:tasks.ceph:Compressing logs... 2024-09-19T04:21:16.828 DEBUG:teuthology.orchestra.run.smithi076:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:21:16.859 DEBUG:teuthology.orchestra.run.smithi116:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T04:21:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35536.log 2024-09-19T04:21:16.875 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35035.log 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36705.log 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35035.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35536.log.gz 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45876.log 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35035.log.gz 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50319.log 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37065.log 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36705.log.gz 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45876.log.gz 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45034.log 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50319.log.gz 2024-09-19T04:21:16.878 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37065.log: -- /var/log/ceph/ceph-client.admin.43161.log 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41811.log 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37065.log.gz 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51235.log 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45034.log.gz 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.43161.log.gz 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44484.log 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41811.log.gz 2024-09-19T04:21:16.882 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38633.log 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51235.log.gz 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44484.log.gz 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38324.log 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38633.log: /var/log/ceph/ceph-client.admin.43972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35432.log 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37195.log 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38324.log: /var/log/ceph/ceph-client.admin.35432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43672.log 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38324.log.gz 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37195.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35432.log.gz 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40144.log 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37195.log.gz 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38633.log.gz 2024-09-19T04:21:16.883 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35380.log/var/log/ceph/ceph-client.admin.43672.log: 2024-09-19T04:21:16.891 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45910.log 2024-09-19T04:21:16.891 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43672.log.gz 2024-09-19T04:21:16.891 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35380.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40144.log.gz 2024-09-19T04:21:16.891 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35380.log.gz 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51829.log 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45910.log.gz 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50215.log 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47236.log 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51829.log.gz 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39204.log 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50215.log.gz 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36595.log 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44722.log 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39204.log.gz 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.2.36595.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47236.log.gz 2024-09-19T04:21:16.892 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49692.log 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40615.log 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.2.36595.log.gz 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44722.log.gz 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41655.log 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39739.log 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49692.log.gz 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41655.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40615.log.gz 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45603.log 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41655.log.gz 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39092.log 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45603.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39739.log.gz 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40771.log 2024-09-19T04:21:16.893 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45603.log.gz 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39092.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39092.log.gz 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47117.log 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.0.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.40771.log.gz 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42732.log 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48470.log 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.47117.log.gz 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36108.log 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40068.log 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42732.log.gz 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0%/var/log/ceph/ceph-client.admin.36108.log: -- replaced with /var/log/ceph/ceph-client.admin.48470.log.gz 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44024.log 2024-09-19T04:21:16.894 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36108.log.gz 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47922.log 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40068.log.gz 2024-09-19T04:21:16.895 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35694.log 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28971.log 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.28971.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi116.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28971.log.gz 2024-09-19T04:21:16.908 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29115.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34558.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.29115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29019.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29115.log.gz 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.1.34558.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.29019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29043.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29019.log.gz 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.1.34558.log.gz 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.29043.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29043.log.gz 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29091.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28995.log 2024-09-19T04:21:16.909 INFO:teuthology.orchestra.run.smithi116.stderr:gzip/var/log/ceph/ceph-client.admin.29091.log: -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.28995.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29091.log.gz 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28995.log.gz 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.34679.log 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.3.34679.log: 92.3% 88.5% -- replaced with /var/log/ceph/ceph-client.3.34679.log.gz 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29139.log 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-client.admin.29139.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29139.log.gz 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-19T04:21:16.910 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-mon.b.log: 94.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-19T04:21:16.913 INFO:teuthology.orchestra.run.smithi116.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29067.log 2024-09-19T04:21:16.927 INFO:teuthology.orchestra.run.smithi116.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.29067.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29067.log.gz 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44024.log: /var/log/ceph/ceph-client.admin.47922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48522.log 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44024.log.gz 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47922.log.gz 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39452.log 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35694.log.gz 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39126.log 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48522.log.gz 2024-09-19T04:21:16.945 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.39452.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38995.log 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39126.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39452.log.gz 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49360.log 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39126.log.gz 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44263.log 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.49360.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47694.log 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.38995.log.gz 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44263.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.49360.log.gz 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51600.log 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44263.log.gz 2024-09-19T04:21:16.946 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47694.log: -- /var/log/ceph/ceph-client.admin.37820.log 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34877.log 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51600.log.gz 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49139.log 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.47694.log.gz 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37820.log.gz 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37117.log 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34877.log.gz 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46033.log 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37117.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49139.log.gz 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41213.log 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37117.log.gz 2024-09-19T04:21:16.947 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44057.log 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46033.log.gz 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38683.log 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50241.log 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44057.log.gz 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41870.log 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41213.log.gz 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47462.log 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38683.log.gz 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41870.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50241.log.gz 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40727.log 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41870.log.gz 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34784.log 2024-09-19T04:21:16.948 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47462.log.gz 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40727.log: /var/log/ceph/ceph-client.admin.43776.log 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34784.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.40727.log.gz 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38138.log 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36894.log 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr: 87.4%/var/log/ceph/ceph-client.admin.43776.log: -- replaced with /var/log/ceph/ceph-client.admin.34784.log.gz 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38138.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43776.log.gz 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51131.log 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38138.log.gzgzip 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.39687.log 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36894.log: /var/log/ceph/ceph-client.admin.51131.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36894.log.gz 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50849.log 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51131.log.gz 2024-09-19T04:21:16.949 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49391.log 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39687.log: /var/log/ceph/ceph-client.admin.50849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46233.log 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39687.log.gz 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35484.log 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49391.log.gz 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.50849.log.gz/var/log/ceph/ceph-client.admin.46233.log: 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38464.log 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46233.log.gz 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39152.log 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35484.log: /var/log/ceph/ceph-client.admin.38464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45210.log 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39152.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35484.log.gz 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42642.log 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39152.log.gz 2024-09-19T04:21:16.950 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.38464.log.gz 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42832.log 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42323.log 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42642.log.gz 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42832.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.45210.log.gz 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45525.log 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40094.log 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42832.log.gz 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45525.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42323.log.gz 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50345.log 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40094.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45525.log.gz 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41565.log 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34905.log 2024-09-19T04:21:16.951 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.40094.log.gz 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.50345.log.gz 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38086.log 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34905.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41565.log.gz 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41968.log 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34905.log.gz 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44298.log 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38086.log.gz 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49826.log 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41968.log.gz 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44536.log 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44298.log.gz 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37651.log 2024-09-19T04:21:16.952 INFO:teuthology.orchestra.run.smithi076.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.49826.log.gz/var/log/ceph/ceph-client.admin.44536.log: 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45850.log 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44536.log.gz 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35458.log 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37731.log 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37651.log.gz 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45850.log.gz 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47073.log 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35458.log.gz 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51027.log 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37731.log: /var/log/ceph/ceph-client.admin.47073.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37731.log.gz 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41785.log 2024-09-19T04:21:16.953 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47073.log.gz 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51354.log 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51027.log: /var/log/ceph/ceph-client.admin.41785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41513.log 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41785.log.gz 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51027.log.gz 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39504.log 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51354.log.gz 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37980.log 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41513.log: /var/log/ceph/ceph-client.admin.39504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35137.log 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41513.log.gz 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37980.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39504.log.gz 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37980.log.gz 2024-09-19T04:21:16.954 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38414.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35137.log: /var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52089.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35137.log.gz 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37008.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46767.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.38414.log.gz 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37008.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.52089.log.gz 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42137.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46767.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.37008.log.gz 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39621.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51157.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.46767.log.gz 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39621.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39374.log 2024-09-19T04:21:16.955 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42137.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39621.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34931.log 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34636.log 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39374.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.51157.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49870.log 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39374.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34931.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52037.log 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34636.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.49870.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28875.log 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52037.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-09-19T04:21:16.956 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51751.log 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52037.log.gz 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.28875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36861.log 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51751.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.28875.log.gz 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48660.log 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4%/var/log/ceph/ceph-client.admin.36861.log: -- replaced with /var/log/ceph/ceph-client.admin.51751.log.gz 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45499.log 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36861.log.gz 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46415.log 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38564.log 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2%/var/log/ceph/ceph-client.admin.45499.log: -- replaced with /var/log/ceph/ceph-client.admin.48660.log.gz 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41163.log 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45499.log.gz 2024-09-19T04:21:16.957 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50039.log 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38564.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.46415.log.gz 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45746.log 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38564.log.gz 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50039.log: /var/log/ceph/ceph-client.admin.45746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50899.log 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41163.log.gz 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45746.log.gz 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose 91.0% -- /var/log/ceph/ceph-client.admin.47870.log -- replaced with /var/log/ceph/ceph-client.admin.50039.log.gz 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47744.log 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42706.log 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47744.log: gzip -5 --verbose -- 91.1% /var/log/ceph/ceph-client.admin.38951.log 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50899.log.gz 2024-09-19T04:21:16.958 INFO:teuthology.orchestra.run.smithi076.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47870.log.gz 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42706.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42706.log.gz 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.47744.log.gz 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42955.log 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49061.log 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42782.log 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38951.log.gz 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35588.log 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42782.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49061.log.gz 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48132.log 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.42955.log.gz 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49165.log/var/log/ceph/ceph-client.admin.35588.log: 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48132.log: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40016.log 2024-09-19T04:21:16.959 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35588.log.gz 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42782.log.gz 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48132.log.gz 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49165.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49165.log.gz 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42048.log 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52115.log 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40016.log.gz 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46612.log 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35692.log 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52115.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42048.log.gz 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39860.log 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52115.log.gz 2024-09-19T04:21:16.960 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46612.log.gz 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34983.log 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35692.log: /var/log/ceph/ceph-client.admin.39860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44612.log 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0%/var/log/ceph/ceph-client.admin.34983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44380.log 88.9% 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39860.log.gz 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35692.log.gz 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34983.log.gz 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51380.log 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44612.log.gz 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47976.log 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44380.log.gz 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51985.log 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47976.log: 89.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48757.log 2024-09-19T04:21:16.961 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51380.log.gz 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41681.log 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47976.log.gz 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48757.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51985.log.gz 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49087.log 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48838.log 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48757.log.gz 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41681.log: /var/log/ceph/ceph-client.admin.49087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40254.log 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41681.log.gz 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49087.log.gz 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41341.log 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36809.log 2024-09-19T04:21:16.962 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48838.log.gz 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40254.log: /var/log/ceph/ceph-client.admin.41341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48927.log 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40254.log.gz 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41341.log.gz 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36809.log: /var/log/ceph/ceph-client.admin.48927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41289.log 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42454.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48927.log.gz 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51725.log 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36809.log.gz 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41289.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39767.log 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41289.log.gz 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40042.log 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51725.log: /var/log/ceph/ceph-client.admin.39767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-09-19T04:21:16.963 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51725.log.gz 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39767.log.gz 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46889.log 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40042.log.gz 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37396.log 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46889.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50497.log 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39017.log 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41539.log 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37396.log.gz 90.9% -- replaced with /var/log/ceph/ceph-client.admin.46889.log.gz 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50497.log: /var/log/ceph/ceph-client.admin.39017.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.39017.log.gz 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.50497.log.gz 2024-09-19T04:21:16.964 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42882.log 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51671.log 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42882.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41539.log.gz 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45084.log 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51671.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42882.log.gz 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43646.log 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45084.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36954.log 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51671.log.gz 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46151.log 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr: 90.8% 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43646.log.gz 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51183.log 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45084.log.gz 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36954.log: /var/log/ceph/ceph-client.admin.46151.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36954.log.gz 2024-09-19T04:21:16.965 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46151.log.gz 2024-09-19T04:21:16.966 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39426.log 2024-09-19T04:21:16.986 INFO:teuthology.orchestra.run.smithi116.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-19T04:21:16.987 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35323.log 2024-09-19T04:21:16.987 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39426.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.51183.log.gz 2024-09-19T04:21:16.987 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36980.log 2024-09-19T04:21:16.987 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35323.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39426.log.gz 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36980.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35323.log.gz 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42076.log 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43252.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36980.log.gz 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42297.log 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40200.log 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42297.log: 91.1% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42076.log.gz 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz --verbose 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.46641.log 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42297.log.gz 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50697.log 2024-09-19T04:21:16.988 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.40200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41063.log 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46641.log: /var/log/ceph/ceph-client.admin.50697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36500.log 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46641.log.gz 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41063.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50697.log.gz 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41063.log.gz 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37928.log 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37846.log 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37928.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36500.log.gz 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47051.log 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40749.log 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37928.log.gz 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37846.log: /var/log/ceph/ceph-client.admin.47051.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37846.log.gz 2024-09-19T04:21:16.989 INFO:teuthology.orchestra.run.smithi076.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47051.log.gz 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37169.log 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52141.log 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48866.log 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.40749.log.gz 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37169.log.gz 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52141.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52141.log.gz 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49517.log 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35562.log 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3%/var/log/ceph/ceph-client.admin.49517.log: -- replaced with /var/log/ceph/ceph-client.admin.48866.log.gz 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46521.log 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35562.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35562.log.gz 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.49517.log.gz 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-09-19T04:21:16.990 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.46521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42530.log 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50597.log 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42530.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42530.log.gz 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51105.log 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37703.log 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51105.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50597.log.gz 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51105.log.gz 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr: 94.1%gzip -- replaced with /var/log/ceph/ceph-client.0.40200.log.gz -5 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50747.log 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28984.log 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37703.log: /var/log/ceph/ceph-client.admin.50747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39061.log 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37703.log.gz 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50747.log.gz 2024-09-19T04:21:16.991 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.28984.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28984.log.gz 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46337.log 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39061.log: /var/log/ceph/ceph-client.admin.45259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50397.log 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46337.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39061.log.gz 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46337.log.gz 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38798.log 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35350.log 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50397.log: /var/log/ceph/ceph-client.admin.38798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44154.log 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35350.log: 91.1% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35350.log.gz 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38798.log.gz 2024-09-19T04:21:16.992 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50397.log.gz 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44235.log 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44154.log: /var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42271.log 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44235.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44154.log.gz 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35688.log 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44235.log.gz 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.46521.log.gz 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42271.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42271.log.gz 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38904.log 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48328.log 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38904.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41263.log 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35688.log.gz 2024-09-19T04:21:16.993 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48328.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.38904.log.gz 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48328.log.gz 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43560.log 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36835.log 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41263.log.gz 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35406.log 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43560.log.gz 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36835.log: /var/log/ceph/ceph-client.admin.43454.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36835.log.gz 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43698.log 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35406.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47592.log 2024-09-19T04:21:16.994 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44458.log 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35406.log.gz 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43698.log: /var/log/ceph/ceph-client.admin.47592.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43698.log.gz 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47592.log.gz 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48957.log 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39178.log 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45798.log 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44458.log.gz 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48957.log.gz 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39178.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39178.log.gz 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34754.log 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35640.log 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34754.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45798.log.gz 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36186.log 2024-09-19T04:21:16.995 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34754.log.gz 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35640.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35640.log.gz 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38006.log 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41759.log 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36186.log.gz/var/log/ceph/ceph-client.admin.38006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48276.log 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38006.log.gz 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34957.log 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48276.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41759.log.gz 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48276.log.gz 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35087.log 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44813.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34957.log.gz 2024-09-19T04:21:16.996 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45357.log 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51777.log 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35087.log: /var/log/ceph/ceph-client.admin.45357.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.35087.log.gz 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.45357.log.gz 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43117.log 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47095.log 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51777.log: /var/log/ceph/ceph-client.admin.43117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37299.log 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr: 88.0%/var/log/ceph/ceph-client.admin.47095.log: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.43117.log.gz 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51777.log.gz 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.47095.log.gz 2024-09-19T04:21:16.997 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43404.log 2024-09-19T04:21:17.000 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37299.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-19T04:21:17.000 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43404.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49035.log 2024-09-19T04:21:17.000 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37299.log.gz 2024-09-19T04:21:17.000 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-osd.1.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.43404.log.gz 2024-09-19T04:21:17.001 INFO:teuthology.orchestra.run.smithi076.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-19T04:21:17.001 INFO:teuthology.orchestra.run.smithi116.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-19T04:21:17.006 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36673.log 2024-09-19T04:21:17.006 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35691.log 2024-09-19T04:21:17.006 INFO:teuthology.orchestra.run.smithi076.stderr:gzip/var/log/ceph/ceph-client.admin.36673.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50139.log 2024-09-19T04:21:17.007 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49035.log.gz 2024-09-19T04:21:17.007 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2%/var/log/ceph/ceph-client.admin.35691.log: -- replaced with /var/log/ceph/ceph-client.admin.36673.log.gz 2024-09-19T04:21:17.008 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37599.log 2024-09-19T04:21:17.008 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35691.log.gz 2024-09-19T04:21:17.010 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48002.log 2024-09-19T04:21:17.010 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34847.log 2024-09-19T04:21:17.010 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37599.log.gz 2024-09-19T04:21:17.010 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48002.log: -- replaced with /var/log/ceph/ceph-client.admin.50139.log.gz 2024-09-19T04:21:17.010 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48600.log 2024-09-19T04:21:17.010 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48002.log.gz 2024-09-19T04:21:17.010 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39256.log 2024-09-19T04:21:17.011 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48600.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34847.log.gz 2024-09-19T04:21:17.011 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46939.log 2024-09-19T04:21:17.011 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35690.log 2024-09-19T04:21:17.011 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48600.log.gz 2024-09-19T04:21:17.013 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39256.log: /var/log/ceph/ceph-client.admin.46939.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41733.log 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39256.log.gz 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48164.log 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46939.log.gz 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35690.log.gz 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41733.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41733.log.gz 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46116.log 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37428.log 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46116.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48164.log.gz 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46989.log 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:gzip 88.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.46116.log.gz 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41487.log 2024-09-19T04:21:17.014 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46989.log: /var/log/ceph/ceph-client.admin.37428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50773.log 2024-09-19T04:21:17.017 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41487.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37428.log.gz 2024-09-19T04:21:17.017 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50089.log 2024-09-19T04:21:17.017 INFO:teuthology.orchestra.run.smithi076.stderr: 91.6% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41487.log.gz 2024-09-19T04:21:17.017 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46989.log.gz 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50773.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.50773.log.gz 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36783.log 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38112.log 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37247.log 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr: 91.2%/var/log/ceph/ceph-client.admin.38112.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36783.log.gz 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50089.log.gz 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38112.log.gz 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48354.log 2024-09-19T04:21:17.018 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40815.log 2024-09-19T04:21:17.021 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48354.log: 87.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36238.log 2024-09-19T04:21:17.021 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37247.log.gz 2024-09-19T04:21:17.021 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40815.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48354.log.gz 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48444.log 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36212.log 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48444.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40815.log.gz 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36238.log.gz 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43354.log 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48444.log.gz 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35009.log 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36212.log: /var/log/ceph/ceph-client.admin.43354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47488.log 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35009.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36212.log.gz 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36264.log 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35009.log.gz 2024-09-19T04:21:17.022 INFO:teuthology.orchestra.run.smithi076.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.43354.log.gz 2024-09-19T04:21:17.024 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47488.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47488.log.gz 2024-09-19T04:21:17.024 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47336.log 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45551.log 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44324.log 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45551.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.36264.log.gz 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47336.log.gz 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45551.log.gz 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38733.log 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40665.log 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38733.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44324.log.gz 2024-09-19T04:21:17.025 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49271.log 2024-09-19T04:21:17.027 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39400.log 2024-09-19T04:21:17.027 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40665.log: /var/log/ceph/ceph-client.0.49271.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.38733.log.gz 2024-09-19T04:21:17.027 INFO:teuthology.orchestra.run.smithi076.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.40665.log.gz 2024-09-19T04:21:17.027 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50949.log 2024-09-19T04:21:17.027 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38164.log 2024-09-19T04:21:17.028 INFO:teuthology.orchestra.run.smithi076.stderr: 89.4%/var/log/ceph/ceph-client.admin.50949.log: -- replaced with /var/log/ceph/ceph-client.admin.39400.log.gz 2024-09-19T04:21:17.028 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50949.log.gz 2024-09-19T04:21:17.028 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48198.log 2024-09-19T04:21:17.029 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48809.log 2024-09-19T04:21:17.029 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38164.log: /var/log/ceph/ceph-client.admin.48198.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48198.log.gz 2024-09-19T04:21:17.030 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38164.log.gz 2024-09-19T04:21:17.030 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38214.log 2024-09-19T04:21:17.030 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41113.log 2024-09-19T04:21:17.031 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48809.log: /var/log/ceph/ceph-client.admin.38214.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38214.log.gz 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48809.log.gz 2024-09-19T04:21:17.031 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:17.031 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52193.log 2024-09-19T04:21:17.032 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42375.log 2024-09-19T04:21:17.033 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41113.log: /var/log/ceph/ceph-client.admin.52193.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.52193.log.gz 2024-09-19T04:21:17.033 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.41113.log.gz 2024-09-19T04:21:17.033 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45824.log 2024-09-19T04:21:17.033 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39886.log 2024-09-19T04:21:17.034 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42375.log: /var/log/ceph/ceph-client.admin.45824.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42375.log.gz 2024-09-19T04:21:17.034 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45824.log.gz 2024-09-19T04:21:17.034 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48380.log 2024-09-19T04:21:17.034 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47186.log 2024-09-19T04:21:17.036 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39886.log: /var/log/ceph/ceph-client.admin.48380.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39886.log.gz 2024-09-19T04:21:17.036 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48380.log.gz 2024-09-19T04:21:17.036 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51302.log 2024-09-19T04:21:17.036 INFO:teuthology.orchestra.run.smithi076.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.49271.log.gz 2024-09-19T04:21:17.037 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47186.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47186.log.gz 2024-09-19T04:21:17.037 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51328.log 2024-09-19T04:21:17.038 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35897.log 2024-09-19T04:21:17.038 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51328.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51484.log 2024-09-19T04:21:17.038 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51302.log.gz 2024-09-19T04:21:17.038 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9%/var/log/ceph/ceph-client.admin.35897.log: -- replaced with /var/log/ceph/ceph-client.admin.51328.log.gz 2024-09-19T04:21:17.039 INFO:teuthology.orchestra.run.smithi076.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35897.log.gz 2024-09-19T04:21:17.039 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47644.log 2024-09-19T04:21:17.039 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48574.log 2024-09-19T04:21:17.040 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9%/var/log/ceph/ceph-client.admin.47644.log: -- replaced with /var/log/ceph/ceph-client.admin.51484.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50371.log 2024-09-19T04:21:17.040 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:17.040 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48574.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48574.log.gz 2024-09-19T04:21:17.040 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47644.log.gz 2024-09-19T04:21:17.041 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-09-19T04:21:17.041 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50371.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-19T04:21:17.041 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37902.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.50371.log.gz 2024-09-19T04:21:17.041 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43205.log 2024-09-19T04:21:17.041 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47161.log 2024-09-19T04:21:17.041 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-09-19T04:21:17.042 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43205.log: /var/log/ceph/ceph.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43205.log.gz 2024-09-19T04:21:17.043 INFO:teuthology.orchestra.run.smithi076.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-19T04:21:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36325.log 2024-09-19T04:21:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44406.log 2024-09-19T04:21:17.044 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36134.log 2024-09-19T04:21:17.044 INFO:teuthology.orchestra.run.smithi076.stderr: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.47161.log.gz 2024-09-19T04:21:17.045 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36325.log.gz 2024-09-19T04:21:17.045 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44406.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44406.log.gz 2024-09-19T04:21:17.045 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45447.log 2024-09-19T04:21:17.046 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37344.log 2024-09-19T04:21:17.046 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45447.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45637.log 2024-09-19T04:21:17.047 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36134.log.gz 2024-09-19T04:21:17.047 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45447.log.gz 2024-09-19T04:21:17.047 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37344.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37344.log.gz 2024-09-19T04:21:17.048 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48224.log 2024-09-19T04:21:17.048 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52011.log 2024-09-19T04:21:17.048 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4%/var/log/ceph/ceph-client.admin.48224.log: -- replaced with /var/log/ceph/ceph-client.admin.45637.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39964.log 2024-09-19T04:21:17.048 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:17.048 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48224.log.gz 2024-09-19T04:21:17.048 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.52011.log.gz 2024-09-19T04:21:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46817.log 2024-09-19T04:21:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40862.log 2024-09-19T04:21:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46817.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39964.log.gz 2024-09-19T04:21:17.049 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50189.log 2024-09-19T04:21:17.050 INFO:teuthology.orchestra.run.smithi076.stderr:gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.46817.log.gz --verbose 2024-09-19T04:21:17.051 INFO:teuthology.orchestra.run.smithi076.stderr: -- /var/log/ceph/ceph-client.admin.37546.log 2024-09-19T04:21:17.051 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40862.log: /var/log/ceph/ceph-client.admin.50189.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.50189.log.gz 2024-09-19T04:21:17.051 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40862.log.gz 2024-09-19T04:21:17.052 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48250.log 2024-09-19T04:21:17.052 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44562.log 2024-09-19T04:21:17.052 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7%/var/log/ceph/ceph-client.admin.48250.log: -- replaced with /var/log/ceph/ceph-client.admin.37546.log.gzgzip 2024-09-19T04:21:17.052 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46177.log 2024-09-19T04:21:17.052 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48250.log.gz 2024-09-19T04:21:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44562.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.44562.log.gz 2024-09-19T04:21:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46259.log 2024-09-19T04:21:17.053 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51548.log 2024-09-19T04:21:17.054 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7%/var/log/ceph/ceph-client.admin.46259.log: -- replaced with /var/log/ceph/ceph-client.admin.46177.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35682.log 2024-09-19T04:21:17.054 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:17.054 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46259.log.gz 2024-09-19T04:21:17.054 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51548.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51548.log.gz 2024-09-19T04:21:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2024-09-19T04:21:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-09-19T04:21:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42402.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35682.log.gz 2024-09-19T04:21:17.055 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48054.log 2024-09-19T04:21:17.056 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37520.log 2024-09-19T04:21:17.056 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2024-09-19T04:21:17.057 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47286.log: /var/log/ceph/ceph-client.admin.48054.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48054.log.gz 2024-09-19T04:21:17.057 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-09-19T04:21:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46007.log 2024-09-19T04:21:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40344.log 2024-09-19T04:21:17.058 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37520.log.gz 2024-09-19T04:21:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51881.log 2024-09-19T04:21:17.058 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46007.log: /var/log/ceph/ceph-client.admin.40344.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46007.log.gz 2024-09-19T04:21:17.059 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40344.log.gz 2024-09-19T04:21:17.059 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42588.log 2024-09-19T04:21:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39318.log 2024-09-19T04:21:17.060 INFO:teuthology.orchestra.run.smithi076.stderr: 87.6%/var/log/ceph/ceph-client.0.42588.log: -- replaced with /var/log/ceph/ceph-client.admin.51881.log.gzgzip 2024-09-19T04:21:17.060 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37143.log 2024-09-19T04:21:17.060 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39318.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39318.log.gz 2024-09-19T04:21:17.061 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50267.log 2024-09-19T04:21:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49642.log 2024-09-19T04:21:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50267.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37143.log.gz 2024-09-19T04:21:17.062 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.50267.log.gz 2024-09-19T04:21:17.062 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47794.log 2024-09-19T04:21:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42219.log 2024-09-19T04:21:17.063 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47794.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49642.log.gz 2024-09-19T04:21:17.063 INFO:teuthology.orchestra.run.smithi076.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47794.log.gz 2024-09-19T04:21:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48626.log 2024-09-19T04:21:17.064 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48496.log 2024-09-19T04:21:17.065 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5%/var/log/ceph/ceph-client.admin.48626.log: -- replaced with /var/log/ceph/ceph-client.admin.42219.log.gz 2024-09-19T04:21:17.065 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48626.log.gz 2024-09-19T04:21:17.066 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45413.log 2024-09-19T04:21:17.066 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37273.log 2024-09-19T04:21:17.066 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6%/var/log/ceph/ceph-client.admin.45413.log: -- replaced with /var/log/ceph/ceph-client.admin.48496.log.gz 2024-09-19T04:21:17.066 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45413.log.gz 2024-09-19T04:21:17.067 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37494.log 2024-09-19T04:21:17.067 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36731.log 2024-09-19T04:21:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37494.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37273.log.gz 2024-09-19T04:21:17.068 INFO:teuthology.orchestra.run.smithi076.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37494.log.gz 2024-09-19T04:21:17.068 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49009.log 2024-09-19T04:21:17.069 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38032.log 2024-09-19T04:21:17.069 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3%/var/log/ceph/ceph-client.admin.49009.log: -- replaced with /var/log/ceph/ceph-client.admin.36731.log.gz 2024-09-19T04:21:17.069 INFO:teuthology.orchestra.run.smithi076.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49009.log.gz 2024-09-19T04:21:17.070 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39282.log 2024-09-19T04:21:17.070 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51933.log 2024-09-19T04:21:17.071 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38032.log: /var/log/ceph/ceph-client.admin.39282.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38032.log.gz 2024-09-19T04:21:17.071 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39282.log.gz 2024-09-19T04:21:17.071 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51626.log 2024-09-19T04:21:17.071 INFO:teuthology.orchestra.run.smithi076.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.42588.log.gz 2024-09-19T04:21:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43946.log 2024-09-19T04:21:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48692.log 2024-09-19T04:21:17.072 INFO:teuthology.orchestra.run.smithi076.stderr: 88.7%/var/log/ceph/ceph-client.admin.51626.log: -- replaced with /var/log/ceph/ceph-client.admin.51933.log.gz 2024-09-19T04:21:17.072 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43946.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.51626.log.gz 2024-09-19T04:21:17.073 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43946.log.gz 2024-09-19T04:21:17.073 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37794.log 2024-09-19T04:21:17.074 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48028.log 2024-09-19T04:21:17.074 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6%/var/log/ceph/ceph-client.admin.37794.log: -- replaced with /var/log/ceph/ceph-client.admin.48692.log.gz 2024-09-19T04:21:17.074 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50647.log 2024-09-19T04:21:17.074 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37794.log.gz 2024-09-19T04:21:17.074 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48028.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48028.log.gz 2024-09-19T04:21:17.075 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46088.log 2024-09-19T04:21:17.075 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46389.log 2024-09-19T04:21:17.076 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46285.log 2024-09-19T04:21:17.076 INFO:teuthology.orchestra.run.smithi076.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.50647.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46088.log.gz 2024-09-19T04:21:17.076 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:17.076 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46389.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46389.log.gz 2024-09-19T04:21:17.076 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41315.log 2024-09-19T04:21:17.078 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46059.log 2024-09-19T04:21:17.078 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8%/var/log/ceph/ceph-client.admin.41315.log: -- replaced with /var/log/ceph/ceph-client.admin.46285.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37954.log 2024-09-19T04:21:17.078 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:17.078 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41315.log.gz 2024-09-19T04:21:17.078 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46059.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46059.log.gz 2024-09-19T04:21:17.078 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37370.log 2024-09-19T04:21:17.078 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40289.log 2024-09-19T04:21:17.080 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37370.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37954.log.gz 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48783.log 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39039.log 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37370.log.gz 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40289.log: /var/log/ceph/ceph-client.admin.48783.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40289.log.gz 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48783.log.gz 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43594.log 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43139.log 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38514.log 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.39039.log.gz 2024-09-19T04:21:17.081 INFO:teuthology.orchestra.run.smithi076.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43594.log.gz 2024-09-19T04:21:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43139.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43139.log.gz 2024-09-19T04:21:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49467.log 2024-09-19T04:21:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41922.log 2024-09-19T04:21:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46717.log 2024-09-19T04:21:17.083 INFO:teuthology.orchestra.run.smithi076.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.38514.log.gz 2024-09-19T04:21:17.083 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41922.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49467.log.gz 2024-09-19T04:21:17.083 INFO:teuthology.orchestra.run.smithi076.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41922.log.gz 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45577.log 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34535.log 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35900.log 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.46717.log.gz 88.9% 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34535.log: -- replaced with /var/log/ceph/ceph-client.admin.45577.log.gz 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34535.log.gz 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50293.log 2024-09-19T04:21:17.085 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46465.log 2024-09-19T04:21:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50293.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45184.log 2024-09-19T04:21:17.087 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35900.log.gz 2024-09-19T04:21:17.087 INFO:teuthology.orchestra.run.smithi076.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.50293.log.gz 2024-09-19T04:21:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46465.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46465.log.gz 2024-09-19T04:21:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47950.log 2024-09-19T04:21:17.087 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49325.log 2024-09-19T04:21:17.088 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47950.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45184.log.gz 2024-09-19T04:21:17.088 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51699.log 2024-09-19T04:21:17.089 INFO:teuthology.orchestra.run.smithi076.stderr:gzip 89.4% -- replaced with /var/log/ceph/ceph-client.admin.47950.log.gz -5 2024-09-19T04:21:17.089 INFO:teuthology.orchestra.run.smithi076.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35899.log 2024-09-19T04:21:17.089 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51699.log: /var/log/ceph/ceph-client.admin.49325.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49325.log.gz 2024-09-19T04:21:17.089 INFO:teuthology.orchestra.run.smithi076.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51699.log.gz 2024-09-19T04:21:17.089 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51406.log 2024-09-19T04:21:17.090 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47844.log 2024-09-19T04:21:17.090 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41435.log 2024-09-19T04:21:17.090 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51406.log: /var/log/ceph/ceph-client.admin.47844.log: 88.3% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35899.log.gz 2024-09-19T04:21:17.091 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51406.log.gz 2024-09-19T04:21:17.091 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47844.log.gz 2024-09-19T04:21:17.091 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41409.log 2024-09-19T04:21:17.091 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45719.log 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50547.log 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41435.log.gz 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41409.log: /var/log/ceph/ceph-client.admin.45719.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41409.log.gz 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45719.log.gz 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49989.log 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38973.log 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43504.log 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50547.log.gz 2024-09-19T04:21:17.093 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38973.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49989.log.gz 2024-09-19T04:21:17.094 INFO:teuthology.orchestra.run.smithi076.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.38973.log.gz 2024-09-19T04:21:17.095 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35265.log 2024-09-19T04:21:17.095 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43504.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-19T04:21:17.095 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51574.log 2024-09-19T04:21:17.095 INFO:teuthology.orchestra.run.smithi076.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43504.log.gz 2024-09-19T04:21:17.096 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.audit.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.35265.log.gz 2024-09-19T04:21:17.096 INFO:teuthology.orchestra.run.smithi076.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-19T04:21:17.097 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49804.log 2024-09-19T04:21:17.097 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51574.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-19T04:21:17.098 INFO:teuthology.orchestra.run.smithi076.stderr: 88.6%/var/log/ceph/ceph-client.admin.49804.log: -- replaced with /var/log/ceph/ceph-client.admin.51574.log.gzgzip 2024-09-19T04:21:17.098 INFO:teuthology.orchestra.run.smithi076.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.51959.log 2024-09-19T04:21:17.098 INFO:teuthology.orchestra.run.smithi076.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.49804.log.gz/var/log/ceph/ceph-mds.c.log: 2024-09-19T04:21:17.102 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39830.log 2024-09-19T04:21:17.102 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44913.log 2024-09-19T04:21:17.103 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51959.log: /var/log/ceph/ceph-client.admin.39830.log: 89.1% 89.2% -- replaced with /var/log/ceph/ceph-client.admin.51959.log.gz 2024-09-19T04:21:17.103 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39830.log.gz 2024-09-19T04:21:17.103 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49892.log 2024-09-19T04:21:17.104 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46311.log 2024-09-19T04:21:17.105 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44913.log: /var/log/ceph/ceph-client.admin.49892.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.49892.log.gz 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44913.log.gz 2024-09-19T04:21:17.105 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:17.105 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37221.log 2024-09-19T04:21:17.106 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40912.log 2024-09-19T04:21:17.106 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46311.log: /var/log/ceph/ceph-client.admin.37221.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46311.log.gz 2024-09-19T04:21:17.107 INFO:teuthology.orchestra.run.smithi076.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37221.log.gz 2024-09-19T04:21:17.107 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-19T04:21:17.107 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40912.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40912.log.gz 2024-09-19T04:21:17.108 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39478.log 2024-09-19T04:21:17.108 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29086.log 2024-09-19T04:21:17.109 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.39478.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.39478.log.gz 2024-09-19T04:21:17.123 INFO:teuthology.orchestra.run.smithi116.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-19T04:21:17.130 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36082.log 2024-09-19T04:21:17.131 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.29086.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29086.log.gz 2024-09-19T04:21:17.131 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36923.log 2024-09-19T04:21:17.132 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36082.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36082.log.gz 2024-09-19T04:21:17.133 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39912.log 2024-09-19T04:21:17.133 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36923.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.36923.log.gz 2024-09-19T04:21:17.134 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47139.log 2024-09-19T04:21:17.135 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39912.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39912.log.gz 2024-09-19T04:21:17.135 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46577.log 2024-09-19T04:21:17.136 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47139.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.47139.log.gz 2024-09-19T04:21:17.137 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51053.log 2024-09-19T04:21:17.137 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46577.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46577.log.gz 2024-09-19T04:21:17.138 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51268.log 2024-09-19T04:21:17.139 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51053.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51053.log.gz 2024-09-19T04:21:17.139 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49914.log 2024-09-19T04:21:17.140 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51268.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51268.log.gz 2024-09-19T04:21:17.140 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36757.log 2024-09-19T04:21:17.141 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49914.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49914.log.gz 2024-09-19T04:21:17.142 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35901.log 2024-09-19T04:21:17.143 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36757.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36757.log.gz 2024-09-19T04:21:17.143 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45473.log 2024-09-19T04:21:17.144 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35901.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35901.log.gz 2024-09-19T04:21:17.144 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50447.log 2024-09-19T04:21:17.145 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45473.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45473.log.gz 2024-09-19T04:21:17.145 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44510.log 2024-09-19T04:21:17.147 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50447.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.50447.log.gz 2024-09-19T04:21:17.147 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42111.log 2024-09-19T04:21:17.148 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44510.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44510.log.gz 2024-09-19T04:21:17.148 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34702.log 2024-09-19T04:21:17.149 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42111.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42111.log.gz 2024-09-19T04:21:17.149 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37759.log 2024-09-19T04:21:17.150 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34702.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-09-19T04:21:17.151 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49567.log 2024-09-19T04:21:17.152 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37759.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37759.log.gz 2024-09-19T04:21:17.152 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43227.log 2024-09-19T04:21:17.153 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49567.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49567.log.gz 2024-09-19T04:21:17.153 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51209.log 2024-09-19T04:21:17.154 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43227.log: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.43227.log.gz 2024-09-19T04:21:17.155 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35510.log 2024-09-19T04:21:17.155 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51209.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.51209.log.gz 2024-09-19T04:21:17.156 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51432.log 2024-09-19T04:21:17.156 INFO:teuthology.orchestra.run.smithi116.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-19T04:21:17.157 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35510.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.35510.log.gz 2024-09-19T04:21:17.157 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45942.log 2024-09-19T04:21:17.158 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51432.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.51432.log.gz 2024-09-19T04:21:17.158 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51907.log 2024-09-19T04:21:17.159 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45942.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45942.log.gz 2024-09-19T04:21:17.160 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39795.log 2024-09-19T04:21:17.161 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51907.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.51907.log.gz 2024-09-19T04:21:17.161 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40793.log 2024-09-19T04:21:17.162 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39795.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39795.log.gz 2024-09-19T04:21:17.162 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35676.log 2024-09-19T04:21:17.163 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40793.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.40793.log.gz 2024-09-19T04:21:17.163 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-09-19T04:21:17.164 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35676.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35676.log.gz 2024-09-19T04:21:17.165 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37573.log 2024-09-19T04:21:17.166 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49848.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-09-19T04:21:17.166 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45134.log 2024-09-19T04:21:17.167 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37573.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37573.log.gz 2024-09-19T04:21:17.167 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36450.log 2024-09-19T04:21:17.169 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45134.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.45134.log.gz 2024-09-19T04:21:17.169 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35898.log 2024-09-19T04:21:17.170 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.36450.log: 90.0% -- replaced with /var/log/ceph/ceph-client.0.36450.log.gz 2024-09-19T04:21:17.171 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45308.log 2024-09-19T04:21:17.172 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35898.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35898.log.gz 2024-09-19T04:21:17.172 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46667.log 2024-09-19T04:21:17.173 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45308.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45308.log.gz 2024-09-19T04:21:17.174 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44432.log 2024-09-19T04:21:17.175 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46667.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.46667.log.gz 2024-09-19T04:21:17.175 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44757.log 2024-09-19T04:21:17.176 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44432.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44432.log.gz 2024-09-19T04:21:17.176 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.24050.log 2024-09-19T04:21:17.177 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44757.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44757.log.gz 2024-09-19T04:21:17.178 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2024-09-19T04:21:17.178 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph.tmp-client.admin.24050.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.24050.log.gz 2024-09-19T04:21:17.179 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35693.log 2024-09-19T04:21:17.180 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49417.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2024-09-19T04:21:17.180 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48106.log 2024-09-19T04:21:17.181 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35693.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35693.log.gz 2024-09-19T04:21:17.182 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47436.log 2024-09-19T04:21:17.182 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48106.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48106.log.gz 2024-09-19T04:21:17.183 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44350.log 2024-09-19T04:21:17.184 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47436.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47436.log.gz 2024-09-19T04:21:17.184 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47896.log 2024-09-19T04:21:17.185 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44350.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44350.log.gz 2024-09-19T04:21:17.185 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47540.log 2024-09-19T04:21:17.186 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47896.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47896.log.gz 2024-09-19T04:21:17.186 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-09-19T04:21:17.187 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47540.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47540.log.gz 2024-09-19T04:21:17.188 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37035.log 2024-09-19T04:21:17.189 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42245.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-09-19T04:21:17.189 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39938.log 2024-09-19T04:21:17.190 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37035.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37035.log.gz 2024-09-19T04:21:17.190 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39589.log 2024-09-19T04:21:17.191 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39938.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39938.log.gz 2024-09-19T04:21:17.192 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46363.log 2024-09-19T04:21:17.192 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39589.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39589.log.gz 2024-09-19T04:21:17.193 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-09-19T04:21:17.194 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46363.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46363.log.gz 2024-09-19T04:21:17.194 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39556.log 2024-09-19T04:21:17.195 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37876.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-09-19T04:21:17.195 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35061.log 2024-09-19T04:21:17.196 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39556.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39556.log.gz 2024-09-19T04:21:17.197 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40394.log 2024-09-19T04:21:17.197 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35061.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35061.log.gz 2024-09-19T04:21:17.198 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51079.log 2024-09-19T04:21:17.199 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40394.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.40394.log.gz 2024-09-19T04:21:17.199 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45772.log 2024-09-19T04:21:17.200 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51079.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.51079.log.gz 2024-09-19T04:21:17.201 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37091.log 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45772.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45772.log.gz 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37091.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37091.log.gz 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52167.log 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41837.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52167.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52167.log.gz 2024-09-19T04:21:17.206 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38388.log 2024-09-19T04:21:17.207 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49215.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2024-09-19T04:21:17.207 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44963.log 2024-09-19T04:21:17.208 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38388.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38388.log.gz 2024-09-19T04:21:17.208 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43302.log 2024-09-19T04:21:17.209 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44963.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44963.log.gz 2024-09-19T04:21:17.210 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41461.log 2024-09-19T04:21:17.210 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43302.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.43302.log.gz 2024-09-19T04:21:17.211 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43005.log 2024-09-19T04:21:17.212 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41461.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41461.log.gz 2024-09-19T04:21:17.212 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38848.log 2024-09-19T04:21:17.213 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43005.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43005.log.gz 2024-09-19T04:21:17.214 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49113.log 2024-09-19T04:21:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38848.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38848.log.gz 2024-09-19T04:21:17.215 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45693.log 2024-09-19T04:21:17.216 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49113.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49113.log.gz 2024-09-19T04:21:17.216 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-09-19T04:21:17.217 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.45693.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45693.log.gz 2024-09-19T04:21:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41599.log 2024-09-19T04:21:17.218 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41373.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-09-19T04:21:17.219 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35902.log 2024-09-19T04:21:17.220 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41599.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41599.log.gz 2024-09-19T04:21:17.220 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43998.log 2024-09-19T04:21:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35902.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35902.log.gz 2024-09-19T04:21:17.221 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49742.log 2024-09-19T04:21:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43998.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43998.log.gz 2024-09-19T04:21:17.222 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41994.log 2024-09-19T04:21:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49742.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49742.log.gz 2024-09-19T04:21:17.224 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47618.log 2024-09-19T04:21:17.225 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41994.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41994.log.gz 2024-09-19T04:21:17.225 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38058.log 2024-09-19T04:21:17.226 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47618.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47618.log.gz 2024-09-19T04:21:17.226 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40565.log 2024-09-19T04:21:17.227 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38058.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38058.log.gz 2024-09-19T04:21:17.228 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51458.log 2024-09-19T04:21:17.229 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40565.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40565.log.gz 2024-09-19T04:21:17.229 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46203.log 2024-09-19T04:21:17.230 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51458.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51458.log.gz 2024-09-19T04:21:17.230 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52245.log 2024-09-19T04:21:17.231 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.46203.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46203.log.gz 2024-09-19T04:21:17.232 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39530.log 2024-09-19T04:21:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52245.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52245.log.gz 2024-09-19T04:21:17.233 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43724.log 2024-09-19T04:21:17.234 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39530.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39530.log.gz 2024-09-19T04:21:17.234 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43892.log 2024-09-19T04:21:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43724.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43724.log.gz 2024-09-19T04:21:17.235 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40318.log 2024-09-19T04:21:17.236 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43892.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-09-19T04:21:17.237 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40494.log 2024-09-19T04:21:17.238 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40318.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40318.log.gz 2024-09-19T04:21:17.238 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36645.log 2024-09-19T04:21:17.239 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40494.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40494.log.gz 2024-09-19T04:21:17.239 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42020.log 2024-09-19T04:21:17.240 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36645.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36645.log.gz 2024-09-19T04:21:17.240 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43750.log 2024-09-19T04:21:17.241 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42020.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42020.log.gz 2024-09-19T04:21:17.242 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-19T04:21:17.242 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43750.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43750.log.gz 2024-09-19T04:21:17.243 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35884.log 2024-09-19T04:21:17.244 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44180.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-19T04:21:17.244 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43055.log 2024-09-19T04:21:17.245 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35884.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35884.log.gz 2024-09-19T04:21:17.245 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42163.log 2024-09-19T04:21:17.246 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43055.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43055.log.gz 2024-09-19T04:21:17.247 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35614.log 2024-09-19T04:21:17.248 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42163.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42163.log.gz 2024-09-19T04:21:17.248 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51803.log 2024-09-19T04:21:17.249 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35614.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35614.log.gz 2024-09-19T04:21:17.249 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37677.log 2024-09-19T04:21:17.250 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51803.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51803.log.gz 2024-09-19T04:21:17.251 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35893.log 2024-09-19T04:21:17.251 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37677.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37677.log.gz 2024-09-19T04:21:17.252 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44668.log 2024-09-19T04:21:17.253 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.35893.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35893.log.gz 2024-09-19T04:21:17.253 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39713.log 2024-09-19T04:21:17.260 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.44668.log: 93.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38359.log 2024-09-19T04:21:17.260 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.0.44668.log.gz 2024-09-19T04:21:17.261 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39713.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39713.log.gz 2024-09-19T04:21:17.261 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39990.log 2024-09-19T04:21:17.262 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.38359.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38359.log.gz 2024-09-19T04:21:17.262 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36160.log 2024-09-19T04:21:17.263 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39990.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39990.log.gz 2024-09-19T04:21:17.263 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40444.log 2024-09-19T04:21:17.264 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.36160.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36160.log.gz 2024-09-19T04:21:17.265 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41707.log 2024-09-19T04:21:17.266 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40444.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.40444.log.gz 2024-09-19T04:21:17.266 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42677.log 2024-09-19T04:21:17.267 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.41707.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.41707.log.gz 2024-09-19T04:21:17.267 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48901.log 2024-09-19T04:21:17.268 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42677.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42677.log.gz 2024-09-19T04:21:17.269 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39230.log 2024-09-19T04:21:17.269 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48901.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48901.log.gz 2024-09-19T04:21:17.270 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52063.log 2024-09-19T04:21:17.271 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.39230.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39230.log.gz 2024-09-19T04:21:17.271 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50999.log 2024-09-19T04:21:17.272 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52063.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.52063.log.gz 2024-09-19T04:21:17.272 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49939.log 2024-09-19T04:21:17.273 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50999.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.50999.log.gz 2024-09-19T04:21:17.274 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50799.log 2024-09-19T04:21:17.275 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.49939.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49939.log.gz 2024-09-19T04:21:17.275 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51855.log 2024-09-19T04:21:17.277 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.50799.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50799.log.gz 2024-09-19T04:21:17.277 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44787.log 2024-09-19T04:21:17.278 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.51855.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51855.log.gz 2024-09-19T04:21:17.278 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2024-09-19T04:21:17.279 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44787.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44787.log.gz 2024-09-19T04:21:17.279 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48548.log 2024-09-19T04:21:17.280 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43183.log: 89.9% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2024-09-19T04:21:17.281 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43620.log 2024-09-19T04:21:17.281 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48548.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48548.log.gz 2024-09-19T04:21:17.282 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34812.log 2024-09-19T04:21:17.283 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43620.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43620.log.gz 2024-09-19T04:21:17.283 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42428.log 2024-09-19T04:21:17.284 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34812.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34812.log.gz 2024-09-19T04:21:17.284 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42480.log 2024-09-19T04:21:17.285 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42428.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42428.log.gz 2024-09-19T04:21:17.285 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38270.log 2024-09-19T04:21:17.287 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42480.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.42480.log.gz 2024-09-19T04:21:17.287 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52271.log 2024-09-19T04:21:17.299 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.0.38270.log: 94.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44089.log 2024-09-19T04:21:17.300 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52271.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.52271.log.gz 2024-09-19T04:21:17.300 INFO:teuthology.orchestra.run.smithi076.stderr: -- replaced with /var/log/ceph/ceph-client.0.38270.log.gz 2024-09-19T04:21:17.300 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47514.log 2024-09-19T04:21:17.301 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44089.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44089.log.gz 2024-09-19T04:21:17.301 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42349.log 2024-09-19T04:21:17.302 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47514.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47514.log.gz 2024-09-19T04:21:17.303 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37625.log 2024-09-19T04:21:17.304 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42349.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42349.log.gz 2024-09-19T04:21:17.304 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40837.log 2024-09-19T04:21:17.305 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.37625.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37625.log.gz 2024-09-19T04:21:17.305 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34603.log 2024-09-19T04:21:17.306 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40837.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.40837.log.gz 2024-09-19T04:21:17.306 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44863.log 2024-09-19T04:21:17.307 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.34603.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34603.log.gz 2024-09-19T04:21:17.308 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47386.log 2024-09-19T04:21:17.309 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.44863.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.44863.log.gz 2024-09-19T04:21:17.309 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48983.log 2024-09-19T04:21:17.310 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47386.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.47386.log.gz 2024-09-19T04:21:17.311 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42193.log 2024-09-19T04:21:17.312 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48983.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48983.log.gz 2024-09-19T04:21:17.312 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2024-09-19T04:21:17.313 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.42193.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42193.log.gz 2024-09-19T04:21:17.313 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52221.log 2024-09-19T04:21:17.314 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.40962.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz 2024-09-19T04:21:17.315 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48302.log 2024-09-19T04:21:17.316 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.52221.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.52221.log.gz 2024-09-19T04:21:17.316 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43866.log 2024-09-19T04:21:17.317 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48302.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48302.log.gz 2024-09-19T04:21:17.317 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43920.log 2024-09-19T04:21:17.318 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43866.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43866.log.gz 2024-09-19T04:21:17.319 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47566.log 2024-09-19T04:21:17.319 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43920.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43920.log.gz 2024-09-19T04:21:17.320 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48080.log 2024-09-19T04:21:17.321 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.47566.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47566.log.gz 2024-09-19T04:21:17.321 INFO:teuthology.orchestra.run.smithi076.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43840.log 2024-09-19T04:21:17.322 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.48080.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48080.log.gz 2024-09-19T04:21:17.323 INFO:teuthology.orchestra.run.smithi076.stderr:/var/log/ceph/ceph-client.admin.43840.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43840.log.gz 2024-09-19T04:21:17.390 INFO:teuthology.orchestra.run.smithi076.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-19T04:21:17.477 INFO:teuthology.orchestra.run.smithi076.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-19T04:21:17.628 INFO:teuthology.orchestra.run.smithi076.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-19T04:21:17.919 INFO:teuthology.orchestra.run.smithi116.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-19T04:21:17.924 INFO:teuthology.orchestra.run.smithi116.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-19T04:21:17.984 INFO:teuthology.orchestra.run.smithi116.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-19T04:21:18.022 INFO:teuthology.orchestra.run.smithi116.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-19T04:21:18.022 INFO:teuthology.orchestra.run.smithi116.stderr: 2024-09-19T04:21:18.022 INFO:teuthology.orchestra.run.smithi116.stderr:real 0m1.147s 2024-09-19T04:21:18.022 INFO:teuthology.orchestra.run.smithi116.stderr:user 0m4.769s 2024-09-19T04:21:18.022 INFO:teuthology.orchestra.run.smithi116.stderr:sys 0m0.159s 2024-09-19T04:21:18.031 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-19T04:21:18.044 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-19T04:21:18.175 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-19T04:21:18.175 INFO:teuthology.orchestra.run.smithi076.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-19T04:21:18.177 INFO:teuthology.orchestra.run.smithi076.stderr: 2024-09-19T04:21:18.177 INFO:teuthology.orchestra.run.smithi076.stderr:real 0m1.315s 2024-09-19T04:21:18.177 INFO:teuthology.orchestra.run.smithi076.stderr:user 0m6.559s 2024-09-19T04:21:18.177 INFO:teuthology.orchestra.run.smithi076.stderr:sys 0m0.321s 2024-09-19T04:21:18.177 INFO:tasks.ceph:Archiving logs... 2024-09-19T04:21:18.178 DEBUG:teuthology.misc:Transferring archived files from smithi076:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/remote/smithi076/log 2024-09-19T04:21:18.179 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:21:19.346 DEBUG:teuthology.misc:Transferring archived files from smithi116:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/remote/smithi116/log 2024-09-19T04:21:19.346 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T04:21:19.935 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T04:21:19.944 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T04:21:19.944 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:21:19.947 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T04:21:20.024 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T04:21:20.033 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T04:21:20.033 DEBUG:teuthology.orchestra.run.smithi076:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:21:20.036 DEBUG:teuthology.orchestra.run.smithi116:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T04:21:20.048 INFO:teuthology.orchestra.run.smithi076.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T04:21:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:============================================================================== 2024-09-19T04:21:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:+hv01.front.sepi 67.205.162.81 3 u 41 64 377 0.068 -2.050 1.418 2024-09-19T04:21:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:*hv02.front.sepi 63.231.80.2 3 u 31 64 377 0.100 -1.392 1.687 2024-09-19T04:21:20.048 INFO:teuthology.orchestra.run.smithi076.stdout:+hv03.front.sepi 74.6.168.72 3 u 32 64 377 0.089 -2.899 1.579 2024-09-19T04:21:20.048 INFO:teuthology.orchestra.run.smithi076.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-09-19T04:21:20.050 INFO:teuthology.orchestra.run.smithi116.stdout: remote refid st t when poll reach delay offset jitter 2024-09-19T04:21:20.050 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================== 2024-09-19T04:21:20.050 INFO:teuthology.orchestra.run.smithi116.stdout:+hv01.front.sepi 67.205.162.81 3 u 36 64 377 0.072 -1.467 0.625 2024-09-19T04:21:20.050 INFO:teuthology.orchestra.run.smithi116.stdout:*hv02.front.sepi 63.231.80.2 3 u 43 64 377 0.064 -1.590 0.860 2024-09-19T04:21:20.050 INFO:teuthology.orchestra.run.smithi116.stdout:+hv03.front.sepi 74.6.168.72 3 u 32 64 377 0.099 -1.772 1.041 2024-09-19T04:21:20.050 INFO:teuthology.orchestra.run.smithi116.stdout: hv04.front.sepi .INIT. 16 u - 128 0 0.000 +0.000 0.000 2024-09-19T04:21:20.051 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T04:21:20.060 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T04:21:20.060 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T04:21:20.068 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T04:21:20.077 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T04:21:20.085 INFO:teuthology.task.internal:Duration was 1147.707013 seconds 2024-09-19T04:21:20.085 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T04:21:20.101 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T04:21:20.102 DEBUG:teuthology.orchestra.run.smithi076:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:21:20.104 DEBUG:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T04:21:20.143 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T04:21:20.144 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi076.front.sepia.ceph.com 2024-09-19T04:21:20.144 DEBUG:teuthology.orchestra.run.smithi076:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:21:20.191 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi116.front.sepia.ceph.com 2024-09-19T04:21:20.191 DEBUG:teuthology.orchestra.run.smithi116:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T04:21:20.201 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T04:21:20.201 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:21:20.235 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T04:21:20.271 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T04:21:20.272 DEBUG:teuthology.orchestra.run.smithi076:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:21:20.307 DEBUG:teuthology.orchestra.run.smithi116:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T04:21:20.376 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T04:21:20.386 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T04:21:20.386 DEBUG:teuthology.orchestra.run.smithi076:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:21:20.399 DEBUG:teuthology.orchestra.run.smithi116:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T04:21:20.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T04:21:20.421 DEBUG:teuthology.orchestra.run.smithi076:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:21:20.443 DEBUG:teuthology.orchestra.run.smithi116:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:21:20.451 INFO:teuthology.orchestra.run.smithi076.stdout:kernel.core_pattern = core 2024-09-19T04:21:20.463 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = core 2024-09-19T04:21:20.476 DEBUG:teuthology.orchestra.run.smithi076:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:21:20.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:21:20.505 DEBUG:teuthology.orchestra.run.smithi116:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T04:21:20.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T04:21:20.524 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T04:21:20.534 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T04:21:20.535 DEBUG:teuthology.misc:Transferring archived files from smithi076:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/remote/smithi076 2024-09-19T04:21:20.535 DEBUG:teuthology.orchestra.run.smithi076:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:21:20.570 DEBUG:teuthology.misc:Transferring archived files from smithi116:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911626/remote/smithi116 2024-09-19T04:21:20.570 DEBUG:teuthology.orchestra.run.smithi116:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T04:21:20.591 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T04:21:20.591 DEBUG:teuthology.orchestra.run.smithi076:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:21:20.615 DEBUG:teuthology.orchestra.run.smithi116:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T04:21:20.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T04:21:20.646 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T04:21:20.646 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T04:21:20.660 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T04:21:20.660 DEBUG:teuthology.orchestra.run.smithi076:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:21:20.663 DEBUG:teuthology.orchestra.run.smithi116:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T04:21:20.666 INFO:teuthology.orchestra.run.smithi076.stdout: 655475 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:21 /home/ubuntu/cephtest 2024-09-19T04:21:20.681 INFO:teuthology.orchestra.run.smithi116.stdout: 658623 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 04:21 /home/ubuntu/cephtest 2024-09-19T04:21:20.682 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T04:21:20.694 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T04:21:20.750 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/multimds_misc} duration: 1147.707013130188 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-19T04:21:20.750 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T04:21:20.825 INFO:teuthology.run:pass